SLDEV Discussion - Texture Cache Plan of Attack

Discussion is underway in the SLDev Mailing List regarding options to overhaul the texture cache. Mostly the plan involves storing decompressed textures, rather than the JPG2000 compressed versions, which allows the Texture Fetch operations to skip the decode, which is currently the major bottleneck in the cache system. This article is for summarizing, planning, and status of this project. Please discuss via the SLDev mailing list.

Overview

Current Implementation

Currently, the client will download uncached JPG2000 textures from the region/asset servers, progressively decode and apply them to the faces rendered in whatever quality level is currently available. As more data is downloaded, it is further decoded, and the quality rendered goes up. Textures are cached in a three-part system sharing space with the VFS, described in the Texture Cache article.

New Plan

The discussion on the SLDev list has been very helpful in nailing down ways to improve the cache. The biggest change will be caching of textures in an un-compressed format - TGA or Raw - while preserving the existing obfuscation technique to prevent direct ripping of the cached images. In addition, the cache settings will be split into two separate caches - the VFS for objects that use it currently, and a completely new layer of cache exclusively for textures.

Steps

Rework Cache to store multiple formats

Currently, the texture cache can read TGA and JPG2000 files from the cache. However, it has no way to write TGA to the cache. The interface between llTextureFetch and llTextureCache will need to be refactored to support storing and retrieving in a format-agnostic way, allowing JPG2000, TGA, and Raw to exist simultaneously in the cache, with the possibility of adding others down the line if needed relatively easily.

In short, the calls from llTextureFetch to llTextureCache, and the return data, should include a Format Identifier, a block of data for the image itself (in whatever format is indicated by the Format Identifier), and any other details that may be required such as data size, image dimensions, and (if necessary) color depth. llTexureCache should be able to store and retrieve this data in a format-agnostic way, ideally returning decoded data to llTextureFetch as a llRawImage object. Metadata would be stored in the cache index, discussed below, with the bulk of the data stored in the manner it is done now - subfolders with filenames equivalent to the UUID of the texture.

Rework Cache Index for Greater Storage

Currently, the Texture Cache shares space with the VFS, and is limited to 800MB - 80% of the maximum total "cache size", the other 20% of which is reserved for the VFS. With the cache capable of storing uncompressed files, it could grow much faster; and with the cost of disk space these days, limiting the cache to 1GB is ludicrously inadequate. A separate slider in the "Network" preferences tab should allow the individual specification of max VFS ("Object Cache") and max Texture caches. In discussion, many commenters stated that they would allocate as much as 100GB to a texture cache. Aside from the speed benefit garnered from a larger cache, it also will dramatically reduce the amount of textures that would need to be re-downloaded due to a lower cache discard rate.

The Cache Index would store Metadata about the texture - the Format ID, the dimensions, file size, and color depth (if necessary), and, like the current cache index, would also have the first (arbitrary number of) bytes of the actual data - primarily for obfuscation purposes, to prevent the casual ripping of textures from the cache. Multiple copies of the same texture could be stored in the cache, in different formats - for example, stored as JPG2000 while progressively downloading, then stored in uncompressed format once complete. The Texture Fetcher would simply request a texture, and the cache would respond with the cached version available with the lowest CPU-overhead - Raw first, then TGA, then JPG2000, with the network fetch as the fail-through option. Ideally, once a texture has been downloaded completely and decoded, it would be stored as a Raw image, and the JPG2000 version would be marked as "expired" or otherwise scheduled for purging from the cache. The bulk of the texture data, stored in the UUID-named file in the subfolders of the cache, will need to have an extension added to it that signifies the Format ID to allow co-existence of multiple encodings of the same texture.

Some discussion was brought up about XORing the raw texture data with a hard-coded key, or perhaps one based on the Hardware Hash of each individual machine - in effect a fast but low-quality encryption. This has been largely disregarded in the interest of attaining the best speed - although it remains an option if the community demands more obfuscation than currently exists in the texture cache.

Cache Discard Optimization

There is discussion on the Texture Cache page regarding further optimization or replacement of the Cache Discard algorithms, which should probably be moved here - however, this portion is not a part of the current ongoing discussion in SLDev. I will leave it to someone else to take the initiative on this.

Implementation Experiments: Separate cache size controls

1. Add separate UI controls for VFS and texture decode caches

IN FILE: panel_preferences_network.xml

Widget: Slider CacheSize / cache_size

Originally used to control compressed texture cache and VFS asset cache size. 80% of this value set the texture cache size and 20% the VFS asset cache size. Original slider scaling: 50 to 1000 megabytes

Changes: Will now only set the compressed texture cache size at 100% of this value. Scaling is still 50 to 1000 megabytes

Widget: Checkbox CacheUseDecodeTextures / cache_use_decoded_textures

New, for enabling or disabling use of the new decoded texture cache. Disabling would not clear the cache but just stop use of it, for troubleshooting purposes.

Widget: Slider CacheSizeDecodedTextures / cache_size_decoded_textures

Sets the size of the new decoded cache. Since this has the opportunity to greatly increase performance if more space is allocated to it, the size scaling is in gigabytes from 1 to 1000 GB, with a default size of 1 GB.

Widget: Checkbox CacheRemoveDecodedTextures / cache_remove_decoded_textures

This option deletes the original compressed texture out of the standard cache once it has been added to the decode cache, to conserve disk space usage.

Widget: RadioButton CacheUseVFS / cache_use_vfs

This enables use of the current, existing VFS asset caching method.

Widget: Slider CacheSizeVFS / cache_size_vfs

This sets the size of the existing VFS, as a separate control from the old single-function texture cache slider. Size is set as a percentage of total installed system memory, with a range from 1% to 50% of total memory.

Widget: RadioButton CacheUseDiskAsset / cache_use_disk_asset

This enables use of a purely disk-based cache for assets that is similar to the texture and decode caches, and completely deallocates the VFS from memory and disables it.

Widget: Slider: CacheSizeDiskAsset / cache_size_disk_asset

This sets the size of the new disk-based asset cache, and also has a very large scaling from 1 to 1000 GB, with a default size of 1 GB.

Widget: Button CacheClearStandardTexture/ cache_clear_standard_texture

Only clear the standard compressed texture cache.

Widget: Button CacheClearStdVFS/ cache_clear_vfs

Only clear the RAM-disk VFS. May need a viewer restart to achieve this.

Widget: Button CacheClearDecodedTexture / cache_clear_decoded_texture

Only clear the new decoded texture cache. Should offer a warning that this may take a while for gigabytes of data deletion.

Widget: Button CacheClearDiskAsset / cache_clear_disk_asset

Only clear the new disk-based asset cache. Should offer a warning that this may take a while for gigabytes of data deletion.

Widget: Checkbox CacheClearOnMove / cache_clear_on_move

Offer the option to delete all data from the old cache location, before creating a cache in the new location.