Texture Cache

LLTextureCache is a worker thread interface class used for reading and writing texture data to the local disk cache (part of the Image System)

Current design

• There are two different disk caches used by the Second Life Viewer:
  • The static texture cache
    • Located in the Second Life/skins/textures directory
    • This is a read-only cache which contains common textures shipped with the Viewer
  • The general texture cache
    • Stored in a 'textures' subdirectory in the cache directory, specified in the preferences
    • This is a two tiered read/write cache
      • A list of all files in the cace is stored in texture.entries
      • The header for each image (enough to identify the size and load the first mip level) is stored in texture.cache
      • The body (minus the header) for a limited (by the cache size) number of entries is stored in textures/[0-9a-f]/textureid
• Requests to read or write to the cache are processed in the priority given
• The results are given through a Responder class when they are ready

Disk storage format

• Currently all textures in Second Life are stored in the JPEG2000 compressed format
• The header consists of two files
  • cache/texture.entries
    • 8 byte header:

	struct EntriesInfo
	{
		F32 mVersion;
		U32 mEntries;
	};

• • • Array of 24 byte entries

	struct EntriesInfo
	{
		LLUUID mID; // 128 bits
		S32 mSize; // total size of image if known (NOT size cached)
		U32 mTime; // seconds since 1/1/1970
	};

Analysis

(so far:)

In the discussion of the texture cache has, it is apparent that there are two separate issues involved. Beyond the view of the texture cache, there is the method to which to determine how long a texture is stored in the cache and there is medium format of the textures being stored. At this time, the format of the medium is still under discussion. However, there is significant data to move to a new system for the method to determine the storage time.

• 2Q cache system. Benefits include that the 2Q method has already been studied by other groups and that it is possible to fit this method into the current design. Only drawback noticed so far is that it appears to carry a specific implementation, but this drawback will need to be confirmed. The 2Q's intended implementation may not easily allow for extra values in the cache system to determine factors of pixel coverage. The drawback is of low impact to the the current design since the significant pixel coverage data is stored in memory. However, in order to further implement the ideas being suggested in the discussion about the medium format and pixel coverage, we could adapt the 2Q system.

• ARC Algorithm designed by IBM Research as an improvement over LRU. Patent encumbered, however IBM will grant license for open source use only.

Discussion about improvements

The texture cache is a hot topic for optimization. Current ideas/subjects discussed from the mailing list include:

• Multi-level cache for uncompressed and compressed textures

• Example Cache Levels:
  • Memory
  • Uncompressed Disk
  • Compressed Disk
  • Low Rez Compressed
  • Network
  • A flexible policy architecture would be able to discard, cache, retrieve, and migrate texture data from one level to the next as is appropiate with an external XML file to facilitate fine tuning of policy parameters.

• Fixed block size cache to store images of any size that is a power of 2
• Reformatted textures, being changed from jpeg2000 to TGA (with or without RLE), BMP, GIF, or downsampled
• Storage:
  • Filesystem
  • Database
  • Network
  • Memory mapped
• Quality of cached images
• System Requirements
  • Minimal
  • Optimal
  • Surveys
  • Core(s)

Please use the Talk page to continue discussion.

Related articles: Image System, VFS