Image System

See Also

• Texture Cache
• Image Pipeline

Notes

• This documentation describes the Image Subsystem as of 1.14.0, first released as "First Look: Render Pipeline Improvements".

Overview

The image system handles the requesting, loading, and decoding of texture data. It also handles passing the data to OpenGL, and discarding unwanted data. It uses a priority scheme to determine what data to load and to discard.

Definitions

Format

SecondLife transmits JPEG 2000 image codestreams as described in Appendix A of JPEG 2000 Part I Final Committee Draft Version 1.0 which are decoded in the client and sent to OpenGL as uncompressed 24 bit or 32 bit textures.

JPEG-2000 is a highly efficient compression method, allowing for very small compressed file sizes, which in turn helps to reduce Linden Lab's asset storage. However, the cost to users of the Second Life client is that JPEG-2000 is much more processor intensive than other compression image methods, and so is a major contributing factor for the slow "rezzing" of textures by the client, even when these compressed textures exist in a local disk cache.

After the image data has been decompressed, it is stored only in system memory of the client. When the avatar teleports to a new but similar location using similar textures, the decompressed data is simply discarded and every texture is regenerated from the original compressed JPEG-2000 textures. No attempt is made to write the decoded image data to a secondary image cache to eliminate the decompression penalty.

Discard Level and Mip Mapping

• Discard level

Discard level 0 represents the highest resolution version of a texture. Discard level 1 represents a texture half the resolution of discard level 0 in each dimension, so one quarter of the pixel area and one quarter of the required texture memory. It is primarily a function of total pixel area, e.g. if a 256x256 texture is covering a 128x128 portion of the screen, the desired discard level is 1.

• Mip Mapping is a technique where multiple resolutions of a texture are stored in order to optimize rendering. To support mipmapping, when discard level 0 is loaded, discard levels 1-N are also loaded (where N represents the smallest useful image). e.g. When a 256x128 texture is loaded, the following textures are also generated and loaded: 128x64, 64x32, 32x16, 16x8, 8x4. This incurs a 33% increase in the amount of texture memory consumed, but makes a significant improvement in performance and visual quality.

Prioritization

• Total pixel area

This is an approximation of the total number of pixels of a texture from all faces that are currently being rendered. This is a factor in determining the priority (see below) of a texture.

• Maximum pixel area

This is an approximation of the maximum number of pixels of a texture on a single face that are currently being rendered. This is the primary factor in determining the desired discard level (see below) of a texture.

• Boost level

This is used to increase the prioritization of some textures over others. For example, textures on selected faces are boosted.

• Priority

Also referred to as decode priority, this is used to determine in what order texture data is downloaded and decoded. It is a function of the total pixel area, the boost level, and the difference between the current discard level and the desired discard level.

Feature Description

Texture Pipeline

• See Also: Image Pipeline
• The render pipeline determines which textures are required, their desired discard level, and their priority.
• At this point, the following steps are followed:

1. Requests are sent to the texture fetcher for texture data
2. The texture fetcher does the following with each request:
   1. Load data from cache if available
   2. Send requests to the servers for additional texture data if needed
   3. Receive data from the servers
   4. Cache received data
   5. Decode the texture
3. When the texture data is ready the render pipeline is notified
4. The decoded texture data is passed to OpenGL for rendering

• Each frame a limited number of textures are examined and processed as follows:
  • If the desired discard level of the texture decreases significantly, more texture data is requested and the texture is updated with the higher resolution data
  • If the desired discard level of the texture increases significantly, the undesired discard levels are discarded from memory

Code Description

LLImage

• LLImageBase defines a set of generic interfaces to essential image information: Width, Height, Components, Data, and Data Size. It is only used as a base class.
• LLImageFormatted defines a set of interfaces to formatted image data, e.g. JPEG2000 or TGA.
• LLImageRaw defines special interfaces for handling raw (i.e. uncompressed 24 bit RGB, 32 bit RGBA, or 8 bit A) image data.
• LLImageWorker defines an interface for decoding textures which supports threading.

LLViewerImage

• LLImageGL defines the interface between Second Life image data and OpenGL. It includes methods for setting the image data, reading it back, discarding mip levels, setting GL parameters (like clamping), and various accessors.
• LLViewerImage contains additional state data for calculating the image priority and desired discard level. It also has methods for texture fetching.
• Note: In 1.13.0 and earlier, the texture fetching code is integrated in LLViewerImage, and only supports textures over UDP (instead of HTTP) and the VFS for local caching (instead of a specialized texture cache). This code is being migrated to LLTextureFetch for 1.13.1 or 1.13.2, described below.

LLTextureFetch

• See Image Pipeline

• LLTextureFetch is a worker thread interface class used for "fetching" texture data.
• It executes a state machine that manages the following states:
  • Load from disk cache (see LLTextureCache)
  • Load from the network (from the simulator process or through an HTTP request to the simulator host)
  • Decode the image
  • Write the image to the disk cache (see LLTextureCache)

LLTextureCache

• LLTextureCache is a worker thread interface class used for reading and writing texture data to the local disk cache

More information can be found in the Texture cache article