Linden Lab Official:Media Rendering Plugin Operation and Data Flow

High level view

From a high level view, when the Viewer wants to render a media URI:

1. The Viewer retrieves the MIME type of the resource with an asynchronous HTTP HEAD request. If there is no MIME type associated with this URI (for example, a VNC session), it uses a type or subtype in the "x-" or "vnd-" range. When the MIME type is known, the Viewer uses type and subtype fields to determine which media plugin to use via a lookup in an XML file.
2. The Viewer creates an instance of the plugin.
3. When the plugin is fully created and initialized, it informs the Viewer that it is ready.
4. The Viewer asks the plugin to start rendering data.
5. The Plugin receives the message and starts to render the media.
6. When the media data being rendered changes, the plugin tells the Viewer that something changed and the Viewer updates itself accordingly.

Detailed view

The following sections give a detailed view of what happens during each stage of plugin operation.

Initialization

TODO: what specific messages are sent and received by the plugin? what are the mandatory messages? list other possible messages.

During initialization, a plugin tells the PLS the set of messages it supports, including versioning information to maintain backwards compatibility.

Time-based media rendering plugins

differs from browser-like media rendering plugins in initial size negotiation? if this is small, it doesn't need its own section.

Shared memory size negotiation

This section has not been edited, except to move any command parameter info to its command description on the Media Rendering Plugin System Messages page.

A plugin and the PLS negotiate the size and format of the shared memory pixel buffer into which the plugin draws with the texture_params, size_change_request, and size_change messages.

The plugin specifies the buffer's pixel size and format by sending a texture_params message. Indexed pixel modes and pixels which use less than eight bits per channel are not currently supported. Recommended modes are 24 bit RGB or 32 bit ARGB, or variants thereof (BGRA, etc).

Size negotiation is a bit more complicated:

• PLS initially sets up the dimensions of the pixel buffer.
  • If the plugin doesn't have any special sizing requirements, it can just draw to the buffer and never send a size_change_request message.
• If the plugin sends a size_change_request message, the PLS takes that as the "native size" of the media.
  • Media is allowed to change sizes multiple times during playback, such as with streaming QuickTime movies.
• If the media is playing as parcel media and the "auto scale" option is set, the draw dimensions may be increased to the next power of two.
• PLS sends the requested drawing dimensions, the actual size of the buffer, and the name of the shared memory segment to the plugin in a size_change_response message.
  • The plugin must draw within the specified dimensions, and should NOT send a size_change_request in response to the size_change_response

When changing buffer sizes, the new size may require a larger or smaller memory buffer than the old size. In this case, the shared memory segment name received with the size_change_response may be different than the one the plugin was previously drawing to. If the plugin receives a size_change_response with a shared memory segment name it hasn't seen yet, it MUST not draw to the old shared memory segment using the new parameters, or it may overrun the end of the buffer and crash.

In this case, the plugin should expect to receive a shm_added message for the new shared memory segment either before or after the size_change_response, and must wait until it has received both before drawing to the new buffer.

Likewise, when a plugin receives a shm_remove message and it's drawing to the shared memory buffer with that name, it needs to stop doing so, since the memory buffer will be deallocated once the message handler returns.

When the plugin is running

TODO: what specific messages are sent and received by the plugin? what are the mandatory messages? list other possible messages?

Time-based media rendering plugins

TODO

Browser-like media rendering plugins

TODO

Error handling

TODO: how should the plugin handle errors? how does the PLS handle plugin errors?

Exit and cleanup

TODO: what does the plugin need to do when it exits? what does the PLS need to do?

Media Rendering Plugin Developer Guide Media rendering plugin system Frequently Asked Questions Getting started Plugin operation and data flow System messages Plugin Test App: LLMediaPluginTest