Linden Lab Official:Media Plugin System Messages

From Second Life Wiki
Jump to navigation Jump to search
 
NOTE: This document reflects ongoing internal work on the Media Plugin System. The details may not match the current publicly-available code; however the overall approach and high-level features reflect the project direction.

Overview

This article specifies the messages used in the Second Life Viewer media plugin system to communicate between the Plugin Loader Shell (PLS) and individual media rendering plugins.

For information on messages passed between the PLS and the Viewer, see Media Plugin System Internal Messages.

Message categories

Messages fall into these general categories:

  • Base messages for initialization, connection management, and shared memory setup. All plugins must implement these basic messages.
    • Media messages for negotiating the pixel buffer and handling updates of subsections of it, specifying a media source URL, and passing mouse and keyboard events. All media rendering plugins must implement these messages.
      • Time-based media messages for functions such as play, pause, stop, seek, and so on. All time-based media (video-like) rendering plugins must implement these messages.
      • Browser-like media messages for functions such as forward, back, reload, and so on. All browser-like media rendering plugins must implement these messages.
  • Internal messages for communication between the viewer and the plugin loader shell. These messages are not delivered to a plugin. Plugin developers need not be concerned with them; their documentation is provided for Viewer developers.

During initialization, a plugin tells the PLS the set of messages it supports, including versioning information to maintain backwards compatibility.

Buffer size negotiation

A plugin and the PLS negotiate the size and format of the pixel buffer into which the plugin draws with the texture_params, size_change_request, and size_change messages.

The plugin determines the buffer's pixel size and format according to what it sends in the texture_params message. Indexed pixel modes and pixels which use less than eight bits per channel are not currently supported. Recommended modes are 24 bit RGB or 32 bit ARGB, or variants thereof (BGRA, etc).

Size negotiation is a bit more complicated:

  • PLS initially sets up the dimensions of the pixel buffer.
    • If the plugin doesn't have any special sizing requirements, it can just draw to the buffer and never send a size_change_request message.
  • If the plugin sends a size_change_request message, the PLS takes that as the "native size" of the media.
    • Media is allowed to change sizes multiple times during playback, such as with streaming QuickTime movies.
  • If the media is playing as parcel media and the "auto scale" option is set, the draw dimensions may be increased to the next power of two.
  • If the plugin sets the allow_downsample field in the texture_params message and the media's priority has been lowered, the host may reduce the requested draw dimensions.
    • The plugin should only set allow_downsample if it can draw the media at different scales without changing the overall look of the media, such as scaling down a video stream
    • Media which looks different when drawn at different pixel scales (such as a web browser where font sizes depend on pixel scale) should NOT set allow_downsample
    • If it's more expensive to draw media downsampled than at native resolution, the plugin should NOT set allow_downsample: it's supposed to be a way to reduce load on the system, and if it doesn't accomplish that it shouldn't be used.
  • If the plugin sets the 'padding' field in the texture_params message, the buffer's byte width will be padded to the requested alignment.
    • Note that it's possible to request pixel size/alignment combinations that don't come out even when using 3-byte RGB pixels. If the host detects this, it will disable padding.
  • PLS sends the requested drawing dimensions, the actual size of the buffer, and the name of the shared memory segment to the plugin in a size_change_response message.
    • The plugin must draw within the specified dimensions, and should NOT send a size_change_request in response to the size_change_response

When changing buffer sizes, the new size may require a larger or smaller memory buffer than the old size. In this case, the shared memory segment name received with the size_change_response may be different than the one the plugin was previously drawing to. If the plugin receives a size_change_response with a shared memory segment name it hasn't seen yet, it MUST not draw to the old shared memory segment using the new parameters, or it may overrun the end of the buffer and crash.

In this case, the plugin should expect to receive a shm_added message for the new shared memory segment either before or after the size_change_response, and must wait until it has received both before drawing to the new buffer.

Likewise, when a plugin receives a shm_remove message and it's drawing to the shared memory buffer with that name, it needs to stop doing so, since the memory buffer will be deallocated once the message handler returns.

Base messages

All plugins need to handle these messages.

Messages the plugin receives

init

Initialization message the PLS sends. Plugin responds with init_response.

No parameters.

idle

Call-in for plugin to do processing. No response required.

Parameters

Key Type Description
time Real interval (in seconds) at which idle messages are being sent.

cleanup

The plugin is about to be unloaded, and should perform any cleanup it needs to do beforehand.

shm_added

Shared memory segment has been set up.

Parameters

Key Type Description
name string
size String ?
address string Since this is going from the PLS to the plugin (not crossing a process boundary),

the pointer will actually be valid. LLSD doesn't have an encoding for a pointer (or even an unsigned 32 bit value), so this will be the pointer's lowest 32-bits of value as a hexadecimal string.

address_1 string Optional. The next 32-bits of the shm address, as above. Usually 0 or absent (unneeded) on 32-bit hosts.

shm_remove

PLS will remove the shared memory segment will be removed when plugin responds with shm_remove_response.

Parameters

Key Type Description
name String Name of memory segment to remove.

Messages the plugin sends

init_response

This message is the plugin's response to the init message sent by the PLS.

Parameters

Key Type Description
versions Map Lists version corresponding to each plugin class.
  • Key: message class name
  • Value: version string. All versions start at "1.0".
plugin_version String Free-form version string for the plugin. This is expected to contain the versions of libraries the plugin uses (for example, quicktime, webkit, and so on) and may end up in the Viewer's about box.

shm_remove_response

Plugin is done with the shared memory segment, safe to delete. Plugin responds with this message when it receives a shm_remove message.

Parameters

Key Type Description
name String Name of shared memory segment to delete.

Media messages

Messages the plugin receives

load_uri

Parameters

Key Type Description
uri String URI to load.

key_event

Parameters

Key Type Description
event String One of:
  • down
  • repeat
  • up
key String Platform-specific keycode
modifiers String Contains zero or more of the following, concatenated together with '|' characters in between. There may be a spurious trailing '|'.
  • control
  • alt -- the 'option' key on the mac
  • shift
  • meta -- the 'command' key on the mac

text_event

Text input event not associated with a single keystroke.

Parameters

Key Type Description
text String Encoded in utf8


mouse_event

Parameters

Key Type Description
event String One of:
  • down
  • move
  • up
  • double_click
x Integer x-coordinate in texture space
y Integer Y-coordinate in texture space (taking into account whether vertical flip was requested or not).
modifiers string Contains zero or more of the following, concatenated together with '|' characters in between. There may be a spurious trailing '|'.
  • control
  • alt -- the 'option' key on the mac
  • shift
  • meta -- the 'command' key on the mac

scroll_event

Parameters

Key Type Description
x Integer x-coordinate in texture space
y Integer Y-coordinate in texture space (taking into account whether vertical flip was requested or not).
modifiers string Contains zero or more of the following, concatenated together with '|' characters in between. There may be a spurious trailing '|'.
  • control
  • alt -- the 'option' key on the mac
  • shift
  • meta -- the 'command' key on the mac

size_change

Either host-initiated or after a request from the plugin.

Parameters

Key Type Description
width String
height String
texture_width String
texture_height String


name String Name of the shared memory segment the texture will draw into.

set_priority

Tells the plugin its relative priority.

Parameters

Key Type Description
priority String One of:
  • stopped: media is not playing, shouldn't need to update at all
  • hidden: media is not being displayed or is out of view, don't need to do graphic updates, but may still update audio, playhead, etc.
  • slideshow: media is being displayed, but is being severely throttled to reduce resource usage
  • low: media is in the far distance, may be rendered at reduced size
  • normal: default priority
  • high: media has user focus and/or is taking up most of the screen

edit_cut

Perform an edit-menu "cut" operation to the system clipboard.

No parameters

edit_copy

Perform an edit-menu "copy" operation to the system clipboard.

No parameters

edit_paste

Perform an edit-menu "paste" operation from the system clipboard.

No parameters

Messages the plugin sends

texture_params

  • Note that the hexadecimal GLint and GLenum values used for internalformat, format and type are fixed by the OpenGL ARB, so passing them across process boundaries should be safe.
  • Although the pixel size in bytes is completely determined by the format and type, deriving it from only those values is complex. To simplify things, the plugin must provide the pixel size to the host.

Parameters

Key Type Description
default_width String Optional. Default width for the texture.
default_height String Optional. Default height for the texture.
depth String Pixel size in bytes.
internalformat String Texture internal format for glTexImage2D. The value is a string containing hexadecimal digits, (e.g. "0xDEADBEEF") representing a GLint.
format String Texture format for glTexImage2D. The value is a string containing hexadecimal digits, (e.g. "0xDEADBEEF") representing a GLenum.
type String Texture type for glTexImage2D. The value is a string containing hexadecimal digits, (e.g. "0xDEADBEEF") representing a GLenum.
swap_bytes Boolean Optional. True if the host should use glPixelStorei(GL_UNPACK_SWAP_BYTES, 1) when loading the data. If not specified, defaults to false.
coords_opengl Boolean True if the plugin wants to work in OpenGL coordinates (where (0,0) is the lower left corner of the texture), false to work in a coordinate system where (0,0) is the upper left.
allow_downsample Boolean Optional. If true, the host may negotiate a smaller texture when the plugin's priority is reduced. The plugin should not set this if downscaling the media is more expensive than drawing it at native size.
padding String Optional. Specifies how the rows of the texture should be padded. Useful values:
  • 0 (the default) -- no padding is added
  • -1 -- pad width to the next power of two
  • 16 -- start each row on a 16 byte boundary -- NOTE: may not work correctly with odd-sized pixels like GL_RGB
  • 32 -- start each row on a 32 byte boundary -- NOTE: may not work correctly with odd-sized pixels like GL_RGB

updated

Parameters

Key Type Description
left String
top String
right String
bottom String
current_time Real Optional. Current playhead time in seconds.
duration Real Optional. Total duration of the media in seconds
current_rate Real Optional. Rate at which the media is currently playing.
loaded_duration Real Optional. For media which is partially loaded, the duration through which the media could currently play or seek. For media which can't be loaded incrementally, either omit this parameter or set it to the same value as 'duration'.

media_status

Parameters

Key Type Description
status String One of:
  • <empty string> -- not initialized or no media loaded
  • loading -- loading or navigating
  • loaded -- preroll/navigation complete
  • error -- navigation/preroll failed
  • playing -- playing (only for time-based media)
  • paused -- paused (only for time-based media)

size_change_request

When the stream wants to change its native size.

Parameters

Key Type Description
width String
height String
name String Name of the shared memory segment the texture will draw into. This may induce the host to send a size_change message. The plugin may stop drawing to the texture until an appropriate size_change message has been received.

size_change_response

Sent by the plugin in response to size_change when the size change is complete.

Parameters

Key Type Description
width String
height String
texture_width String
texture_height String
name String Name of the shared memory segment the texture will draw into

cursor_changed

Parameters

Key Type Description
name String One of:
  • <empty string>
  • arrow
  • ibeam
  • splith
  • splitv
  • hand

edit_state

Update the state of the edit menu items in the host.

Parameters

Key Type Description
cut Boolean OPTIONAL
copy Boolean OPTIONAL
paste Boolean OPTIONAL
  • items that are not present in the update message should have their state unchanged.
  • All states start out false in the host.
  • False means the operation is not possible and the menu should be disabled.


Media_browser messages

Messages the plugin receives

focus

Parameters

Key Type Description
focused Boolean true if the browser instance has focus

clear_cache

No Parameters.

clear_cookies

No Parameters.

enable_cookies

Parameters

Key Type Description
enable Boolean Whether to enable cookies.

proxy_setup

Parameters

Key Type Description
enable Boolean
host String
port String

browse_stop

No Parameters.

browse_reload

Parameters

Key Type Description
ignore_cache Boolean If true, bypass cache for this reload.

browse_forward

No Parameters.

browse_back

No Parameters.

set_status_redirect

Parameters

Key Type Description
code String The HTTP status code on which to redirect (currently, only be 404).
url String URL to which to redirect.

init_history

Parameters

Key Type Description
history Array of strings Array of string URIs to add to the browser history.

Messages the plugin sends

navigate_begin

Parameters

Key Type Description
uri String URI to which the browser is navigating.

navigate_complete

Parameters

Key Type Description
uri String URI to which the browser was navigating.
result_code String For example, 200, 404.
result_string String "OK", "Not Found", etc.
history_back_available Boolean True if the "back" button should be enabled
history_forward_available Boolean True if the "forward" button should be enabled

progress

Parameters

Key Type Description
percent String Percent of page load that has completed

status_text

Parameters

Key Type Description
status String New browser status string.

location_changed

Parameters

Key Type Description
uri String New browser URI for the navigation bar.

click_href

Parameters

Key Type Description
uri String URI of the link.
target String Target property of the link.

click_nofollow

Parameters

Key Type Description
uri String URI of the link.

Media_time messages

Messages the plugin receives

stop

No parameters.

start

Key Type Description
rate Real Optional. Rate to play at:
  • 0.0 is the movie's default rate.
  • 1.0 is normal forward (on most movies).
  • -1.0 is reverse.
  • Greater than 1.0 is fast foward.
  • Less than 1.0 is slow motion.

pause

No parameters.

seek

Key Type Description
time Real

set_loop

Key Type Description
loop Boolean True to make the media loop, false to make it play once.

set_volume

Key Type Description
volume Real
Retrieved from "https://wiki.secondlife.com/w/index.php?title=Linden_Lab_Official:Media_Plugin_System_Messages&oldid=591213"