LLMedia Documentation
Abstract
The LLMedia API provides a consistent approach to embedding various Rich Media Rendering Engines (Implementations) in a manner consistent with the needs of Second Life. The API provides a framework for linking a URL type (loosely composed of MIME Type and/or Scheme) to the interface layer of the associated Media Engine, and for passing back the rendered OpenGL texture. It also supports a variety of input controls to enable common media navigation paradigms.
Overview
- Located in the Second Life client source tree at
$BRANCH_NAME/indra/llmedia
- Allows consumers of the library to render different types of media to a well defined memory buffer
- Specifically, in the Second Life client, this memory buffer is used as the source for OpenGL textures
- It is deliberately loosely coupled to the rest of the viewer - no dependencies whatsoever on viewer code
- Each media type is defined using its MIME type (if present) or URL scheme (if present)
- Different media types are handled by implementation files specific to each type - hereafter called Impls
- A media manager class handles registration, creation, management and destruction of Impls
- Impls for new media types are easily integrated - write a C++ impl class and tell the media manager class which MIME type or URL scheme you support
Currently supported media types (Impls)
- LLMediaImplQuickTime
- Apple QuickTime media
- Broadly speaking, supports everything the Apple QuickTime player does
- Available on Win32 and Mac
- LLMediaImplGStreamer
- Supports a wide variety of QuickTime/AVI/other media depending on installed system plug-ins
- Available on Linux
- LLMediaImplLLMozLib
- LLMozLib media
- Web based media
- Very broadly speaking, supports everything that Firefox 2.0.x supports
- Notable exceptions are popups, plugins, .htaccess based authorization, feedback dialogs
API class layout
- Abstract base class (
llMediaBase
)
- Common Impl class (
llMediaImplCommon
) derives from this abstract base class and contains a set of matching (non-pure) methods along with some utility functions.
- Each Impl derives from
llMediaImplCommon
and overrides as much of it as it needs to do its work.
- Each Impl must implement an
llMediaImplMaker
and override the create()
function. This object is also used to register for mime types and schemes.
- A factory class (
llMediaImplFactory
) is responsible for registering an Impl and then making one as and when required.
- A manager class (
llMediaManager
) manages the list of Impls (including creation/deletion/interrogation) and also contains common utility functions.
- An event system based on the well known emitter/observer pattern is implemented via the (
llMediaEmitter
and llMediaObserver
)
API functions
Housekeeping
Function: |
bool init();
|
Description: |
- Local initialization, called by the media manager when creating a media source
- Different from the global
(Impl)::startUp() & (Impl)::closeDown() static methods which are designed to be called once at application startup and closedown.
|
Return value: |
true on success, false on failure
|
Function: |
bool reset();
|
Description: |
- Undoes everything init() did called when the media manager destroys a media source
|
Return value: |
true on success, false on failure
|
Function: |
bool setMimeType( const std::string mime_type );
std::string getMimeType() const;
|
Description: |
- Accessor/mutator for MIME type
|
Return value: |
true on success, false on failure
|
Function: |
std::string getMediaURL() const;
|
Description: |
|
Return value: |
- The current media URL
- This may be different from the original URL that was passed to
navigateTo - a HTTP redirect may have occurred for example
|
Function: |
std::string getVersion();
|
Description: |
- Get a version string specific to the Impl
- Format of version string is specific to each Impl
|
Return value: |
|
Function: |
bool set404RedirectUrl( std::string redirect_url );
bool clr404RedirectUrl();
|
Description: |
- Set/clear URL to visit when a 404 page is reached
- This (unusual) functionality was required for Web based Search because Amazon S3 404 pages are XML based and not HTML
|
Return value: |
true on success, false on failure
|
Function: |
bool setBackgroundColor( unsigned int red, unsigned int green, unsigned int blue ) const;
|
Description: |
- Sets the background color of the media surface
- Pass in 0 - 255 for each color component
|
Return value: |
true on success, false on failure
|
Function: |
bool setCaretColor( unsigned int red, unsigned int green, unsigned int blue ) const;
|
Description: |
- Sets the color of the caret in media impls that have one - Web related ones do for example.
- Pass in 0 - 255 for each color component
|
Return value: |
true on success, false on failure
|
Media Management
Function: |
bool updateMedia();
|
Description: |
- Tells Impl to update itself
- Must be needs to be called on a regular basis
- No guarantee in this version that this will return quickly - this is up to the Impl.
|
Return value: |
true on success, false on failure
|
Function: |
bool setRequestedMediaSize( int media_width, int media_height );
|
Description: |
- Request that the Impl change the media height and width to given dimensions in pixels
- May fail if the Impl doesn't support changing size
|
Return value: |
true on success, false on failure
|
Function: |
int getMediaWidth() const;
|
Description: |
- Gets current media width
- May change throughout lifetime of media stream
- Note: event is emitted that consumers of this library can catch when media size changes
|
Return value: |
- The current media width in pixels
|
Function: |
int getMediaHeight() const;
|
Description: |
- Gets current media height
- May change throughout lifetime of media stream
- Note: event is emitted that consumers of this library can catch when media size changes
|
Return value: |
- The current media height in pixels
|
Function: |
bool setMediaDepth( int media_depth );
|
Description: |
- Request that the Impl change the media depth to given dimensions in bytes
- May fail if the Impl doesn't support changing size
- TODO: ought this to be called setRequestedMediaDepth() ?
|
Return value: |
true on success, false on failure
|
Function: |
int getMediaDepth() const;
|
Description: |
- Gets current media depth
- May change throughout lifetime of media stream
- Note: event is emitted that consumers of this library can catch when media size changes
|
Return value: |
- The current media depth in bytes
|
Function: |
int getMediaBufferSize() const;
|
Description: |
- Gets size of media buffer for current media surface
- Important: this might NOT be the same as media width * height * depth if the Impl is padding stride length for example
|
Return value: |
- The size in bytes of the current media buffer
|
Function: |
unsigned char* getMediaData();
|
Description: |
- Returns pointer to raw media pixels
|
Return value: |
- A pointer to the current media data
|
Function: |
int getMediaDataWidth() const;
|
Description: |
- Get the width of the media data
- May be different from the size of the media if the Impl is padding stride length for example
|
Return value: |
- The width of the media data in pixels
|
Function: |
int getMediaDataHeight() const;
|
Description: |
- Get the height of the media data
- May be different from the size of the media if the Impl is padding height
- This is almost always the same as media height
|
Return value: |
- The height of the media data in pixels
|
Texture Management
Function: |
int getTextureFormatInternal() const;
|
Description: |
- Gets internal format to use for OpenGL texture
|
Return value: |
- A value for internal texture format that mirrors the OpenGL version
- Valid values are:
LL_MEDIA_RGB
LL_MEDIA_RGBA
LL_MEDIA_RGB8
LL_MEDIA_BGR
LL_MEDIA_BGRA
|
Function: |
int getTextureFormatPrimary() const;
|
Description: |
- Gets primary format to use for OpenGL texture
|
Return value: |
- A value for primary texture format that mirrors the OpenGL version
- Valid values are:
LL_MEDIA_RGB
LL_MEDIA_RGBA
LL_MEDIA_RGB8
LL_MEDIA_BGR
LL_MEDIA_BGRA
|
Function: |
int getTextureFormatType() const;
|
Description: |
- Gets format type to use for OpenGL texture
|
Return value: |
- A value for texture format type that mirrors the OpenGL version
- Valid values are:
LL_MEDIA_UNSIGNED_BYTE
LL_MEDIA_UNSIGNED_INT_8_8_8_8
LL_MEDIA_UNSIGNED_INT_8_8_8_8_REV
|
Audio
Function: |
bool setVolume( float volume );
float getVolume() const;
|
Description: |
- Accessor/mutator for volume from media stream if present
- Volume level is 0.0 (off) to 1.0 (maximum)
|
Return value: |
true on success, false on failure
|
Transport Control
Function: |
bool addCommand( ECommand cmd );
|
Description: |
- Add a transport command to the list of ones to be processed
- Note: Currently the list is only 1 entry long so adding a command will remove the existing one
- Supported commands are currently:
ECommand::COMMAND_STOP - Stop media stream if appropriate
ECommand::COMMAND_START - Start media stream if appropriate
ECommand::COMMAND_PAUSE - Pause media stream if appropriate
ECommand::COMMAND_BACK - Go back to last location in media stream if appropriate
ECommand::COMMAND_FORWARD - Go forward to next location in media stream if appropriate
|
Return value: |
true on success, false on failure
|
Function: |
bool clearCommand();
|
Description: |
- Remove a transport command from the list of ones to be processed
- Note: Currently the list is only 1 entry long so clearing a command will empty the list
|
Return value: |
true on success, false on failure
|
Function: |
bool updateCommand();
|
Description: |
- Tells transport command queue to update itself
- Must be needs to be called on a regular basis
|
Return value: |
true on success, false on failure
|
Function: |
EStatus getStatus();
|
Description: |
- Returns a state code that describes what the current transport status is
- Supported values are currently:
EStatus::STATUS_UNKNOWN - Status code is unknown
EStatus::STATUS_INITIALIZING - media source is initializing
EStatus::STATUS_NAVIGATING - media source is navigating to a new URL
EStatus::STATUS_STARTED - media source has been started
EStatus::STATUS_STOPPED - media source has been stopped
EStatus::STATUS_PAUSED - media source has been paused
EStatus::STATUS_RESETTING - media source is resetting (uninitializing)
|
Return value: |
true on success, false on failure
|
Function: |
bool seek( double time );
|
Description: |
- Seeks to a location within a media source stream
- Time is in seconds since the start of the stream
- Only meaningful for video type streams like QuickTime for example
|
Return value: |
true on success, false on failure
|
Function: |
bool setLooping( bool enable);
|
Description: |
- Turn looping on or off.
- If looping is on, media will restart from the beginning when it reaches the end
- If looping is off, media still pause on the last frame when it reaches the end
|
Return value: |
true on success, false on failure
|
Function: |
bool isLooping();
|
Description: |
- Determine if looping is switched on or off
|
Return value: |
true if looping is enabled, false if looping is disabled
|
Scaling
Function: |
bool setAutoScaled( bool auto_scaled );
|
Description: |
- Turn media autoscaling on or off
- Autoscale means try to scale media to size of texture
- May fail if media doesn't support size change
|
Return value: |
true on success, false on failure
|
Function: |
bool isAutoScaled() const;
|
Description: |
- Determine if autoscaling is turned on or off
|
Return value: |
true if autoscaling is turned on, false if autoscaling is turned off
|
Mouse and Keyboard
Function: |
bool mouseDown( int x_pos, int y_pos );
|
Description: |
- Inject left mouse button press at given X/Y coordinate within media surface
|
Return value: |
true on success, false on failure
|
Function: |
bool mouseUp( int x_pos, int y_pos );
|
Description: |
- Inject left mouse button release at given X/Y coordinate within media surface
|
Return value: |
true on success, false on failure
|
Function: |
bool mouseLeftDoubleClick( int x_pos, int y_pos );
|
Description: |
- Inject left mouse button double click at given X/Y coordinate within media surface
|
Return value: |
true on success, false on failure
|
Function: |
bool mouseMove( int x_pos, int y_pos );
|
Description: |
- Inject mouse cursor move to given X/Y coordinate within media surface
|
Return value: |
true on success, false on failure
|
Function: |
bool keyPress( int key_code );
|
Description: |
- Inject key press with given ASCII value into media surface
- No support currently for discrete key down/up events
|
Return value: |
true on success, false on failure
|
Function: |
bool unicodeInput( unsigned long uni_char );
|
Description: |
- Inject key press with given UNICODE value into media surface
- No support currently for discrete key down/up events
|
Return value: |
true on success, false on failure
|
Function: |
bool scrollByLines( int lines );
|
Description: |
- Scroll display by number of lines if appropriate (for example, in a Web related media)
|
Return value: |
true on success, false on failure
|
Function: |
bool focus( bool focus );
|
Description: |
- Enable/disable focus
- If media Impl doesn't have focus, keyboard input will not succeed
|
Return value: |
true on success, false on failure
|
Navigation
Function: |
bool navigateTo( const std::string url );
|
Description: |
- Request navigation to given URL
- Typically the actual navigation happens asynchronously
|
Return value: |
true on success, false on failure
- Note: success here means the navigate command succeeded. The actual navigation may fail later
|
Function: |
bool navigateForward();
|
Description: |
- Navigate forward to next location in media source
- In a Web related media source this means go to next page in history stack
- In a video related media source, this means go to next chapter (currently unimplemented)
|
Return value: |
true on success, false on failure
|
Function: |
bool navigateBack();
|
Description: |
- Navigate backward to previous location in media source
- In a Web related media source this means go to previous page in history stack
- In a video related media source, this means go to previous chapter (currently unimplemented)
|
Return value: |
true on success, false on failure
- Note: success here means the navigate command succeeded. The actual navigation may fail later
|
Function: |
bool canNavigateForward();
|
Description: |
- Determine if it's possible to go forward to next location in media source
|
Return value: |
true if it's possible to navigate forwards in the history stack, false if it's not possible to navigate forwards in the history stack
|
Function: |
bool canNavigateBack();
|
Description: |
- Determine if it's possible to go back to previous location in media source
|
Return value: |
true if it's possible to navigate backwards in the history stack, false if it's not possible to navigate backwards in the history stack
|
Caching and Cookies
Function: |
bool enableCookies( bool enable );
|
Description: |
- Enable/disable the saving of cookies
- Currently only supported on a global level - all or nothing
- Probably only meaningful for Web related media sources
|
Return value: |
true on success, false on failure
|
Function: |
bool clearCookies();
|
Description: |
- clears any current set cookies
- Only meaningful for Web related media sources
|
Return value: |
true on success, false on failure
|
Function: |
bool clearCache();
|
Description: |
- Clears the cache
- Meaningful for Web related media
- May be meaningful for QuickTime media too
|
Return value: |
true on success, false on failure
|
Proxy
Function: |
bool enableProxy(bool enable, std::string proxy_host_name, int proxy_port);
|
Description: |
- Enable/disable support for network proxying.
- Each Impl is responsible for implementing the proxy support itself
- Currently only supported for LLMediaImplLLMozLib
|
Return value: |
true on success, false on failure
|
Emitter/Observer
Function: |
bool addObserver( LLMediaObserver* subject );
|
Description: |
- Add yourself as an observer of the media library
- See sample code for examples of how to do this
|
Return value: |
true on success, false on failure
|
Observable events
- Add class as an observer to the media library using
addObserver
and remObserver
- To observe specific events, override one or more of these methods in your own code
onMediaPreroll
- emitted when the media is prerolling
onMediaLoaded
- emitted when the media is loaded
onMediaSizeChange
- emitted when the media changes size
onMediaContentsChange
- emitted when the contents of the media stream changed
onMediaStatusTextChange
- emitted when the status text associated with the media stream changed
onNavigateBegin
- emitted when navigation to a new URL starts
onNavigateComplete
- emitted when navigation to a new URL ends
onUpdateProgress
- emitted when navigation to a new URL makes progress
onLocationChange
- emitted when a URL changes - for example, an HTTP redirect
onClickLinkHref
- emitted when a link is clicked in a Web related media source
onClickLinkNoFollow
- emitted when a link that has been marked as not to follow is clicked in a Web related media source - a secondlife:// URL for example
- Event payload is also passed in with data specific to the nature of the event
Examples
- Example code exists in 2 places:
- In the LLMedia library itself, there are 2 example Impls called (unimaginatively) Example1 and Example2.
- Example1 drags a random pattern on the screen and changes the contents when you move the mouse over it
- Example2 bounces a square around the media area
- In the llmediatest application in the indra/test_apps directory.
- Simple media browser
- Dependency on GLUI for its UI
Current Impls
- LLMediaImplExample1 - Example Impl that displays a random pattern of squares and writes data into the media surface at the mouse position
- LLMediaImplExample2 - Example Impl that bounces a random colored box around within the media surface
- LLMediaImplLLMozLib - provides Web based content via LLMozLib library (which, itself wraps Mozilla/Gecko 1.8.1)
- LLMediaImplQuickTime - provides QuickTime media content (Movies, images, RTSP streams)