LLMedia API

LLMedia Documentation

Abstract

The LLMedia API provides a consistent approach to embedding various Rich Media Rendering Engines (Implementations) in a manner consistent with the needs of Second Life. The API provides a framework for linking a URL type (loosely composed of MIME Type and/or Scheme) to the interface layer of the associated Media Engine, and for passing back the rendered OpenGL texture. It also supports a variety of input controls to enable common media navigation paradigms.

Overview

• Located in the Second Life client source tree at $BRANCH_NAME/indra/llmedia
• Allows consumers of the library to render different types of media to a well defined memory buffer
• Specifically, in the Second Life client, this memory buffer is used as the source for OpenGL textures
• It is deliberately loosely coupled to the rest of the viewer - no dependencies whatsoever on viewer code
• Each media type is defined using it's MIME type (if present) or URL scheme (if present)
• Different media types are handled by implementation files specific to each type - hereafter called Impls
• A media manager class handles registration, creation, management and destruction of Impls
• Impls for new media types are easily integrated - write a C++ impl class and tell the media manager class which MIME type or URL scheme you support

Currently supported media types (Impls)

• LLMediaImplQuickTime
  • Apple QuickTime media
  • Broadly speaking, supports everything the Apple QuickTime player does
  • Available on Win32 and Mac

• LLMediaImplGStreamer
  • Supports a wide variety of QuickTime/AVI/other media depending on installed system plug-ins
  • Available on Linux

• LLMediaImplLLMozLib
  • LLMozLib media
  • Web based media
  • Very broadly speaking, supports everything that Firefox 2.0.x supports
  • Notable exceptions are popups, plugins, .htaccess based authorization, feedback dialogs

API class layout

• Abstract base class (llMediaBase)
• Common Impl class (llMediaImplCommon) derrives from this abstract base class and contains a set of matching (non-pure) methods along with some utility functions.
• Each Impl derrives from llMediaImplCommon and overrides as much of it as it needs to do its work.
• Each Impl must implement an llMediaImplMaker and override the create() function. This object is also used to register for mime types and schemes.
• A factory class (llMediaImplFactory) is responsible for registering an Impl and then making one as and when required.
• A manager class (llMediaManager) manages the list of Impls (including creation/deletion/interrogation) and also contains common utility functions.
• An event system based on the well known emitter/observer pattern is implemented via the (llMediaEmitter and llMediaObserver)

API functions

Housekeeping

Function:	bool init();
Description:	Local initialization, called by the media manager when creating a media source Different from the global (Impl)::startUp() & (Impl)::closeDown() static methods which are designed to be called once at application startup and closedown.
Return value:	true on success, false on failure

---

 

Function:	bool reset();
Description:	Undoes everything init() did called when the media manager destroys a media source
Return value:	true on success, false on failure

---

 

Function:	bool setMimeType( const std::string mime_type ); std::string getMimeType() const;
Description:	Accessor/mutator for MIME type
Return value:	true on success, false on failure

---

 

Function:	std::string getMediaURL() const;
Description:	Accessor for current URL
Return value:	The current media URL This may be different from the original URL that was passed to navigateTo - a HTTP redirect may have occurred for example

---

 

Function:	std::string getVersion();
Description:	Get a version string specific to the Impl Format of version string is specific to each Impl
Return value:	The version string

---

 

Function:	bool set404RedirectUrl( std::string redirect_url ); bool clr404RedirectUrl();
Description:	Set/clear URL to visit when a 404 page is reached This (unusual) functionality was required for Web based Search because Amazon S3 404 pages are XML based and not HTML
Return value:	true on success, false on failure

---

 

Function:	bool setBackgroundColor( unsigned int red, unsigned int green, unsigned int blue ) const;
Description:	Sets the background color of the media surface Pass in 0 - 255 for each color component
Return value:	true on success, false on failure

---

 

Function:	bool setCaretColor( unsigned int red, unsigned int green, unsigned int blue ) const;
Description:	Sets the color of the caret in media impls that have one - Web related ones do for example. Pass in 0 - 255 for each color component
Return value:	true on success, false on failure

---

 

Media Management

Function:	bool updateMedia();
Description:	Tells Impl to update itself Must be needs to be called on a regular basis No guarantee in this version that this will return quickly - this is up to the Impl.
Return value:	true on success, false on failure

---

 

Function:	bool setRequestedMediaSize( int media_width, int media_height );
Description:	Request that the Impl change the media height and width to given dimensions in pixels May fail if the Impl doesn't support changing size
Return value:	true on success, false on failure

---

 

Function:	int getMediaWidth() const;
Description:	Gets current media width May change throughout lifetime of media stream Note: event is emitted that consumers of this library can catch when media size changes
Return value:	The current media width in pixels

---

 

Function:	int getMediaHeight() const;
Description:	Gets current media height May change throughout lifetime of media stream Note: event is emitted that consumers of this library can catch when media size changes
Return value:	The current media height in pixels

---

 

Function:	bool setMediaDepth( int media_depth );
Description:	Request that the Impl change the media depth to given dimensions in bytes May fail if the Impl doesn't support changing size TODO: ought this to be called setRequestedMediaDepth() ?
Return value:	true on success, false on failure

---

 

Function:	int getMediaDepth() const;
Description:	Gets current media depth May change throughout lifetime of media stream Note: event is emitted that consumers of this library can catch when media size changes
Return value:	The current media depth in bytes

---

 

Function:	int getMediaBufferSize() const;
Description:	Gets size of media buffer for current media surface Important: this might NOT be the same as media width * height * depth if the Impl is padding stride length for example
Return value:	The size in bytes of the current media buffer

---

 

Function:	unsigned char* getMediaData();
Description:	Returns pointer to raw media pixels
Return value:	A pointer to the current media data

---

 

Function:	int getMediaDataWidth() const;
Description:	Get the width of the media data May be different from the size of the media if the Impl is padding stride length for example
Return value:	The width of the media data in pixels

---

 

Function:	int getMediaDataHeight() const;
Description:	Get the height of the media data May be different from the size of the media if the Impl is padding height This is almost always the same as media height
Return value:	The height of the media data in pixels

---

 

Texture Management

Function:	int getTextureFormatInternal() const;
Description:	Gets internal format to use for OpenGL texture
Return value:	A value for internal texture format that mirrors the OpenGL version Valid values are: LL_MEDIA_RGB LL_MEDIA_RGBA LL_MEDIA_RGB8 LL_MEDIA_BGR LL_MEDIA_BGRA

---

 

Function:	int getTextureFormatPrimary() const;
Description:	Gets primary format to use for OpenGL texture
Return value:	A value for primary texture format that mirrors the OpenGL version Valid values are: LL_MEDIA_RGB LL_MEDIA_RGBA LL_MEDIA_RGB8 LL_MEDIA_BGR LL_MEDIA_BGRA

---

 

Function:	int getTextureFormatType() const;
Description:	Gets format type to use for OpenGL texture
Return value:	A value for texture format type that mirrors the OpenGL version Valid values are: LL_MEDIA_UNSIGNED_BYTE LL_MEDIA_UNSIGNED_INT_8_8_8_8 LL_MEDIA_UNSIGNED_INT_8_8_8_8_REV

---

 

Audio

Function:	bool setVolume( float volume ); float getVolume() const;
Description:	Accessor/mutator for volume from media stream if present Volume level is 0.0 (off) to 1.0 (maximum)
Return value:	true on success, false on failure

---

 

Transport Control

Function:	bool addCommand( ECommand cmd );
Description:	Add a transport command to the list of ones to be processed Note: Currently the list is only 1 entry long so adding a command will remove the existing one Supported commands are currently: ECommand::COMMAND_STOP - Stop media stream if appropriate ECommand::COMMAND_START - Start media stream if appropriate ECommand::COMMAND_PAUSE - Pause media stream if appropriate ECommand::COMMAND_BACK - Go back to last location in media stream if appropriate ECommand::COMMAND_FORWARD - Go forward to next location in media stream if appropriate
Return value:	true on success, false on failure

---

 

Function:	bool clearCommand();
Description:	Remove a transport command from the list of ones to be processed Note: Currently the list is only 1 entry long so clearing a command will empty the list
Return value:	true on success, false on failure

---

 

Function:	bool updateCommand();
Description:	Tells transport command queue to update itself Must be needs to be called on a regular basis
Return value:	true on success, false on failure

---

 

Function:	EStatus getStatus();
Description:	Returns a state code that describes what the current transport status is Supported values are currently: EStatus::STATUS_UNKNOWN - Status code is unknown EStatus::STATUS_INITIALIZING - media source is initializing EStatus::STATUS_NAVIGATING - media source is navigating to a new URL EStatus::STATUS_STARTED - media source has been started EStatus::STATUS_STOPPED - media source has been stopped EStatus::STATUS_PAUSED - media source has been paused EStatus::STATUS_RESETTING - media source is resetting (uninitializing)
Return value:	true on success, false on failure

---

 

Function:	bool seek( double time );
Description:	Seeks to a location within a media source stream Time is in seconds since the start of the stream Only meaningful for video type streams like QuickTime for example
Return value:	true on success, false on failure

---

 

Function:	bool setLooping( bool enable);
Description:	Turn looping on or off. If looping is on, media will restart from the beginning when it reaches the end If looping is off, media still pause on the last frame when it reaches the end
Return value:	true on success, false on failure

---

 

Function:	bool isLooping();
Description:	Determine if looping is switched on or off
Return value:	true if looping is enabled, false if looping is disabled

---

 

Scaling

Function:	bool setAutoScaled( bool auto_scaled );
Description:	Turn media autoscaling on or off Autoscale means try to scale media to size of texture May fail if media doesn't support size change
Return value:	true on success, false on failure

---

 

Function:	bool isAutoScaled() const;
Description:	Determine if autoscaling is turned on or off
Return value:	true if autoscaling is turned on, false if autoscaling is turned off

---

 

Mouse and Keyboard

Function:	bool mouseDown( int x_pos, int y_pos );
Description:	Inject left mouse button press at given X/Y coordinate within media surface
Return value:	true on success, false on failure

---

 

Function:	bool mouseUp( int x_pos, int y_pos );
Description:	Inject left mouse button release at given X/Y coordinate within media surface
Return value:	true on success, false on failure

---

 

Function:	bool mouseLeftDoubleClick( int x_pos, int y_pos );
Description:	Inject left mouse button double click at given X/Y coordinate within media surface
Return value:	true on success, false on failure

---

 

Function:	bool mouseMove( int x_pos, int y_pos );
Description:	Inject mouse cursor move to given X/Y coordinate within media surface
Return value:	true on success, false on failure

---

 

Function:	bool keyPress( int key_code );
Description:	Inject key press with given ASCII value into media surface No support currently for discrete key down/up events
Return value:	true on success, false on failure

---

 

Function:	bool unicodeInput( unsigned long uni_char );
Description:	Inject key press with given UNICODE value into media surface No support currently for discrete key down/up events
Return value:	true on success, false on failure

---

 

Function:	bool scrollByLines( int lines );
Description:	Scroll display by number of lines if appropriate (for example, in a Web related media)
Return value:	true on success, false on failure

---

 

Function:	bool focus( bool focus );
Description:	Enable/disable focus If media Impl doesn't have focus, keyboard input will not succeed
Return value:	true on success, false on failure

---

 

Navigation

Function:	bool navigateTo( const std::string url );
Description:	Request navigation to given URL Typically the actual navigation happens asynchronously
Return value:	true on success, false on failure Note: success here means the navigate command succeeded. The actual navigation may fail later

---

 

Function:	bool navigateForward();
Description:	Navigate forward to next location in media source In a Web related media source this means go to next page in history stack In a video related media source, this means go to next chapter (currently unimplemented)
Return value:	true on success, false on failure

---

 

Function:	bool navigateBack();
Description:	Navigate backward to previous location in media source In a Web related media source this means go to previous page in history stack In a video related media source, this means go to previous chapter (currently unimplemented)
Return value:	true on success, false on failure Note: success here means the navigate command succeeded. The actual navigation may fail later

---

 

Function:	bool canNavigateForward();
Description:	Determine if it's possible to go forward to next location in media source
Return value:	true if it's possible to navigate forwards in the history stack, false if it's not possible to navigate forwards in the history stack

---

 

Function:	bool canNavigateBack();
Description:	Determine if it's possible to go back to previous location in media source
Return value:	true if it's possible to navigate backwards in the history stack, false if it's not possible to navigate backwards in the history stack

---

 

Caching and Cookies

Function:	bool enableCookies( bool enable );
Description:	Enable/disable the saving of cookies Currently only supported on a global level - all or nothing Probably only meaningful for Web related media sources
Return value:	true on success, false on failure

---

 

Function:	bool clearCookies();
Description:	clears any current set cookies Only meaningful for Web related media sources
Return value:	true on success, false on failure

---

 

Function:	bool clearCache();
Description:	Clears the cache Meaningful for Web related media May be meaningful for QuickTime media too
Return value:	true on success, false on failure

---

 

Proxy

Function:	bool enableProxy(bool enable, std::string proxy_host_name, int proxy_port);
Description:	Enable/disable support for network proxying. Each Impl is responsible for implementing the proxy support itself Currently only supported for LLMediaImplLLMozLib
Return value:	true on success, false on failure

---

 

Emitter/Observer

Function:	bool addObserver( LLMediaObserver* subject );
Description:	Add yourself as an observer of the media library See sample code for examples of how to do this
Return value:	true on success, false on failure

---

 

Observable events

• Add class as an observer to the media library using addObserver and remObserver
• To observe specific events, override one or more of these methods in your own code
  • onMediaPreroll - emitted when the media is prerolling
  • onMediaLoaded - emitted when the media is loaded
  • onMediaSizeChange - emitted when the media changes size
  • onMediaContentsChange - emitted when the contents of the media stream changed
  • onMediaStatusTextChange - emitted when the status stext associated with the medias stream changed
  • onNavigateBegin - emitted when navigation to a new URL starts
  • onNavigateComplete - emitted when navigation to a new URL ends
  • onUpdateProgress - emitted when navigation to a new URL makes progress
  • onLocationChange - emitted when a URL changes - for example, an HTTP redirect
  • onClickLinkHref - emitted when a link is clicked in a Web related media source
  • onClickLinkNoFollow - emitted when a link that has been marked as not to follow is clicked in a Web related media source - a secondlife:// URL for example
• Event payload is also passed in with data specific to the nature of the event

Examples

• Example code exists in 2 places:

• • In the LLMedia library itself, there are 2 example Impls called (unimaginatively) Example1 and Example2.
    • Example1 drags a random pattern on the screen and changes the contents when you move the mouse over it
    • Example2 bounces a square around the media area

• • In the llmediatest application in the indra/test_apps directory.
    • Simple media browser
    • Dependency on GLUI for its UI

Current Impls

• LLMediaImplExample1 - Example Impl that displays a random pattern of squares and writes data into the media surface at the mouse position
• LLMediaImplExample2 - Example Impl that bounces a random colored box around within the media surface
• LLMediaImplLLMozLib - provides Web based content via LLMozLib library (which, itself wraps Mozilla/Gecko 1.8.1)
• LLMediaImplQuickTime - provides QuickTime media content (Movies, images, RTSP streams)