LLMedia API

From Second Life Wiki
Jump to navigation Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.
NOTE: This system is deprecated. Use the new Media Rendering Plugin Framework instead.


LLMedia Documentation

Abstract

The LLMedia API provides a consistent approach to embedding various Rich Media Rendering Engines (Implementations) in a manner consistent with the needs of Second Life. The API provides a framework for linking a URL type (loosely composed of MIME Type and/or Scheme) to the interface layer of the associated Media Engine, and for passing back the rendered OpenGL texture. It also supports a variety of input controls to enable common media navigation paradigms.


Overview

  • Located in the Second Life client source tree at $BRANCH_NAME/indra/llmedia
  • Allows consumers of the library to render different types of media to a well defined memory buffer
  • Specifically, in the Second Life client, this memory buffer is used as the source for OpenGL textures
  • It is deliberately loosely coupled to the rest of the viewer - no dependencies whatsoever on viewer code
  • Each media type is defined using its MIME type (if present) or URL scheme (if present)
  • Different media types are handled by implementation files specific to each type - hereafter called Impls
  • A media manager class handles registration, creation, management and destruction of Impls
  • Impls for new media types are easily integrated - write a C++ impl class and tell the media manager class which MIME type or URL scheme you support


Currently supported media types (Impls)

  • LLMediaImplQuickTime
    • Apple QuickTime media
    • Broadly speaking, supports everything the Apple QuickTime player does
    • Available on Win32 and Mac
  • LLMediaImplGStreamer
    • Supports a wide variety of QuickTime/AVI/other media depending on installed system plug-ins
    • Available on Linux
  • LLMediaImplLLMozLib
    • LLMozLib media
    • Web based media
    • Very broadly speaking, supports everything that Firefox 2.0.x supports
    • Notable exceptions are popups, plugins, .htaccess based authorization, feedback dialogs

API class layout

  • Abstract base class (llMediaBase)
  • Common Impl class (llMediaImplCommon) derives from this abstract base class and contains a set of matching (non-pure) methods along with some utility functions.
  • Each Impl derives from llMediaImplCommon and overrides as much of it as it needs to do its work.
  • Each Impl must implement an llMediaImplMaker and override the create() function. This object is also used to register for mime types and schemes.
  • A factory class (llMediaImplFactory) is responsible for registering an Impl and then making one as and when required.
  • A manager class (llMediaManager) manages the list of Impls (including creation/deletion/interrogation) and also contains common utility functions.
  • An event system based on the well known emitter/observer pattern is implemented via the (llMediaEmitter and llMediaObserver)

API functions

Housekeeping

Function:

 bool init();

Description:
  • Local initialization, called by the media manager when creating a media source
  • Different from the global (Impl)::startUp() & (Impl)::closeDown() static methods which are designed to be called once at application startup and closedown.
Return value:
  • true on success, false on failure

 

Function:

 bool reset();

Description:
  • Undoes everything init() did called when the media manager destroys a media source
Return value:
  • true on success, false on failure

 

Function:

 bool setMimeType( const std::string mime_type ); std::string getMimeType() const;

Description:
  • Accessor/mutator for MIME type
Return value:
  • true on success, false on failure

 

Function:

 std::string getMediaURL() const;

Description:
  • Accessor for current URL
Return value:
  • The current media URL
  • This may be different from the original URL that was passed to navigateTo - a HTTP redirect may have occurred for example

 

Function:

 std::string getVersion();

Description:
  • Get a version string specific to the Impl
  • Format of version string is specific to each Impl
Return value:
  • The version string

 

Function:

 bool set404RedirectUrl( std::string redirect_url ); bool clr404RedirectUrl();

Description:
  • Set/clear URL to visit when a 404 page is reached
  • This (unusual) functionality was required for Web based Search because Amazon S3 404 pages are XML based and not HTML
Return value:
  • true on success, false on failure

 

Function:

 bool setBackgroundColor( unsigned int red, unsigned int green, unsigned int blue ) const;

Description:
  • Sets the background color of the media surface
  • Pass in 0 - 255 for each color component
Return value:
  • true on success, false on failure

 

Function:

 bool setCaretColor( unsigned int red, unsigned int green, unsigned int blue ) const;

Description:
  • Sets the color of the caret in media impls that have one - Web related ones do for example.
  • Pass in 0 - 255 for each color component
Return value:
  • true on success, false on failure

 

Media Management

Function:

 bool updateMedia();

Description:
  • Tells Impl to update itself
  • Must be needs to be called on a regular basis
  • No guarantee in this version that this will return quickly - this is up to the Impl.
Return value:
  • true on success, false on failure

 

Function:

 bool setRequestedMediaSize( int media_width, int media_height );

Description:
  • Request that the Impl change the media height and width to given dimensions in pixels
  • May fail if the Impl doesn't support changing size
Return value:
  • true on success, false on failure

 

Function:

 int getMediaWidth() const;

Description:
  • Gets current media width
  • May change throughout lifetime of media stream
  • Note: event is emitted that consumers of this library can catch when media size changes
Return value:
  • The current media width in pixels

 

Function:

 int getMediaHeight() const;

Description:
  • Gets current media height
  • May change throughout lifetime of media stream
  • Note: event is emitted that consumers of this library can catch when media size changes
Return value:
  • The current media height in pixels

 

Function:

 bool setMediaDepth( int media_depth );

Description:
  • Request that the Impl change the media depth to given dimensions in bytes
  • May fail if the Impl doesn't support changing size
  • TODO: ought this to be called setRequestedMediaDepth() ?
Return value:
  • true on success, false on failure

 

Function:

 int getMediaDepth() const;

Description:
  • Gets current media depth
  • May change throughout lifetime of media stream
  • Note: event is emitted that consumers of this library can catch when media size changes
Return value:
  • The current media depth in bytes

 

Function:

 int getMediaBufferSize() const;

Description:
  • Gets size of media buffer for current media surface
  • Important: this might NOT be the same as media width * height * depth if the Impl is padding stride length for example
Return value:
  • The size in bytes of the current media buffer

 

Function:

 unsigned char* getMediaData();

Description:
  • Returns pointer to raw media pixels
Return value:
  • A pointer to the current media data

 

Function:

 int getMediaDataWidth() const;

Description:
  • Get the width of the media data
  • May be different from the size of the media if the Impl is padding stride length for example
Return value:
  • The width of the media data in pixels

 

Function:

 int getMediaDataHeight() const;

Description:
  • Get the height of the media data
  • May be different from the size of the media if the Impl is padding height
  • This is almost always the same as media height
Return value:
  • The height of the media data in pixels

 

Texture Management

Function:

 int getTextureFormatInternal() const;

Description:
  • Gets internal format to use for OpenGL texture
Return value:
  • A value for internal texture format that mirrors the OpenGL version
  • Valid values are:
    • LL_MEDIA_RGB
    • LL_MEDIA_RGBA
    • LL_MEDIA_RGB8
    • LL_MEDIA_BGR
    • LL_MEDIA_BGRA

 

Function:

 int getTextureFormatPrimary() const;

Description:
  • Gets primary format to use for OpenGL texture
Return value:
  • A value for primary texture format that mirrors the OpenGL version
  • Valid values are:
    • LL_MEDIA_RGB
    • LL_MEDIA_RGBA
    • LL_MEDIA_RGB8
    • LL_MEDIA_BGR
    • LL_MEDIA_BGRA

 

Function:

 int getTextureFormatType() const;

Description:
  • Gets format type to use for OpenGL texture
Return value:
  • A value for texture format type that mirrors the OpenGL version
  • Valid values are:
    • LL_MEDIA_UNSIGNED_BYTE
    • LL_MEDIA_UNSIGNED_INT_8_8_8_8
    • LL_MEDIA_UNSIGNED_INT_8_8_8_8_REV

 

Audio

Function:

 bool setVolume( float volume ); float getVolume() const;

Description:
  • Accessor/mutator for volume from media stream if present
  • Volume level is 0.0 (off) to 1.0 (maximum)
Return value:
  • true on success, false on failure

 

Transport Control

Function:

 bool addCommand( ECommand cmd );

Description:
  • Add a transport command to the list of ones to be processed
  • Note: Currently the list is only 1 entry long so adding a command will remove the existing one
  • Supported commands are currently:
    • ECommand::COMMAND_STOP - Stop media stream if appropriate
    • ECommand::COMMAND_START - Start media stream if appropriate
    • ECommand::COMMAND_PAUSE - Pause media stream if appropriate
    • ECommand::COMMAND_BACK - Go back to last location in media stream if appropriate
    • ECommand::COMMAND_FORWARD - Go forward to next location in media stream if appropriate
Return value:
  • true on success, false on failure

 

Function:

 bool clearCommand();

Description:
  • Remove a transport command from the list of ones to be processed
  • Note: Currently the list is only 1 entry long so clearing a command will empty the list
Return value:
  • true on success, false on failure

 

Function:

 bool updateCommand();

Description:
  • Tells transport command queue to update itself
  • Must be needs to be called on a regular basis
Return value:
  • true on success, false on failure

 

Function:

 EStatus getStatus();

Description:
  • Returns a state code that describes what the current transport status is
  • Supported values are currently:
    • EStatus::STATUS_UNKNOWN - Status code is unknown
    • EStatus::STATUS_INITIALIZING - media source is initializing
    • EStatus::STATUS_NAVIGATING - media source is navigating to a new URL
    • EStatus::STATUS_STARTED - media source has been started
    • EStatus::STATUS_STOPPED - media source has been stopped
    • EStatus::STATUS_PAUSED - media source has been paused
    • EStatus::STATUS_RESETTING - media source is resetting (uninitializing)
Return value:
  • true on success, false on failure

 

Function:

 bool seek( double time );

Description:
  • Seeks to a location within a media source stream
  • Time is in seconds since the start of the stream
  • Only meaningful for video type streams like QuickTime for example
Return value:
  • true on success, false on failure

 

Function:

 bool setLooping( bool enable);

Description:
  • Turn looping on or off.
  • If looping is on, media will restart from the beginning when it reaches the end
  • If looping is off, media still pause on the last frame when it reaches the end
Return value:
  • true on success, false on failure

 

Function:

 bool isLooping();

Description:
  • Determine if looping is switched on or off
Return value:
  • true if looping is enabled, false if looping is disabled

 

Scaling

Function:

 bool setAutoScaled( bool auto_scaled );

Description:
  • Turn media autoscaling on or off
  • Autoscale means try to scale media to size of texture
  • May fail if media doesn't support size change
Return value:
  • true on success, false on failure

 

Function:

 bool isAutoScaled() const;

Description:
  • Determine if autoscaling is turned on or off
Return value:
  • true if autoscaling is turned on, false if autoscaling is turned off

 

Mouse and Keyboard

Function:

 bool mouseDown( int x_pos, int y_pos );

Description:
  • Inject left mouse button press at given X/Y coordinate within media surface
Return value:
  • true on success, false on failure

 

Function:

 bool mouseUp( int x_pos, int y_pos );

Description:
  • Inject left mouse button release at given X/Y coordinate within media surface
Return value:
  • true on success, false on failure

 

Function:

 bool mouseLeftDoubleClick( int x_pos, int y_pos );

Description:
  • Inject left mouse button double click at given X/Y coordinate within media surface
Return value:
  • true on success, false on failure

 

Function:

 bool mouseMove( int x_pos, int y_pos );

Description:
  • Inject mouse cursor move to given X/Y coordinate within media surface
Return value:
  • true on success, false on failure

 

Function:

 bool keyPress( int key_code );

Description:
  • Inject key press with given ASCII value into media surface
  • No support currently for discrete key down/up events
Return value:
  • true on success, false on failure

 

Function:

 bool unicodeInput( unsigned long uni_char );

Description:
  • Inject key press with given UNICODE value into media surface
  • No support currently for discrete key down/up events
Return value:
  • true on success, false on failure

 

Function:

 bool scrollByLines( int lines );

Description:
  • Scroll display by number of lines if appropriate (for example, in a Web related media)
Return value:
  • true on success, false on failure

 

Function:

 bool focus( bool focus );

Description:
  • Enable/disable focus
  • If media Impl doesn't have focus, keyboard input will not succeed
Return value:
  • true on success, false on failure

 

Navigation

Function:

 bool navigateTo( const std::string url );

Description:
  • Request navigation to given URL
  • Typically the actual navigation happens asynchronously
Return value:
  • true on success, false on failure
  • Note: success here means the navigate command succeeded. The actual navigation may fail later

 

Function:

 bool navigateForward();

Description:
  • Navigate forward to next location in media source
  • In a Web related media source this means go to next page in history stack
  • In a video related media source, this means go to next chapter (currently unimplemented)
Return value:
  • true on success, false on failure

 

Function:

 bool navigateBack();

Description:
  • Navigate backward to previous location in media source
  • In a Web related media source this means go to previous page in history stack
  • In a video related media source, this means go to previous chapter (currently unimplemented)
Return value:
  • true on success, false on failure
  • Note: success here means the navigate command succeeded. The actual navigation may fail later

 

Function:

 bool canNavigateForward();

Description:
  • Determine if it's possible to go forward to next location in media source
Return value:
  • true if it's possible to navigate forwards in the history stack, false if it's not possible to navigate forwards in the history stack

 

Function:

 bool canNavigateBack();

Description:
  • Determine if it's possible to go back to previous location in media source
Return value:
  • true if it's possible to navigate backwards in the history stack, false if it's not possible to navigate backwards in the history stack

 

Caching and Cookies

Function:

 bool enableCookies( bool enable );

Description:
  • Enable/disable the saving of cookies
  • Currently only supported on a global level - all or nothing
  • Probably only meaningful for Web related media sources
Return value:
  • true on success, false on failure

 

Function:

 bool clearCookies();

Description:
  • clears any current set cookies
  • Only meaningful for Web related media sources
Return value:
  • true on success, false on failure

 

Function:

 bool clearCache();

Description:
  • Clears the cache
  • Meaningful for Web related media
  • May be meaningful for QuickTime media too
Return value:
  • true on success, false on failure

 

Proxy

Function:

 bool enableProxy(bool enable, std::string proxy_host_name, int proxy_port);

Description:
  • Enable/disable support for network proxying.
  • Each Impl is responsible for implementing the proxy support itself
  • Currently only supported for LLMediaImplLLMozLib
Return value:
  • true on success, false on failure

 

Emitter/Observer

Function:

 bool addObserver( LLMediaObserver* subject );

Description:
  • Add yourself as an observer of the media library
  • See sample code for examples of how to do this
Return value:
  • true on success, false on failure

 


Observable events

  • Add class as an observer to the media library using addObserver and remObserver
  • To observe specific events, override one or more of these methods in your own code
    • onMediaPreroll - emitted when the media is prerolling
    • onMediaLoaded - emitted when the media is loaded
    • onMediaSizeChange - emitted when the media changes size
    • onMediaContentsChange - emitted when the contents of the media stream changed
    • onMediaStatusTextChange - emitted when the status text associated with the media stream changed
    • onNavigateBegin - emitted when navigation to a new URL starts
    • onNavigateComplete - emitted when navigation to a new URL ends
    • onUpdateProgress - emitted when navigation to a new URL makes progress
    • onLocationChange - emitted when a URL changes - for example, an HTTP redirect
    • onClickLinkHref - emitted when a link is clicked in a Web related media source
    • onClickLinkNoFollow - emitted when a link that has been marked as not to follow is clicked in a Web related media source - a secondlife:// URL for example
  • Event payload is also passed in with data specific to the nature of the event

Examples

  • Example code exists in 2 places:
    • In the LLMedia library itself, there are 2 example Impls called (unimaginatively) Example1 and Example2.
      • Example1 drags a random pattern on the screen and changes the contents when you move the mouse over it
      • Example2 bounces a square around the media area
    • In the llmediatest application in the indra/test_apps directory.
      • Simple media browser
      • Dependency on GLUI for its UI


Current Impls

  • LLMediaImplExample1 - Example Impl that displays a random pattern of squares and writes data into the media surface at the mouse position
  • LLMediaImplExample2 - Example Impl that bounces a random colored box around within the media surface
  • LLMediaImplLLMozLib - provides Web based content via LLMozLib library (which, itself wraps Mozilla/Gecko 1.8.1)
  • LLMediaImplQuickTime - provides QuickTime media content (Movies, images, RTSP streams)
Retrieved from "https://wiki.secondlife.com/w/index.php?title=LLMedia_API&oldid=494363"