LLMedia API

From Second Life Wiki
Jump to navigation Jump to search

LLMedia Documentation

Abstract

The LLMedia API provides a consistent approach to embedding various Rich Media Rendering Engines (Implementations) in a manner consistent with the needs of Second Life. The API provides a framework for linking a URL type (loosely composed of MIME Type and/or Scheme) to the interface layer of the associated Media Engine, and for passing back the rendered OpenGL texture. It also supports a variety of input controls to enable common media navigation paradigms.


Overview

  • Located in the Second Life client source tree at $BRANCH_NAME/indra/llmedia
  • Allows consumers of the library to render different types of media to a well defined memory buffer
  • Specifically, in the Second Life client, this memory buffer is used as the source for OpenGL textures
  • It is deliberately loosely coupled to the rest of the viewer - no dependencies whatsoever on viewer code
  • Each media type is defined using it's MIME type (if present) or URL scheme (if present)
  • Different media types are handled by implementation files specific to each type - hereafter called Impls
  • A media manager class handles registration, creation, management and destruction of Impls
  • Impls for new media types are easily integrated - write a C++ impl class and tell the media manager class which MIME type or URL scheme you support


Currently supported media types (Impls)

  • LLMediaImplQuickTime
    • Apple QuickTime media
    • Broadly speaking, supports everything the Apple QuickTime player does
    • Available on Win32 and Mac
  • LLMediaImplGStreamer
    • Supports a wide variety of QuickTime/AVI/other media depending on installed system plug-ins
    • Available on Linux
  • LLMediaImplLLMozLib
    • LLMozLib media
    • Web based media
    • Very broadly speaking, supports everything that Firefox 2.0.x supports
    • Notable exceptions are popups, plugins, .htaccess based authorization, feedback dialogs

API class layout

  • Abstract base class (llMediaBase)
  • Common Impl class (llMediaImplCommon) derrives from this abstract base class and contains a set of matching (non-pure) methods along with some utility functions.
  • Each Impl derrives from llMediaImplCommon and overrides as much of it as it needs to do its work.
  • Each Impl must implement an llMediaImplMaker and override the create() function. This object is also used to register for mime types and schemes.
  • A factory class (llMediaImplFactory) is responsible for registering an Impl and then making one as and when required.
  • A manager class (llMediaManager) manages the list of Impls (including creation/deletion/interrogation) and also contains common utility functions.
  • An event system based on the well known emitter/observer pattern is implemented via the (llMediaEmitter and llMediaObserver)


API functions

Housekeeping

Function:

 bool init();

Description:
  • Local initialization, called by the media manager when creating a media source
  • Different from the global (Impl)::startUp() & (Impl)::closeDown() static methods which are designed to be called once at application startup and closedown.
Return value:
  • true on success, false on failure

 

Function:

 bool reset();

Description:
  • Undoes everything init() did called when the media manager destroys a media source
Return value:
  • true on success, false on failure

 

Function:

 bool setMimeType( const std::string mime_type ); std::string getMimeType() const;

Description:
  • Accessor/mutator for MIME type
Return value:
  • true on success, false on failure

 

Function:

 std::string getMediaURL() const;

Description:
  • Accessor for current URL
Return value:
  • The current media URL
  • This may be different from the original URL that was passed to navigateTo - a HTTP redirect may have occurred for example

 

Function:

 std::string getVersion();

Description:
  • Get a version string specific to the Impl
  • Format of version string is specific to each Impl
Return value:
  • The version string

 

Function:

 bool set404RedirectUrl( std::string redirect_url ); bool clr404RedirectUrl();

Description:
  • Set/clear URL to visit when a 404 page is reached
  • This (unusual) functionality was required for Web based Search because Amazon S3 404 pages are XML based and not HTML
Return value:
  • true on success, false on failure

 

Function:

 bool setBackgroundColor( unsigned int red, unsigned int green, unsigned int blue ) const;

Description:
  • Sets the background color of the media surface
  • Pass in 0 - 255 for each color component
Return value:
  • true on success, false on failure

 

Function:

 bool setCaretColor( unsigned int red, unsigned int green, unsigned int blue ) const;

Description:
  • Sets the color of the caret in media impls that have one - Web related ones do for example.
  • Pass in 0 - 255 for each color component
Return value:
  • true on success, false on failure

 

Media Management

Function:

 bool updateMedia();

Description:
  • Tells Impl to update itself
  • Must be needs to be called on a regular basis
  • No guarantee in this version that this will return quickly - this is up to the Impl.
Return value:
  • true on success, false on failure

 

Function:

 bool setRequestedMediaSize( int media_width, int media_height );

Description:
  • Request that the Impl change the media height and width to given dimensions in pixels
  • May fail if the Impl doesn't support changing size
Return value:
  • true on success, false on failure

 

Function:

 int getMediaWidth() const;

Description:
  • Gets current media width
  • May change throughout lifetime of media stream
  • Note: event is emitted that consumers of this library can catch when media size changes
Return value:
  • The current media width in pixels

 

Function:

 int getMediaHeight() const;

Description:
  • Gets current media height
  • May change throughout lifetime of media stream
  • Note: event is emitted that consumers of this library can catch when media size changes
Return value:
  • The current media height in pixels

 

Function:

 bool setMediaDepth( int media_depth );

Description:
  • Request that the Impl change the media depth to given dimensions in bytes
  • May fail if the Impl doesn't support changing size
  • TODO: ought this to be called setRequestedMediaDepth() ?
Return value:
  • true on success, false on failure

 

Function:

 int getMediaDepth() const;

Description:
  • Gets current media depth
  • May change throughout lifetime of media stream
  • Note: event is emitted that consumers of this library can catch when media size changes
Return value:
  • The current media depth in bytes

 

Function:

 int getMediaBufferSize() const;

Description:
  • Gets size of media buffer for current media surface
  • Important: this might NOT be the same as media width * height * depth if the Impl is padding stride length for example
Return value:
  • The size in bytes of the current media buffer

 

Function:

 unsigned char* getMediaData();

Description:
  • Returns pointer to raw media pixels
Return value:
  • A pointer to the current media data

 

Function:

 int getMediaDataWidth() const;

Description:
  • Get the width of the media data
  • May be different from the size of the media if the Impl is padding stride length for example
Return value:
  • The width of the media data in pixels

 

Function:

 int getMediaDataHeight() const;

Description:
  • Get the height of the media data
  • May be different from the size of the media if the Impl is padding height
  • This is almost always the same as media height
Return value:
  • The height of the media data in pixels

 

Texture Management

Function:

 int getTextureFormatInternal() const;

Description:
  • Gets internal format to use for OpenGL texture
Return value:
  • A value for internal texture format that mirrors the OpenGL version
  • Valid values are:
    • LL_MEDIA_RGB
    • LL_MEDIA_RGBA
    • LL_MEDIA_RGB8
    • LL_MEDIA_BGR
    • LL_MEDIA_BGRA

 

Function:

 int getTextureFormatPrimary() const;

Description:
  • Gets primary format to use for OpenGL texture
Return value:
  • A value for primary texture format that mirrors the OpenGL version
  • Valid values are:
    • LL_MEDIA_RGB
    • LL_MEDIA_RGBA
    • LL_MEDIA_RGB8
    • LL_MEDIA_BGR
    • LL_MEDIA_BGRA

 

Function:

 int getTextureFormatType() const;

Description:
  • Gets format type to use for OpenGL texture
Return value:
  • A value for texture format type that mirrors the OpenGL version
  • Valid values are:
    • LL_MEDIA_UNSIGNED_BYTE
    • LL_MEDIA_UNSIGNED_INT_8_8_8_8
    • LL_MEDIA_UNSIGNED_INT_8_8_8_8_REV

 

Audio

Function:

 bool setVolume( float volume ); float getVolume() const;

Description:
  • Accessor/mutator for volume from media stream if present
  • Volume level is 0.0 (off) to 1.0 (maximum)
Return value:
  • true on success, false on failure

 

Transport Control

Function:

 bool addCommand( ECommand cmd );

Description:
  • Add a transport command to the list of ones to be processed
  • Note: Currently the list is only 1 entry long so adding a command will remove the existing one
  • Supported commands are currently:
    • ECommand::COMMAND_STOP - Stop media stream if appropriate
    • ECommand::COMMAND_START - Start media stream if appropriate
    • ECommand::COMMAND_PAUSE - Pause media stream if appropriate
    • ECommand::COMMAND_BACK - Go back to last location in media stream if appropriate
    • ECommand::COMMAND_FORWARD - Go forward to next location in media stream if appropriate
Return value:
  • true on success, false on failure

 

Function:

 bool clearCommand();

Description:
  • Remove a transport command from the list of ones to be processed
  • Note: Currently the list is only 1 entry long so clearing a command will empty the list
Return value:
  • true on success, false on failure

 

Function:

 bool updateCommand();

Description:
  • Tells transport command queue to update itself
  • Must be needs to be called on a regular basis
Return value:
  • true on success, false on failure

 

Function:

 EStatus getStatus();

Description:
  • Returns a state code that describes what the current transport status is
  • Supported values are currently:
    • EStatus::STATUS_UNKNOWN - Status code is unknown
    • EStatus::STATUS_INITIALIZING - media source is initializing
    • EStatus::STATUS_NAVIGATING - media source is navigating to a new URL
    • EStatus::STATUS_STARTED - media source has been started
    • EStatus::STATUS_STOPPED - media source has been stopped
    • EStatus::STATUS_PAUSED - media source has been paused
    • EStatus::STATUS_RESETTING - media source is resetting (uninitializing)
Return value:
  • true on success, false on failure

 

Function:

 bool seek( double time );

Description:
  • Seeks to a location within a media source stream
  • Time is in seconds since the start of the stream
  • Only meaningful for video type streams like QuickTime for example
Return value:
  • true on success, false on failure

 

Function:

 bool setLooping( bool enable);

Description:
  • Turn looping on or off.
  • If looping is on, media will restart from the beginning when it reaches the end
  • If looping is off, media still pause on the last frame when it reaches the end
Return value:
  • true on success, false on failure

 

Function:

 bool isLooping();

Description:
  • Determine if looping is switched on or off
Return value:
  • true if looping is enabled, false if looping is disabled

 

Scaling

Function:

 bool setAutoScaled( bool auto_scaled );

Description:
  • Turn media autoscaling on or off
  • Autoscale means try to scale media to size of texture
  • May fail if media doesn't support size change
Return value:
  • true on success, false on failure

 

Function:

 bool isAutoScaled() const;

Description:
  • Determine if autoscaling is turned on or off
Return value:
  • true if autoscaling is turned on, false if autoscaling is turned off

 

Mouse and Keyboard

Function:

 bool mouseDown( int x_pos, int y_pos );

Description:
  • Inject left mouse button press at given X/Y coordinate within media surface
Return value:
  • true on success, false on failure

 

Function:

 bool mouseUp( int x_pos, int y_pos );

Description:
  • Inject left mouse button release at given X/Y coordinate within media surface
Return value:
  • true on success, false on failure

 

Function:

 bool mouseLeftDoubleClick( int x_pos, int y_pos );

Description:
  • Inject left mouse button double click at given X/Y coordinate within media surface
Return value:
  • true on success, false on failure

 

Function:

 bool mouseMove( int x_pos, int y_pos );

Description:
  • Inject mouse cursor move to given X/Y coordinate within media surface
Return value:
  • true on success, false on failure

 

Function:

 bool keyPress( int key_code );

Description:
  • Inject key press with given ASCII value into media surface
  • No support currently for discrete key down/up events
Return value:
  • true on success, false on failure

 

Function:

 bool unicodeInput( unsigned long uni_char );

Description:
  • Inject key press with given UNICODE value into media surface
  • No support currently for discrete key down/up events
Return value:
  • true on success, false on failure

 

Function:

 bool scrollByLines( int lines );

Description:
  • Scroll display by number of lines if appropriate (for example, in a Web related media)
Return value:
  • true on success, false on failure

 

Function:

 bool focus( bool focus );

Description:
  • Enable/disable focus
  • If media Impl doesn't have focus, keyboard input will not succeed
Return value:
  • true on success, false on failure

 

Navigation

Function:

 bool navigateTo( const std::string url );

Description:
  • Request navigation to given URL
  • Typically the actual navigation happens asynchronously
Return value:
  • true on success, false on failure
  • Note: success here means the navigate command succeeded. The actual navigation may fail later

 

Function:

 bool navigateForward();

Description:
  • Navigate forward to next location in media source
  • In a Web related media source this means go to next page in history stack
  • In a video related media source, this means go to next chapter (currently unimplemented)
Return value:
  • true on success, false on failure

 

Function:

 bool navigateBack();

Description:
  • Navigate backward to previous location in media source
  • In a Web related media source this means go to previous page in history stack
  • In a video related media source, this means go to previous chapter (currently unimplemented)
Return value:
  • true on success, false on failure
  • Note: success here means the navigate command succeeded. The actual navigation may fail later

 

Function:

 bool canNavigateForward();

Description:
  • Determine if it's possible to go forward to next location in media source
Return value:
  • true if it's possible to navigate forwards in the history stack, false if it's not possible to navigate forwards in the history stack

 

Function:

 bool canNavigateBack();

Description:
  • Determine if it's possible to go back to previous location in media source
Return value:
  • true if it's possible to navigate backwards in the history stack, false if it's not possible to navigate backwards in the history stack

 

Caching and Cookies

Function:

 bool enableCookies( bool enable );

Description:
  • Enable/disable the saving of cookies
  • Currently only supported on a global level - all or nothing
  • Probably only meaningful for Web related media sources
Return value:
  • true on success, false on failure

 

Function:

 bool clearCookies();

Description:
  • clears any current set cookies
  • Only meaningful for Web related media sources
Return value:
  • true on success, false on failure

 

Function:

 bool clearCache();

Description:
  • Clears the cache
  • Meaningful for Web related media
  • May be meaningful for QuickTime media too
Return value:
  • true on success, false on failure

 

Proxy

Function:

 bool enableProxy(bool enable, std::string proxy_host_name, int proxy_port);

Description:
  • Enable/disable support for network proxying.
  • Each Impl is responsible for implementing the proxy support itself
  • Currently only supported for LLMediaImplLLMozLib
Return value:
  • true on success, false on failure

 

Emitter/Observer

Function:

 bool addObserver( LLMediaObserver* subject );

Description:
  • Add yourself as an observer of the media library
  • See sample code for examples of how to do this
Return value:
  • true on success, false on failure

 


Observable events

  • Add class as an observer to the media library using addObserver and remObserver
  • To observe specific events, override one or more of these methods in your own code
    • onMediaPreroll - emitted when the media is prerolling
    • onMediaLoaded - emitted when the media is loaded
    • onMediaSizeChange - emitted when the media changes size
    • onMediaContentsChange - emitted when the contents of the media stream changed
    • onMediaStatusTextChange - emitted when the status stext associated with the medias stream changed
    • onNavigateBegin - emitted when navigation to a new URL starts
    • onNavigateComplete - emitted when navigation to a new URL ends
    • onUpdateProgress - emitted when navigation to a new URL makes progress
    • onLocationChange - emitted when a URL changes - for example, an HTTP redirect
    • onClickLinkHref - emitted when a link is clicked in a Web related media source
    • onClickLinkNoFollow - emitted when a link that has been marked as not to follow is clicked in a Web related media source - a secondlife:// URL for example
  • Event payload is also passed in with data specific to the nature of the event


Examples

  • Example code exists in 2 places:
    • In the LLMedia library itself, there are 2 example Impls called (unimaginatively) Example1 and Example2.
      • Example1 drags a random pattern on the screen and changes the contents when you move the mouse over it
      • Example2 bounces a square around the media area
    • In the llmediatest application in the indra/test_apps directory.
      • Simple media browser
      • Dependency on GLUI for its UI


Current Impls

  • LLMediaImplExample1 - Example Impl that displays a random pattern of squares and writes data into the media surface at the mouse position
  • LLMediaImplExample2 - Example Impl that bounces a random colored box around within the media surface
  • LLMediaImplLLMozLib - provides Web based content via LLMozLib library (which, itself wraps Mozilla/Gecko 1.8.1)
  • LLMediaImplQuickTime - provides QuickTime media content (Movies, images, RTSP streams)
Retrieved from "https://wiki.secondlife.com/w/index.php?title=LLMedia_API&oldid=69463"