|
|
| Line 1: |
Line 1: |
| ='''Media On A Parcel'''= | | ='''Media On A Parcel'''= |
| This project comprises tasks related to supporting various different types of media in Second Life. Currently we support a single quicktime compliant URL per parcel, which is associated with a texture ID and thus displayed on any prim using that texture. We would like to expand this to include multiple URLs per parcel, and abstract the underlying media layer to allow for the easy addition of new media support. Eventually we will also support a single URL per material component, and thereby allow users to display media on any face of a prim.
| | The purpose of this project is to shore up and expand support for Media within Second Life. Currently we support a single Quicktime URL per parcel, which is used in place of a given texture. We plan to rewrite the underlying Media Library and expand it to support a wide variety of Internet Media types, such as HTML and Flash. Also we will support Media display in other areas of Second Life, such as the HUD and on individual Prims. |
| | |
| | The Project will deploy in several milestones. The first of these will deliver the underlying re-architecture of the Media Library and support for HTML on a parcel. Subsequent milestones will provide for Media in the HUD, multiple URLs per parcel, and implement new Media Type support. |
|
| |
|
| * [[MediaForum |Transcript of SLDev Forum, 2 Oct 07]] | | * [[MediaForum |Transcript of SLDev Forum, 2 Oct 07]] |
|
| |
| == Goals ==
| |
| The goal of this project is to expand the current functionality for playing movies on a parcel to include a variety of media types. The media system itself will be rewritten to be more scalable and allow for user created media handling and flexible URL types. It will also be extricated from the viewer and exist as a standalone application with a minimum of external dependencies. This will allow for external testing of the component.
| |
|
| |
|
| == Design == | | == Design == |
| | * The Media Library will use common MIME types to map URLs to specific decoders in the Library. |
| | * These will all be part of the SL Viewer Open Source project. |
|
| |
|
| ===User interaction model=== | | ===Media Setup=== |
| ====Creating a media URL====
| | * The parcel owner will initially set the URL in the About Land dialog, and will add information such as Type and Media Size if applicable. |
| [[Image:Land_Parcel_Media_Options.PNG|thumb]]
| | * They also specify the ID of the Texture to replace with this URL. |
| * Parcel owner right clicks on land and selects "About Land" and chooses Media tab | | * This information is stored on the server side with the rest of the Parcel information. |
| | | * If the parcel owner is unsure of the media type a lookup can be done of the URL header. |
| * Enters media details:
| | [[AboutLandMedia.png|Thumb]] |
| ** media URL
| | ===Playing Media=== |
| ** media Type (dropdown menu from LLMediaRegistry file)
| | * When a resident enters a parcel which has a media URL defined, an appropriate UI control pops up allowing the resident to load the media. |
| ** texture to replace
| | * When the resident invokes the media control the Media Library begins decoding the media based on the MIME type for the media. |
| ** auto scaling checkbox
| | * Frames will appear on any face displaying the texture ID specified in the Media Setup. |
| *** media size (if required, otherwise grey'ed out)
| | * Milestone 2 will also add support for displaying media in the HUD, so rather than being drawn in world, the media would appear on a UI widget displayed in the HUD. |
| | |
| * Hits (new) 'Apply' button | |
| | |
| * Client uses HTTP 'GetHeaders' to retrieve the MIME type of the media URL
| |
| | |
| * 'Pleasing' client UI shows progress bar whilst HTTP request happens
| |
| | |
| * If this fails (or it's not an HTTP URL) then some more primitive heuristics are used (string matching etc.) and a MIME type generated
| |
| | |
| * Media details (including MIME type) are sent to the simulator and stored | |
| [[Image:URLSetFlow.jpg|420px]]
| |
| ====Consuming a media URL====
| |
| | |
| * Resident enters a parcel which has a media URL defined
| |
| | |
| * Simulator sends media information as a fragment of LLSD.
| |
| | |
| * Transport control appears based on the media type (since the simulator knows the type now via MIME)
| |
| | |
| * User selects transport control that 'starts' media (E.G. play the movie for QuickTime) | |
| | |
| * Media type (via MIME) used to select the corresponding media impl
| |
| ** Note: consider re-getting MIME type via HTTP headers in case the contents of the source URL changed since it was added
| |
| | |
| * Media impl created and local init function called (typically lightweight init that wasn't done at startup)
| |
| | |
| * Media Manager calls method in the impl via well defined abstract interface - e.g. 'start()'
| |
| | |
| * Media impl does the right thing
| |
| [[Image:URLDecodingFlow.jpg|420px]] | |
| | |
| | |
| ===Changes to UI=== | |
| * Dialog to set media URL now needs to support: | |
| ** Selecting which media URL/texture you are changing
| |
| ** Allow name and description fields for media (to help with selecting which one you want to see later)
| |
| ** Allow for setting media type from a dropdown populated by LLMediaRegistry
| |
| ** Media width/height (if required, e.g. for Web browser)
| |
| ** Button to get MIME type and "GETTING..." progress bar
| |
| | |
| * Playback UI now needs to support
| |
| ** More than one media source per parcel (selection via name perhaps)
| |
| ** Different transport controls based on media type (e.g. Play/pause for QuickTime, back,forward,refresh for Web)
| |
| *** May punt on this for now since Richard is making large changes in that area too - will talk to him about it. | |
| | |
| | |
| | |
| ===Top level class hierarchy===
| |
| | |
| * Singleton: LLMediaManager. All access to media is via this class.
| |
| ** Note: name change from LLMediaEngine was done on purpose to strongly differentiate it from current version
| |
| | |
| * Singleton: LLMediaRegistry. determines, stores and updates media registry XML file
| |
| | |
| * Singleton: LLMediaFactory. Creates instances of LLMediaImpls based on the URL MIME type.
| |
| * Pure virtual base class: LLMediaBase.h:
| |
| ** provides well defined interface through which media impls are accessed
| |
| | |
| * LLMediaImplCommon.cpp/.h:
| |
| ** Support for common functionality between impls - E.G. setting flags, allocating memory etc.
| |
| ** Each media impl derives from this class
| |
| | |
| * Media impls: LLMediaImpl*.cpp/.h:
| |
| ** Impls for each media type | |
| ** May be more than one per media type - e.g. Web may have LLMediaImplLLMozLib and LLMediaImplLLOpera
| |
| | |
| * Inheritance chain:
| |
| ** LLmediaBase is an abstract class
| |
| ** LLMediaImplCommon derrives from LLmediaBase
| |
| ** LLMediaImplCommon provides default (stubbed) impls for LLmediaBase interface
| |
| ** LLMediaImpl* derrives from LLMediaImplCommon and overrides the methods it needs to change
| |
| | |
| * Observer/emitter implementation: LLMediaObserver.h and LLMediaEmitter.h
| |
| | |
| === Media Registration ===
| |
| * Media types are associated with Impls through the Media Registry XML file
| |
| * Each entry in the file contains the following:
| |
| ** Unique Name - e.g. Quicktime
| |
| ** MIME-type reference - e.g. video/quicktime | |
| ** Impl reference - e.g. QuicktimeImpl
| |
| | |
| The Name field need not be unique, however there is no current use-case that would support duplicate names.
| |
| | |
| | |
| | |
| ==Implementation==
| |
| | |
| | |
| ===System startup===
| |
| * LLMediaRegistry is loaded and
| |
| * At startup, global init function (static) called for each media impl type. (breaks abstraction but any initialization that takes a long time is done here)
| |
| | |
| | |
| ===Creating a media URL===
| |
| * Client side:
| |
| ** Retrieve media URL
| |
| ** Gets HTTP headers using CURL and retrieves MIME type from Content-type: field
| |
| *** If no HTTP headers (or not HTTP URL) use heuristics to find media type.
| |
| *** Default to Web content
| |
| ** Retrieve texture ID, name, description, width, height and send to simulator as a block of LLSD using new message system.
| |
| | |
| * Server side:
| |
| ** Simulator stores media information
| |
| | |
| | |
| ===Playing back a media URL===
| |
| * Client side:
| |
| ** Resident enters a parcel
| |
| ** Asks simulator for media details
| |
| ** Media details arrive via new message system
| |
| ** Display UI (transport controls) based on media type
| |
| ** When user presses 'start':
| |
| ** Look up media type and create appropriate impl
| |
| ** Client sets itself up to observe Media Manager
| |
| ** Store impl and return media ID
| |
| ** Client uses this media ID to reference this media source
| |
| ** Load media using this ID (E.G. LLMediaManager::getInstance()->play( mMediaId ); )
| |
| ** Start playing back media
| |
| ** When new frame of data ready, media manager emits event that client sees
| |
| ** Client decides whether to display it or not.
| |
| ** Client sees other events emitted from Media Manager (for example, media size changes, media stopped, media started)
| |
| | |
| * Server side:
| |
| ** Client asks for media details
| |
| ** Send media details to client using new message system
| |
|
| |
|
| | | ===Supported Media Types=== |
| | | * Implemented in Milestone 1 |
| === Message Details ===
| |
| The simulator changes for this project are reasonably small. The current message will be deprecated in favor of an LLSD style TCP message that will allow for a more robust media format and multiple URL's per packet. The structure will look like:
| |
| | |
| {| border="1"
| |
| |+ Media LLSD Message
| |
| ! Value !! Type !! Comment
| |
| |-
| |
| | URL ||String || Link to the Media
| |
| |-
| |
| |-
| |
| | MediaType||String || MIME Type
| |
| |-
| |
| |-
| |
| | Flags||U32|| Flags associated with the media
| |
| |-
| |
| |-
| |
| |Name||String || A user specified name for the media
| |
| |-
| |
| |-
| |
| |Description||String || A user specified description for the media
| |
| |-
| |
| |}
| |
| | |
| | |
| | |
| ===Media Types=== | |
| * Implemented in this project | |
| ** QuickTime (refactor existing one and make sure we're using the latest version of the QuickTime SDK) | | ** QuickTime (refactor existing one and make sure we're using the latest version of the QuickTime SDK) |
| ** LLMozLib (Will use existing version that's in the client - updating that is another project) | | ** Mozilla (Will use existing version that's in the client - updating that is another project) |
|
| |
|
| * Implemented shortly afterwards: | | * Implemented in Milestone 2 |
| ** LLOperaLib (Alternative impl for handling Web based content - based on Opera browser 9.5) | | ** Opera (Alternative impl for handling Web based content - based on Opera browser 9.5) |
| | ** Flash |
|
| |
|
| * Implemented sometime in the future by us or ideally, by residents: | | * Implemented sometime in the future by us or ideally, by residents: |
| Line 187: |
Line 37: |
| ** Flickr -e.g. flickr://keyword/dog/rand or flickr://interestiness/date/20070410/rand | | ** Flickr -e.g. flickr://keyword/dog/rand or flickr://interestiness/date/20070410/rand |
| ** VNC | | ** VNC |
|
| |
|
| |
|
| |
| ===Media Registry File===
| |
| * The Media Registry file provides data-driven association between media types and decoders (Impls)
| |
| * The Type Name can be used for type-specific UI
| |
| * Type Name is specified in About Land UI
| |
|
| |
| {| border="1"
| |
| |+ Media Registry File
| |
| ! Status!! MIME Type!! Impl Type !! Type
| |
| |-
| |
| |enabled||image/jpeg||LLQUICKTIME||image
| |
| |-
| |
| |enabled||image/bmp||LLQUICKTIME||image
| |
| |-
| |
| |enabled||image/png||LLQUICKTIME||image
| |
| |-
| |
| |enabled||stream/mov||LLQUICKTIME||movie
| |
| |-
| |
| |enabled||stream/mpeg||LLQUICKTIME||movie
| |
| |-
| |
| |enabled||text/html||LLOPERA||web
| |
| |-
| |
| |disabled||text/html||LLMOZLIB||web
| |
| |-
| |
| |}
| |
|
| |
| {| border="1"
| |
| |+ XUI Combobox definition
| |
| ! User Friendly Name !! Type !! UI controls visible
| |
| |-
| |
| |Image||image||none
| |
| |-
| |
| |Movie||movie||Source image size, auto-scale
| |
| |-
| |
| |Web Page||web||Source page size
| |
| |-
| |
| |}
| |
|
| |
| ----
| |
|
| |
| {| border="1"
| |
| |+ Alternate Approach
| |
| ! UI String !! MIME Type !! Enabled !! Impl Type !! UI Controls Visible
| |
| |-
| |
| |Image (JPEG) ||image/jpeg||enabled||LLQUICKTIME||image
| |
| |-
| |
| |Image (BMP) ||image/bmp||enabled||LLQUICKTIME||image
| |
| |-
| |
| |Image (PNG) ||image/png||enabled||LLQUICKTIME||image
| |
| |-
| |
| |Movie (QuickTime) ||stream/mov||enabled||LLQUICKTIME||movie
| |
| |-
| |
| |Movie (MPEG) ||stream/mpeg||enabled||LLQUICKTIME||movie
| |
| |-
| |
| |Web Page||text/html||enabled||LLOPERA||web
| |
| |-
| |
| |Web Page||text/html||disabled||LLMOZLIB||web
| |
| |-
| |
| |}
| |
|
| |
| {| border="1"
| |
| |+ XUI Combobox definition
| |
| ! Type !! UI controls visible
| |
| |-
| |
| |image||none
| |
| |-
| |
| |movie||Source image size, auto-scale
| |
| |-
| |
| |web||Source page size
| |
| |-
| |
| |}
| |
|
| |
| * media_registry.xml lives in the /app_settings directory
| |
| * XUI for pop-up contains user-friend name -> type mapping
| |
|
| |
| ===Changes to LSL===
| |
| * All LSL media commands now needs to specify some kind of media index since we can have more than one per parcel
| |
| * Extra commands needed to:
| |
| ** specify MIME type directly (maybe)
| |
| ** set MIME type based on HTTP headers (remember this process may take some time)
| |
| ** specify media width/height
| |
|
| |
|
| |
|
| |
| ==Milestone targets==
| |
| ===Milestone 1===
| |
| * Tentative Alpha Date: October 31st
| |
| * Tentative Release Date: December 1st
| |
| ====Features====
| |
| * Allow parcel owner to specify Web based media URL that gets rendered in the same way as QuickTime does currently
| |
| * No interactivity
| |
| * Dynamic web content is supported
| |
| * Includes foundation for later work - for example:
| |
| ** flexible media type discovery via MIME types
| |
| ** multiple URLs per parcel
| |
| ** unification of Web based media and LLWebBrowser control widget
| |
|
| |
| ====Tasks====
| |
| * Media Manager -- 10 days
| |
| ** LLMediaManager ('''callum''')
| |
| ** LLMediaRegistry ('''callum''')
| |
| *** Make and test version of LLMediaRegistry xml ('''callum''')
| |
| *** Create and test CURL getHeader functionality ('''callum''')
| |
| ** LLMediaFactory ('''callum''')
| |
| ** LLMediaBase ('''callum''')
| |
| ** LLMediaImplCommon ('''callum''')
| |
| * Media Impls -- 15 days
| |
| ** LLMediaImplQuicktime ('''callum''')
| |
| ** LLMediaImplMozilla ('''callum''')
| |
| * Build unit test app ('''callum''') -- 5 days
| |
| * Viewer Changes -- 7 days
| |
| ** Alter About Land UI xui and .cpp file ('''james''')
| |
| ** Tie in to new transport controls UI ('''sam''')
| |
| ** Alter media rendering code to use data written to LLTextureRaw ('''callum/steve''')
| |
| * Simulator/Database -- 10 days
| |
| ** Deprecate UDP msg and create TCP msg from About Land panel to Sim using capability ('''sam''')
| |
| ** Alter Simulator code to use new msg ('''sam''')
| |
| ** Create code to send URL to viewer via secure TCP ('''sam''')
| |
| ** Add database support for new parcel media fields ('''sam''')
| |
| *** Generate upgrade scripts ('''sam''')
| |
| * Write Milestone 1 test plan ('''sam''') -- 1 day
| |
|
| |
| ===Milestone 2===
| |
| ====Features====
| |
| * Adds multiple URLs per parcel
| |
|
| |
| ====Tasks====
| |
| * Alter About Land UI to support multiple URLs
| |
| * Alter transport controls to support multiple URLs
| |
| * Alter msg to support multiple URLs
| |
| * Write simulator code to sanity check URL limit
| |
| * Add database handling for multiple URLs
| |
| ** Generate upgrade script
| |
| * Write Milestone 2 test plan
| |
|
| |
|
| |
| ===Milestone 3a===
| |
| * Adds interactivity
| |
| ** Individual interactivity (all interaction is local)
| |
|
| |
|
| |
| ===Milestone 3b===
| |
| * Poor mans shared interactivity via clicks or URLS (mouse clicks or URLs passed around parcel)
| |
| * Will need a permissions system so parcel owner can specify who is able to interact with media and how.
| |
|
| |
|
| |
| ===Milestone 3c===
| |
| ** True shared interactivity via browser running on server (sim)
| |
|
| |
|
| |
| ===Milestone 4===
| |
| * Media per prim.
| |
| * Almost certainly a completely separate project but listed here for completeness.
| |
|
| |
|
| |
|
| |
| == Limitations ==
| |
| * One URL per parcel initially supported.
| |
| * Limited support of media types in early releases
| |
| * No Interaction initially
| |
| * Limited media size?
| |
|
| |
|
| |
|
| |
| == Performance ==
| |
| * Open questions remain about performance of the HTTP::get from the simulator. This might need to be done asynchronously, or on the simulator to prevent hitches
| |
| * The Media Manager will cache URLs to prevent re-getting same data again
| |
| * Might have to limit number of URLs once multi-url support is added
| |
|
| |
|
| |
|
| |
| ==Security concerns==
| |
| * Don't autoplay media content in case it p0wns the parcel visitor
| |
| * How does the new secondlife:// play ball with media (I.E. don't open a classified floater when I walk onto your parcel)
| |
| * We don't handle media dialogs for passwords etc. so any site that uses .htaccess to protect its contents or one using SSL won't work. Should we do the work to support this (HARD)?
| |
|
| |
|
| |
|
| |
|
| ==Use cases== | | ==Use cases== |
| Line 419: |
Line 90: |
|
| |
|
| Also: some ideas here: http://wiki.secondlife.com/wiki/User:Zero_Linden/Office_Hours/HTML_on_a_Prim_Use_Cases | | Also: some ideas here: http://wiki.secondlife.com/wiki/User:Zero_Linden/Office_Hours/HTML_on_a_Prim_Use_Cases |
|
| |
| == Test Plan==
| |
| To Be Determined
| |
|
| |
|
| |
|
| |
| ==Miscellaneous unanswered questions==
| |
| * media engine should emit event when media is ready - client observes this events and acts on it when it comes in
| |
| * limit maximum media size (2k x2k)?
| |
| * timed updated per media impl ? N per second
| |
| * what do we need to consider to make this work later on a HUD
| |
| * Should we be able to render HTML content from notecards
| |
| * Should we be able to render HTML content from LSL
| |
|
| |
| == Sample media URLs ==
| |
| '''```QuickTime``` Movies non-streaming (.mov)'''
| |
| *http://movies.apple.com/movies/miramax/sin_city/sin_city_480.mov
| |
| *http://movies.apple.com/movies/disney/the_incredibles/the_incredibles-tlr_a480.mov
| |
| *http://movies.apple.com/movies/disney/cars/cars-teaser_320.mov
| |
| *http://www2.bc.edu/~lewbel/shortsha.mov
| |
| *http://movies.apple.com/movies/fox/fantastic_four/fantastic_four_480.mov
| |
| *http://www.madoverlord.com/Movies/TSN/I-TornadoMer-Megabyte.mov
| |
| *http://webdev.lindenlab.com/media/guinesssurfer.mov (amazing ad a friend worked on)
| |
| *http://webdev.lindenlab.com/media/johnniewalkerfish.mov (more of the same)
| |
|
| |
| '''```QuickTime``` Movies streaming (.mov)'''
| |
| *http://stream.qtv.apple.com/events/jan/macworld/2005/macworld_hi_300_100_56_ref.mov
| |
|
| |
| '''MPEG4 (.mp4)'''
| |
| *http://www.archive.org/download/doa_1949/doa_1949_256kb.mp4
| |
|
| |
| '''Flash 5 (.swf)'''
| |
| *only Flash5 or earlier will play - movies created with latest version or FlashMX probably won't work
| |
| *although we have no interactivity with flash movies, if there is a single "click to play" at the start, pressing play multiple times will often start it playing
| |
| *http://www.campchaos.com/content/232_napsterbad_56k.swf
| |
|
| |
| '''Animated GIF (.gif)'''
| |
| *```http://www.geocities.com/callumprentice/lindenlab/weather.gif```
| |
|
| |
| '''```QuickTime``` VR panoramas (.mov)'''
| |
| *http://www.panoramas.dk/movies3/ts2004_3600r.mov
| |
|
| |
| '''```QuickTime``` VR objects (.mov)'''
| |
| *http://images.apple.com/hardware/gallery/media/pmac_g5_jun2004_480.mov
| |
|
| |
| '''Real Time Streaming (Video Camera)'''
| |
| *rtsp://216.23.173.121/mystream.sdp (live stream from hosting company we're using for GDC party)
| |
| *rtsp://aragorn.lindenlab.com/mystream.sdp (from callum's imac - not always on and only available internally)
| |
|
| |
| '''Streaming via HTTP'''
| |
| * http://stream.qtv.apple.com/events/jan/macworld/2005/macworld_hi_300_100_56_ref.mov
| |
|
| |
| '''```QuickTime``` Text files'''
| |
| *rudimentary markup language - supports fonts, colours, size, position etc.
| |
| *syntax here: http://www.apple.com/quicktime/tools_tips/tutorials/textdescriptors.html
| |
| *http://www.geocities.com/callumprentice/lindenlab/example.txt (temporary location)
| |
|
| |
| '''URL that causes a crash with QT6 installed'''
| |
| http://www.americafree.tv/unicast_mov/H264test.mov
| |
|
| |
| [[Category:Design Documents]]
| |
| [[Category:System Architecture]]
| |
