Media Plugin System Project Notes
Changes to existing system
Client side code
- A large section of the client code that deals with moving bytes from memory into a texture managed by the client that OpenGL understands will have to be rewritten. There are several reasons for this - firstly, that code has grown "organically" and needs a significant clean-up. Secondly, the process by which dynamic textures are created and managed hasn't changed since QuickTime was added to the client 4 years ago. Lastly and most importantly, the programming paradigm changes completely with this new design from a synchronous model where you call functions to a message based model. When you send a message to a plugin, consider it as being shot into the dark. You may (or may not) get a response back at some future time. Whether a particular message will cause a response to be sent will be defined by the particular type of plugin.
- plugin will specify which transport controls should be used via a fragment of XML.
- The number and type of transport controls can vary over time and should support an enabled and disabled state.
- QUESTION: How will we do this? Built in set of icons that XML file references plus ability to specify your own or something more straightforward?
Other notes
One-shote rendering
In one-shot mode, a message is sent to the plugin requesting that it render a particular media URL into a shared memory buffer. It renders the media once, then sends a message indicating the render has completed. This may be used when rendering large numbers of media streams at lower priorities (such as when there are a large number of streams in the user's view, but most of them are far away).
Plugins in the Viewer process
An addition to the design that allows the plugin implementation shared libraries to be loaded directly by the plugin manager into the same process space is being considered. There are some circumstances when this might be preferred and whilst this functionality may not be needed immediately, it will be useful in the future. For example, in a world where many simultaneous media sources are required, having a large number of separate processes may consume a lot of system resource. If a plugin has been shown to be reliable, loading in into the same process may be reasonable. Certainly from the plugin author's point of view (and ideally for most of the system), the interface should be the same for both use cases. Reword, but want devs to think about possibility of loading into Viewer process.
Existing media implementations
- New versions of the embedded Web browser (using LLMozLib) and movie support (QuickTime and GStreamer) will be written using the new API.
- Ideally, we will include at least one additional Linden Lab written plugin and one more 3rd party plugin.
- Some simple examples will be included in the API documentation.
Proxy Object
A proxy object is an instance of a C++ class in the viewer which has detailed knowledge of the behaviors and messages of a particular type of plugin. Each instance of a plugin being managed by the plugin manager will have a corresponding instance of a proxy object in the viewer.
Parts of the viewer code that need to use the services of a plugin will do so by interacting with the plugin's proxy object. The proxy object will provide any necessary interfaces for interacting with the plugin, and will mirror any necessary state information about the plugin that may need to be queried by clients of that plugin.
The proxy object instance may persist for some time after the plugin instance goes away, if necessary (i.e. so that all clients of the plugin can be cleanly notified). If a plugin crashes and is relaunched, the proxy object need not be destroyed, although any clients using the proxy will be notified that the plugin's state has been reset.
The proxy object may mirror some of the state of the plugin. This can be maintained by the plugin sending unsolicited state messages when its state changes in ways meaningful to its clients. State queries can then be answered directly by the proxy object without having to do a round-trip query with messages.
The hierarchy of proxy object classes may mirror the hierarchy of message types where it makes sense to do so. For example, there will probably be different proxy object classes for browser-like and time-based media. We started out this way, but ended up merging the browser-like and time-based proxy objects into a single class, LLPluginClassMedia.
Each proxy object will have member functions that can be called which will cause messages to be sent to the plugin for which it is a proxy. Likewise, each proxy object will have a mechanism (probably a listener-type interface) that will allow other code to sign up to receive relevant messages from the plugin.
The functions in the proxy object that clients use will have a conventional function-argument signature. The message will be built up (for outgoing messages) or decoded (for incoming messages) inside the proxy object's implementation, so that the details of message formats aren't exposed to code that doesn't need to know them. If the semantics of messages need to change over time, this is the level where compatibility with older plugins/message formats will be maintained.
Testing philosophy
- Lots of QA and user testing via a 'First Look' release and blog post encouraging users to test the more esoteric existing media set ups.
- Script to grab all media URIs from the grid, add them to a playlist that is used as input to a test harness that tries to open and render each one for a few seconds. Clearly this won't test the visuals but may find potential crashers. A lot of this exists already. It's takes a long time - several days to grab URIs and a day to run through them.
- We should think about unit tests for each media implementation. A script can run through the directory and load each one, initialize it and render a known media URL at a given "timecode". The output can be captured and written to a file. We can either visually examine the file or look for an all black frame (for example) and pass/fail automatically. Not perfect but it would catch big failures.