Difference between revisions of "Media Plugin System Project Notes"

From Second Life Wiki
Jump to navigation Jump to search
Line 15: Line 15:
== Future enhancements ==
== Future enhancements ==


In the future, the plugin loader may be responsible for querying a central authority for a certificate of authenticity for a plugin it has been asked to load. Ultimately, we will have different levels of signing so users can differentiate between plugins from Linden Lab, plugins from partner companies and 3rd party providers (Granitehead Chislehome and his penis plugins). Users will be able to set a security "zone" so that no plugin outside of their zone is ever loaded. The default setting is probably obvious. None of this will be implemented in this phase of the project and is only mentioned here in case it's appropriate to lay some groundwork earlier on.
In the future, the plugin loader may be responsible for querying a central authority for a certificate of authenticity for a plugin it has been asked to load. Ultimately, we will have different levels of signing so users can differentiate between plugins from Linden Lab, plugins from partner companies and 3rd party providers. Users will be able to set a security "zone" so that no plugin outside of their zone is ever loaded. The default setting is probably obvious. None of this will be implemented in this phase of the project and is only mentioned here in case it's appropriate to lay some groundwork earlier on.


* One invocation of the plugin per media source (per URI).
* One invocation of the plugin per media source (per URI).

Revision as of 12:27, 17 August 2009

Changes to existing system

A large section of the client code that deals with moving bytes from memory into a texture managed by the client that OpenGL understands has been rewritten. There are several reasons for this:

  • This code has grown "organically" and needs a significant clean-up.
  • The process by which dynamic textures are created and managed hasn't changed since QuickTime was added to the Viewer four years ago.
  • Most importantly, the programming paradigm changes completely with this new design from a synchronous model where you call functions to a message-based model.

A plugin will specify which transport controls should be used via a fragment of XML.

The number and type of transport controls can vary over time and should support an enabled and disabled state.

Future enhancements

In the future, the plugin loader may be responsible for querying a central authority for a certificate of authenticity for a plugin it has been asked to load. Ultimately, we will have different levels of signing so users can differentiate between plugins from Linden Lab, plugins from partner companies and 3rd party providers. Users will be able to set a security "zone" so that no plugin outside of their zone is ever loaded. The default setting is probably obvious. None of this will be implemented in this phase of the project and is only mentioned here in case it's appropriate to lay some groundwork earlier on.

  • One invocation of the plugin per media source (per URI).
    • However, consideration should be given to a system that allows multiple media sources (URIs) to share a plugin invocation. For example, for rendering one-shot Web pages, it might be perfectly acceptable for a part of the client to wait a short time before displaying a Web page whilst a different page is rendered.
    • In addition, it may be optimal to cache one or more plugin invocations - for example, a significant amount of content in the Second Life client uses the embedded Web Browser so when the login page is closed and the user logs in, there is probably no point in destroying the embedded Web browser media implementation.

One-shot rendering

In one-shot mode, a message is sent to the plugin requesting that it render a particular media URL into a shared memory buffer. It renders the media once, then sends a message indicating the render has completed. This may be used when rendering large numbers of media streams at lower priorities (such as when there are a large number of streams in the user's view, but most of them are far away).

Plugins in the Viewer process

An addition to loading a plugin as a separate process, the architecture provides for loading plugins into the same process space as the Viewer. Although this entails additional risk (if the plugin crashes, it can crash the Viewer), but it may be desirable in certain cases. For example, in a situation with multiple simultaneous media, the resulting large number of separate processes will consume a lot of system resources. If a plugin has been shown to be reliable, loading it into the same process may be reasonable.

Technical notes

Shared memory

The Viewer sets up shared memory segments shared with plugin processes. Shared memory segments are identified by a name unique within each instance of the plugin. Memory segments can be created, resized (reallocated), and destroyed.

Shared memory segment setup uses message semantics: when you make a request to the Viewer to modify a segment, nothing actually happens until you receive a message telling you it completed.

The plugin interacts with the PLS through messages to set up and tear down shared memory segements, but some of those messages may be handled (or partially handled and changed) by the PLS -- they do not go directly over the control channel.

We will create native implementations for the shared memory segment with a lightweight platform abstraction. This would use the CreateFileMapping API on Windows and either mmap() or shm_open() on Mac and Linux. The details of this implementation will be hidden from plugin authors, since it will reside in the viewer and the plugin loader shell.

Proxy object

A proxy object is an instance of a C++ class in the viewer which has detailed knowledge of the behaviors and messages of a particular type of plugin. Each instance of a plugin being managed by the plugin manager will have a corresponding instance of a proxy object in the viewer.

Parts of the viewer code that need to use the services of a plugin will do so by interacting with the plugin's proxy object. The proxy object will provide any necessary interfaces for interacting with the plugin, and will mirror any necessary state information about the plugin that may need to be queried by clients of that plugin.

The proxy object instance may persist for some time after the plugin instance goes away, if necessary (i.e. so that all clients of the plugin can be cleanly notified). If a plugin crashes and is relaunched, the proxy object need not be destroyed, although any clients using the proxy will be notified that the plugin's state has been reset.

The proxy object may mirror some of the state of the plugin. This can be maintained by the plugin sending unsolicited state messages when its state changes in ways meaningful to its clients. State queries can then be answered directly by the proxy object without having to do a round-trip query with messages.

The hierarchy of proxy object classes may mirror the hierarchy of message types where it makes sense to do so. For example, there will probably be different proxy object classes for browser-like and time-based media. We started out this way, but ended up merging the browser-like and time-based proxy objects into a single class, LLPluginClassMedia.

Each proxy object will have member functions that can be called which will cause messages to be sent to the plugin for which it is a proxy. Likewise, each proxy object will have a mechanism (probably a listener-type interface) that will allow other code to sign up to receive relevant messages from the plugin.

The functions in the proxy object that clients use will have a conventional function-argument signature. The message will be built up (for outgoing messages) or decoded (for incoming messages) inside the proxy object's implementation, so that the details of message formats aren't exposed to code that doesn't need to know them. If the semantics of messages need to change over time, this is the level where compatibility with older plugins/message formats will be maintained.

Control channel

The Viewer and plugin will use the control channel to pass messages back and forth.

To establish the control channel, the Viewer listens on a local TCP socket, and passes the socket's port to the PLS. The plugin loader shell connects back to the indicated local TCP port.

To keep latency low, the system does not use the control channel for bulk data transfer. If large amounts of data need to be passed between the plugin host and a plugin, that data should not be embedded in messages sent across the control channel. Instead use either:

  • Shared memory segments.
  • Separate streams negotiated via messages.

Testing philosophy

  • Lots of QA and user testing via a 'First Look' release and blog post encouraging users to test the more esoteric existing media set ups.
  • Script to grab all media URIs from the grid, add them to a playlist that is used as input to a test harness that tries to open and render each one for a few seconds. Clearly this won't test the visuals but may find potential crashers. A lot of this exists already. It's takes a long time - several days to grab URIs and a day to run through them.
  • We should think about unit tests for each media implementation. A script can run through the directory and load each one, initialize it and render a known media URL at a given "timecode". The output can be captured and written to a file. We can either visually examine the file or look for an all black frame (for example) and pass/fail automatically. Not perfect but it would catch big failures.