Media Plugin System Project Notes

Media Rendering Plugin System

• Frequently Asked Questions
• Technical Overview
• Using the Plugin Test App
• Getting Started
  • Hello World Plugin Code
• Plugin Operation and Data Flow
• Messages
  • Internal Messages

Changes to existing system

A large section of the client code that deals with moving bytes from memory into a texture managed by the client that OpenGL understands has been rewritten. There are several reasons for this:

• This code has grown "organically" and needs a significant clean-up.
• The process by which dynamic textures are created and managed hasn't changed since QuickTime was added to the Viewer four years ago.
• Most importantly, the programming paradigm changes completely with this new design from a synchronous model where you call functions to a message-based model.

A plugin will specify which transport controls should be used via a fragment of XML.

The number and type of transport controls can vary over time and should support an enabled and disabled state.

Technical notes

Shared memory

The Viewer sets up shared memory segments shared with plugin processes. Shared memory segments are identified by a name unique within each instance of the plugin. Memory segments can be created, resized (reallocated), and destroyed.

Shared memory segment setup uses message semantics: when you make a request to the Viewer to modify a segment, nothing actually happens until you receive a message telling you it completed.

The plugin interacts with the Plugin Loader Shell (PLS) through messages to set up and tear down shared memory segments, but some of those messages may be handled (or partially handled and changed) by the PLS -- they do not go directly over the control channel.

We will create native implementations for the shared memory segment with a lightweight platform abstraction. This would use the CreateFileMapping API on Windows and either mmap() or shm_open() on Mac and Linux. The details of this implementation will be hidden from plugin authors, since it will reside in the viewer and the plugin loader shell.

Proxy object

A proxy object is an instance of a C++ class in the viewer which has detailed knowledge of the behaviors and messages of a particular type of plugin. Each instance of a plugin being managed by the plugin manager will have a corresponding instance of a proxy object in the viewer.

Parts of the viewer code that need to use the services of a plugin will do so by interacting with the plugin's proxy object. The proxy object will provide any necessary interfaces for interacting with the plugin, and will mirror any necessary state information about the plugin that may need to be queried by clients of that plugin.

The proxy object instance may persist for some time after the plugin instance goes away, if necessary (i.e. so that all clients of the plugin can be cleanly notified). If a plugin crashes and is relaunched, the proxy object need not be destroyed, although any clients using the proxy will be notified that the plugin's state has been reset.

The proxy object may mirror some of the state of the plugin. This can be maintained by the plugin sending unsolicited state messages when its state changes in ways meaningful to its clients. State queries can then be answered directly by the proxy object without having to do a round-trip query with messages.

Each proxy object will have member functions that can be called which will cause messages to be sent to the plugin for which it is a proxy. Likewise, each proxy object will have a mechanism (probably a listener-type interface) that will allow other code to sign up to receive relevant messages from the plugin.

The functions in the proxy object that clients use will have a conventional function-argument signature. The message will be built up (for outgoing messages) or decoded (for incoming messages) inside the proxy object's implementation, so that the details of message formats aren't exposed to code that doesn't need to know them. If the semantics of messages need to change over time, this is the level where compatibility with older plugins/message formats will be maintained.

Control channel

The Viewer and plugin will use the control channel to pass messages back and forth.

To establish the control channel, the Viewer listens on a local TCP socket, and passes the socket's port to the PLS. The plugin loader shell connects back to the indicated local TCP port.

To keep latency low, the system does not use the control channel for bulk data transfer. If large amounts of data need to be passed between the plugin host and a plugin, that data should not be embedded in messages sent across the control channel. Instead use either:

• Shared memory segments.
• Separate streams negotiated via messages.

Testing philosophy

Think about unit tests for each media implementation. A script can run through the directory and load each one, initialize it and render a known media URL at a given "timecode". The output can be captured and written to a file. We can either visually examine the file or look for an all black frame (for example) and pass/fail automatically. Not perfect but it would catch big failures.