Linden Lab Official:Getting Started Developing Media Rendering Plugins

Media Rendering Plugin System

• Frequently Asked Questions
• Technical Overview
• Using the Plugin Test App
• Getting Started
  • Hello World Plugin Code
• Plugin Operation and Data Flow
• Messages
  • Internal Messages

Introduction

The Media Rendering Plugin system enables you to create your own media rendering plugins for the Second Life Viewer. Currently, you must download the Snowglobe source code and build the Viewer to do so; Future releases will enable you to build plugins without building the entire Viewer.

Media rendering plugins can display content:

• Via parcel media.
• In the Viewer's [Parcel_media#The_Media_Browser_Window media browser] (the built-in web browser window).
• In the Plugin Test App provided with Viewer source code (for use during development only).

Setting up your environment

Before creating a new plugin, set up your development environment and make sure the necessary tools are in place.

System requirements and tools

• Operating System: Windows XP or Vista
• Visual Studio 2005 (Warning: Visual Studio 2008 may not work)

This guide currently assumes you are using Visual Studio. Some of the specific instructions will not apply to Mac or Linux users.

Downloading and building the Viewer

To download and build the Viewer:

• Follow the Snowglobe instructions for getting and compiling the source code

Checking that your environment was built correctly

To check that your environment was built correctly, run the example plugin both on the Plugin Test App and inworld.

The example plugin: media_plugin_example

The source code includes a simple example plugin: indra\media_plugins\example\media_plugin_example.cpp. It displays a checkerboard pattern background overlaid with bouncing squares, with automatically changing colors. Keyboard input causes the checkerboard pattern to change. You can also draw lines by holding the left mouse button down.

Running the example plugin in the Plugin Test App

Open the "(EX) Example Plugin" bookmark in a panel using the Plugin Test App.

Running the example plugin in the Viewer

1. Add the example plugin type to the Viewer by adding the following lines to indra\newview\skins\default\xui\en\mime_types.xml: <xml><scheme name="example"> <label name="example_label"> Example Plugin </label> <widgettype> image </widgettype> <impl> media_plugin_example </impl> </scheme></xml>
2. Add the example plugin to an object inworld: Using example://foo as the URL, follow the instructions for adding media to an object to use the example plugin inworld. You will see the example plugin animation displayed on the face of the object you selected.

Creating your own Hello World plugin

To create your own Hello World plugin, you need to both create the plugin C++ file and edit some supporting files.

Creating and editing the new plugin files

To create and edit the new Hello World plugin files:

1. Create a new directory: indra\media_plugins\helloworld.
2. In the new helloworld directory, create a new file called media_plugin_helloworld.cpp by copying and pasting this basic plugin code. This code implements the basic functions that are necessary in any plugin.
3. Find the "TODO" comment in the MediaPluginHelloworld::update() function, and replace it with the code snippet here. This code displays a moving square with changing color on a black background.
4. In the helloworld directory, create another new file called CMakeLists.txt by copying and pasting everything on this page.
5. Add the following line to the bottom of indra\media_plugins\CMakeLists.txt:

   add_subdirectory(helloworld)

6. Make sure the code compiles:
   1. In the indra\ directory, run develop.py.
   2. In the Visual Studio Solution Explorer pane, right-click media_plugin_helloworld and Build.

Testing your new plugin before trying it in the Viewer

Use the Plugin Test App to test your new plugin before trying it in the Viewer:

1. Add your new plugin to the Plugin Test App. Note: You can use the example code as-is.
2. Test your new plugin using the Plugin Test App.

Using your new plugin inworld

To use your new plugin inworld:

1. Add your new plugin to the Viewer:
   1. Add the following lines to indra\newview\skins\default\xui\en\mime_types.xml. This is similar to the way you modified mime_types.xml for the example plugin. <xml><scheme name="helloworld"> <label name="helloworld_label"> Hello World Plugin </label> <widgettype> image </widgettype> <impl> media_plugin_helloworld </impl> </scheme></xml>
   2. Add your plugin to the list of dependencies for the Viewer in indra\newview\CMakeList.txt:
      • Search for every instance of "media_plugin_example" in an add_dependencies() line and add "media_plugin_helloworld". Example:

        Old: add_dependencies(${VIEWER_BINARY_NAME} SLPlugin media_plugin_quicktime media_plugin_webkit media_plugin_example)

        New: add_dependencies(${VIEWER_BINARY_NAME} SLPlugin media_plugin_quicktime media_plugin_webkit media_plugin_example media_plugin_helloworld)

      • Search for "media_plugin_example" in a get_target_property() line. Copy and paste the entire get_target_property() section, replacing "example" with "helloworld". The new section will look like this:

          get_target_property(BUILT_HELLOWORLD_PLUGIN media_plugin_helloworld LOCATION)
            add_custom_command(
                TARGET ${VIEWER_BINARY_NAME} POST_BUILD
                COMMAND ${CMAKE_COMMAND}
                ARGS
                  -E
                  copy_if_different
                  ${BUILT_HELLOWORLD_PLUGIN}
                  ${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_CFG_INTDIR}/llplugin
                COMMENT "Copying Hello World Plugin to the runtime folder."
                )
        

2. Re-build the Viewer.
3. Start the Viewer, choose Help, then in the media browser window that opens, enter the URL of your media, helloworld://foo. If desired, you can also log into Second Life add your media as Parcel media.

Congratulations! You have successfully created your first simple plugin.

How the Hello World plugin works

The Hello World plugin implements the basic functions that any media rendering plugin needs to work correctly.

Messages

A media rendering plugin uses messages to communicate with the Plugin Loader Shell, which mediates requests between the plugin and the Viewer.

Initialization

When a plugin is loaded, the plugin and Plugin Loader Shell exchange specific messages to initialize the plugin. While a plugin is running, the Plugin Loader Shell uses messages to notify it of mouse and keyboard input.

In the Hello World plugin, init_media_plugin() creates the message interface between the plugin and Plugin Loader Shell: <cpp> int init_media_plugin( LLPluginInstance::sendMessageFunction host_send_func,

                 void* host_user_data,
                 LLPluginInstance::sendMessageFunction *plugin_send_func,
                 void **plugin_user_data )

{

  MediaPluginHelloworld* self = new MediaPluginHelloworld( host_send_func, host_user_data );
  *plugin_send_func = MediaPluginHelloworld::staticReceiveMessage;
  *plugin_user_data = ( void* )self;

  return 0;

} </cpp>

To initialize the plugin, the Plugin Loader Shell sends an "init" message to the Hello World plugin. The plugin responds with messages that contain information about the plugin version ("init_response") and the object's texture parameters ("texture_params"): <cpp>

  if ( message_name == "init" )
  {
     LLPluginMessage message( "base", "init_response" );
     LLSD versions = LLSD::emptyMap();
     versions[ LLPLUGIN_MESSAGE_CLASS_BASE ] = LLPLUGIN_MESSAGE_CLASS_BASE_VERSION;
     versions[ LLPLUGIN_MESSAGE_CLASS_MEDIA ] = LLPLUGIN_MESSAGE_CLASS_MEDIA_VERSION;
     versions[ LLPLUGIN_MESSAGE_CLASS_MEDIA_BROWSER ] = LLPLUGIN_MESSAGE_CLASS_MEDIA_BROWSER_VERSION;
     message.setValueLLSD( "versions", versions );

     std::string plugin_version = "Hello World media plugin, Hello World Version 1.0.0.0";
     message.setValue( "plugin_version", plugin_version );
     sendMessage( message );

     // Plugin gets to decide the texture parameters to use.
     message.setMessage( LLPLUGIN_MESSAGE_CLASS_MEDIA, "texture_params" );
     message.setValueS32( "default_width", mWidth ); // width in pixels
     message.setValueS32( "default_height", mHeight ); // height in pixels
     message.setValueS32( "depth", mDepth ); // pixel size in bytes
     message.setValueU32( "internalformat", GL_RGBA );
     message.setValueU32( "format", GL_RGBA );
     message.setValueU32( "type", GL_UNSIGNED_BYTE );
     message.setValueBoolean( "coords_opengl", false );
     sendMessage( message );
  }

</cpp>

Idle messages

The Plugin Loader Shell also sends frequent "idle" messages to the plugin. The "idle" messages indicate that time is passing, so the plugin can update its state.

When the Hello World plugin receives an "idle" message, it updates the color and position of the square in the image by calling update(): <cpp>

  if ( message_name == "idle" )
  {
     // no response is necessary here.
     double time = message_in.getValueReal( "time" );

     std::cout << "MediaPluginHelloworld::receiveMessage(): idle"<<std::endl;
     // Convert time to milliseconds for update()
     update( time );
  }

</cpp>

Keyboard input

If keyboard input occurs, the Plugin Loader Shell sends a key_event message to the plugin. The plugin can see what key was pressed and act accordingly.

In the Hello World plugin, pressing the space key causes the plugin to update: <cpp>

  if ( message_name == "key_event" )
  {
     std::string event = message_in.getValue( "event" );
     S32 key = message_in.getValueS32( "key" );
     std::string modifiers = message_in.getValue( "modifiers" );

     if ( event == "down" )
     {
        if ( key == ' ')
        {
           mLastUpdateTime = 0;
           update( 0.0f );
        };
     };
  }

</cpp>

Mouse input

If the Hello World plugin receives a "left mouse button down" message, it detects the location of the mouse cursor and creates a white dot by writing to shared memory: <cpp>

  if ( message_name == "mouse_event" )
  {
     std::string event = message_in.getValue( "event" );
     S32 button = message_in.getValueS32( "button" );

     // left mouse button
     if ( button == 0 )
     {
        int mouse_x = message_in.getValueS32( "x" );
        int mouse_y = message_in.getValueS32( "y" );
        std::string modifiers = message_in.getValue( "modifiers" );

        if ( event == "move" )
        {
           if ( mMouseButtonDown )
              write_pixel( mouse_x, mouse_y, 0xFF, 0xFF, 0xFF);
        }
        else
        if ( event == "down" )
        {
           mMouseButtonDown = true;
        }
        else
        if ( event == "up" )
        {
           mMouseButtonDown = false;
        }
        else
        if ( event == "double_click" )
        {
        };
     };
  }

</cpp>

Shared memory

The shared memory represents the area displayed by the Viewer. In response to the "texture_params" message, the Plugin Loader Shell sends a "shm_added" message with a pointer to the shared memory that the plugin will use to transfer display data: <cpp>

  if ( message_name == "shm_added" )
  {
     SharedSegmentInfo info;
     info.mAddress = message_in.getValuePointer( "address" );
     info.mSize = ( size_t )message_in.getValueS32( "size" );
     std::string name = message_in.getValue( "name" );

     mSharedSegments.insert( SharedSegmentMap::value_type( name, info ) );

  }

</cpp>

Display

The Hello World plugin writes an image to shared memory using RGB color values. The Viewer reads the image from shared memory and displays it. In the Hello World plugin, the update() function writes the image to shared memory: <cpp>

  if ( time( NULL ) > mLastUpdateTime + 1 )
  {
     // Draw black background
     int bkgnd_r = 0;
     int bkgnd_g = 0;
     int bkgnd_b = 0;
     for (int bkgnd_pix = 0; bkgnd_pix < getSurfaceSize(); bkgnd_pix = bkgnd_pix+mDepth)
     {
        mBackgroundPixels [bkgnd_pix + 0] = bkgnd_r;
        mBackgroundPixels [bkgnd_pix + 1] = bkgnd_g;
        mBackgroundPixels [bkgnd_pix + 2] = bkgnd_b;
     }
   
     // Set the color of the moving square
     squareR = (squareR+1) % (0xFF-0x20) + 0x20;
     squareG = (squareG+7) % (0xFF-0x20) + 0x20;
     squareB = (squareB+7) % (0xFF-0x20) + 0x20;
   
     time( &mLastUpdateTime );

  }
  memcpy( mPixels, mBackgroundPixels, getSurfaceSize() );

  // Black out last position of square
  for ( int xcnt = 0; xcnt < squareWidth; ++xcnt )
  {
     for ( int ycnt = 0; ycnt < squareHeight; ++ycnt )
     {
        mPixels [ (squareXpos + xcnt)*mDepth + (squareYpos + ycnt)*mWidth*mDepth + 0 ] = 0;
        mPixels [ (squareXpos + xcnt)*mDepth + (squareYpos + ycnt)*mWidth*mDepth + 1 ] = 0;
        mPixels [ (squareXpos + xcnt)*mDepth + (squareYpos + ycnt)*mWidth*mDepth + 2 ] = 0;
     }
  }

  // Calculate the new position of the moving square. Note: (0,0) is the top left corner.
  //
  if (rand() % 400 == 0) // only change direction once in a while
  {
     randomizeDirection();
  }

  if ((squareXpos + squareXinc < 0) || (squareXpos + squareXinc >= mWidth - squareWidth))
     squareXinc = -squareXinc;

  if ((squareYpos + squareYinc < 0) || (squareYpos + squareYinc >= mHeight - squareHeight))
     squareYinc = -squareYinc;

  squareXpos += squareXinc;
  squareYpos += squareYinc;

  // Draw square
  for ( int xcnt = 0; xcnt < squareWidth; ++xcnt )
  {
     for ( int ycnt = 0; ycnt < squareHeight; ++ycnt )
     {
        mPixels [ (squareXpos + xcnt)*mDepth + (squareYpos + ycnt)*mWidth*mDepth + 0 ] = squareR;
        mPixels [ (squareXpos + xcnt)*mDepth + (squareYpos + ycnt)*mWidth*mDepth + 1 ] = squareG;
        mPixels [ (squareXpos + xcnt)*mDepth + (squareYpos + ycnt)*mWidth*mDepth + 2 ] = squareB;
     }
  }

</cpp>

Troubleshooting tips

Setting breakpoints in plugin code

A plugin runs as a separate process from the Viewer, so breakpoints can only be set after the Viewer is running and the plugin has also started running.

To set breakpoints in plugin code:

1. In Visual Studio, with the project set to secondlife-bin, select Debug->Start Without Debugging.
2. Log into Second Life and go to the object using your media rendering plugin.
3. Select Debug->Attach To Process, and select the SLPlugin.exe items in the list.

You can now set breakpoints in your plugin code.