Puppeteering

Objectives

• Further the ability to communicate in Second Life by enabling residents to 'puppeteer' their avatars
• Create a Physical Avatar code layer which enables the addition of ragdoll physics and inverse kinematics to second life avatars

Overview

Puppeteering is real-time, user-manipulated animation of avatar joints.

It refers to nascent avatar expression technology. As with a real puppet, people will be able to manipulate the joints of their avatar, and others will be able to see these changes. The initial release will be rather humble -- the ability to temporarily adjust the position of individual joints. Enabling this is the deeper infrastructure of the Physical Avatar which, in subsequent evolutions, will allow (besides Puppeteering) a variety of new tools for real-time gesturing, inverse kinematics, rag-doll physics, in-world animation building, and scripted animation.

Project details

Puppeteering is a project within Studio Shiny. Cube Linden, Aura Linden, and Ventrella Linden are designing and implementing puppeteering. Periapse Linden is assisting with project management.

The code to puppeteer one's avatar locally (visible only to the person doing the puppeteering) is complete. A simple feature to save a puppeted pose as a still animation asset is complete. A proposed UI rework (cosmetic only) is designed but not yet implemented. The networking layer, which will allow others to see the real time puppeteering of one's avatar has not been thoroughly tested, and several issues remain. The foundational work to allow puppeteering to blend in with traditional animations (for example, waving an arm or pointing while dancing) is in place, however work remains to do smooth blending.

Note: The Puppeteering team has decided to put this project on the back burner until further notice.

Roadmap

Project in hibernation so that developers have time to devote to projects that increase Second Life's stability and performance

Prototype Release: Expressive Puppeteering

NOTE: this list hasn't been edited since the project was put on hold. The features listed here may or may not be available in the release on svn.secondlife.com

• The first release is about laying the foundation for the future of avatar manipulation. Key is the development of the Physical Avatar, a code layer which models an avatar as physically-based joint balls connected by springs, and allows influence by external forces (from UI and from world). As this model is being manipulated, it continually converts back to the standard avatar representation so it can be rendered.
• The visible component of this is a simple UI enabling the momentary manipulation of avatar body parts. Initially this will be triggered by holding the CTRL key down. Mouse cursor proximity will cause joints to highlight, and a click and drag allows the positioning of that joint. As long as the CTRL key is held down the pose will be fixed, allowing the adjustment of several joints. All these manipulations will be visible to other residents. Movement, or the ESC key, will end puppeteering and revert the avatar back to original animation.
• A base set of normal joint constraints should be available to users that allow for realistic movement
• You will be able to save a puppeteering pose as an animation asset, for later use as a pose or in a gesture.

Vision

The Physical Avatar layer enables the development of several Grid-changing technologies.

• Real-time gesturing: An advanced UI, perhaps using keyboard chording or even alternate input devices to produce complicated avatar expression. Real time dancing.
• IK and rag-doll physics: Residents could walk hand in hand with realistic motion. Avatars tumble realistically when falling (or when shot, pushed).
  • Play awesomely with Havok. When bumped, ragdoll friction relative to impact force. :)
• In-world animation tools: Residents can build and save their own gestures, animations in world. No need for external modeling applications, exporting, and uploading.
• Scripted animations: Additions to LSL would allow the scripted adjustment of the Physical Avatar joint tree.
• Procedural animations: Ability to blend puppeteered actions into running animations including previously puppeteered loops.
• Touch a surface: Ride on the 'texture surface' code to move a joint to a surface on a nearby object, pulling IK chain along.

Source code

NOTE - THIS CODE IS NOT SUPPORTED, PROVIDED AS IS, AND DOES NOT REPRESENT THE DIRECTION LINDEN LAB WILL WANT TO TAKE THIS FEATURE. This source code is released purely for demonstration and research purposes

SVN URL:http://svn.secondlife.com/svn/linden/branches/2008/Puppeteering080323/

The code itself is somewhat complete - there are definitely some things we would do differently and there are still definitely bugs in there, but right now it's pretty unlikely that we will return to this to complete it or refactor that which we would want to change with the code and intent as it stands. It's based off 1.19.1 and we have a region on aditi that is running the simulator code (note, again, this code is relatively incomplete and the simulator could well crash) named Puppeteering. You may need to log directly into that region for it to work.

Direct any questions you may have about this to the SLDev mailing list.

See Also

• Render Roadmap
• avatarpuppeteering.com - JJ Ventrella's site discussing the feature.