Morph Target Community Proposal
From Second Life Wiki
This page is to contain the overall community proposal for "Morph Targets".
Please do not add to this page without authorization from a co-chair or a co-chair's assistant from the Content Creation Improvement User Group first.
Please discuss any relevant changes in the discussion tab of this page.
What is a Morph Target?
A morph target, also known as a blend shape, or a shape key in some 3D applications, is a special kind of "vertex map" containing the positions of vertices in relation to their original position on a mesh. They are relatively easy to work with, only requiring the content creator to "sculpt" or otherwise move the vertices on their meshes to the positions they'd like for a morph target.
Most 3D applications support authoring morph targets, such as Luxology Modo, Autodesk 3ds Max, Autodesk Maya, and Blender. The Collada file format also supports morph targets.
What are they used for?
Most modern video games use morph targets for complex animations that would otherwise be hard to accomplish with skeletal animation, such as facial animation, and even things like character customization. The viewer its self actually uses morph targets to enable avatar customization already.
Why do we need them?
A a small list of examples of what they could be used for in the context of Second Life include:
- Complex facial animations on rigged meshes
- Additional ways for a user to customize the appearance of their avatar
- Animating the surface of a prim without the need to use a custom skeleton
- Fine grained control for content creators over how their clothing and avatars deform
There's a lot that can be achieved with morph targets, and this is just a small list of things that can be accomplished
Caveats of Existing Techniques
Existing techniques, such as RedPoly's proposed cBones technique and Qarl Fizz's Mesh Deformer, have their own disadvantages that morph targets are intended to either mitigate, or solve completely.
"cBones" as they're called, have the caveat of requiring content creators to rig many more bones than what would otherwise be necessary for simply animating a rigged mesh. It extends the workflow of rigging a mesh substantially into taking several hours of work; sometimes with a significant amount of trial and error. On top of that, a content creator sometimes is working "blind" with regards to what the final outcome will be in-world. This is not seen as an ideal workflow, as it increases the burden on content creators to produce this content. It does, however, provide more control over the deformation process, and is potentially faster than Qarl's mesh deformer.
Qarl's mesh deformer has the benefit, and caveat, of deforming a rigged mesh in accordance to the avatar's current shape automatically. The reason why some may see this as a caveat, is because this doesn't ensure correct deformation all the time. An example of where this may be undesirable can include custom mesh avatars, where the entire avatar mesh is replaced. A content creator may want to make their avatar's belly deform differently than the avatar mesh's belly does, or may want to have their avatar's breasts deform differently. This is a problem that morph targets could easily solve, by putting content creators in control of how a rigged mesh deforms. The mesh deformer also currently depends on the content creator making their content around a specific shape for it to work properly. Morph targets would lessen the need for this to some degree.
Morph Target Advantages
There are a few advantages of morph targets that existing solutions do not provide.
Morph targets allow a content creator to literally "sculpt" how a morph target should appear, directly within their 3D application of choice. This gives them the ability to directly control over how a mesh "deforms" around the avatar, or in the context of rigged mesh replacement avatars, how an avatar "deforms". An ideal workflow would enable content creators to create a "morph map", a special vertex map in their 3D application of choice, and begin sculpting how that morph target should look in-world with respect to the associated avatar mesh morph target.
Each morph target currently should correspond to an existing morph target on the avatar mesh. Some examples include:
- Belly size
- Torso muscularity
- Saddle bags
- Body fat
- Breast size
There are already several pre-existing tools that can work with morph targets, and there's several ways morph targets can be tested. 3D modeling applications such as Autodesk Maya, Autodesk 3ds Max, and Luxology Modo all support morph targets out of the box, and the Collada file format also supports morph targets. One could effectively test different morph target combinations directly within their 3D modeling application of choice, or even existing game-engine toolsets. This would allow for content creators to rapidly iterate with their work, making adjustments on the fly with minimal slow down by cutting out the need to upload meshes to test their work.
Another workflow possibility is one could effectively "bake" the different potential morphs of a mesh, and simply tweak them until the content creator has the general effect they're looking for.
Morph targets do have some performance implications, however they're largely mitigable. Depending on the implementation, morph targets could have a very small performance impact that could be less than that of Qarl's deformer. This would mean more avatars could be in view using morph targets on more rigged meshes, enabling more people to experience the benefits of mesh clothing that fits properly.
Furthermore, morph targets could some day become an optimization to Qarl's deformer by precalculating the morph targets for a mesh on import. This would provide the potential performance benefits that morph targets provide, while still enabling an automatic solution for content creators to make use of.
Morph Target Disadvantages
Morph targets are not entirely a perfect solution. There are a few notable disadvantages relating to bandwidth usage, and potential workflow disadvantages relating to specific implementations.
Morph targets require additional information to be encoded either within a mesh, or within a texture. This means that additional bandwidth will be required to transmit any mesh with morph targets.
The bandwidth cost could be mitigated by either determining what vertices have a "similar" position across different morph targets and simply reusing those values, by only including a partial vertex map for the morph target by excluding vertices that don't have unique positions in the vertex map, or by even using textures that can be loaded progressively.
The amount of bandwidth required to support morph targets will vary depending on the implementation approach taken.
Although the workflow is largely more streamlined than that of rigging a mesh, it does require content creators learn how to efficiently create morph targets. This will vary largely upon the implementation approach taken to support morph targets on Second Life.
There are a few possible implementation approaches. They each of their advantages and disadvantages associated with them.
Vector Displacement Maps
A vector displacement map encodes the positional information for a given vertex on a mesh. They can operate with relatively low overhead, and can be accessed through a vertex shader to displace an object's vertices. They can also be "baked" and transmitted to other viewers after a morph has been finalized. They're also capable of working with or without tessellation, and can operate reasonably well at lower resolutions.
They would also benefit from progressive loading, and could effectively function as a regular texture on the asset backend. Being regular images effectively means they can also take advantage of image compression.
They can also be used on practically any prim face as well, potentially paving the path towards a general purpose solution in the future for assigning morph targets to standard primitives, and virtually any arbitrary mesh in-world furthering the overall customization of in-world content.
The primary disadvantage to this technique, would be the ability to "copy" the content without the creator's permission by simply using the UUID to the VDM.
The VDM workflow could be largely transparent to content creators, by creating them on mesh upload.
A VDM would be applied to the surface of a mesh, or a prim, through a special set of parameters. Multiple VDMs should be supported for morph targets, each one having a parameter that controls the strength of the morph.
Either the viewer could create an interpolated VDM on the CPU which would then be uploaded to the GPU for displacing a surface's vertices, or it could be a CPU-bound process wherein the VDMs are interpolated and applied to a surface's vertices.
Vertex Map Pruning
Vertex map pruning could work by storing a "common" position look up table that's derived from the mesh's morph targets. Each common position could be determined by comparing vertex positions within a given threshold, and positions that are similar are averaged and added to the lookup table. This entire process would be completely transparent to the content creator, and would be determined at import.
This would help mitigate the bandwidth concerns, while also mitigating content theft concerns as meshes are not currently trivial to "copybot".
The downside would be that this requires extensions to the mesh format that could create problems for older viewers that wouldn't support the new morph target functionality. Fallback mechanisms may need to be devised to ensure backwards compatibility. Performance may also become an issue, but ways to mitigate it is to expose a single vertex map to the GPU that contains interpolated vertex positions, from which the GPU can apply within a vertex program better enabling hardware acceleration on a wider range of hardware.
As with VDMs, vertex map pruning would be transparent to content creators since much of the process would be done at import (meaning content creators will not need to know how to "prune" a morph target's vertices; just that they will be pruned within a given threshold).
There are a few ways this could potentially be implemented, but the simplest would be to have a look up table of common vertex positions derived from the mesh's morph maps, with each reference to this look up table being a 16-bit index that corresponds to the appropriate location within the lookup table.
Morph extensions could be thought of as a partial implementation of morph targets specifically for use in combination with Qarl's deformer. This would be a blend between Qarl's mesh deformer, and vertex map pruning.
The idea behind this is to define unique vertex positions that would then be encoded into a vertex map on the mesh, much like how morph targets work in general. The defining factor however, is the mesh deformer would do the majority of the deformation, with the morph target only containing the vertex positions that otherwise wouldn't be included if the content creator were to just rely on the mesh deformer.
Deformer extensions could be calculated upon upload, using the mesh deformer to help determine which portions of the morph targets to discard in the process.
This could be seen as a partial solution, and potentially has the highest level of bandwidth mitigation, while still retaining some of the control that full morph targets would give content creators.
The biggest disadvantage with this would primarily involve educating content creators how to create morph targets that would yield the best results with this method.
The workflow for deformer extensions has the potential to get tedious. Content creators would have to know how the deformer would deform their content, and plan their morph targets accordingly. Although it has the ability to cut down on the amount of bandwidth needed for morph targets, the workflow may be untenable for content creators.
Deformer extensions would effectively blend one of the two previous options with Qarl's mesh deformer. Using vertex map pruning, the majority of the morph target would reference the same exact vertex position within the common position look up table. A new set of parameters that get transmitted with rigged meshes that controls the overall morph target value would accompany an avatar's shape texture which would then control how much a morph is applied. A reference position could also be encoded for each morph target as well, allowing the viewer to decide how far a rigged mesh's vertices should extend before a given morph is applied, reducing the need to transmit additional parameters.
Proposed Initial Feature Set
Since the initial release of morph targets would be geared towards fashion and avatar designers, the initial version should primarily have features that appeal to those people. As such, here is the proposed initial feature set for a version one release:
- Import of morph targets within a Collada file
- Application of morph targets on a worn rigged mesh
- Support for existing shape parameters (i.e., a morph target named "saddle_bags" in a mesh would correspond to the saddle bag shape parameter)
- The ability to define custom morph parameters (i.e., foot_toe to control toe size, face_expression_smile to control a mesh's smile and other facial expressions, etc.)
- The ability to animate morph parameters
- The ability to define morph targets on static meshes