Second Life Wiki - User contributions [en]

Compiling the viewer (Mac OS X)

2010-06-24T01:50:04Z

Joshua Linden: Link to SNOW-223 for issues with 10.6 SDK

{{Multi-lang}}
{{CompileNav}}

The following are instructions for building the Second Life viewer on Mac OS X. For other platforms, see [[Compiling the viewer]]

You will also need to check the '''Build Notes''' column of the table on [[source downloads]] page.

= Development Environment =

Most Lindens use XCode 3.0 on Leopard for building on Macintosh computers (though some still use XCode 2.4). For simplicity, we suggest installing everything from the mpkg.

You will need to install the proprietary libraries as well as all of the other third party libraries below.

= Installing Files =

== CMake (1.21 and later) ==

Install CMake 2.6.2 from [http://www.cmake.org cmake.org].

== Viewer Source ==

You can get source either from [[#Source Snapshot|a source snapshot tarball]], or [[#Check Out From Source Control|check out from the source control system]].

=== Source Snapshot ===

Sources for each release are available on the [[source downloads]] page.

If you have Stuffit installed on your system, it may automacially unstuff .gz, so use tar xvf, without the z to extract the tar file. It may also automatically unzip files. Instead of clicking on the file link, right click on the file link and select Download Linked File which will keep .gz and .zip intact.

When you have downloaded, create a working directory (the directory name must not contain any spaces - most Linden Lab developers use '''~/linden''') and unpack the snapshot (replace ''x.x.x.x'' with the version number of the snapshot you downloaded):

$ mkdir ''linden''
$ cd ''linden''
$ tar xvfz ''path-to-download/''slviewer-src-''x.x.x.x''.tar.gz

then proceed to [[#Viewer Libraries and Artwork]].

The exact filenames will differ with the version number. If you open them with the double click file extract, remember that dragging folders on top of each other will overwrite the original contents, not merge them as in Windows.

Check the [[source downloads]] page for any special Build Notes associated with the viewer version, if any.

=== Check Out From Source Control ===

To check out sources from the subversion source control system, choose a working directory name (the directory name must not contain any spaces - most Linden Lab developers use ''~/linden''), and check out the sources:

$ svn co ''repository-url'' ''linden''

where
;''repository-url''
:is the url for the appropriate branch; see '''TBD'''
;''linden''
:is the working directory name you have chosen (you do not need to create this directory before doing the checkout)

then proceed to [[#Viewer Libraries and Artwork]].

== Viewer Libraries and Artwork ==

The required pre-compiled third-party libraries, and artwork files are available from the [[source downloads]] page. Download the Mac library package, and the Artwork zip file.

Unpack them into the working directory that you created in the [[#Viewer Source]] step. The directory that you work in should should have no spaces in the name. For example, from the directory where you downloaded the tarballs :

$ cd ''linden/''..
$ tar xvfz ''path-to-downloads/''snowglobe-darwin-libs-''x.x.x.x''.tar.gz
$ unzip ''path-to-downloads/''snowglobe-artwork-''x.x.x.x''.zip

If you have Stuffit installed on your system, it may automacially unstuff .gz, so use tar xvf, without the z to extract the tar file. It may also automatically unzip files. Instead of clicking on the file link, right click on the file link and select Download Linked File which will keep .gz and .zip intact.

The exact filenames will differ with the version number. If you open them with the double click file extract, remember that dragging folders on top of each other will overwrite the original contents, not merge them as in Windows.

Check the [[source downloads]] page for any special Build Notes associated with the viewer version, if any.

Proceed to [[#Installing Proprietary Libraries]].

=== Installing Libraries From Scratch (Optional) ===

For convenience, Lindens package up the libraries they are allowed to distribute so you can download and unpack them into your development working directory. The [[#Viewer Source, Libraries, and Artwork|above procedure]] installs pre-compiled third-party library files. Alternatively, if you want to build the libraries yourself, See [[Compiling the viewer libraries (Mac OS X)|another page]].

== Installing Proprietary Libraries ==

The Viewer depends on some proprietary libraries. Lindens do not distribute these libraries, so you will need to fetch and install these even if you download the libraries packages. (This is due to licensing restrictions. Don't ask, Lindens already did, and can't get permission. So you do have to get them yourself.)

=== Fmod ===
#Download & extract [http://www.fmod.org/files/fmod3/ fmod 3.75 programmers api for macintosh].
#:''You do '''not''' want the latest version''
#Copy the extracted files, combining the separate PowerPC and x86 libraries into a universal binary.

mkdir -p ''linden/''libraries/include
mkdir -p ''linden/''libraries/universal-darwin/lib_debug
mkdir -p ''linden/''libraries/universal-darwin/lib_release
cp -p fmodapi375mac/api/inc/*.h ''linden/''libraries/include
lipo -create fmodapi375mac/api/lib/libfmod.a fmodapi375mac/api/lib/libfmodx86.a -output ''linden/''libraries/universal-darwin/lib_debug/libfmod.a
touch -r fmodapi375mac/api/lib/libfmodx86.a ''linden/''libraries/universal-darwin/lib_debug/libfmod.a
cp -p ''linden/''libraries/universal-darwin/lib_debug/libfmod.a ''linden/''libraries/universal-darwin/lib_release/libfmod.a

= Building the Viewer with CMake =
[http://cmake.org CMake] is a system for generating per-platform build files. On OS X, it will generate your choice of Makefiles or Xcode project files.

:'''NOTE:''' - these instructions only apply to version 1.21 and later of the Second Life viewer. For instructions about compiling earlier versions, please see [[Compiling the viewer (Mac OS X, 1.20 and earlier)]]

== Quickstart ==

#Go into the <code>''linden/''indra</code> directory, and run <code>develop.py</code>.
#Launch XCode, open the project file "''linden/''indra/build-darwin-universal/SecondLife.xcodeproj",
#In XCode's project settings, set the base SDK to Mac OS X 10.6 (NOTE: see [http://jira.secondlife.com/browse/SNOW-223]) (or 10.5, if you're not on Snow Leopard), and the compiler version to GCC 4.0.
#Build the project. This can take a very long time, depending on your system, but will likely use nearly 100% CPU for most of the process. Get a cup of coffee.

== Build Configurations ==

; Debug: This configuration is more suitable for debugging. The build process will create the SecondLife application targeted for your host architecture. Unoptimized, includes debug symbols.
; RelWithDebInfo : Essentially the same as the Debug target without LL_DEBUG defined. This disables a significant amount of sanity checking which slows down the viewer. Unoptimized, includes debug symbols.
; Release : Optimised, without debug info.

== Configuring your tree ==
Before you first run a build, you'll need to configure things. There's a <code>develop.py</code> script that will create a reasonably sane default configuration for you.

From the command line, '''cd into the indra subdirectory''' and run one of the following commands (depending on your choice of platform and build environment):
** XCode: <code>"./develop.py"</code>
** make: <code>"./develop.py -G 'Unix Makefiles'"</code>

'''NOTE:''' The above commands will configure a "non-standalone" version of the source code tree. This means that the required third party library packages (as built by Linden Lab) will be downloaded during the CMake process.

== Compiling ==

In the CMake world, we keep source and object files separate. The <code>develop.py</code> script will create and populate a build directory for you, located in the <code>build-darwin-universal</code> directory.

To start a build,
# Run "./develop.py build").

-- or --

# Change to the build directory (<code>build-darwin-universal</code>)
# Run <code>make</code>

-- or --

# Launch XCode, open the project file 'linden/indra/newview/build-darwin-universal/SecondLife.xcodeproj',

... and away you go!

== Where's the built viewer? ==

On OS X, your viewer build will be here by default:
<pre>
build-darwin-universal/newview/RelWithDebInfo/Second Life.app
</pre>

If you change the kind of build you use, the intermediate directory will also change, e.g. from <code>RelWithDebInfo</code> to <code>Release</code>.

= Building the Unit Tests =
From XCode, open the project 'linden/indra/test/MacTester.xcodeproj', set 'MacTester' as the active target, and build.

= What to do if it doesn't work for you =

* Check your environment. Ensure you have the OS X 10.6 (NOTE: see [http://jira.secondlife.com/browse/SNOW-223]) or 10.5 SDK selected and installed (reinstall XCode if needed to get it) and have set the Compiler Version to GCC 4.0 in XCode's Project Settings.
* If the build complains of missing proprietary libraries, such as libopenal.dylib, you may need to copy them manually to the location the build scripts expect them to be.
* If you can't find certain files that the build process expects, you can copy them an already-compiled version of the viewer. You can do this by looking inside the viewer's application package [right click > Show Package Contents]. This applies equally to branched or third-party variants of the viewer which may have additional dependencies.
* Ask for help on [[IRC]] (irc.freenode.net #opensl)
* Ask for help on the [[SLDev|SLDev mailing list]]
* Fix it: [[Modifying CMake Files‎]] (and please, submit a patch!)

= Submitting Patches =
This is probably far down the road, but if you make changes to the source and want to submit them, see the page about [[Submitting patches|submitting patches]].

[[Category:Compiling viewer]]

Channel and Version Requirements

2009-12-11T18:10:24Z

Joshua Linden: /* Updating an Open Source Viewer */

== Setting a Channel and Version for an Open Source Viewer ==
1. Choose a channel name
* The default channel name is "Second Life Release"
* Other Linden Lab channel names are "Second Life Release Candidate" and "Second Life First Look [Feature]"
* The channel name can be any ASCII string, however we suggest something along the lines of:
** "Second Life Open Source [Brand and/or Feature]"
2. Choose a version number
* The version number is in the form Major.Minor.Patch.Build
* The version number can be any four numbers
* We recommend using the Major, Minor, and Patch numbers from the most recently merged Linden Lab source code.
* We recommend using a Build number >= 100 to indicate a non Linden Lab version.
** Linden Lab uses the Subversion revision number for the build number for non Release / Release Candidate builds
3. Set the Channel and Version in the code
* Edit indra/llcommon/llversionviewer.h
** Modify LL_CHANNEL with the Channel Name
** Modify LL_VERSION_BUILD with the appropriate build number
* Update the version number in the following files:
** indra/newview/res/newViewRes.rc - ''"FileVersion" and "ProductVersion" in the Version section''
** indra/newview/Info-SecondLife.plist - ''CFBundleVersion''
** indra/newview/English.lproj/InfoPlist.strings - ''CFBundleShortVersionString''
* Build the code

== Packaging for Distribution ==
* From the indra/newview directory run viewer_manifest.py with the following arguments:
** viewer_manifest.py --login_url=http://your.login.url.com
* See http://wiki.secondlife.com/wiki/Viewer_Manifest for more information on packaging the Viewer

== Updating an Open Source Viewer ==
* Anyone distributing an Open Source version of the Viewer is responsible for messaging updates to residents using that client.
* We recommend using your login url for directing residents to updates.
* Clients not based on the Linden viewer code base should ensure that the XMLRPC login connection sends:
** channel = the channel name, e.g. "My Viewer"
** version = the channel name and a four-part version, e.g. "My Viewer 1.2.3.4"

== Linden Lab Updates ==
* '''Linden Lab updates will not affect non Linden Lab channels.''' This includes optional '''and''' mandatory updates. It is the responsibility of other distributors to ensure that security patches are provided to residents using their client.
* '''NOTE: If Linden Lab determines that a particular client distribution contains security vulnerabilities or exploits, Linden Lab may block that particular version of the client from logging in.'''
* '''NOTE: If Linden Lab introduces a server update that requires a particular capability in order to connect to the server, Viewers that do not include the required capability will be unable to connect to ''regions running that version of the server.''''' It is the intention of Linden Lab to make these required updates infrequent, and to make the Viewer code available well in advance of the server update.

IEEE Internet Computing: VWRAP for Virtual Worlds interop

2009-11-05T18:47:43Z

Joshua Linden: Blanked the page

IEEE Internet Computing: VWRAP for Virtual Worlds interop

2009-10-30T21:29:01Z

Joshua Linden: /* A brief history of virtual worlds */

''' ''Tentative skeletal pre-draft of a possible article.'' ''' --- '''Remember the wiki motto, "Be Bold."''' :-)

== Introduction ==

* <<<Set the scene with an epic literary start>>

Mankind is an imaginative, storytelling species. From our first glimmers of intelligence in the wielding of a bone as a weapon, through to today's visions of our future among the stars, we have always imagined new worlds that are different to those of yesterday. We have continually redefined our physical world through progress, created innumerable fantasy worlds through sheer imagination, and told stories about past worlds and future worlds to keep our culture and visions alive. Some new worlds we have even reached physically by departing from our little planet of origin into the darkness of space. We're really not happy with just a single world.

Computing is a microcosm of everything else that Mankind does, so it's no surprise that our worlds of the imagination have become part of this new space as well, indeed an essential part. We call our computer-driven worlds ''virtual worlds'', although the adjective is oddly superfluous. As has been the case with all our worlds throughout history, these CPU-assisted worlds gain their ''immersive'' reality as worlds entirely within our minds.

This article is about a small corner of the huge space of virtual worlds, a corner in which one large established virtual world and a constellation of small newer ones share sufficient commonality that a desire for interoperability between them has taken root, under the auspices of the IETF.

== A brief history of virtual worlds ==

* Here's my take - it probably needs to be edited down significantly. --[[User:Joshua Linden|Joshua Linden]] 21:29, 30 October 2009 (UTC)

Computer-based virtual worlds can be defined as having representations of multiple users, often called characters or avatars, and locations which those user-representations occupy. While virtual environments can be single user (with or without an explicit user representation), we restrict our focus to multi-user systems which generally implies a network, though not necessarily any particular topology or capability of the nodes. Such a definition spans the gamut of visualization options for the world - from text through 2D to fully immersive 3D graphics. It also spans genres such as combat role playing to purely social.

Given that definition, there is a 40-year history to virtual worlds, although through many cycles of reinvention. In the 1970s, multi-player games on the PLATO system and Maze War used graphics to portray immersive worlds, while MUDs used rich textual descriptions to bring combat, social interaction and user creativity to the realm of formerly solitary text adventures - but only to those lucky enough to have access to such systems and dedicated networks. With the advent of microcomputers in the early 1980s, millions of users potentially had access to more powerful gateways into virtual worlds but few were had access to networks - and even then, only over dial-up connections, so evolution was slow. Initially, local bulletin boards offered text-based games like MUDs that could be experienced over slow dial-up connections, but eventually larger service providers offered graphical games such as Air Warrior (3D flight simulation combat) and Lucasfilm's Habitat (a 2D social world with persistent spaces).

Towards the end of the decade, what are now called Massively Multiplayer Online Roleplaying Games (MMORPGs or simply MMOs) emerged, with entrants such as Ultima Online supporting thousands of simultaneous users. Meanwhile, military simulations over private networks for training and planning purposes continued to advance. In the early 1990s, with the start of widespread adoption of the Internet, purely social worlds such as The Palace appeared allowing 2D user created content. Initiatives such as VRML hoped to achieve the same success for distributed creation and sharing of 3D content as HTML had done for text and 2D, but the technology pushed contemporary computers past their limits and without shared destinations (despite efforts by Blaxxun and others), 3D on the desktop was limited to immersive spaces with only canned content such Quake. However, the stage was set for the next generation of virtual worlds.

Around the turn of the millennium, home computers were powerful enough that MMOs like EverQuest and World of Warcraft arose alongside social worlds such as Active Worlds, There, and Second Life. The characteristics of this generation of services in many ways defines how virtual worlds are portrayed by the media and understood by popular culture today: a fully immersive 3D world rendered by a stand-alone client application, portraying realistic humanoid avatars moving and interacting in shared spaces with extensive customizations and user-generated content and locations, used for ad-hoc social interaction and organized events including games and music, and a micro-economy with user to user transactions.

However, virtual worlds are not limited to that model. Many virtual worlds aimed at children (such as Club Penguin or Webkinz) or casual game play (such as Metaplace)rely on cartoon-style 2D or isomorphic 3D visualization and take advantage of widely deployed technologies such as Flash in Web browsers to avoid an initial download and keep interaction simple. Services such as IMVU add a sense of location to instant messaging and allow purchase of content. The latest generation of network capable game consoles provide user avatars and shared social spaces such as PlayStation Home.

== The Second Life virtual world model ==

* <<<Intro to Linden Lab and the rise of SL --- I'm leaving the company bit to Josh. :P Morg.>>>

* <<<Description of the SL grid/simulator/client architecture and virtual space model>>>

At the outermost level, the Second Life architecture is comprised of a large number of region ''simulators'', each of which implements a 256m x 256m unit of virtual acreage. These simulators and their separate regions are organized into a ''grid'', which defines a 2D-tiled flat virtual world. A user's client application or ''viewer'' connects to this world, and the user is given an identity and a virtual presence at a single location in one specific region. Viewer controls then allow free movement within the current region, transparent crossings into adjacent regions, and teleport to remote regions of this virtual world.

* <<<Description of the SL data model --- it's obvious to us, but not to everyone!>>>

Grids, simulators and clients are the macroscale entities of the Second Life platform, but this architectural view says little about virtual worlds as an immersive experience. That experience is provided by the visible and audible components of the 3D world as displayed by the SL client application, which renders the 3D graphics objects as a 2D projection for the screen and delivers spatialized audio to the user's sound system.

In order for the client to do this, those world components are ''streamed'' from the simulators to each client whenever they are in visualization range, and cached by the client for future use. This data model is a fixed point of the world architecture of Second Life. The fact that user generated content is copied to clients ''by design'' has fundamental implications for non-technical issues such as object ownership and copyright which often arise.

The main components of the virtual world which together provide the immersive experience are ''avatars'' which are the visible instantiation of a logged-in user, ''assets'' such as textures, animations and sound files, and ''objects'' which are user-built assemblies constructed out of parametrized geometric primitives, as well as the data which defines land geometry and visuals. The client-side rendering of these miscellaneous components creates a ''mashup'' of these visual elements on the screen, and if done well, the end result is a highly believable and hence ''immersive'' representation of the user's avatar within an imagined world.

It is worth noting that the concept of ''mashup'' provides a useful alternative way of thinking about interop. Virtual worlds interoperate whenever the mashup displayed by a client contains components from more than one virtual world. This is an effective definition which avoids the pitfalls of trying to define interop architecturally amid the many and varied VW architectures in use today.

== Opensim and open source virtual worlds ==

* <<<libsecondlife and reverse engineering of SL protocols>>>

* <<<Founding of Opensim project and goals>>>

* <<<OSgrid and other Opensim-based grids>>>

* <<<Interop mechanisms available in Opensim>>>

* <<<User expectations for the SL/Opensym ecosystem>>>

== Defining the VW interop problem space ==

* <<<LL+IBM, founding of the AWG, and scary numbers>>>

* <<<The OGP agent domain + region domain model>>>

* <<<One-sentence description of capabilities>>>

* <<<The OGP interop trial of Summer 2008>>>

* <<<Taking OGP to the IETF --- MMOX and problem space overload>>>

* <<<Narrowing the VW interop problem space: OGPX>>>

== VWRAP: a services model of VW interop ==

* <<<Extending OGP to multiple VWs through policy-defining RDs == OGPX>>> (acceptable simplification?)

* <<<A successful OGPX BoF, and the foundation of VWRAP workgroup>>>

* <<<Brief description of expected VWRAP services>>>

* <<<High level overview of protocol entities>>>

== VWRAP: the work ahead ==
* <<<Goals, rough timeline, and open invitation>>>

IEEE Internet Computing: VWRAP for Virtual Worlds interop

2009-10-30T20:00:47Z

Joshua Linden: /* A brief history of virtual worlds */

''' ''Tentative skeletal pre-draft of a possible article.'' ''' --- '''Remember the wiki motto, "Be Bold."''' :-)

== Introduction ==

* <<<Set the scene with an epic literary start>>

Mankind is an imaginative, storytelling species. From our first glimmers of intelligence in the wielding of a bone as a weapon, through to today's visions of our future among the stars, we have always imagined new worlds that are different to those of yesterday. We have continually redefined our physical world through progress, created innumerable fantasy worlds through sheer imagination, and told stories about past worlds and future worlds to keep our culture and visions alive. Some new worlds we have even reached physically by departing from our little planet of origin into the darkness of space. We're really not happy with just a single world.

Computing is a microcosm of everything else that Mankind does, so it's no surprise that our worlds of the imagination have become part of this new space as well, indeed an essential part. We call our computer-driven worlds ''virtual worlds'', although the adjective is oddly superfluous. As has been the case with all our worlds throughout history, these CPU-assisted worlds gain their ''immersive'' reality as worlds entirely within our minds.

This article is about a small corner of the huge space of virtual worlds, a corner in which one large established virtual world and a constellation of small newer ones share sufficient commonality that a desire for interoperability between them has taken root, under the auspices of the IETF.

== A brief history of virtual worlds ==

* <<<The only thing I can bring to this section is a small acquaintance since 1996 with what became ActiveWorlds, a decade of excessive online gaming in 3 major MMOs, and 5 years of Second Life + a little Opensim. I don't consider this a usable background for writing even a brief history of VWs. Morg>>>

Computer-based virtual worlds can be defined as having representations of multiple users, often called characters or avatars, and locations which those user-representations occupy. While virtual environments can be single user (with or without an explicit user representation), we restrict our focus to multi-user systems which generally implies a network, though not necessarily any particular topology or capability of the nodes. Such a definition spans the gamut of visualization options for the world - from text through 2D to fully immersive 3D graphics. It also spans genres such as combat role playing to purely social.

Given that definition, there is a 40-year history to virtual worlds, although through many cycles of reinvention. In the 1970s, multi-player games on the PLATO system and Maze War used graphics to portray immersive worlds, while MUDs used rich textual descriptions to bring combat, social interaction and user creativity to the realm of formerly solitary text adventures - but only to those lucky enough to have access to such systems and dedicated networks. With the advent of microcomputers in the early 1980s, millions of users potentially had access to more powerful gateways into virtual worlds but few were had access to networks - and even then, only over dial-up connections, so evolution was slow. Initially, local bulletin boards offered text-based games like MUDs that could be experienced over slow dial-up connections, but eventually larger service providers offered graphical games such as Air Warrior (3D flight simulation combat) and Lucasfilm's Habitat (a 2D social world with persistent spaces).

Towards the end of the decade, what are now called Massively Multiplayer Online Roleplaying Games (MMORPGs or simply MMOs) emerged, with entrants such as Ultima Online supporting thousands of simultaneous users. Meanwhile, military simulations over private networks for training and planning purposes continued to advance. In the early 1990s, with the start of widespread adoption of the Internet, purely social worlds such as The Palace appeared allowing graphical user created content. Initiatives such as VRML hoped to achieve the same success for distributed 3D content as HTML had done for text and 2D content, but failed to garner widespread use.

ActiveWorlds, MMOs, Second Life, IMVU, Club Penguin, Sony Home, ...

== The Second Life virtual world model ==

* <<<Intro to Linden Lab and the rise of SL --- I'm leaving the company bit to Josh. :P Morg.>>>

* <<<Description of the SL grid/simulator/client architecture and virtual space model>>>

At the outermost level, the Second Life architecture is comprised of a large number of region ''simulators'', each of which implements a 256m x 256m unit of virtual acreage. These simulators and their separate regions are organized into a ''grid'', which defines a 2D-tiled flat virtual world. A user's client application or ''viewer'' connects to this world, and the user is given an identity and a virtual presence at a single location in one specific region. Viewer controls then allow free movement within the current region, transparent crossings into adjacent regions, and teleport to remote regions of this virtual world.

* <<<Description of the SL data model --- it's obvious to us, but not to everyone!>>>

Grids, simulators and clients are the macroscale entities of the Second Life platform, but this architectural view says little about virtual worlds as an immersive experience. That experience is provided by the visible and audible components of the 3D world as displayed by the SL client application, which renders the 3D graphics objects as a 2D projection for the screen and delivers spatialized audio to the user's sound system.

In order for the client to do this, those world components are ''streamed'' from the simulators to each client whenever they are in visualization range, and cached by the client for future use. This data model is a fixed point of the world architecture of Second Life. The fact that user generated content is copied to clients ''by design'' has fundamental implications for non-technical issues such as object ownership and copyright which often arise.

The main components of the virtual world which together provide the immersive experience are ''avatars'' which are the visible instantiation of a logged-in user, ''assets'' such as textures, animations and sound files, and ''objects'' which are user-built assemblies constructed out of parametrized geometric primitives, as well as the data which defines land geometry and visuals. The client-side rendering of these miscellaneous components creates a ''mashup'' of these visual elements on the screen, and if done well, the end result is a highly believable and hence ''immersive'' representation of the user's avatar within an imagined world.

It is worth noting that the concept of ''mashup'' provides a useful alternative way of thinking about interop. Virtual worlds interoperate whenever the mashup displayed by a client contains components from more than one virtual world. This is an effective definition which avoids the pitfalls of trying to define interop architecturally amid the many and varied VW architectures in use today.

== Opensim and open source virtual worlds ==

* <<<libsecondlife and reverse engineering of SL protocols>>>

* <<<Founding of Opensim project and goals>>>

* <<<OSgrid and other Opensim-based grids>>>

* <<<Interop mechanisms available in Opensim>>>

* <<<User expectations for the SL/Opensym ecosystem>>>

== Defining the VW interop problem space ==

* <<<LL+IBM, founding of the AWG, and scary numbers>>>

* <<<The OGP agent domain + region domain model>>>

* <<<One-sentence description of capabilities>>>

* <<<The OGP interop trial of Summer 2008>>>

* <<<Taking OGP to the IETF --- MMOX and problem space overload>>>

* <<<Narrowing the VW interop problem space: OGPX>>>

== VWRAP: a services model of VW interop ==

* <<<Extending OGP to multiple VWs through policy-defining RDs == OGPX>>> (acceptable simplification?)

* <<<A successful OGPX BoF, and the foundation of VWRAP workgroup>>>

* <<<Brief description of expected VWRAP services>>>

* <<<High level overview of protocol entities>>>

== VWRAP: the work ahead ==
* <<<Goals, rough timeline, and open invitation>>>

IEEE Internet Computing: VWRAP for Virtual Worlds interop

2009-10-30T19:23:45Z

Joshua Linden: /* A brief history of virtual worlds */

''' ''Tentative skeletal pre-draft of a possible article.'' ''' --- '''Remember the wiki motto, "Be Bold."''' :-)

== Introduction ==

* <<<Set the scene with an epic literary start>>

Mankind is an imaginative, storytelling species. From our first glimmers of intelligence in the wielding of a bone as a weapon, through to today's visions of our future among the stars, we have always imagined new worlds that are different to those of yesterday. We have continually redefined our physical world through progress, created innumerable fantasy worlds through sheer imagination, and told stories about past worlds and future worlds to keep our culture and visions alive. Some new worlds we have even reached physically by departing from our little planet of origin into the darkness of space. We're really not happy with just a single world.

Computing is a microcosm of everything else that Mankind does, so it's no surprise that our worlds of the imagination have become part of this new space as well, indeed an essential part. We call our computer-driven worlds ''virtual worlds'', although the adjective is oddly superfluous. As has been the case with all our worlds throughout history, these CPU-assisted worlds gain their ''immersive'' reality as worlds entirely within our minds.

This article is about a small corner of the huge space of virtual worlds, a corner in which one large established virtual world and a constellation of small newer ones share sufficient commonality that a desire for interoperability between them has taken root, under the auspices of the IETF.

== A brief history of virtual worlds ==

* <<<The only thing I can bring to this section is a small acquaintance since 1996 with what became ActiveWorlds, a decade of excessive online gaming in 3 major MMOs, and 5 years of Second Life + a little Opensim. I don't consider this a usable background for writing even a brief history of VWs. Morg>>>

Computer-based virtual worlds can be defined as having representations of users (often called characters or avatars) and locations which those user-representations occupy. While that definition precludes some single-user virtual environments (such as the Colossal Cave Adventure or Myst) since a first-person aspect does not necessitate an embodiment of the user, we restrict our focus to multi-user systems (which generally implies a network, though not necessarily any particular topology or capacity in the nodes). Such a definition spans the visualization of the world - from text through 2D to fully immersive 3D graphics. It also spans genres such as combat role playing to purely social.

Given that definition, there is a 40-year history to virtual worlds, although through many cycles of reinvention. Multi-player games on the PLATO system and Maze War used graphics to portray simple worlds to those lucky enough to have access to such systems in the early 1970s, while MUDs used rich textual descriptions to bring combat, social interaction and user creativity to the realm of formerly solitary text adventures. With the advent of microcomputers, millions of users had access to more powerful gateways into virtual worlds but the networks were nascent. Initially, local bulletin boards offered text-based games like MUDs that could be experienced over slow dial-up connections, but eventually larger service providers offered graphical games such as Air Warrior (3D flight simulation combat) and Lucasfilm's Habitat (a 2D social world with persistent spaces).

Habitat, military sims, the Palace, VRML, ActiveWorlds, MMOs, Second Life, IMVU, Club Penguin, Sony Home, ...

== The Second Life virtual world model ==

* <<<Intro to Linden Lab and the rise of SL --- I'm leaving the company bit to Josh. :P Morg.>>>

* <<<Description of the SL grid/simulator/client architecture and virtual space model>>>

At the outermost level, the Second Life architecture is comprised of a large number of region ''simulators'', each of which implements a 256m x 256m unit of virtual acreage. These simulators and their separate regions are organized into a ''grid'', which defines a 2D-tiled flat virtual world. A user's client application or ''viewer'' connects to this world, and the user is given an identity and a virtual presence at a single location in one specific region. Viewer controls then allow free movement within the current region, transparent crossings into adjacent regions, and teleport to remote regions of this virtual world.

* <<<Description of the SL data model --- it's obvious to us, but not to everyone!>>>

Grids, simulators and clients are the macroscale entities of the Second Life platform, but this architectural view says little about virtual worlds as an immersive experience. That experience is provided by the visible and audible components of the 3D world as displayed by the SL client application, which renders the 3D graphics objects as a 2D projection for the screen and delivers spatialized audio to the user's sound system.

In order for the client to do this, those world components are ''streamed'' from the simulators to each client whenever they are in visualization range, and cached by the client for future use. This data model is a fixed point of the world architecture of Second Life. The fact that user generated content is copied to clients ''by design'' has fundamental implications for non-technical issues such as object ownership and copyright which often arise.

The main components of the virtual world which together provide the immersive experience are ''avatars'' which are the visible instantiation of a logged-in user, ''assets'' such as textures, animations and sound files, and ''objects'' which are user-built assemblies constructed out of parametrized geometric primitives, as well as the data which defines land geometry and visuals. The client-side rendering of these miscellaneous components creates a ''mashup'' of these visual elements on the screen, and if done well, the end result is a highly believable and hence ''immersive'' representation of the user's avatar within an imagined world.

It is worth noting that the concept of ''mashup'' provides a useful alternative way of thinking about interop. Virtual worlds interoperate whenever the mashup displayed by a client contains components from more than one virtual world. This is an effective definition which avoids the pitfalls of trying to define interop architecturally amid the many and varied VW architectures in use today.

== Opensim and open source virtual worlds ==

* <<<libsecondlife and reverse engineering of SL protocols>>>

* <<<Founding of Opensim project and goals>>>

* <<<OSgrid and other Opensim-based grids>>>

* <<<Interop mechanisms available in Opensim>>>

* <<<User expectations for the SL/Opensym ecosystem>>>

== Defining the VW interop problem space ==

* <<<LL+IBM, founding of the AWG, and scary numbers>>>

* <<<The OGP agent domain + region domain model>>>

* <<<One-sentence description of capabilities>>>

* <<<The OGP interop trial of Summer 2008>>>

* <<<Taking OGP to the IETF --- MMOX and problem space overload>>>

* <<<Narrowing the VW interop problem space: OGPX>>>

== VWRAP: a services model of VW interop ==

* <<<Extending OGP to multiple VWs through policy-defining RDs == OGPX>>> (acceptable simplification?)

* <<<A successful OGPX BoF, and the foundation of VWRAP workgroup>>>

* <<<Brief description of expected VWRAP services>>>

* <<<High level overview of protocol entities>>>

== VWRAP: the work ahead ==
* <<<Goals, rough timeline, and open invitation>>>

PyOGP Client Library

2009-10-22T16:00:49Z

Joshua Linden: /* Multiple Agent Login */

{{PyOGP/navigation}}
== Intro ==

PyOGP is a young, open source, python client library which provides an interface to an agent (client) of a Second Life grid. The code aims to enable automated testing of Second Life grids, to encourage and enable exploration into the client/server relationship and related protocols of Second Life, and to make possible simple prototyping of various client applications.

Hosted on svn.secondlife.com, it does require a contributor's agreement for commit access, and currently has a few contributors from the Second Life open source community.

P.S. We'll likely be moving to hg relatively soon...

* mailing list: pyogp@lists.secondlife.com
* irc: irc://irc.freenode.com/#pyogp
* wiki: http://wiki.secondlife.com/wiki/Pyogp
* licensing: Apache 2.0 license, copyright Linden Lab

=== Goals ===

In the very near future, we can have tests available to be run as soon as a deploy is completed that exercise a simulator/grid in the same way we do a smoke test. We will use these as automated tests run at build time, post deploy validation, and regression testing of simulators and backend systems.

This provides early feedback on code quality. QA is then able to dive deeper in testing the changes specific to a branch.

Having this library available also allows us to test potential changes before we have finalized design and are ready to submit to QA. Not sure how something will play out? Try it, and test it with PyOGP....

=== A Brief History ===

PyOGP was originally created as a tool for testing [[OGP]] related changes to the Second Life grid. By the end of the summer in 2008, the '''pyogp.lib.base''' package provided a skeleton messaging system and was able to test Linden Lab's first implementation of an agent domain and the related changes in an agent's region handoff and teleport. As the development effort around OGP waned, we started to extend pyogp by adding higher level client related functionality. Recently, this functionality was split out into a separate python package ('''pyogp.lib.client'''), and sample scripts (and future apps) were moved into '''pyogp.apps'''.

== Architecture Overview==

=== Structure ===

PyOGP is comprised of three python packages. The library consists of pyogp.lib.base and pyogp.lib.client, while sample scripts, and in time applications, live in pyogp.apps.
:'''pyogp.lib.base''' - consists of basic networking, messaging systems (UDP and event queue) and capabilities, custom datatypes, and a low level message related event system
:'''pyogp.lib.client''' - consists of 'convenience' classes mapping methods and handlers to specific messages (e.g. Agent().say(msg) tells the client to send the ChatFromViewer message to the host region). Raises application oriented events based on processing of messages (these are currently sparsely implemented)
:'''pyogp.apps''' - sample scripts and works in progress, the scripts here generally illustrate simple usage of classes as related to in world interactions by an agent of a Second Life grid

=== Dependencies ===

==== Platform / Python version compatibility ====

PyOGP aims to be compatible across platforms, though there are known problems with various environments. We'll be focusing on ensuring better compatibility soon.

'''Known good configurations'''

:Windows XP + Python 2.5
:Mac + Python 2.5, 2.6
:Linux + Python 2.4, 2.5, 2.6 (Linden hosts fall into this group)

'''Known bad configurations'''
:Windows Vista + Python 2.6
:Windows 7 + Python 2.6

There have been challenges in ensuring compatibility between the various dependencies, largely due to greenlet, eventlet, and pyopenssl. Please report bugs on [http://jira.secondlife.com/browse/PYO pJira]

==== Python module dependencies ====

The packages that make up PyOGP have some dependencies on python modules not included in a standard install, or sometimes not available on an older Python distribution.

'''pyogp.lib.base dependencies:'''

<python> #from setup.py

install_requires=[
'setuptools',
# -*- Extra requirements: -*-
'uuid',
'elementtree',
'indra.base',
'WebOb',
'wsgiref',
'eventlet==0.8.14'
]</python>

'''pyogp.lib.client dependencies:'''

<python> install_requires=[
'setuptools',
# -*- Extra requirements: -*-
'pyogp.lib.base'
] </python>

'''pyogp.apps dependencies:'''

<python> install_requires=[
'setuptools',
'pyogp.lib.client'
]</python>

=== How to install ===

''Lindens can see internal documentation for more specific guidance. https://wiki.lindenlab.com/wiki/Pyogp#How_to_Install''

==== Standalone dev environment using buildout ====

Buildout is a type of Python development environment, organizing and configuring various components and their dependencies. On a desktop, one may checkout such an environment for working with PyOGP. One may optionally use a virtualenv Python environment to isolate the development code and it's runtime environment on one's host.

Dependencies: buildout takes care of everything, grabbing needed modules etc.

Wiki instructions: https://wiki.secondlife.com/wiki/Pyogp/Client_Lib/The_Development_Sandbox

==== Installing the PyOGP packages ====

Each of the PyOGP package may be installed to one's Python install or to a [http://pypi.python.org/pypi/virtualenv virtualenv]. '''Buyer beware''' if installing into your system's install: you'll want to be able to uninstall manually, as we haven't hooked up the uninstall. PyOGP is still coming up to speed with respect to distutils and pypi and the like, but it's relatively close now.

To install a package, simply run 'python setup.py install' in a package's root.

==== Referencing PyOGP packages via the PATH ====

Source code can be referenced directly if one simply ensures that a package, and it's dependencies, are available in the PYTHONPATH environmental variable.

== Current functional coverage ==

Anything not listed as covered is probably not yet covered.

pyogp.lib.base:
:* base udp messaging system (message.*)
:** UDP serialization/deserialization
:** message_template.msg parsing
:* base event queue messaging system (event_queue.EventQueueClient())
:* capabilities and their methods. Seed capabilities are a special case. (caps.Capability())
:* Message-based events (message.message_handler)

pyogp.lib.client:
:* agents (agent.Agent())
:** L$ balance request, friending, and walk/fly/sit/stand actions...
:* OGP agentdomain (agentdomain.AgentDomain())
:* application level events (event_system.AppEventsHandler())
:* some object handling (objects.*)
:** edit name, description, next-owner permissions and more
:** object creation is possible
:* some inventory handling (inventory.*)
:** login inv skeletons
:** fetching inventory, including AIS (caps based Agent Inventory Services))
:** some creating of new inventory items (LSL scripts, notecards)
:* regions (region.Region())
:** host and neighboring regions are handled slightly differently
:** udp and event queue connections are optionally enabled for each case
:** currently only pulling caps available to the agent via the seed cap (plus using the inventory related caps in the AIS context)
:* some appearance handling (appearance.AppearanceManager),
:* parcels
:* chat
:* some ImprovedInstantMessage handling
:** raises events on received instant messages
:** can deal with inventory offers/accepts/declines
:** other cases in this message are currently only logged
:* groups
:* group chat
:* LSL script uploading

=== Sample Scripts ===

There are a variety of scripted examples that have been used to exercise and test functionality as it is added to the library. These persist as coded documentation.

The source code is available in https://svn.secondlife.com/svn/linden/projects/2008/pyogp/pyogp.apps/trunk/pyogp/apps/examples/.

The following refers to a buildout context. If one installs pyogp.apps vi setup.py, these scripts will exist in the python environment's bin/ directory. In the buildout context, these scripts are available in {buildout root}/bin.

The scripts are derived from a package's 'setup.py' via the entry_points parameter, and essentially build executable Python scripts configured to run in the correct environment with the proper dependencies added the the path used by the script. These scripts are currently just simple illustrations of some uses of the PyOGP codebase.

<python> #the current entry_points in setup.py og pyogp.apps:

entry_points={
'console_scripts': [
'AIS_inventory_handling = pyogp.apps.examples.AIS_inventory_handling:main',
'agent_login = pyogp.apps.examples.agent_login:main',
'agent_manager = pyogp.apps.examples.agent_manager:main',
'appearance_management = pyogp.apps.examples.appearance_management:main',
'chat_and_instant_messaging = pyogp.apps.examples.chat_and_instant_messaging:main',
'group_chat = pyogp.apps.examples.group_chat:main',
'group_creation = pyogp.apps.examples.group_creation:main',
'inventory_handling = pyogp.apps.examples.inventory_handling:main',
'inventory_transfer = pyogp.apps.examples.inventory_transfer:main',
'inventory_transfer_specify_agent = pyogp.apps.examples.inventory_transfer_specify_agent:main',
'login = pyogp.apps.examples.login:main',
'multi_region_connect = pyogp.apps.examples.multi_region_connect:main',
'object_create_edit = pyogp.apps.examples.object_create_edit:main',
'object_create_permissions = pyogp.apps.examples.object_create_permissions:main',
'object_create_rez_script = pyogp.apps.examples.object_create_rez_script:main',
'object_creation = pyogp.apps.examples.object_creation:main',
'object_properties = pyogp.apps.examples.object_properties:main',
'object_tracking = pyogp.apps.examples.object_tracking:main',
'parcel_management = pyogp.apps.examples.parcel_management:main',
'parse_packets = pyogp.apps.examples.parse_packets:main',
'region_connect = pyogp.apps.examples.region_connect:main',
'smoke_test = pyogp.apps.examples.smoke_test:main',
'chat = pyogp.apps.examples.chat_interface:main',
]
}</python>

== How it Works (High Level) ==

=== Eventlet ===

PyOGP use [[Eventlet]] to run coroutines to handle multiple 'concurrent' processes, rather than threads or multiple processes. Each client agent instance will spawn a handful of coroutines to handles e.g. the UDP pipe, the Event Queue, various monitors, while yielding time to the parent process which should ensure it yields to the other routines as well.

PyOGP uses eventlet in very elementary ways at this point, but will perhaps start to use blocking queues in some cases, so that the coroutine only is allocated processing time if there is work for it to do.

=== pyogp.lib.base ===

This package handles the protocols used when communicating with a Second Life grid. A high level perspective on the package reveals a MessageManager() (still in development) which provides an interface to the UDP and Event Queue connections, as well as basic networking with enables login and capabilities interactions. The base package also has a low level even system through which all messages are passed and sent to subscribers.

Any subcomponent is available for direct interaction at any time, the MessageManager() and the MessageHandler() are the simple access points.

==== Events & Callbacks ====

The event implementation in pyogp follows the observer pattern, where observers subscribe to and are notified when an event occurs. Data is passed throughout the client instance via events.

* MessageManager - is an attribute of a Region and every packet received/sent is filtered through here. subscriptions are by message name
** MessageHandler - is an attribute of MessageManager, and every categorized message received from the event queue or udp dispatcher is filtered through here. (message as defined in message_template.msg, or one of ['ChatterBoxInvitation', 'ChatterBoxSessionEventReply', 'ChatterBoxSessionAgentListUpdates', 'ChatterBoxSessionStartReply', 'EstablishAgentCommunication']. There may be unhandled messages, I just haven't seen em yet :))

===== Message Events =====

In the most fundamental implementation of event usage, all packets are passed through a MessageManager() instance for evaluation. Observers may register to receive udp packets serialized into the form of Message() instances. The MessageHandler() is a consolidation point for subscribing to messages keyed by message name, and created on demand via subscription.

See pyogp.lib.base.message.message_handler.MessageHandler() for more details.

The pyogp agent's Region() instances each monitor their stream of packets (e.g. the host region: agent.region.message_manager.message_handler). (Perhaps this should be changed to a generalized Network() class where all packets (coupled to their originating regions) are evaluated.

Event firing passes data on to a callback handler defined in the subscription, in the form of (handler, *args, **kwargs).

The Agent class monitors the ImprovedInstantMessage packet:
<python> onImprovedInstantMessage_received = self.region.message_handler.register('ImprovedInstantMessage')
onImprovedInstantMessage_received.subscribe(self.onImprovedInstantMessage)

def onImprovedInstantMessage(self, packet):
""" handles the many cases of data being passed in this message """

{code} # parse and handle the data...</python>

The messaging system then fires the event when an ImprovedInstantMessage message is received, which calls onImprovedInstantMessage method above to handle the message contents. Multiple subscribers may be listening for any message related event, and each would be notified of the same Message() instance.

Unsubscribing from an event:
<python> onImprovedInstantMessage_received.unsubscribe(self.onImprovedInstantMessage)</python>

There are various event and callback implementations viewable in pyogp, poke around and help consolidate things if you like.

=== pyogp.lib.client ===

The client package generally provides a convenient interface to initiate or interpret interactions with host region (or neighboring regions). By listening to the messaging related event system in pyogp.lib.base, the client package interprets the messages that come in off the wire, and executes business logic in building responses. pyogp.lib.client also provides simple methods to enable the ending of messages to a grid.

==== Events ====

* EventsHandler - an attribute of an Agent, also able to be passed in, that is intended as the primary interface of a pyogp application into the internal state and data events within the lib. This system uses the same base classes as used by the MessageHandler() in the base package, and the descriptions about events and callbacks above apply here as well. The api for subscribing to these events is similar to the MessageHandler(), with an additional timeout parameter passed in the _register() method. When the specified timeout expires, the subscription returns None and expires the subscription.

==== Agent Login (examples) ====

===== Single Agent Login & Chat=====

Spawn a client in a co-routine, allowing persistent presence until forcefully terminated.

<python>from eventlet import api

from pyogp.lib.client.agent import Agent
from pyogp.lib.client.settings import Settings

settings = Settings()

settings.ENABLE_INVENTORY_MANAGEMENT = True
settings.MULTIPLE_SIM_CONNECTIONS = False

client = Agent(settings = settings)

api.spawn(client.login, options.loginuri, 'first', 'last', 'password', start_location = options.region)

# wait for the agent to connect to it's region
while client.connected == False:
api.sleep(0)

while client.region.connected == False:
api.sleep(0)

client.say("Hello World!")

# once connected, live until someone kills me
while client.running:
api.sleep(0)</python>

===== Multiple Agent Login =====

Each agent instance in logged in in a separate coroutine.

<python>from pyogp.lib.client.agent import Agent
from pyogp.lib.client.agentmanager import AgentManager

credentials= [('agent1', 'lastname', 'password'), ('agent2', 'lastname', 'password')]
agents = []

# prime the Agent instances
for firstname, lastname, password in credentials:
agents.append(Agent(settings, firstname, lastname, password))

agentmanager = AgentManager()
agentmanager.initialize(agents)

# log them in
for key in agentmanager.agents:
agentmanager.login(key, options.loginuri, options.region)

# while they are connected, stay alive
while agentmanager.has_agents_running():
api.sleep(0)</python>

== Extending Functionality ==

While the implementations and structures in pyogp.lib.base can (and are in the process of) being refactored to improve performance or usability, it is a fairly complete package.

The functional coverage PyOGP provides on the other hand is not complete, and there are a variety of needs to complete the implementation in pyogp.lib.base. We need to improve coverage of message handling (dealing with messages sent to the client), add more wrappers for sending various messages and performing multistep tasks (to simplify the initiation of interactions with the region), and we need to raise more application level events in the client package so that applications have easy access to incoming data.

=== Sending Messages ===

PyOGP, like the Viewer, communicates with the Second Life simulator by sending messages over UDP.

In order to extend PyOGP, you'll build a representation of a new UDP message, and send it through the pyogp.lib.base modules for serialization and wire handling.

=== Example: Sending an IM ===

To send an IM to the simulator, send an ImprovedInstantMessage packet. The base class for message packets is defined in <code>base/message/message.py</code>

Packets are assembled using a Message() instance which has the message name and Block() instances passed in through its constructor. Similarly, Blocks are assembled by passing in the Block name and the value name and values for each of the Block values. (The ability to build Message() instances via an llsd payload is expected to be introduced soon.)

Note: It is important that the message name, block name, and value names and types should match what is specified in the message template. (It is also possible to manipulate the representation of the stored template, or to use a custom message template.)

Example:
<python> def send_ImprovedInstantMessage(self, AgentID = None, SessionID = None,
FromGroup = None, ToAgentID = None,
ParentEstateID = None, RegionID = None,
Position = None, Offline = None,
Dialog = None, _ID = None, Timestamp = None,
FromAgentName = None, _Message = None,
BinaryBucket = None):
"""
sends an instant message packet to ToAgentID. this is a
multi-purpose message for inventory offer handling, im, group chat,
and more
"""

packet = Message('ImprovedInstantMessage',
Block('AgentData',
AgentID = AgentID,
SessionID = SessionID),
Block('MessageBlock',
FromGroup = FromGroup,
ToAgentID = ToAgentID,
ParentEstateID = ParentEstateID,
RegionID = RegionID,
Position = Position,
Offline = Offline,
Dialog = Dialog,
ID = UUID(str(_ID)),
Timestamp = Timestamp,
FromAgentName = FromAgentName,
Message = _Message,
BinaryBucket = BinaryBucket))

# Send the message:
self.region.enqueue_message(packet, True)</python>

=== Handling Incoming Messages and Raising an Event ===

To listen for when messages of a particular type are sent to the client instance, subscribe to the MessageHandler() on the host region's MessageManager(), like the example that follows:

<python> onImprovedInstantMessage_received = self.region.message_handler.register('ImprovedInstantMessage')
onImprovedInstantMessage_received.subscribe(self.onImprovedInstantMessage)</python>

When the event is fired upon receipt of the message matching the name, the specified callback handles the data passed along, and in this case raises an event notifying observers of the 'InstantMessageReceived' event in pyogp.lib.client of the important data.

<python> def onImprovedInstantMessage(self, packet):
""" callback handler for received ImprovedInstantMessage messages. much is passed in this message, and handling the data is only partially implemented """

Dialog = packet.blocks['MessageBlock'][0].get_variable('Dialog').data
FromAgentID = packet.blocks['AgentData'][0].get_variable('AgentID').data

if Dialog == ImprovedIMDialogue.InventoryOffered:

self.inventory.handle_inventory_offer(packet)

# ...
# some of the Dialogue types this message can contain are handled, we are showing 2
# ...

elif Dialog == ImprovedIMDialogue.FromAgent:

# ... code parses the data from the Message() instance ...

message = AppEvent('InstantMessageReceived', FromAgentID = FromAgentID, RegionID = RegionID, Position = Position, ID = ID, FromAgentName = FromAgentName, Message = _Message)

logger.info("Received instant message from %s: %s" % (FromAgentName, _Message))

self.events_handler.handle(message)</python>

== Logging ==

Uses python's standard logging module (http://docs.python.org/library/logging.html). The library defines logging events throughout, it is up to the application/script to determine the output.

Hooking logging into a new module:

<python>from logging import getLogger

# initialize logging
logger = getLogger('pyogp.lib.client.agent')

class Agent(object):
""" our agent class """

def __init__(self, params):

self.params = params

logger.debug("Initializing agent with params: %s" % (params))</python>

An application can then set up the logging output as follows (or any other way it pleases):

<python>console = logging.StreamHandler()
formatter = logging.Formatter('%(asctime)-30s%(name)-30s: %(levelname)-8s %(message)s')
console.setFormatter(formatter)
logging.getLogger('').addHandler(console)
logging.getLogger('').setLevel(logging.DEBUG)</python>

The output to console is then:

<bash>2009-04-21 22:08:58,681 pyogp.lib.base.agent : DEBUG agent with params: params</bash>

== Pyogp Unit Tests ==

See [[PyOGP_Package_Unittests]].

== Sphinx (api docs) ==

'''Api documentation is now available for PyOGP packages''' (well, not for apps yet, but someday....)!

In pyogp/docs in each of the pyogp.lib.base and pyogp.lib.client packages, one will find source, last revision, and build files for sphinx based documents. Output is available at {package root}/docs/html/index.html.

We plan on sharing the api documentation on the web soon, and will work to make simple build wrappers work on various platforms, though this is a low priority.

Ask Enus for updated documentation to be checked in, or, build a better refresh.py.

== Roadmap ==

See [[Pyogp/Roadmap]] for details, or ask in irc or on the mailing list. Here's what's up in the near term for pyogp:

* better '''packaging''' and '''platform compatibility'''
* more '''functional coverage''' of message. We are covering 32% of the messages the viewers sends or handles when received (though it's estimated at more like 50% of normal use cases.
* '''permissions system testing''' to save QA from 3-5 day regression passes on perms
* '''appearance''' - this will require enabling upload and download, plus baking. Anyone have some spare time? :)

[[Category: Pyogp_Client_Lib]]
[[Category: Pyogp]]
[[Category:Architecture Working Group]]
[[Category:Grid_Interoperability]]
[[Category:AW Groupies]]
[[Category:MMOX]]

PyOGP Client Library

2009-10-19T20:30:10Z

Joshua Linden: /* Intro */

{{PyOGP/navigation}}
== Intro ==

PyOGP is a young open source python client library which provides an interface to an agent (client) of a Second Life grid. Conceived as a mechanism for testing [[OGP]] grid changes, it carries on with a charter of enabling automated testing of Second Life grids, enabling exploration into the client/server relationship and related protocols of Second Life, and enabling prototyping of various client applications.

Hosted on svn.secondlife.com, it does require a contributor's agreement for commit access, and currently has a few contributors from the Second Life community.

P.S. We'll likely be moving to hg relatively soon...

* mailing list: pyogp@lists.secondlife.com
* irc: irc://irc.freenode.com/#pyogp
* wiki: http://wiki.secondlife.com/wiki/Pyogp
* licensing: Apache 2.0 license, copyright Linden Lab

=== Goals ===

In the very near future, we can have tests available to be run as soon as a deploy is completed that exercise a simulator/grid in the same way we do a smoke test. We will use these as automated tests run at build time, post deploy validation, and regression testing of simulators and backend systems.

This provides early feedback on code quality. QA is then able to dive deeper in testing the changes specific to a branch.

Having this library available also allows us to test potential changes before we have finalized design and are ready to submit to QA. Not sure how something will play out? Try it, and test it with PyOGP....

== Architecture Overview==

=== Structure ===

PyOGP is comprised of two python packages.
:'''pyogp.lib.base''' - consists of basic networking, messaging systems (UDP and event queue) and capabilities, custom datatypes, and a low level message related event system
:'''pyogp.lib.client''' - consists of 'convenience' classes mapping methods and handlers to specific messages (e.g. Agent().say(msg) tells the client to send the ChatFromViewer message to the host region). Raises application oriented events based on processing of messages (these are currently sparsely implemented)
:'''pyogp.apps''' - sample scripts and works in progress, the scripts here generally illustrate simple usage of classes as related to in world interactions by an agent of a Second Life grid

=== Dependencies ===

The packages that make up PyOGP have some dependencies on python modules not included in a standard install, or sometimes not available on an older Python distribution.

PyOGP is known to work well in Python 2.4.x and 2.5.x, 2.6.x. Known issues are preventing successful use on Python 2.6 on Linux and Windows.

'''pyogp.lib.base dependencies:'''

<python> #from setup.py

install_requires=[
'setuptools',
# -*- Extra requirements: -*-
'uuid',
'elementtree',
'indra.base',
'WebOb',
'wsgiref',
'eventlet==0.8.14'
]</python>

'''pyogp.lib.client dependencies:'''

<python> install_requires=[
'setuptools',
# -*- Extra requirements: -*-
'pyogp.lib.base'
] </python>

'''pyogp.apps dependencies:'''

<python> install_requires=[
'setuptools',
'pyogp.lib.client'
]</python>

PyOGP aims to be compatible across platforms, though there are known problems with various environments. PyOGP runs well on Windows XP with Python 2.5.x, on Mac with Python 2.5.x, and on Linden hosts running Python 2.4.4. Problems have been observed on Windows Vista and 7 (due to Python 2.6 and or pyopenssl related problems). We'll be focusing on ensuring better compatibility soon.

=== How to install and use ===

==== Standalone dev environment using buildout ====

Buildout is a type of Python development environment, organizing and configuring various components and their dependencies. On a desktop, one may checkout such an environment for working with PyOGP. One may optionally use a virtualenv Python environment to isolate the development code and it's runtime environment on one's host.

Dependencies: buildout takes care of everything, grabbing needed modules etc.

Wiki instructions: https://wiki.secondlife.com/wiki/Pyogp/Client_Lib/The_Development_Sandbox

==== Installing the PyOGP packages ====

Each of the PyOGP package may be installed to one's Python install or to a [http://pypi.python.org/pypi/virtualenv virtualenv]. '''Buyer beware''' if installing into your system's install (you'll want to be able to uninstall manually, as we haven't hooked up the uninstall).

To install a package, simply download the package, cd into the root folder, and run 'python setup.py install' (or for virtual env {path to virtualenv python exectuable}/python setup.py install.

PyOGP is still coming up to speed with respect to distutils and pypi and the like, but it's relatively close now.

==== Referencing PyOGP packages via the PATH ====

Source code can be referenced directly if one simply ensures that a package, and it's dependencies, are available in the environmental variable Python references when searching it's PATH. If you choose to go this route, please use the proper environmental variable settings method for your platform, and include the appropriate directories in the PYTHONPATH variable.

=== Sample Scripts ===

There are a variety of scripted examples that have been used to exercise and test functionality as it is added to the library. These persist as coded documentation.

The source code is available in https://svn.secondlife.com/svn/linden/projects/2008/pyogp/pyogp.apps/trunk/pyogp/apps/examples/.

The following refers to a buildout context. If one installs pyogp.apps vi setup.py, these scripts will exist in the python environment's bin/ directory. In the buildout context, these scripts are available in {buildout root}/bin.

The scripts are derived from a package's 'setup.py' via the entry_points parameter, and essentially build executable Python scripts configured to run in the correct environment with the proper dependencies added the the path used by the script. These scripts are currently just simple illustrations of some uses of the PyOGP codebase.

<python> #the current entry_points in setup.py og pyogp.apps:

entry_points={
'console_scripts': [
'AIS_inventory_handling = pyogp.apps.examples.AIS_inventory_handling:main',
'agent_login = pyogp.apps.examples.agent_login:main',
'agent_manager = pyogp.apps.examples.agent_manager:main',
'appearance_management = pyogp.apps.examples.appearance_management:main',
'chat_and_instant_messaging = pyogp.apps.examples.chat_and_instant_messaging:main',
'group_chat = pyogp.apps.examples.group_chat:main',
'group_creation = pyogp.apps.examples.group_creation:main',
'inventory_handling = pyogp.apps.examples.inventory_handling:main',
'inventory_transfer = pyogp.apps.examples.inventory_transfer:main',
'inventory_transfer_specify_agent = pyogp.apps.examples.inventory_transfer_specify_agent:main',
'login = pyogp.apps.examples.login:main',
'multi_region_connect = pyogp.apps.examples.multi_region_connect:main',
'object_create_edit = pyogp.apps.examples.object_create_edit:main',
'object_create_permissions = pyogp.apps.examples.object_create_permissions:main',
'object_create_rez_script = pyogp.apps.examples.object_create_rez_script:main',
'object_creation = pyogp.apps.examples.object_creation:main',
'object_properties = pyogp.apps.examples.object_properties:main',
'object_tracking = pyogp.apps.examples.object_tracking:main',
'parcel_management = pyogp.apps.examples.parcel_management:main',
'parse_packets = pyogp.apps.examples.parse_packets:main',
'region_connect = pyogp.apps.examples.region_connect:main',
'smoke_test = pyogp.apps.examples.smoke_test:main',
'chat = pyogp.apps.examples.chat_interface:main',
]
}</python>

== Current functional coverage ==

Anything not listed as covered is probably not yet covered.

pyogp.lib.base:
:* base udp messaging system (message.*)
:** UDP serialization/deserialization
:** message_template.msg parsing
:* base event queue messaging system (event_queue.EventQueueClient())
:* capabilities and their methods. Seed capabilities are a special case. (caps.Capability())
:* Message-based events (message.message_handler)

pyogp.lib.client:
:* agents (agent.Agent())
:** L$ balance request, friending, and walk/fly/sit/stand actions...
:* OGP agentdomain (agentdomain.AgentDomain())
:* application level events (event_system.AppEventsHandler())
:* some object handling (objects.*)
:** edit name, description, next-owner permissions and more
:** object creation is possible
:* some inventory handling (inventory.*)
:** login inv skeletons
:** fetching inventory, including AIS (caps based Agent Inventory Services))
:** some creating of new inventory items (LSL scripts, notecards)
:* regions (region.Region())
:** host and neighboring regions are handled slightly differently
:** udp and event queue connections are optionally enabled for each case
:** currently only pulling caps available to the agent via the seed cap (plus using the inventory related caps in the AIS context)
:* some appearance handling (appearance.AppearanceManager),
:* parcels
:* chat
:* some ImprovedInstantMessage handling
:** raises events on received instant messages
:** can deal with inventory offers/accepts/declines
:** other cases in this message are currently only logged
:* groups
:* group chat
:* LSL script uploading

== How it Works (High Level) ==

=== Eventlet ===

PyOGP use [[Eventlet]] to run coroutines to handle multiple 'concurrent' processes, rather than threads or multiple processes. Each client agent instance will spawn a handful of coroutines to handles e.g. the UDP pipe, the Event Queue, various monitors, while yielding time to the parent process which should ensure it yields to the other routines as well.

PyOGP uses eventlet in very elementary ways at this point, but will perhaps start to use blocking queues in some cases, so that the coroutine only is allocated processing time if there is work for it to do.

=== pyogp.lib.base ===

This package handles the protocols used when communicating with a Second Life grid. A high level perspective on the package reveals a MessageManager() (still in development) which provides an interface to the UDP and Event Queue connections, as well as basic networking with enables login and capabilities interactions. The base package also has a low level even system through which all messages are passed and sent to subscribers.

Any subcomponent is available for direct interaction at any time, the MessageManager() and the MessageHandler() are the simple access points.

==== Events & Callbacks ====

The event implementation in pyogp follows the observer pattern, where observers subscribe to and are notified when an event occurs. Data is passed throughout the client instance via events.

* MessageManager - is an attribute of a Region and every packet received/sent is filtered through here. subscriptions are by message name
** MessageHandler - is an attribute of MessageManager, and every categorized message received from the event queue or udp dispatcher is filtered through here. (message as defined in message_template.msg, or one of ['ChatterBoxInvitation', 'ChatterBoxSessionEventReply', 'ChatterBoxSessionAgentListUpdates', 'ChatterBoxSessionStartReply', 'EstablishAgentCommunication']. There may be unhandled messages, I just haven't seen em yet :))

===== Message Events =====

In the most fundamental implementation of event usage, all packets are passed through a MessageManager() instance for evaluation. Observers may register to receive udp packets serialized into the form of Message() instances. The MessageHandler() is a consolidation point for subscribing to messages keyed by message name, and created on demand via subscription.

See pyogp.lib.base.message.message_handler.MessageHandler() for more details.

The pyogp agent's Region() instances each monitor their stream of packets (e.g. the host region: agent.region.message_manager.message_handler). (Perhaps this should be changed to a generalized Network() class where all packets (coupled to their originating regions) are evaluated.

Event firing passes data on to a callback handler defined in the subscription, in the form of (handler, *args, **kwargs).

The Agent class monitors the ImprovedInstantMessage packet:
<python> onImprovedInstantMessage_received = self.region.message_handler.register('ImprovedInstantMessage')
onImprovedInstantMessage_received.subscribe(self.onImprovedInstantMessage)

def onImprovedInstantMessage(self, packet):
""" handles the many cases of data being passed in this message """

{code} # parse and handle the data...</python>

The messaging system then fires the event when an ImprovedInstantMessage message is received, which calls onImprovedInstantMessage method above to handle the message contents. Multiple subscribers may be listening for any message related event, and each would be notified of the same Message() instance.

Unsubscribing from an event:
<python> onImprovedInstantMessage_received.unsubscribe(self.onImprovedInstantMessage)</python>

There are various event and callback implementations viewable in pyogp, poke around and help consolidate things if you like.

=== pyogp.lib.client ===

The client package generally provides a convenient interface to initiate or interpret interactions with host region (or neighboring regions). By listening to the messaging related event system in pyogp.lib.base, the client package interprets the messages that come in off the wire, and executes business logic in building responses. pyogp.lib.client also provides simple methods to enable the ending of messages to a grid.

==== Events ====

* EventsHandler - an attribute of an Agent, also able to be passed in, that is intended as the primary interface of a pyogp application into the internal state and data events within the lib. This system uses the same base classes as used by the MessageHandler() in the base package, and the descriptions about events and callbacks above apply here as well. The api for subscribing to these events is similar to the MessageHandler(), with an additional timeout parameter passed in the _register() method. When the specified timeout expires, the subscription returns None and expires the subscription.

==== Agent Login (examples) ====

===== Single Agent Login & Chat=====

Spawn a client in a co-routine, allowing persistent presence until forcefully terminated.

<python>from eventlet import api

from pyogp.lib.client.agent import Agent
from pyogp.lib.client.settings import Settings

settings = Settings()

settings.ENABLE_INVENTORY_MANAGEMENT = True
settings.MULTIPLE_SIM_CONNECTIONS = False

client = Agent(settings = settings)

api.spawn(client.login, options.loginuri, 'first', 'last', 'password', start_location = options.region)

# wait for the agent to connect to it's region
while client.connected == False:
api.sleep(0)

while client.region.connected == False:
api.sleep(0)

client.say("Hello World!")

# once connected, live until someone kills me
while client.running:
api.sleep(0)</python>

===== Multiple Agent Login =====

Each agent instance in logged in in a separate coroutine.

<python>from pyogp.lib.client.agent import Agent
from pyogp.lib.client.agentmanager import AgentManager

params = [['agent1', 'lastname', 'password'], ['agent2', 'lastname', 'password']]
agents = []

# prime the Agent instances
for params in clients:

agents.append(Agent(settings, params[0], params[1], params[2]))

agentmanager = AgentManager()
agentmanager.initialize(agents)

# log them in
for key in agentmanager.agents:
agentmanager.login(key, options.loginuri, options.region)

# while they are connected, stay alive
while agentmanager.has_agents_running():
api.sleep(0)</python>

== Extending Functionality ==

While the implementations and structures in pyogp.lib.base can (and are in the process of) being refactored to improve performance or usability, it is a fairly complete package.

The functional coverage PyOGP provides on the other hand is not complete, and there are a variety of needs to complete the implementation in pyogp.lib.base. We need to improve coverage of message handling (dealing with messages sent to the client), add more wrappers for sending various messages and performing multistep tasks (to simplify the initiation of interactions with the region), and we need to raise more application level events in the client package so that applications have easy access to incoming data.

=== Sending Messages ===

PyOGP, like the Viewer, communicates with the Second Life simulator by sending messages over UDP.

In order to extend PyOGP, you'll build a representation of a new UDP message, and send it through the pyogp.lib.base modules for serialization and wire handling.

=== Example: Sending an IM ===

To send an IM to the simulator, send an ImprovedInstantMessage packet. The base class for message packets is defined in <code>base/message/message.py</code>

Packets are assembled using a Message() instance which has the message name and Block() instances passed in through its constructor. Similarly, Blocks are assembled by passing in the Block name and the value name and values for each of the Block values. (The ability to build Message() instances via an llsd payload is expected to be introduced soon.)

Note: It is important that the message name, block name, and value names and types should match what is specified in the message template. (It is also possible to manipulate the representation of the stored template, or to use a custom message template.)

Example:
<python> def send_ImprovedInstantMessage(self, AgentID = None, SessionID = None,
FromGroup = None, ToAgentID = None,
ParentEstateID = None, RegionID = None,
Position = None, Offline = None,
Dialog = None, _ID = None, Timestamp = None,
FromAgentName = None, _Message = None,
BinaryBucket = None):
"""
sends an instant message packet to ToAgentID. this is a
multi-purpose message for inventory offer handling, im, group chat,
and more
"""

packet = Message('ImprovedInstantMessage',
Block('AgentData',
AgentID = AgentID,
SessionID = SessionID),
Block('MessageBlock',
FromGroup = FromGroup,
ToAgentID = ToAgentID,
ParentEstateID = ParentEstateID,
RegionID = RegionID,
Position = Position,
Offline = Offline,
Dialog = Dialog,
ID = UUID(str(_ID)),
Timestamp = Timestamp,
FromAgentName = FromAgentName,
Message = _Message,
BinaryBucket = BinaryBucket))

# Send the message:
self.region.enqueue_message(packet, True)</python>

=== Handling Incoming Messages and Raising an Event ===

To listen for when messages of a particular type are sent to the client instance, subscribe to the MessageHandler() on the host region's MessageManager(), like the example that follows:

<python> onImprovedInstantMessage_received = self.region.message_handler.register('ImprovedInstantMessage')
onImprovedInstantMessage_received.subscribe(self.onImprovedInstantMessage)</python>

When the event is fired upon receipt of the message matching the name, the specified callback handles the data passed along, and in this case raises an event notifying observers of the 'InstantMessageReceived' event in pyogp.lib.client of the important data.

<python> def onImprovedInstantMessage(self, packet):
""" callback handler for received ImprovedInstantMessage messages. much is passed in this message, and handling the data is only partially implemented """

Dialog = packet.blocks['MessageBlock'][0].get_variable('Dialog').data
FromAgentID = packet.blocks['AgentData'][0].get_variable('AgentID').data

if Dialog == ImprovedIMDialogue.InventoryOffered:

self.inventory.handle_inventory_offer(packet)

# ...
# some of the Dialogue types this message can contain are handled, we are showing 2
# ...

elif Dialog == ImprovedIMDialogue.FromAgent:

# ... code parses the data from the Message() instance ...

message = AppEvent('InstantMessageReceived', FromAgentID = FromAgentID, RegionID = RegionID, Position = Position, ID = ID, FromAgentName = FromAgentName, Message = _Message)

logger.info("Received instant message from %s: %s" % (FromAgentName, _Message))

self.events_handler.handle(message)</python>

== Logging ==

Uses python's standard logging module (http://docs.python.org/library/logging.html). The library defines logging events throughout, it is up to the application/script to determine the output.

Hooking logging into a new module:

<python>from logging import getLogger

# initialize logging
logger = getLogger('pyogp.lib.base.agent')

class Agent(object):
""" our agent class """

def __init__(self, params):

self.params = params

logger.debug("Initializing agent with params: %s" % (params))</python>

An application can then set up the logging output as follows (or any other way it pleases):

<python>console = logging.StreamHandler()
formatter = logging.Formatter('%(asctime)-30s%(name)-30s: %(levelname)-8s %(message)s')
console.setFormatter(formatter)
logging.getLogger('').addHandler(console)
logging.getLogger('').setLevel(logging.DEBUG)</python>

The output to console is then:

<bash>2009-04-21 22:08:58,681 pyogp.lib.base.agent : DEBUG agent with params: params</bash>

== Pyogp Unit Tests ==

See [[PyOGP_Package_Unittests]].

== Sphinx (api docs) ==

'''Api documentation is now available for PyOGP packages''' (well, not for apps yet, but someday....)!

In pyogp/docs in each of the pyogp.lib.base and pyogp.lib.client packages, one will find source, last revision, and build files for sphinx based documents. Output is available at {package root}/docs/html/index.html.

We plan on sharing the api documentation on the web soon, and will work to make simple build wrappers work on various platforms, though this is a low priority.

Ask Enus for updated documentation to be checked in, or, build a better refresh.py.

== Roadmap ==

There is so much yet to implement that it is frightening. Here's what's up in the near term for pyogp:

* better '''packaging''' and '''platform compatibility'''
* more '''functional coverage''' of message. We are covering 32% of the messages the viewers sends or handles when received (though it's estimated at more like 50% of normal use cases.
* '''permissions system testing''' to save QA from 3-5 day regression passes on perms
* '''appearance''' - this will require enabling upload and download, plus baking. Anyone have some spare time? :)

[[Category: Pyogp_Client_Lib]]
[[Category: Pyogp]]
[[Category:Architecture Working Group]]
[[Category:Grid_Interoperability]]
[[Category:AW Groupies]]
[[Category:MMOX]]

Pyogp/Roadmap

2009-10-13T20:29:28Z

Joshua Linden: Added voice sample notes

== Meta Issues ==

# Update and maintain the published docs at:
## [http://wiki.secondlife.com/wiki/PyOGP_Client_Library pWiki Pyogp Tech Overview]
# we need to spend time improving unit test coverage very soon
# we need to use pylint to help clean up pyogp
# we need to come up with pyogp specific coding guidelines
## wiki page indicating style, naming conventions, docstring standards, and examples of adding code (in the lib context)

== Functional Changes ==

# Raise an application event when disconnected
# Messaging changes
## add a MessagingManager, refactor UDPDispatcher and EventQueueClient to purely parse/pack data, and handle messaging concerns in this new class
### incorporate message.xml handling
## remove packets.py and refactor accordingly
## <s>Merge UPDPacket and Message</s> (Done)
## <s>merge packet_handler and event_queue_handler</s> (Done)
## <s>fix http://jira.secondlife.com/browse/PYO-62 (kotler)</s> (Done)
# <s>update the event handler</s> (Done)
## <s>create a generic event message class with name and payload</s> (Done)
## add convenience methods (e.g. to_llsd(), to_dict())
# PyOGP Application support
## adding application level events throughout
# Functional Additions
## teleport
##* <s>teleport to region/x/y/z</s> (Done 2009-06-15)
##* <s>teleport to landmark</s> (Done 2009-09-01)
##* receive teleport request
##* accept teleport request
##* accept god teleport request
##* <s>teleport status</s> (Done 2009-06-15)
## L$
##* <s>Send money</s> (Done 2009-06-17)
##* <s>money balance updates</s> (Done 2009-06-17)
## Actions
##* <s>Sit on ground</s> (Done 2009-07-07)
##* Sit on target
##* <s>Stand</s> (Done 2009-07-07)
##* Gestures
## Friends
##* <s>List of friends</s> (Done 2009-06-15)
##* <s>offline/online notifications</s> (Done 2009-06-15)
##* send/accept friend invite
##* de-friending
## Parcels
##* parcel update events (media)
## Voice
##* <s>Voice URI (Can be done using caps; should we bother with it outside a sample?)</s> (Sample added 2009-10-13)
##* <s>Voice credentials</s> (Sample added 2009-10-13)
## Object Tracking
##* Avatars (name cache)
## Appearance management (e.g. avoid having to bake locally)
##* <s>Initialization of appearance manager and required messages</s> (Done)
##* <s>Requesting cached Texture ids</s>(Done)
##* <s>VisualParams classes</s> (Done)
##* <s>AgentSetAppearance</s> (Done)
##* <s>Fetch own avatar visualParams from wearables</s> (Done)
##* Cases where sim does not have cached TextureEntry(ObjectUpdate) or TextureIDs (AgentCachedTextureResponse)
##* Attachments
##* Animations
## a thousand other things

=== Apps to develop using pyogp ===

# create a viewer proxy using pyogp
# create something akin to libomv's TestClient

Pyogp/Roadmap

2009-10-09T22:16:25Z

Joshua Linden: /* Functional Changes */

== Meta Issues ==

# Update and maintain the published docs at:
## [http://wiki.secondlife.com/wiki/PyOGP_Client_Library pWiki Pyogp Tech Overview]
# we need to spend time improving unit test coverage very soon
# we need to use pylint to help clean up pyogp
# we need to come up with pyogp specific coding guidelines
## wiki page indicating style, naming conventions, docstring standards, and examples of adding code (in the lib context)

== Functional Changes ==

# Raise an application event when disconnected
# Messaging changes
## add a MessagingManager, refactor UDPDispatcher and EventQueueClient to purely parse/pack data, and handle messaging concerns in this new class
### incorporate message.xml handling
## remove packets.py and refactor accordingly
## <s>Merge UPDPacket and Message</s> (Done)
## <s>merge packet_handler and event_queue_handler</s> (Done)
## <s>fix http://jira.secondlife.com/browse/PYO-62 (kotler)</s> (Done)
# <s>update the event handler</s> (Done)
## <s>create a generic event message class with name and payload</s> (Done)
## add convenience methods (e.g. to_llsd(), to_dict())
# PyOGP Application support
## adding application level events throughout
# Functional Additions
## teleport
##* <s>teleport to region/x/y/z</s> (Done 2009-06-15)
##* <s>teleport to landmark</s> (Done 2009-09-01)
##* receive teleport request
##* accept teleport request
##* accept god teleport request
##* <s>teleport status</s> (Done 2009-06-15)
## L$
##* <s>Send money</s> (Done 2009-06-17)
##* <s>money balance updates</s> (Done 2009-06-17)
## Actions
##* <s>Sit on ground</s> (Done 2009-07-07)
##* Sit on target
##* <s>Stand</s> (Done 2009-07-07)
##* Gestures
## Friends
##* <s>List of friends</s> (Done 2009-06-15)
##* <s>offline/online notifications</s> (Done 2009-06-15)
##* send/accept friend invite
##* de-friending
## Parcels
##* parcel update events (media)
## Voice
##* Voice URI (Can be done using caps; should we bother with it outside a sample?)
##* Voice credentials
## Object Tracking
##* Avatars (name cache)
## Appearance management (e.g. avoid having to bake locally)
##* <s>Initialization of appearance manager and required messages</s> (Done)
##* <s>Requesting cached Texture ids</s>(Done)
##* <s>VisualParams classes</s> (Done)
##* <s>AgentSetAppearance</s> (Done)
##* <s>Fetch own avatar visualParams from wearables</s> (Done)
##* Cases where sim does not have cached TextureEntry(ObjectUpdate) or TextureIDs (AgentCachedTextureResponse)
##* Attachments
##* Animations
## a thousand other things

=== Apps to develop using pyogp ===

# create a viewer proxy using pyogp
# create something akin to libomv's TestClient

Avatar Appearance

2009-05-22T20:03:44Z

Joshua Linden: /* File Structure */

{{ProtocolNav}}
[[Category:Viewer Architecture]]
<br/>
For a good introduction to the elements of 3D character graphics, including meshes, morphs, bones, rendering, quaternions, and more see [http://www.gup.uni-linz.ac.at/~gk/Diplom/CART-Martin.pdf Martin Garstenauer's thesis].<br/><br/>

== Appearance Components ==
SL Avatar has "wearables" plus "attachments" for static appearance, and "animations".

=== Wearables ===
Wearables encompass both clothing and body parts – represent customizations of the basic avatar mesh via parameterized changes and textures.

Clothing can be taken off, but body parts cannot. Otherwise, they are similar.

Wearables are created by the viewer when appearance is edited and the results are saved; changed parts of appearance (clothing, bodyparts) are saved as new wearables which are uploaded as assets. There is no support for directly importing or exporting wearables.

=== Attachments ===
An "attachment" is an object ("linked set of primitives") which is rooted in the scene graph on an attachment point on an avatar. They are otherwise the same as other objects in SL, and can contain and execute scripts, inventory, maintain state, etc.

Attachments are heavily used for hair, some parts of clothing (dresses, cuffs) and accessories (jewelery, weapons). Non-humanoid avatars may use attachments to completely obscure the underlying humanoid avatar mesh.

Attachments are never serialized as files on the viewer, and there is no import/export support. Attachments are only transmitted to/from the viewer as ObjectUpdate messages which describe the constantly updating scene graph of the simulation. Attachments are created/modified using the "build" tools in the viewer; modifications are transmitted from the viewer to the simulator as editing messages. The simulator then broadcasts the changes to other users as ObjectUpdates. When an attachment is saved to inventory, the linked set of prims structure is serialized to a structure that includes script state.

=== Animations ===
Animations are Poser BVH files. Animations can be added to a scripted object to be played (e.g. a "poseball"), or incorporated into a gesture (a sequence of actions including text, sounds and animations invoked manually or triggered by a keyword). However, animations are often thought of as parts of avatars when they are controlled by an attachment to alter the avatar's default animations.

One common case is as an "Animation Override" script. These are used along with creative animations to deform the skeleton. Sometimes these override only some of the default animations (e.g. directed attention), sometimes everything is overridden. For example, "tinies" are created by folding the skeleton's limbs back on themselves. Non-humanoid avatars often use "meatballs" animations that compress the avatar to such an extent that it can be hidden inside an attachment.

Animations can be created in third party tools (e.g. Poser, QAvimator) and uploaded via File > Upload Animation (L$10). The animation is added to the inventory, and can then be incorporated into a script or gesture.

==Packets==

* [[AgentWearablesRequest]]
** This is where it starts. The viewer asks the simulator what it is wearing, and an AgentWearablesUpdate packet is returned.
* [[AgentWearablesUpdate]]
** A mapping of wearable types to asset IDs and item IDs is returned to the client. There are currently 13 different wearable types and all 13 will always be returned in this packet. If your avatar is not wearing a wearable type the asset ID and item ID for that type will be null IDs (all zeros). The next step is to request an asset transfer for all of the non-null asset IDs and download the wearable assets.
* [[AgentIsNowWearing]]
** Like the AgentWearablesUpdate packet but in the other direction. The viewer sends a list of the 13 wearable types and their associated item IDs, or null if nothing is being worn in that wearable slot. This is typically sent right before AgentSetAppearance.
* [[AgentSetAppearance]]
** Serves two purposes, to tell the server how our avatar mesh is deformed (with visual parameters) and what textures we are wearing. There are currently 218 VisualParam blocks sent with each AgentSetAppearance describing everything from the color of the avatar eyes to the gender slider. A TextureEntry is also sent that uses the same TextureEntry format objects use, but each "face" is hard-coded to describe a particular texture. This may or may not include baked textures for the five different baked layers (head, upper, lower, eyes, and skirt).
**The ParamValue fields do not have parameter IDs associated with them in the packet, as it's assumed all viewers have the same parameter map in the same sequence. The parameters are stored internally as a floating point integer, and are converted to a single byte by inputting the value along with minimum and maximum weights for that parameter to a conversion function.

* [[AvatarAppearance]]
* [[AgentCachedTexture]]
* [[AgentCachedTextureResponse]]

==Linden avatar definition file==

The parameters affecting avatar appearance are defined in an XML file
located in the executable at
: SecondLife\character\avatar_lad.xml
and in the source code at
: linden\indra\newview\character\avatar_lad.xml

With a few exceptions
: genepool.xml
: rouge_alpha.tga
: underpants_trial_female.tga
: underpants_trial_male.tga
: undershirt_trial_female.tga
: upperbodyfreckles_alpha.tga
all of the other files in the same directory are referenced by avatar_lad.xml. These include the mesh files (.llm), the texture files (.tga) and the avatar_skeleton.xml file (see below).

genepool.xml contains parameter sets for wearable types WT_SHAPE, WT_SKIN, WT_HAIR, and WT_EYES for several archetypal avatar definitions. You can see the current parameter set for your own avatar by invoking the debug menus (Ctrl-Alt-D) and selecting
Client->Character->Character Tests->Appearance To XML. This will create a file
: SecondLife\character\new archetype.xml
in the same format as genepool.xml.

The avatar definition file affects the appearances on your viewer only. Any change to the file will not affect how others see the avatars. You can set value_default="1" for "Express_Tongue_Out" to make the avatars on your viewer give you the raspberries, but no one else will notice.

The interesting thing is that this file defines parameters that can be used in animations.

===File structure===

'''<?xml version="1.0" encoding="US-ASCII" standalone="yes"?>'''

'''<linden_avatar version="1.0" wearable_definition_version="22">'''

: '''<skeleton file_name="avatar_skeleton.xml">'''
:: '''''<attachment_point'' id="1" group="6" name="Chest" joint="mChest"'''
:: position="0.15 0 -0.1" rotation="0 90 90" visible_in_first_person="true"
:: pie_slice="2" hud="true" max_attachment_offset="1.5" '''''/>'''''
:: '''''<param ... >'''''
::: '''<param_skeleton>'''
:::: '''''<bone'' name="mNeck" scale="0 0 .2"''' offset="0 0 .004" '''''/>'''''
::: '''</param_skeleton>'''
:: '''''</param>'''''
: '''</skeleton>'''

:Sets of
: '''<mesh type="hairMesh" lod="0" file_name="avatar_hair.llm"'''
: min_pixel_width="320" min_pixel_area="0.1" '''>'''
:: '''''<param ... >'''''
::: '''<param_morph />'''
:::or
::: '''<param_morph>'''
:::: '''''<volume_morph''''' name="HEAD" scale="-0.008 -0.006 0.015"
:::: pos="0.07 0 -0.07" '''''/>'''''
::: '''</param_morph>'''
:: '''''</param>'''''
: '''</mesh>'''
: ''<mesh'' '''type="hairMesh" lod="1" file_name="avatar_hair_1.llm"'''
: min_pixel_width="160" min_pixel_area="0.1" '''reference="avatar_hair.llm"'''''>''
: ''</mesh>''

: '''''<global_color'' name="skin_color"''>'''''
:: '''<param ... >'''
::: '''<param_color''' operation="blend" '''>'''
:::: '''''<value'' color="252, 215, 200, 255" ''/>'''''
::: '''</param_color>'''
:: '''</param>'''
: '''''</global_color>'''''

: '''''<layer_set'' body_region="head" width="512" height="512"'''
: clear_alpha="false" alpha_tga_file="head_alpha.tga" '''''>'''''

:: '''''<layer'' name="head bump definition"''' render_pass="bump"
:: global_color="hair_color" fixed_color="128,128,128,255"
:: write_all_channels="true" '''''>'''''

::: <texture '''local_texture="lower_jacket"''' local_texture_alpha_only="true" />
:::and if not fixed_color
::: <texture '''tga_file="bump_head_base.tga"''' file_is_mask="FALSE" />

::: ''<morph_mask'' morph_name="Displace_Loose_Upperbody" invert="FALSE" ''/>''

::: ''<param ... >''
:::: '''<param_alpha''' tga_file="bump_face_wrinkles.tga" skip_if_zero="true"
:::: multiply_blend="false" domain="0" '''/>'''
::::or
:::: '''<param_color''' operation="multiply" '''>'''
::::: '''''<value'' color="0, 0, 0, 0" ''/>'''''
:::: '''</param_color>'''
::: ''</param>''
:: '''''</layer>'''''
: '''''</layer_set>'''''

: '''<driver_parameters>'''
:: '''''<param ... >'''''
::: '''<param_driver>'''
:::: '''''<driven'' id="628"''' min1="0.1" max1="0.5" max2="1" min2="1" '''''/>'''''
::: '''</param_driver>'''
:: '''''</param>'''''
: '''</driver_parameters>'''

'''</linden_avatar>'''

where '''bold''' text indicates required tags or parameters, standard text
indicates a tag that can occur zero or one time, ''italics''
indicates that the tag can be repeated zero or more times, and '''''bold italics''''' indicates
that at least one is required but more are possible.

Note that the values shown for parameters here may not be consistent. They are shown only
to indicate the value type.

===Parameter detail===

<param ... > is

: '''<param id="32"''' group="1" name="Male_Skeleton"
: value_min="0" value_max="1" value_default="0"
: edit_group="shape_body" edit_group_order="1"
: label="Skeleton" label_min="Female" label_max="Male"
: wearable="shape" clothing_morph="true" sex="male" shared="1"
: show_simple="true" simple_percent_min="40" simple_percent_max="100"
: camera_distance="2.2" camera_elevation=".1" camera_angle="0" '''>'''

Parameters are parsed in LLViewerVisualParamInfo::parseXml and LLVisualParamInfo::parseXml and are used by LLViewerVisualParam and LLVisualParam respectively when LLVOAvatar::loadAvatar is called.

Fields parsed in LLVisualParamInfo are used to describe the meaning of the parameters.

;id -- mID
;name -- mName
:default ""
;group -- mGroup
:"0" is VISUAL_PARAM_GROUP_TWEAKABLE
:"1" is VISUAL_PARAM_GROUP_ANIMATABLE
:default "0"
;value_min -- mMinWeight
:default 0.
;value_max -- mMaxWeight
:default 1.
;value_default -- mDefaultWeight
:default 0.
;sex -- mSex
{|
| || || "both" || SEX_BOTH
|-
| || || "male" || SEX_MALE
|-
| || || "female" || SEX_FEMALE
|-
| || || default || SEX_BOTH
|}
;label -- mDisplayName
:default is name
;label_min -- mMinName
:default is "Less"
;label_max -- mMaxName
:default is "More"
;shared
:Used only by mesh params. Indicates that the param affects more than one mesh

Fields parsed in LLViewerVisualParamInfo describe the appearance of the entries on the Edit->Appearance editing panel.

;wearable -- mWearableType
{|
| || || "shape" || WT_SHAPE
|-
| || || "skin" || WT_SKIN
|-
| || || "hair" || WT_HAIR
|-
| || || "eyes" || WT_EYES
|-
| || || "shirt" || WT_SHIRT
|-
| || || "pants" || WT_PANTS
|-
| || || "shoes" || WT_SHOES
|-
| || || "socks" || WT_SOCKS
|-
| || || "jacket" || WT_JACKET
|-
| || || "gloves" || WT_GLOVES
|-
| || || "undershirt" || WT_UNDERSHIRT
|-
| || || "underpants" || WT_UNDERPANTS
|-
| || || "skirt" || WT_SKIRT
|-
| || || default || WT_INVALID
|}
;edit_group -- mEditGroup
:default ""
;edit_group_order -- mEditGroupDisplayOrder
:By default, value 0, parameters are displayed in the order in which they appear in the xml file. "edit_group_order" overriddes this.

Optional camera offsets from the current joint center. Used for generating "hints" (thumbnails).
;camera_distance -- mCamDist
:default 0.5
;camera_angle -- mCamAngle
:default 0.
;camera_elevation -- mCamElevation
:default 0.

;show_simple -- mShowSimple : value is ignored
:default FALSE
;simple_percent_min -- mSimpleMin : value is ignored
:default 0.
;simple_percent_max -- mSimpleMax : value is ignored
:default 100.

===Loading the file===

The file is parsed in LLVOAvatar::initClass() primarily by calls to LLVOAvatarInfo. Both classes are defined in indra\newview\llvoavatar.h(.cpp).
There is one method for each of the major XML nodes
:LLVOAvatarInfo::parseXmlSkeletonNode
:LLVOAvatarInfo::parseXmlMeshNodes
:LLVOAvatarInfo::parseXmlColorNodes
:LLVOAvatarInfo::parseXmlLayerNodes
:LLVOAvatarInfo::parseXmlDriverNodes

This data is then used to create instances of LLVOAvatar in LLVOAvatar::loadAvatar()
which invokes LLVOAvatar::loadSkeletonNode() and LLVOAvatar::loadMeshNodes().

==LLVisualParam==

The avatar visual parameters are divided into those that define the avatar's appearance and those that are used in animations. Theoretically, they all could be used for animations, but the animations would have to start and end at the settings defined for the avatar rather than at the default values.

The group field of the <param> tag identifies those that are directly set in Edit->Appearance and sent in [[AgentSetAppearance]] messages as VISUAL_PARAM_GROUP_TWEAKABLE. The rest are identified as VISUAL_PARAM_GROUP_ANIMATABLE, but this is a bit misleading because it includes all of the driven parameters defined in the <driver_parameters> stanza. These are set by the driver parameters which are themselves directly tweakable.

Those that are not directly or indirectly tweakable include the emotes
:* 300 Express_Closed_Mouth
:* 301 Express_Tongue_Out
:* 302 Express_Surprise_Emote
:* 303 Express_Wink_Emote
:* 304 Express_Embarrassed_Emote
:* 305 Express_Shrug_Emote
:* 306 Express_Kiss
:* 307 Express_Bored_Emote
:* 308 Express_Repulsed_Emote
:* 309 Express_Disdain
:* 310 Express_Afraid_Emote
:* 311 Express_Worry_Emote
:* 312 Express_Cry_Emote
:* 313 Express_Sad_Emote
:* 314 Express_Anger_Emote
:* 315 Express_Frown
:* 316 Express_Laugh_Emote
:* 317 Express_Toothsmile
:* 318 Express_Smile
:* 632 Express_Open_Mouth
the hand morphs
:* 101 Hands_Relaxed
:* 102 Hands_Point
:* 103 Hands_Fist
:* 666 Hands_Relaxed_L
:* 667 Hands_Point_L
:* 668 Hands_Fist_L
:* 669 Hands_Relaxed_R
:* 670 Hands_Point_R
:* 671 Hands_Fist_R
:* 672 Hands_Typing
:* 766 Hands_Salute_R
:* 791 Hands_Peace_R
:* 792 Hands_Spread_R
and blinking eyes
:* 58 Blink_Left
:* 59 Blink_Right

The remaining visual params include a few useful for animations
:* 51 Furrowed_Eyebrows
:* 53 Surprised_Eyebrows
:* 54 Worried_Eyebrows
:* 55 Frown_Mouth
:* 57 Smile_Mouth
and a couple that seem to have been abandoned
:* 41 Old
:* 128 Wrinkles

==Linden skeleton definition file==

The skeletal structure is defined in an XML file located in the executable at
: SecondLife\character\avatar_skeleton.xml
and in the source code at
: linden\indra\newview\character\avatar_skeleton.xml

It is referenced through the avatar definition file (see above) and can be
considered to be part of the avatar definition.

===File structure===

'''<?xml version="1.0" encoding="US-ASCII" standalone="yes"?>'''

'''<linden_skeleton version="1.0" num_bones="46" num_collision_volumes="19">'''

: '''<bone name="mPelvis" pos="0.000 0.000 1.067" rot="0.000000 0.000000 0.000000"
: '''scale="1.000 1.000 1.000" pivot="0.000000 0.000000 1.067015">'''
:: <collision_volume '''name="PELVIS" pos = "-0.01 0 -0.02"'''
:: '''rot="0.000000 8.00000 0.000000" scale="0.12 0.16 0.17"'''/>
:: ''<bone ...>''
:: ''</bone>''
: '''</bone>'''

'''</linden_skeleton>'''

where '''bold''' text indicates required tags or parameters, standard text
indicates a tag that can occur zero or one time, and ''italics''
indicates that the tag can be repeated zero or more times.

The <bone ...> tag above is to indicate that the structure is recursive.

The "46" for num_bones is actually a typo in the file. There are only 26 bones defined.

===Loading the file===

The file is parsed in LLVOAvatar::initClass() using LLVOAvatarSkeletonInfo::parseXml. Both classes are defined in indra\newview\llvoavatar.h(.cpp).

This data is then used to create instances of LLVOAvatar in LLVOAvatar::loadAvatar()
which invokes LLVOAvatar::buildSkeleton(). A special-purpose "screen" joint named "mScreen" is also created.

==Linden binary mesh file==

Meshes are defined in binary files located in the executable at
: SecondLife\character\*.llm
and in the source code at
: linden\indra\newview\character\*.llm

They are referenced through the avatar definition file (see above) and can be
considered to be part of the avatar definition.

The morphName in the binary mesh file matches the name field of the visual params with <param_morph> tags in the <mesh> stanzas. There are a few morphs defined in the mesh files that are not used by the avatar definition file
:* Blink_Both
:* Fingernails_Long
:* Wide_Lips

There are two types of mesh files: the base mesh definition and the level of detail (LOD) mesh definition, which always references a base mesh definition file. LOD meshes are used to simplify the rendering when the avatar is distant from the camera. There are several LOD meshes with progressively decreasing resolution.

The vertex information referenced by the LOD meshes is stored at the beginning of the base mesh vertex information.

===File structure===
====Base mesh====

:char[24] "Linden Binary Mesh 1.0"
:U8 hasWeights
:U8 hasDetailTexCoords
:LLVector3 position
:LLVector3 rotationAngles
:U8 rotationOrder (ignored, set to 0)
:LLVector3 scale
:U16 numVertices
:LLVector3[ numVertices ] baseCoords
:LLVector3[ numVertices ] baseNormals
:LLVector3[ numVertices ] baseBinormals
:LLVector2[ numVertices ] texCoords
:if (hasDetailTexCoords)
::LLVector2[ numVertices ] detailTexCoords
:if (hasWeights)
::F32[ numVertices ] weights
:U16 numFaces
:struct faces[ numFaces ]
:{
::S16[3] face
:}
:if ( hasWeights )
:{
::U16 numSkinJoints
::struct skinJoints[ numSkinJoints ]
::{
:::char[64] jointName
::}
:}
:struct morph []
:{
::char[64] morphName
::S32 numVertices
::struct vertices[ numVertices ]
::{
:::U32 vertexIndex
:::LLVector3 coord
:::LLVector3 normal
:::LLVector3 binormal
:::LLVector2 texCoord
::}
:}
:char[64] "End Morphs"
:S32 numRemaps (or EOF)
:struct vertexRemap[ numRemaps ]
:{
::S32 remapSource
::S32 remapDestination
:}

====Level Of Detail (LOD) mesh====

:char[24] "Linden Binary Mesh 1.0"
:U8 hasWeights
:U8 hasDetailTexCoords
:LLVector3 position
:LLVector3 rotationAngles
:U8 rotationOrder (ignored, set to 0)
:LLVector3 scale
:U16 numFaces
:struct faces[ numFaces ]
:{
::S16[3] face
:}

where LLVector2 and LLVector3 are 2 and 3 floats respectively, and all numbers are little endian. Integers are 8, 16, or 32 bits signed (S) or unsigned (U).

The LOD .llm files actually contain fields for (if hasWeights) numSkinJoints set to zero, char[64] "End Morphs", and numRemaps set to zero, but these are not read by the viewer.

Each weight actually contains two pieces of information. The number to the left of the decimal point is the index of the joint and also implicitly indexes to the following joint. The actual weight is to the right of the decimal point and interpolates between these two joints. The index is into an "expanded" list of joints, not just a linear array of the joints as defined in the skeleton file. In particular, any joint that has more than one child will be repeated in the list for each of its children.

===Loading the file===

This data is used to create instances of LLVOAvatar in LLVOAvatar::loadAvatar()
through LLVOAvatar::loadMeshNodes, LLPolyMesh::getMesh, LLPolyMeshSharedData::loadMesh, and LLPolyMorphData::loadBinary. These are found in indra\newview\llvoavatar.h(.cpp), indra\newview\llpolymesh.h(.cpp), and indra\newview\llpolymorph.h(.cpp).

== Linden wearable definition file ==

The format is used for client/server communication - wearable assets are downloaded to cache/uploaded on creation. No import/export ability currently exists, although wearables are manipulated as files both on the viewer and server.

=== Source References ===

* <code>newview/llwearable.h</code>
** <code>enum EWearableType</code>
* <code>newview/llwearable.cpp</code>
** <code>LLWearable::importFile</code> – parses assets when "worn" from inventory, after download to cache
** <code>LLWearable::exportFile</code> – called by <code>LLWearable::saveNewAsset</code> when created, does upload to asset system
* <code>newview/llvoavatardefines.h</code>
** <code>enum ETextureIndex</code>

=== File Structure ===

Details:
* The file is line-oriented. UNIX line endings (LF 0x0A) are expected.
* File encoding for strings is implicitly UTF-8. No byte-order mark is allowed.
* Whitespace is used as delimiters. Spaces (0x20) and tabs (0x09) are considered whitespace, and any number may be used.

The file consists of the following blocks, which must occur in the specified order:

* Format/Version
* Name
* Description
* Permissions
* Sale Info
* Wearable type
* Parameters
* Textures

'''Format/Version'''

The string <code>LLWearable version</code> followed by the version number (currently <code>22</code>)

Example:

LLWearable version 22

'''Name'''

A single line string, terminated with a UNIX newline (LF 0x0A). Maximum length is 2047 bytes, not counting the newline. Encoding is UTF-8. A blank line is used to indicate the absence of a name.

Example:

Space Cadet Shirt

'''Description'''

A single line string, terminated with a UNIX newline (LF 0x0A). Maximum length is 2047 bytes, not counting the newline. Encoding is UTF-8. A blank line is used to indicate the absence of a description.

Example:

This is my awesome shirt!

'''Permissions block'''

* First line: The token <code>permissions</code> followed by the permission block version <code>0</code>
* Second line: open brace <code>{</code>
* A sequence of permission name/value pairs, each on its own line
* Final line: close brace <code>}</code>

The permission name/value pairs are:

* <code>base_mask</code>, with 8-digit hexadecimal value
* <code>owner_mask</code>, with 8-digit hexadecimal value
* <code>group_mask</code>, with 8-digit hexadecimal value
* <code>everyone_mask</code>, with 8-digit hexadecimal value
* <code>next_owner_mask</code>, with 8-digit hexadecimal value
* <code>creator_id</code>, with agent UUID in hexadecimal 8-4-4-4-12 format
* <code>owner_id</code>, with agent UUID in hexadecimal 8-4-4-4-12 format
* <code>last_owner_id</code>, with agent UUID in hexadecimal 8-4-4-4-12 format
* <code>group_id</code>, with group UUID in hexadecimal 8-4-4-4-12 format
* <code>group_owned</code>, with <code>0</code> (false) or <code>1</code> (true) flag

If a name/value is missing, a default value is assumed. For maximum compatibility, everything should be explicitly specified except <code>group_owned</code> when false.

Flags are defined in <code>llinventory/llpermissionsflags.h</code>:
* <code>PERM_TRANSFER</code> = 0x00002000
* <code>PERM_MODIFY</code> = 0x00004000
* <code>PERM_COPY</code> = 0x00008000
* <code>PERM_MOVE</code> = 0x00080000

Common masks are:
* <code>PERM_NONE</code> 0x00000000
* <code>PERM_ALL</code> 0x7FFFFFFF

And common values are:
* no modify: <code>PERM_ALL & ~PERM_MODIFY</code> = 0x7fffbfff
* no copy: <code>PERM_ALL & ~PERM_COPY</code> = 0x7fff7fff
* no modify or copy: = 0x7fff3fff
* no transfer: <code>PERM_ALL & ~PERM_TRANSFER</code> = 0x7fffdfff
* no modify, no transfer = 0x7fff9fff

Example:

permissions 0
{
base_mask 7fffffff
owner_mask 7fffffff
group_mask 00000000
everyone_mask 7fffbfff
next_owner_mask 7fffffff
creator_id d929385f-41e3-4a34-a04e-f1fc39f24f12
owner_id ff9a71eb-7414-4bf8-866e-a70ddeb7c3cf
last_owner_id ff9a71eb-7414-4bf8-866e-a70ddeb7c3cf
group_id 00000000-0000-0000-0000-000000000000
}

'''Sale Info block'''

* First line: The token <code>sale_info</code> followed by the sale_info block version <code>0</code>
* Second line: open brace <code>{</code>
* A sequence of sale info name/value pairs, each on its own line
* Final line: close brace <code>}</code>

The sale info name/value pairs are:
* <code>sale_type</code>: one of:
** <code>not</code> - not for sale
** <code>orig</code> - the original is purchased
** <code>copy</code> - a copy is purchased
** <code>cntn</code> - the content is purchased
* <code>sale_price</code> - price in L$

Example:

sale_info 0
{
sale_type not
sale_price 10
}

'''Wearable Type'''

* A single line with the token <code>type</code> followed by the wearable type as a decimal integer

The type is defined in the <code>EWearableType</code> enumeration in <code>newview/llwearable.h</code>:

* Bodyparts
** <code>WT_SHAPE</code> = 0
** <code>WT_SKIN</code> = 1
** <code>WT_HAIR</code> = 2
** <code>WT_EYES</code> = 3
* Clothing
** <code>WT_SHIRT</code> = 4
** <code>WT_PANTS</code> = 5
** <code>WT_SHOES</code> = 6
** <code>WT_SOCKS</code> = 7
** <code>WT_JACKET</code> = 8
** <code>WT_GLOVES</code> = 9
** <code>WT_UNDERSHIRT</code> = 10
** <code>WT_UNDERPANTS</code> = 11
** <code>WT_SKIRT</code> = 12

Example:

type 4

'''Parameters'''

* A line with the token <code>parameters</code> followed by the number of parameter ID/value pairs as a decimal integer
* A sequence of parameter/ID value pairs, each on its own line
** Parameter ID from the [[#Linden avatar definition file]], a decimal integer
** Parameter value (or weight) is a decimal floating point value

Example:

parameters 10
781 1
800 .92
801 1
802 1
803 1
804 1
805 1
828 .18
840 0
868 .27

'''Texture'''

* A line with the token <code>textures</code> followed by the number of texture index/ID pairs as a decimal integer
* A sequence of texture index/ID pairs, each on its own line
** Texture index is defined in the <code>ETextureIndex</code> enumeration in <code>newview/llvoavatardefines.h</code>
** Texture ID is the texture's UUID in hexadecimal 8-4-4-4-12 format

Texture indexes:
* <code>TEX_HEAD_BODYPAINT</code> = 0
* <code>TEX_UPPER_SHIRT</code> = 1
* <code>TEX_LOWER_PANTS</code> = 2
* <code>TEX_EYES_IRIS</code> = 3
* <code>TEX_HAIR</code> = 4
* <code>TEX_UPPER_BODYPAINT</code> = 5
* <code>TEX_LOWER_BODYPAINT</code> = 6
* <code>TEX_LOWER_SHOES</code> = 7
* <code>TEX_HEAD_BAKED</code> = 8 (* See below)
* <code>TEX_UPPER_BAKED</code> = 9 (* See below)
* <code>TEX_LOWER_BAKED</code> = 10 (* See below)
* <code>TEX_EYES_BAKED</code> = 11 (* See below)
* <code>TEX_LOWER_SOCKS</code> = 12
* <code>TEX_UPPER_JACKET</code> = 13
* <code>TEX_LOWER_JACKET</code> = 14
* <code>TEX_UPPER_GLOVES</code> = 15
* <code>TEX_UPPER_UNDERSHIRT</code> = 16
* <code>TEX_LOWER_UNDERPANTS</code> = 17
* <code>TEX_SKIRT</code> = 18
* <code>TEX_SKIRT_BAKED</code> = 19 (* See below)
* <code>TEX_HAIR_BAKED</code> = 20 (* See below)

''* When the avatar changes a wearable, the simulator requests that the viewer rebake the *_BAKED textures; these are then uploaded as new temporary assets to the system so that the individual assets do not need to be downloaded. The *_BAKED textures are not referenced by wearables.''

Example:

textures 1
1 08115557-ad4d-09ed-6b39-2ffc1f2b58b2

Avatar Appearance

2009-05-22T20:01:40Z

Joshua Linden:

{{ProtocolNav}}
[[Category:Viewer Architecture]]
<br/>
For a good introduction to the elements of 3D character graphics, including meshes, morphs, bones, rendering, quaternions, and more see [http://www.gup.uni-linz.ac.at/~gk/Diplom/CART-Martin.pdf Martin Garstenauer's thesis].<br/><br/>

== Appearance Components ==
SL Avatar has "wearables" plus "attachments" for static appearance, and "animations".

=== Wearables ===
Wearables encompass both clothing and body parts – represent customizations of the basic avatar mesh via parameterized changes and textures.

Clothing can be taken off, but body parts cannot. Otherwise, they are similar.

Wearables are created by the viewer when appearance is edited and the results are saved; changed parts of appearance (clothing, bodyparts) are saved as new wearables which are uploaded as assets. There is no support for directly importing or exporting wearables.

=== Attachments ===
An "attachment" is an object ("linked set of primitives") which is rooted in the scene graph on an attachment point on an avatar. They are otherwise the same as other objects in SL, and can contain and execute scripts, inventory, maintain state, etc.

Attachments are heavily used for hair, some parts of clothing (dresses, cuffs) and accessories (jewelery, weapons). Non-humanoid avatars may use attachments to completely obscure the underlying humanoid avatar mesh.

Attachments are never serialized as files on the viewer, and there is no import/export support. Attachments are only transmitted to/from the viewer as ObjectUpdate messages which describe the constantly updating scene graph of the simulation. Attachments are created/modified using the "build" tools in the viewer; modifications are transmitted from the viewer to the simulator as editing messages. The simulator then broadcasts the changes to other users as ObjectUpdates. When an attachment is saved to inventory, the linked set of prims structure is serialized to a structure that includes script state.

=== Animations ===
Animations are Poser BVH files. Animations can be added to a scripted object to be played (e.g. a "poseball"), or incorporated into a gesture (a sequence of actions including text, sounds and animations invoked manually or triggered by a keyword). However, animations are often thought of as parts of avatars when they are controlled by an attachment to alter the avatar's default animations.

One common case is as an "Animation Override" script. These are used along with creative animations to deform the skeleton. Sometimes these override only some of the default animations (e.g. directed attention), sometimes everything is overridden. For example, "tinies" are created by folding the skeleton's limbs back on themselves. Non-humanoid avatars often use "meatballs" animations that compress the avatar to such an extent that it can be hidden inside an attachment.

Animations can be created in third party tools (e.g. Poser, QAvimator) and uploaded via File > Upload Animation (L$10). The animation is added to the inventory, and can then be incorporated into a script or gesture.

==Packets==

* [[AgentWearablesRequest]]
** This is where it starts. The viewer asks the simulator what it is wearing, and an AgentWearablesUpdate packet is returned.
* [[AgentWearablesUpdate]]
** A mapping of wearable types to asset IDs and item IDs is returned to the client. There are currently 13 different wearable types and all 13 will always be returned in this packet. If your avatar is not wearing a wearable type the asset ID and item ID for that type will be null IDs (all zeros). The next step is to request an asset transfer for all of the non-null asset IDs and download the wearable assets.
* [[AgentIsNowWearing]]
** Like the AgentWearablesUpdate packet but in the other direction. The viewer sends a list of the 13 wearable types and their associated item IDs, or null if nothing is being worn in that wearable slot. This is typically sent right before AgentSetAppearance.
* [[AgentSetAppearance]]
** Serves two purposes, to tell the server how our avatar mesh is deformed (with visual parameters) and what textures we are wearing. There are currently 218 VisualParam blocks sent with each AgentSetAppearance describing everything from the color of the avatar eyes to the gender slider. A TextureEntry is also sent that uses the same TextureEntry format objects use, but each "face" is hard-coded to describe a particular texture. This may or may not include baked textures for the five different baked layers (head, upper, lower, eyes, and skirt).
**The ParamValue fields do not have parameter IDs associated with them in the packet, as it's assumed all viewers have the same parameter map in the same sequence. The parameters are stored internally as a floating point integer, and are converted to a single byte by inputting the value along with minimum and maximum weights for that parameter to a conversion function.

* [[AvatarAppearance]]
* [[AgentCachedTexture]]
* [[AgentCachedTextureResponse]]

==Linden avatar definition file==

The parameters affecting avatar appearance are defined in an XML file
located in the executable at
: SecondLife\character\avatar_lad.xml
and in the source code at
: linden\indra\newview\character\avatar_lad.xml

With a few exceptions
: genepool.xml
: rouge_alpha.tga
: underpants_trial_female.tga
: underpants_trial_male.tga
: undershirt_trial_female.tga
: upperbodyfreckles_alpha.tga
all of the other files in the same directory are referenced by avatar_lad.xml. These include the mesh files (.llm), the texture files (.tga) and the avatar_skeleton.xml file (see below).

genepool.xml contains parameter sets for wearable types WT_SHAPE, WT_SKIN, WT_HAIR, and WT_EYES for several archetypal avatar definitions. You can see the current parameter set for your own avatar by invoking the debug menus (Ctrl-Alt-D) and selecting
Client->Character->Character Tests->Appearance To XML. This will create a file
: SecondLife\character\new archetype.xml
in the same format as genepool.xml.

The avatar definition file affects the appearances on your viewer only. Any change to the file will not affect how others see the avatars. You can set value_default="1" for "Express_Tongue_Out" to make the avatars on your viewer give you the raspberries, but no one else will notice.

The interesting thing is that this file defines parameters that can be used in animations.

===File structure===

'''<?xml version="1.0" encoding="US-ASCII" standalone="yes"?>'''

'''<linden_avatar version="1.0" wearable_definition_version="22">'''

: '''<skeleton file_name="avatar_skeleton.xml">'''
:: '''''<attachment_point'' id="1" group="6" name="Chest" joint="mChest"'''
:: position="0.15 0 -0.1" rotation="0 90 90" visible_in_first_person="true"
:: pie_slice="2" hud="true" max_attachment_offset="1.5" '''''/>'''''
:: '''''<param ... >'''''
::: '''<param_skeleton>'''
:::: '''''<bone'' name="mNeck" scale="0 0 .2"''' offset="0 0 .004" '''''/>'''''
::: '''</param_skeleton>'''
:: '''''</param>'''''
: '''</skeleton>'''

:Sets of
: '''<mesh type="hairMesh" lod="0" file_name="avatar_hair.llm"'''
: min_pixel_width="320" min_pixel_area="0.1" '''>'''
:: '''''<param ... >'''''
::: '''<param_morph />'''
:::or
::: '''<param_morph>'''
:::: '''''<volume_morph''''' name="HEAD" scale="-0.008 -0.006 0.015"
:::: pos="0.07 0 -0.07" '''''/>'''''
::: '''</param_morph>'''
:: '''''</param>'''''
: '''</mesh>'''
: ''<mesh'' '''type="hairMesh" lod="1" file_name="avatar_hair_1.llm"'''
: min_pixel_width="160" min_pixel_area="0.1" '''reference="avatar_hair.llm"'''''>''
: ''</mesh>''

: '''''<global_color'' name="skin_color"''>'''''
:: '''<param ... >'''
::: '''<param_color''' operation="blend" '''>'''
:::: '''''<value'' color="252, 215, 200, 255" ''/>'''''
::: '''</param_color>'''
:: '''</param>'''
: '''''</global_color>'''''

: '''''<layer_set'' body_region="head" width="512" height="512"'''
: clear_alpha="false" alpha_tga_file="head_alpha.tga" '''''>'''''

:: '''''<layer'' name="head bump definition"''' render_pass="bump"
:: global_color="hair_color" fixed_color="128,128,128,255"
:: write_all_channels="true" '''''>'''''

::: <texture '''local_texture="lower_jacket"''' local_texture_alpha_only="true" />
:::and if not fixed_color
::: <texture '''tga_file="bump_head_base.tga"''' file_is_mask="FALSE" />

::: ''<morph_mask'' morph_name="Displace_Loose_Upperbody" invert="FALSE" ''/>''

::: ''<param ... >''
:::: '''<param_alpha''' tga_file="bump_face_wrinkles.tga" skip_if_zero="true"
:::: multiply_blend="false" domain="0" '''/>'''
::::or
:::: '''<param_color''' operation="multiply" '''>'''
::::: '''''<value'' color="0, 0, 0, 0" ''/>'''''
:::: '''</param_color>'''
::: ''</param>''
:: '''''</layer>'''''
: '''''</layer_set>'''''

: '''<driver_parameters>'''
:: '''''<param ... >'''''
::: '''<param_driver>'''
:::: '''''<driven'' id="628"''' min1="0.1" max1="0.5" max2="1" min2="1" '''''/>'''''
::: '''</param_driver>'''
:: '''''</param>'''''
: '''</driver_parameters>'''

'''</linden_avatar>'''

where '''bold''' text indicates required tags or parameters, standard text
indicates a tag that can occur zero or one time, ''italics''
indicates that the tag can be repeated zero or more times, and '''''bold italics''''' indicates
that at least one is required but more are possible.

Note that the values shown for parameters here may not be consistent. They are shown only
to indicate the value type.

===Parameter detail===

<param ... > is

: '''<param id="32"''' group="1" name="Male_Skeleton"
: value_min="0" value_max="1" value_default="0"
: edit_group="shape_body" edit_group_order="1"
: label="Skeleton" label_min="Female" label_max="Male"
: wearable="shape" clothing_morph="true" sex="male" shared="1"
: show_simple="true" simple_percent_min="40" simple_percent_max="100"
: camera_distance="2.2" camera_elevation=".1" camera_angle="0" '''>'''

Parameters are parsed in LLViewerVisualParamInfo::parseXml and LLVisualParamInfo::parseXml and are used by LLViewerVisualParam and LLVisualParam respectively when LLVOAvatar::loadAvatar is called.

Fields parsed in LLVisualParamInfo are used to describe the meaning of the parameters.

;id -- mID
;name -- mName
:default ""
;group -- mGroup
:"0" is VISUAL_PARAM_GROUP_TWEAKABLE
:"1" is VISUAL_PARAM_GROUP_ANIMATABLE
:default "0"
;value_min -- mMinWeight
:default 0.
;value_max -- mMaxWeight
:default 1.
;value_default -- mDefaultWeight
:default 0.
;sex -- mSex
{|
| || || "both" || SEX_BOTH
|-
| || || "male" || SEX_MALE
|-
| || || "female" || SEX_FEMALE
|-
| || || default || SEX_BOTH
|}
;label -- mDisplayName
:default is name
;label_min -- mMinName
:default is "Less"
;label_max -- mMaxName
:default is "More"
;shared
:Used only by mesh params. Indicates that the param affects more than one mesh

Fields parsed in LLViewerVisualParamInfo describe the appearance of the entries on the Edit->Appearance editing panel.

;wearable -- mWearableType
{|
| || || "shape" || WT_SHAPE
|-
| || || "skin" || WT_SKIN
|-
| || || "hair" || WT_HAIR
|-
| || || "eyes" || WT_EYES
|-
| || || "shirt" || WT_SHIRT
|-
| || || "pants" || WT_PANTS
|-
| || || "shoes" || WT_SHOES
|-
| || || "socks" || WT_SOCKS
|-
| || || "jacket" || WT_JACKET
|-
| || || "gloves" || WT_GLOVES
|-
| || || "undershirt" || WT_UNDERSHIRT
|-
| || || "underpants" || WT_UNDERPANTS
|-
| || || "skirt" || WT_SKIRT
|-
| || || default || WT_INVALID
|}
;edit_group -- mEditGroup
:default ""
;edit_group_order -- mEditGroupDisplayOrder
:By default, value 0, parameters are displayed in the order in which they appear in the xml file. "edit_group_order" overriddes this.

Optional camera offsets from the current joint center. Used for generating "hints" (thumbnails).
;camera_distance -- mCamDist
:default 0.5
;camera_angle -- mCamAngle
:default 0.
;camera_elevation -- mCamElevation
:default 0.

;show_simple -- mShowSimple : value is ignored
:default FALSE
;simple_percent_min -- mSimpleMin : value is ignored
:default 0.
;simple_percent_max -- mSimpleMax : value is ignored
:default 100.

===Loading the file===

The file is parsed in LLVOAvatar::initClass() primarily by calls to LLVOAvatarInfo. Both classes are defined in indra\newview\llvoavatar.h(.cpp).
There is one method for each of the major XML nodes
:LLVOAvatarInfo::parseXmlSkeletonNode
:LLVOAvatarInfo::parseXmlMeshNodes
:LLVOAvatarInfo::parseXmlColorNodes
:LLVOAvatarInfo::parseXmlLayerNodes
:LLVOAvatarInfo::parseXmlDriverNodes

This data is then used to create instances of LLVOAvatar in LLVOAvatar::loadAvatar()
which invokes LLVOAvatar::loadSkeletonNode() and LLVOAvatar::loadMeshNodes().

==LLVisualParam==

The avatar visual parameters are divided into those that define the avatar's appearance and those that are used in animations. Theoretically, they all could be used for animations, but the animations would have to start and end at the settings defined for the avatar rather than at the default values.

The group field of the <param> tag identifies those that are directly set in Edit->Appearance and sent in [[AgentSetAppearance]] messages as VISUAL_PARAM_GROUP_TWEAKABLE. The rest are identified as VISUAL_PARAM_GROUP_ANIMATABLE, but this is a bit misleading because it includes all of the driven parameters defined in the <driver_parameters> stanza. These are set by the driver parameters which are themselves directly tweakable.

Those that are not directly or indirectly tweakable include the emotes
:* 300 Express_Closed_Mouth
:* 301 Express_Tongue_Out
:* 302 Express_Surprise_Emote
:* 303 Express_Wink_Emote
:* 304 Express_Embarrassed_Emote
:* 305 Express_Shrug_Emote
:* 306 Express_Kiss
:* 307 Express_Bored_Emote
:* 308 Express_Repulsed_Emote
:* 309 Express_Disdain
:* 310 Express_Afraid_Emote
:* 311 Express_Worry_Emote
:* 312 Express_Cry_Emote
:* 313 Express_Sad_Emote
:* 314 Express_Anger_Emote
:* 315 Express_Frown
:* 316 Express_Laugh_Emote
:* 317 Express_Toothsmile
:* 318 Express_Smile
:* 632 Express_Open_Mouth
the hand morphs
:* 101 Hands_Relaxed
:* 102 Hands_Point
:* 103 Hands_Fist
:* 666 Hands_Relaxed_L
:* 667 Hands_Point_L
:* 668 Hands_Fist_L
:* 669 Hands_Relaxed_R
:* 670 Hands_Point_R
:* 671 Hands_Fist_R
:* 672 Hands_Typing
:* 766 Hands_Salute_R
:* 791 Hands_Peace_R
:* 792 Hands_Spread_R
and blinking eyes
:* 58 Blink_Left
:* 59 Blink_Right

The remaining visual params include a few useful for animations
:* 51 Furrowed_Eyebrows
:* 53 Surprised_Eyebrows
:* 54 Worried_Eyebrows
:* 55 Frown_Mouth
:* 57 Smile_Mouth
and a couple that seem to have been abandoned
:* 41 Old
:* 128 Wrinkles

==Linden skeleton definition file==

The skeletal structure is defined in an XML file located in the executable at
: SecondLife\character\avatar_skeleton.xml
and in the source code at
: linden\indra\newview\character\avatar_skeleton.xml

It is referenced through the avatar definition file (see above) and can be
considered to be part of the avatar definition.

===File structure===

'''<?xml version="1.0" encoding="US-ASCII" standalone="yes"?>'''

'''<linden_skeleton version="1.0" num_bones="46" num_collision_volumes="19">'''

: '''<bone name="mPelvis" pos="0.000 0.000 1.067" rot="0.000000 0.000000 0.000000"
: '''scale="1.000 1.000 1.000" pivot="0.000000 0.000000 1.067015">'''
:: <collision_volume '''name="PELVIS" pos = "-0.01 0 -0.02"'''
:: '''rot="0.000000 8.00000 0.000000" scale="0.12 0.16 0.17"'''/>
:: ''<bone ...>''
:: ''</bone>''
: '''</bone>'''

'''</linden_skeleton>'''

where '''bold''' text indicates required tags or parameters, standard text
indicates a tag that can occur zero or one time, and ''italics''
indicates that the tag can be repeated zero or more times.

The <bone ...> tag above is to indicate that the structure is recursive.

The "46" for num_bones is actually a typo in the file. There are only 26 bones defined.

===Loading the file===

The file is parsed in LLVOAvatar::initClass() using LLVOAvatarSkeletonInfo::parseXml. Both classes are defined in indra\newview\llvoavatar.h(.cpp).

This data is then used to create instances of LLVOAvatar in LLVOAvatar::loadAvatar()
which invokes LLVOAvatar::buildSkeleton(). A special-purpose "screen" joint named "mScreen" is also created.

==Linden binary mesh file==

Meshes are defined in binary files located in the executable at
: SecondLife\character\*.llm
and in the source code at
: linden\indra\newview\character\*.llm

They are referenced through the avatar definition file (see above) and can be
considered to be part of the avatar definition.

The morphName in the binary mesh file matches the name field of the visual params with <param_morph> tags in the <mesh> stanzas. There are a few morphs defined in the mesh files that are not used by the avatar definition file
:* Blink_Both
:* Fingernails_Long
:* Wide_Lips

There are two types of mesh files: the base mesh definition and the level of detail (LOD) mesh definition, which always references a base mesh definition file. LOD meshes are used to simplify the rendering when the avatar is distant from the camera. There are several LOD meshes with progressively decreasing resolution.

The vertex information referenced by the LOD meshes is stored at the beginning of the base mesh vertex information.

===File structure===
====Base mesh====

:char[24] "Linden Binary Mesh 1.0"
:U8 hasWeights
:U8 hasDetailTexCoords
:LLVector3 position
:LLVector3 rotationAngles
:U8 rotationOrder (ignored, set to 0)
:LLVector3 scale
:U16 numVertices
:LLVector3[ numVertices ] baseCoords
:LLVector3[ numVertices ] baseNormals
:LLVector3[ numVertices ] baseBinormals
:LLVector2[ numVertices ] texCoords
:if (hasDetailTexCoords)
::LLVector2[ numVertices ] detailTexCoords
:if (hasWeights)
::F32[ numVertices ] weights
:U16 numFaces
:struct faces[ numFaces ]
:{
::S16[3] face
:}
:if ( hasWeights )
:{
::U16 numSkinJoints
::struct skinJoints[ numSkinJoints ]
::{
:::char[64] jointName
::}
:}
:struct morph []
:{
::char[64] morphName
::S32 numVertices
::struct vertices[ numVertices ]
::{
:::U32 vertexIndex
:::LLVector3 coord
:::LLVector3 normal
:::LLVector3 binormal
:::LLVector2 texCoord
::}
:}
:char[64] "End Morphs"
:S32 numRemaps (or EOF)
:struct vertexRemap[ numRemaps ]
:{
::S32 remapSource
::S32 remapDestination
:}

====Level Of Detail (LOD) mesh====

:char[24] "Linden Binary Mesh 1.0"
:U8 hasWeights
:U8 hasDetailTexCoords
:LLVector3 position
:LLVector3 rotationAngles
:U8 rotationOrder (ignored, set to 0)
:LLVector3 scale
:U16 numFaces
:struct faces[ numFaces ]
:{
::S16[3] face
:}

where LLVector2 and LLVector3 are 2 and 3 floats respectively, and all numbers are little endian. Integers are 8, 16, or 32 bits signed (S) or unsigned (U).

The LOD .llm files actually contain fields for (if hasWeights) numSkinJoints set to zero, char[64] "End Morphs", and numRemaps set to zero, but these are not read by the viewer.

Each weight actually contains two pieces of information. The number to the left of the decimal point is the index of the joint and also implicitly indexes to the following joint. The actual weight is to the right of the decimal point and interpolates between these two joints. The index is into an "expanded" list of joints, not just a linear array of the joints as defined in the skeleton file. In particular, any joint that has more than one child will be repeated in the list for each of its children.

===Loading the file===

This data is used to create instances of LLVOAvatar in LLVOAvatar::loadAvatar()
through LLVOAvatar::loadMeshNodes, LLPolyMesh::getMesh, LLPolyMeshSharedData::loadMesh, and LLPolyMorphData::loadBinary. These are found in indra\newview\llvoavatar.h(.cpp), indra\newview\llpolymesh.h(.cpp), and indra\newview\llpolymorph.h(.cpp).

== Linden wearable definition file ==

The format is used for client/server communication - wearable assets are downloaded to cache/uploaded on creation. No import/export ability currently exists, although wearables are manipulated as files both on the viewer and server.

=== Source References ===

* <code>newview/llwearable.h</code>
** <code>enum EWearableType</code>
* <code>newview/llwearable.cpp</code>
** <code>LLWearable::importFile</code> – parses assets when "worn" from inventory, after download to cache
** <code>LLWearable::exportFile</code> – called by <code>LLWearable::saveNewAsset</code> when created, does upload to asset system
* <code>newview/llvoavatardefines.h</code>
** <code>enum ETextureIndex</code>

=== File Structure ===

Details:
* The file is line-oriented. UNIX line endings (LF 0x0A) are expected.
* File encoding for strings is implicitly UTF-8. No byte-order mark is allowed.
* Whitespace is used as delimiters. Spaces (0x20) and tabs (0x09) are considered whitespace, and any number may be used.

The file consists of the following blocks, which must occur in the specified order:

* Format/Version
* Name
* Description
* Permissions
* Sale Info
* Wearable type
* Parameters
* Textures

==== Format/Version ====

The string <code>LLWearable version</code> followed by the version number (currently <code>22</code>)

Example:

LLWearable version 22

==== Name ====

A single line string, terminated with a UNIX newline (LF 0x0A). Maximum length is 2047 bytes, not counting the newline. Encoding is UTF-8. A blank line is used to indicate the absence of a name.

Example:

Space Cadet Shirt

==== Description ====

A single line string, terminated with a UNIX newline (LF 0x0A). Maximum length is 2047 bytes, not counting the newline. Encoding is UTF-8. A blank line is used to indicate the absence of a description.

Example:

This is my awesome shirt!

==== Permissions block ====

* First line: The token <code>permissions</code> followed by the permission block version <code>0</code>
* Second line: open brace <code>{</code>
* A sequence of permission name/value pairs, each on its own line
* Final line: close brace <code>}</code>

The permission name/value pairs are:

* <code>base_mask</code>, with 8-digit hexadecimal value
* <code>owner_mask</code>, with 8-digit hexadecimal value
* <code>group_mask</code>, with 8-digit hexadecimal value
* <code>everyone_mask</code>, with 8-digit hexadecimal value
* <code>next_owner_mask</code>, with 8-digit hexadecimal value
* <code>creator_id</code>, with agent UUID in hexadecimal 8-4-4-4-12 format
* <code>owner_id</code>, with agent UUID in hexadecimal 8-4-4-4-12 format
* <code>last_owner_id</code>, with agent UUID in hexadecimal 8-4-4-4-12 format
* <code>group_id</code>, with group UUID in hexadecimal 8-4-4-4-12 format
* <code>group_owned</code>, with <code>0</code> (false) or <code>1</code> (true) flag

If a name/value is missing, a default value is assumed. For maximum compatibility, everything should be explicitly specified except <code>group_owned</code> when false.

Flags are defined in <code>llinventory/llpermissionsflags.h</code>:
* <code>PERM_TRANSFER</code> = 0x00002000
* <code>PERM_MODIFY</code> = 0x00004000
* <code>PERM_COPY</code> = 0x00008000
* <code>PERM_MOVE</code> = 0x00080000

Common masks are:
* <code>PERM_NONE</code> 0x00000000
* <code>PERM_ALL</code> 0x7FFFFFFF

And common values are:
* no modify: <code>PERM_ALL & ~PERM_MODIFY</code> = 0x7fffbfff
* no copy: <code>PERM_ALL & ~PERM_COPY</code> = 0x7fff7fff
* no modify or copy: = 0x7fff3fff
* no transfer: <code>PERM_ALL & ~PERM_TRANSFER</code> = 0x7fffdfff
* no modify, no transfer = 0x7fff9fff

Example:

permissions 0
{
base_mask 7fffffff
owner_mask 7fffffff
group_mask 00000000
everyone_mask 7fffbfff
next_owner_mask 7fffffff
creator_id d929385f-41e3-4a34-a04e-f1fc39f24f12
owner_id ff9a71eb-7414-4bf8-866e-a70ddeb7c3cf
last_owner_id ff9a71eb-7414-4bf8-866e-a70ddeb7c3cf
group_id 00000000-0000-0000-0000-000000000000
}

==== Sale Info block ====

* First line: The token <code>sale_info</code> followed by the sale_info block version <code>0</code>
* Second line: open brace <code>{</code>
* A sequence of sale info name/value pairs, each on its own line
* Final line: close brace <code>}</code>

The sale info name/value pairs are:
* <code>sale_type</code>: one of:
** <code>not</code> - not for sale
** <code>orig</code> - the original is purchased
** <code>copy</code> - a copy is purchased
** <code>cntn</code> - the content is purchased
* <code>sale_price</code> - price in L$

Example:

sale_info 0
{
sale_type not
sale_price 10
}

==== Wearable Type ====

* A single line with the token <code>type</code> followed by the wearable type as a decimal integer

The type is defined in the <code>EWearableType</code> enumeration in <code>newview/llwearable.h</code>:

* Bodyparts
** <code>WT_SHAPE</code> = 0
** <code>WT_SKIN</code> = 1
** <code>WT_HAIR</code> = 2
** <code>WT_EYES</code> = 3
* Clothing
** <code>WT_SHIRT</code> = 4
** <code>WT_PANTS</code> = 5
** <code>WT_SHOES</code> = 6
** <code>WT_SOCKS</code> = 7
** <code>WT_JACKET</code> = 8
** <code>WT_GLOVES</code> = 9
** <code>WT_UNDERSHIRT</code> = 10
** <code>WT_UNDERPANTS</code> = 11
** <code>WT_SKIRT</code> = 12

Example:

type 4

==== Parameters ====

* A line with the token <code>parameters</code> followed by the number of parameter ID/value pairs as a decimal integer
* A sequence of parameter/ID value pairs, each on its own line
** Parameter ID from the [[#Linden avatar definition file]], a decimal integer
** Parameter value (or weight) is a decimal floating point value

Example:

parameters 10
781 1
800 .92
801 1
802 1
803 1
804 1
805 1
828 .18
840 0
868 .27

==== Texture ====

* A line with the token <code>textures</code> followed by the number of texture index/ID pairs as a decimal integer
* A sequence of texture index/ID pairs, each on its own line
** Texture index is defined in the <code>ETextureIndex</code> enumeration in <code>newview/llvoavatardefines.h</code>
** Texture ID is the texture's UUID in hexadecimal 8-4-4-4-12 format

Texture indexes:
* <code>TEX_HEAD_BODYPAINT</code> = 0
* <code>TEX_UPPER_SHIRT</code> = 1
* <code>TEX_LOWER_PANTS</code> = 2
* <code>TEX_EYES_IRIS</code> = 3
* <code>TEX_HAIR</code> = 4
* <code>TEX_UPPER_BODYPAINT</code> = 5
* <code>TEX_LOWER_BODYPAINT</code> = 6
* <code>TEX_LOWER_SHOES</code> = 7
* <code>TEX_HEAD_BAKED</code> = 8 (* See below)
* <code>TEX_UPPER_BAKED</code> = 9 (* See below)
* <code>TEX_LOWER_BAKED</code> = 10 (* See below)
* <code>TEX_EYES_BAKED</code> = 11 (* See below)
* <code>TEX_LOWER_SOCKS</code> = 12
* <code>TEX_UPPER_JACKET</code> = 13
* <code>TEX_LOWER_JACKET</code> = 14
* <code>TEX_UPPER_GLOVES</code> = 15
* <code>TEX_UPPER_UNDERSHIRT</code> = 16
* <code>TEX_LOWER_UNDERPANTS</code> = 17
* <code>TEX_SKIRT</code> = 18
* <code>TEX_SKIRT_BAKED</code> = 19 (* See below)
* <code>TEX_HAIR_BAKED</code> = 20 (* See below)

''* When the avatar changes a wearable, the simulator requests that the viewer rebake the *_BAKED textures; these are then uploaded as new temporary assets to the system so that the individual assets do not need to be downloaded. The *_BAKED textures are not referenced by wearables.''

Example:

textures 1
1 08115557-ad4d-09ed-6b39-2ffc1f2b58b2

SLDev Open Source Viewer

2009-05-01T15:25:19Z

Joshua Linden: /* Get it committed to the repository */

=What is the SLDev Open Source Viewer?=

See [https://blogs.secondlife.com/community/technology/blog/2009/03/30/intensifying-open-source-efforts Philip's blog post] for the initial announcement.

Merov posted on SLDev:

''We do hope that most of the innovations battle-tested here will
eventually make it to the official viewer codebase but things that are
clearly not mainstream enough or not very popular with this community
simply won't make it there and that's ok. People will still be able to
get those features in the "Second Life OSS" download though.''

This document is an edited version of some of Merov's other comments on the SLDev list combined with an edited outline originally by Mike Monkowski.

=How do I contribute to it?=
==Talk to the mailing list==

First, you might want to have a discussion about your idea. Chat on the SLDev mailing list. Create a JIRA and see if others agree with you. Refine the JIRA to get the idea clearly defined.

==Write your code==

Writing code isn't just writing the lines of C++. It's also documenting the code, designing a test plan, doing the actual testing, and making sure it works in a variety of situations.

When you think it's solid, you should create a patch file (using a diff tool like svn diff) and attach the patch to the JIRA. You should attach the test plan in the JIRA, either directly in a comment, an attached document, or a nonvolatile link to it on the web.

==Get your code reviewed==

You should then ask in the JIRA, and possibly on SLDev, for comments on the code and for people to help by trying out the test plan. Give people time (at least a day, maybe quite a bit more depending on the complexity of your submission and the timing) to evaluate it and comment.

Keep in mind that a positive, engaged attitude on SLDev and in JIRA is likely to garner more support than a more negative approach.

==Get it committed to the repository==

It's like almost any other FLOSS project: there is a group of
'''committers''' that do have commit privileges to the svn tree. The big
difference with the "old LL way of doing things" is that this group of
committers has Lindens '''and''' non-Lindens. Engage folks on SLDev to get support: are people interested by
the feature, is this a worthy issue to fix?

If there is community
support and a good review of the code from committers, the patch will
be taken in and committed by one of the committers. Eventually, if your
contributions are significant and in line with the project,
other committers might propose that you to become a committer himself. The
committers group grows by internal recommendations only.

If someone objects (and if our community is alive, there '''will''' be
someone to object...), we debate and try to reach a consensus on this
list. If there's a really contentious decision to be made (split
debate), Philip, our BDFL (Benevolent Dictator For Life), will weight
in and make the decision.

==Track it through the merge process==

Some contributions may merge back to the official viewer someday. That
will be up to the LL
viewer team to cherry pick and merge patches/commits separately.

This is not easy to do. Making this more manageable is one
of the big reason for the push to hg (mercurial) which does make
merges from various sources much more easier than svn.

SLDev Open Source Viewer

2009-05-01T15:25:01Z

Joshua Linden: /* Get it committed to the repository */

=What is the SLDev Open Source Viewer?=

See [https://blogs.secondlife.com/community/technology/blog/2009/03/30/intensifying-open-source-efforts Philip's blog post] for the initial announcement.

Merov posted on SLDev:

''We do hope that most of the innovations battle-tested here will
eventually make it to the official viewer codebase but things that are
clearly not mainstream enough or not very popular with this community
simply won't make it there and that's ok. People will still be able to
get those features in the "Second Life OSS" download though.''

This document is an edited version of some of Merov's other comments on the SLDev list combined with an edited outline originally by Mike Monkowski.

=How do I contribute to it?=
==Talk to the mailing list==

First, you might want to have a discussion about your idea. Chat on the SLDev mailing list. Create a JIRA and see if others agree with you. Refine the JIRA to get the idea clearly defined.

==Write your code==

Writing code isn't just writing the lines of C++. It's also documenting the code, designing a test plan, doing the actual testing, and making sure it works in a variety of situations.

When you think it's solid, you should create a patch file (using a diff tool like svn diff) and attach the patch to the JIRA. You should attach the test plan in the JIRA, either directly in a comment, an attached document, or a nonvolatile link to it on the web.

==Get your code reviewed==

You should then ask in the JIRA, and possibly on SLDev, for comments on the code and for people to help by trying out the test plan. Give people time (at least a day, maybe quite a bit more depending on the complexity of your submission and the timing) to evaluate it and comment.

Keep in mind that a positive, engaged attitude on SLDev and in JIRA is likely to garner more support than a more negative approach.

==Get it committed to the repository==

It's like almost any other FLOSS project: there is a group of
''committers'' that do have commit privileges to the svn tree. The big
difference with the "old LL way of doing things" is that this group of
committers has Lindens *and* non-Lindens. Engage folks on SLDev to get support: are people interested by
the feature, is this a worthy issue to fix?

If there is community
support and a good review of the code from committers, the patch will
be taken in and committed by one of the committers. Eventually, if your
contributions are significant and in line with the project,
other committers might propose that you to become a committer himself. The
committers group grows by internal recommendations only.

If someone objects (and if our community is alive, there *will* be
someone to object...), we debate and try to reach a consensus on this
list. If there's a really contentious decision to be made (split
debate), Philip, our BDFL (Benevolent Dictator For Life), will weight
in and make the decision.

==Track it through the merge process==

Some contributions may merge back to the official viewer someday. That
will be up to the LL
viewer team to cherry pick and merge patches/commits separately.

This is not easy to do. Making this more manageable is one
of the big reason for the push to hg (mercurial) which does make
merges from various sources much more easier than svn.

Release Notes/Second Life Beta Server/1.24

2008-08-20T20:45:52Z

Joshua Linden: /* Release Notes for Second Life Beta Server 1.24 */

== Release Notes for Second Life Beta Server 1.24 ==

Changes:
* Mono is an alternate way to run ordinary LSL scripts. It has advantages of speed and memory management over the current script engine. For more information about Mono see the wiki article: https://wiki.secondlife.com/wiki/Mono.
** Note: Although no special viewer is needed to run Mono scripts, to create them you will need a viewer version of 1.21 or later.

* Enabled group moderation features for voice chat.

Changes that require both 1.24 Server *and* 1.21 Viewer (not yet released):
* New LSL calls for touch "position" in the touch_start(), touch(), and touch_end() events. Each touch() event provides the current surface point data (allowing grabbing, sliders, levers, and all sorts of pseudo GUI builds!):
** [[llDetectedTouchUV]]() - returns the UV coordinates of the point touched.
** [[llDetectedTouchFace]]() - returns the face of the point touched.
** [[llDetectedTouchPos]]() - returns the world coordinates of the point touched.
** [[llDetectedTouchNormal]]() - returns the surface normal of the point touched.
** [[llDetectedTouchBinormal]]() - returns the surface bi-normal of the point touched.
** [[llDetectedTouchST]]() - call to query surface coordinates.

* New feature to return all objects on an estate
** [https://jira.secondlife.com/browse/MISC-713 MISC-713]: Add "Return All Objects" as an estate-wide feature for a specified resident

* New info added:
** Add Time Stamp to Parcel Object List
** Add Time Stamp to Top Scripts Colliders List
** Add more categories to Help -> Report Abuse..

Fixes:
* Three security fixes

Release Notes/Second Life Beta Server/1.23

2008-07-02T22:54:53Z

Joshua Linden:

== Release Notes for Second Life Beta Server 1.23: ==

Changes:
* New LSL call for count of agents in region: [[llGetRegionAgentCount]]()
** Expected in 1.21 viewer
* New LSL call to get user's client language: [[llGetAgentLanguage]](llDetectedKey(0)))
** Expected in 1.21 viewer
* Dynamic sound throttling (test your content!)

Fixes:
* Havok4 fixes:
** [https://jira.secondlife.com/browse/SVC-2485 SVC-2485]: Phantom Objects Collide with Virtual Prims in 1.22.2
** [https://jira.secondlife.com/browse/SVC-2484 SVC-2484]: [[llVolumeDetect]]() Fails to detect some avatars
** [https://jira.secondlife.com/browse/SVC-2448 SVC-2448]: [[llUnSit]] Attack/Bug
** [https://jira.secondlife.com/browse/MISC-1261 MISC-1261]: Sim crashes following unlinking of a group of overlapping individually scripted prims rotating with llTargetOmega.
** Fix to [[llSetLinkPrimitiveParams ]]
** Toxic message alert ('Unable to create item that has caused problems on this region) is now a script error and throttled
** Fix to XML parser to improve simulator performance
** Fixed: Sim crash by rezzing complex objects.

* Other Fixes:
** [https://jira.secondlife.com/browse/SVC-1507 SVC-1507]: Notices failing for larger groups
** [https://jira.secondlife.com/browse/SVC-216 SVC-216]: New autoreturn email message shows incorrect region name & position
** [https://jira.secondlife.com/browse/VWR-4513 VWR-4513]: SL takes my money but doesn't give me the service I paid for
** [https://jira.secondlife.com/browse/VWR-3729 VWR-3729]: Notices still received when "Receive Group Notices" is unchecked
** Fixed: LSL [[Changed]] event on region cross behaves poorly
** Fixed: Gestures deactivate when moving them between folders

Service Disruptions

2008-04-09T23:27:48Z

Joshua Linden: /* Dataservers */

Second Life is a complex system with many inter-operating components, from simulators and databases to the viewer you run, and the Internet connections over which data flows. Although it is intended to be as [http://en.wikipedia.org/wiki/Loose_coupling loosely coupled] as possible and resilient against problems, events can still occur which lead to service disruptions.

In some cases the failures are beyond the control of Linden Lab. However, in nearly all cases there is active work being done to mitigate the disruptions - either prevent them from happening or significantly reduce the impact.

When a disruption occurs, the following sequence usually occurs:
* a system stops responding
* automated notifications go off, alerting our operations team
* residents often notice immediately, and alert in-world support, who confirm the problem to our operations team
* the operations team identifies the system that is the root cause of the problem
* the communications team is notified, and asked to provide information about the disruption to the blog
* if the disruption lasts more a few minutes, updates are made to the blog regularly
* once the problem is solved, an "all clear" is reported to the blog

Note that in many cases a disruption may be solved before any information can make it to the blog explaining the details of the problem. One purpose of this document is to provide a "clearing house" for types of service disruptions, so that in the event of a system failure the blog post can reference this page.

At the time of this writing, and in no particular order, these are the systems which have been known to cause service disruptions:

== Asset Storage Cluster ==

'''What is it:''' A cluster of machines that form a whopping huge [http://en.wikipedia.org/wiki/WebDAV WebDAV] (think "web-based disk drive") storage mechanism with terabytes of space for storing assets, including uploaded textures, snapshots, scripts, objects taken into inventory, script states, saved region states (simstates), etc that make up Second Life. The technology (software and hardware) is licensed from a third party.

'''How it can fail:''' The system should be resilient against single node failures. In the case of multiple disk failures, software upgrades, removing problem nodes or adding new nodes, some or all of the cluster can fall offline. If this happens, asset uploads and downloads fail - this causes texture uploads and simstate saves to fail. Since transient data during region crossings (attachment states, etc) are written as assets, region crossings will also often fail.

'''How we fix it:''' When detected, we often disable logins and message in-world (if possible) to help avoid data loss. Failed nodes can be taken out of rotation. A restart of other nodes may be necessary. When upgrading the software on the nodes, the grid is usually closed to prevent data loss during any inadvertent outages. An asset system failure requiring a reboot occurred on [http://blog.secondlife.com/2008/03/28/asset-server-issues/ March 28th, 2008]

== Central Database Cluster ==

'''What is it:''' A cluster of databases that store the core persistent information about Second Life - including resident profiles, groups, regions, parcels, L$ transactions and classifieds.

'''How it can fail:''' The database can become loaded enough during normal operations that some fraction of transactions fail and either must be manually retried or are automatically retried. Hardware failure and software bugs in the database code can also cause the database to crash or stop responding. Logins will fail, transactions in-world and on the web site will fail, and so forth.

'''How we fix it:''' If the primary database fails, we swap to one of the secondaries. If the database load is high but hasn't failed we can turn off services to try and reduce the load.

''Eliminating this cluster as a scalability bottleneck and failure point is a very high priority for Linden Lab. While this is in progress, load mitigation is occurring. Watch the [http://blog.secondlife.com/2008/04/07/profile-and-group-services-temporarily-unavailable/ Second Life Blog] for updates''

== Agent ("Inventory") Database Cluster ==

'''What is it:''' Storage for most agent-specific data such as the inventory tree is partitioned across a series of databases. Each agent is associated with a particular inventory partition (a primary database and its secondary backups). At the time of this writing, we have approximately 15 agent database partitions.

The initial use of these agent-partitioned databases was for inventory, so they are often referred to as "inventory databases" by Lindens, but this is no longer the extent of what agent-specific data is stored within them.

'''How it can fail:''' Hardware or software failure can affect the primary database within the partition, so that it either stops responding to queries or becomes excessively slow.

'''How we fix it:''' When an agent database fails we can swap to the backup within that partition, which takes a few minutes. If this will not happen immediately or if problems are encountered, that particular agent partition is "blacklisted" temporarily; this causes logins of agents who are associated with that partitions to be blocked and any agents logged in are "kicked", while the fix is in progress. This will affect some fraction of the grid, but not everybody.

''This is an example of a system that in the past was prone to causing global service disruption. It was re-designed to limit the impact to Residents even in the face of hardware failure; only Residents associated with a particular partition are affected during such a failure.''

== Other Database Clusters ==

'''What is it:''' There are a handful other database clusters in use. One is used for logging data.

'''How it can fail:''' Hardware or software failures can take a database cluster offline. There should be no in-world effect from one of these other database clusters failing, but occasionally a software design flaw does introduce a dependency that is not caught. For example, logins used to require a successful connection to the login database to record the login and viewer statistics, but this dependency has been removed.

'''How we fix it:''' All databases act in clusters with a primary machine and several secondaries. In case of failure, a secondary can be swapped into place as the new primary.

''Our data warehousing team has been doing significant work over the past year to ensure that the ever increasing amount of data being logged about simulator and other system performance can be analyzed, and that the collection of this data is "transparent" to the other systems - logging database failures should no longer cause service disruptions.''

== Transient Data Services ==

'''What is it:''' A cluster of machines (currently: 16) that store data in memory for "transient" state. This includes things like agent presence ("who is logged on?"), group chat participation, inbound email to script mapping, and so forth. This data is not stored in a database and is either constantly refreshed (e.g. simulators update agent presence every few minutes) or otherwise recoverable (e.g. rejoining a group chat).

'''How it can fail:''' Hardware failure can reduce the capacity of these machines, or take them offline entirely. Software bugs can also cause poor performance - for example, a memory leak in a service could cause the services to start responding slowly. While the specific service is disrupted, the overall service remains functional.

'''How we fix it:''' Because the state is transient, a replacement host can be brought online quickly and the data "heals" itself over time. If the error is software-side the services can be restarted as soon as a fix for the bug is found, with little impact to residents.

== Simulators ==

'''What is it:''' A [[Glossary#Sim|sim]] is a machine that runs [[Glossary#Simulator|simulators]], which are the computer processes that runs [[region|regions]]. (Think of a ''region'' like a document, the ''simulator'' like a word processor, and the ''sim'' as the computer itself that runs the program.) Since these are closely coupled concepts, jargon/terminology tends to be somewhat loose, e.g. "simstate" should really be "region state". The simulator divides its time between communicating with viewers, communicating with other system components, simulating physics and executing scripts.

'''How it can fail:''' A bug in the simulator code can cause a crash. Most crashes cause a region simstate to be saved, and another simulator will load that simstate after a few minutes. Often, the bug is triggered by some of the region content - a script or physical object.

Other problems fall into two categories - problems with a specific sim, or grid-wide. Problems with a specific sim may include overloading (e.g. 4 high-traffic regions on the same sim) or failures (disk full, network interfaces lost, hardware failure). Grid-wide problems are usually caused by the other factors listed here, such as loss of network or database or asset cluster failure (either of which, for example, could prevent simulators from loading simstates. New simulator code releases occasionally introduce bugs with grid-wide consequences (e.g. excessive logging causing network traffic congestion)

'''How we fix it:''' Simulator crashes are reported just as viewer crashes are. We can use the data to determine the general subsystem that caused the crash (initialization, physics, scripts, messaging, etc.) If the crash is caused by content (script or physical object) we can use the crash data to determine why this occurred. In the mean time (since the fix may take several days or, in the case of physics, a project like the move to Havok 4) the region is brought up with scripts/physics disabled and the offending objects removed.

Problems with a specific sim can be addressed by restarting the regions, which causes a simulator process on a different sim to run it. Grid-wide problems are fixed either at the source (e.g. repair the network). Bugs from new code releases require either a configuration change (to turn off a new feature) or a rolling restart with updated code.

''Reducing simulator crashes was the main motivation behind the movement to [[Havok4]] for physics simulation, and the upcoming move to [[Mono]] for script execution.''

== Dataservers ==

'''What is it:''' Most of the simulator to database communication proxies through a process called "dataserver"; there are a few dataserver processes on each sim host. This eliminates a direct dependency on the database and allows the dataserver to block on a lengthy query while the simulator targets a fixed frame rate.

'''How it can fail:''' The dataserver process can crash as a result of bugs related to unforseen circumstances. For example, if the network hiccups, a connection to a database may be lost. Usually the system recovers gracefully and transparently from a dataserver failure, but on a particular simulator some transactions may fail temporarily. The service disruption is localized to the specific simulator. It is also possible that a software update could introduce bugs that cause grid-wide effects (for example, increased load on the central database cluster, or just more frequent crashes.) When a database is not responding to connections, the dataserver process watcher will automatically stop and restart the dataserver so new requests can be services.

'''How we fix it:''' When an individual dataserver crashes, it is automatically restored. If a bug is introduced that causes grid-wide effects the dataserver processes can usually be replaced without downtime.

''The dataserver component is being phased out and replaced with web dataservices; simulators will use HTTP to talk to a new set of hosts that in turn relay queries to the database. This will allow us to more easily tweak the system to improve performance and eliminate disruptions.''

== Login Server Cluster ==

'''What is it:''' A cluster of servers which represent the first service that the viewer connects to when attempting to log in. This validates the resident's credentials, checks the viewer version for possible updates, ensures the latest Terms of Service have been updated. Assuming those check out, it sends the viewer an initial overview of the resident's inventory folders and a few other chunks of data. Finally, it negotiates with the simulator for the requested start location and lets the viewer know which simulator to talk to.

'''How it can fail:''' If one drops offline, some percentage of logins will fail. Additionally, since the login sequence is database-intensive, if the central database or inventory database cluster are having problems then logins will also fail. Finally, after a major disruption that leads to many Residents being kicked or unable to connect, there may be more Residents trying to connect than our Second Life can handle (roughly 1000 logins/minute); this can appear to Residents trying to log in as though the login service is failing, even though it is fully functional and just at maximum capacity.

'''How we fix it:''' If a login server itself fails, we take it out of rotation. If the problem is in another system or service, we fix it there.

== Web Site ==

'''What is it:''' A cluster of machines that serve the web pages and web services exposed to the public - including secondlife.com, lindenlab.com, slurl.com, etc.

'''How it can fail:''' Hardware failures can slow down or shut down a machine in the web cluster. In that case, a load balancer should automatically redirect web traffic away from machines that are performing poorly, but the load balancer itself may have bugs (e.g. it may not detect such failures properly, or itself become blocked up). Web site bugs can be introduced by code updates to the web site, which are made daily. In addition, the web site relies on the central database cluster for many service actions, so failures there will affect web site actions such as the LindeX and transaction history, land store, friends online, and so forth.

'''How we fix it:''' Problematic hardware can be taken out of rotation to restore the responsiveness of the web site. Problems in other systems such as the central database cluster need to be addressed there.

== Linden Network ==

'''What is it:''' The tubes through which stuff travels. Most notably, the connections between our co-location facilities ("colos"), e.g. SF and Dallas, but also the plumbing within colos. This includes "VPNs", switches, routers, and other esoteric stuff. Some of this is Linden equipment, some of this is leased equipment (e.g. we pay a third party to have dedicated use of their "tubes" between our colos), and public Internet pipes are also used.

'''How it can fail:''' A component can go bad, for example, a router can start dropping packets. This often appears as one of the other problems (asset storage, database, simulators, logins) since the systems can no longer talk to each other. The [http://blog.secondlife.com/2008/04/05/logins-disabled-until-further-notice/ failure on April 5th, 2008] is an example of this kind of failure.

'''How we fix it:''' isolate the affected component and take it out of service or replace it as quickly as possible. If this is a leased component we need to talk to our provider.

== Internet ==

'''What is it:''' A [http://en.wikipedia.org/wiki/Series_of_tubes series of tubes] that bring Second Life to your computer, from the large trans-oceanic and trans-continental pipes that link the world down to high-speed connection to your home from your Internet Service Provider (ISP).

'''How it can fail:''' Failures occur on several levels. If this happens at a high level - for example, a major Internet trunk to Europe drops offline - thousands of residents can be disconnected from Second Life.

'''How we fix it:''' This is usually beyond our control. If we can isolate the problem we can report it to network contacts, but otherwise we just need to wait for the issue to get fixed, like the residents.

Service Disruptions

2008-04-09T23:26:37Z

Joshua Linden: /* Dataservers */

Second Life is a complex system with many inter-operating components, from simulators and databases to the viewer you run, and the Internet connections over which data flows. Although it is intended to be as [http://en.wikipedia.org/wiki/Loose_coupling loosely coupled] as possible and resilient against problems, events can still occur which lead to service disruptions.

In some cases the failures are beyond the control of Linden Lab. However, in nearly all cases there is active work being done to mitigate the disruptions - either prevent them from happening or significantly reduce the impact.

When a disruption occurs, the following sequence usually occurs:
* a system stops responding
* automated notifications go off, alerting our operations team
* residents often notice immediately, and alert in-world support, who confirm the problem to our operations team
* the operations team identifies the system that is the root cause of the problem
* the communications team is notified, and asked to provide information about the disruption to the blog
* if the disruption lasts more a few minutes, updates are made to the blog regularly
* once the problem is solved, an "all clear" is reported to the blog

Note that in many cases a disruption may be solved before any information can make it to the blog explaining the details of the problem. One purpose of this document is to provide a "clearing house" for types of service disruptions, so that in the event of a system failure the blog post can reference this page.

At the time of this writing, and in no particular order, these are the systems which have been known to cause service disruptions:

== Asset Storage Cluster ==

'''What is it:''' A cluster of machines that form a whopping huge [http://en.wikipedia.org/wiki/WebDAV WebDAV] (think "web-based disk drive") storage mechanism with terabytes of space for storing assets, including uploaded textures, snapshots, scripts, objects taken into inventory, script states, saved region states (simstates), etc that make up Second Life. The technology (software and hardware) is licensed from a third party.

'''How it can fail:''' The system should be resilient against single node failures. In the case of multiple disk failures, software upgrades, removing problem nodes or adding new nodes, some or all of the cluster can fall offline. If this happens, asset uploads and downloads fail - this causes texture uploads and simstate saves to fail. Since transient data during region crossings (attachment states, etc) are written as assets, region crossings will also often fail.

'''How we fix it:''' When detected, we often disable logins and message in-world (if possible) to help avoid data loss. Failed nodes can be taken out of rotation. A restart of other nodes may be necessary. When upgrading the software on the nodes, the grid is usually closed to prevent data loss during any inadvertent outages. An asset system failure requiring a reboot occurred on [http://blog.secondlife.com/2008/03/28/asset-server-issues/ March 28th, 2008]

== Central Database Cluster ==

'''What is it:''' A cluster of databases that store the core persistent information about Second Life - including resident profiles, groups, regions, parcels, L$ transactions and classifieds.

'''How it can fail:''' The database can become loaded enough during normal operations that some fraction of transactions fail and either must be manually retried or are automatically retried. Hardware failure and software bugs in the database code can also cause the database to crash or stop responding. Logins will fail, transactions in-world and on the web site will fail, and so forth.

'''How we fix it:''' If the primary database fails, we swap to one of the secondaries. If the database load is high but hasn't failed we can turn off services to try and reduce the load.

''Eliminating this cluster as a scalability bottleneck and failure point is a very high priority for Linden Lab. While this is in progress, load mitigation is occurring. Watch the [http://blog.secondlife.com/2008/04/07/profile-and-group-services-temporarily-unavailable/ Second Life Blog] for updates''

== Agent ("Inventory") Database Cluster ==

'''What is it:''' Storage for most agent-specific data such as the inventory tree is partitioned across a series of databases. Each agent is associated with a particular inventory partition (a primary database and its secondary backups). At the time of this writing, we have approximately 15 agent database partitions.

The initial use of these agent-partitioned databases was for inventory, so they are often referred to as "inventory databases" by Lindens, but this is no longer the extent of what agent-specific data is stored within them.

'''How it can fail:''' Hardware or software failure can affect the primary database within the partition, so that it either stops responding to queries or becomes excessively slow.

'''How we fix it:''' When an agent database fails we can swap to the backup within that partition, which takes a few minutes. If this will not happen immediately or if problems are encountered, that particular agent partition is "blacklisted" temporarily; this causes logins of agents who are associated with that partitions to be blocked and any agents logged in are "kicked", while the fix is in progress. This will affect some fraction of the grid, but not everybody.

''This is an example of a system that in the past was prone to causing global service disruption. It was re-designed to limit the impact to Residents even in the face of hardware failure; only Residents associated with a particular partition are affected during such a failure.''

== Other Database Clusters ==

'''What is it:''' There are a handful other database clusters in use. One is used for logging data.

'''How it can fail:''' Hardware or software failures can take a database cluster offline. There should be no in-world effect from one of these other database clusters failing, but occasionally a software design flaw does introduce a dependency that is not caught. For example, logins used to require a successful connection to the login database to record the login and viewer statistics, but this dependency has been removed.

'''How we fix it:''' All databases act in clusters with a primary machine and several secondaries. In case of failure, a secondary can be swapped into place as the new primary.

''Our data warehousing team has been doing significant work over the past year to ensure that the ever increasing amount of data being logged about simulator and other system performance can be analyzed, and that the collection of this data is "transparent" to the other systems - logging database failures should no longer cause service disruptions.''

== Transient Data Services ==

'''What is it:''' A cluster of machines (currently: 16) that store data in memory for "transient" state. This includes things like agent presence ("who is logged on?"), group chat participation, inbound email to script mapping, and so forth. This data is not stored in a database and is either constantly refreshed (e.g. simulators update agent presence every few minutes) or otherwise recoverable (e.g. rejoining a group chat).

'''How it can fail:''' Hardware failure can reduce the capacity of these machines, or take them offline entirely. Software bugs can also cause poor performance - for example, a memory leak in a service could cause the services to start responding slowly. While the specific service is disrupted, the overall service remains functional.

'''How we fix it:''' Because the state is transient, a replacement host can be brought online quickly and the data "heals" itself over time. If the error is software-side the services can be restarted as soon as a fix for the bug is found, with little impact to residents.

== Simulators ==

'''What is it:''' A [[Glossary#Sim|sim]] is a machine that runs [[Glossary#Simulator|simulators]], which are the computer processes that runs [[region|regions]]. (Think of a ''region'' like a document, the ''simulator'' like a word processor, and the ''sim'' as the computer itself that runs the program.) Since these are closely coupled concepts, jargon/terminology tends to be somewhat loose, e.g. "simstate" should really be "region state". The simulator divides its time between communicating with viewers, communicating with other system components, simulating physics and executing scripts.

'''How it can fail:''' A bug in the simulator code can cause a crash. Most crashes cause a region simstate to be saved, and another simulator will load that simstate after a few minutes. Often, the bug is triggered by some of the region content - a script or physical object.

Other problems fall into two categories - problems with a specific sim, or grid-wide. Problems with a specific sim may include overloading (e.g. 4 high-traffic regions on the same sim) or failures (disk full, network interfaces lost, hardware failure). Grid-wide problems are usually caused by the other factors listed here, such as loss of network or database or asset cluster failure (either of which, for example, could prevent simulators from loading simstates. New simulator code releases occasionally introduce bugs with grid-wide consequences (e.g. excessive logging causing network traffic congestion)

'''How we fix it:''' Simulator crashes are reported just as viewer crashes are. We can use the data to determine the general subsystem that caused the crash (initialization, physics, scripts, messaging, etc.) If the crash is caused by content (script or physical object) we can use the crash data to determine why this occurred. In the mean time (since the fix may take several days or, in the case of physics, a project like the move to Havok 4) the region is brought up with scripts/physics disabled and the offending objects removed.

Problems with a specific sim can be addressed by restarting the regions, which causes a simulator process on a different sim to run it. Grid-wide problems are fixed either at the source (e.g. repair the network). Bugs from new code releases require either a configuration change (to turn off a new feature) or a rolling restart with updated code.

''Reducing simulator crashes was the main motivation behind the movement to [[Havok4]] for physics simulation, and the upcoming move to [[Mono]] for script execution.''

== Dataservers ==

'''What is it:''' Most of the simulator to database communication proxies through a process called "[[dataserver]]"; there are a few dataserver processes on each sim host. This eliminates a direct dependency on the database and allows the dataserver to block on a lengthy query while the simulator targets a fixed frame rate.

'''How it can fail:''' The dataserver process can crash as a result of bugs related to unforseen circumstances. For example, if the network hiccups, a connection to a database may be lost. Usually the system recovers gracefully and transparently from a dataserver failure, but on a particular simulator some transactions may fail temporarily. The service disruption is localized to the specific simulator. It is also possible that a software update could introduce bugs that cause grid-wide effects (for example, increased load on the central database cluster, or just more frequent crashes.) When a database is not responding to connections, the dataserver process watcher will automatically stop and restart the dataserver so new requests can be services.

'''How we fix it:''' When an individual dataserver crashes, it is automatically restored. If a bug is introduced that causes grid-wide effects the dataserver processes can usually be replaced without downtime.

''The dataserver component is being phased out and replaced with web dataservices; simulators will use HTTP to talk to a new set of hosts that in turn relay queries to the database. This will allow us to more easily tweak the system to improve performance and eliminate disruptions.''

== Login Server Cluster ==

'''What is it:''' A cluster of servers which represent the first service that the viewer connects to when attempting to log in. This validates the resident's credentials, checks the viewer version for possible updates, ensures the latest Terms of Service have been updated. Assuming those check out, it sends the viewer an initial overview of the resident's inventory folders and a few other chunks of data. Finally, it negotiates with the simulator for the requested start location and lets the viewer know which simulator to talk to.

'''How it can fail:''' If one drops offline, some percentage of logins will fail. Additionally, since the login sequence is database-intensive, if the central database or inventory database cluster are having problems then logins will also fail. Finally, after a major disruption that leads to many Residents being kicked or unable to connect, there may be more Residents trying to connect than our Second Life can handle (roughly 1000 logins/minute); this can appear to Residents trying to log in as though the login service is failing, even though it is fully functional and just at maximum capacity.

'''How we fix it:''' If a login server itself fails, we take it out of rotation. If the problem is in another system or service, we fix it there.

== Web Site ==

'''What is it:''' A cluster of machines that serve the web pages and web services exposed to the public - including secondlife.com, lindenlab.com, slurl.com, etc.

'''How it can fail:''' Hardware failures can slow down or shut down a machine in the web cluster. In that case, a load balancer should automatically redirect web traffic away from machines that are performing poorly, but the load balancer itself may have bugs (e.g. it may not detect such failures properly, or itself become blocked up). Web site bugs can be introduced by code updates to the web site, which are made daily. In addition, the web site relies on the central database cluster for many service actions, so failures there will affect web site actions such as the LindeX and transaction history, land store, friends online, and so forth.

'''How we fix it:''' Problematic hardware can be taken out of rotation to restore the responsiveness of the web site. Problems in other systems such as the central database cluster need to be addressed there.

== Linden Network ==

'''What is it:''' The tubes through which stuff travels. Most notably, the connections between our co-location facilities ("colos"), e.g. SF and Dallas, but also the plumbing within colos. This includes "VPNs", switches, routers, and other esoteric stuff. Some of this is Linden equipment, some of this is leased equipment (e.g. we pay a third party to have dedicated use of their "tubes" between our colos), and public Internet pipes are also used.

'''How it can fail:''' A component can go bad, for example, a router can start dropping packets. This often appears as one of the other problems (asset storage, database, simulators, logins) since the systems can no longer talk to each other. The [http://blog.secondlife.com/2008/04/05/logins-disabled-until-further-notice/ failure on April 5th, 2008] is an example of this kind of failure.

'''How we fix it:''' isolate the affected component and take it out of service or replace it as quickly as possible. If this is a leased component we need to talk to our provider.

== Internet ==

'''What is it:''' A [http://en.wikipedia.org/wiki/Series_of_tubes series of tubes] that bring Second Life to your computer, from the large trans-oceanic and trans-continental pipes that link the world down to high-speed connection to your home from your Internet Service Provider (ISP).

'''How it can fail:''' Failures occur on several levels. If this happens at a high level - for example, a major Internet trunk to Europe drops offline - thousands of residents can be disconnected from Second Life.

'''How we fix it:''' This is usually beyond our control. If we can isolate the problem we can report it to network contacts, but otherwise we just need to wait for the issue to get fixed, like the residents.

2007-07-19T18:43:19Z

Joshua Linden:

{{CompileNav}}

The public [[version control repository]] provided by Linden Lab has many of the same branches that Linden Lab's internal repository has.

There are generally four major branches at all times, along with many temporary development branches:

* '''Numbered branch (currently "Branch_1-18-0")''' - this is where the current release resides, and where very minor fixes get checked in for the next minor release (e.g., if 1.15.0.2 is the last release, then changes for 1.15.0.3 would get checked in here).
* '''"release"''' - the main trunk of development. This is often referred to as "head" or "trunk" in other organizations. Linden Lab is rather conservative about what gets checked into release, though, preferring that development work mainly happens on branches (either a dedicated feature branch, or "maintenance" for the small stuff, and only gets merged into release once QA has happened. We aspire to keep release compatible with the agni (main) grid. All branches occasionally pick up changes from release, and are occasionally merged back into release.
* ''DEPRECATED:'' '''"release-candidate"''' - when Linden Lab is working toward a major release, this branch provides a similar function to release, but is for changes that are compatible with the beta grid (usually aditi) rather than the release grid. ''Following the release of 1.18, this branch has been deprecated. Incompatible changes go directly into numbered branches.''
* '''"maintenance"''' - bug fixes and small features that don't warrant their own branch get checked into maintenance. Changes from maintenance are periodically merged up to release after QA has tested a build from the branch. Many source code contributions from the community land in this branch first.

Other branches are typically branched from release or release candidate.

Linden Lab does feature development on temporary branches. As those features mature, they are merged into the "release-candidate" branch. The [[QA_Portal|QA group]] tests the feature, and often, the feature is deployed on the beta grid for public testing. QA will then either reject or approve it for inclusion in "release". When a major release is about to happen, a branch is created for work on that release (e.g. "Branch_1-13-2"), which gets synced with the "release" branch periodically. Then, stability work is done on the branch, and the new code is deployed. After that, any stability work gets merged back into "release", and the process repeats for the next major release. In the meantime, stability work continues on the branch for maintenance releases.

Inventory UI Design

2007-03-13T02:31:23Z

Joshua Linden: /* Email */

{{OSWikiContribBox}}
== Thoughts on Inventory UI in SL ==
By [[User:Kelly Linden|Kelly Linden]] 14:01, 12 March 2007 (PDT)

Right now our inventory is already too complicated for new users with few items and not powerful enough for advanced users with thousands of items. In the end we end up with an inventory that is not good for anyone. At the same time, making it more complex, adding filters and links and more and more is not likely to make it really usable for anyone but the most hard core.

I think something drastic is probably needed to make our inventory actually work, and I'd love to be part of a discussion about it.

== What do other's do? ==
There are existing software systems that have the same problem our inventory does (lots of files). A couple of those are File Systems and Email.

=== OSX Applications ===
Apple has shown a will to put ease of use before any other priority so it makes sense to look a bit at some things they do.

One thing I do really like is how they handle applications. An application on windows is a single executable file burried somewhere with all the other files needed to run the program, and linked to from your desktop or start menu. In OSX an Application is *actually* a folder. Usually it doesn't look like one and it doesn't act like one. You double click it to start the app, and it is the only thing you have to move around if you want the app somewhere else. There are no links (though you can make them if you want) and you only see at the top level the data you need. At the same time an advanced user can browse the contents to see the actual executable(s) and other files needed for running the application.

=== Windows Start Menu ===
Windows has its cruft and kludge but they do get some things right. In particular the Start menu contains a list of recently or frequently used applications. I don't even know the algorithm used to decide what goes there, but the applications I use most are usually in that short list.

=== File Systems (more generally) ===
Both Windows and Mac, and even Linux, limit the locations user created files generally go. Your files go on the desktop or Documents (My Documents) or your home directory. These locations are easy to get to, relatively few and they don't contain any of that 'extra' stuff (programs, config info) that is actually filling your computer.

=== GUI Filesystems ===
GUI Filesystems support icons. Any recent incarnation supports ''custom'' icons on a per application basis. More recent versions even support thumbnail icons based on the actual data in the file. All these help to quickly and easily visually sort a collection of data and find specific items within the larger set.

=== Email ===
Email has it's own problems. However it is a (mostly) full text medium which makes searching it easier. All email clients highlight 'new' items - those that have not yet been viewed - within the normal list. Additionally Google's gmail and 'Threaded' views in traditional email readers automatically attempt to group related data.

Recent versions of Outlook (2003 and 2007) are possibly worth looking at, as they they try and deal with making an inbox with hundreds/thousands of messages more usable:
* Two-line view - adapt to thin window for message list by using two lines of text to display Subject, From, Date, Status, Flags and Size in < 200px of width.
* Smart grouping - by name, date (e.g. today, yesterday, last week, last month)
* Persistent "Search Folders" (also seen in Windows Media Player) which preserve searches
* Single-click to flag items as important

== Design Ideas ==
=== Packaging, Containers, Outfits and Items ===
A new class of inventory items that are containers. Containers all:
* Can contain an 'icon' image, which if present is the icon (at 16x16 probably) for the item in inventory.
* Can contain a 'help' notecard, which can be opened by context menu for the container (right click -> help)
* Have their own edit interface for adding or removing items
Specific Inventory Types:
* New inventory type '''Outfit'''
** Contains all the attachments, body parts, clothing and gestures to make the outfit.
** Outfits when worn are atomic.
*** If you attach an outfit that has any piece that conflicts with an already worn outfit then the entire previous outfit is removed.
*** If the overlap is small (1 piece out of 10 for each of two 10 piece outfits for example), potentially warn user about removing the old outfit.
* New inventory type '''Item''' (or 'Bundle', 'Package', 'Set' etc)
** A container object that can hold many different things, including outfits but maybe not other Items.
** May contain meta data about what to do for various actions - drag into world, drag onto avatar etc.
*** For example: When dragged onto an avatar attach this outfit, rez this vehicle on the ground, attach these gestures.
*** Defaults should make sense - dragging an item onto your avatar that contains an outfit will wear the outfit.
* New inventory type '''Container'''
** An easy generic container type.
** Can't be put into the world
** Dragging it *anywhere except another avatar* opens UI for seeing what is inside
*** Existing or improved 'Open' interface.

=== Icons and Previews ===
* Containers contain the icon as described above
* Other items - objects, notecards, scripts - do not get to set an icon.
* Icons are only shown at 16x16 in inventory by default.
* Preview mode will show large icons (32x32 or 64x64)
* Optional preview pane will show large icon (64x64) for the selected item or the item under the cursor.
* Images, although they can not set a seperate icon, can be their own icon in the 'Preview' modes, but not the normal 16x16 mode.
* (no copy) (no mod) (no transfer) are replaced by an icon:
** A fully permissive object has no extra icon.
** There is 1 icon each for each permissions combination
=== Directory Structure ===
* Always sort Special Folders to the top.
* New base folder ''''Images'''' replaces ''''Textures'''' and contains ''''Photo Album''''
* New base folder ''''Clothing'''' contains ''''Clothes & Bodyparts'''', ''''Gestures & Animations''''
** Outfits go into the ''''Clothing'''' folder itself
* New base folder ''''Items'''' contains ''''Objects'''', ''''Scripts'''', ''''Notecards'''', ''''Lost and Found'''', ''''Sounds''''
** Items and Bags/Containers go into the base ''''Items'''' folder
* Calling cards go away completely - any missing features are added to Friends and IM interfaces
=== Misc ===
* Trash gets cleaned every login of items older than X days. Defaults to 30, definable in Preferences.

=== Recently Used ===
* List the last five outfits worn ''in a menu''.
* The menu could be part of the bottom bar and pop up, or part of the inventory window.
** The main top menu needs its own roasting before adding this there.
* List the last five locations teleported to ''in a menu''.

=== New and Unread ===
* Any item that has not yet been rezed or viewed is shown in bold or otherwise highlighted.
* Optional filter to only show those items which have not been opened.
* Right click -> 'Mark as Viewed' option to un-highlight the item.

=== Tagging, Searching, Filters ===
Alright, we can get all web 2.0 on this ;)
* Every inventory item gets a new text field 'Tags'
** Comma separated list of tags
** Always modifiable by the owner, independant of permissions
** Default 'tags' could be the description, with spaces replaced with commas.
* Search finds by tag, name, owner or creator - who cares! If you type it, find the related items.
** Searching your inventory for Chair will work! Even if your chair is named 'Ultra Recliner IV'
* Replace the Filter window with an icon to the left of the search box that drops down a menu when clicked.
** The menu contains all the object types (Textures, Sounds etc) and All (default)
*** No you can't filter for multiple types.
*** Maybe an exception for Textures and Snapshots, or replace the 2 types in the filter with 'Images'
[[Category: Design Discussions]]