Second Life Wiki - User contributions [en]

Asset Wrapper

2010-05-27T00:48:04Z

Poppy Linden:

{| border="1"
! Reference !! extension !! id !! mime type !! LSL
|-
{{Asset Type
|wiki_page=Asset Wrapper
|extension=asset
|id=unassigned
|mime_type=application/vnd.ll.asset
}}
|}

= Requirements =
A simple container format has been invented to wrap assets in a somewhat flexible and concise manner. Specifically we needed a format which:

* provides features for attribution.
* the meta-data needs to be in the first network packet. (specifically a requirement of texture attribution)
* supports arbitrary meta-data.
* introduces minimal byte overhead and does not require increased payload size.

Other container formats were considered but rejected for one of:

* too much overhead
* poor cross-language and cross-platform support
* geared for streaming content rather than fixed assets.

= Specification =
The asset type is an [[LLSD]] array of arrays. Each sub-array will contain a [[LLSD]] map of meta-data followed by the data segment. Asset files are serialized in binary [[LLSD]] unless a header in asset file indicates otherwise.

The asset file:
* '''must''' be an array of arrays.
* '''should''' be binary
* '''may not''' have a header to indicate binary serialization
* '''must''' indicate if they are not serialized in the binary format in the first line of the file.
* '''should not''' contain more than one segment of a particular content-type.

The meta-data segment:
* '''must''' contain the 'content-type' of the data.
* '''should''' contain a 'creation-date' of the data.
* '''should not''' contain a 'content-length'.

= examples =
This is a [[LLSD#Notation_Serialization|notation based]] mock-up of what an asset will actually look like:
<pre>
[[{'content-type':'image/x-j2c',
'creation-date':'2007-01-01T00:01:00Z',
'creator-id':u3c115e51-04f4-523c-9fa6-98aff1034730},
<BINARY IMAGE DATA>]]
</pre>

This example shows two different wrapped assets. Note that the first binary blob is serialized as raw data and the curly braces are inside the 160 bytes of content. The vnd.ll.lslbyte presented here is a (somewhat truncated) base64 encoding of the byte code generated by compiling the related vnd.ll.lsltext.
<pre>
[
[
{
'content-type':'application/vnd.ll.lsltext',
'creation-date':'2007-03-15T18:30:00Z',
'creator-id':u3c115e51-04f4-523c-9fa6-98aff1034730
},
b(160)"default
{
state_entry()
{
llSay(0, "Hello, Avatar!");
}

touch_start(integer total_number)
{
llSay(0, "Touched.");
}
}"
],
[
{
'content-type':'application/vnd.ll.lslbyte',
'creation-date':'2007-03-15T18:30:18Z',
'creator-id':u3c115e51-04f4-523c-9fa6-98aff1034730
},
b64"AABAAAAAAAAAAAIAAAA//wAAP/8AAADgAAAA5wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AABkAAAAZAAAAAAAAAAAAAAAZAAAAAAAAAABAAAAAAAAAAAAAAAAAAAABQAAAAEAAAAQAAAAAAAA
AAUAAAAFAAAAABAAAAAAAAAAPgAAAAQAAAAFAGNbXgAAAABgSGVsbG8sIEF2YXRhciEAZgAAAABc
XgAAAAhwEQjRABeVAAAABQBjW14AAAAAYFRvdWNoZWQuAGYAAAAAXF4AAAAIcBEI0QAXAZUAAEAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
]
]
</pre>

[[Category:Assets]]

Gestures

2010-05-27T00:47:47Z

Poppy Linden:

{{KBmaster}}
{{Navbox/Controls}}

== What are gestures? ==

Gestures are an inventory item type (shown as [[Image:Inv_item_gesture.png]]) that allow you to '''create a [http://en.wikipedia.org/wiki/Macro_%28computer_science%29 series of steps] which trigger your avatar to animate, play sounds, and emit text chat'''. These can be used for practical purposes or to amuse friends. For example, a cartoon "Squish!" noise accompanied by your avatar falling down. In Second Life [[culture]], many gestures are openly circulated like [http://knowyourmeme.com/ memes], as they add a lot of spark to social situations.

Gestures can be triggered with:

* Keyboard shortcuts
* Text chat words/phrases
* Voice volume.

== Where do I get gestures? ==

There are a variety of sample gestures in your inventory's Library folder. Search your inventory for "<code>gesture</code>" and scroll down.

You can also [https://www.xstreetsl.com/modules.php?name=Marketplace&CategoryID=233 purchase many readymade gestures at Xstreet]. [[:Category:Xstreet|Learn more about Xstreet.]]

== How do I make a gesture? ==

Many variations exist, but here's a simple exercise to get you started.

# Click the '''Inventory''' button in the bottom right of the Viewer window to open the Inventory window.
# Right-click the '''Gestures''' folder (if you don't see it, open the '''My Inventory''' folder) and select '''New Gesture''' from the context menu.
# Type in a unique name for the gesture. You can always change this later. The Gesture editor window appears:
#: [[File:Gesture_editor_window.png|640px]]
# In the Gesture window, under '''Steps''', there are example steps. Click '''Preview''' button to see and hear what the gesture does as a whole. For example:
## Click the first step, '''Start Animation: Wave'''.
## Click the dropdown below and select a different animation, like '''Afraid'''. The steps update to show this.
## Click '''Preview''' again.
# Add and remove steps using the '''Add >>''' and '''Remove''' buttons.
# Change the order of the steps by selecting a step, then clicking '''Move Up''' and '''Move Down'''.
# As-desired, enter a '''Trigger''' and/or '''Shortcut Key'''. Changing the '''Description''' is optional, but you should at least have one way to easily trigger the gesture.
# Make sure '''Active''' is checked. Gestures must be active to be triggered.
# When you're done, click '''Save''' button and close the gesture window. Test the trigger or shortcut key.

You should also look at existing gestures to understand how to construct them from scratch.

Since gestures are best shown in action, watch this video tutorial to learn more:

{{KBvideo|4168991}}

Also, did you know gestures can be used as a productivity tool? With text chat macros, you can type in abbreviations which expand into phrases, or even paragraphs.

{{KBvideo|4242104}}

== How do I see all my active gestures? ==

There are several ways:

* Search your inventory for "<code>(active)</code>".
* There's a dropdown called '''Gestures''' to the right of your chat bar. Click it to see gestures sorted by text trigger.
* Go to '''View''' > '''Active Gestures'''. A special window opens that shows you the trigger, key, and name for all your activated gestures.

== Why don't my gestures work? ==

You're either not using the correct trigger or they're deactivated. Gestures ''must'' be activated to be triggered. Activated gestures appear in '''bold''' in your inventory and have "(active)" next to their names.

There are a few ways to activate gestures:

* Right-click a gesture and choose '''Activate'''.
** You can also Shift-click a range of gestures, then choose '''Activate''' to activate multiple gestures at once.
* Drag a gesture from your inventory onto your avatar to activate it.
** This also works with folders containing gestures. It works similar to wearing clothes or attaching objects — active gestures are effectively "worn".

Conversely, you can follow the same steps, then choose '''Deactivate''' to turn off gestures.

== Why do I sound like a woman if I have a male avatar (and vice-versa)? ==

The default gestures come in two folders, one for '''Male Gestures''' and another for '''Female Gestures''' depending on which gender you picked when you first logged into Second Life. Their steps are the same except for the sound files they reference. You can always switch to the other by as described above.

If you have a microphone, you can record your own voice, upload sound clips, then use them in gestures for an extra-personal touch.

== Why can't I use an animation or sound in a gesture? ==

In other words, why doesn't an animation or sound appear in the dropdown list when you're editing a gesture?

Check if it has full permissions. '''Animations and sounds must have ''full'' permissions to be included in a gesture.''' This is because a gesture functions like a "wrapper" that references individual animations and sounds. If an animation was no-transfer but the gesture could include it, that'd be an unfair way to work around the creator's permissions. A similar case applies for non-copyable animations and sounds.

It's true you can't do much to modify animations and sounds after uploading them aside from renaming and changing their description, but the rule still holds true.

== Can I trigger multiple gestures in a single line of chat? ==

No. The Viewer only reads the first trigger. If you have many actions you want to do, you can include them with multiple actions in a single gesture.

If you want to show someone a gesture trigger, type it twice in text chat. For example, with the default "/bow" gesture activated, type: "<code>/bow /bow</code>". It appears as a single "<code>/bow</code>" — other avatars see you bowing but also see the trigger.

== How come my text trigger disappears? ==

In the Gesture editor window, the '''Trigger''' is what's used to start the gesture, but is independent from what shows up. For example, if you have "<code>/wave</code>" used to do a wave animation, you might want the text chat that appears to everyone else to be "<code>Friendly greetings!</code>.

If you want your trigger and the emitted text output (independent of further '''Steps''') to be the same, simply enter the same text next to '''Trigger''' and '''Replace with'''.

== How do I use voice levels to trigger gestures? ==

There exist ''special'' triggers to do this. They are:

<pre>/voicelevel1
/voicelevel2
/voicelevel3</pre>

Each basically correlates to the loudness (amplitude) of your incoming [[How_to_use_voice|voice chat]], from 1 being softest to 3 being loudest. When there are several speech gestures which share the same trigger, they're randomly shuffled to add variety.

To get started, search your inventory for the '''Speech Gestures''' folder (which is included in the '''Library''' > '''Gestures''' folder), then drag that folder onto your avatar. Then, try talking on voice chat, from quiet to loud. Your avatar's arms should gesticulate more dramatically when speaking loudly, as shown here:

{{KBvideo|4243627}}

{{KBnote|The '''Speech Gestures''' folder may not appear copied into your own inventory until after relogging. This is a bug.}}

Like any other gesture, you can edit a speech gesture. Leave its trigger alone, but you can change its steps. For more info, see these videos:

{{KBvideo|4168800}}

== Tips & tricks ==

For the experienced gesture connoisseur.

* Active gestures starting with a slash "<code>/</code>" in the chat bar are autocompleted. Typing a preceding slash also hides the fact you're typing from others.
* In the Gesture editor window, default animations appear at the top of the dropdown menu and are sorted alphabetically, while custom ones appear below and are also sorted alphabetically.
* While you can string many steps together, there's a limit to how much can be packed into a gesture. You'll get an error message when you try to Save if you have exceeded that limit.
* More than one gesture can share the same trigger, meaning both '''Trigger''' and '''Shortcut Key''', as shown with the speech gestures. Each time you use a shared trigger, it'll randomly pick one of the gestures — a great way to add life and variety. For example, if you have 4 different waving animations and trigger them all with "<code>/wave</code>", you can add nuances to your avatar body language.
* Trigger multiple gestures simultaneously to mix and mash them up.
* In an Animation step, the '''Start''' and '''Stop''' radio buttons are for looped animations like still poses and repetitive dances. If you don't want your avatar to groove into eternity, insert a '''Stop Animation''' step for the same animation sometime after the '''Start Animation''' one.
* Gestures can add emphasis for keywords. For example, if '''Trigger''' and '''Replace with''' are both "<code>cat</code>" and you have a gesture set to play a meowing sound with an animated swipe of your hand, every time you type "<code>cat</code>" in a sentence, you exhibit this feline behavior. Similarly, [http://en.wikipedia.org/wiki/Emoticon emoticons] can correspond with avatar body language. The Library's gesture provides many examples.
* Gestures can be connected to [[LSL Portal|scripts]]. If you have scripts which use channel commands like "<code>/1 activate</code>, these can be triggered via gesture faster than typing the long way.
* See the [[Limits#Gestures|Limits]] page for some numerical limits.
* Share gestures you've created with friends. It can be such a satisfying experience to trigger them one after another in a group. You know what they say about laughter being contagious!
** But at the same time, ''[http://secondlife.com/corporate/cs.php be considerate]''. ''Don't'' gesture-spam others if they don't want to hear you going "Woooooo!" for the 100th time.

== See also ==
* '''Link helpful pages here!'''

[[Category:Assets]]

Animations

2010-05-27T00:47:27Z

Poppy Linden:

{{OSWikiFeatureNav}}
=== Feature Design Document ===
(none)

=== Functional Spec ===
The format uses normalized key framed quaternions stored in shorts. The format is closer to that of Poser than {{HoverText|BVH|Biovision file format for animations}}. It consists of a header that tells the viewer what morphs to set, how long the animation plays and at what priority. After that there is an array of bones with each set of positions and rotations. The position array is only used in the root bone (or you can use it to animate the HUD, course there is no default animation for the HUD so once you do, you can't get it back into the default position). Both position & rotations use 3 shorts (16 bits * 3 = 6 bytes) per entry. For quaternions, it regenerates the 4th value since the quaternion is normalized.

=== Test scripts ===
[[Animation Test]]

=== Discussion for future improvements ===
(none)

=== Relationship to other features ===
 List of features that need to be tested when this feature changes, and why. 

[[Sitting]] - Make sure animations still work while sitting.

[[Region crossing]] - Make sure animations still work while sitting.. in vehicles.. while crossing a region boundary.

[[Scripted permissions]] - specifically, animation permissions.

[[llGetAnimationList]]

[[Asset Upload]] - If animation uploading was changed, verify other uploads still work.

=== User Guides ===
(none)

===See also===
* [[Animation]]

[[Category:Assets]]

LSO

2010-05-27T00:47:02Z

Poppy Linden:

{{LSL Header|ml=*}}{{RightToc}}
The compiled form of LSL scripts - a mostly stack-based language with a reference-counted heap and a very odd function calling convention in which the caller allocates storage for the callee's local variables. All the gory details can be found at http://www.libsecondlife.org/wiki/LSO, including some opcodes that are implemented in the interpreter but never outputted by the compiler (such as the handy DUP family of instructions for Forth-style programming, and PUSHIP/POPIP for implementing custom function calls/function pointers, jumptable-based switch statements and the like).

It's actually a reasonably powerful instruction set in some ways, but the compiler doesn't make use of it very well. Of course, with the planned switch to [[Mono]], it's probably really not sensible to do anything with it. Besides, rumor has it that script compilation will move server-side in the near future.

=Bytecodes=

{|
|style="text-align: center;"|'''Argument Types'''
|colspan="2" style="text-align: center;"|'''Variable type codes'''
|- vAlign="top"
|
{| {{Prettytable}}
|-{{Hl2}}
! Type
! title="Size in Bytes" | Size
!
|-
| dword
| 4
|
|-
| short
| 2
|
|-
| byte
| 1
|
|-
| type
| 1
| upper 4 bits are type.Left lower 4 bits are type.Right
|}
|
{| {{Prettytable}}
|-{{Hl2}}
! Value
! Type
! title="Size in Bytes" | Size
|-
| 0
| void
| ?
|-
| 1
| integer
| 4
|-
| 2
| float
| 4
|-
| 3
| string
| 4
|}
|
{| {{Prettytable}}
|-{{Hl2}}
! Value
! Type
! title="Size in Bytes" | Size
|-
| 4
| key
| 4
|-
| 5
| vector
| 12
|-
| 6
| rotation
| 16
|-
| 7
| list
| 4
|}
|}
==By Task==
=== Pop Value ===
{| class="sortable" {{Prettytable}}
|-{{Hl2}}
!Name
!Value
!class="unsortable"|Description
!class="unsortable"|arg0 size
{{LSO Bytecodes/pop-value}}
|}

=== Duplicate Value ===
{| class="sortable" {{Prettytable}}
|-{{Hl2}}
!Name
!Value
!class="unsortable"|Description
{{LSO Bytecodes/duplicate}}
|}

=== Store to Local ===
{| class="sortable" {{Prettytable}}
|-{{Hl2}}
!Name
!Value
!class="unsortable"|Description
!class="unsortable"|arg0 size
{{LSO Bytecodes/store-local}}
|}

=== Store to Global ===
{| class="sortable" {{Prettytable}}
|-{{Hl2}}
!Name
!Value
!class="unsortable"|Description
!class="unsortable"|arg0 size
{{LSO Bytecodes/store-global}}
|}

=== Load to Local ===
{| class="sortable" {{Prettytable}}
|-{{Hl2}}
!Name
!Value
!class="unsortable"|Description
!class="unsortable"|arg0 size
{{LSO Bytecodes/load-local}}
|}

=== Load to Global ===
{| class="sortable" {{Prettytable}}
|-{{Hl2}}
!Name
!Value
!class="unsortable"|Description
!class="unsortable"|arg0 size
{{LSO Bytecodes/load-global}}
|}

=== Push Local ===
{| class="sortable" {{Prettytable}}
|-{{Hl2}}
!Name
!Value
!class="unsortable"|Description
!class="unsortable"|arg0 size
{{LSO Bytecodes/push-local}}
|}

=== Push Global ===
{| class="sortable" {{Prettytable}}
|-{{Hl2}}
!Name
!Value
!class="unsortable"|Description
!class="unsortable"|arg0 size
{{LSO Bytecodes/push-global}}
|}

=== Push into List ===
{| class="sortable" {{Prettytable}}
|-{{Hl2}}
!Name
!Value
!class="unsortable"|Description
!class="unsortable"|arg0 size
{{LSO Bytecodes/push-list}}
|}

=== Push Zero Values ===
{| class="sortable" {{Prettytable}}
|-{{Hl2}}
!Name
!Value
!class="unsortable"|Description
!class="unsortable"|arg0 size
{{LSO Bytecodes/push-zeros}}
|}

=== Operators ===
{| class="sortable" {{Prettytable}}
|-{{Hl2}}
!Name
!Value
!class="unsortable"|Description
!class="unsortable"|arg0 size
{{LSO Bytecodes/operators}}
|}

=== Jump ===
{| class="sortable" {{Prettytable}}
|-{{Hl2}}
!Name
!Value
!class="unsortable"|Description
!class="unsortable"|arg0 size
!class="unsortable"|arg1 size
{{LSO Bytecodes/jump}}
|}

=== Return ===
{| class="sortable" {{Prettytable}}
|-{{Hl2}}
!Name
!Value
!class="unsortable"|Description
!class="unsortable"|arg0 size
{{LSO Bytecodes/return}}
|}

=== Stack to Heap ===
{| class="sortable" {{Prettytable}}
|-{{Hl2}}
!Name
!Value
!class="unsortable"|Description
!class="unsortable"|arg0 size
{{LSO Bytecodes/stack-to-heap}}
|}

=== Call Functions ===
{| class="sortable" {{Prettytable}}
|-{{Hl2}}
!Name
!Value
!class="unsortable"|Description
!class="unsortable"|arg0 size
{{LSO Bytecodes/functions}}
|}

=== Special Registers ===
{| class="sortable" {{Prettytable}}
|-{{Hl2}}
!Name
!Value
!class="unsortable"|Description
{{LSO Bytecodes/push-register}}
|-
{{LSO Bytecodes/pop-register}}
|}

=== Miscellaneous ===
{| class="sortable" {{Prettytable}}
|-{{Hl2}}
!Name
!Value
!class="unsortable"|Description
!class="unsortable"|arg0 size
{{LSO Bytecodes/misc}}
|}

== All Bytecodes ==
{| class="sortable" {{Prettytable}}
|-{{Hl2}}
!Name
!Value
!class="unsortable"{{!}}Description
!class="unsortable"{{!}}arg0 size
!class="unsortable"{{!}}arg1 size
{{LSO Bytecodes/misc}}
|-
{{LSO Bytecodes/pop-value}}
|-
{{LSO Bytecodes/pop-register}}
|-
{{LSO Bytecodes/duplicate}}
|-
{{LSO Bytecodes/store-local}}
|-
{{LSO Bytecodes/store-global}}
|-
{{LSO Bytecodes/load-local}}
|-
{{LSO Bytecodes/load-global}}
|-
{{LSO Bytecodes/push-local}}
|-
{{LSO Bytecodes/push-global}}
|-
{{LSO Bytecodes/push-register}}
|-
{{LSO Bytecodes/push-list}}
|-
{{LSO Bytecodes/push-zeros}}
|-
{{LSO Bytecodes/operators}}
|-
{{LSO Bytecodes/jump}}
|-
{{LSO Bytecodes/return}}
|-
{{LSO Bytecodes/stack-to-heap}}
|-
{{LSO Bytecodes/functions}}
|}

[[Category:Assets]]

Notecards

2010-05-27T00:46:24Z

Poppy Linden:

{{Help |Glossary=*}}

A '''notecard''' is an [[inventory]] item containing text and/or embedded [[texture]]s, [[snapshot]]s, [[object]]s, or other notecards. Items embedded in notecards must have copy/modify/transfer [[permission]]s.

The inventory [[icon]] for a notecard is [[Image:Inv_item_notecard.png]]. New notecards are stored in the inventory's "Notecards" folder [[Image:Inv_folder_notecard.png]] by default.

The maximum size of a notecard is 65,536 [[byte]]s.

==[[Open source]]==
The file format for a simple notecard (no embedded inventory items) looks like this:

<pre>Linden text version 1
{
LLEmbeddedItems version 1
{
count 0
}
Text length 21
Editing this notecard}
</pre>

==Scripting==
* [[:Category:LSL Notecard]]

== Video tutorial ==
* [[Video Tutorial/Notecard attachments]]

[[Category:Assets]]

Primitive

2010-05-27T00:45:44Z

Poppy Linden:

{{Help
|BugFixes=
|Avatar=
|Object=
|LandSim=
|Community=
|Viewer=
|Communication=
|Multimedia=
|Navigation=
|Misc=
|Description=
|Wiki=
|Glossary=*
}}

{{Otheruses4‎|primitives|[[LSL]] related information|:Category:LSL Prim}}

A '''primitive''', or '''prim''', is a single-part [[object]]. Multi-part objects will have multiple parts ("prims").

In [[Second Life]], virtual physical objects such as cars, houses, jewelry, and even less obvious things like [[hair]] and [[clothing]] are made out of one or more prims. Objects made from prims are usually created in-world using the built-in [[edit window|object editing tool]]. This tool is used for all sorts of 3D modeling in Second Life, playing the same role as 3D Max, Maya, or Blender, but customized for the Second Life way of doing things.

Each prim is represented by a set of parameters, including shape/type, [[position]], [[scale]]/size, [[rotation]], [[cut]], [[hollow]], [[twist]], [[shear]], etc. These parameters are sent from a [[server]] to the [[viewer]] running on the [[resident]]'s desktop, where the local video card is used to render the visual appearance of everything. (Rendering on the server would probably create a much higher amount of network traffic.)

The [[color]], [[texture]], [[bumpiness]], [[shininess]], and [[transparency]] of prims can also be adjusted, and images ([[textures]]) can be applied to each surface ([[face]]/side) of a prim to change its appearance. Box, cylinder, and prism prim shape types can also be made [[flexible]].

Prims can be linked together into [[link set]]s. They can also be attached to avatars, but this process is separate from (thought similar to) linking.

In the Viewer [[source code]], primitives are implemented in LLPrimitive, with vertex generation performed in LLVolume and rendering in LLVOVolume.

==Shape types==
There are eight primitive shape types ("Building Block Type" in build tools):
# box: all kinds of rectangular shapes
# cylinder: round tables, floors, long pipes
# prism: a box with one very small face
# sphere: can be used for ellipsoids of all shapes
# torus: most complex among normal prims, can be modified in many exotic ways
# tube: another form of hollow cylinder
# ring: another variant of torus
# [[sculpt]]ed: new as of 2007, used for highly variable organic shapes

==Properties==

==== Mass ====

The [[mass]] of a prim is a function of it's volume, different prim shapes have different volumes thus different masses.
* [[Mass Lab]] demonstrates how different shapes have different masses.

==== Survival ====

The survival of properties depends upon the type of the property and what happens to the object.

{| {{Prettytable}}
|+ '''Property survival'''
|- {{Hl2}}
! Property
! Script Set Not Running
! Script Removal
! Take unscripted and re-rez
! Shift-drag-copy
|-
|| [[llSitTarget|Sit Target]] || Yes || Yes || Yes || ?
|-
|| [[llParticleSystem|Particles]] || Yes || Yes || Yes || ?
|-
|| [[llSetText|Floating Text]] || Yes || Yes || Yes || No
|-
|| [[llTargetOmega|Spin]] || Yes || No || ? || ?
|-
|| [[llCollisionSound|Collision Sound]] || Yes || Yes || ? || ?
|-
|| [[llLoopSound|Looped Sound]] || Yes || Yes || Yes || No
|-
|| [[llSetRemoteScriptAccessPin|Remote Script Access Pin]] || Yes || Yes || ? || ?
|-
|| [[PRIM_POINT_LIGHT|Light]] || Yes || Yes || Yes || Yes
|-
|| {{LSLGC|Status}} || Yes || Yes || Yes || Yes
|-
|| [[llSetBuoyancy|Buoyancy]]
|| ? || No || No || ?
|-
|| [[llSetTextureAnim|Texture Animation]]
|| Yes || Yes || Yes || No
|}

It should be noted that when a script (in an object) moves from one simulator into another while being set as "Not Running" (either by [[llSetScriptState]] or via the checkbox in the script editor) the script's state will be lost. See {{Jira|SVC-1853}} for details.

* {{Jira|SVC-3925}}
* {{Jira|VWR-640}}
* {{Jira|SVC-1242}}

==Materials==
There are seven primitive [[material]] types that determine things like friction and default [[llCollisionSound|collision sound]]:
# stone
# metal
# glass
# wood
# flesh
# plastic
# rubber

==Help==
* [[Basic Prim Creation and Editing]]

==Scripting==
* [[:Category:LSL_Prim]]

==Video tutorials==
* [[Video Tutorial/What is a prim?]]
* [[Video Tutorial/Prim and mesh hair differences]]

===See also===
* [[Invisiprim]]
* [[Mega Prim]]
* [[Object]]
* [[Prims]]
* [[Prim and Object Hierarchy]]
* [[Prim flicker]]
* [[Primfficiency]]
* [[Sculpted prim]]

==External links==
* [http://en.wikipedia.org/wiki/Geometric_primitive Wikipedia: Geometric primitive]

[[Category:Assets]]

Sounds

2010-05-27T00:44:17Z

Poppy Linden:

{{OSWikiFeatureNav}}
=== Feature Design Document ===
The Sounds folder contains audio clips. (give specs for audio clips, max length, freq, codec, file extension)

=== Functional Spec ===
(none)

=== Test scripts ===
[http://secondlife.com/knowledgebase/article.php?id=246 Audio test]

=== Discussion for future improvements ===
Sound Synthesis
[pardon the ramble... if it is inappropriate here please contact me]

IMHO a much ignored aspect of game sound design is Sound Synthesis. Presently the pervasive model is 'event driven file playback' which, although often workable (e.g. gunshots, explosions, physical interactions), also limits the sound designer's ability to craft more dynamic, deeply interactive sonic events.
A simple example of this would be:
Imagine a 'machine', emitting a low rumble, with a switch on the side which when touched, changed the sound (say to a higher pitch).
In contrast, imagine the same machine with a lever on it, which allowed the user to arbitrarily change the sound (again say pitch) as they interacted with it.

Whereas the former can be created with well crafted looping file playback and an interactive trigger to another looping file....
the latter (arbitrary interaction on the users part) would be almost impossible to create using only 'file playback'.
Instead, this type of dynamic interaction can only be created using standard sound design synthesis tools (oscillators, filters, etc.).
And a quick look at the API of sound engine for SL (FMOD) reveals that it is fully capable of this.

So it is my suggestion that these features be revealed to the designer.

Wether this is functionally possible I do not know... but if so: It would allow sound designers to craft much more dynamic and interactive sonic content.
- Robb

=== Relationship to other features ===
 List of features that need to be tested when this feature changes, and why. 

(none)

=== User Guides ===
(none)

[[Category:Assets]]

Textures

2010-05-27T00:43:54Z

Poppy Linden:

{{Help |Glossary=*}}
{{otheruses4|textures|[[LSL]] related info|:Category:LSL Texture|opensource specs|Textures (OS)}}

A texture can be used to cover the faces of a prim as a visual representation of the material and look of an object or be used to make clothing or other tatoos or be put in a notecard.

Textures can be purchased or found for free in-world, or created in third-party graphics programs and uploaded to Second Life for {{L$|10}} per image.

Textures are marked with a texture icon [[Image:Inv_item_texture.png]] (and [[Image:Inv_item_snapshot.png]] for [[snapshots]]) in the [[inventory]].

==Related Links==
* [[Saving Textures]]
* [[Sculpted_Prims:_Resident-made_Tools#Importers|Importing Textures]]
* [[Video Tutorial/Making transparent textures]]
* [[Profile Image]]

[[Category:Assets]]

AssetType

2010-05-27T00:34:30Z

Poppy Linden:

<pre>
/// <summary>
/// The different types of grid assets
/// </summary>
public enum AssetType : sbyte
{
/// <summary>Unknown asset type</summary>
Unknown = -1,
/// <summary>Texture asset, stores in JPEG2000 J2C stream format</summary>
Texture = 0,
/// <summary>Sound asset</summary>
Sound = 1,
/// <summary>Calling card for another avatar</summary>
CallingCard = 2,
/// <summary>Link to a location in world</summary>
Landmark = 3,
/// <summary>Legacy script asset, you should never see one of these</summary>
//[Obsolete]
//Script = 4,
/// <summary>Collection of textures and parameters that can be
/// worn by an avatar</summary>
Clothing = 5,
/// <summary>Primitive that can contain textures, sounds,
/// scripts and more</summary>
Object = 6,
/// <summary>Notecard asset</summary>
Notecard = 7,
/// <summary>Holds a collection of inventory items</summary>
Folder = 8,
/// <summary>Root inventory folder</summary>
RootFolder = 9,
/// <summary>Linden scripting language script</summary>
LSLText = 10,
/// <summary>LSO bytecode for a script</summary>
LSLBytecode = 11,
/// <summary>Uncompressed TGA texture</summary>
TextureTGA = 12,
/// <summary>Collection of textures and shape parameters that can
/// be worn</summary>
Bodypart = 13,
/// <summary>Trash folder</summary>
TrashFolder = 14,
/// <summary>Snapshot folder</summary>
SnapshotFolder = 15,
/// <summary>Lost and found folder</summary>
LostAndFoundFolder = 16,
/// <summary>Uncompressed sound</summary>
SoundWAV = 17,
/// <summary>Uncompressed TGA non-square image, not to be used as a
/// texture</summary>
ImageTGA = 18,
/// <summary>Compressed JPEG non-square image, not to be used as a
/// texture</summary>
ImageJPEG = 19,
/// <summary>Animation</summary>
Animation = 20,
/// <summary>Sequence of animations, sounds, chat, and pauses</summary>
Gesture = 21,
/// <summary>Simstate file</summary>
Simstate = 22,
}
</pre>

[[Category:Assets]]

Category:Assets

2010-05-27T00:05:54Z

Poppy Linden:

Assets are the serialized representation of the objects and entities of Second Life.

Build clothing

2010-05-27T00:04:41Z

Poppy Linden:

{{OSWikiFeatureNav}}
=== Feature Design Document ===
Clothing are asset that can be worn on the avatar. They each have at least one texture swatch, a color swatch, some modification sliders, and a unique Inventory icon.

[[Undershirt]]

[[Shirt]]

[[Underpants]]

[[Pants]]

[[Jacket]]

[[Gloves]]

[[Socks]]

[[Shoes]]

[[Dress]]

=== Functional Spec ===
(none)

=== Test scripts ===
(none)

=== Discussion for future improvements ===
(none)

=== Relationship to other features ===
 List of features that need to be tested when this feature changes, and why. 

(none)

=== User Guides ===
(none)

=== See also ===
* [[Clothing Tutorials]]

[[Category:Assets]]

Landmarks

2010-05-27T00:03:41Z

Poppy Linden:

{{OSWikiFeatureNav}}
=== Feature Design Document ===
Landmarks are inventory assets that contain precise locations in Second Life. Their main purpose is to enable teleportation to that location from somewhere else. Landmarks are stored, by default, in the Landmarks folder in My Inventory. Landmarks may be given to others, stored in notecards and in any field used for teleportation. There are two versions of landmarks, one of which is obsolete but still functional. The previous version stored a location in the form of a global position. Version 2 stores the local position and the RegionID of the region in question.

You may create landmarks in your own inventory by use of the Create Landmark Here option on the World menu. It is possible to edit the underlying landmark asset, however not via the Second Life client.

=== Functional Spec ===
(none)

=== Test scripts ===
[[Landmarks test]]

=== Discussion for future improvements ===
(none)

=== Relationship to other features ===
 List of features that need to be tested when this feature changes, and why. 

[[Teleport OS|Teleport]]

=== User Guides ===
(none)

[[Category:Assets]]

Asset System

2010-05-27T00:03:04Z

Poppy Linden:

{{ViewerArchNav}}
[[Category:Viewer Architecture]]
The asset system is the primary interface for requesting assets from an upstream provider. The upstream provider for the client is the current region with the main agent.

{{All Asset Types}}

[[Category:Assets]]

Category:LSL Permissions/Asset

2010-05-26T23:48:56Z

Poppy Linden:

{{LSL Header|ml=*}}
{{LSLC|Permissions}}

==Articles==
*[[Help:Permissions]]
*[[Permissions Explained]]

==Constants==
{{LSL Constants Perm Mask}}

[[Category:Assets]]

Category:LSL Notecard

2010-05-26T23:43:33Z

Poppy Linden:

{{LSL Header|ml=*}}
{{LSLC|}}{{LSLC|Inventory}}

== Notecards & Parameters ==

In public domain scripts released with modify privileges, operational parameters (e.g. opening sounds for drawers, etc.) are usually set and changed right in the script by editing the script.

Scripted products that are sold, however, often use instead a notecard as the place where customers can change parameters to meet their individual needs. Doing this negates both the need to distribute proprietary scripts with modify privileges, and the need for customers to edit scripts (which most find frightening.)

To pull information from a notecard, you use the [[llGetNotecardLine]] function. As it comes into the script, it is captured in the [[dataserver]] event, during which event you parse the information and assign it to variables. There is an illustration of the procedure here: [[Notecard_reading]].

== Limits ==

Notecards are limited in several ways:
* 64KiB
* When read by script:
** Only the first 255 bytes of each line are returned.
** Cannot contain embedded assets.
* Immutability. (They don't appear to be immutable, but they really are.)

== Notecard Immutability ==

If you create a notecard, then you can easily edit the notecard and save changes. However internally within the Second Life asset server, notecard objects are immutable. This means that when you are editing the notecard, you are actually creating a new notecard with an entirely different key than the original notecard. (The avatar's or object's inventory will not contain a reference to the old notecard any more -- it is updated to reference the key of the new notecard.) The user probably thinks it is the same notecard object, but it is really an entirely new object.

This can be exploited in a script, to detect changes to a notecard and ignore other changes in an object's inventory. The following script illustrates this technique. The configuration notecard is read when the script is started, or when the configuration notecard itself changes. (It will have a new key, and this will trigger reading it to get the changes.) If something else changes in the object's inventory, then the change is ignored and the configuration notecard is not re-read. This can save a lot of overhead.

This script assumes that there is a notecard named "Configuration" in the object's inventory, and that one or more lines of text are stored in the notecard.
<lsl>key kQuery;
integer iLine = 0;
string notecard_name = "Configuration";
key notecard_key = NULL_KEY;

config_init()
{
key nc_key = llGetInventoryKey(notecard_name);
if (nc_key == notecard_key)
{
// Some other inventory item changed, not the configuration notecard.
return; // Skip reading the notecard in this case.
}
// Remember the new notecard key from now on.
notecard_key = nc_key;
kQuery = llGetNotecardLine(notecard_name, iLine);
}

default
{
state_entry()
{
// Read the notecard once at startup.
config_init();
}

changed(integer change)
{
if (change & CHANGED_INVENTORY)
{
// Read the notecard when the inventory has changed.
config_init();
}
}

dataserver(key query_id, string data)
{
if (query_id == kQuery)
{
// this is a line of our notecard
if (data == EOF)
{
llOwnerSay("Finished reading configuration.");
}
else
{
// TODO: handle notecard line here. For not just inform the owner.
llOwnerSay("Read notecard line: " + data);
// increment line count
++iLine;
//request next line of notecard.
kQuery = llGetNotecardLine(notecard_name, iLine);
}
}
}
}
</lsl>

== Sharing Notecard Access ==

In the course of shared projects, you may often need to let team members access notecards in objects owned by you, or need yourself to access notecards in objects owned by them.

To enable this, see: [[Editing someone else's scripts and notecards]].

== Embedded Assets in Notecards ==
Notecards can contain embedded assets, such as textures, landmarks, etc.

These cannot be accessed via script.

Tip! While it appears that the contents of one notecard, text and embedded objects, can manually be copied from one notecard to a second notecard, in fact you won't be able to save that second notecard. What you must do is drag the embedded objects to your inventory, and then from there drag into the second notecard.

[[Category:Assets]]

Upload Assets

2010-05-26T23:34:13Z

Poppy Linden:

{{OSWikiFeatureNav}}
=== Feature Design Document ===
(none)

=== Functional Spec ===
(none)

=== Test scripts ===
[[Upload Audio test]]

[[Animation Test]]

[[Upload Texture test]]

[[Upload Heightfield test]]

=== Discussion for future improvements ===
(none)

=== Relationship to other features ===
 List of features that need to be tested when this feature changes, and why. 

(none)

=== User Guides ===
[http://secondlife.com/knowledgebase/article.php?id=142 Uploading Files]

[http://secondlife.com/knowledgebase/article.php?id=288 Uploading Heightfields]

[[Category:Assets]]

Asset Server

2010-05-26T23:33:41Z

Poppy Linden:

{{Help |Glossary=*}}

Database that tracks all Second Life objects in-world and in inventory. Overloaded asset servers can cause search, inventory, rez, teleport, and friends list failures.

== See also ==
* [[Inventory Recovery Steps]]
* [[Asset]]

[[Category:Assets]]

Asset

2010-05-26T23:33:30Z

Poppy Linden:

{{Help |Glossary=*}}

A data resource such as an image, sound, script, object, etc. Assets can be downloaded to the viewer or uploaded into a central asset store. Assets are identified by [[UUID]] and type (Types are in llassettype.h)

== See also ==
* [[Inventory Recovery Steps]]
* [[Asset Server]]

[[Category:Assets]]

Transfer Manager

2010-05-26T23:30:33Z

Poppy Linden:

{{ViewerArchNav}}

The LLTransferManager class is a throttled data transfer system which is used primarily by the [[Asset System]] to fetch assets from Second Life. A transfer manager client makes a new instance of a class derived from LLTransferSourceParams, a new instance of a class derived from LLTransferTargetParams, and issues the request with a [[TransferRequest]] message. The source parameters are thrown away, and the client process creates an appropriate LLTransferTarget instance which copies the target parameters.

When the [[TransferRequest]] comes in, the transfer id is used as a primary key in the server to index the transfers. Since duplicates are discarded and the key should be unguessable, it should be safe to key on a client generated uuid in this case. Colluding clients generating identical keys will get one successful request and one client with a likely memory leak.

Requests for file sources will fail. Requests for assets are constrained to assets which the viewer is allowed to request via asset id. Asset requests are never prioritized in the asset queue regardless of client specified priority. Requests for inventory items are constrained to the assets in agent or task inventory according to the client source parameters on initialization.

Textures are moved the through the [[Texture Pipeline]] -- not the transfer manager.

When the server has collected the requested data, it sends a [[TransferInfo]] message to the viewer, followed by as many [[TransferPacket]] messages as necessary to transmit all data. If a [[TransferPacket]] is received before the [[TransferInfo]], or out of order, the data will be stored and replayed once the information shows up or the preceding packets arrive. If too many [[TransferPacket]] are queued for delayed delivery, the transfer will be aborted.

The [[TransferAbort]] message can be sent from either side to end a transfer. The message is sent in response to incorrect accounting information, reliable packet delivery failure, or too many delayed packets.

[[Category:Assets]]

Protocol

2010-05-26T23:28:17Z

Poppy Linden:

{{OSWikiLearnBox}}

==Message Formats==
===Binary UDP===
* [[Message Layout]] -- Explanation of message format and the message template
* [[Packet Layout]] -- the layout of the UDP payload.
* [[Circuits]] -- Establishment and common operation of UDP circuits between two nodes
* [[Packet Accounting]] -- how the message system handles packet sequencing, reliability, suppression, throttling, etc.
* [[:Category:Common Messages|Common Messages]] -- messages that are handled by every Indra message system instance.
* [[Xfer Manager]] -- messages used by the <code>LLXFerManager</code> class.
* [[Transfer Manager]] -- messages used by the <code>LLTransferManger</code> class.
* [[:Category:Messages|All messages]] -- an alphabetical listing of all messages in the system

===Capabilities===
The Capabilities system allows viewers to request abilities and be temporarily granted those abilities through a unique URL. The transport format is LLSD XML, and clients can either post new events to CAPS URLs or wait for events to come from the server on the event queue connection the client holds open.

* [[Capabilities]]
* [[Message_System_and_Capabilities]]

Capabilities use the Linden Lab Structured Data system to represent data.
See [[LLSD]].

===XML-RPC===
Older releases of the official viewer used XML-RPC as part of the [[Login Protocol]]. The official viewer now uses a web based authentication protocol, however non-official viewers may still use XML-RPC for login.

See also [[Authentication Flow]].

==Protocol Systems==
===Animations===

See [[Internal Animation Format]].

===Appearance===
See [[Avatar Appearance]].

===Assets===

===Avatars===

===Channels and Versions===

* [[Channel]] - definition of a channel
** [[Channel and Version Requirements]] - Linden Lab policy on the use of channels.

===Directory Searches===
See [[:Category:Search Messages|Search Messages]].

===Estate Tools===
===Friends===
See:
* [[:Category:Relationship Messages|Relationship Messages]]
* [[Mute List]]

===Grid===
===Groups===
* [[Group Messages]]

===Inventory===
* [[:Category:Inventory Messages|Inventory Messages]]
* [[Task Inventory]]

===Agent===
* [[:Category:Agent Messages|Agent Messages]]
* [[:Category:Money Messages|Money Messages]]
* [[:Category:Communication Messages|Communication Messages]]
* [[Buying Land and Currency]]
===Login===
* [[Login_Protocol|Login]]

===Movement===
===Objects===
* [[:Category:Object Messages|Object Messages]]

===Images===
===Parcels===
* [[:Category:Parcel Messages|Parcel Messages]]

===Simulators===
* [[:Category:Region Messages|Region Messages]]

===Sounds===
===Terrain===

==File Formats==
===Asset Files===
{{All Asset Types}}

[[Category:Assets]]

Category:Assets

2010-05-26T23:27:56Z

Poppy Linden: Created page with 'Assets are the on-disk representation of the objects and entities of Second Life.'

Assets are the on-disk representation of the objects and entities of Second Life.

Asset

2010-05-26T22:47:02Z

Poppy Linden:

Asset Server

2010-05-26T22:46:13Z

Poppy Linden:

LLSD

2010-05-14T22:01:35Z

Poppy Linden: /* Notation Serialization */

== Summary ==

Linden Lab Structured Data (LLSD) provides a flexible data system similar to the data facilities of dynamic languages like Perl and Python. It supports exchange of structured data between loosely-coupled systems (those not compiled into the same module).

Data in such exchanges must be highly tolerant of:
* Recompilation
* Implementation in a different language
* Addition of extra parameters
* Execution of older versions (with fewer parameters)

To this end, LLSD strives to be easy to use, and to default to "the right thing" wherever possible. It is extremely tolerant of errors and unexpected situations.

The fundamental class is LLSD. LLSD is a value holding object. It holds one value that is either undefined, one of the scalar types, or a map or an array. LLSD objects have value semantics (copying them copies the value, though it can be considered efficient, due to sharing), and are mutable.

Undefined is the singular value given to LLSD objects that are not initialized with any data.

The scalar data types are:
* Boolean - true or false
* Integer - a 32 bit signed integer
* Real - a 64 bit IEEE 754 floating point value
* UUID - a 128 bit unique value
* String - a sequence of zero or more Unicode chracters
* Date - an absolute point in time, UTC, with resolution to the second
* URI - a String that is a URI
* Binary - a sequence of zero or more octets (unsigned bytes)

A map is a dictionary mapping String keys to LLSD values. The keys are unique within a map, and have only one value (though that value could be an LLSD array).

An array is a sequence of zero or more LLSD values.

== Scalar Accessors ==

Function: Fetch a scalar value, converting if needed and possible.

Conversion among the basic types, Boolean, Integer, Real and String, is fully defined. Each type can be converted to another with a reasonable interpretation. These conversions can be used as a convenience even when you know the data is in one format, but you want it in another. Of course, many of these conversions lose information.

Note: These conversions are not the same as Perl's. In particular, when converting a String to a Boolean, only the empty string converts to false. Converting the String "0" to Boolean results in true.

Conversion to and from UUID, Date, and URI is only defined to and from String. Conversion is defined to be information preserving for valid values of those types. These conversions can be used when one needs to convert data to or from another system that cannot handle these types natively, but can handle strings.

Conversion to and from Binary isn't defined.

Conversion of the Undefined value to any scalar type results in a reasonable null or zero value for the type.

== Automatic Cast Protection ==

These are not implemented on purpose. Without them, C++ can perform some conversions that are clearly not what the programmer intended.

If you get a linker error about these being missing, you have made mistake in your code. DO NOT IMPLEMENT THESE FUNCTIONS as a fix.

All of thse problems stem from trying to support char* in LLSD or in std::string. There are too many automatic casts that will lead to using an arbitrary pointer or scalar type to std::string.

== Attributes and Data ==
Attributes are only used for encoding parser and formatting instructions. The data in the elements is always data.

== Root Element ==
The root element is llsd. The root must have only one child element which can be any container or atomic type.

== Atomic Types ==
Each atomic type represents one value with type information. An atomic does not have a name, but may have attributes to specify format or processing considerations for the parser.
Consumers of atomics are encouraged to massage the data into the preferred native representation, but further serialization should honor the original type information if possible.

The default value for any type is represented in XML as <code><''type'' /></code>. For example, <code><integer /></code> represents zero (0).

{| class="lltable" border="1"
! Datatype
!width="300"| Description
! Default Value
! Examples

|---
| Undefined
| A placeholder for a value.
| <code>undef</code>
| <code>undef</code>

|---
| Boolean
| A true or false value.
| False
| <code>true</code>
<code>0</code>

|---
| Integer
| A signed 32-bit integer value; any natural number between -2147483648 and 2147483647 inclusive.
| 0
| <code><integer>289343</integer></code>
<code>-3</code>

|---
| Real
| A signed 64-bit floating point number.
| 0.0
|<code>-0.28334</code>
<code>2983287453.3848387</code>

|---
| String
| A sequence of zero or more Unicode characters intended to be human readable. See the [http://tools.ietf.org/html/draft-hamrick-llsd-00#section-2.1.5 IETF draft] for details.

|"" (the empty string)
|<code>The quick brown fox, etc.</code>
<code>foo</code>

|---
| UUID
| A universally unique identifier: an unsigned 128-bit integer.
| 00000000-0000-0000-
0000-000000000000
|<code>d7f4aeca-88f1-42a1- b385-b9db18abb255</code>

|---
| Date
| A specific point in time, UTC; Values range between January 1, 1970, through at least January 1, 2038. The serialization and parser only understand ISO-8601 numeric encoding in UTC.
| The "epoch," Midnight, January 1, 1970 (UTC).
| <code>2006-02-01T14:29:53.43Z</code>

|---
| URI
| A Uniform Resource Identifier, specifying a link to an external resource and conforming to [http://www.ietf.org/rfc/rfc2396.txt RFC 2396].
| The empty URI.

|<code><nowiki>http://sim956.agni.lindenlab.com:12035/runtime/agents</nowiki></code>

|---
| Binary
|width="300"| A sequence of zero or more octets. Serialization format may include base64 (the default), base16, and base85.

| A sequence of zero octets
| base16: <code>cmFuZG9t</code>
base64: <code>dGhlIHF1aWNrIGJyb3duIGZveA==</code>

|}

=== undefined ===
The undefined type is a placeholder to indicate something is there, but it has no value, and cannot be converted to any other atomic type. Though limited in this way, an undefined is still considered a first-class atomic, and is expected to behave like any other atomic structured data type at runtime.
==== Serialization example ====
<code>
<undef />
</code>

=== boolean ===
A true or false value.
==== Conversion ====
{|
|'''type'''||'''rules'''||
|-
|boolean||unity||
|-
|integer||true => 1, false => 0||
|-
|real||true => 1.0, false => 0.0||
|-
|uuid||n/a||
|-
|string||'true', 'false'||
|-
|binary||one byte us-ascii where true => 1, false => 0||
|-
|date||n/a||
|-
|uri||n/a||
|}

==== Serialization examples ====
<pre>

<boolean>1</boolean>
<boolean>true</boolean>


<boolean>0</boolean>
<boolean>false</boolean>
<boolean />
</pre>

=== integer ===
A signed integer value with a representation of 32 bits.
==== Conversion ====
{|
|'''type'''||'''rules'''||
|-
|boolean||0 => false, all other values => true||
|-
|integer||unity||
|-
|real||closest representable number||
|-
|uuid||n/a||
|-
|string||human readable string||
|-
|binary||8 byte network byte order representation||
|-
|date||seconds since epoch||
|-
|uri||n/a||
|}

==== Serialization examples ====
<pre>
<integer>289343</integer>
<integer>-3</integer>
<integer /> 
</pre>

=== real ===
A 64 bit double as defined by IEEE.
==== Conversion ====
{|
|'''type'''||'''rules'''||
|-
|boolean||exactly 0 => false, all other values => true||
|-
|integer||rounded to closest representable number||
|-
|real||unity||
|-
|uuid||n/a||
|-
|string||human readable string||
|-
|binary||8 byte network byte order representation||
|-
|date||seconds since epoch||
|-
|uri||n/a||
|}

==== Serialization examples ====
<pre>
<real>-0.28334</real>
<real>2983287453.3848387</real>
<real /> 
</pre>

=== uuid ===
A 128 bit unsigned integer.
==== Conversion ====
{|
|'''type'''||'''rules'''||
|-
|boolean||null uuid => false, all other values => true||
|-
|integer||n/a||
|-
|real||n/a||
|-
|uuid||unity||
|-
|string||standard 8-4-4-4-12 serialization format||
|-
|binary||16 byte raw representation||
|-
|date||n/a||
|-
|uri||n/a||
|}

==== Serialization examples ====
<pre>
<uuid>d7f4aeca-88f1-42a1-b385-b9db18abb255</uuid>
<uuid /> 
</pre>

=== string ===
A simple string of any character data which is intended to be human comprehensible.

Strings in the system that hold text a user might see or enter (chat, IM, notecards, AV names, region names,... basically almost everything!) should move to using a consistent set of acceptable characters. This set is:

* Unicode code points U+20 through U+10FFFD
** except U+D800 through U+DFFF
** except U+FFFE, U+FFFF, U+1FFFE, U+1FFFF, U+2FFFE, U+2FFFF ... U+10FFFE, U+10FFFF
** except U+FDD0 through U+FDEF
* U+9 (tab, '\t')
* U+A (newline or line feed, '\n')
* U+D (carriage return, '\r')

Strings may be sequences of zero or more of these characters. Strings *may* be normalized by mapping line ending sequences to U+A. Do not rely on differences in strings that normalize to the same string.

These choices of valid strings are chosen from Unicode 4.0 which defines the following valid code points:
* Unicode code points U+0 through U+10FFFD
** except U+D800 through U+DFFF (the UTF-16 surrogate pair range)
** except U+FFFE, U+FFFF, U+1FFFE, U+1FFFF, U+2FFFE, U+2FFFF ... U+10FFFE, U+10FFFF
** except U+FDD0 through U+FDEF (some historical screw up with Arabic)

The choice for special characters < U+20 is because XML defines acceptable text as all valid Unicode code points >= U+20, and U+9, U+A and U+D. The normalization is because XML defines that all line ending sequences are normalized to U+A.

See: [[Unicode In 5 Minutes]] for a brief introduction to Unicode.

==== String escaping ====
Strings that contain non-printable characters delimited with quotes or double quotes require escaping. If a single quote delimited string contains single quotes, those must be escaped. If a double quote delimited string contains double quotes, the double quotes must be escaped.

To escape the delimiter character, prefix a backslash. Backslashes must always be escaped with another backslash.

"And then he said, \"I have nothing more to say on the subject.\""

'Look in "C:\\linden\\"'

The most generic escaping is to specify a hex value of the byte after a literal backslash and character 'x'. This can be used for any character and is required for all non-printable characters which do not have an abbreviation. For example:

\x0C

Serialized strings should only contain UTF-8 characters, so non-printable characters other than tab, newline, and carriage return should be avoided. However, common non-printable characters have short-hand abbreviations.

{| class="lltable" border="1"
|+ Notation abbreviations
! character !! value !! serialization
|-
| alert/bell || 0x7 || \a
|-
| backspace || 0x8 || \b
|-
| form feed || 0xc || \f
|-
| newline || 0xa || \n
|-
| carriage return || 0xd || \r
|-
| horizontal tab || 0x9 || \t
|-
| vertical tab || 0xb || \v
|}

==== Conversion ====

{| class="lltable" border="1"
|-
! Type
! Rules

|-
|boolean||Empty => false, all other values => true
|-
|integer||A simple conversion of the initial characters to an integer
|-
|real||A simple conversion of the initial characters to a real number
|-
|uuid||A valid 8-4-4-4-12 is converted to a uuid, all other values => null uuid
|-
|string||Unity
|-
|binary||Raw representation of the characters
|-
|date||An interpretation of the string as a date
|-
|uri||An interpretation of the string as a link
|}

==== Serialization examples ====
<pre>
<string>The quick brown fox jumped over the lazy dog.</string>
<string>540943c1-7142-4fdd-996f-fc90ed5dd3fa</string>
<string /> 
</pre>

=== binary data ===
A chunk of binary data.
The serialization format is allowed to specify an encoding. Parsers must support base64 encoding. Parsers may support base16 and base85.
==== Conversion ====
{|
||'''type'''||'''rules'''||
|-
||boolean||empty => false, all other values => true||
|-
||integer||len < 4 => 0, otherwise first four bytes are interpreted as a network byte order integer||
|-
||real||len < 8 => 0, otherwise first eight bytes are interpreted as a network byte order double||
|-
||uuid||len < 16 => null uuid, otherwise first sixteen bytes are interpreted as the raw binary uuid||
|-
||string||the raw binary data interpreted as utf-8 character data||
|-
||binary||unity||
|-
||date||n/a||
|-
||uri||the raw binary data interpreted as a utf-8 serialized link||
|}

==== Serialization examples ====
<pre>
<binary encoding="base64">cmFuZG9t</binary> 
<binary>dGhlIHF1aWNrIGJyb3duIGZveA==</binary> 
<binary /> 
</pre>

=== date ===
A specific point in time. Intervals or relative dates are not supported. The serialization and parser only understand ISO-8601 numeric encoding in UTC. The time may be omitted which will be interpreted as midnight at the start of the day.
==== Conversion ====
{|
|'''type'''||'''rules'''||
|-
|boolean||n/a||
|-
|integer||seconds since epoch||
|-
|real||seconds since epoch||
|-
|uuid||n/a||
|-
|string||standard serialization format||
|-
|binary||n/a||
|-
|date||unity||
|-
|uri||n/a||
|}

==== Serialization examples ====
<pre>
<date>2006-02-01T14:29:53.43Z</date>
<date /> 
</pre>

=== uri ===
A link to an external resource. The data is expected to conform to [http://www.ietf.org/rfc/rfc2396.txt rfc 2396] for interpretation, meaning, serialization, and deserialization.
==== Conversion ====
{|
||'''type'''||'''rules'''||
|-
||boolean||n/a||
|-
||integer||n/a||
|-
||real||n/a||
|-
||uuid||n/a||
|-
||string||standard serialization format||
|-
||binary||n/a||
|-
||date||n/a||
|-
||uri||unity||
|}

==== Serialization examples ====
<pre>
<uri>http://sim956.agni.lindenlab.com:12035/runtime/agents</uri>
<uri /> 
</pre>

== Containers ==
Containers is a special data type which can contain any other data type including other containers.

=== map ===
A map of key and value pairs where key ordering is unspecified and keys are unique. The key is always interpreted as a character string and any character string is acceptable. If there are any elements in the map, it is serialized as a key followed by an atomic or container value. For every key, there must be one value.
Well formed and valid serialized maps may contain more non-unique keys. When a deserialized, the implementation should choose one of the the value objects, but that choice is not specified.
==== Serialization example ====
<pre>
<map>
<key>foo</key>
<string>bar</string>
<key>agent info</key>
<map>
<key>agent_id</key>
<uuid>93c73b16-cd86-434d-8b4a-76e12eee950a</uuid>
<key>name</key>
<string>testtest tester</string>
</map>
</map>
</pre>

=== array ===
An ordered collection of data members. Any member can be any atomic or container type.
==== Serialization example ====
<pre>
<array>
<real>7343.0194</real>
<string>foo</string>
<array>
<map>
<key>offset</key>
<integer>9847</integer>
</map>
<string>da boom</string>
</array>
</array>
</pre>

== XML Serialization ==
MIME type: <code>application/llsd+xml</code>

When possible, prefer using us-ascii or or UTF-8 xml encoding.

XML is the "standard" serialization format, being future-proof and readable by a wide variety of tools. The XML serialization should be preferred unless profiling reveals that the [[#Binary_Serialization | binary serialization]] provides an essential performance benefit. All the serialization examples in the above sections are of the XML serialization.

=== DTD ===
<xml><!DOCTYPE llsd [
<!ELEMENT llsd (DATA)>
<!ELEMENT DATA (ATOMIC|map|array)>
<!ELEMENT ATOMIC (undef|boolean|integer|real|uuid|string|date|uri|binary)>
<!ELEMENT KEYDATA (key,DATA)>
<!ELEMENT key (#PCDATA)>
<!ELEMENT map (KEYDATA*)>
<!ELEMENT array (DATA*)>
<!ELEMENT undef (EMPTY)>
<!ELEMENT boolean (#PCDATA)>
<!ELEMENT integer (#PCDATA)>
<!ELEMENT real (#PCDATA)>
<!ELEMENT uuid (#PCDATA)>
<!ELEMENT string (#PCDATA)>
<!ELEMENT date (#PCDATA)>
<!ELEMENT uri (#PCDATA)>
<!ELEMENT binary (#PCDATA)>

<!ATTLIST string xml:space (default|preserve) 'preserve'>
<!ATTLIST binary encoding (base64|base16|base85) 'base64'>
]></xml>

=== Example XML Output ===
This is a sample from a recently running sim (indention for readability):

<xml><?xml version="1.0" encoding="UTF-8"?>
<llsd>
<map>
<key>region_id</key>
<uuid>67153d5b-3659-afb4-8510-adda2c034649</uuid>
<key>scale</key>
<string>one minute</string>
<key>simulator statistics</key>
<map>
<key>time dilation</key><real>0.9878624</real>
<key>sim fps</key><real>44.38898</real>
<key>pysics fps</key><real>44.38906</real>
<key>agent updates per second</key><real>nan</real>
<key>lsl instructions per second</key><real>0</real>
<key>total task count</key><real>4</real>
<key>active task count</key><real>0</real>
<key>active script count</key><real>4</real>
<key>main agent count</key><real>0</real>
<key>child agent count</key><real>0</real>
<key>inbound packets per second</key><real>1.228283</real>
<key>outbound packets per second</key><real>1.277508</real>
<key>pending downloads</key><real>0</real>
<key>pending uploads</key><real>0.0001096525</real>
<key>frame ms</key><real>0.7757886</real>
<key>net ms</key><real>0.3152919</real>
<key>sim other ms</key><real>0.1826937</real>
<key>sim physics ms</key><real>0.04323055</real>
<key>agent ms</key><real>0.01599029</real>
<key>image ms</key><real>0.01865955</real>
<key>script ms</key><real>0.1338836</real>
</map>
</map>
</llsd></xml>

= Binary Serialization =
MIME type: <code>application/llsd+binary</code>

We also have support for binary serialization and deserialization in c++ and python. The binary format is useful when dealing where optimal parse time is necessary. Binary LLSD is the binary llsd prefix followed by a single LLSD element of any type.

<pre>
<?llsd/binary?>\n
</pre>

{| border="1" class="lltable"
|+ Binary element serialization
! type !! serialization !! notes
|-
| undef || '!'
|-
| true || '1'
|-
| false || '0'
|-
| integer || 'i' + htonl(value)
|-
| real || 'r' + htond(value)
|-
| uuid || 'u' + uuid || uuid is 16 bytes
|-
| binary || 'b' + htonl(binary.size()) + binary
|-
| string || 's' + htonl(string.size()) + string || notation serialization is considered valid
|-
| uri || 'l' + htonl(uri.size()) + uri
|-
| date || 'd' + htond(seconds_since_epoch)
|-
| array || '[' + htonl(array.length()) + (child0, child1, ...) + ']' || order is always preserved
|-
| map || '{' + htonl(map.length()) + ((key0,value0), (key1, value1), ...)+ '}' || order is not always preserved.
|}

size() is a byte count.

length() is a child count.

htonl() is a function to generate a 4 byte network byte order integer.

htond() is a function to generate an 8 byte network byte order double. htond is not a standard system call, but you can find a c implementation in <code>indra/llcommon/llsdserialize.cpp</code>.

= Notation Serialization =
The (<strike>deprecated</strike>?) notation serialization format is meant for human readability. It is similar to, but different from, JSON. See the [[LLSD#Notation_versus_JSON |comparison notes]] below.

Parsing and formatting are currently available <strike>only in c++</strike> in c++ (indra/llcommon/llsd.h) and in python (llbase.llsd). Notation LLSD is the notation llsd prefix followed by a single LLSD element of any type.

<pre>
<?llsd/notation?>\n
</pre>

{| border="1" class="lltable"
|+ Binary element serialization
! type !! serialization !! notes
|-
| undef || '!'
|-
| true || <nowiki>'1' | 't' | 'T' | 'true' | 'TRUE'</nowiki>
|-
| false || <nowiki>'0' | 'f' | 'F' | 'false' | 'FALSE'</nowiki>
|-
| integer || 'i' str(value)
|-
| real || 'r' str(value)
|-
| uuid || 'u' str(uuid)
|-
| binary || <nowiki>'b(' str(size) ')"' raw_data '"' | 'b' base '"' encoded_data '"'</nowiki> || Base 16 and 64 encodings are supported.
|-
| string || <nowiki>" escaped_string " | ' escaped_string ' | 's(' str(size) ')"' raw_string '"'</nowiki> || When using single quotes, double quotes do not need escaping and vice versa.
|-
| uri || 'l"' escaped_uri '"' || See [http://www.faqs.org/rfcs/rfc1738.html rfc 1738] for encoding rules.
|-
| date || 'd"' YYYY-MM-DD 'T' HH:MM:SS [.FF] 'Z"' || Fractional seconds are optional
|-
| array || '[' object0 ',' object1 ',' ... ']' || order is always preserved
|-
| map || '{' string0:object0 ',' string1:object1 ',' ... '}' || order is not always preserved. The string is any supported string serialization format
|}

== Example Notation Output ==
This is an excerpt from an agent request to enter a region serialized as notation:
<pre>
[
{'destination':'http://secondlife.com'},
{'version':i1},
{
'agent_id':u3c115e51-04f4-523c-9fa6-98aff1034730,
'session_id':u2c585cec-038c-40b0-b42e-a25ebab4d132,
'circuit_code':i1075,
'first_name':'Phoenix',
'last_name':'Linden',
'position':[r70.9247,r254.378,r38.7304],
'look_at':[r-0.043753,r-0.999042,r0],
'granters':[ua2e76fcd-9360-4f6d-a924-000000000003],
'attachment_data':
[
{
'attachment_point':i2,
'item_id':ud6852c11-a74e-309a-0462-50533f1ef9b3,
'asset_id':uc69b29b1-8944-58ae-a7c5-2ca7b23e22fb
},
{
'attachment_point':i10,
'item_id':uff852c22-a74e-309a-0462-50533f1ef900,
'asset_id':u5868dd20-c25a-47bd-8b4c-dedc99ef9479
}
]
}
]
</pre>

<pre>
[
{
'creation-date':d"2007-03-15T18:30:18Z",
'creator-id':u3c115e51-04f4-523c-9fa6-98aff1034730
},
s(10)"0123456789",
"Where's the beef?",
'Over here.',
b(160)"default
{
state_entry()
{
llSay(0, "Hello, Avatar!");
}

touch_start(integer total_number)
{
llSay(0, "Touched.");
}
}",
b64"AABAAAAAAAAAAAIAAAA//wAAP/8AAADgAAAA5wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AABkAAAAZAAAAAAAAAAAAAAAZAAAAAAAAAABAAAAAAAAAAAAAAAAAAAABQAAAAEAAAAQAAAAAAAA
AAUAAAAFAAAAABAAAAAAAAAAPgAAAAQAAAAFAGNbXgAAAABgSGVsbG8sIEF2YXRhciEAZgAAAABc
XgAAAAhwEQjRABeVAAAABQBjW14AAAAAYFRvdWNoZWQuAGYAAAAAXF4AAAAIcBEI0QAXAZUAAEAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
]
</pre>

== Notation versus JSON ==

The plan is to eventually move to JSON, after its spec in ECMAScript 5th edition is finalized. There are three reasons this is not a current priority:

# The binary and XML serialization formats work for our use cases, so there's no driving need for JSON.
# The notation format has not been useful in our experience and we expect JSON to fill a similar use niche.
# JSON was not in wide, common use when LLSD was invented.

= Guidelines =

=== Questions & Things To Do ===

Would Binary be more convenient as usigned char* buffer semantics?

Should Binary be convertable to/from String, and if so how?
* as UTF8 encoded strings (making not like UUID<->String)
* as Base64 or Base96 encoded (making like UUID<->String)

Conversions to std::string and LLUUID do not result in easy assignment to std::string, LLString or LLUUID due to non-unique conversion paths.

[[Category:Open Source Portal]]

File:PoppyLinden.png

2010-05-12T21:51:35Z

Poppy Linden:

LLMessageSystem Burst Metrics

2010-02-10T00:59:38Z

Poppy Linden: /* Examples */

== Why care about message processing? ==

The dataserver receives a message, makes a database query or two, then sends a response. The next message isn't received until the response is sent. LLMessageSystem in other LLApp instances is used in a similar way - receive message, do processing, receive next message. The question becomes, what is the longest amount of time that a message has to wait to be processed? In many instances, this wait manifests itself as "lag", and in the worst cases can result in timeouts and failed operations.

== What do we want to know about messages? ==

There are several concerns. Among the reasons this metric was added:
* How much longer is response time when database network latency is increased? (dataserver specific)
* What is gridwide message response time? For which application?

Ideally, we would like to be able to make a list of information about each message:
* time received
* time processing started
* time processing finished

In the case of the dataserver, over a specified time period the maximum time processing finished minus time received would be the worst case of message latency on that app. Think about this instance:

[[Image:MessageBurstTiming_1.png]]

Message E has the highest response latency - it wasn't done processing until 15ms after it was received.

== The way the world actually works ==

Unfortunately, with the way the message system currently works, there is no way to tell when a message was received. Worse, the packet buffer is extremely small and there is no way to tell if a message is dropped. Let's run through the above example in a more realistic environment.

[[Image:MessageBurstTiming_2.png]]

Message A, B, C received in the same order, but C doesn't fit in the buffer, so is silently dropped. In this case we're assuming the sender will re-send after a certain time if an ACK isn't sent back. For the sake of the example, assume an ACK is sent right when a message begins processing, and resends are at 4ms. (it's more complicated than this, but this example is far more convoluted than the previous one already.)

[[Image:MessageBurstTiming_3.png]]

Time 4 comes around, A is pulled out of the buffer for processing, B moves forward in the incoming queue, and D is received. E is dropped, because by the time 5ms rolls around, the buffer is full again. B is resent because it hasn't been ACK'd by 6ms - A is still processing until 7ms. The resend of B happens to a full buffer, and is dropped.

[[Image:MessageBurstTiming_4.png]]

At the tick of 8ms message B is pulled in for processing, and the resend of C hits the buffer. Before B is done processing, E gets resent, but dropped again.

[[Image:MessageBurstTiming_5.png]]

By the time D gets pulled in, the third E comes around and makes it to the buffer. There are no more new packets in the rest of the timeframe we're looking at, so we can see what times all of these packets came in and out.

[[Image:MessageBurstTiming_6.png]]

Looks a little different than the ideal example, but the sum of the processing times remains the same. The receiving application was constantly busy in either case. The worst case wait is still message E and is also matched by C - 15ms (20 - 5, and 18 - 3, respectively).

[[Image:MessageBurstTiming_7.png]]

The end result is the same as if all of the messages had arrived at once. And we've got a little bit more information in the diagrams than there is in the system itself - remember, there's no way to tell what the actual arrival times are. Hence, from the system, we can't tell the difference between this case and the previous case.

[[Image:MessageBurstTiming_8.png]]

This is all the information we have - we may know a message is a resend, but we don't know when the original was intended to be received.

== The Burst, defined ==

This brings us to the concept of the message burst. If we started processing messages at 4ms, and stopped at 20ms, we know that the theoretical worst case would be if all the messages arrived simultaneously the last message would be waiting 16ms for a reply. That's worse than the truth, but it's the closest thing we have. There's also another approximation - that we're always processing messages, and they don't have to wait long to start processing (clearly not the case for packet A).

A burst is this constant arrival of packets. It ends when the buffer is found to be empty. Second Life backend traffic tends to be very bursty with long break periods. Unfortunately, these bursts can last many full seconds, causing terrible lag. This metric, combined with the many others that we already have, should give us a better idea about what our current message latency is today, and if changes we make help or hurt it. It's just one part of the connection, but hopefully it helps.

== Implementation details ==

* Burst metrics are disabled by default, because timer calls can be expensive - 12ns * 4 may be significant if you are expecting to process several messages per 1ms!
* The bursts are recorded as matching lists of the duration of the burst and the count of messages in the burst.
* The metrics are available both as calculated statistics, and raw data (if you have need of more accurate stats)
* If burst tracking is enabled, but you want to dynamically suppress either log output, you need merely add a "log-messages" section to you app's LiveConfig file, add a key for the log message ("logMessageBurstStats" and "logMessageBurstData", respectively) and set "use-syslog" to integer 0.

== Examples ==
This is an example from an artificially-busy development LLApp I've been working on so the numbers look ''really huge'' but that's, unfortunately, not real.

<code>
2010-02-09T04:48:47Z INFO: log: LLLOGMESSAGE (1) logMessageBurstStats {'avg_message_burst_throughput_rate':r1768.89, 'data_points':i702, 'frame_time_dur':r53.1146, 'max_message_break_duration':r5.85154, 'max_message_burst_count':i227, 'max_message_burst_duration':r0.307471, 'max_message_burst_throughput_rate':r12422.4, 'min_message_burst_throughput_rate':r163.62, 'mode':'worker', 'total_message_break_duration':r47.1424, 'total_message_burst_duration':r6.48097, 'total_message_count':i6341, 'total_message_throughput_rate':r119.383}
</code>

<code>
2010-02-09T04:48:57Z INFO: log: LLLOGMESSAGE (2) logMessageBurstStats {'avg_message_burst_throughput_rate':r1097.77, 'data_points':i115, 'frame_time_dur':r10.1428, 'max_message_break_duration':r2.9249, 'max_message_burst_count':i216, 'max_message_burst_duration':r0.314165, 'max_message_burst_throughput_rate':r3458.5, 'min_message_burst_throughput_rate':r438.102, 'mode':'worker', 'total_message_break_duration':r9.07722, 'total_message_burst_duration':r1.16166, 'total_message_count':i1082, 'total_message_throughput_rate':r106.676}
</code>

== Homework ==
* I've only looked at the dataserver. What would the viewer message system output look like?
* What is the highest message latency that you've seen?
* What in-world experience corresponds to what burst statistics?

LLMessageSystem Burst Metrics

2010-02-10T00:59:16Z

Poppy Linden: /* Examples */

== Why care about message processing? ==

The dataserver receives a message, makes a database query or two, then sends a response. The next message isn't received until the response is sent. LLMessageSystem in other LLApp instances is used in a similar way - receive message, do processing, receive next message. The question becomes, what is the longest amount of time that a message has to wait to be processed? In many instances, this wait manifests itself as "lag", and in the worst cases can result in timeouts and failed operations.

== What do we want to know about messages? ==

There are several concerns. Among the reasons this metric was added:
* How much longer is response time when database network latency is increased? (dataserver specific)
* What is gridwide message response time? For which application?

Ideally, we would like to be able to make a list of information about each message:
* time received
* time processing started
* time processing finished

In the case of the dataserver, over a specified time period the maximum time processing finished minus time received would be the worst case of message latency on that app. Think about this instance:

[[Image:MessageBurstTiming_1.png]]

Message E has the highest response latency - it wasn't done processing until 15ms after it was received.

== The way the world actually works ==

Unfortunately, with the way the message system currently works, there is no way to tell when a message was received. Worse, the packet buffer is extremely small and there is no way to tell if a message is dropped. Let's run through the above example in a more realistic environment.

[[Image:MessageBurstTiming_2.png]]

Message A, B, C received in the same order, but C doesn't fit in the buffer, so is silently dropped. In this case we're assuming the sender will re-send after a certain time if an ACK isn't sent back. For the sake of the example, assume an ACK is sent right when a message begins processing, and resends are at 4ms. (it's more complicated than this, but this example is far more convoluted than the previous one already.)

[[Image:MessageBurstTiming_3.png]]

Time 4 comes around, A is pulled out of the buffer for processing, B moves forward in the incoming queue, and D is received. E is dropped, because by the time 5ms rolls around, the buffer is full again. B is resent because it hasn't been ACK'd by 6ms - A is still processing until 7ms. The resend of B happens to a full buffer, and is dropped.

[[Image:MessageBurstTiming_4.png]]

At the tick of 8ms message B is pulled in for processing, and the resend of C hits the buffer. Before B is done processing, E gets resent, but dropped again.

[[Image:MessageBurstTiming_5.png]]

By the time D gets pulled in, the third E comes around and makes it to the buffer. There are no more new packets in the rest of the timeframe we're looking at, so we can see what times all of these packets came in and out.

[[Image:MessageBurstTiming_6.png]]

Looks a little different than the ideal example, but the sum of the processing times remains the same. The receiving application was constantly busy in either case. The worst case wait is still message E and is also matched by C - 15ms (20 - 5, and 18 - 3, respectively).

[[Image:MessageBurstTiming_7.png]]

The end result is the same as if all of the messages had arrived at once. And we've got a little bit more information in the diagrams than there is in the system itself - remember, there's no way to tell what the actual arrival times are. Hence, from the system, we can't tell the difference between this case and the previous case.

[[Image:MessageBurstTiming_8.png]]

This is all the information we have - we may know a message is a resend, but we don't know when the original was intended to be received.

== The Burst, defined ==

This brings us to the concept of the message burst. If we started processing messages at 4ms, and stopped at 20ms, we know that the theoretical worst case would be if all the messages arrived simultaneously the last message would be waiting 16ms for a reply. That's worse than the truth, but it's the closest thing we have. There's also another approximation - that we're always processing messages, and they don't have to wait long to start processing (clearly not the case for packet A).

A burst is this constant arrival of packets. It ends when the buffer is found to be empty. Second Life backend traffic tends to be very bursty with long break periods. Unfortunately, these bursts can last many full seconds, causing terrible lag. This metric, combined with the many others that we already have, should give us a better idea about what our current message latency is today, and if changes we make help or hurt it. It's just one part of the connection, but hopefully it helps.

== Implementation details ==

* Burst metrics are disabled by default, because timer calls can be expensive - 12ns * 4 may be significant if you are expecting to process several messages per 1ms!
* The bursts are recorded as matching lists of the duration of the burst and the count of messages in the burst.
* The metrics are available both as calculated statistics, and raw data (if you have need of more accurate stats)
* If burst tracking is enabled, but you want to dynamically suppress either log output, you need merely add a "log-messages" section to you app's LiveConfig file, add a key for the log message ("logMessageBurstStats" and "logMessageBurstData", respectively) and set "use-syslog" to integer 0.

== Examples ==
This is an example from an artificially-busy development LLApp I've been working on, so the numbers look ''really huge'' but that's, unfortunately, not real.
<code>
2010-02-09T04:48:47Z INFO: log: LLLOGMESSAGE (1) logMessageBurstStats {'avg_message_burst_throughput_rate':r1768.89, 'data_points':i702, 'frame_time_dur':r53.1146, 'max_message_break_duration':r5.85154, 'max_message_burst_count':i227, 'max_message_burst_duration':r0.307471, 'max_message_burst_throughput_rate':r12422.4, 'min_message_burst_throughput_rate':r163.62, 'mode':'worker', 'total_message_break_duration':r47.1424, 'total_message_burst_duration':r6.48097, 'total_message_count':i6341, 'total_message_throughput_rate':r119.383}
</code>
<code>
2010-02-09T04:48:57Z INFO: log: LLLOGMESSAGE (2) logMessageBurstStats {'avg_message_burst_throughput_rate':r1097.77, 'data_points':i115, 'frame_time_dur':r10.1428, 'max_message_break_duration':r2.9249, 'max_message_burst_count':i216, 'max_message_burst_duration':r0.314165, 'max_message_burst_throughput_rate':r3458.5, 'min_message_burst_throughput_rate':r438.102, 'mode':'worker', 'total_message_break_duration':r9.07722, 'total_message_burst_duration':r1.16166, 'total_message_count':i1082, 'total_message_throughput_rate':r106.676}
</code>

== Homework ==
* I've only looked at the dataserver. What would the viewer message system output look like?
* What is the highest message latency that you've seen?
* What in-world experience corresponds to what burst statistics?

LLMessageSystem Burst Metrics

2010-02-10T00:56:54Z

Poppy Linden: /* Examples */

== Why care about message processing? ==

The dataserver receives a message, makes a database query or two, then sends a response. The next message isn't received until the response is sent. LLMessageSystem in other LLApp instances is used in a similar way - receive message, do processing, receive next message. The question becomes, what is the longest amount of time that a message has to wait to be processed? In many instances, this wait manifests itself as "lag", and in the worst cases can result in timeouts and failed operations.

== What do we want to know about messages? ==

There are several concerns. Among the reasons this metric was added:
* How much longer is response time when database network latency is increased? (dataserver specific)
* What is gridwide message response time? For which application?

Ideally, we would like to be able to make a list of information about each message:
* time received
* time processing started
* time processing finished

In the case of the dataserver, over a specified time period the maximum time processing finished minus time received would be the worst case of message latency on that app. Think about this instance:

[[Image:MessageBurstTiming_1.png]]

Message E has the highest response latency - it wasn't done processing until 15ms after it was received.

== The way the world actually works ==

Unfortunately, with the way the message system currently works, there is no way to tell when a message was received. Worse, the packet buffer is extremely small and there is no way to tell if a message is dropped. Let's run through the above example in a more realistic environment.

[[Image:MessageBurstTiming_2.png]]

Message A, B, C received in the same order, but C doesn't fit in the buffer, so is silently dropped. In this case we're assuming the sender will re-send after a certain time if an ACK isn't sent back. For the sake of the example, assume an ACK is sent right when a message begins processing, and resends are at 4ms. (it's more complicated than this, but this example is far more convoluted than the previous one already.)

[[Image:MessageBurstTiming_3.png]]

Time 4 comes around, A is pulled out of the buffer for processing, B moves forward in the incoming queue, and D is received. E is dropped, because by the time 5ms rolls around, the buffer is full again. B is resent because it hasn't been ACK'd by 6ms - A is still processing until 7ms. The resend of B happens to a full buffer, and is dropped.

[[Image:MessageBurstTiming_4.png]]

At the tick of 8ms message B is pulled in for processing, and the resend of C hits the buffer. Before B is done processing, E gets resent, but dropped again.

[[Image:MessageBurstTiming_5.png]]

By the time D gets pulled in, the third E comes around and makes it to the buffer. There are no more new packets in the rest of the timeframe we're looking at, so we can see what times all of these packets came in and out.

[[Image:MessageBurstTiming_6.png]]

Looks a little different than the ideal example, but the sum of the processing times remains the same. The receiving application was constantly busy in either case. The worst case wait is still message E and is also matched by C - 15ms (20 - 5, and 18 - 3, respectively).

[[Image:MessageBurstTiming_7.png]]

The end result is the same as if all of the messages had arrived at once. And we've got a little bit more information in the diagrams than there is in the system itself - remember, there's no way to tell what the actual arrival times are. Hence, from the system, we can't tell the difference between this case and the previous case.

[[Image:MessageBurstTiming_8.png]]

This is all the information we have - we may know a message is a resend, but we don't know when the original was intended to be received.

== The Burst, defined ==

This brings us to the concept of the message burst. If we started processing messages at 4ms, and stopped at 20ms, we know that the theoretical worst case would be if all the messages arrived simultaneously the last message would be waiting 16ms for a reply. That's worse than the truth, but it's the closest thing we have. There's also another approximation - that we're always processing messages, and they don't have to wait long to start processing (clearly not the case for packet A).

A burst is this constant arrival of packets. It ends when the buffer is found to be empty. Second Life backend traffic tends to be very bursty with long break periods. Unfortunately, these bursts can last many full seconds, causing terrible lag. This metric, combined with the many others that we already have, should give us a better idea about what our current message latency is today, and if changes we make help or hurt it. It's just one part of the connection, but hopefully it helps.

== Implementation details ==

* Burst metrics are disabled by default, because timer calls can be expensive - 12ns * 4 may be significant if you are expecting to process several messages per 1ms!
* The bursts are recorded as matching lists of the duration of the burst and the count of messages in the burst.
* The metrics are available both as calculated statistics, and raw data (if you have need of more accurate stats)
* If burst tracking is enabled, but you want to dynamically suppress either log output, you need merely add a "log-messages" section to you app's LiveConfig file, add a key for the log message ("logMessageBurstStats" and "logMessageBurstData", respectively) and set "use-syslog" to integer 0.

== Examples ==
<code>
2010-02-09T04:48:47Z INFO: log: LLLOGMESSAGE (1) logMessageBurstStats {'avg_message_burst_throughput_rate':r1768.89, 'data_points':i702, 'frame_time_dur':r53.1146, 'max_message_break_duration':r5.85154, 'max_message_burst_count':i227, 'max_message_burst_duration':r0.307471, 'max_message_burst_throughput_rate':r12422.4, 'min_message_burst_throughput_rate':r163.62, 'mode':'worker', 'total_message_break_duration':r47.1424, 'total_message_burst_duration':r6.48097, 'total_message_count':i6341, 'total_message_throughput_rate':r119.383}
</code>

== Homework ==
* I've only looked at the dataserver. What would the viewer message system output look like?
* What is the highest message latency that you've seen?
* What in-world experience corresponds to what burst statistics?

LLMessageSystem Burst Metrics

2010-02-10T00:55:18Z

Poppy Linden: /* Examples */

== Why care about message processing? ==

The dataserver receives a message, makes a database query or two, then sends a response. The next message isn't received until the response is sent. LLMessageSystem in other LLApp instances is used in a similar way - receive message, do processing, receive next message. The question becomes, what is the longest amount of time that a message has to wait to be processed? In many instances, this wait manifests itself as "lag", and in the worst cases can result in timeouts and failed operations.

== What do we want to know about messages? ==

There are several concerns. Among the reasons this metric was added:
* How much longer is response time when database network latency is increased? (dataserver specific)
* What is gridwide message response time? For which application?

Ideally, we would like to be able to make a list of information about each message:
* time received
* time processing started
* time processing finished

In the case of the dataserver, over a specified time period the maximum time processing finished minus time received would be the worst case of message latency on that app. Think about this instance:

[[Image:MessageBurstTiming_1.png]]

Message E has the highest response latency - it wasn't done processing until 15ms after it was received.

== The way the world actually works ==

Unfortunately, with the way the message system currently works, there is no way to tell when a message was received. Worse, the packet buffer is extremely small and there is no way to tell if a message is dropped. Let's run through the above example in a more realistic environment.

[[Image:MessageBurstTiming_2.png]]

Message A, B, C received in the same order, but C doesn't fit in the buffer, so is silently dropped. In this case we're assuming the sender will re-send after a certain time if an ACK isn't sent back. For the sake of the example, assume an ACK is sent right when a message begins processing, and resends are at 4ms. (it's more complicated than this, but this example is far more convoluted than the previous one already.)

[[Image:MessageBurstTiming_3.png]]

Time 4 comes around, A is pulled out of the buffer for processing, B moves forward in the incoming queue, and D is received. E is dropped, because by the time 5ms rolls around, the buffer is full again. B is resent because it hasn't been ACK'd by 6ms - A is still processing until 7ms. The resend of B happens to a full buffer, and is dropped.

[[Image:MessageBurstTiming_4.png]]

At the tick of 8ms message B is pulled in for processing, and the resend of C hits the buffer. Before B is done processing, E gets resent, but dropped again.

[[Image:MessageBurstTiming_5.png]]

By the time D gets pulled in, the third E comes around and makes it to the buffer. There are no more new packets in the rest of the timeframe we're looking at, so we can see what times all of these packets came in and out.

[[Image:MessageBurstTiming_6.png]]

Looks a little different than the ideal example, but the sum of the processing times remains the same. The receiving application was constantly busy in either case. The worst case wait is still message E and is also matched by C - 15ms (20 - 5, and 18 - 3, respectively).

[[Image:MessageBurstTiming_7.png]]

The end result is the same as if all of the messages had arrived at once. And we've got a little bit more information in the diagrams than there is in the system itself - remember, there's no way to tell what the actual arrival times are. Hence, from the system, we can't tell the difference between this case and the previous case.

[[Image:MessageBurstTiming_8.png]]

This is all the information we have - we may know a message is a resend, but we don't know when the original was intended to be received.

== The Burst, defined ==

This brings us to the concept of the message burst. If we started processing messages at 4ms, and stopped at 20ms, we know that the theoretical worst case would be if all the messages arrived simultaneously the last message would be waiting 16ms for a reply. That's worse than the truth, but it's the closest thing we have. There's also another approximation - that we're always processing messages, and they don't have to wait long to start processing (clearly not the case for packet A).

A burst is this constant arrival of packets. It ends when the buffer is found to be empty. Second Life backend traffic tends to be very bursty with long break periods. Unfortunately, these bursts can last many full seconds, causing terrible lag. This metric, combined with the many others that we already have, should give us a better idea about what our current message latency is today, and if changes we make help or hurt it. It's just one part of the connection, but hopefully it helps.

== Implementation details ==

* Burst metrics are disabled by default, because timer calls can be expensive - 12ns * 4 may be significant if you are expecting to process several messages per 1ms!
* The bursts are recorded as matching lists of the duration of the burst and the count of messages in the burst.
* The metrics are available both as calculated statistics, and raw data (if you have need of more accurate stats)
* If burst tracking is enabled, but you want to dynamically suppress either log output, you need merely add a "log-messages" section to you app's LiveConfig file, add a key for the log message ("logMessageBurstStats" and "logMessageBurstData", respectively) and set "use-syslog" to integer 0.

== Examples ==
<code>
2010-02-09T04:47:54Z INFO: log: LLLOGMESSAGE (0) logMessageBurstStats {'data_points':i1, 'frame_time_dur':r0.12202, 'max_message_break_duration':r0.1219, 'max_message_burst_count':i0, 'max_message_burst_duration':r0, 'mode':'worker', 'total_message_break_duration':r0.1219, 'total_message_burst_duration':r0, 'total_message_count':i0, 'total_message_throughput_rate':r0}
</code>

== Homework ==
* I've only looked at the dataserver. What would the viewer message system output look like?
* What is the highest message latency that you've seen?
* What in-world experience corresponds to what burst statistics?

LLMessageSystem Burst Metrics

2010-01-05T22:05:29Z

Poppy Linden:

== Why care about message processing? ==

The dataserver receives a message, makes a database query or two, then sends a response. The next message isn't received until the response is sent. LLMessageSystem in other LLApp instances is used in a similar way - receive message, do processing, receive next message. The question becomes, what is the longest amount of time that a message has to wait to be processed? In many instances, this wait manifests itself as "lag", and in the worst cases can result in timeouts and failed operations.

== What do we want to know about messages? ==

There are several concerns. Among the reasons this metric was added:
* How much longer is response time when database network latency is increased? (dataserver specific)
* What is gridwide message response time? For which application?

Ideally, we would like to be able to make a list of information about each message:
* time received
* time processing started
* time processing finished

In the case of the dataserver, over a specified time period the maximum time processing finished minus time received would be the worst case of message latency on that app. Think about this instance:

[[Image:MessageBurstTiming_1.png]]

Message E has the highest response latency - it wasn't done processing until 15ms after it was received.

== The way the world actually works ==

Unfortunately, with the way the message system currently works, there is no way to tell when a message was received. Worse, the packet buffer is extremely small and there is no way to tell if a message is dropped. Let's run through the above example in a more realistic environment.

[[Image:MessageBurstTiming_2.png]]

Message A, B, C received in the same order, but C doesn't fit in the buffer, so is silently dropped. In this case we're assuming the sender will re-send after a certain time if an ACK isn't sent back. For the sake of the example, assume an ACK is sent right when a message begins processing, and resends are at 4ms. (it's more complicated than this, but this example is far more convoluted than the previous one already.)

[[Image:MessageBurstTiming_3.png]]

Time 4 comes around, A is pulled out of the buffer for processing, B moves forward in the incoming queue, and D is received. E is dropped, because by the time 5ms rolls around, the buffer is full again. B is resent because it hasn't been ACK'd by 6ms - A is still processing until 7ms. The resend of B happens to a full buffer, and is dropped.

[[Image:MessageBurstTiming_4.png]]

At the tick of 8ms message B is pulled in for processing, and the resend of C hits the buffer. Before B is done processing, E gets resent, but dropped again.

[[Image:MessageBurstTiming_5.png]]

By the time D gets pulled in, the third E comes around and makes it to the buffer. There are no more new packets in the rest of the timeframe we're looking at, so we can see what times all of these packets came in and out.

[[Image:MessageBurstTiming_6.png]]

Looks a little different than the ideal example, but the sum of the processing times remains the same. The receiving application was constantly busy in either case. The worst case wait is still message E and is also matched by C - 15ms (20 - 5, and 18 - 3, respectively).

[[Image:MessageBurstTiming_7.png]]

The end result is the same as if all of the messages had arrived at once. And we've got a little bit more information in the diagrams than there is in the system itself - remember, there's no way to tell what the actual arrival times are. Hence, from the system, we can't tell the difference between this case and the previous case.

[[Image:MessageBurstTiming_8.png]]

This is all the information we have - we may know a message is a resend, but we don't know when the original was intended to be received.

== The Burst, defined ==

This brings us to the concept of the message burst. If we started processing messages at 4ms, and stopped at 20ms, we know that the theoretical worst case would be if all the messages arrived simultaneously the last message would be waiting 16ms for a reply. That's worse than the truth, but it's the closest thing we have. There's also another approximation - that we're always processing messages, and they don't have to wait long to start processing (clearly not the case for packet A).

A burst is this constant arrival of packets. It ends when the buffer is found to be empty. Second Life backend traffic tends to be very bursty with long break periods. Unfortunately, these bursts can last many full seconds, causing terrible lag. This metric, combined with the many others that we already have, should give us a better idea about what our current message latency is today, and if changes we make help or hurt it. It's just one part of the connection, but hopefully it helps.

== Implementation details ==

* Burst metrics are disabled by default, because timer calls can be expensive - 12ns * 4 may be significant if you are expecting to process several messages per 1ms!
* The bursts are recorded as matching lists of the duration of the burst and the count of messages in the burst.
* The metrics are available both as calculated statistics, and raw data (if you have need of more accurate stats)
* If burst tracking is enabled, but you want to dynamically suppress either log output, you need merely add a "log-messages" section to you app's LiveConfig file, add a key for the log message ("logMessageBurstStats" and "logMessageBurstData", respectively) and set "use-syslog" to integer 0.

== Examples ==
Forthcoming.

== Homework ==
* I've only looked at the dataserver. What would the viewer message system output look like?
* What is the highest message latency that you've seen?
* What in-world experience corresponds to what burst statistics?

LLMessageSystem Burst Metrics

2010-01-05T21:59:11Z

Poppy Linden:

File:MessageBurstTiming 8.png

2010-01-05T21:55:48Z

Poppy Linden:

File:MessageBurstTiming 7.png

2010-01-05T21:55:35Z

Poppy Linden:

File:MessageBurstTiming 6.png

2010-01-05T21:54:30Z

Poppy Linden:

File:MessageBurstTiming 5.png

2010-01-05T21:54:17Z

Poppy Linden:

File:MessageBurstTiming 4.png

2010-01-05T21:54:04Z

Poppy Linden:

File:MessageBurstTiming 3.png

2010-01-05T21:53:47Z

Poppy Linden:

File:MessageBurstTiming 2.png

2010-01-05T21:53:22Z

Poppy Linden:

File:MessageBurstTiming 1.png

2010-01-05T21:52:12Z

Poppy Linden:

LLMessageSystem Burst Metrics

2010-01-05T21:51:18Z

Poppy Linden:

LLMessageSystem Burst Metrics

2009-11-24T21:47:02Z

Poppy Linden: Created page with 'This is a placeholder page for LLMessageSystem Burst Metrics. I'm drafting this privately, and will post it when it starts to look useful. If it's after 2009-12-15, please ping m...'

This is a placeholder page for LLMessageSystem Burst Metrics. I'm drafting this privately, and will post it when it starts to look useful. If it's after 2009-12-15, please ping me (Poppy Linden) for this info! Until then search for "burst" in llmessage to find the code.

Logging System Overview

2009-10-02T19:36:03Z

Poppy Linden: Created page with 'This document explains both the usage of our log message system in code and how to control it at runtime to vary the granularity of logging that is actually done. :Original auth...'

This document explains both the usage of our log message system in code and how to control it at runtime to vary the granularity of logging that is actually done.

:Original author is BigPapi Linden, I'm just copying the doc from internal -> external. ([[User:Poppy Linden|Poppy Linden]] 19:36, 2 October 2009 (UTC))

= Overview =

All developer functionality is enabled by "'''llerror.h'''" (in llcommon).

All runtime changing of logging granularity is done by editing "'''logcontrol.xml'''" or "'''logcontrol-dev.xml'''", which are located in the "'''app_settings/'''" directory for the viewer and in "'''/opt/linden/etc/'''" for the server ("'''<branch>/etc/'''" on Windows).

In order to get the maximum utility out of the log system, you should make sure your classes include "'''LOG_CLASS'''(LLFoo);" in the private section of your class declaration. For example:

class LLFoo
{
'''LOG_CLASS'''(LLFoo);
public:
...
};

==Output==
For the viewer, log output ends up in <tt>LL_PATH_APP_SETTINGS/logs/SecondLife.log</tt>, where LL_PATH_APP_SETTINGS is <tt>%APPDATA%\SecondLife\</tt> on Windows, <tt>$HOME/Library/Application Support/SecondLife/</tt> on Mac, and <tt>$HOME/.secondlife/</tt> on Linux.

For the server, log output ends up in <tt>/var/log/indra.log</tt>.

= Logging Macros =

The old "'''llerrs'''", "'''llwarns'''", "'''llinfos'''", "'''lldebugs'''", and "'''llendl'''" logging macros are '''deprecated''', but still work for backwards compatibility. The new ones are capitalized per our compiler preprocessor directives naming conventions in our [[Coding_Standard#Naming_Convention|Coding Standards]].

The new base logging macros allow you to optionally add one or two arbitrary string tags to the log message (a "broad" scope tag and a "narrow" scope tag), which then allows you to enable or disable log messages at runtime based on their tag (see '''<key>tags</key>''' block in sample "'''logcontrol.xml'''" [[Logging_System_Overview#Varying_amount_of_logging_at_runtime_.28logcontrol.xml.29|Here]]). NULL can be passed in to the macro if you '''REALLY''' don't want to tag a message, but it only takes a second and will make logging much more useful for all of us (think of the children).

The new base macros are used as follows:

'''LL_DEBUGS'''("BroadScopeTag") << "A debug level log message" << '''LL_ENDL'''; // '''NOTE:''' This is a macro, not a string stream. The "'''LL_ENDL'''" is required.
'''LL_INFOS'''("BroadScopeTag") << "An info level log message" << '''LL_ENDL''';
'''LL_WARNS'''("BroadScopeTag") << "A warning level log message" << '''LL_ENDL''';
'''LL_ERRS'''("BroadScopeTag") << "An error level log message" << '''LL_ENDL'''; // '''NOTE:''' '''LL_ERRS''' force a crash of the viewer.

'''Note:''' Tags should be formatted in [[Logging_System_Overview#Tag_Formatting|CamelCase]]

The logging macros behave like iostreams, and can have anything appended to them that can be appended to an iostream.

= Multiple Tag Logging Macros =

There also exist versions of the macros that allow you to bind 2 string tags to a log message (a "broad" scope tag and a "narrow" scope tag), which is very useful when you want to have both high and low granularity for tagging a log message (for example "AppInit" and "SystemInfo", where "AppInit" is the broad scope tag and describes the long Application Initialization process of which the narrow scope tag "SystemInfo" printing is a small portion).

'''LL_DEBUGS2'''("BroadScopeTag", "NarrowScopeTag") << "A debug level log message" << '''LL_ENDL'''; // '''NOTE:''' This is a macro, not a string stream. The "'''LL_ENDL'''" is required.
'''LL_INFOS2'''("BroadScopeTag", "NarrowScopeTag") << "An info level log message" << '''LL_ENDL''';
'''LL_WARNS2'''("BroadScopeTag", "NarrowScopeTag") << "A warning level log message" << '''LL_ENDL''';
'''LL_ERRS2'''("BroadScopeTag", "NarrowScopeTag") << "An error level log message" << '''LL_ENDL'''; // '''NOTE:''' '''LL_ERRS2''' force a crash of the viewer.

'''Note:''' Tags should be formatted in [[Logging_System_Overview#Tag_Formatting|CamelCase]]

= Logging a repeating message only once =

There are currently some log messages that if triggered end up spamming the log file with the same exact message string over and over. As a result, we now also have the ability to use a special macro that keeps these messages from filling up the log and drowning out other messages. These macros will print the message the first time it gets triggered and prepends "ONCE: " to the message string. Any further time it is triggered '''with the same message''' it will not print it, unless it is the 10th, 50th, or every 100th time there after that it is encountered. Then, it updates you by prepending "ONCE (Nth time seen): " to the log message, where N is the number of times it has been seen.

'''Note:''' This works on unique final log messages, so log messages that depend on one or more variables have the final output treated as different messages if any part of the string changes. Thus, you should not use _ONCE macros for log messages that will print many times but with different messages, as this would be slightly slower and would slowly fill up a std::map with memory occupying junk.

'''LL_DEBUGS_ONCE'''("BroadScopeTag") << "A debug level log message" << '''LL_ENDL'''; // NOTE: This is a macro, not a string stream. The "'''LL_ENDL'''" is required.
'''LL_INFOS_ONCE'''("BroadScopeTag") << "An info level log message" << '''LL_ENDL''';
'''LL_WARNS_ONCE'''("BroadScopeTag") << "A warning level log message" << '''LL_ENDL''';

'''LL_DEBUGS2_ONCE'''("BroadScopeTag", "NarrowScopeTag") << "A debug level log message" << '''LL_ENDL'''; // NOTE: This is a macro, not a string stream. The "'''LL_ENDL'''" is required.
'''LL_INFOS2_ONCE'''("BroadScopeTag", "NarrowScopeTag") << "An info level log message" << '''LL_ENDL''';
'''LL_WARNS2_ONCE'''("BroadScopeTag", "NarrowScopeTag") << "A warning level log message" << '''LL_ENDL''';

'''Note:''' Tags should be formatted in [[Logging_System_Overview#Tag_Formatting|CamelCase]]

For example, the following log message:

'''LL_WARNS_ONCE'''("Drawable") << "Drawable becomes static with active parent!" << '''LL_ENDL''';

would print the following messages to "'''SecondLife.log'''" if it kept getting called repeatedly:

WARNING: LLDrawable::makeStatic: ONCE: Drawable becomes static with active parent!
WARNING: LLDrawable::makeStatic: ONCE (10th time seen): Drawable becomes static with active parent!
WARNING: LLDrawable::makeStatic: ONCE (50th time seen): Drawable becomes static with active parent!
WARNING: LLDrawable::makeStatic: ONCE (100th time seen): Drawable becomes static with active parent!
WARNING: LLDrawable::makeStatic: ONCE (200th time seen): Drawable becomes static with active parent!
... etc ...

= Tag Formatting =

Tags should be formatted in "'''CamelCase'''", such as "RenderInit" and "TextureCache"

= Varying amount of logging at runtime (logcontrol.xml) =

Upon initialization of the application, the logging system tries to load a "'''logcontrol-dev.xml'''" file and if it doesn't exist, it loads "'''logcontrol.xml'''" instead. These files are located in the "app_settings/" directory for the viewer and in "/opt/linden/etc/" for the server ("<branch>/etc/" on Windows). Whichever file is loaded gets checked for changes every couple of seconds and reloaded if necessary. The following is a sample '''logcontrol.xml''' file that sets the default loggging level to WARN and then conditionally turns on some warnings at INFO level and DEBUG level based on "function name", "class name", "file name", and "tag".

Sample "'''logcontrol.xml'''" file:

<llsd>
<map>
<'''!--''' default-level can be '''ALL''', '''DEBUG''', '''INFO''', '''WARN''', '''ERROR''', or '''NONE''' '''--'''>
<key>'''default-level'''</key> <string>'''WARN'''</string>
<key>'''print-location'''</key> <boolean>'''false'''</boolean>
<key>'''settings'''</key
<array>
<map>
<key>'''level'''</key><string>'''INFO'''</string>
<key>'''functions'''</key>
<array>
<string>'''LLAgentInfo::handleImageRequest'''</string>
</array>
<key>'''classes'''</key>
<array>
<string>'''LLBufferArray'''</string>
</array>
<key>'''files'''</key>
<array>
<string>'''lldrawpoolavatar.cpp'''</string>
</array>
<key>'''tags'''</key>
<array>
<string>'''AppInit'''</string>
<string>'''TextureCache'''</string>
</array>
</map>
<map>
<key>'''level'''</key><string>'''DEBUG'''</string>
<key>'''functions'''</key>
<array>
</array>
<key>'''classes'''</key>
<array>
</array>
<key>'''files'''</key>
<array>
<string>'''llbutton.cpp'''</string>
<string>'''llcombobox.cpp'''</string>
</array>
<key>'''tags'''</key>
<array>
<string>'''RenderInit'''</string>
</array>
</map>
</array>
</map>
</llsd>

= Logging granularity conflict resolution =

The logging system decides in the following order what logging granularity to apply to a message:

'''Default log level''' < '''BroadScopeTag''' < '''File Name''' < '''Class Name''' < '''Function Name''' < '''NarrowScopeTag'''

Thus, highest priority level match found in "'''logcontrol.xml'''" is used to decide if a particular log message should be printed. For example, if a given log message is of a class that has been set in "'''logcontrol.xml'''" to print all '''LL_DEBUGS()''' and below, but there is an entry for its function name that states to only print '''LL_WARNS()''' and below, then only '''LL_WARNS()''' and '''LL_ERRS()''' will be printed.

= Continuing a log message after further computation =

Use the '''LL_CONT''' macro if you need to do computation in the middle of a log message:

'''LL_DEBUGS'''("AgentGesture") << "the agent " << agend_id;
switch (f)
{
case FOP_SHRUGS: '''LL_CONT''' << "shrugs"; break;
case FOP_TAPS: '''LL_CONT''' << "points at " << who; break;
case FOP_SAYS: '''LL_CONT''' << "says " << message; break;
}
'''LL_CONT''' << " for " << t << " seconds" << '''LL_ENDL'''; // Make sure to remember the '''LL_ENDL''' at the end and '''ONLY''' at the end.

Such computation is done only if the message will be logged.

This means that you can also use the macro functionality to conditionally call separate functions only if that granularity of logging is enabled, such as in the following example from llfeaturemanager.cpp:

'''LL_DEBUGS'''("RenderInit") << "After applying mask " << mask.mName << std::endl;
// Will conditionally call dump only if the above message will be logged, thanks
// to it being wrapped by the '''LL_DEBUGS''' and '''LL_ENDL''' macros.
dump();
'''LL_CONT''' << '''LL_ENDL''';

[[Category:Development]]
[[Category:Design Documents]]

User:Poppy Linden/PyOGP \ libdev Initial Setup Experience

2009-07-29T01:19:52Z

Poppy Linden: /* random hacking */

sequential, oldest first.

== bootstrapping ==
* omg tons of docs
* omg repeats internally / externally... lemme fix that...
* code?
* ok, what deps are required? wait.. how does python pathing work? oh, i can just put it in my python_path, cool
* wait, this says i need to install a friggin million things.
* I Hate Everything Python Right Now.
* I bootstrapped a wicked python env in my homedir.
<pre>
mkdir -p ~/local/lib/python2.4/site-packages
vi ~/.bashrc
>> export PYTHONPATH=$PYTHONPATH:~/local/lib/python2.4/site-packages
>> export PATH=$PATH:/usr/local/bin:~/bin:~/local/bin
. ~/.bashrc

curl -L 'http://pypi.python.org/packages/2.4/s/setuptools/setuptools-0.6c9-py2.4.egg#md5=260a2be2e5388d66bdaee06abec6342a' > setuptools-0.6c9-py2.4.egg
sh setuptools-0.6c9-py2.4.egg --prefix=~/local
easy_install --prefix=~/local pip
easy_install --prefix=~/local virtualenv
</pre>
* ok, now what again? Oh yeah, virtualenv for pyogp... the point of this...
** virtualenv . --no-site-packages
** bin/python bootstrap.py
** bin/buildout -v
** oh, so virtualenv is preventing the very borknorking i am trying to prevent to my system. Fair game.

== bootstrap / sanity verification ==
* bin/client_unittest
** FAILBOAT
** pip -E . install greenlet
** bin/client_unittest
** now a bunch of crazy shit instead of tracebacks, that's... good?
** 114 tests and 0 failures! that's good!

== initial investigations ==
* vi src/pyogp.lib.base/pyogp/lib/base/tests/base.py
** everything starts with login, even with base. What about the delicious primitives i crave?
** ls -l is your friend. There are subfolders in src/pyogp.lib.base/pyogp/lib/base/, namely one named network, and one named message.
* reading tests is like looking at a tautology; they have no docs. "Yes, i bet that does in fact work that way." What do I see to bootstrap an app?
* i say: "(i want to know what code to see to understand) how to init main, send a message, quit" enus: "well, region.py uses UDPDispatcher. i've not tried using it w/o the entire context" "maybe test_udpconnection.py is a useful reference. it instantiates UDPDispatcher, against a mock host, and sends a message"
* enus: "the Message usage is changing soon. look at the Message() class." "packets.py transforms representations of packet data into Message instqances, but packets.py is going away soon"
* there *really* needs to be a step-by-step to get bootstrapped, I hope this helps write that! I'm still not writing "Hello World" with pyogp yet!
* test app, try 1
** mkdir test_dataserver; cd test_dataserver
** virtualenv . --no-site-packages
** RTFM: pip help install | less
** pip install -E . -e ../libdev/ libdev ... FAIL
** pip install -E . -e svn+../libdev/ libdev ... FAIL
** pip install -E . -e svn+http://svn.secondlife.com/svn/linden/projects/2008/pyogp/buildouts/libdev/trunk libdev ... FAIL
** OH WAIT the problem isn't the dir, it's the "libdev" afterwards
** pip install -E . -e svn+http://svn.secondlife.com/svn/linden/projects/2008/pyogp/buildouts/libdev/trunk ... FAIL but a new error, no setup.py... wait a sec...
** pip install -E . -e svn+http://svn.secondlife.com/svn/linden/projects/2008/pyogp/pyogp.lib.base/trunk/ ... WIN
* should work to just copy a unit test into my src/ dir, copying test_udpconnection.py as it sounds like a good start
* pip install -E . greenlet
* bin/python src/test_udpconnection.py ... aaaaaand... no output. Is that a good thing for a test?
* need some of this:
<python>
if __name__ == "__main__":
unittest.main()
</python>
* tests pass. Rad! Let's send some garbage.

== learn to send messages ==
* pyogp.lib.base/pyogp/lib/base/message/circuit.py - the Host class looks a little thin; doesn't take a FQDN, just an IP addy?
* resetting the host in test_find_circuit in my modified copy of test_udpconnection.py doesn't seem to be hitting the desired host / port
* try to reverse-engineer class diagram using pylint and graphviz... hmm...
* finally get *something* - a couple circuits established with the UDPConnectionTest replacing the mock server with my code
* it's making me crazy that MockupTestServer and Client don't seem to map to a real object. It appears to be impossible to trivially modify test_udpconnection.py to connect to a real server instead of a mock.
* so after some tinkering, this works:
<pre>
p = packet_types.AvatarPropertiesRequestPacket()
poppy = UUID()
print dir(poppy)
print poppy.uuid.bytes
p.AgentData['AgentID'] = poppy
p.AgentData['SessionID'] = UUID()
p.AgentData['AvatarID'] = poppy
self.send_message(p, True)
</pre>
* need a coroutine to sit and wait for response, not that far yet
* much of this functionality is in region.py, should definitely be brought out and into the message package somehow
* chat with enus about branching, possibly mercurial?
* full test, with reliable acks:
<python>
def test_asdf(self):
# build message, send it
packet = packet_types.AvatarPropertiesRequestPacket()
poppy = UUID()
packet.AgentData['AgentID'] = poppy
packet.AgentData['SessionID'] = UUID()
packet.AgentData['AvatarID'] = poppy
#self.messenger.send_message(packet, self.host)
self.messenger.send_reliable(packet, self.host, 0)
# wait for Ack, socket timeout after 1s wait (1000ms)
self.messenger.socket.settimeout(1.0)
msg_buf, msg_size = self.messenger.udp_client.receive_packet(self.messenger.socket)
recv = self.messenger.receive_check(self.messenger.udp_client.get_sender(), msg_buf, msg_size)
if recv is None or recv.name != 'PacketAck':
self.fail("no data waiting after 1s")
</python>

== random hacking ==
* can't seem to introspect then send all the packet types from dir() on pyogp.lib.base.message.packets
* that's gone, but no obvious way to get message from template...
* eventlet 0.8.12 cheeseshop install seems borked, using 'pip install -E . -f http://pypi.python.org/packages/source/e/eventlet/eventlet-0.8.11.tar.gz eventlet-0.8.11'
* install nose: easy_install --prefix=~/local nose
* bin/python `which nosetests` src/*
* yeah, i want to keep a mercurial fork: hg convert http://svn.secondlife.com/svn/linden/projects/2008/pyogp/pyogp.lib.base/trunk

User:Poppy Linden/PyOGP \ libdev Initial Setup Experience

2009-07-28T05:42:47Z

Poppy Linden: /* random hacking */

User:Poppy Linden/PyOGP \ libdev Initial Setup Experience

2009-07-28T05:30:40Z

Poppy Linden: /* random hacking */

User:Poppy Linden/PyOGP \ libdev Initial Setup Experience

2009-07-28T03:16:23Z

Poppy Linden: /* random hacking */

User:Poppy Linden/PyOGP \ libdev Initial Setup Experience

2009-07-24T01:55:43Z

Poppy Linden:

User:Poppy Linden/PyOGP \ libdev Initial Setup Experience

2009-07-10T02:27:41Z

Poppy Linden: /* learn to send messages */

User:Poppy Linden/PyOGP \ libdev Initial Setup Experience

2009-07-09T02:54:07Z

Poppy Linden: /* learn to send messages */

User:Poppy Linden/PyOGP \ libdev Initial Setup Experience

2009-07-08T23:28:14Z

Poppy Linden: /* bootstrap / sanity verification */