Difference between revisions of "Packet Layout"

From Second Life Wiki
Jump to navigation Jump to search
(Changed message_template URL to viewer source repo which is kept up to date)
 
(19 intermediate revisions by 8 users not shown)
Line 6: Line 6:


<pre width=80>
<pre width=80>
  +-+-+-+-+-------+---------------+---------------+---------------+
  +-+-+-+-+----+--------+--------+--------+--------+--------+-----...-----+
  |Z|R|R|A|       |                                               |
  |Z|R|R|A|   |                                   |        |  Extra      |
  |E|E|E|C|       |           Sequence number (24 bits)           |
  |E|E|E|C|   |   Sequence number (4 bytes)     | Extra  |  Header    |
  |R|L|S|K|       |                                               |
  |R|L|S|K|   |                                   | (byte) | (N bytes)  |
  +-+-+-+-+-------+---------------+---------------+---------------+
  +-+-+-+-+----+--------+--------+--------+--------+--------+-----...-----+
|                                                              |
:                                                              :
</pre>
</pre>
* Byte 0, 4 most significant bits: Flags
 
** LL_ZERO_CODE_FLAG 0x80 -- 0's in packet body are run length encoded, such that series of 1 to 255 zero bytes are encoded to take 2 bytes.
* Byte 0: Flags
** LL_RELIABLE_FLAG 0x40 -- This packet was sent reliably (implies please ack this packet)
** <code>0x80</code> <code>LL_ZERO_CODE_FLAG</code> -- zeroes in ''packet body'' are [https://en.wikipedia.org/wiki/Run-length_encoding run-length encoded], such that series of 1 to 255 zero-bytes are encoded to take 2 bytes (first the zero, then the count). See [[Zerocode]] for example implementations.
** LL_RESENT_FLAG 0x20 -- This packet is a resend from the source.
** <code>0x40</code> <code>LL_RELIABLE_FLAG</code> -- This packet was sent reliably (meaning "please acknowledge this packet")
** LL_ACK_FLAG 0x10 -- This packet contains appended acks.
** <code>0x20</code> <code>LL_RESENT_FLAG</code> -- This packet is a resend from the source.
* Byte 0, 4 least significant bits: currently unused
** <code>0x10</code> <code>LL_ACK_FLAG</code> -- This packet contains appended ACKs.
* Bytes 1-3: Sequence number
** The 4 least significant bits of byte 0 are currently unused.
* Bytes 1-4: Sequence number as a big-endian unsigned integer
* Byte 5: How many bytes of extra header is available.
 
If byte 5 is non-zero, then there is some extra header information. Clients which are not expecting that header information may skip it by jumping forward 'Extra' bytes into the message payload.


The term "Packet ID" is sometimes used to refer to all 4 bytes and sometimes just the sequence number.
The term "Packet ID" is sometimes used to refer to all 4 bytes and sometimes just the sequence number.
Line 27: Line 29:


==Body==
==Body==
* Bytes 4 to 4 + (data length)
===Message number===
** Message number.  This is a numeric encoding of the message types defined in the message_template.msg file. It may be 1, 2 or 4 bytes in length, depending on the message frequency (High, Medium, Low or Fixed).
Starting from byte 6, this is a variable-length numeric encoding (big-endian) of the message number and frequency defined in the [https://github.com/secondlife/viewer/blob/develop/scripts/messages/message_template.msg message_template.msg] file. The message number may be 1, 2, or 4 bytes in length, depending on the message frequency, and they are used to look up the correct packet in the message template; the contents of that template are then used to decode the rest of the packet, which is specified in terms of blocks and fields.
*** High frequency messages are assigned (at run time) numbers 0x01 - 0xFE.
 
*** Medium frequency messages are assigned (at run time) numbers 0xFF01 - 0xFFFE.
<pre>
*** Low frequency messages are assigned (at run time) numbers 0xFFFF0001 and up.
        [  header  ] [ msg num ] [ data ]
*** Messages with "Fixed" frequency are really those with fixed message numbers, i.e. the numbers are assigned in the message_template.msg file itself. There are currently 6 of these, 0xFFFFFFFA - 0xFFFFFFFF.
Fixed:  .. .. .. .. .. FF FF FF XX .. .. ..
** Message dataThis is different for each message type, and is defined in the message_template.msg file.
Low:    .. .. .. .. .. FF FF XX XX .. .. ..
Medium: .. .. .. .. .. FF XX .. .. .. .. ..
High:  .. .. .. .. .. XX .. .. .. .. .. ..
</pre>
 
<code>0xFFFFFFFA</code> - <code>0xFFFFFFFF</code> are Fixed frequency messages.
 
<code>0xFFFF0001</code> - <code>0xFFFFFFF9</code> are Low frequency messages.
 
<code>0xFF01</code> - <code>0xFFFE</code> are Medium frequency messages.
 
<code>0x01</code> - <code>0xFE</code> are High frequency messages.
 
Messages with "Fixed" frequency are those with truly fixed message numbers, i.e. the numbers are assigned in the template file itself, for example [[PacketAck]] is assigned <code>0xFFFFFFFB</code>. Even though all messages have numbers assigned to them in the template file, the Fixed frequency is kept around for legacy reasons.
 
===Message data===
This is different for each message type, and is defined in the template file.
 
The message template details how many blocks are in the packet, which can be a fixed or variable number specified in the packet. (The block names in the template are for human readers; they are not encoded in the message.)
 
* For blocks marked <code>Single</code>, <code>Fixed</code>, or <code>Multiple</code>, no block header bytes are added.
* If a block is marked <code>Variable</code>, one byte is stored in the message indicating how many times that block has been repeated.
** If a block is marked <code>Variable 2</code>, two bytes is stored as little-endian. A count of 7 would be specified with <code>0x07 0x00</code>.
 
Each packet contains exactly one message. Thus, the length of the message is known before message decoding:
 
* <code>5</code> bytes for the header.
* <code>4</code> bytes for message number. (+1 if the packet is zerocoded)
* <code>N</code> extra bytes denoted by the 5th byte. (see above)
* <code>M</code> bytes for message data. (see template)
 
===Extensions for backwards compatibility===
Some features were added in 2023-2024 to allow extensions to the message format while retaining backwards compatibility.
 
# Messages with excess bytes beyond what the template calls for are accepted and decoded, not rejected as errors. This allows old message recipients to process newly expanded messages.
# Messages with fewer bytes than expected are allowed for a specific special case. If the last block in the message is "Variable", indicating zero or more repeats of the following block, the decoder expects a one-byte count of the number of repeats of the block. If the message body ends just before that byte, the message should be interpreted as a "Variable" block with zero repeats.
 
An example of such a message is "AvatarAppearance". Compare these versions of the message template:
 
* Old: [https://github.com/secondlife/viewer/blob/DRTVWR-591-maint-X/scripts/messages/message_template.msg]
 
* New: [https://github.com/secondlife/viewer/blob/DRTVWR-600-maint-A/scripts/messages/message_template.msg]
 
Newer versions of the message template entry for AvatarAppearance contain the section
<pre>
{
    AttachmentBlock Variable
    { ID              LLUUID  }
    { AttachmentPoint U8 }
}
</pre>
 
The first extended parsing rule above allows a reader working from the old template to parse a message generated with the new template, ignoring the new fields. The second rule allows a reader working from the new template to parse a message generated with the old one, treating the new fields as not present.


==Appended Acks==
==Appended Acks==
* Bytes 4+(data length) thru acks
* Bytes (6 + extra + (data length)) through acks
** Rest of packet is filled with as many acks from previous reliable messages as will fit.
** Rest of packet is filled with as many acks from previous reliable messages as will fit.
** The last byte of the packet is a count of how many acks are included (if LL_ACK_FLAG is set).


 
See also:
See also: [http://www.libsecondlife.org/wiki/Protocol_%28network%29 libsecondlife documentation]
:[http://lib.openmetaverse.org/wiki/Protocol_%28network%29 libopenmetaverse documentation]
:[[AW_Groupies#Documenting_current_protocols|AW Groupies documentation]]

Latest revision as of 21:03, 22 September 2024


Header (Packet ID)

 +-+-+-+-+----+--------+--------+--------+--------+--------+-----...-----+
 |Z|R|R|A|    |                                   |        |  Extra      |
 |E|E|E|C|    |    Sequence number (4 bytes)      | Extra  |  Header     |
 |R|L|S|K|    |                                   | (byte) | (N bytes)   |
 +-+-+-+-+----+--------+--------+--------+--------+--------+-----...-----+
  • Byte 0: Flags
    • 0x80 LL_ZERO_CODE_FLAG -- zeroes in packet body are run-length encoded, such that series of 1 to 255 zero-bytes are encoded to take 2 bytes (first the zero, then the count). See Zerocode for example implementations.
    • 0x40 LL_RELIABLE_FLAG -- This packet was sent reliably (meaning "please acknowledge this packet")
    • 0x20 LL_RESENT_FLAG -- This packet is a resend from the source.
    • 0x10 LL_ACK_FLAG -- This packet contains appended ACKs.
    • The 4 least significant bits of byte 0 are currently unused.
  • Bytes 1-4: Sequence number as a big-endian unsigned integer
  • Byte 5: How many bytes of extra header is available.

If byte 5 is non-zero, then there is some extra header information. Clients which are not expecting that header information may skip it by jumping forward 'Extra' bytes into the message payload.

The term "Packet ID" is sometimes used to refer to all 4 bytes and sometimes just the sequence number.

For further discussion of the bits in the header, see Messages and Packet Accounting

Body

Message number

Starting from byte 6, this is a variable-length numeric encoding (big-endian) of the message number and frequency defined in the message_template.msg file. The message number may be 1, 2, or 4 bytes in length, depending on the message frequency, and they are used to look up the correct packet in the message template; the contents of that template are then used to decode the rest of the packet, which is specified in terms of blocks and fields.

        [   header   ] [ msg num ] [ data ]
Fixed:  .. .. .. .. .. FF FF FF XX .. .. ..
Low:    .. .. .. .. .. FF FF XX XX .. .. ..
Medium: .. .. .. .. .. FF XX .. .. .. .. ..
High:   .. .. .. .. .. XX .. .. .. .. .. ..

0xFFFFFFFA - 0xFFFFFFFF are Fixed frequency messages.

0xFFFF0001 - 0xFFFFFFF9 are Low frequency messages.

0xFF01 - 0xFFFE are Medium frequency messages.

0x01 - 0xFE are High frequency messages.

Messages with "Fixed" frequency are those with truly fixed message numbers, i.e. the numbers are assigned in the template file itself, for example PacketAck is assigned 0xFFFFFFFB. Even though all messages have numbers assigned to them in the template file, the Fixed frequency is kept around for legacy reasons.

Message data

This is different for each message type, and is defined in the template file.

The message template details how many blocks are in the packet, which can be a fixed or variable number specified in the packet. (The block names in the template are for human readers; they are not encoded in the message.)

  • For blocks marked Single, Fixed, or Multiple, no block header bytes are added.
  • If a block is marked Variable, one byte is stored in the message indicating how many times that block has been repeated.
    • If a block is marked Variable 2, two bytes is stored as little-endian. A count of 7 would be specified with 0x07 0x00.

Each packet contains exactly one message. Thus, the length of the message is known before message decoding:

  • 5 bytes for the header.
  • 4 bytes for message number. (+1 if the packet is zerocoded)
  • N extra bytes denoted by the 5th byte. (see above)
  • M bytes for message data. (see template)

Extensions for backwards compatibility

Some features were added in 2023-2024 to allow extensions to the message format while retaining backwards compatibility.

  1. Messages with excess bytes beyond what the template calls for are accepted and decoded, not rejected as errors. This allows old message recipients to process newly expanded messages.
  2. Messages with fewer bytes than expected are allowed for a specific special case. If the last block in the message is "Variable", indicating zero or more repeats of the following block, the decoder expects a one-byte count of the number of repeats of the block. If the message body ends just before that byte, the message should be interpreted as a "Variable" block with zero repeats.

An example of such a message is "AvatarAppearance". Compare these versions of the message template:

Newer versions of the message template entry for AvatarAppearance contain the section

{
    AttachmentBlock	Variable
    {	ID              LLUUID  }
    {	AttachmentPoint U8 }
}

The first extended parsing rule above allows a reader working from the old template to parse a message generated with the new template, ignoring the new fields. The second rule allows a reader working from the new template to parse a message generated with the old one, treating the new fields as not present.

Appended Acks

  • Bytes (6 + extra + (data length)) through acks
    • Rest of packet is filled with as many acks from previous reliable messages as will fit.
    • The last byte of the packet is a count of how many acks are included (if LL_ACK_FLAG is set).

See also:

libopenmetaverse documentation
AW Groupies documentation