Difference between revisions of "Packet Layout"
(Changed message_template URL to viewer source repo which is kept up to date) |
|||
(39 intermediate revisions by 11 users not shown) | |||
Line 1: | Line 1: | ||
{{OSWikiLearnBox|parent=Protocol}} | |||
==Header (Packet ID)== | |||
See also: | <pre width=80> | ||
+-+-+-+-+----+--------+--------+--------+--------+--------+-----...-----+ | |||
|Z|R|R|A| | | | Extra | | |||
|E|E|E|C| | Sequence number (4 bytes) | Extra | Header | | |||
|R|L|S|K| | | (byte) | (N bytes) | | |||
+-+-+-+-+----+--------+--------+--------+--------+--------+-----...-----+ | |||
</pre> | |||
* Byte 0: Flags | |||
** <code>0x80</code> <code>LL_ZERO_CODE_FLAG</code> -- zeroes in ''packet body'' are [https://en.wikipedia.org/wiki/Run-length_encoding run-length encoded], such that series of 1 to 255 zero-bytes are encoded to take 2 bytes (first the zero, then the count). See [[Zerocode]] for example implementations. | |||
** <code>0x40</code> <code>LL_RELIABLE_FLAG</code> -- This packet was sent reliably (meaning "please acknowledge this packet") | |||
** <code>0x20</code> <code>LL_RESENT_FLAG</code> -- This packet is a resend from the source. | |||
** <code>0x10</code> <code>LL_ACK_FLAG</code> -- This packet contains appended ACKs. | |||
** The 4 least significant bits of byte 0 are currently unused. | |||
* Bytes 1-4: Sequence number as a big-endian unsigned integer | |||
* Byte 5: How many bytes of extra header is available. | |||
If byte 5 is non-zero, then there is some extra header information. Clients which are not expecting that header information may skip it by jumping forward 'Extra' bytes into the message payload. | |||
The term "Packet ID" is sometimes used to refer to all 4 bytes and sometimes just the sequence number. | |||
For further discussion of the bits in the header, see [[Messages]] and [[Packet Accounting]] | |||
==Body== | |||
===Message number=== | |||
Starting from byte 6, this is a variable-length numeric encoding (big-endian) of the message number and frequency defined in the [https://github.com/secondlife/viewer/blob/develop/scripts/messages/message_template.msg message_template.msg] file. The message number may be 1, 2, or 4 bytes in length, depending on the message frequency, and they are used to look up the correct packet in the message template; the contents of that template are then used to decode the rest of the packet, which is specified in terms of blocks and fields. | |||
<pre> | |||
[ header ] [ msg num ] [ data ] | |||
Fixed: .. .. .. .. .. FF FF FF XX .. .. .. | |||
Low: .. .. .. .. .. FF FF XX XX .. .. .. | |||
Medium: .. .. .. .. .. FF XX .. .. .. .. .. | |||
High: .. .. .. .. .. XX .. .. .. .. .. .. | |||
</pre> | |||
<code>0xFFFFFFFA</code> - <code>0xFFFFFFFF</code> are Fixed frequency messages. | |||
<code>0xFFFF0001</code> - <code>0xFFFFFFF9</code> are Low frequency messages. | |||
<code>0xFF01</code> - <code>0xFFFE</code> are Medium frequency messages. | |||
<code>0x01</code> - <code>0xFE</code> are High frequency messages. | |||
Messages with "Fixed" frequency are those with truly fixed message numbers, i.e. the numbers are assigned in the template file itself, for example [[PacketAck]] is assigned <code>0xFFFFFFFB</code>. Even though all messages have numbers assigned to them in the template file, the Fixed frequency is kept around for legacy reasons. | |||
===Message data=== | |||
This is different for each message type, and is defined in the template file. | |||
The message template details how many blocks are in the packet, which can be a fixed or variable number specified in the packet. (The block names in the template are for human readers; they are not encoded in the message.) | |||
* For blocks marked <code>Single</code>, <code>Fixed</code>, or <code>Multiple</code>, no block header bytes are added. | |||
* If a block is marked <code>Variable</code>, one byte is stored in the message indicating how many times that block has been repeated. | |||
** If a block is marked <code>Variable 2</code>, two bytes is stored as little-endian. A count of 7 would be specified with <code>0x07 0x00</code>. | |||
Each packet contains exactly one message. Thus, the length of the message is known before message decoding: | |||
* <code>5</code> bytes for the header. | |||
* <code>4</code> bytes for message number. (+1 if the packet is zerocoded) | |||
* <code>N</code> extra bytes denoted by the 5th byte. (see above) | |||
* <code>M</code> bytes for message data. (see template) | |||
===Extensions for backwards compatibility=== | |||
Some features were added in 2023-2024 to allow extensions to the message format while retaining backwards compatibility. | |||
# Messages with excess bytes beyond what the template calls for are accepted and decoded, not rejected as errors. This allows old message recipients to process newly expanded messages. | |||
# Messages with fewer bytes than expected are allowed for a specific special case. If the last block in the message is "Variable", indicating zero or more repeats of the following block, the decoder expects a one-byte count of the number of repeats of the block. If the message body ends just before that byte, the message should be interpreted as a "Variable" block with zero repeats. | |||
An example of such a message is "AvatarAppearance". Compare these versions of the message template: | |||
* Old: [https://github.com/secondlife/viewer/blob/DRTVWR-591-maint-X/scripts/messages/message_template.msg] | |||
* New: [https://github.com/secondlife/viewer/blob/DRTVWR-600-maint-A/scripts/messages/message_template.msg] | |||
Newer versions of the message template entry for AvatarAppearance contain the section | |||
<pre> | |||
{ | |||
AttachmentBlock Variable | |||
{ ID LLUUID } | |||
{ AttachmentPoint U8 } | |||
} | |||
</pre> | |||
The first extended parsing rule above allows a reader working from the old template to parse a message generated with the new template, ignoring the new fields. The second rule allows a reader working from the new template to parse a message generated with the old one, treating the new fields as not present. | |||
==Appended Acks== | |||
* Bytes (6 + extra + (data length)) through acks | |||
** Rest of packet is filled with as many acks from previous reliable messages as will fit. | |||
** The last byte of the packet is a count of how many acks are included (if LL_ACK_FLAG is set). | |||
See also: | |||
:[http://lib.openmetaverse.org/wiki/Protocol_%28network%29 libopenmetaverse documentation] | |||
:[[AW_Groupies#Documenting_current_protocols|AW Groupies documentation]] |
Latest revision as of 21:03, 22 September 2024
Header (Packet ID)
+-+-+-+-+----+--------+--------+--------+--------+--------+-----...-----+ |Z|R|R|A| | | | Extra | |E|E|E|C| | Sequence number (4 bytes) | Extra | Header | |R|L|S|K| | | (byte) | (N bytes) | +-+-+-+-+----+--------+--------+--------+--------+--------+-----...-----+
- Byte 0: Flags
0x80
LL_ZERO_CODE_FLAG
-- zeroes in packet body are run-length encoded, such that series of 1 to 255 zero-bytes are encoded to take 2 bytes (first the zero, then the count). See Zerocode for example implementations.0x40
LL_RELIABLE_FLAG
-- This packet was sent reliably (meaning "please acknowledge this packet")0x20
LL_RESENT_FLAG
-- This packet is a resend from the source.0x10
LL_ACK_FLAG
-- This packet contains appended ACKs.- The 4 least significant bits of byte 0 are currently unused.
- Bytes 1-4: Sequence number as a big-endian unsigned integer
- Byte 5: How many bytes of extra header is available.
If byte 5 is non-zero, then there is some extra header information. Clients which are not expecting that header information may skip it by jumping forward 'Extra' bytes into the message payload.
The term "Packet ID" is sometimes used to refer to all 4 bytes and sometimes just the sequence number.
For further discussion of the bits in the header, see Messages and Packet Accounting
Body
Message number
Starting from byte 6, this is a variable-length numeric encoding (big-endian) of the message number and frequency defined in the message_template.msg file. The message number may be 1, 2, or 4 bytes in length, depending on the message frequency, and they are used to look up the correct packet in the message template; the contents of that template are then used to decode the rest of the packet, which is specified in terms of blocks and fields.
[ header ] [ msg num ] [ data ] Fixed: .. .. .. .. .. FF FF FF XX .. .. .. Low: .. .. .. .. .. FF FF XX XX .. .. .. Medium: .. .. .. .. .. FF XX .. .. .. .. .. High: .. .. .. .. .. XX .. .. .. .. .. ..
0xFFFFFFFA
- 0xFFFFFFFF
are Fixed frequency messages.
0xFFFF0001
- 0xFFFFFFF9
are Low frequency messages.
0xFF01
- 0xFFFE
are Medium frequency messages.
0x01
- 0xFE
are High frequency messages.
Messages with "Fixed" frequency are those with truly fixed message numbers, i.e. the numbers are assigned in the template file itself, for example PacketAck is assigned 0xFFFFFFFB
. Even though all messages have numbers assigned to them in the template file, the Fixed frequency is kept around for legacy reasons.
Message data
This is different for each message type, and is defined in the template file.
The message template details how many blocks are in the packet, which can be a fixed or variable number specified in the packet. (The block names in the template are for human readers; they are not encoded in the message.)
- For blocks marked
Single
,Fixed
, orMultiple
, no block header bytes are added. - If a block is marked
Variable
, one byte is stored in the message indicating how many times that block has been repeated.- If a block is marked
Variable 2
, two bytes is stored as little-endian. A count of 7 would be specified with0x07 0x00
.
- If a block is marked
Each packet contains exactly one message. Thus, the length of the message is known before message decoding:
5
bytes for the header.4
bytes for message number. (+1 if the packet is zerocoded)N
extra bytes denoted by the 5th byte. (see above)M
bytes for message data. (see template)
Extensions for backwards compatibility
Some features were added in 2023-2024 to allow extensions to the message format while retaining backwards compatibility.
- Messages with excess bytes beyond what the template calls for are accepted and decoded, not rejected as errors. This allows old message recipients to process newly expanded messages.
- Messages with fewer bytes than expected are allowed for a specific special case. If the last block in the message is "Variable", indicating zero or more repeats of the following block, the decoder expects a one-byte count of the number of repeats of the block. If the message body ends just before that byte, the message should be interpreted as a "Variable" block with zero repeats.
An example of such a message is "AvatarAppearance". Compare these versions of the message template:
- Old: [1]
- New: [2]
Newer versions of the message template entry for AvatarAppearance contain the section
{ AttachmentBlock Variable { ID LLUUID } { AttachmentPoint U8 } }
The first extended parsing rule above allows a reader working from the old template to parse a message generated with the new template, ignoring the new fields. The second rule allows a reader working from the new template to parse a message generated with the old one, treating the new fields as not present.
Appended Acks
- Bytes (6 + extra + (data length)) through acks
- Rest of packet is filled with as many acks from previous reliable messages as will fit.
- The last byte of the packet is a count of how many acks are included (if LL_ACK_FLAG is set).
See also: