Pyogp/Client Lib/Packet

From Second Life Wiki
Jump to navigation Jump to search

This information can also be found in a couple of places. It is listed here as another explanation, hopefully to make it more understandable.

General Knowledge

1 Byte = 8 Bits
One Hex character uses 4 bits, or half a Byte. So it takes 2 Hex characters to make up a Byte.

  • bits 11110000 = F0 hex (or 0xF0)
  • in other words, hex 0xF0 can be turned into 11110000

This means that a given Byte can be broken up into two groups of 4 bits, with each group of 4 bits being represented by a single Hex value. The first 4 bits (starting from the left) can be considered the most significant bits. The last 4 bits (the four bits at the right) can be considered the most significant bits.

  • In the binary number 11110000, 11110000 are most significant bits, 11110000 are least significant bits
  • Shown in hex, 0xF0, where F are the most significant bits, 0 are the least significant bits

Examples

  • 00010001 binary (bits) = 11 hex = 17 decimal
  • 00010001 = 0x11 = 17

Packets

This is an attempt to make packets more clear, using specific examples in Python.
References:

Packets are made up of a header and a body. To construct the packets in Python, use the Python struct module. The struct module has a function called pack, which takes a format that the packet will be formatted into, and also takes the values (in Hex form). The pack function outputs a string that can be sent over the network through a socket, which can then be unpacked and put back into a form that can be read easier (hex, decimal, etc). For more information, see Struct module.

Overview

Byte 0: packet flags
Bytes 1 - 4: sequence number
Byte 5: extra header information byte (thought about as the offset length to the data/payload)
Bytes 6 - n: packet payload. This is a sequence of block information and block variable information.
    START (REPEAT FOR EACH BLOCK)
        If block type is VARIABLE, one byte dedicated to number of repeats for the block
            START (REPEAT FOR EACH VARIABLE)
                If variable type is VARIABLE, either 1,2, or 4 bytes is dedicated to the size of the variable
                Variable data
            END
    END

Header

The header is usually made up of 5 bytes, all in Big Endian form.

Byte 0 (unsigned char)

This should be a 2-digit hex value. The most significant bits are used to indicate a characteristic of the packet. The 4 least significant bits are unused. So, as a hex value, the value should always be of the form 0xV0, where V is the flag to set. The following flags are available:

  • 0x80 - LL_ZERO_CODE_FLAG - 0's in packet body are run length encoded, such that series of 1 to 255 zero bytes are encoded to take 2 bytes.
  • 0x40 - LL_RELIABLE_FLAG - This packet was sent reliably (implies please ack this packet)
  • 0x20 - LL_RESENT_FLAG - This packet is a resend from the source.
  • 0x10 - LL_ACK_FLAG - This packet contains appended acks.

Bytes 1-4 (unsigned long)

Bytes 1 through 4 (4 Bytes is the size of a typical integer, or long) are the sequence number. This number is pass back and forth between the client and server to note which packet is being sent in the sequence. The sequence number essentially represents where the packet is in the sequence of communication between client and server, so to be sure that we are receiving messages in order. The first packet, then, should have the sequence number 1, or 0x00000001.

Byte 5 (unsigned char)

This is a byte that signifies if there is any extra header information. It is usually 0 to signify that there is not an extra header. If it is non-zero, the number represents how many extra bytes are in the extra header.

Body

Bytes 6-n (sequences of unsigned chars) aka Packet ID

The next series of Bytes are used to represent the Packet ID (or Message ID). The Packet ID is used to determine which message is being sent, and therefore find out what the message data format will be. However, the Packet ID is actually logically broken up into 2 parts: the message frequency and a message ID number. The combination of the frequency and the Packet ID can be used as a key in the Message Template to find exactly what message is being sent. They take the form (frequency, message ID number). The Message Template can be found in scripts\messages\message_template.msg of the release, which will be included in the Pyogp project repo.

  • As an example, a key to get the AddCircuitCode packet will be ('Low', 2), with 'Low' being the frequency and 2 being the message ID number. However, this combination is actually represented by a single sequence of Bytes, 0xFFFF0002. The AddCircuitCode packet can be determined either directly by the value the Bytes represent, or we can parse the Bytes to find its sequence and message ID. The Message Template represents the packets by the form ('Low', 2) and not by the hex value, so we usually must parse the Bytes to find the frequency and ID in them.
  • ('Low, 2) is the AddCircuitCode packet, aka, 0xFFFF0002
  • ('Medium', 2) is the MultipleObjectUpdate packet, aka, 0xFF02
  • ('High', 2) is the CompletePingCheck packet, aka, 0x02

In order to determine the Packet ID to be used as a look-up into the Message Template, we have to parse Bytes 6-n. There can be 1, 2, or 4 Bytes (n can be either 1, 2, or 4) used to represent the Packet ID, hence the 6-n. The number of Bytes used is determined by the frequency of the packet. Every message has a particular frequency (also called message number or category), or how often it is typically sent over the network. There are 4 different frequencies of messages, "Low", "Medium","High", and "Fixed". Due to their frequency, messages are assigned a certain number of Bytes to represent their Packet IDs (a High frequency packet should need less Bytes so to not clog the network). High frequency packets have a single Byte to represent the Packet ID. Medium frequency packets use 2 Bytes to represent their Packet ID. Low frequency packets use 4 Bytes to represent their Packet ID. Knowing how many Bytes each category of messages use, we also have to know how to parse the Bytes to determine their frequency and message ID number.

Fixed:  FF FF FF xx 
Low:    FF FF xx xx 
Medium: FF xx .. .. 
High:   xx .. .. ..

Where FF means that the Byte must be FF, and where xx represents where the message ID number will be in the sequence of Bytes.
Frequency Bytes How to identify Values Examples
High frequency messages 1 Byte does not have 0xFF as Byte 6 assigned (from the template file) numbers 0x01 - 0xFE
  • 0x01 - ('High', 1)
  • 0xFE - ('High', 254)
Medium frequency messages 2 Bytes 0xFF as Byte 6, does not have an 0xFF as Byte 7 assigned (derived from numbers in the template file) numbers 0xFF01 - 0xFFFE
  • 0xFF01 - ('Medium', 1)
  • 0xFFFE - ('Medium', 254)
  • Notice that in both cases, the first FF are not used for the message ID number. The first FF Byte is used only to indicate that it is a medium frequency packet. The following Byte (or 2 Hex values) represents the message ID.
Low frequency messages 4 Bytes 0xFFFF as Byte 6 and Byte 7 are assigned (derived from numbers in the template file) numbers 0xFFFF0001 and up
  • 0xFFFF0001 - ('Low', 1)
  • 0xFFFFFFF9 - ('Low', 65529)
  • Notice that in both cases, the first FFFF Bytes are not used for the message ID number. The first FFFF Bytes are only used to indicate that it is a low frequency packet. The following 2 Bytes (or 4 Hex values) actually represent the message ID.
Fixed Frequency

Messages with "Fixed" frequency are really those with fixed message numbers, i.e. the numbers are assigned in the message_template.msg file itself. There are currently 6 of these, 0xFFFFFFFA - 0xFFFFFFFF. Even though all messages have numbers assigned to them in the template file, the Fixed frequency is kept around for legacy reasons.

Bytes after Packet ID aka Payload

After we get and decode the Bytes for the Packet ID, we can look up the packet in the Message Template and determine what its payload will be. We can figure out what blocks and data will be in the payload and so we can read in all the data.
What is written as payload depends on the message. Each message has a different payload, based on the message's block specifications as well as each of the blocks' variable specifications.
After the Packet ID is decoding each of the blocks for the message are written to the payload. A message can have any number of blocks. The first thing written for a block depends on the block type. If the block type is VARIABLE, then the first byte written for the block is the number of repeats for the block. Note: if it is MULTIPLE we don't have to write it because it is stored in the template (and is therefore FIXED). Also, if it is SINGLE, we don't have to write the number because it is always 1.
Then, the variables for the block are written to the payload. Again, the first thing written depends on the type of the variable. If the variable is type VARIABLE (confusing, I know) then the first thing written for the payload is the size of the variable. The maximum size of the variable can be either 1,2, or 4 bytes long, which is determined by the message template (and hence is read from the template). In other words, if the message template variable that is being written is listed as VARIABLE 1, then the variable's payload can have a length that is no longer than 1 byte (so, max length is 255). If it is VARIABLE 2, then max length is 2 bytes (65535).
Finally, the actual data is packed. The data is packed based on the variable type. For instance, if the type is U8, then a single unsigned byte is written to the payload and so on.

Bytes after Payload aka Acks

After we get the payload, there will be Bytes there for any Acks that were previously requested.