Hegemons Login Analysis

This page is intended to share/organize the information I learn while tinkering with the Second Life client now that it's been GPLed.

I am not a guru, developer etc... I'm just playing around as a learning experience.

I am attempting to create an extremely basic server that just allows the Client to login, hopefully to a empty null world and maybe send some very basic information such as the suns position, area name, amount of money, 1 or 2 prims, flat terrain etc... I want to create it from scratch without relying on the SL viewer code or the libsecondlife stuff to ensure i understand everything. I would also like the server to handle all the elements of the login, ie not using a web server for the login stuff.

So far the server allows the completion of step 1 of the Authentication Flow

Its not very advanced or prwdy and works by parroting the clients requests back to it, this is enough of a hack to trick the client into continuing however incorrect sequence numbers etc are sent.

Stuff used: netcat, wireshark(Ethereal), C#/Mono/Monodevelop

I'm doing this under Ubuntu but any form of Linux should be fine (theres also a port of SL on Solaris now apparently), netcat and wireshark have windows ports also. I have never actually learned C#, but i have done Java, C and C++ so with the help of Google its fairly easy to work it out as I go. I chose C# as I haven't used it much previously, its also being incorporate into SL for the scripting language and libsecondlife appears to be written in it.

Step 1.

The Authentication Flow just describes this as:

Viewer(client)--------->User Server

    * Secure Message checkum request - Protocol Level Supported? Yes/No.
    * Port 12036 

which doesn't really give us an in depth description of everything thats happening or enough to make a server that does what is required, probably the quickest way is to analyses what the client does and try to emulate the servers responses.

Redirecting the client

Firstly the client needs to be started with the correct command line parameter (found here). This tells the client to send the initial UDP packets to the localhost.

./secondlife -user localhost

Then netcat need to be set to listen on UDP port 12036:

hegemon@ender:~$ sudo nc -u -l -p 12036 
@������+U_{q�Y)$�ͫ

                        `������+U_{q�Y)$�ͫ

                                                @������+U_{q�Y)$�ͫ

                                                                        ����`������+U_{q�Y)$�ͫ

                `������+U_{q�Y)$�ͫ

                                        @������+U_{q�Y)$�ͫ

                                                                ���`������+U_{q�Y)$�ͫ

        @������+U_{q�Y)$�ͫ

                                ���`������+U_{q�Y)$�ͫ

                                                        `������+U_{q�Y)$�ͫ

                                                                               `������+U_{q�Y)$�ͫ

                        �����+U_{q�Y)$�ͫ

                                                        ���`������+U_{q�Y)$�ͫ

                                                                               �����+U_{q�Y)$�ͫ[[]]

This successfully makes the client send its response to localhost, its then not receiving anything back and disconnecting. The outputted information isn't very usefull so the next step is to fireup wireshark, set it to listen on the loopback interface and find out whats being sent (Alternatively netcat has the ability to output a hexdump also with the -o flag, its also possible to pipe the output to the 'hexdump' unix command however wireshark is probably nicer and can be used to analyze the client connecting to the actual server for comparison later).

A timeout analysis

Raw Dump

Wireshark gives us:

Client>Server
0000  00 00 00 00 00 00 00 00  00 00 00 00 08 00 45 00   ........ ......E.
0010  00 34 00 00 40 00 40 11  b7 62 c0 a8 01 03 c0 a8   .4..@.@. .b......
0020  01 03 94 ad 2f 04 00 20  83 88 40 00 00 01 ff ff   ..../..  ..@.....
0030  ff fa 2f c7 ac c1 79 ca  f6 24 2a bc 12 e6 36 48   ../...y. .$*...6H
0040  be 04                                              ..               

0000  00 00 00 00 00 00 00 00  00 00 00 00 08 00 45 00   ........ ......E.
0010  00 34 00 00 40 00 40 11  b7 62 c0 a8 01 03 c0 a8   .4..@.@. .b......
0020  01 03 94 ad 2f 04 00 20  83 88 60 00 00 01 ff ff   ..../..  ..`.....
0030  ff fa 2f c7 ac c1 79 ca  f6 24 2a bc 12 e6 36 48   ../...y. .$*...6H
0040  be 04                                              ..               

0000  00 00 00 00 00 00 00 00  00 00 00 00 08 00 45 00   ........ ......E.
0010  00 34 00 00 40 00 40 11  b7 62 c0 a8 01 03 c0 a8   .4..@.@. .b......
0020  01 03 94 ad 2f 04 00 20  83 88 40 00 00 02 ff ff   ..../..  ..@.....
0030  ff fa 2f c7 ac c1 79 ca  f6 24 2a bc 12 e6 36 48   ../...y. .$*...6H
0040  be 04                                              ..               

Description

These are the 1st 3 packets, there where around 16 in total but its the same thing over and over.

So now we begin the packet analysis. We are only interested in the Data part of the packets, the first 42 bytes contains information such as source and destination ports/addresses, protocol information etc... which is what routers use to determin where to push these things around the internet to get it to your computer. This is shown in wireshark, so just select the data which highlights the following data packets.

Packet info

40 00 00 01 ff ff ff fa 2f c7 ac c1 79 ca f6 24 2a bc 12 e6 36 48 be 04

60 00 00 01 ff ff ff fa 2f c7 ac c1 79 ca f6 24 2a bc 12 e6 36 48 be 04

40 00 00 02 ff ff ff fa 2f c7 ac c1 79 ca f6 24 2a bc 12 e6 36 48 be 04

You will notice there are only 2 bytes that are changing, the 1st and 4th, this is because there is no response so the client will keep resending the same data until there is one or it gives up.

By looking at the packet layout we can start to see what is being sent. There is also the libsecondlife analysis.

The first byte is the byte contains the bitmask flags, LL_ZERO_CODE_FLAG 0x80, LL_RELIABLE_FLAG 0x40, LL_RESENT_FLAG 0x20, LL_ACK_FLAG 0x10. To understand exactly whats happening in the first byte its best to look at it in binary. In the first packet we have 0x40, 4 in binary is 0100 and 0 is obviously 0000, which is 01000000, so the LL_RELIABLE_FLAG is set, and packet 2 has the LL_RELIABLE_FLAG and the LL_RESENT_FLAG set.

The first byte in packet 2 is 0x60, in binary 01100000, in this packet it wants an ack and it is a packet that has been resent. The 3rd packet is 0x40 again so its a new packet. not a resent one.

The next 3 bytes are marked as the sequence number, 000001, the first packet sent, then it is repeated in the resent packed and when the client decides to send a new packet rather then resend it increments it to 2. When an ack is sent back I believe it uses this sequence number for the ID, which tells the client it was received and doesn't need it to be resent.

The next bytes are listed as the 'frequency', its really means the messageID so the system knows the difference between a request for the version information and a request to close the connection, they are defined in the message template file /linden/scripts/messages/message_template.msg in the viewers source code, they are also available from libsecondlife here and you can also find the individual ones on this wiki here, there are 6 'fixed' ones shown near the top, we are encountering 0xFFFFFFFA which is the RequestSecureChecksum, we can check the template which shows that the packet only contains a LLUUID, that is the next 16 bytes and is a unique number for our client which is the remaining bytes. This is the client check to make sure the server is talking the same protocol version.

A majority of the messages are not 'fixed' ones. The number of bytes that makes the frequency is variable, so if a byte isn't 0xFF then that byte is the last byte in the frequency. Currently I'm not sure how to convert the frequency number to the specific message, or workout the checksum as it appears the client loads the template file in and docs say it assigns the numbers at runtime, but the fixed ones are enough for the first login step and we can use what we know is a valid checksum number by asking the client for its checksum.

The function to parse the message_template.msg file is in /linden/indrea/llmessage/message.cpp in the Second Life viewer source and the number just appears to be in the order they are in the file (seperate counters for how medium and low frequency messages.

The code used to generate the checksum, around line 1000:

	snprintf(formatString, sizeof(formatString), "%%%ds", MAX_MESSAGE_INTERNAL_NAME_SIZE);
	messagefilep = LLFile::fopen(filename, "r");
	if (messagefilep) {

		fseek(messagefilep, 0L, SEEK_SET );
		while(fscanf(messagefilep, formatString, token) != EOF)	{
			// skip comments
			if (token[0] == '/') {
				// skip to end of line
				while (token[0] != 10)
					fscanf(messagefilep, "%c", token);
				continue;
			}
	
			checkp = token;
			while (*checkp)	{
				mMessageFileChecksum += ((U32)*checkp++) << checksum_offset;
				checksum_offset = (checksum_offset + 8) % 32;
			}
	// Code here to actually process the entries, see source for more info

Valid Response

Lets looks at a valid response by sniffing an actual connection to the server

Raw

Client:
0000   00 e0 a0 a6 66 70 00 18 f3 04 90 4c 08 00 45 00  ....fp.....L..E.
0010   00 34 00 00 40 00 40 11 41 e0 c0 a8 01 03 42 96  .4..@.@.A.....B.
0020   f4 97 94 6e 2f 04 00 20 2f e6 40 00 00 01 ff ff  ...n/.. /.@.....
0030   ff fa 16 89 02 b0 75 38 50 f1 8f 41 fe a7 6e 9a  ......u8P..A..n.
0040   f7 98                                            ..

Server:
0000   00 18 f3 04 90 4c 00 e0 a0 a6 66 70 08 00 45 00  .....L....fp..E.
0010   00 40 72 bf 40 00 2d 11 e2 14 42 96 f4 97 c0 a8  .@r.@.-...B.....
0020   01 03 2f 04 94 6e 00 2c 14 38 00 00 00 01 ff ff  ../..n.,.8......
0030   ff ff 00 02 c4 e4 95 9d 00 00 00 00 01 0d 16 89  ................
0040   02 b0 75 38 50 f1 8f 41 fe a7 6e 9a f7 98        ..u8P..A..n...

Server:
0000   00 18 f3 04 90 4c 00 e0 a0 a6 66 70 08 00 45 00  .....L....fp..E.
0010   00 29 72 c3 40 00 2d 11 e2 27 42 96 f4 97 c0 a8  .)r.@.-..'B.....
0020   01 03 2f 04 94 6e 00 15 42 79 00 00 00 02 ff ff  ../..n..By......
0030   ff fb 01 01 00 00 00 00 00 00 00 00              ............

Client:
0000   00 e0 a0 a6 66 70 00 18 f3 04 90 4c 08 00 45 00  ....fp.....L..E.
0010   00 24 00 00 40 00 40 11 41 f0 c0 a8 01 03 42 96  .$..@.@.A.....B.
0020   f4 97 94 6e 2f 04 00 10 43 82 00 00 00 02 ff ff  ...n/...C.......
0030   ff fd                                            ..

Info

In a slightly more readible form:

/-Flags
|
|     /-Sequence Number
|     |
|     |          /-Frequency
|     |          |
+- +--+---+ +----+----+ <<<<<<<<<<<<<<<<<<<<<<<<< DATA >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
40 00 00 01 ff ff ff fa 16 89 02 b0 75 38 50 f1 8f 41 fe a7 6e 9a f7 98 <--- DATA is LLUID
00 00 00 01 ff ff ff ff 00 02 c4 e4 95 9d 00 00 00 00 01 0d 16 89 02 b0 75 38 50 f1 8f 41 fe a7 6e 9a f7 98
00 00 00 01 ff ff ff fb 01 01 00 00 00 00 00 00 00 00  \--\  \-------------------------------------------/ 
00 00 00 02 ff ff ff fd  \  \--------\  \--\--\--\--\      \                                     LLUUID-/
                          \      \             \            \-Version info at this end?
                           \      \-ACK_ID      \
                            \       (reversed)   \-5 Trailing Zeros, not actually part of data
                             \
                              \-Number of ACKs in message

Description

In english please?:

[Client>Server] 0xFFFFFFFA - Please ACK this, Sequence #01, SecureTemplateChecksumRequest, LLUUID is [16-89-02-b0-75-38-50-f1-8f-41-fe-a7-6e-9a-f7-98]
Here out client is requesting a checksum of the template file so it can check its version matches.

[Server>Client] 0xFFFFFFFF No special flags, Sequence #01, TemplateChecksumReply, <This is the checksum and version info>, LLUUID is [16-89-02-b0-75-38-50-f1-8f-41-fe-a7-6e-9a-f7-98]
This is the servers response, checking the correct template shows that it has 32bits(4bytes) of checksum info, then 8bits (1byte) for each for major, minor, patch and server version information and then another 4 bytes for flags, it then has a LUUID.

The data bytes aren't in the same order as the template, and each one is reversed, the major version is 0x01 and the minor version id 0x0D (13). The checksum for this template version is 0x9D 0x95 0xE4 0xC4. I would gess that the flags are the 4 0x00 and the other patch information is the 02 and 00 but its speculative at this point.

[Server>Client] 0xFFFFFFFB - NoFlags, I am acknowledging that i received 1 packet from you, packet number #1
One of the 01 bytes is the number of packets that are being acknowledged, its possible to acknowledge more than more packet in one ACK reply. The other is the ID of the acknowledged packet. Looking at the PacketAck template you will notice there is just one unsigned 32 bit, no counting byte is listed, I believe that is because of the word 'variable', each ACK is assigned 4 bytes but the 01 appears in the first byte so it would seem to be in reverse order (might be relevant for decoding the previous entry). The 4 zeros bytes at the end Wireshark lists as trailing bytes so they are probably padding required by the network protocols.

The server replied to the packet befoure it ACK it, its also possible to embed the ACK into other packets by setting the ACK flag and adding the number of ACKs to the end of the packet and then the sequence numbers.

[Client>Server] 0xFFFFFFFD - NoFlags, CloseCircuit
Having received the version info, the client closes this connection, it will now attempt to connect to the login server which is a TSL encrypted webserver, with XML-RPC calls. No extra data is needed.

Very simple C# server

// This just listens on the port and sends back the packets to the client. It done some basic analysis of the packets
// Its very horrible code

using System;
using System.Net;
using System.Net.Sockets;

namespace SLLoginTest
{
	public class Slog
	{
		
		public Slog()
		{
				int recv;
				byte[] data = new byte[1024];
				
				// Create the listening server socket...

				Socket socServer = new Socket(AddressFamily.InterNetwork,SocketType.Dgram,ProtocolType.Udp);		

				IPEndPoint ipServer = new IPEndPoint(IPAddress.Any, 12036);
				

				// Bind to local Address...

				socServer.Bind(ipServer);
				
				IPEndPoint ipClient = new IPEndPoint(IPAddress.Any, 0);
				EndPoint Remote = (EndPoint)(ipClient);
							
				// FIXME: These should be defines or static finals or whatever C# has
				byte MASK_LL_ZERO_CODE_FLAG = 0x80;
				byte MASK_LL_RELIABLE_FLAG = 0x40;
				byte MASK_LL_RESENT_FLAG = 0x20;
				byte MASK_LL_ACK_FLAG = 0x10;
				
				
				while(true) {
					data = new byte[1024];
					Console.WriteLine("Awaiting packets...");
					recv = socServer.ReceiveFrom(data, ref Remote);
					Console.WriteLine("{0} bytes recived...", recv);
					
					
                                        int seek = 0;
					int i = 0;

                                        // Dump the raw data
					Console.Write("DATA:");
					for(int j = 0; j<recv; j++) {
						Console.Write(" 0x{0:X2} ", data[j]);
					}
					Console.WriteLine();

                                        // This bit checks the packet flags
					bool LL_ZERO_CODE_FLAG, LL_RELIABLE_FLAG, LL_RESENT_FLAG, LL_ACK_FLAG;
					
					if((byte)(data[0] & MASK_LL_ZERO_CODE_FLAG) != 0) {
						LL_ZERO_CODE_FLAG = true;
						Console.WriteLine("Zero encoded");
					}
					if((byte)(data[0] & MASK_LL_RELIABLE_FLAG) != 0) {
						LL_RELIABLE_FLAG = true;
						Console.WriteLine("Reliable");
					}
					if((byte)(data[0] & MASK_LL_RESENT_FLAG) != 0) {
						LL_RESENT_FLAG = true;
						Console.WriteLine("Resent");
					}
					if((byte)(data[0] & MASK_LL_ACK_FLAG) != 0) {
						LL_ACK_FLAG = true;
						Console.WriteLine("ACK");
					}
					seek++; // Move to the next byte

                                        // Read the sequence number into a 3 byte array
					byte[] sequence = new byte[3];
					
					sequence[0] = data[seek++];
					sequence[1] = data[seek++];
					sequence[2] = data[seek++];

					Console.WriteLine("Sequence: 0x{0:X2}{1:X2}{2:X2}", sequence[0], sequence[1], sequence[2]);
					
					byte[] frequency = new byte[4];
					byte freqByte = data[1];
					
					// Add each 0xFF byte  until we reach something thats not 0xFF
                                        // FIXME: Should stop at 4 bytes
					do {
						frequency[i] = data[seek];
						i++;
						seek++;
					} while(data[seek] == 0xFF);

					// We add the final frequency byte, unless it was already added in the case of 0xFFFFFFFF
					if(i < 4) {
						frequency[i++] = data[seek++];
					}

				// Print the frequency
        			Console.Write("Frequency: 0x");
        			for(int j = 0; j < i; j++) {
        				Console.Write("{0:X2}", frequency[j]);
        			}
        			Console.WriteLine();
        			
				// These are the basic templaces (Theres 2 more not listed here)
        			byte[] SecureTemplateChecksumRequest = {0xFF, 0xFF, 0xFF, 0xFA};
        			byte[] TemplateChecksumReply = {0xFF, 0xFF, 0xFF, 0xFF};
        			byte[] Ack = {0xFF, 0xFF, 0xFF, 0xFB};
        			byte[] CloseCircuit = {0xFF, 0xFF, 0xFF, 0xFD};
        			
        			Console.Write("ST: 0x");
        			for(int j = 0; j < 4; j++) {
        				Console.Write("{0:X2}", SecureTemplateChecksumRequest[j]);
        			}
        			Console.WriteLine();

				// Check to see which message was send
				// This isn't a good way to handle it, it assumes a 4byte frequency and may crash
				// I haven't bothered to check them all either
        			if(frequency[3] == SecureTemplateChecksumRequest[3]) {
        				Console.WriteLine("SecureTemplateChecksumRequest");
        			} else if(frequency[3] == CloseCircuit[3]) {
        				Console.WriteLine("CloseCircuit");
        			}
        			
        			// By sending the packet back to the client we can fake the start of the login
        			//
        			// 1. V>S SecureTemplateChecksumRequest? - Viewer asks for the servers checksum
        			// 2. S>V SecureTemplateChecksumRequest? - Server asks for the viewers checksum
        			// 3. V>S TemplaceChecksumReply - Client replys to request 2
        			// 4. S>V TemplaceChecksumReply - Sending the clients on checksum back, answers request in 1
        			// 5. V>S CloseCircuit - The client has checked it matches the checksum and closes
        			// 6. S>V CloseCircuit - Parroted back (probably useless/ignored by client)
        			Console.WriteLine("Sending parrot response.");
				socServer.SendTo(data, recv, SocketFlags.None, Remote);
				}
		}
	}
}

Step 2

Now that the client passes step 1, it will now attempt to access the login page using XML-RPC calls.

We can override the website with the -loginuri command. Normally the SL client will want to use encrypted TSL 1.0 but we can specify a nonencrypted url.

./secondlife -user ender -loginuri http://ender/

Then netcat can listen on port 80 and we see the following

POST / HTTP/1.1
Host: localhost
Pragma: no-cache
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: text/xml
Content-Length: 1825

<?xml version="1.0"?><methodCall><methodName>login_to_simulator</methodName><params><param><value><struct><member><name>first</name><value><string>Hegemon</string></value></member><member><name>last</name><value><string>Skall</string></value></member><member><name>passwd</name><value><string>$1$4534d5a3257d801ba927ce39bca33aab</string></value></member><member><name>start</name><value><string>last</string></value></member><member><name>major</name><value><string>1</string></value></member><member><name>minor</name><value><string>13</string></value></member><member><name>patch</name><value><string>2</string></value></member><member><name>build</name><value><string>57209</string></value></member><member><name>platform</name><value><string>Win</string></value></member><member><name>mac</name><value><string>b5fcfb83e74517f2a8be6092995c51c7</string></value></member><member><name>id0</name><value><string/></value></member><member><name>viewer_digest</name><value><string>2bd23799-b279-8665-500a-ce27025b315e</string></value></member><member><name>options</name><value><array><data><value><string>inventory-root</string></value><value><string>inventory-skeleton</string></value><value><string>inventory-lib-root</string></value><value><string>inventory-lib-owner</string></value><value><string>inventory-skel-lib</string></value><value><string>initial-outfit</string></value><value><string>gestures</string></value><value><string>event_categories</string></value><value><string>event_notifications</string></value><value><string>classified_categories</string></value><value><string>buddy-list</string></value><value><string>ui-config</string></value><value><string>login-flags</string></value><value><string>global-textures</string></value></data></array></value></member></struct></value></param></params></methodCall>

No thats not my real encrypted password :)

The libsecondlife login page has more information about the valid responses, pasting one of those into netcat should yield some results, looks like you could use it to make custom messages pop up :)

The next step is to take the request the client produced and attempt to send it to the login server and capture the reply from the server. The login servers use SSL encryption so we need to tunnel through the encryption, fortunately there is a tool similar to netcat called stunnel that supports SSL.

stunnel -c -r login.agni.lindenlab.com:443 > loginreply.xml

You can then paste the clients xml request into stunnel and recieve the reply from the server, by default the server sends gziped documents that aren't very human readable, by removing the ", gzip" from the Accept-Encoding in the heading we can get the server to send us plain text back.

Note: When manually changing the xml files, its important to keep the Content-Length field in the header valid, otherwise it will be rejected and rerequested.

The servers reply contains things like the inventory layout, gestures and some character details (check the libsecondlife wiki). This response also contains the sim_ip address, by changing this and sending it to the client, we can redirect the client again to connect to netcat.

There is also a new field containing the url of the capabilities (its not shown on the libsecondlife page), once again we can redirect it and receive the xml requests.

<member><name>seed_capability</name><value><string>https://sim2049.agni.lindenlab.com:12043/cap/fa3c0b48-2db5-c73a-c8c8-9cf68be4edb2</string></value></member>

Client request to sim_ip/sim_port (it was port 13005 in this case) UDP:

[FL]  [Seq Num] [Message Num]  [------------------------------Data, 2 LLUIDS and a 4 byte 'code' = 36 bytes------------------------------]
 40   00 00 01   ff ff 00 03   ad 65 5e b6 5f f6 4b 67 af ee 6f da f1 f8 f1 65 38 ba 9a a4 7b 41 42 6e 90 c5 f4 76 3c 51 e8 37 e4 2a 1e 03                            

Here we encounter the first non-fixed message number FFFF0003. FFFF indicates that it is a low frequency message as seen in the packet layout page. We can then find out this message by getting the 3rd Low frequency message in the message_templates.msg file, in this case it seems to be UseCircitCode.

The client also sends a request to the capabilities server of the simulator obtained for the XML-RPC response:

POST /cap/cee88818-d777-ced4-af57-a3aa39757f4d HTTP/1.1
Host: localhost:12043
Pragma: no-cache
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/xml
Content-Length: 185

<llsd><array><string>MapLayer</string><string>MapLayerGod</string><string>NewAgentInventory</string><string>EventQueueGet</string><string>RequestTextureDownload</string></array></llsd>

To be continued

Find an appropriate response to the XML.

Fix the server so it actually responds rather that parroting the packets it gets.

Figure out how to incorporate the messages_template.msg file How to get the checksum/non fixed messageIDs

Step 3

This is a server>server bit, it basically checks the password is correct, initially it won't have any security and will just auto accept the client or lookup a hardcoded value. If i get things more advanced maybe a SQL database lookup.

Another idea would be to use OpenID, the firstname would be the id and the last would be the OpenID server (the password could be anything). OpenID would be a good choice for 3rd party SL servers as it wouldn't be to hard to transfer the player from one server to another when the servers don't trust each other.

Step 4,5

Another Server>Server, This is intended to find which sim in the grid to use and autorize, since this is just a basic self contained server it will be itself and this can just be skipped. It will need to tell the client that though.

Step 6

Tell the client the sim info and other basic information.

Step 7,8

Client sends server information, if possible skip most of it.