User:Pixel Gausman/Interop Viewer

Open Grid Protocol implementation in Snowglobe

This page is only for describing the OGP implementation of the viewer, not general OGP protocol description.

Open Grid Protocol (OGP) currently includes a style of login and teleport that separates control of the agent into a separate domain named agent domain (also called agentd or agent service). This separate service handles moving an avatar from region to region, including between regions that are run by different entities. This is helpful to end users because they can maintain a consistent appearance and inventory as they move from region to region. They can also trust the agent service to maintain their inventory without having to "hand over the keys" to their inventory to every region they visit.

It is a protocol under development, and that's why we need it in Snowglobe. I've developed a patch for Snowglobe to enable OGP. Protocols are best standardized with the experience of implementation, and I believe OGP needs a publicly available implementation to experiment with. An open implementation will help ground the design of protocols in the reality of working code that the community can work with.

Note: In the near future, IBM will open source an OGP implementation for the community to experiment with.

OGP protocols are not supported on the main Linden grid or the Linden test grid, which for now is a good thing. The protocol needs further development before Linden or anyone else should use it on a production level grid. OGP is new and distinct protocol from the current SL UDP and HTTP protocols, so it is the choice of the deployer to expose these interfaces or not. The specific HTTP caps for keeping inventory at the agent domain, manipulating the agent domain inventory, and resolving remote assets do not exist on the Linden Lab main grid or test grid, so no assets/inventory will cross the boundary between Second Life and OpenSim by using this viewer unless Linden enables it in their deployment.OpenSim grids that choose to expose the interfaces can experiment with distributed assets and with an inventory that is maintained by the agent domain.

Note: If appropriate, I intend to replace OGP references in the code and this wiki page to match the IETF working group name.

General information about OGP is at https://wiki.secondlife.com/wiki/User:Infinity_Linden/IETF_Drafts_and_Meetings until the IETF working group is officially formed. General information about IBM's open source OGP implementation is at TBD.

OGP History

OGP was developed by LL with input from the Architecture Working Group (AWG). OGP's initial proof of concept implementation was a joint project between LL and IBM. IBM did a portion of that OGP viewer code with some help from Linden developers. The viewer code for that initial Proof of Concept is currently located in the OGP9 public svn branch. I ported the OGP9 branch to SnowGlobe, gave the code some cleanup, and added some of the additional inventory caps we have been experimenting with. The additional caps are not supported in Linden's agent domain implementation, so no assets/inventory will cross the boundary between Second Life and OpenSim by using this viewer unless Linden enables it.

When looking at the patch, gentle readers, please remember that it was originally proof of concept level code.

Implementation

OGP viewer code affects small portions of the UI, added some command line parameters, modified the login and teleport sequences, and implements some additional agent domain caps involving inventory and assets.

UI

If -ogp is passed on the command line, the panel_login layout is modified. Last/Home/Region has no meaning in OGP, so that space in the panel is instead used for the regionuri_edit combo box. The user enters the region uri in this combo box. "Start Location:" is used in both instances,because from an end user point of view, a region uri is a starting location. The combo box is populated using URIs that are saved in a LLURLHistory named "region_uri", but only if the user's First/Last name is saved. This is because if I don't have a First/Last when building the login panel, it's impossible to pull/save *user* specific data (like the specific LLURLHistory for the user Pixel Gausman). The selected entry for the combo box is set to the command line parameter --regionuri ( gSavedSettings.getString("CmdLineRegionURI")), which is an additional way to specify the desired initial region.

Also enabled/disabled by the -ogp command line switch is the "Teleport Region..." selection in the World menu. This menu selection pops a floater that allows teleport via OGP, it also uses the LLURLHistory to store region uris. The floater can also be popped via Ctrl-Alt-Shift-L.

Command line parameters

-ogp: enables OGP mode. Viewer is either OGP or non OGP for the life of the session
--regionuri <uri>: allows you to pass in the name of the region you wish to login to.
--agenturi <uri>: allows you to specify the name of the agent domain you wish to use. This is passed to the authenticate call. If you do not specify one, it is still possible to request and get a seed cap on an agent domain from authenticate.
--loginuri <uri>: While not a new flag, it is needed when authenticating against any agent domain. Use it to point to whatever authentication authority your agent domain accepts.
--set PurgeCacheOnNextStartup TRUE: Also not a new flag, but simplified testing in the OGP viewer. Not needed in XLM-RPC Legacy mode. On the TODO list is to test with the viewer's caching in OGP mode.

Example: If you are part of the OGP beta from Summer 08 (yay Gridnauts!), this would be the command line parameter to authenticate against vaak, where <regionuri> comes from http://wiki.secondlife.com/wiki/Open_Grid_Public_Beta/Public_Regions#Linden_Lab_OGP_Regions :

-ogp --set PurgeCacheOnNextStartup TRUE  --regionuri  <regionuri>  --loginuri https://login1.aditi.lindenlab.com/cgi-bin/auth.cgi

Login sequence

The login sequence for SnowGlobe is messier than what exists in OGP9 SVN. This is because OGP9 did not support XML-RPC style logins, and the removal of the XML-RPC path in OGP9 made the code simpler. We will need to maintain the XML-RPC path in SnowGlobe, so the patch attempts to do that with minimal disturbance (and therefore risk) to the XML-RPC code path.

The login sequence is kicked off in idle_startup() in llstartup.cpp. The code loops continuously in idle_startup() as it moves through the different stages of the login sequence. OGP and XML-RPC diverge states for AUTHENTICATE and XMLRPC_LEGACY, and then share the code for STATE_LOGIN_PROCESS_RESPONSE. STATE_LOGIN_PROCESS_RESPONSE is exceptionally long, there are now separate XLMRPC and OGP blocks in that state that cope with all the responses gathered through the stages of the login process. Note that the login for OGP daisy chains together several HTTP POSTs, kicking off the next one in the responder of the previous one. Some of the HTTP POSTs must be successful (for instance, rez_avatar/place must succeed to move to STATE_LOGIN_PROCESS_RESPONSE), but some need to be optional, and still proceed with the login process if they fail (for instance, fetching agent/inventory needs to be optional)

After the overly complex STATE_LOGIN_PROCESS_RESPONSE section of the code, OGP and XMLRPC processing are essentially the same in STATE_WORLD_INIT and onward. The viewer still does it's handshake with the region, and goes through it's AgentMovementComplete processing. In our initial POC coding last year, we had experimented with getting rid of AgentMovementComplete messages, but were unsuccessful.

The login sequence is intended to work for agent domains that do not expose inventory and asset HTTP caps (like the one on vaak). It has been tested with vaak for login and teleport, despite vaak being slightly tempermental. Members if Gridnauts group on Aditi should be able to test with vaak, but vaak has intermittent issues with login.

On logout, instead of sending a UDP message to the region, in OGP a rez_avatar/place request is sent with a null URI. The null URI signals to the agent domain that it should derez the agent from it's current region.

One difference between this viewer and the OGP9 branch is that this viewer does not include LLSD "legacy" login, only XMLRPC legacy and OGP.

Teleport sequence

The user invokes OGP teleport via Ctrl-Alt-Shift-L or 'Teleport Region...' in the World menu. This pops up an instance of LLFoaterTeleport, which has a combo box of the same region_uri LLURLHistory. LLFloaterTeleport's function in life is to make a rez_avatar/place request on the agent domain using the URI from the floater. The reponse is caught by LLPlaceAvatarTeleportResponder, and relevant values are plucked from its contents. Note that some of the values in the response to rez_avatar/place are *only* passed on login, and not passed in teleport.

After the response to rez_avatar/place is received from the agentd, the code for Teleport is very similar to XML-RPC Teleport.

Assets and Inventory

Inventory can now be managed by the agent domain, and fetched via HTTP using agent/inventory cap. Temporarily Inventory manipulation (CreateInventoryItem, etc.) still goes through UDP. The UDP calls in OpenSim are reflected to the agent domain via an HTTP interface for handling. This was done as a temporary measure until proper HTTP Inventory manipulation calls could be designed properly and done from the viewer.

Fetching of the inventory is done via the existing WebFetchInventoryDescendents HTTP cap code in the viewer, with the cap renamed to be agent/inventory for the agentd case. For the time being, the format of agent/inventory is the same as WebFetchInventoryDescendents. In cases where there is both a agent/inventory cap on the agent domain and a WebFetchInventoryDescendents cap on the region, the cap on the agent domain is used.

There is a new cap in agentd called agent/inventory-skeleton. It's responder, LLInventorySkeletonResponder, grabs the skeleton, and also makes a new (dangerous?) assumption. It assumes that content["Skeleton"][0]["folder_id"] is the inventory-root UUID. It makes sense, but is a change to how things worked in legacy mode. In legacy XML-RPC authentication, the inventory-root UUID is passed back from authenticate, separate from the skeleton. Just like agent/inventory, the values passed back for agent/inventory-skeleton mimic the values received in an XML-RPC login.

The interfaces of agent/inventory and agent/inventory-skeleton will change as the design of agent domain moves forward. Also, assets will eventually be distributed, meaning that a region on a grid might contain an asset that is not stored on that grid. An agent's inventory might be made up of items scattered across the web. This means some interesting protocol changes, while keeping an eye on performance.

Shortcomings, future needs, and dirty laundry

Refactor Teleport: The teleport code needs to be refactored into a new class when OGP support is added for:

LLAgent::telportViaLure - This function is invoked when one agent offers one or more other agents to teleport to her location. LLAgent::teleportViaLure() is called when the current agent has accepted a lure and desires to TP to a friend. Currently all the protocol gives me is a lure_id. I'd really like to get a region_uri, so a protocol change would be needed.
LLAgent::teleportViaLandmark - Similar to teleportViaLure issue. I have a landmark_asset_id, and need a URI.
LLAgent::teleportVialocation - This is invoked from about a dozen different places in the viewer (for instance, when the user clicks on the map). A global position is passed in, which is then used to compute the regionhandle. With the regionhandle, the legacy teleportRequest() is called. The assumptions around the use of regionhandle need to be reworked in the viewer (and in the OGP protocol).

regionhandles: The viewer assumes that all regions it can get to are arranged in one contiguous XY grid. It's not its fault, when the viewer was written, there *was* only one grid. A region's regionhandle denotes its XY location in the one known grid, i.e. where it shows up on the map. When teleporting between two distinct grids, it is very possible that two regions report the same regionhandle. This makes the viewer very unhappy. The viewer uses the regionhandle to check that an XY location is a valid region that the viewer has a connection to, and to do location based things like determine neighboring regions. It's even checked when determining if you are flying. When doing In the OGP beta, we worked around this issue by having all the OGP region owners register their regionhandles on a website. That allowed us to avoid the issue. Longer term, the issue needs to be resolved differently.

STATE_LOGIN_PROCESS_RESPONSE: This section of idle_startup() is rather brittle, and will be easy to break. Consider rewriting it, or separating XML-RPC and OGP into two different states instead of both of them using STATE_LOGIN_PROCESS_RESPONSE. I think the original OGP9 code usedSTATE_LOGIN_PROCESS_RESPONSE as a common state between legacy and OGP because OGP9 only dealt with LLSD login (and had chopped all the XML-RPC code.

makeNewOutfit() crashes when in OGP mode.: In the original Proof of Concept, we chose to disable the UI button for Make New Outfit. It has been re-enabled, and needs to be fixed for both agent domains that support inventory, and ones that do not.

PurgeCacheOnNextStartup flag: This flag is only needed in OGP mode, we need to test the viewer without this flag and make sure viewer's internal caching mechanisms work with OGP. Careful definition of the testing scenarios need to be defined, and it also needs to work with agent domains that do not support Inventory.

LLPlaceAvatarTeleportResponder: Rework how values are caught from the rez_avatar/place responder, and put into mResult. Currently I think we might wipe out some mResult values on teleport.

Disclaimer

Anything resembling an opinion is my own, and not that of my employer, IBM. This is experimental code, meant to give the community a relevant viewer to use in interoperability work.