OGP Base Draft 3

Open Grid Protocol: Foundation

Draft 3

September 2008

Notice: This draft is for public comment.

Mark Lentczner (Zero Linden) Linden Lab zero.linden@secondlife.com

Copyright 2008, Linden Lab. All rights reserved.

This work is licensed under the Creative Commons Attribution-Share Alike 3.0 license. See http://creativecommons.org/licenses/by-sa/3.0/for details.

All contributions to this document must be contributed under the Second Life Project Contribution Agreement. See http://wiki.secondlife.com/wiki/Project:Contribution_Agreementfor details

Abstract

The Second Life Open Grid Protocol documents define the protocols by which a vast, Internet wide virtual world can operate. This protocol enables different regions of the virtual world to be operated independently, yet interoperate to form a cohesive experience.

Status

As of Summer 2008, this protocol is a work in progress. While the major structural elements have been designed, the specific elements for many needed resources have not. This work is being undertaken by Linden Lab and its community based Architecture Working Group.

This protocol is being defined in light of feature set of Second Life. For now, where this document is currently lacking details, familiarity with Second Life will be assumed. It is an express aim of this work to enable the live, gradual migration of Second Life to using this new protocol as the development proceeds.

discuss

Structure

discuss

Domains

This protocol is about a three way interaction between viewer, agent and region in order to facilitate a shared experience between people.

The viewer is the element that senses and acts on the state of the virtual world. The viewer does so from the vantage point of an agent. An agent is persistent identity and persona that interacts in a virtual world. The agent persists and can be interacted with even when the user controlling it (though a viewer) is off-line. Regions are persistent locations in the virtual world. Multiple agents may be present in a region at the same time, and when they are they have a shared experience.

Groups of regions and agents are managed by domains. A region domain is responsible for a collection of regions. An agent domain manages agent accounts.

This protocol makes few assumptions about how a domain manages its collection of elements. In particular, it does not assume that a region will be entirely managed on a single host, nor that an agent will or won’t be managed by a single process.

It is useful to think of the “stance” that each element takes in the three-way protocol:

The viewer is the direct proxy for a human that wants to control an agent. This control can be direct as in the case of an interactive 3D viewer, or indirect as in the case of a web site that the user directs to display their agent’s status.

The agent domain is responsible for the agent itself. The persistent state of the agent is held within the agent domain, and requests to interact with the agent, even by the viewer, are mediated by the agent domain.

The region domain runs the live simulations of regions in the virtual world. The region domain manages the persistent state of these regions.

discuss

Basic Flow

The basic flow of the protocol is:

1. The viewer authenticates to an agent domain for the authorized control of a particular agent.
2. The viewer directs the agent domain to to place the agent in a region.
3. The agent domain contacts the region domain for the region, and negotiates placement of the agent.
4. The region grants access to the agent domain, which in turn passes some of that granted access on to the viewer.

At this stage, each entity will have access to many resources in the other entities. For example:

• The viewer has access to region resources that let it move the avatar.
• The region has access to viewer resources that update the state of objects in the region.
• The viewer and agent have access to resources in each other to facilitate text messaging.

discuss

Structure of the Protocol

The protocol is fundamentally composed of individual resources that can be invoked by one entity in the system upon another. Each resource is a member of a resource class that describes the syntax and semantics of invoking the resource. The bulk of this document, when complete, will describe the several hundred resource classes that make up the virtual world.

The resource classes are composed into suites that form logical groupings, though suites do not otherwise play a part in the protocol.

In order to facilitate migration from the currently running version of Second Life, legacy resources return information that allow entities to continue to communicate using the existing protocols and structures. These protocols and structures are not described by this document. It is the intention that when this work is complete, none of these legacy resources will be in use.

Agent and region domains have a few resources that are available at well known URLs. All other resources in the agent and region domains are accessed via capabilities obtained from the those few initial resources.

Since viewers are typically behind firewalls that do not allow connection, resources in the viewer are accessed by event queues held in the agent and region for the viewer. The viewer uses the “long poll” technique to efficiently proxy these inward resource invocations.

discuss

Document Structure

OGP is a large suite of interrelated protocols. Each major protocol set is described in its own document. For examples, see the OGP Authentication and OGP Teleport documents. This document describes the base facilities and concepts upon which the other protocols are based. To be compliant with OGP, an implementation must conform to this document, and may implement any of the other protocol sets that are deemed relevant.

discuss

Base Protocols

discuss

Resources, HTTP & REST

All interaction between entities is through a client invoking a resource. Resources are invoked either directly via HTTP, or through an event queue.

For each resource class, this protocol defines how the client obtains the URL, the HTTP verb (or verbs) to be used, the request and response bodies (if any), and significant status codes. Resource classes are designed with REST style semantics.

In general, HTTP & REST are used as follows: The URL will be either well-known in advance or returned in a response from another resource. The latter is called a capability. Except for security reasons, URLs are always treated as opaque. Clients should not modify them. Parameters are never added to them via the query section. Resource handlers must be prepared to ignore query sections.

Resources follow general REST semantics and so respond to one of these verb sets:

GET	for cacheable resources
GET, PUT	for cacheable resources that can be modified
GET, PUT, DELETE	for cacheable resources that can be modified and deleted
POST	for non-cacheable resources

Unless otherwise stated, if a resource accepts PUT, it accepts multiple PUT invocations.

Note: We are considering having all resources support OPTIONS

The request and response bodies are transmitted as serialized LLSD data. If a resource has no response defined, then it can return either an undefined value, an empty map, or have a zero length response body.

HTTP status codes should only be used to indicate the status of the HTTP interaction itself. In general, if the resource is reachable, and the request understood, a 2xx code should be returned.

Note: Something about redirection - is it supported? Probably not…

HTTP headers, both for the request and the response are never part of a resource class definition. Headers are handled as per the HTTP standard.

discuss

LLSD

All data in this system is defined by LLSD. LLSD is an abstract way of talking about structured data. It is defined in the document "LLSD".

Note: This needs to be a normative reference.

Serialization

When used as part of OGP, the XML and JSON serializations of LLSD >>> rfc2119 must be supported.

Note: As deployed, only XML serialization is implemented.

discuss

Capabilities

This protocol makes extensive use of capabilities. A capability is an opaque HTTP (or HTTPS) URL used for accessing a particular resource. The provider of the resource has three logical parts: the grantor , the capability host , and the service .

The grantor uses the capability host to construct a capability that maps to the service that provides the resource, then returns that capability to the client. At some point in the future, the client invokes the capability which makes a connection to the capability host. The capability host then proxies to the service to provide the resource.

The parts that make up the provider may be separate entities or may be the same.

The client can’t invoke the resource without the capability. Typically the capability is a URL with a cryptographically secure path component. Within the capability host, this URL is mapped to the actual internal resource URL.

The client is free to hand the capability to other entities who become clients of the capability as well. Other than for the security considerations below, the client must not rely on any assumed structure of the capability URL.

Obtaining

For each resource a client wants to invoke, the capability must be obtained. In a few cases, the capability will have been expressly returned in the result of some other resource. Usually, the system uses a seed capability (see below) to request the capability for a given resource by name.

Invocation

To invoke a capability, the holder performs an HTTP transaction with the capability as the URL. The resource class the capability represents will dictate which verb (or verbs) can be used, and what the request and response bodies (if any) should be.

Lifetime & Revocation

Capabilities can be either unlimited or one-shot. Unlimited capabilities can be used multiple times, whereas one-shot can be used only once and are automatically revoked on invocation.

Note: If we support the OPTIONS method, then invoking a one-shot with OPTIONS does not revoke it.

Note: What about using HEAD on a one-shot?

Any capability can be revoked at will by the provider of the resource. Clients must be prepared to handle revoked capabilities. A revoked capability, when invoked must return a 4xx HTTP status code. The capability host may return a 404, even if the capability had been previously active.

Names

The resource a capability performs is identified by name. When requesting a capability, or when returning a capability, the opaque URL is identified with this name. The names of such resources are intended to be globally unique.

Names are URIs. When a name appears without a scheme component, then it is a relative URL, considered relative to the base:

http://xmlns.secondlife.com/capability/name/

While names do exhibit path-link structure, they are to be considered opaque identifiers. For example, while the following three capability names are indeed from the same sub-system, nothing should be inferred about a capability that starts with their common prefix:

• inventory/root
• inventory/folder_contents
• inventory/move_folder

While not required, this protocol prefers names that are all lower case roman letters, separated by underscores.

Seed Capability (Resource Class)

In many cases, a sub-system will return a seed capability from which other capabilities can be requested.

Seed Cabability
Name	various, this is a generic resource and used in a variety of places
URL	various
Verb	POST
Request	{ capabilities: [ name1, name2, … ] }
	an array of the names of the capabilities being requested
Response	{ capabilities: { name1: url1, name2: url2, … } }
	a map from the names requested to granted capabilities.

The request contains an array of all resource names for which capabilities are desired. The response contains a map with an entry for each capability granted. Note: a grantor may grant all, some or none of the requested capabilities. The grantor may also grant additional capabilities that were requested, or none at all. If the grantor grants none, the response array must be empty and the HTTP status code should still be 200.

Note: The seed capability protocol deployed today uses the 'caps' key instead of 'capabilities'

Security

If an end-point receives a capability from an untrusted source, it is permissible for security reasons to check the following aspects of the URL before use:

• The scheme should be http: or https:.
• The authority (in particular, the resolved host name) should not resolve to ports on the local machine that aren‚Äôt publicly accessible.

discuss

Event Queues

An event queue enables an entity to invoke resources in the viewer, which cannot be directly contact via HTTP. This is usually the case because the viewer is behind a firewall that doesn’t allow incoming TCP (and hence HTTP) connections from the region or agent domains.

In such a situation, the client establishes a queue of invocation requests for resources in the viewer. At the same time, the viewer uses an event_queue/get capability to effectively tunnel the requests from the client to itself.

Note: The event queue protocol described here matches what is deployed today, but is limited in functionality. It is expected to be superseded by a more general facility soon.

Note: The deployed event queue on the agent domain

Restrictions

Resources accessed this way have the following restrictions:

• Resources are identified by their resource class name. With capabilities, there can be several resources in an entity that all conform to the same resource class. With event queues only one resource can exist for each resource class within the viewer. This is not usually a severe restriction.

Note: The next three are temporary limitations in the current protocol and are expected to be removed. Resource classes that conform to these restrictions are equivalent to messages in the current Second Life protocol.

• The only verb allowed is POST.
• No response body is allowed.
• The only status codes are 200, or 500 if the queue shuts down before all events are ack‚Äôd.

Requests

The event queue is an unordered list of requests. Each request is formatted as follows:

{ message: name , body: arbitrary-data }

Basic Flow

When the viewer invokes event_queue/get , the entity replies with the list of messages that have been queued up. The viewer takes the response, breaks it apart into a series of requests that it processes on itself, as resource invocations that the entity wanted to perform. When those invocations are processed, the viewer indicates in its next invocation of event_queue/get that the previous set was completed. While it takes two resource invocations of event_queue/get to tunnel a set of invocations in the other directions, subsequent transactions are chained, since the acknowledgement of a previous set of requests is performed in the same invocation that gets the next set.

Acknowledgement

The response to event_queue/get includes a sequence number for the batch of requests. When the viewer has processed a batch of requests, acknowledges them in the next invocation of event_queue/get . If unacknowledged, the event queue in the entity may either resend the same requests batched with any new requests, or may treat them as lost.

Long Poll

Both viewers and entities must be prepared to handle use the “long poll” technique to keep the flow of requests timely. Viewers must be prepared to handle that invoking event_queue/get may take a relatively long time to return, as the entity may choose to delay responding if there are no requests pending, or if it believes it would be better to wait for more requests to queue. Entities must be prepared to handle viewers that request as soon as they are ready for events with no delay. Both sides must be prepared to handle time outs and retries.

Closing the Queue

When the viewer is ready to terminate the queue, meaning that it wishes to be done accepting requests, it may signal such by including the done flag in the next invocation of event_queue/get . This value is purely advisory, but enables entities to cleanly flush remaining events, and release resources.

Event Queue Get (Resource Class)

This resource is a capability both in the agent and in the region, for implementing a tunneled series of resource invocations from the entity back to the client:

Event Queue Get
Name	event_queue/get
URL	capability
Verb	POST
Request	{ ack: sequence-number , done: done-flag }
	On the first invocation for a given resource, sequence-number must be undef. If done-flag is true, then the client is signaling its intention to stop polling if there are no more events.
Response	{ id: sequence-number , events: [ requests , … ] }
	See above for format of requests. Events may be empty.See above for format of requests. Events may be empty.