Difference between revisions of "LSL HTTP server"

From Second Life Wiki
Jump to navigation Jump to search
m (Previous text "Testing this have it has been found that..." made ZERO sense.)
 
(88 intermediate revisions by 35 users not shown)
Line 1: Line 1:
== Goals ==
{{LSL Header|ml=*}}
Create an alternative to the XMLRPC server and email gateway for communication with LSL scripts initiated from outside Second Life that is easy to use and scalable.  Extra bonus for enabling LSL -> LSL communication at the same time.


== Design ==
= Introduction =
=== URLs / Namespace ===
public url: ''https://sim123.agni/cap/f23b4b94-012d-44f2-bd0c-16c328321221''
request: ''curl https://sim123.agni/cap/f23b4b94-012d-44f2-bd0c-16c328321221/{untrusted-path}''


=== LSL ===
This is the counterpart to [[llHTTPRequest]]. While [[llHTTPRequest]] enables [[Script|LSL scripts]] to request data from HTTP-accessible sources, HTTP-in enables outside sources to request data from [[Script|LSL scripts]] in Second Life. The key difference is that [[llHTTPRequest]] exchanges data when the script in SL wants; HTTP-in allows outside sources to determine when they need to communicate with [[Script|LSL scripts]] in SL.
* '''llHTTPServerRequest(integer request-type, string data)'''
: An asynchronous event that will trigger an http_server event with a result.
:* SERVER_REQUEST_URL: Request a new LSL Server public URL. data is the 'trusted-path'
lsl: llHTTPServerRequest(SERVER_REQUEST_URL,"test-case/foo");
:* SERVER_CLEAR_URL: Clear a specific public url, data is the public url to be cleared, no http_server event triggered
lsl: llHTTPServerRequest(SERVER_CLEAR_URL,"https://sim123.agni/cap/f23b4b94-012d-44f2-bd0c-16c328321221");
:* SERVER_CLEAR_ALL_URLS: Clears all URLs associated with this task, no http_server event triggered.
lsl: llHTTPServerRequest(SERVER_CLEAR_ALL_URLS,"");
:* SERVER_GET_URLS: Requests a list of all URLs associated with this task
lsl: llHTTPServerRequest(SERVER_GET_URLS,"");


* '''http_server(string method, list meta, string body)'''
Prior to HTTP-in, similar functionality could be achieved by [[HTTP Polling|polling]] with [[llHTTPRequest]], [[llEmail]] and [[:Category:LSL_XML-RPC|XML-RPC]]. All three are cumbersome and the latter two have serious scalability bottlenecks.
: Event triggered when an URL is hit or with results of llHTTPServerRequest
:* method is GET/POST/PUT/DELETE for normal hits
::* meta: A list of headers of the format [header1,value1,header2,value2], standard headers:
:::* "x-trusted-path": A string as set when the url is requested, essentially trusted meta data associated with a specific cap.
:::* "x-untrusted-path": A string that is any trailing characters from the external request
:::* "x-forwarded-for": The host that made the request
:::* "x-forwarded-for-port": The port from the request (is this the port of the requester?)
::* body: The body of the request.
:* method is one of the following string constants for special case events.
::* URL_REQUEST_GRANTED: The script now has a public url.  The url is in the body.
::* URL_REQUEST_FAILED: Unable to get an URL, if possible a reason is given in the body.
::* GET_URLS: meta is a list of urls held by this task
::* URL_LOST: body is the url that was lost.
Is there a case where we won't know what urls were lost, just that they all were?
[[User:Kelly Linden|Kelly]] 15:30, 14 May 2008 (PDT)


http_server(string method, list meta, string body)
It is important to note that [[LSL HTTP server|LSL HTTP servers]] cannot use [[llSetContentType]] with [[CONTENT_TYPE_HTML]] for an [[llHTTPResponse]] except in very limited circumstances. See [[#Other_Limitations|Other Limitations]] for details.
{
    if (method == URL_REQUEST_GRANTED)
    {
      string public_url = body;
      // We now have the url, probably want to let someone else know - llHTTPRequest or similar
    }
    else if (method == URL_REQUEST_FAILED)
    {
      // Retry and/or let owner know we are broken.
    }
    else if (method == "GET")
    {
      integer i = llListFindList(meta,TRUSTED_PATH);
      string trusted = llList2String(meta,i+1);
      if (trusted == "echo")
      {
          llHTTPServerResponse(200,body);
      }
      else if (trusted == "status")
      {
          i = llListFindList(meta,UNTRUSTED_PATH);
          string untrusted = llList2String(meta,i+1);
          llRequestAgentData(AGENT_ONLINE,(key)(untrusted));
          // !! Won't have anything to return via llHTTPServerResponse until we get a dataserver() event!
      }
    }
}


* '''integer llHTTPServerResponse(integer status, string body)'''
'''This section forms the [[:Category:LSL HTTP#Understanding_LSL_HTTP_Communications | Incoming]] HTTP pipeline.'''
: Send ''body'' to the requester with status code ''status''
: Returns TRUE if the response was sent, FALSE if the connection was no longer available to respond on.
Can we do this outside this event? 
We need to time out pending requests anyway if the script takes too long.
[[User:Kelly Linden|Kelly Linden]] 15:30, 14 May 2008 (PDT)


If we're going to respond outside of the event, every request will need an id.
= Uses =
Another bad thing about doing it that way is that it allows people to hold
onto a file descriptor for a long time.
[[User:Phoenix Linden|Phoenix]] 16:32, 14 May 2008 (PDT)


Any more than if they plunk a while(1) in there? Or just take a really long
* Easily get data from [[Script|LSL scripts]] to outside viewers, [[Script|LSL scripts]] or servers.
time to do whatever?  We need to handle the case of 'they
** Web front end for a visitor counter or other statistics accumulator.
are taking too long to send a response' even if we limit them to sending
* Easily get data into [[Script|LSL scripts]] from outside viewers, [[Script|LSL scripts]] or servers.
responses from within the event. Allowing responses to be sent outside the
** A store with a web front end that communicates to an in-world object to exchange L$ and inventory items.
event allows for a lot more flexibility - sensors, notecard reading, link messages
** Inworld game for which an external program handles the primary game logic that needs to manipulate inworld items.
and all the dataserver requests all become possibilities.
* HUD UI when used with [[llSetContentType]]. See [[HTML HUD Demo]].
[[User:Kelly Linden|Kelly Linden]] 13:46, 16 May 2008 (PDT)
Gory Technical Details follow. Or jump straight to the [[LSL_http_server/examples | Script Examples]].


=== Simulator ===
= Script API =
What the simulator needs to do:
* '''Passes the method, body and these headers into the lsl script:'''
:* 'x-forwarded-for' and 'x-forwarded-for-port': for the ip/port of the requester.
:* 'x-trusted-path': The string passed into the url request
:* 'x-untrusted-path': A header appended by the cap server containing any path appended to the cap


* '''Clear/invalidate caps in some situations''':
== Events ==
Caps will be automatically be revoked when the region goes down.
* '''[[http_request|http_request(key id, string method, string body)]]'''
The caps can also be granted using the object as the "key" - and revoked when the object is destroyed.  
: Event triggered when an URL is hit:
[[User:Zero Linden|Zero Linden]] 14:37, 16 November 2007 (PST)
:* id is unique to this request
:* Supported methods are <code>"GET"</code>, <code>"POST"</code>, <code>"PUT"</code>, <code>"DELETE"</code>
:* body: The body of the request.
: Event also triggered with response to [[llRequestURL]] and [[llRequestSecureURL]]
:* id matches the key returned by [[llRequestURL]] or [[llRequestSecureURL]]
:* method == [[URL_REQUEST_GRANTED]] for success, [[URL_REQUEST_DENIED]] for failure to get an URL
:* body is the public URL. If unable to get a public URL body will be empty.
* '''[[Changed|changed(integer change)]]'''
:* [[CHANGED_REGION_START]]: New [[changed]] [[event]] triggered on region startup.
:* [[CHANGED_REGION]]: New [[changed]] [[event]] triggered when crossing regions or teleporting. Please read [[CHANGED_REGION]] for more info.
:* [[CHANGED_TELEPORT]]: New [[changed]] [[event]] triggered when the avatar teleports. Please read [[CHANGED_TELEPORT]] for more info.


Need more details about this.
== Functions ==
[[User:Kelly Linden|Kelly Linden]] 11:15, 14 May 2008 (PDT)
{{LSL Function/Head
:* Object removed from world
|func_id=345|func_sleep=0.0|func_energy=10.0
:* Object region change
|func=llRequestURL
:* Object owner change
|func_desc=Requests one HTTP:// url for use by the script. The [[http_request]] [[event]] is triggered with the result of the request. HTTP-in uses port 12046.
:* Region startup (clear all by region)
|return_type=key|return_text=that is the handle used for identifying the result in the [[http_request]] [[event]].
:* Script request
|func_footer
* '''Trigger an lsl http_server event with URL_LOST whenever a cap becomes invalid but the object and script still exist'''
|caveats=none
:* Object first rezed
}}
:* Object changed regions
{{LSL Function/Head
:* Region restarted
|func_id=346|func_sleep=0.0|func_energy=10.0
:* Capability was successfully removed by script request
|func=llRequestSecureURL
:* Object owner changed
|func_desc=Requests one HTTPS:// (SSL) url for use by the script. The [[http_request]] [[event]] is triggered with the result of the request. HTTPS-in uses port 12043.
* '''Track LSL Public URLs as a [[Limited Script Resource]].'''
|return_type=key|return_text=that is the handle used for identifying the result in the [[http_request]] [[event]].
:* This is a first use for a more general Limited Script Resource system that could eventually also handle script memory and cpu cycles.
|func_footer
:* Not all requests for an url will succeed, the scripter is expected to handle the failure case - we must let the scripter know when it fails.
|caveats=none
:* The number of available urls will be based on the amount of land owned in the region
}}
:* Need the ability to figure out how many/much of a resource is used and available
{{LSL Function/Head
''Do we need to support transient users or objects (where they own no land)?''
|func_id=347|func_sleep=0.0|func_energy=10.0
''Do we have special handling for sandbox regions?''
|func=llReleaseURL
''How do we handle rented parcels in private estates?  What controls (if any) do we need to give estate owners?''
|p1_type=string|p1_name=url|p1_desc=URL to release
[[User:Kelly Linden|Kelly Linden]] 15:58, 14 May 2008 (PDT)
|func_desc=Releases the specified URL, it will no longer be usable.
|func_footer
|caveats=none
}}
{{LSL Function/Head
|func_id=348|func_sleep=0.0|func_energy=10.0
|func=llHTTPResponse
|p1_type=key|p1_name=request_id|p1_desc=A valid HTTP request key
|p2_type=integer|p2_name=status|p2_desc=[http://en.wikipedia.org/wiki/List_of_HTTP_status_codes HTTP_Status] (200, 400, 404, etc)
|p3_type=string|p3_name=body|p3_desc=Contents of the response.
|func_desc=Responds to {{LSLP|request_id}} with {{LSLP|status}} and {{LSLP|body}}.
|func_footer=The response need not be made inside the [[http_request]] [[event]] but if it does not happen in a timely fashion the request will time out (within 25 seconds).
|caveats=none
}}
{{LSL Function/Head
|func_id=344|func_sleep=0.0|func_energy=10.0
|func=llGetFreeURLs
|return_type=integer|return_text=that is the number of available URLs.
|caveats=none
}}
{{LSL Function/Head
|func_id=349|func_sleep=0.0|func_energy=10.0
|func=llGetHTTPHeader
|p1_type=key|p1_name=request_id|p1_desc=A valid HTTP request key
|p2_type=string|p2_name=header|p2_desc=Lower case header value name
|return_type=string|return_text=that is the value for {{LSLP|header}} for {{LSLP|request_id}}.
|func_footer=Returns an empty [[string]] if the {{LSLP|header}} is not found or if the headers can no longer be accessed. Headers can only be accessed before [[llHTTPResponse]] is called and with-in the first 30 seconds after the [[http_request]] [[event]] is queued.


=== Cap Server ===
====Generated Headers====
This is the cap server on the same host as the region
These headers are automatically generated by the simulator, they were not actually sent by the requesting client. They supply information about the request to make parsing easier.
* Grants caps per normal that look like:
public url: https://sim123.agni/cap/f23b4b94-012d-44f2-bd0c-16c328321221
* Trailing path after the cap ID is passed to the internal url as the 'x-untrusted-path' header:
request: https://sim123.agni/cap/f23b4b94-012d-44f2-bd0c-16c328321221/foo/bar
forwarded to: <internal-url>
  w/ header: x-untrusted-path: foo/bar
* Caps are cleared when a region shuts down or restarts
* Caps are cleared by lsl request (via specific lsl function)
* Caps are cleared by region request (object deleted or moves to another region)
* Simulator must be able to verify a cap still exists.
* Separate apache config and/or resource pools for agent and viewer related caps vs task/LSL caps
** Would give us opportunities to ensure good behavior for agent critical features at the expense of LSL/Caps performance if needed
** Potentially allow us to minimize the impact of DOS style attacks against the LSL caps.


==== Questions / Issues ====
Sample URL: [https://sim3015.aditi.lindenlab.com:12043/cap/a7717681-2c04-e4ac-35e3-1f01c9861322/foo/bar?arg=gra {{HoverTextStyle|style=color:green;|<nowiki>https://sim3015.aditi.lindenlab.com:12043/cap/a7717681-2c04-e4ac-35e3-1f01c9861322</nowiki>|2={{String|x-script-url}} = {{String|https://sim3015.aditi.lindenlab.com:12043/cap/a7717681-2c04-e4ac-35e3-1f01c9861322}}}}{{HoverTextStyle|style=color:blue;|/foo/bar|2={{String|x-path-info}} = {{String|/foo/bar}}}}?{{HoverTextStyle|style=color:red;|1=arg=gra|2={{String|x-query-string}} = {{String|1=arg=gra}}}}]
* Should these caps time out?
* Define response codes - no script/object found, request throttled.
* Define mime-handling
I believe that we should, like the llHTTPRequest() call, be very clear in our handling of bodies and mime-types. 
In particular, accept only text/* mime types, and be sure to do proper charset handling and conversion into the
UTF-8 that LSL strings use.  (The code should all be cullable from the llHTTPRequest() implementation.)
[[User:Zero Linden|Zero Linden]] 14:37, 16 November 2007 (PST)


== Interface Requirements ==
{{{!}} class="lltable" border="1"
* No GUI components.
! header
* LSL Functions are written in stone, must get them right.
! description
! example
{{!}}-
{{!}} style="white-space: nowrap;" {{!}} "x-script-url"
{{!}} The base url, as originally received from [[llRequestURL]]/[[llRequestSecureURL]]
{{!}} <font color="green"><nowiki>https://sim3015.aditi.lindenlab.com:12043/cap/a7717681-2c04-e4ac-35e3-1f01c9861322</nowiki></font>
{{!}}-
{{!}} style="white-space: nowrap;" {{!}} "x-path-info"
{{!}} Any trailing path information from the requested url
{{!}} <font color="blue">/foo/bar</font>
{{!}}-
{{!}} style="white-space: nowrap;" {{!}} "x-query-string"
{{!}} Any query arguments, the text past the first "?" in the url
{{!}} <font color="red">arg=gra</font>
{{!}}-
{{!}} style="white-space: nowrap;" {{!}} "x-remote-ip"
{{!}} IP address of the host that made the request
{{!}}
{{!}}}


== Performance Requirements ==
====Common Headers====
This should add no database, assetserver or viewer load.
Simulator:
* In general load should be no higher than existing alternatives (xmlrpc, llemail and llhttprequest) for any single action. 
* Connections need to be throttled. 
: It would be nice if this could happen before the simulator on a per-cap basis, but throttling in the simulator handler would probably work as well.
Capserver:
* TBD: More info on cap server's load characteristics.  We believe this will be ok but we need to verify.
* Separate apache config and/or resource pools for agent caps vs task/LSL caps would give us opportunities to ensure good behavior for agent critical features at the expense of LSL/Caps performance if needed and allow us to minimize the impact of DOS style attacks against the LSL caps.
TODO:
* Get statistics on # of XMLRPC connections, llHTTPRequests and llEmails per unit of time per region to set expectations for level of usage.
We currently have 2 XMLRPC servers each processing about 30 concurrent requests.
I don't know the actual rate of requests, but I do know that we start failing at ~150
concurrent requests (out of a theoretical max of 200) per server, or ~300 total.
[[User:Kelly Linden|Kelly Linden]] 12:52, 14 May 2008 (PDT)


== Security Impact ==
{{{!}} class="lltable" border="1"
Creating a server accessible in any way from outside needs to be done with care.  The cap server already does this, and security concerns should already be handled here.  This isn't something to take for granted though.
! header
* Scripts should not be blocked from using llHTTPRequest to contact the ''public interface'' of the cap server.
! description
* Need to be careful about this since these are requests that originate from inside our network.
! example
{{!}}-
{{!}} style="white-space: nowrap;" {{!}} "user-agent"
{{!}} The user-agent header as reported by the requester
{{!}}
{{!}}}


== Limitations ==
====[[llHTTPRequest]] Headers====
* Size of the body of the requests will probably need to be limited.  At least to something that will fit within script memory - 2k is probably right.
{{LSL_Constants/HTTP_Headers|lower-case=*}}
This is needed to keep from overloading scripts and to reduce the overall potential load of handling large sets of data. 
|caveats=none
We probably also need to limit the size of returned data
}}
[[User:Kelly Linden|Kelly Linden]] 12:58, 14 May 2008 (PDT)
* Incoming requests will need to be throttled at some rate.  Script event queuing acts as a throttle for what connections the script will be able to hanlde, but that probably isn't enough.  I would guess that near the rate of llHTTPRequest throttling is probably good - ~1 / second / cap.
Can we throttle before we reach the sim? As part of apache, the cap server or squid?
[[User:Kelly Linden|Kelly Linden]] 12:58, 14 May 2008 (PDT)
* Need to limit the size of the 'trusted-path' and 'untrusted-path'


== Interactions ==
= URL Lifetime Limitations =
* Relies on capabilities and the cap server
* URLs are '''temporary'''!
* Effects LSL development including MONO
* URLs will be lost in the following cases, all detectable by the script events listed with them.
* [[Limited Script Resource | Limited Script Resources]]
** On object derez/rez: [[on_rez]]
** On script save/reset: ''[[default]] [[state_entry]] (trickier in multi-state [[Script|LSL scripts]])
** On region cross or TP(attachments): [[changed]] {{LSLGC|Events|event}}, [[CHANGED_REGION]] and [[CHANGED_TELEPORT]]. Testing the latter event, it has been found that teleports within a region DO NOT cause a URL to be lost, therefore you do not need to request a new URL on [[CHANGED_TELEPORT]], because [[CHANGED_REGION]] will handle a teleport to a different region. If you choose to request a URL after a teleport, it is recommended to release the old URL to be sure you don't have too many used. (Stone Tremont Aug 8th 2010)
** On region restart: [[changed]] [[event]], flag [[CHANGED_REGION_START]]
* When urls are 'lost' it means that all public urls for that script are gone, new ones will need to be requested and the new urls will '''not''' resemble the old ones.
* Maintaining persistent URLs will require building or using an external service similar to how [http://en.wikipedia.org/wiki/Dynamic_DNS Dynamic DNS] services work for tying domain names to dynamic IP addresses.


== Testing ==
Contributed HTTP-in URL mapping implementations and services:
TODO: Export internal test plan pages, once they exist. :)
* A Dynamic DNS service running on Google App Engine (contributed as an example) can be found in the forums [http://wiki.secondlife.com/wiki/User:Darien_Caldwell/HTTP-DNS here].
* Yet another one, running on GAE, with password protected and private domains. http://wiki.secondlife.com/wiki/Public_Object_DNS
* The [http://www.silverday.net/sqndns SilverDay ObjectDNS] is an easy to use dns-mapping-service with many configurable options (password protection, write protection, etc.) and an optional web-interface. The [[Script|LSL scripts]] are available here on the wiki (LSL-Script Library/[[Silverday ObjectDNS]]).


== Previous Design ==
= Resource Limitations =
[[LSL http server old | Previous design and comments]]
* There are a limited number of URLs available in each [[region]], split by land ownership exactly like prim limits.
** Use [[llGetFreeURLs]] to get the exact number of available URLs for the script.
** The number of allowed URLs is the same as the number of allowed [[prim|prims]] on the [[Land#Parcel|parcel]] the [[object]] is over.
**: ''Object owner does not matter, all objects over a [[Land#Parcel|parcel]] will use the resource pool for that [[Land#Parcel|parcel]].''
**: ''Like [[prim|prims]], all the [[Land#Parcel|parcels]] owned by the same owner and in the same [[region]] share the same pool of resources.''
**: ''If you have two [[Land#Parcel|parcels]] in a [[region]] that each support 100 URLs, then you could use all 200 in object(s) on a single [[Land#Parcel|parcel]].''
** The [[region]]'s [[object]] bonus factor does not apply to available URLs.
**: ''If a [[Land#Parcel|parcel]] has a max of 300 [[prim|prims]] in a [[region]] with a 2x bonus factor there will only be 150 urls allowed.''
* Each resident has their own unique pool of available URLs with a max of 38 URLs per resident.
** This is 1 per attachment point, but all 38 could be used by a single attachment for example.
* Vehicles are special and lazily moved to resident pools by the following logic:
** Any [[object]] that has a resident sitting on it is a 'vehicle'
** Vehicles will use the url resources from the [[Land#Parcel|parcel]] they are over until the cross a [[Land#Parcel|parcel]] border.
**: ''Specifically this prevents anyone from breaking your vending machine by sitting on it and making it a 'vehicle'.''
** When any [[object]] using URL resources with a resident sitting on it crosses a [[Land#Parcel|parcel]] boundary the resources will switch to the first sitting resident with enough resources. If no sitting agents have enough resources then the resources from the [[Land#Parcel|parcel]] being moved onto will be used. If even then there are not enough resources to use then the vehicle will be blocked from moving.
**: ''In short we do everything we can to find a pool to host the resources needed by the vehicle, but will block movement if we can't.''
* Parcel Sale: When a [[Land#Parcel|parcel]] is sold such that it changes the total available URLs in the [[region]] for either resident (seller or buyer) such that more URLs are being used than are available some objects will be returned.
** The objects returned will be from youngest [[object]] to oldest [[object]] of those using URLs in each category in order of [[object]] category: Temporary, Other, Group, Owner, Selected/Sat upon.
**: ''The '''only''' time objects are possibly returned is when [[Land#Parcel|parcels]] change owner, and only if more resources are being used than allowed.''
**: ''We return youngest temporary objects before older temporary objects before younger 'other' (owned by non-group, non-[[Land#Parcel|parcel]]-owner) objects etc.''


[[Category: Feature Requests]]
= Other Limitations =
[[Category: Design Discussions]]
* Size of the body of requests on the '''[[:Category:LSL HTTP#Understanding_LSL_HTTP_Communications | Incoming pipeline]]''' will be [[limit]]ed to 2k [[bytes]], and cannot currently be changed.
* Size of headers of requests will be limited to 255 bytes per header, not total.
* The size of responses to requests is not currently limited, but this is subject to review during testing.
* By default the content type of the returned data is always <code>"text/plain; charset=utf-8"</code>, this however can be changed via [[llSetContentType]] for all subsequent responses via [[llHTTPResponse]].
* There is a cap of 64 in flight requests per script. This is based on the maximum number of pending [[event|events]] in LSL. After hitting the 64 request limit, the simulator cap server returns <code>{{String|503 Service Unavailable}}</code> to the inbound request.
* ''We may throttle the rate we accept hits at the CAP server level as well. This is possible, but has not yet been decided.''
* HTML can be used to in-client browser, including media on a prim, with some restrictions. See [[llSetContentType]] and [[CONTENT_TYPE_HTML]] for further details.
* If making a parser, remember that you should probably restrict access to only Linden Lab HTML URLs as to not have your bandwith stolen. See [[Simulator_IP_Addresses]] for details.
* It is important to note that when appending a query string to a cap URL there '''MUST''' be a trailing slash between the cap guid and the query string token <code>"?"</code>. For example: [https://sim3015.aditi.lindenlab.com:12043/cap/a7717681-2c04-e4ac-35e3-1f01c9861322?arg=gra {{HoverTextStyle|style=color:green;|<nowiki>https://sim3015.aditi.lindenlab.com:12043/cap/a7717681-2c04-e4ac-35e3-1f01c9861322</nowiki>|2={{String|x-script-url}} = {{String|https://sim3015.aditi.lindenlab.com:12043/cap/a7717681-2c04-e4ac-35e3-1f01c9861322}}}}?{{HoverTextStyle|style=color:red;|1=arg=gra|2={{String|x-query-string}} = {{String|1=arg=gra}}}}] will return a 500 HTTP status {{Wikipedia|List_of_HTTP_status_codes#5xx_Server_Error|Server Error code}}, but [https://sim3015.aditi.lindenlab.com:12043/cap/a7717681-2c04-e4ac-35e3-1f01c9861322/?arg=gra {{HoverTextStyle|style=color:green;|<nowiki>https://sim3015.aditi.lindenlab.com:12043/cap/a7717681-2c04-e4ac-35e3-1f01c9861322</nowiki>|2={{String|x-script-url}} = {{String|https://sim3015.aditi.lindenlab.com:12043/cap/a7717681-2c04-e4ac-35e3-1f01c9861322}}}}{{HoverTextStyle|style=color:blue;|/|2={{String|x-path-info}} = {{String|/}}}}?{{HoverTextStyle|style=color:red;|1=arg=gra|2={{String|x-query-string}} = {{String|1=arg=gra}}}}] will succeed.
 
= Links =
* [[LSL_http_server/examples | Script Examples]]
* [[LSL_http_server/design | Design Document]]
* Design Jira:{{jira|SVC-1086}}
 
{{LSLC|HTTP}}
{{LSLC|Communications}}

Latest revision as of 02:54, 16 November 2022

Introduction

This is the counterpart to llHTTPRequest. While llHTTPRequest enables LSL scripts to request data from HTTP-accessible sources, HTTP-in enables outside sources to request data from LSL scripts in Second Life. The key difference is that llHTTPRequest exchanges data when the script in SL wants; HTTP-in allows outside sources to determine when they need to communicate with LSL scripts in SL.

Prior to HTTP-in, similar functionality could be achieved by polling with llHTTPRequest, llEmail and XML-RPC. All three are cumbersome and the latter two have serious scalability bottlenecks.

It is important to note that LSL HTTP servers cannot use llSetContentType with CONTENT_TYPE_HTML for an llHTTPResponse except in very limited circumstances. See Other Limitations for details.

This section forms the Incoming HTTP pipeline.

Uses

  • Easily get data from LSL scripts to outside viewers, LSL scripts or servers.
    • Web front end for a visitor counter or other statistics accumulator.
  • Easily get data into LSL scripts from outside viewers, LSL scripts or servers.
    • A store with a web front end that communicates to an in-world object to exchange L$ and inventory items.
    • Inworld game for which an external program handles the primary game logic that needs to manipulate inworld items.
  • HUD UI when used with llSetContentType. See HTML HUD Demo.

Gory Technical Details follow. Or jump straight to the Script Examples.

Script API

Events

Event triggered when an URL is hit:
  • id is unique to this request
  • Supported methods are "GET", "POST", "PUT", "DELETE"
  • body: The body of the request.
Event also triggered with response to llRequestURL and llRequestSecureURL

Functions

llRequestURL

Function: key llRequestURL( );

Requests one HTTP:// url for use by the script. The http_request event is triggered with the result of the request. HTTP-in uses port 12046.
Returns a key that is the handle used for identifying the result in the http_request event.

llRequestSecureURL

Function: key llRequestSecureURL( );

Requests one HTTPS:// (SSL) url for use by the script. The http_request event is triggered with the result of the request. HTTPS-in uses port 12043.
Returns a key that is the handle used for identifying the result in the http_request event.

llReleaseURL

Function: llReleaseURL( string url );

Releases the specified URL, it will no longer be usable.

• string url – URL to release

llHTTPResponse

Function: llHTTPResponse( key request_id, integer status, string body );

Responds to request_id with status and body.

• key request_id – A valid HTTP request key
• integer status – HTTP_Status (200, 400, 404, etc)
• string body – Contents of the response.

The response need not be made inside the http_request event but if it does not happen in a timely fashion the request will time out (within 25 seconds).

llGetFreeURLs

Function: integer llGetFreeURLs( );

Returns an integer that is the number of available URLs.

llGetHTTPHeader

Function: string llGetHTTPHeader( key request_id, string header );

Returns a string that is the value for header for request_id.

• key request_id – A valid HTTP request keyA valid HTTP request key
• string header – Lower case header value name

Returns an empty string if the header is not found or if the headers can no longer be accessed. Headers can only be accessed before llHTTPResponse is called and with-in the first 30 seconds after the http_request event is queued.

Generated Headers

These headers are automatically generated by the simulator, they were not actually sent by the requesting client. They supply information about the request to make parsing easier.

Sample URL: https://sim3015.aditi.lindenlab.com:12043/cap/a7717681-2c04-e4ac-35e3-1f01c9861322/foo/bar?arg=gra

header description example
"x-script-url" The base url, as originally received from llRequestURL/llRequestSecureURL https://sim3015.aditi.lindenlab.com:12043/cap/a7717681-2c04-e4ac-35e3-1f01c9861322
"x-path-info" Any trailing path information from the requested url /foo/bar
"x-query-string" Any query arguments, the text past the first "?" in the url arg=gra
"x-remote-ip" IP address of the host that made the request

Common Headers

header description example
"user-agent" The user-agent header as reported by the requester

llHTTPRequest Headers

Headers sent by the simulator in the course of calling llHTTPRequest.
Header Description Example data
connection Connection options close
cache-control Maximum response age accepted. max-age=259200
x-forwarded-for Used to show the IP address connected to through proxies. 127.0.0.1
via Shows the recipients and protocols used between the User Agent and the server. 1.1 sim10115.agni.lindenlab.com:3128 (squid/2.7.STABLE9)
content-length The size of the entity-body, in decimal number of octets. 17
pragma The message should be forwarded to the server, even if it has a cached version of the data. no-cache
x-secondlife-shard The environment the object is in. "Production" is the main grid and "Testing" is the preview grid Production
x-secondlife-region The name of the region the object is in, along with the global coordinates of the region's south-west corner Jin Ho (264448, 233984)
x-secondlife-owner-name Legacy name of the owner of the object Zeb Wyler
x-secondlife-owner-key UUID of the owner of the object 01234567-89ab-cdef-0123-456789abcdef
x-secondlife-object-name The name of the object containing the script Object
x-secondlife-object-key The key of the object containing the script 01234567-89ab-cdef-0123-456789abcdef
x-secondlife-local-velocity The velocity of the object 0.000000, 0.000000, 0.000000
x-secondlife-local-rotation The rotation of the object containing the script 0.000000, 0.000000, 0.000000, 1.000000
x-secondlife-local-position The position of the object within the region (173.009827, 75.551231, 60.950001)
user-agent The user-agent header sent by LSL Scripts. Contains Server version. Second Life LSL/16.05.24.315768 (http://secondlife.com)
content-type The media type of the entity body. text/plain; charset=utf-8
accept-charset Acceptable character sets from the server. Q being the quality expected when sending the different character sets. utf-8;q=1.0, *;q=0.5
accept Media types the server will accept. text/*, application/xhtml+xml, application/atom+xml, application/json, application/xml, application/llsd+xml, application/x-javascript, application/javascript, application/x-www-form-urlencoded, application/rss+xml
accept-encoding Acceptable content encodings for the server. deflate, gzip
host The internet host being requested. secondlife.com
  • CGI environments may place the headers into variables by capitalizing the entire name, replacing dashes with underscores, and prefixing the name with "HTTP_", e.g. "x-secondlife-object-name" becomes "HTTP_X_SECONDLIFE_OBJECT_NAME".
  • HTTP header names are case insensitive [1]. Some ISPs may modify the case of header names, as was seen in BUG-5094.

URL Lifetime Limitations

  • URLs are temporary!
  • URLs will be lost in the following cases, all detectable by the script events listed with them.
  • When urls are 'lost' it means that all public urls for that script are gone, new ones will need to be requested and the new urls will not resemble the old ones.
  • Maintaining persistent URLs will require building or using an external service similar to how Dynamic DNS services work for tying domain names to dynamic IP addresses.

Contributed HTTP-in URL mapping implementations and services:

Resource Limitations

  • There are a limited number of URLs available in each region, split by land ownership exactly like prim limits.
    • Use llGetFreeURLs to get the exact number of available URLs for the script.
    • The number of allowed URLs is the same as the number of allowed prims on the parcel the object is over.
      Object owner does not matter, all objects over a parcel will use the resource pool for that parcel.
      Like prims, all the parcels owned by the same owner and in the same region share the same pool of resources.
      If you have two parcels in a region that each support 100 URLs, then you could use all 200 in object(s) on a single parcel.
    • The region's object bonus factor does not apply to available URLs.
      If a parcel has a max of 300 prims in a region with a 2x bonus factor there will only be 150 urls allowed.
  • Each resident has their own unique pool of available URLs with a max of 38 URLs per resident.
    • This is 1 per attachment point, but all 38 could be used by a single attachment for example.
  • Vehicles are special and lazily moved to resident pools by the following logic:
    • Any object that has a resident sitting on it is a 'vehicle'
    • Vehicles will use the url resources from the parcel they are over until the cross a parcel border.
      Specifically this prevents anyone from breaking your vending machine by sitting on it and making it a 'vehicle'.
    • When any object using URL resources with a resident sitting on it crosses a parcel boundary the resources will switch to the first sitting resident with enough resources. If no sitting agents have enough resources then the resources from the parcel being moved onto will be used. If even then there are not enough resources to use then the vehicle will be blocked from moving.
      In short we do everything we can to find a pool to host the resources needed by the vehicle, but will block movement if we can't.
  • Parcel Sale: When a parcel is sold such that it changes the total available URLs in the region for either resident (seller or buyer) such that more URLs are being used than are available some objects will be returned.
    • The objects returned will be from youngest object to oldest object of those using URLs in each category in order of object category: Temporary, Other, Group, Owner, Selected/Sat upon.
      The only time objects are possibly returned is when parcels change owner, and only if more resources are being used than allowed.
      We return youngest temporary objects before older temporary objects before younger 'other' (owned by non-group, non-parcel-owner) objects etc.

Other Limitations

Links

Retrieved from "https://wiki.secondlife.com/w/index.php?title=LSL_HTTP_server&oldid=1212773"