llHTTPRequest

From Second Life Wiki
Revision as of 14:44, 30 March 2022 by Gwyneth Llewelyn (talk | contribs) (Edited mostly because the (Loading...) issue is *still* present as of 2022 (!); but took some time to do a general polishing, cleaning up, correcting typos, and applying a few more formatting rules as used elsewhere in the LSL Wiki)
Jump to navigation Jump to search

Summary

Function: key llHTTPRequest( string url, list parameters, string body );
0.0 Forced Delay
10.0 Energy

Sends an HTTP request to the specified URL with the body of the request and parameters.
Returns a handle (a key) identifying the HTTP request made.

• string url – A valid HTTP/HTTPS URL.
• list parameters – configuration parameters, specified as HTTP_* flag-value pairs

[ parameter1, value1, parameter2, value2, . . . parameterN, valueN]

• string body – Contents of the request.

Flag Parameter(s) Default Parameter Value(s) Description
HTTP_METHOD 0 [string method] ["GET"] "GET", "POST", "PUT" and "DELETE"
HTTP_MIMETYPE 1 [string MIME_type] ["text/plain;charset=utf-8"] text/* MIME types should specify a charset. To emulate HTML forms use application/x-www-form-urlencoded. This allows you to set the body to a properly escaped (llEscapeURL) sequence of <name,value> pairs in the form var=value&var2=value2 and have them automatically parsed by web frameworks.
MIME types must be specified in the format: type/subtype[;option=value]
Some valid examples are
"text/html"
"text/plain;charset=utf-8"
"application/xhtml+xml"
"application/json"
"application/x-www-form-urlencoded"
"application/rss+xml"
"multipart/mixed; boundary="---1234567890---""
HTTP_BODY_MAXLENGTH 2 [integer length] [2048] Sets the maximum (UTF-8 encoded) byte length of the HTTP response body. The maximum that can be set depends upon which VM is used.
KBwarning.png Warning: Applies to the Outgoing pipeline only (HTTP calls invoked by llHTTPRequest,and responses from http_response).
KBtip2.png Tip: When you only need to request a small amount of data from a remote source, consider using the Content-Range header instead.
HTTP_VERIFY_CERT 3 [integer verify] [TRUE] If TRUE, the server SSL certificate must be verifiable using one of the standard certificate authorities[1] when making HTTPS requests. If FALSE, any server SSL certificate will be accepted.
HTTP_VERBOSE_THROTTLE 4 [integer noisy] [TRUE] If TRUE, shout error messages to DEBUG_CHANNEL if the outgoing request rate exceeds the server limit. If FALSE, the error messages are suppressed (llHTTPRequest will still return NULL_KEY).
HTTP_CUSTOM_HEADER 5 [string name, string value] NA Add an extra custom HTTP header to the request. The first string is the name of the parameter to change, e.g. "Pragma", and the second string is the value, e.g. "no-cache". Multiple custom headers may be configured per request, as long as the combined custom header length is no greater than 4096 characters. Note that certain headers, such as the default headers, are blocked for security reasons.
HTTP_PRAGMA_NO_CACHE 6 [integer send_header] [TRUE] Sends "Pragma: no-cache" header (TRUE), or does not send a "Pragma" header (FALSE).
HTTP_USER_AGENT 7 [string user agent value] [(none)] The user agent value is appended to the one generated by LSL itself. It should follow the syntax from the HTTP standard like: "My-Script-Name/1.0 (Mozilla compatible)".

Note: Spaces are not allowed in HTTP User Agent token values, so "My Script Name/1.0" will produce a script error; change the spaces to hyphens ("-")

HTTP_ACCEPT 8 [string MIME_type] ["text/plain;charset=utf-8"] HTTP_ACCEPT parameters can be passed to limit the number of mime types that are sent in the Accept: header of the HTTP request. Specified mime types may include character set and q parameters. This parameter may be specified multiple times.

The specified mime type must be one already recognized by llHTTPRequest. These include any text/ mime type, or the following application mime types: “application/xhtml+xml”, “application/atom+xml”, “application/json”, “application/xml”, “application/llsd+xml”, “application/x-javascript”, “application/javascript”, “application/x-www-form-urlencoded”, or “application/rss+xml”.

The Content-Type header in the response is checked against the specified HTTP_ACCEPT parameters. If the value of the header is not in the list of acceptable mime types, llHTTPRequest will return 415 as a result code and the body will be "Unsupported or unknown Content-Type."

HTTP_EXTENDED_ERROR 9 [integer extended] [FALSE] If TRUE llHTTPRequest will always return a key. If there was an error making the HTTP request. Detailed error information will be returned through the http_response event using the provided key. Error information is delivered in a JSON block as described in RFC 7807. Details about extended return codes can be found below.
Headers sent by the simulator in the course of calling llHTTPRequest.
Header Description Example data
Connection Connection options close
Cache-Control Maximum response age accepted. max-age=259200
X-Forwarded-For Used to show the IP address connected to through proxies. 127.0.0.1
Via Shows the recipients and protocols used between the User Agent and the server. 1.1 sim10115.agni.lindenlab.com:3128 (squid/2.7.STABLE9)
Content-Length The size of the entity-body, in decimal number of octets. 17
Pragma The message should be forwarded to the server, even if it has a cached version of the data. no-cache
X-SecondLife-Shard The environment the object is in. "Production" is the main grid and "Testing" is the preview grid Production
X-SecondLife-Region The name of the region the object is in, along with the global coordinates of the region's south-west corner Jin Ho (264448, 233984)
X-SecondLife-Owner-Name Legacy name of the owner of the object Zeb Wyler
X-SecondLife-Owner-Key UUID of the owner of the object 01234567-89ab-cdef-0123-456789abcdef
X-SecondLife-Object-Name The name of the object containing the script Object
X-SecondLife-Object-Key The key of the object containing the script 01234567-89ab-cdef-0123-456789abcdef
X-SecondLife-Local-Velocity The velocity of the object 0.000000, 0.000000, 0.000000
X-SecondLife-Local-Rotation The rotation of the object containing the script 0.000000, 0.000000, 0.000000, 1.000000
X-SecondLife-Local-Position The position of the object within the region (173.009827, 75.551231, 60.950001)
User-Agent The user-agent header sent by LSL Scripts. Contains Server version. Second Life LSL/16.05.24.315768 (http://secondlife.com)
Content-Type The media type of the entity body. text/plain; charset=utf-8
Accept-Charset Acceptable character sets from the server. Q being the quality expected when sending the different character sets. utf-8;q=1.0, *;q=0.5
Accept Media types the server will accept. text/*, application/xhtml+xml, application/atom+xml, application/json, application/xml, application/llsd+xml, application/x-javascript, application/javascript, application/x-www-form-urlencoded, application/rss+xml
Accept-Encoding Acceptable content encodings for the server. deflate, gzip
Host The internet host being requested. secondlife.com
  • CGI environments may place the headers into variables by capitalizing the entire name, replacing dashes with underscores, and prefixing the name with "HTTP_", e.g. "X-SecondLife-Object-Name" becomes "HTTP_X_SECONDLIFE_OBJECT_NAME".
  • HTTP header names are case insensitive [1]. Some ISPs may modify the case of header names, as was seen in BUG-5094.


Extended Errors

When the HTTP_EXTENDED_ERROR parameter is TRUE, llHTTPRequest will return extended error information to the script through the http_response event. A detailed error explanation is returned as a JSON blob blob in the body of the event and an error code is provided through the HTTP status. The format of the JSON blob is described in detail in [RFC 7807]. 

{
    "type":   "http://wiki.secondlife.com/wiki/llHTTPRequest", // reference to the documentation for llHTTPRequest.
    "title":  "Error Title", // A human readable title for the error.
    "detail": "Description of error condition.", // A human readable description of the error condition.
    "status": 400 // The HTTP Status for the call.
}


Error Description
Method Not Allowed 405 The HTTP method specified for HTTP_METHOD is not supported.
Unsupported Media Type 415 The mime type specified in HTTP_MIMETYPE or HTTP_ACCEPT is not supported.
Enhance Your Calm 420 HTTP requests have been throttled for this object, script, or owner. Back off making further HTTP calls until the throttle has cleared.
HTTP Unavailable 470 HTTP and/or HTTPS have been disabled in this region.
Invalid URL 471 The URL passed to llHTTPRequest contains invalid characters or uses a protocol other than http or https.
Parameter Error 472 There was an issue with the parameters passed to llHTTPRequest. One of the following conditions may be the problem
  • The type of the parameter value may be incorrect.
  • The parameter key may be something other than an integer.
  • A parameter may be missing.
  • The value for a parameter failed validation.
Illegal Request Header 473 A request header passed to HTTP_CUSTOM_HEADER is not allowed.
Too Many Headers 474 Too many request headers have been specified with HTTP_CUSTOM_HEADER.

Caveats

  • Spaces, control characters, and other characters that are not allowed in URLs will cause a run time error.
  • The corresponding http_response event will be triggered in all scripts in the prim, not just in the requesting script.
  • Requests must fully complete after 60 seconds, or else the response will be thrown away and the http_response status code will be 499.
  • The response body is limited to 2048 bytes by default, see HTTP_BODY_MAXLENGTH above to increase it. If the response is longer, it will be truncated.
  • The request body size (eg, of POST and PUT requests) is limited only by available script memory. Scripts can hold at most 32k characters in a string, under Mono, as characters are two bytes each, so, scripts cannot upload over 32k UTF-8 characters
  • Cannot be used to load textures or images from the internet, for more information see Web Textures.
  • If the accessed site is relying on the LSL script to report L$ transactions, then it must check the X-SecondLife-Shard header to see if the script is running on the beta grid.
  • Some servers will return a 405 error if you send POST to a file that can't accept metadata, such as a text or HTML file. Make sure you use the GET method to ensure success in any environment.
  • While the HTTP status code from the server is provided to the script, redirect codes such as 302 will result in the redirect being automatically and transparently followed ONLY IF the HTTP_METHOD is GET, with the resulting response being returned. If the HTTP_METHOD is anything other than GET then you'll get back an http_response with a status code of 302, but without any way to view the headers, you can't know where you were being redirected to unless that was also included in the body.
  • The following applies when making a request to a script using HTTP-In:
  • X-SecondLife-Owner-Name may return "(Loading...)" instead of owner name (still true, 30th of March, 2022)
  • Requests made at approx 0625 SLT may fail with a 503 status code, with "ERROR: The requested URL could not be retrieved", and "(111) Connection refused" in the body of the response. This has been confirmed as expected behaviour by Kelly, due to the nightly maintenance and log rotation. It does reliably impact object to object HTTP at that time, and quite probably may impact object to/from web around the same time. The interruption in service is fairly brief, and the precise timing may vary as LL adjust their nightly maintenance processes, or due to server load.
  • Use HTTP_MIMETYPE to set the Content-Type header. Attempts to use HTTP_CUSTOM_HEADER to set it will cause a runtime script error.

Throttles

The LSL function llHTTPRequest is throttled in two ways: by prim and owner. All group-owned objects are considered together in the same throttle.

The current limits are 25 requests in 20 seconds for each object, and 1000 in 20 seconds for each owner. These may change in the future if needed to prevent problems in regions.

It is possible for a large collection of objects or scripts to make many calls to llHTTPRequest and reach this throttle. When a script calls llHTTPRequest with the throttle blocking the request, it will return NULL_KEY.

The calling script must check for the NULL_KEY result and react properly for the script and object to function correctly. Some things to consider:

  • Pause further requests until the throttle clears
  • Do not make any additional llHTTPRequest calls until enough time has passed for the throttle to reset. They will fail and continue to return NULL_KEY otherwise.
  • Once reached, the throttles will remain in effect as long as requests continue, but will clear if there is a silent period with no requests at least twice the throttle interval, in this case 2 * 20 or 40 seconds.‎
Consider how a group of objects behaves. Developers must consider how multiple objects will interact and how that will affect clearing the throttle.‎
The llHTTPRequest throttle is most likely to be an issue with a large number of objects in a region making requests. To clear the throttle fastest, when an object encounters the throttle, it should broadcast a region-wide chat message to other objects informing them of the event and stopping their requests. If those objects continue making requests, those requests will fail and just prolong recovery.‎
If an object waits and still gets a failure, it may be a good idea to increase the time before the next request and/or add a small random value to the wait time. This may help prevent failures caused by large groups of objects acting nearly in unison.
  • Object requests are throttled at approximately 25 requests per 20 seconds. This is to support a sustained rate of one per second or a maximum burst of up to 25 every 40 seconds (twice the interval for maximum burst), smaller bursts are recommended.
  • NULL_KEY is returned if the request is throttled.
  • See this thread and this thread for older details.

Examples

key http_request_id;

default
{
    state_entry()
    {
        http_request_id = llHTTPRequest("url", [], "");
    }

    http_response(key request_id, integer status, list metadata, string body)
    {
        if (request_id != http_request_id) return;// exit if unknown

        vector COLOR_BLUE = <0.0, 0.0, 1.0>;
        float  OPAQUE     = 1.0;

        llSetText(body, COLOR_BLUE, OPAQUE);
    }
}

Example PHP test script: 

<?php header("content-type: text/plain; charset=utf-8"); ?>
Headers received:
<?php

/**
 * @author Wouter Hobble
 * @copyright 2008
 */

foreach ($_SERVER as $k => $v)
{
    if( substr($k, 0, 5) == 'HTTP_')
    {
        print "\n". $k. "\t". $v;
    }
}
?>

Example PHP wrapper script both capturing Apache headers and global methods 

<?php
    // Author Waster Skronski.
    // General Public License (GPL).
    // Mind that some headers are not included because they're either useless or unreliable.

    $USE_APACHE_HEADERS = TRUE;// switch to false if you need CGI methods

    if ($USE_APACHE_HEADERS)
    {
        $headers    = apache_request_headers();
        $objectgrid = $headers["X-SecondLife-Shard"];
        $objectname = $headers["X-SecondLife-Object-Name"];
        $objectkey  = $headers["X-SecondLife-Object-Key"];
        $objectpos  = $headers["X-SecondLife-Local-Position"];
        $ownerkey   = $headers["X-SecondLife-Owner-Key"];
        $ownername  = $headers["X-SecondLife-Owner-Name"];
        $regiondata = $headers["X-SecondLife-Region"];
        $regiontmp  = explode ("(",$regiondata);        // cut coords off
        $regionpos  = explode (")",$regiontmp[1]);
        $regionname = substr($regiontmp[0],0,-1);       // cut last space from region name
    } else {
        $db         = $GLOBALS;
        $headers    = $db['$_ENV'];
        $objectgrid = $headers["HTTP_X_SECONDLIFE_SHARD"];
        $objectname = $headers["HTTP_X_SECONDLIFE_OBJECT_NAME"];
        $objectkey  = $headers["HTTP_X_SECONDLIFE_OBJECT_KEY"];
        $ownerkey   = $headers["HTTP_X_SECONDLIFE_OWNER_KEY"];
        $objectpos  = $headers["HTTP_X_SECONDLIFE_LOCAL_POSITION"];
        $ownername  = $headers["HTTP_X_SECONDLIFE_OWNER_NAME"];
        $regiondata = $headers["HTTP_X_SECONDLIFE_REGION"];
        $regiontmp  = explode ("(",$regiondata);
        $regionpos  = explode (")",$regiontmp[1]);
        $regionname = substr($regiontmp[0],0,-1);
    }
?>

Example wrapper script for GoDaddy.com Linux PHP servers (fix made by Thomas Conover): 

<?php
// FETCH HEADERS START

if (!function_exists('apache_request_headers'))
{
    function apache_request_headers() {
        foreach($_SERVER as $key=>$value) {
            if (substr($key,0,5)=="HTTP_") {
                $key=str_replace(" ","-",ucwords(strtolower(str_replace("_"," ",substr($key,5)))));
                $out[$key]=$value;
            }else{
                $out[$key]=$value;
            }
        }
        return $out;
    }
}
// Mind that some headers are not included because they're either useless or unreliable (e.g. X-Secondlife-Local-Position)
$headers    = apache_request_headers();
$objectgrid = $headers["X-Secondlife-Shard"];
$objectname = $headers["X-Secondlife-Object-Name"];
$objectkey  = $headers["X-Secondlife-Object-Key"];
$objectpos  = $headers["X-Secondlife-Local-Position"];
$ownerkey   = $headers["X-Secondlife-Owner-Key"];
$ownername  = $headers["X-Secondlife-Owner-Name"];
$regiondata = $headers["X-Secondlife-Region"];
$regiontmp  = explode ("(",$regiondata);            // cut coords off
$regionname = substr($regiontmp[0],0,-1);           // cut last space from region name
$regiontmp  = explode (")",$regiontmp[1]);
$regionpos  = $regiontmp[0];


// FETCH HEADERS END
?>

Notes

If for some reason while using the function llHTTPRequest or the event http_response you are unable to parse a known good RSS feed or some other form of web contents, you will need to work around it outside of Second Life. This is unlikely to change in the near future since checking the headers requires more overhead at the simulator level.

You may find that some web servers return either a null or a nonsensical result when llHTTPRequest is used, even though the same URL in a PC web browser returns the expected result. This may be due to the fact that the llHTTPRequest User Agent string is not recognised by some web servers as it does not contain "Mozilla", which would identify it as a web browser instead of, for example, a Shoutcast or an RSS client. This is also true when the PHP script relies on $_COOKIE. Neither can the function llHTTPRequest set a cookie nor can the event http_request retrieve them.

CGI environments may place the headers into variables by capitalizing the entire name, replacing dashes with underscores, and prefixing the name with HTTP_, e.g. HTTP_X_SECONDLIFE_OBJECT_NAME. PHP $_SERVER variables do this as well.

Apache can include the headers in its logs, using the CustomLog and LogFormat directives. See the docs for details on the syntax.

See Also

Events

•  http_response

Functions

•  llEscapeURL
•  llHTTPResponse
•  llUnescapeURL

Articles

•  Simulator IP Addresses

Deep Notes

Footnotes

  1. ^ A list of acceptable certificate authorities can be found at SCR-473.

Signature

function key llHTTPRequest( string url, list parameters, string body );
Retrieved from "https://wiki.secondlife.com/w/index.php?title=LlHTTPRequest&oldid=1210546"