Difference between revisions of "LlHTTPRequest"
(When the response is received, a http_response event is raised.) |
Monty Linden (talk | contribs) (Update the throttling details. Still not what it should be but the basics are now correct.) |
||
(One intermediate revision by one other user not shown) | |||
Line 36: | Line 36: | ||
==== Throttles ==== | ==== Throttles ==== | ||
The LSL function [[llHTTPRequest]] is throttled in | The LSL function [[llHTTPRequest]] is throttled in three ways: by object, by owner, and by http error. All group-owned objects are considered together in the same throttle. | ||
The current limits are 25 requests in 20 seconds for each object | The current limits are: | ||
* 25 requests in 20 seconds for each object | |||
* 1000 requests in 20 seconds for each owner with higher limits for some regions | |||
* Five HTTP errors (500 or higher) in 60 seconds for each script | |||
These may change in the future if needed to prevent problems in regions. | |||
It is possible for a large collection of objects or scripts to make many calls to [[llHTTPRequest]] and reach | It is possible for a large collection of objects or scripts to make many calls to [[llHTTPRequest]] and reach one or more throttles. When a script calls [[llHTTPRequest]] with a throttle blocking the request, it will return [[NULL_KEY]]. | ||
The calling script must check for the [[NULL_KEY]] result and react properly for the script and object to function correctly. Some things to consider: | The calling script must check for the [[NULL_KEY]] result and react properly for the script and object to function correctly. Some things to consider: | ||
Line 46: | Line 50: | ||
* Pause further requests until the throttle clears | * Pause further requests until the throttle clears | ||
* Do not make any additional [[llHTTPRequest]] calls until enough time has passed for the throttle to reset. They will fail and continue to return [[NULL_KEY]] otherwise. | * Do not make any additional [[llHTTPRequest]] calls until enough time has passed for the throttle to reset. They will fail and continue to return [[NULL_KEY]] otherwise. | ||
* Once reached, the throttles will remain in effect as long as requests continue, but will clear if there is a silent period with no requests at least twice the throttle interval, in | * Once reached, the throttles will remain in effect as long as requests continue, but will clear if there is a silent period with no requests at least twice the throttle interval, in the common case 2 * 20 or 40 seconds. | ||
:Consider how a group of objects behaves. Developers must consider how multiple objects will interact and how that will affect clearing the throttle. | :Consider how a group of objects behaves. Developers must consider how multiple objects will interact and how that will affect clearing the throttle. | ||
Line 194: | Line 198: | ||
You may find that some web servers return either a null or a nonsensical result when [[llHTTPRequest]] is used, even though the same URL in a PC web browser returns the expected result. This may be due to the fact that the [[llHTTPRequest]] User Agent string is not recognised by some web servers as it does not contain <code>"Mozilla"</code>, which would identify it as a web browser instead of, for example, a [https://www.shoutcast.com/ Shoutcast] or an RSS client. This is also true when the PHP script relies on [https://www.php.net/manual/en/reserved.variables.cookies.php $_COOKIE]. Neither can the function [[llHTTPRequest]] set a cookie nor can the event [[http_request]] retrieve them. | You may find that some web servers return either a null or a nonsensical result when [[llHTTPRequest]] is used, even though the same URL in a PC web browser returns the expected result. This may be due to the fact that the [[llHTTPRequest]] User Agent string is not recognised by some web servers as it does not contain <code>"Mozilla"</code>, which would identify it as a web browser instead of, for example, a [https://www.shoutcast.com/ Shoutcast] or an RSS client. This is also true when the PHP script relies on [https://www.php.net/manual/en/reserved.variables.cookies.php $_COOKIE]. Neither can the function [[llHTTPRequest]] set a cookie nor can the event [[http_request]] retrieve them. | ||
CGI environments may place the headers into variables by capitalizing the entire name, replacing dashes with underscores, and prefixing the name with <code>HTTP_</code>, e.g. <code>HTTP_X_SECONDLIFE_OBJECT_NAME</code>. PHP <code>$_SERVER</code> variables do this as well. | CGI environments may place the headers into variables by capitalizing the entire name, replacing dashes with underscores, and prefixing the name with <code>HTTP_</code>, e.g. <code>HTTP_X_SECONDLIFE_OBJECT_NAME</code>. PHP <code>$_SERVER</code> variables do this as well, and so does PHP running inside PHP-FPM (as opposed to an Apache module), which is the standard way to configure PHP on non-Apache webservers (such as [https://nginx.org/nginx {{code|nginx}}]). | ||
Apache can include the headers in its logs, using the <code>CustomLog</code> and <code>LogFormat</code> directives. See [https://httpd.apache.org/docs/2.4/mod/mod_log_config.html#formats the docs] for details on the syntax. | Apache can include the headers in its logs, using the <code>CustomLog</code> and <code>LogFormat</code> directives. See [https://httpd.apache.org/docs/2.4/mod/mod_log_config.html#formats the docs] for details on the syntax. |
Latest revision as of 06:31, 28 June 2024
LSL Portal | Functions | Events | Types | Operators | Constants | Flow Control | Script Library | Categorized Library | Tutorials |
Summary
Function: key llHTTPRequest( string url, list parameters, string body );0.0 | Forced Delay |
10.0 | Energy |
Sends an HTTP request to the specified URL with the body of the request and parameters. When the response is received, a http_response event is raised.
Returns a handle (a key) identifying the HTTP request made.
• string | url | – | A valid HTTP/HTTPS URL. | |
• list | parameters | – | configuration parameters, specified as HTTP_* flag-value pairs
[ parameter1, value1, parameter2, value2, . . . parameterN, valueN] |
|
• string | body | – | Contents of the request. |
Flag | Parameter(s) | Default Parameter Value(s) | Description | |||||
---|---|---|---|---|---|---|---|---|
HTTP_METHOD | 0 | [string method] | ["GET"] | "GET", "POST", "PUT" and "DELETE" | ||||
HTTP_MIMETYPE | 1 | [string MIME_type] | ["text/plain;charset=utf-8"] | text/* MIME types should specify a charset. To emulate HTML forms use application/x-www-form-urlencoded. This allows you to set the body to a properly escaped (llEscapeURL) sequence of <name,value> pairs in the form var=value&var2=value2 and have them automatically parsed by web frameworks.
| ||||
HTTP_BODY_MAXLENGTH | 2 | [integer length] | [2048] | Sets the maximum (UTF-8 encoded) byte length of the HTTP response body. The maximum that can be set depends upon which VM is used.
| ||||
HTTP_VERIFY_CERT | 3 | [integer verify] | [TRUE] | If TRUE, the server SSL certificate must be verifiable using one of the standard certificate authorities[1] when making HTTPS requests. If FALSE, any server SSL certificate will be accepted. | ||||
HTTP_VERBOSE_THROTTLE | 4 | [integer noisy] | [TRUE] | If TRUE, shout error messages to DEBUG_CHANNEL if the outgoing request rate exceeds the server limit. If FALSE, the error messages are suppressed (llHTTPRequest will still return NULL_KEY). | ||||
HTTP_CUSTOM_HEADER | 5 | [string name, string value] | NA | Add an extra custom HTTP header to the request. The first string is the name of the parameter to change, e.g. "Pragma", and the second string is the value, e.g. "no-cache". Multiple custom headers may be configured per request, as long as the combined custom header length is no greater than 4096 characters. Note that certain headers, such as the default headers, are blocked for security reasons. | ||||
HTTP_PRAGMA_NO_CACHE | 6 | [integer send_header] | [TRUE] | Sends "Pragma: no-cache" header (TRUE), or does not send a "Pragma" header (FALSE). | ||||
HTTP_USER_AGENT | 7 | [string user agent value] | [(none)] | The user agent value is appended to the one generated by LSL itself. It should follow the syntax from the HTTP standard like: "My-Script-Name/1.0 (Mozilla compatible)".
Note: Spaces are not allowed in HTTP User Agent token values, so "My Script Name/1.0" will produce a script error; change the spaces to hyphens ("-") | ||||
HTTP_ACCEPT | 8 | [string MIME_type] | ["text/plain;charset=utf-8"] | HTTP_ACCEPT parameters can be passed to limit the number of mime types that are sent in the Accept: header of the HTTP request. Specified mime types may include character set and q parameters. This parameter may be specified multiple times.
The specified mime type must be one already recognized by llHTTPRequest. These include any text/ mime type, or the following application mime types: “application/xhtml+xml”, “application/atom+xml”, “application/json”, “application/xml”, “application/llsd+xml”, “application/x-javascript”, “application/javascript”, “application/x-www-form-urlencoded”, or “application/rss+xml”. The Content-Type header in the response is checked against the specified HTTP_ACCEPT parameters. If the value of the header is not in the list of acceptable mime types, llHTTPRequest will return 415 as a result code and the body will be "Unsupported or unknown Content-Type." | ||||
HTTP_EXTENDED_ERROR | 9 | [integer extended] | [FALSE] | If TRUE llHTTPRequest will always return a key. If there was an error making the HTTP request. Detailed error information will be returned through the http_response event using the provided key. Error information is delivered in a JSON block as described in RFC 7807. Details about extended return codes can be found below. |
Header | Description | Example data |
---|---|---|
Connection | Connection options | close |
Cache-Control | Maximum response age accepted. | max-age=259200 |
X-Forwarded-For | Used to show the IP address connected to through proxies. | 127.0.0.1 |
Via | Shows the recipients and protocols used between the User Agent and the server. | 1.1 sim10115.agni.lindenlab.com:3128 (squid/2.7.STABLE9) |
Content-Length | The size of the entity-body, in decimal number of octets. | 17 |
Pragma | The message should be forwarded to the server, even if it has a cached version of the data. | no-cache |
X-SecondLife-Shard | The environment the object is in. "Production" is the main grid and "Testing" is the preview grid | Production |
X-SecondLife-Region | The name of the region the object is in, along with the global coordinates of the region's south-west corner | Jin Ho (264448, 233984) |
X-SecondLife-Owner-Name | Legacy name of the owner of the object | Zeb Wyler |
X-SecondLife-Owner-Key | UUID of the owner of the object | 01234567-89ab-cdef-0123-456789abcdef |
X-SecondLife-Object-Name | The name of the object containing the script | Object |
X-SecondLife-Object-Key | The key of the object containing the script | 01234567-89ab-cdef-0123-456789abcdef |
X-SecondLife-Local-Velocity | The velocity of the object | 0.000000, 0.000000, 0.000000 |
X-SecondLife-Local-Rotation | The rotation of the object containing the script | 0.000000, 0.000000, 0.000000, 1.000000 |
X-SecondLife-Local-Position | The position of the object within the region | (173.009827, 75.551231, 60.950001) |
User-Agent | The user-agent header sent by LSL Scripts. Contains Server version. | Second Life LSL/16.05.24.315768 (http://secondlife.com) |
Content-Type | The media type of the entity body. | text/plain; charset=utf-8 |
Accept-Charset | Acceptable character sets from the server. Q being the quality expected when sending the different character sets. | utf-8;q=1.0, *;q=0.5 |
Accept | Media types the server will accept. | text/*, application/xhtml+xml, application/atom+xml, application/json, application/xml, application/llsd+xml, application/x-javascript, application/javascript, application/x-www-form-urlencoded, application/rss+xml |
Accept-Encoding | Acceptable content encodings for the server. | deflate, gzip |
Host | The internet host being requested. | secondlife.com |
|
Extended Errors
When the HTTP_EXTENDED_ERROR parameter is TRUE, llHTTPRequest will return extended error information to the script through the http_response event. A detailed error explanation is returned as a JSON blob blob in the body of the event and an error code is provided through the HTTP status. The format of the JSON blob is described in detail in [RFC 7807].
{
"type": "http://wiki.secondlife.com/wiki/llHTTPRequest", // reference to the documentation for llHTTPRequest.
"title": "Error Title", // A human readable title for the error.
"detail": "Description of error condition.", // A human readable description of the error condition.
"status": 400 // The HTTP Status for the call.
}
Error | Description | |
---|---|---|
Method Not Allowed | 405 | The HTTP method specified for HTTP_METHOD is not supported. |
Unsupported Media Type | 415 | The mime type specified in HTTP_MIMETYPE or HTTP_ACCEPT is not supported. |
Enhance Your Calm | 420 | HTTP requests have been throttled for this object, script, or owner. Back off making further HTTP calls until the throttle has cleared. |
HTTP Unavailable | 470 | HTTP and/or HTTPS have been disabled in this region. |
Invalid URL | 471 | The URL passed to llHTTPRequest contains invalid characters or uses a protocol other than http or https. |
Parameter Error | 472 | There was an issue with the parameters passed to llHTTPRequest. One of the following conditions may be the problem
|
Illegal Request Header | 473 | A request header passed to HTTP_CUSTOM_HEADER is not allowed. |
Too Many Headers | 474 | Too many request headers have been specified with HTTP_CUSTOM_HEADER. |
Caveats
- Spaces, control characters, and other characters that are not allowed in URLs will cause a run time error.
- The corresponding http_response event will be triggered in all scripts in the prim, not just in the requesting script.
- Requests must fully complete after 60 seconds, or else the response will be thrown away and the http_response status code will be 499.
- The response body is limited to 2048 bytes by default, see HTTP_BODY_MAXLENGTH above to increase it. If the response is longer, it will be truncated.
- The request body size (e.g., of POST and PUT requests) is limited only by available script memory. Scripts can hold at most 32k characters in a string, under Mono, as characters are two bytes each, so, scripts cannot upload over 32k UTF-8 characters
- Cannot be used to load textures or images from the internet, for more information see Web Textures.
- If the accessed site is relying on the LSL script to report L$ transactions, then it must check the
X-SecondLife-Shard
header to see if the script is running on the beta grid. - Some servers will return a
405
error if you send POST to a file that can't accept metadata, such as a text or HTML file. Make sure you use the GET method to ensure success in any environment. - While the HTTP status code from the server is provided to the script, redirect codes such as
302
will result in the redirect being automatically and transparently followed ONLY IF the HTTP_METHOD is GET, with the resulting response being returned. If the HTTP_METHOD is anything other than GET then you'll get back an http_response with a status code of302
, but without any way to view the headers, you can't know where you were being redirected to unless that was also included in the body. - The following applies when making a request to a script using HTTP-In:
- When appending a query string to a cap URL there MUST be a trailing slash between the cap guid and the query string token
"?"
. For example: https://sim3015.aditi.lindenlab.com:12043/cap/a7717681-2c04-e4ac-35e3-1f01c9861322?arg=gra will return a 500 HTTP status Server Error code, but https://sim3015.aditi.lindenlab.com:12043/cap/a7717681-2c04-e4ac-35e3-1f01c9861322/?arg=gra will succeed.
- When appending a query string to a cap URL there MUST be a trailing slash between the cap guid and the query string token
X-SecondLife-Owner-Name
may return"(Loading...)"
instead of owner name (still true, 30th of March, 2022)- Requests made at approx 0625 SLT may fail with a
503
status code, with"ERROR: The requested URL could not be retrieved"
, and"(111) Connection refused"
in the body of the response. This has been confirmed as expected behavior by Kelly, due to the nightly maintenance and log rotation. It does reliably impact object to object HTTP at that time, and quite probably may impact object to/from web around the same time. The interruption in service is fairly brief, and the precise timing may vary as LL adjust their nightly maintenance processes, or due to server load. - Use
HTTP_MIMETYPE
to set theContent-Type
header. Attempts to useHTTP_CUSTOM_HEADER
to set it will cause a runtime script error. - If the origin server does not include a content-type header in its response, LSL will attempt to treat the incoming data as "text/plain; charset=utf-8". This behavior diverges from RFC 2616.
Throttles
The LSL function llHTTPRequest is throttled in three ways: by object, by owner, and by http error. All group-owned objects are considered together in the same throttle.
The current limits are:
- 25 requests in 20 seconds for each object
- 1000 requests in 20 seconds for each owner with higher limits for some regions
- Five HTTP errors (500 or higher) in 60 seconds for each script
These may change in the future if needed to prevent problems in regions.
It is possible for a large collection of objects or scripts to make many calls to llHTTPRequest and reach one or more throttles. When a script calls llHTTPRequest with a throttle blocking the request, it will return NULL_KEY.
The calling script must check for the NULL_KEY result and react properly for the script and object to function correctly. Some things to consider:
- Pause further requests until the throttle clears
- Do not make any additional llHTTPRequest calls until enough time has passed for the throttle to reset. They will fail and continue to return NULL_KEY otherwise.
- Once reached, the throttles will remain in effect as long as requests continue, but will clear if there is a silent period with no requests at least twice the throttle interval, in the common case 2 * 20 or 40 seconds.
- Consider how a group of objects behaves. Developers must consider how multiple objects will interact and how that will affect clearing the throttle.
- The llHTTPRequest throttle is most likely to be an issue with a large number of objects in a region making requests. To clear the throttle fastest, when an object encounters the throttle, it should broadcast a region-wide chat message to other objects informing them of the event and stopping their requests. If those objects continue making requests, those requests will fail and just prolong recovery.
- If an object waits and still gets a failure, it may be a good idea to increase the time before the next request and/or add a small random value to the wait time. This may help prevent failures caused by large groups of objects acting nearly in unison.
- Object requests are throttled at approximately 25 requests per 20 seconds. This is to support a sustained rate of one per second or a maximum burst of up to 25 every 40 seconds (twice the interval for maximum burst), smaller bursts are recommended.
- NULL_KEY is returned if the request is throttled.
- See this thread and this thread for older details.
Examples
key http_request_id;
default
{
state_entry()
{
http_request_id = llHTTPRequest("url", [], "");
}
http_response(key request_id, integer status, list metadata, string body)
{
if (request_id != http_request_id) return;// exit if unknown
vector COLOR_BLUE = <0.0, 0.0, 1.0>;
float OPAQUE = 1.0;
llSetText(body, COLOR_BLUE, OPAQUE);
}
}
Example PHP test script:
<?php header("content-type: text/plain; charset=utf-8"); ?>
Headers received:
<?php
/**
* @author Wouter Hobble
* @copyright 2008
*/
foreach ($_SERVER as $k => $v)
{
if( substr($k, 0, 5) == 'HTTP_')
{
print "\n". $k. "\t". $v;
}
}
?>
Example PHP wrapper script both capturing Apache headers and global methods
<?php
// Author Waster Skronski.
// General Public License (GPL).
// Mind that some headers are not included because they're either useless or unreliable.
$USE_APACHE_HEADERS = TRUE;// switch to false if you need CGI methods
if ($USE_APACHE_HEADERS)
{
$headers = apache_request_headers();
$objectgrid = $headers["X-SecondLife-Shard"];
$objectname = $headers["X-SecondLife-Object-Name"];
$objectkey = $headers["X-SecondLife-Object-Key"];
$objectpos = $headers["X-SecondLife-Local-Position"];
$ownerkey = $headers["X-SecondLife-Owner-Key"];
$ownername = $headers["X-SecondLife-Owner-Name"];
$regiondata = $headers["X-SecondLife-Region"];
$regiontmp = explode ("(",$regiondata); // cut coords off
$regionpos = explode (")",$regiontmp[1]);
$regionname = substr($regiontmp[0],0,-1); // cut last space from region name
} else {
$db = $GLOBALS;
$headers = $db['$_ENV'];
$objectgrid = $headers["HTTP_X_SECONDLIFE_SHARD"];
$objectname = $headers["HTTP_X_SECONDLIFE_OBJECT_NAME"];
$objectkey = $headers["HTTP_X_SECONDLIFE_OBJECT_KEY"];
$ownerkey = $headers["HTTP_X_SECONDLIFE_OWNER_KEY"];
$objectpos = $headers["HTTP_X_SECONDLIFE_LOCAL_POSITION"];
$ownername = $headers["HTTP_X_SECONDLIFE_OWNER_NAME"];
$regiondata = $headers["HTTP_X_SECONDLIFE_REGION"];
$regiontmp = explode ("(",$regiondata);
$regionpos = explode (")",$regiontmp[1]);
$regionname = substr($regiontmp[0],0,-1);
}
?>
Example wrapper script for GoDaddy.com Linux PHP servers (fix made by Thomas Conover):
<?php
// FETCH HEADERS START
if (!function_exists('apache_request_headers'))
{
function apache_request_headers() {
foreach($_SERVER as $key=>$value) {
if (substr($key,0,5)=="HTTP_") {
$key=str_replace(" ","-",ucwords(strtolower(str_replace("_"," ",substr($key,5)))));
$out[$key]=$value;
}else{
$out[$key]=$value;
}
}
return $out;
}
}
// Mind that some headers are not included because they're either useless or unreliable (e.g. X-Secondlife-Local-Position)
$headers = apache_request_headers();
$objectgrid = $headers["X-Secondlife-Shard"];
$objectname = $headers["X-Secondlife-Object-Name"];
$objectkey = $headers["X-Secondlife-Object-Key"];
$objectpos = $headers["X-Secondlife-Local-Position"];
$ownerkey = $headers["X-Secondlife-Owner-Key"];
$ownername = $headers["X-Secondlife-Owner-Name"];
$regiondata = $headers["X-Secondlife-Region"];
$regiontmp = explode ("(",$regiondata); // cut coords off
$regionname = substr($regiontmp[0],0,-1); // cut last space from region name
$regiontmp = explode (")",$regiontmp[1]);
$regionpos = $regiontmp[0];
// FETCH HEADERS END
?>
Notes
If for some reason while using the function llHTTPRequest or the event http_response you are unable to parse a known good RSS feed or some other form of web contents, you will need to work around it outside of Second Life. This is unlikely to change in the near future since checking the headers requires more overhead at the simulator level.
You may find that some web servers return either a null or a nonsensical result when llHTTPRequest is used, even though the same URL in a PC web browser returns the expected result. This may be due to the fact that the llHTTPRequest User Agent string is not recognised by some web servers as it does not contain "Mozilla"
, which would identify it as a web browser instead of, for example, a Shoutcast or an RSS client. This is also true when the PHP script relies on $_COOKIE. Neither can the function llHTTPRequest set a cookie nor can the event http_request retrieve them.
CGI environments may place the headers into variables by capitalizing the entire name, replacing dashes with underscores, and prefixing the name with HTTP_
, e.g. HTTP_X_SECONDLIFE_OBJECT_NAME
. PHP $_SERVER
variables do this as well, and so does PHP running inside PHP-FPM (as opposed to an Apache module), which is the standard way to configure PHP on non-Apache webservers (such as nginx
).
Apache can include the headers in its logs, using the CustomLog
and LogFormat
directives. See the docs for details on the syntax.
See Also
Events
• | http_response |
Functions
• | llEscapeURL | |||
• | llHTTPResponse | |||
• | llUnescapeURL |
Articles
• | Simulator IP Addresses |