Difference between revisions of "Search API"

From Second Life Wiki
Jump to navigation Jump to search
m (Added footnotes at the bottom in its own section)
 
(6 intermediate revisions by 5 users not shown)
Line 1: Line 1:
{{Unsupported API}}
{{:API Portal/navigation}}
{{:API Portal/navigation}}
__TOC__
__TOC__
Line 13: Line 14:
* Information on this Wiki.  
* Information on this Wiki.  


Linden Lab uses a [http://www.google.com/enterprise/gsa/ Google Search Appliance] to index web pages created at 3AM and 3PM each day and hosted on a separate web server. It can take up to three days for results to actually get indexed, but it is often just one day.
Linden Lab use<s>s</s>d a {{Wikipedia|Google Search Appliance}} to index web pages created at 3AM and 3PM each day and hosted on a separate web server. It can take up to three days for results to actually get indexed, but it is often just one day.{{citation needed}}


User the following URLs:
{{Warning|Most of this page refers to a different, earlier version of the search engine API used by Linden Lab. It's not conclusive if these links still work and are correctly forwarded to the new API. ''Caveat utilitor''.}}
 
Use the following URLs:


{| style="border:1px solid #A3B1BF; text-align:left; background:#f5faff"
{| style="border:1px solid #A3B1BF; text-align:left; background:#f5faff"
Line 31: Line 34:
|Redirects to http://search.secondlife.com/client_search.php
|Redirects to http://search.secondlife.com/client_search.php
|}
|}
==== Important note ====
After the last search API upgrade/update (verified date=February 2024), it seems that ''most'' of the URLs explained below are not operational any more. The ''current'' format of the search API seems to use the following template:
https://search.secondlife.com/?query_term=nothing+works&search_type=standard&collection_chosen=destinations&collection_chosen=places&maturity=g
It's not even clear if Linden Lab still uses PHP on their backend search engine.


=== Paging ===
=== Paging ===
Line 58: Line 69:


=== Location ===
=== Location ===
When searching from the viewer, the location of the avatar is passed along to the Search API. This is optional. The coordinates do not appear to have any effect on the search results. They have the potential to show results based on your current location, (such as displaying nearby snapshots on [http://www.sluniverse.com/pics/ Snapzilla]).
When searching from the viewer, the location of the avatar is passed along to the Search API. This is optional. The coordinates do not appear to have any effect on the search results. They have the potential to show results based on your current location, (such as displaying nearby snapshots on [https://www.sluniverse.com/pics/ Snapzilla]{{Dead link|date=February 2024}}<ref>On [https://www.virtualverse.one/forums/threads/coming-out-of-the-dark-%F0%9F%8C%9F.13126/ VirtualVerse] in November 2023, Cristiano mentioned that he might not be able to recover Snapzilla.</ref>).
The following parameters are used.
The following parameters are used.
{| style="border:1px solid #A3B1BF; text-align:left; background:#f5faff"
{| style="border:1px solid #A3B1BF; text-align:left; background:#f5faff"
Line 83: Line 94:
|}
|}


http://search.secondlife.com/client_search.php?q=hippo&region=Here&x=128&y=128&z=36
https://search.secondlife.com/client_search.php?q=hippo&region=Here&x=128&y=128&z=36


=== Mature Content ===
=== Mature Content ===
By default, the Search API blocks mature content in its results. A parameter "m" may be specified to allow or block mature content. To allow mature content, the value should be "y" meaning "Yes". Blocking mature content will be "n" for "No".
By default, the Search API blocks mature content in its results. A binary parameter "mat" may be specified to what mature content. To have PG content, you add "1" meaning "PG". To allow mature content, you add "2" meaning "Matrue". To allow adult content, you add "4" meaning "Adult".
 
{| style="border:1px solid #A3B1BF; text-align:left; background:#f5faff"
|-
|style="background: #cee0f2; padding: 0.3em; text-align:center;"|'''Parameter'''
|style="background: #cee0f2; padding: 0.3em; text-align:center;"|'''URI'''
|-
|PG Only
|http://search.secondlife.com/client_search.php?q=hippo&mat=1
|-
|PG & Mature
|http://search.secondlife.com/client_search.php?q=hippo&mat=3
|-
|PG & Mature & Adult
|http://search.secondlife.com/client_search.php?q=hippo&mat=7
|-
|Mature Only
|http://search.secondlife.com/client_search.php?q=hippo&mat=2
|-
|Mature & Adult
|http://search.secondlife.com/client_search.php?q=hippo&mat=6
|-
|Adult Only
|http://search.secondlife.com/client_search.php?q=hippo&mat=4
|-
|}
 
 
=== Mature Content Old ===
By default, the Search API blocks mature content in its results. A parameter {{code|m}} may be specified to allow or block mature content. To allow mature content, the value should be {{code|y}} meaning {{code|Yes}}. Blocking mature content will be {{code|n}} for {{code|No}}.


In addition, you can still filter out mature results by appending a flag to the query "+-SLS_MATURE".
In addition, you can still filter out mature results by appending a flag to the query {{code|+-SLS_MATURE}}.


{| style="border:1px solid #A3B1BF; text-align:left; background:#f5faff"
{| style="border:1px solid #A3B1BF; text-align:left; background:#f5faff"
Line 108: Line 148:
|}
|}


Note: %20 represents a URL Encoded space
'''Note:''' {{code|%20}} represents a URL-encoded space character


=== Category ===
=== Category ===
To limit the results to specific categories, the "s" parameter is used. This value defaults to "All", but can also be set to one of the following: All, Events, Groups, People, Places, Wiki.
To limit the results to specific categories, the {{code|s}} parameter is used. This value defaults to {{code|All}}, but can also be set to one of the following: {{code|All, Events, Groups, People, Places, Wiki}}.


{| style="border:1px solid #A3B1BF; text-align:left; background:#f5faff"
{| style="border:1px solid #A3B1BF; text-align:left; background:#f5faff"
Line 140: Line 180:
There are a few parameters that identify the language to display the results in. Although the searchable content provided by the residents is not translated, portions of the page are. This includes headers and informational text (such as the number of matching results).
There are a few parameters that identify the language to display the results in. Although the searchable content provided by the residents is not translated, portions of the page are. This includes headers and informational text (such as the number of matching results).


The language (also known as local or culture) specified for the viewers user interface is passed along in the "lang" parameter. These are ISO 639-1 Values for language, and an optional country code ([http://www.iso.org/iso/country_codes/iso_3166_code_lists/english_country_names_and_code_elements.htm ISO 3166-1]). Not all languages are supported. Some languages are in beta.
The language (also known as ''locale'' or ''culture'') specified for the viewers user interface is passed along in the {{code|lang}} parameter. These are ISO 639-1 Values for language, and an optional country code ([https://www.iso.org/obp/ui/#iso:pub:PUB500001:en ISO 3166]). Not all languages are supported. Some languages are in beta.


{| style="border:1px solid #A3B1BF; text-align:left; background:#f5faff"
{| style="border:1px solid #A3B1BF; text-align:left; background:#f5faff"
Line 158: Line 198:




In addition, another attribute appears in querys when navigating between pages using a standard web browser. It is assumed that this may represent the same value, but only provides the language code without the country code.
In addition, another attribute appears in queries when navigating between pages using a standard web browser. It is assumed that this may represent the same value, but only provides the language code without the country code.


http://search.secondlife.com/client_search.php?q=hippo&hl=en
http://search.secondlife.com/client_search.php?q=hippo&hl=en


=== Output ===
=== Output ===


At the moment, it appears that there are two fields that deal with output. The first "output" is probably telling the search appliance to return xml formatted data. The "client" parameter is then transforming the results into a different format. Since the client is raw_xml_frontend, it is probably not doing any transformation at all and sending us the exact output from the Google appliance.
At the moment{{when}}, it appears that there are two fields that deal with output. The first "output" is probably telling the search appliance to return {{Wikipedia|XML}} formatted data. The {{code|client}} parameter is then transforming the results into a different format. Since the client is {{code|raw_xml_frontend}}, it is probably not doing any transformation at all and sending us the exact output from the Google Appliance.


{| style="border:1px solid #A3B1BF; text-align:left; background:#f5faff"
{| style="border:1px solid #A3B1BF; text-align:left; background:#f5faff"
Line 174: Line 214:
|output
|output
|xml_no_dtd
|xml_no_dtd
|xml results from appliance. output format.
|XML results from appliance. output format.
|-
|-
|client
|client
|raw_xml_frontend
|raw_xml_frontend
|transform into xml results for user agent
|transform into XML results for user agent
|}
|}


http://search.secondlife.com/client_search.php?q=hippo&output=xml_no_dtd&client=raw_xml_frontend
http://search.secondlife.com/client_search.php?q=hippo&output=xml_no_dtd&client=raw_xml_frontend


=== Tags ===
=== Tags ===
Tags allow content to be flagged with special attributes. Tags are used in the query of search results (q=hippo +-SLS_MATURE)
Tags allow content to be flagged with special attributes. Tags are used in the query of search results ({{code|q=hippo +-SLS_MATURE}})


{| style="border:1px solid #A3B1BF; text-align:left; background:#f5faff"
{| style="border:1px solid #A3B1BF; text-align:left; background:#f5faff"
Line 210: Line 250:
|session
|session
|00000000-0000-0000-0000-000000000000
|00000000-0000-0000-0000-000000000000
|This value changes each time that the SL Viewer logs into the main grid. When querying against the appliance with a standard browser and navigating pages, the value is an empty Guid.
|This value changes each time that the SL Viewer logs into the main grid. When querying against the appliance with a standard browser and navigating pages, the value is an empty [[UUID|Guid]].
|-
|-
|site
|site
Line 226: Line 266:
|sort
|sort
|date:D:L:d1
|date:D:L:d1
|Apparently this "phrase" is all over the net in regards to the default sorting for Google search appliance. Guessing it is sorting by date. Other option is relevance?
|Apparently this "phrase" is all over the net in regard to the default sorting for Google Search Appliance. Guessing it is sorting by date. Other option is relevance?
|-
|-
|dbg
|dbg
Line 247: Line 287:
=== Headers ===
=== Headers ===


The following headers have been captured while querying against the google search appliance with an in-world viewer.
The following headers have been captured while querying against the Google Search Appliance with an in-world viewer.


  Host: search.secondlife.com
  Host: search.secondlife.com
Line 259: Line 299:
   __utma=########.##################.##########.##########.##########.###; __utmc=########
   __utma=########.##################.##########.##########.##########.###; __utmc=########


NOTE: Line feeds added to very long lines above for readability.
'''Note:''' Line feeds added to very long lines above for readability.


=== Work-around for LSL ===
=== Work-around for LSL ===


It should be noted that LSL scripts using [[llHTTPRequest]] method can not query against the Google appliance directly. Linden Lab has blocked all access due to security reasons of requests from machines on the local network. To work around this, many people use proxy services. One popular service is offered by Google. It is originally intended to make content available for mobile phones. For an example, see the [http://www.google.com/gwt/n Proxy Search Form].
It should be noted that LSL scripts using [[llHTTPRequest]] method cannot query against the Google appliance directly. Linden Lab has blocked all access due to security reasons of requests from machines on the local network. To work around this, many people use proxy services. One popular service is offered by Google. It is originally intended to make content available for mobile phones. For an example, see the [https://www.google.com/gwt/n Proxy Search Form].


'''Example of proxy search URL'''
'''Example of proxy search URL'''
http://www.google.com/gwt/n?u=http%3A%2F%2Fsearch.secondlife.com%2Fclient_search.php%3Fq%3Dhippo&_gwt_noimg=1
https://www.google.com/gwt/n?u=http%3A%2F%2Fsearch.secondlife.com%2Fclient_search.php%3Fq%3Dhippo&_gwt_noimg=1


=Search Engine Optimization=
=Search Engine Optimization=
Search Engine Optimization (SEO) is the method of fine tuning the name and description of content and links to further promote the position of a listing based on [http://en.wikipedia.org/wiki/Relevance_(information_retrieval) relevance] on the search results.
Search Engine Optimization (SEO) is the method of fine tuning the name and description of content and links to further promote the position of a listing based on {{Wikipedia|Relevance_(information_retrieval)|relevance}} on the search results.
==Groups==
===Groups===
==Regions==
===Regions===
Regions will always show up before parcel listings when their name contains a keyword that the end-user had searched for.
Regions will always show up before parcel listings when their name contains a keyword that the end-user had searched for.


==Parcels==
===Parcels===
The search engine searches for parcel name and description only if the parcel had been marked to show up in search results (30 L$ per week).
The search engine searches for parcel name and description only if the parcel had been marked to show up in search results (30 L$ per week).


Line 316: Line 356:
|}
|}


==Events==
===Events===
==Residents==
===Residents===
Residents must enable their profile to show in search results. When a resident name matches, an additional link directly to the profile is displayed regardless if the resident marked their profile to show up in search results.
Residents must enable their profile to show in search results. When a resident name matches, an additional link directly to the profile is displayed regardless if the resident marked their profile to show up in search results.


==Parcel Objects==
===Parcel Objects===
All objects with the default name of "object" are ignored by search results.
All objects with the default name of "object" are ignored by search results.


The object must be rezzed on a listed parcel in order to be searched. An object within the contents of a vendor will not be searched.
The object must be [[rez|rezzed]] on a listed [[parcel]] in order to be searched. An object within the contents of a vendor will not be searched.


Only the root prim of an object will be searched. Objects made of many linked prims will not index each prim.
Only the root [[prim]] of an object will be searched. Objects made of many [[Link|linked prims]] will not index each prim.


==Classifieds==
===Classifieds===
=References=
==References==
* {{Slkb|4722|Search FAQ}}
* [[Linden_Lab_Official:Search FAQ|Search FAQ]]
* [http://blog.secondlife.com/2007/10/19/new-search-currently-under-development/ New Search Currently Under Development]
* [http://blog.secondlife.com/2007/10/19/new-search-currently-under-development/ New Search Currently Under Development]{{Dead link}}
==Notes==
{{References}}

Latest revision as of 10:14, 27 February 2024

NOTE: This API and its documentation are NOT provided or supported by Linden Lab. Linden Lab does not guarantee that this API works as described or works at all. Use at your own risk!

Getting Started

What is the search API?

The Second Life Search API enables you to search Second Life using a web browser for:

  • Events
  • People
  • Places
  • Groups
  • Information on this Wiki.

Linden Lab usesd a "Wikipedia logo" Google Search Appliance to index web pages created at 3AM and 3PM each day and hosted on a separate web server. It can take up to three days for results to actually get indexed, but it is often just one day.[citation needed]

Warning!

Most of this page refers to a different, earlier version of the search engine API used by Linden Lab. It's not conclusive if these links still work and are correctly forwarded to the new API. Caveat utilitor.


Use the following URLs:

URL Details
http://search.secondlife.com/search.php?q=hippo External Browser
http://search.secondlife.com/client_search.php?q=hippo In-world browser built into SL Viewer/Client
http://secondlife.com/app/search/search_proxy.php?q=hippo Redirects to http://search.secondlife.com/client_search.php

Important note

After the last search API upgrade/update (verified date=February 2024), it seems that most of the URLs explained below are not operational any more. The current format of the search API seems to use the following template: 

https://search.secondlife.com/?query_term=nothing+works&search_type=standard&collection_chosen=destinations&collection_chosen=places&maturity=g

It's not even clear if Linden Lab still uses PHP on their backend search engine.

Paging

Each query will return 10 results at the most. The parameter passed for paging is "start". This is a zero-based index identifying what record to start with. To view page 1, the following URL may be used.

q= [search for something here] &start= [(page - 1) * 10]

Page 1 http://search.secondlife.com/client_search.php?q=hippo&start=0

Page 2 http://search.secondlife.com/client_search.php?q=hippo&start=10

Page 3 http://search.secondlife.com/client_search.php?q=hippo&start=20

...

Page 100 http://search.secondlife.com/client_search.php?q=hippo&start=990

Note Any results after page 100 will return an error message similar to the following: 

The requested URL /client_search.php&lang=en-us&session=00000000-0000-0000-0000-000000000000&dbg=0&m=n&t=n was not found on this server.

Location

When searching from the viewer, the location of the avatar is passed along to the Search API. This is optional. The coordinates do not appear to have any effect on the search results. They have the potential to show results based on your current location, (such as displaying nearby snapshots on Snapzilla[dead link][1]). The following parameters are used.

Parameter Data type Description
Region string Region name where avatar is located.
X integer Position of avatar in region coordinates on X axis
Y integer Position of avatar in region coordinates on Y axis
Z integer Position of avatar in region coordinates on Z axis 
https://search.secondlife.com/client_search.php?q=hippo&region=Here&x=128&y=128&z=36

Mature Content

By default, the Search API blocks mature content in its results. A binary parameter "mat" may be specified to what mature content. To have PG content, you add "1" meaning "PG". To allow mature content, you add "2" meaning "Matrue". To allow adult content, you add "4" meaning "Adult".

Parameter URI
PG Only http://search.secondlife.com/client_search.php?q=hippo&mat=1
PG & Mature http://search.secondlife.com/client_search.php?q=hippo&mat=3
PG & Mature & Adult http://search.secondlife.com/client_search.php?q=hippo&mat=7
Mature Only http://search.secondlife.com/client_search.php?q=hippo&mat=2
Mature & Adult http://search.secondlife.com/client_search.php?q=hippo&mat=6
Adult Only http://search.secondlife.com/client_search.php?q=hippo&mat=4


Mature Content Old

By default, the Search API blocks mature content in its results. A parameter m may be specified to allow or block mature content. To allow mature content, the value should be y meaning Yes. Blocking mature content will be n for No.

In addition, you can still filter out mature results by appending a flag to the query +-SLS_MATURE.

Parameter URI
PG Only http://search.secondlife.com/client_search.php?q=hippo&m=n
PG & Mature http://search.secondlife.com/client_search.php?q=hippo&m=y
PG Only http://search.secondlife.com/client_search.php?q=hippo%20+-SLS_MATURE
Mature Only http://search.secondlife.com/client_search.php?q=hippo%20+SLS_MATURE&m=y

Note: %20 represents a URL-encoded space character

Category

To limit the results to specific categories, the s parameter is used. This value defaults to All, but can also be set to one of the following: All, Events, Groups, People, Places, Wiki.

Category URI
All categories http://search.secondlife.com/client_search.php?q=hippo&s=All
Events http://search.secondlife.com/client_search.php?q=hippo&s=Events
Groups http://search.secondlife.com/client_search.php?q=hippo&s=Groups
People http://search.secondlife.com/client_search.php?q=hippo&s=People
Places http://search.secondlife.com/client_search.php?q=hippo&s=Places
Wiki http://search.secondlife.com/client_search.php?q=hippo&s=Wiki

Language

There are a few parameters that identify the language to display the results in. Although the searchable content provided by the residents is not translated, portions of the page are. This includes headers and informational text (such as the number of matching results).

The language (also known as locale or culture) specified for the viewers user interface is passed along in the lang parameter. These are ISO 639-1 Values for language, and an optional country code (ISO 3166). Not all languages are supported. Some languages are in beta.

Language Code URI
Spanish es http://search.secondlife.com/client_search.php?q=hippo&lang=es
United States English en-us http://search.secondlife.com/client_search.php?q=hippo&lang=en-us


In addition, another attribute appears in queries when navigating between pages using a standard web browser. It is assumed that this may represent the same value, but only provides the language code without the country code. 

http://search.secondlife.com/client_search.php?q=hippo&hl=en

Output

At the moment[when?], it appears that there are two fields that deal with output. The first "output" is probably telling the search appliance to return "Wikipedia logo" XML formatted data. The client parameter is then transforming the results into a different format. Since the client is raw_xml_frontend, it is probably not doing any transformation at all and sending us the exact output from the Google Appliance.

Parameter Value Notes
output xml_no_dtd XML results from appliance. output format.
client raw_xml_frontend transform into XML results for user agent 
http://search.secondlife.com/client_search.php?q=hippo&output=xml_no_dtd&client=raw_xml_frontend

Tags

Tags allow content to be flagged with special attributes. Tags are used in the query of search results ({{{1}}})

Tag Description
SLS_MATURE Page contains mature content

Miscellaneous Information

There are many other parameters that show up when observing the communications of the Search API. The following properties have been seen, but not identified properly.

Parameter Default Value Notes / Guesses
t n Guessing teen grid. Teen = Y, Adult = N. Without proper session, defaults to N
session 00000000-0000-0000-0000-000000000000 This value changes each time that the SL Viewer logs into the main grid. When querying against the appliance with a standard browser and navigating pages, the value is an empty Guid.
site default_collection
lr (blank)
access p authorization to crawl and index secured content ...?
sort date:D:L:d1 Apparently this "phrase" is all over the net in regard to the default sorting for Google Search Appliance. Guessing it is sorting by date. Other option is relevance?
dbg 0 Debug Flag?
filter 0
sa N
ie UTF-8 input encoding

Headers

The following headers have been captured while querying against the Google Search Appliance with an in-world viewer. 

Host: search.secondlife.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; chrome://navigator/locale/navigator.properties; rv:1.8.1.13) 
 Gecko/20080314 SecondLife/1.20.15.92456 (Second Life Release; silver skin)
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Encoding: gzip,deflate
Keep-Alive: 300
Connection: keep-alive
Cookie: __utmz=########.##########.#.#.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none);
  __utma=########.##################.##########.##########.##########.###; __utmc=########

Note: Line feeds added to very long lines above for readability.

Work-around for LSL

It should be noted that LSL scripts using llHTTPRequest method cannot query against the Google appliance directly. Linden Lab has blocked all access due to security reasons of requests from machines on the local network. To work around this, many people use proxy services. One popular service is offered by Google. It is originally intended to make content available for mobile phones. For an example, see the Proxy Search Form.

Example of proxy search URL 

https://www.google.com/gwt/n?u=http%3A%2F%2Fsearch.secondlife.com%2Fclient_search.php%3Fq%3Dhippo&_gwt_noimg=1

Search Engine Optimization

Search Engine Optimization (SEO) is the method of fine tuning the name and description of content and links to further promote the position of a listing based on "Wikipedia logo" relevance on the search results.

Groups

Regions

Regions will always show up before parcel listings when their name contains a keyword that the end-user had searched for.

Parcels

The search engine searches for parcel name and description only if the parcel had been marked to show up in search results (30 L$ per week).

Vector Description Improvement
Date This is when the parcel was last purchased, transferred, subdivided, or joined. Do not sell, transfer, join, or divide your parcel.
Title Describes the parcel Make clear and concise
Description Describe your parcel. Add a few relevant keywords within the content or at the end of the description.
Parcel Object Name Make descriptive of what objects are found on the parcel, products sold, etc. Add a few relevant keywords within the name. Do not use duplicate named objects. Mark to show in search results.
Picks This counts as an incoming link. Paying/Verified residents have better weight on rankings. Encourage others to add your parcel to their picks tab on the profile. (See picks camping)
Classifieds This counts as an incoming link. Keywords matching in a classified as well as a higher price increase the ranking. Find a keyword that will get you into the top 10 classifieds for an affordable price.
Foot Traffic This counts as an incoming link from a hidden "top #" page. For every 10 minutes that individual avatars spend at your parcel, foot traffic is increased by 1. Host popular events. Also see camping chairs and bots (although frowned upon).
Mature If parcel or any parcel object is flagged as mature, then parcel will only be listed in mature results. Provide only PG content to reach larger audience.

Events

Residents

Residents must enable their profile to show in search results. When a resident name matches, an additional link directly to the profile is displayed regardless if the resident marked their profile to show up in search results.

Parcel Objects

All objects with the default name of "object" are ignored by search results.

The object must be rezzed on a listed parcel in order to be searched. An object within the contents of a vendor will not be searched.

Only the root prim of an object will be searched. Objects made of many linked prims will not index each prim.

Classifieds

References

Notes

  1. ↑ On VirtualVerse in November 2023, Cristiano mentioned that he might not be able to recover Snapzilla.
Retrieved from "https://wiki.secondlife.com/w/index.php?title=Search_API&oldid=1216285"