Difference between revisions of "User:SuzannaLinn Resident/ScriptingClasses/Reading Notecards"

From Second Life Wiki
Jump to navigation Jump to search
(Created page with "= Reading Notecards = == Monday class == === How the information is stored in SL === Today we will use something that we haven't seen in my classes yet: asynchronous functi...")
 
 
Line 356: Line 356:
Could we read only parts of the text?
Could we read only parts of the text?
* We are reading lines from 0 to the end, we can read only some lines if we know what numbers of lines we want.
* We are reading lines from 0 to the end, we can read only some lines if we know what numbers of lines we want.
== Wednesday class ==
=== Reading a notecard with llGetNotecardLineSync ===
We studied on Monday that notecards are stored in the central database.
We need to use asynchronous functions to read them, line by line, and we receive the text of the lines in the event dataserver.
This is the traditional way to read notecards, that we have been using since the old times.
Early this year a new function was added to LSL to read notecards faster:
* llGetNotecardLineSync( notecardName, notecardLine)
It has the same parameters than llGetNotecardLine(), the function that we used on Monday.
But the new function reads the lines from the region database, not from the central database, and we get the text of the line in return, not a request id.
llGetNotecardLineSync() is a synchronous function, as its name says.
But there is a problem... how can llGetNotecardLineSync() read the notecards from the region if they are stored in the central database?
Regions has some memory reserved to store notecards, as a cache memory.
And we need to get our notecard stored in the region before using llGetNotecardLineSync().
To store the notecard in the region we need to call an asynchronous function on notecards, like llGetNumberOfNotecardLines() or llGetNotecardLine().
The memory in the region to store notecards is limited. When more notecards are stored, the older ones are removed from the region.
So, if other objects in the region are reading notecards, our notecard could be removed while we are reading it.
If llGetNotecardLineSync() doesn't find the notecard stored in the region, it returns the constant NAK.
We need to look for NAK in each answer, and if it happens, use the asynchronous llGetNotecardLine() instead. We will get the text of the line in the event dataserver, and the notecards stored in the region again.
Please rezz the "Object 3 Reading Notecards sync" that is in the Class Materials folder.
Let's look to the script.
In touch_start, lines 15-21, we look for the first notecard in the contents.
If there is one we read the first line of the notecard in the asynchronous way, as we did on Monday.
With this we get the first line and we get the notecard stored in the region.
The new function is in the event dataserver, lines 23-39.
First we check if it is the answer to our request, comparing the parameter "request" with our variable "requestLineId", in line 24.
The reading happens in the loop    do ... while  in lines 25-34.
do ... while is like a loop while, but with the condition at the end. This loop is always executed at list once.
We use this loop because we want to process the data before checking the condition.
We start checking is the data is EOF, meaning that we are trying to read a line after the end of a notecard.
If is not EOF, we have the text of the line in data.
We add the text to the list "notecard" and we increase the number of line.
Now we read the next line using llGetNotecardLineSync() and we get the answer immediately.
If should be the text of the line, but it could happen that in meanwhile our notecard has been removed from the memory of the region.
In line 30, we check is the answer is NAK.
If it is NAK we go back to the asynchronous method, requesting the line with llGetNotecardLine().
This request will trigger another event dataserver, so we will not do anything else in the current event.
Let's look at the while in line 34. It handles different situations.
* In case of EOF, we have finishing reading the notecard, we finish the loop.
* In case of NAK, our notecard is not in the region, we can't use the sync function, and we have requested, in line 31, the line with the async function.
* In case of NAK, we are finishing the loop too, but the event will be triggered again when we receive the requested line, and the loop will go on again, from the line where it was, which is in the global variable notecardLine.
* If it is not EOF neither NAK, it means that we have read the line with the sync function, and we stay in the loop to read the next line.
If we have finished the do ... while with EOF, lines 35-37, we call list Notecard().
If we have finishied the do ... while with NAK, we do nothing, the loop will restart with the next line when the event is triggered again when we receive the requested line.
listNotecad() is in lines 5-11.
We loop on the lines of the notecard to say them.
Touch the sphere to see it working.
=== Finding text in a notecard with llFindNotecardTextSync ===
This is a vey new function, added some weeks ago.
llFindNotecardTextSync() looks for a text in a notecard and returns a list with the different lines and positions where the text appears in the notecard.
As it name says, it's a synchronous function. So it needs that the notecard is stored in the region before using it.
If has five parameters:
* the name of the notecard
* the text that we want to look for
* usually 0, we can use another value to discard some matches, for instance 3 will start with the 4th match.
* how many matches we want to get, the maximum is 64.
* and empty list, it's reserved to add other options in the future
So we call the function like this:
* lFindNotecardTextSync(  notecardName,  text,  0,  64,  []  );
It returns a strided list, with:
* Row
* Column
* Length
If it finds 64 matches, the maximum, the list will have 192 items.
Please rezz the "Object 4 Finding Text Sync" that is in the Class Materials folder.
Let's look at the script.
In lines 2-5, we have contants for the stride and the items in the stride.
This way we don't need to remember them, and the script is more clear using names instead of numbers.
In touch_start, lines 27-30, we call our function findText() with "or", which is a text that is repeated several times in the example notecard.
All the process is done in findText(), lines 7-23.
We have a  do ... while  here too, lines 11-16.
We start trying the sync function in line 12.
Usually we will get a NAK the first time. We check for it in line 13, and call an asynchronous function to get the notecard stored in the region.
We don't need the number of lines, so we don't store the request id that it returns, and we don't have an event dataserver.
When the answer to llGetNumberOfNotecardLines() arrives, it will be discarded, because there is no event dataserver to trigger.
In line 16, if it has been a NAK, we loop again, and this time we will get the answer.
Using  found == [NAK]  is "wrong", but it works well.
The operator == with two lists only compares the length of the lists, not their contents.
It works because if it is a NAK, the length is 1, otherwise the length is never 1, but 0, or 3, or 6...
In lines 17-22 we say the information, in what lines and positions "or" has been found.
Touch the sphere to see it working.
=== Both together: reading poems ===
Now we are going to see a more practical use of the llFindNotecardTextSync(), combined with llGetNotecardLineSync().
The idea of the function to find text is to allow us to have plenty of read only data stored in notecards, without need to read the full notecards in memory (or in the linkset data, that we will study next week).
For instance, long configuration notecards, where we can look for the section that we want, and read only the lines in this section.
Or a notecard with many usernames, where we can look if a name is there or not, without reading any line.
Or a book, to read only a chapter. Or a dictionary, to read only the definition of a word.
In this example we have a notecard with some poems, and we will read one, chosen by the owner of the object.
Please rezz the "Object 5 Reading Poems" that is in the Class Materials folder.
Touch the sphere to get a list of the poems. Say the number of poem that you want in channel 1, like  /1 5  and it says the poem.
Look at the notecard "Poem" to see how it is organized. All poems written by ChatGPT.
First we have a line with <LIST>, the titles of the poems, and another line with <LIST>.
We will look for the text <LIST> to find the titles, which are in the lines between the two lines <LIST>.
Below we have all the poems, starting with its title between <> and ending with again with the title.
To read a poem we will look for <title_of_the_poem>, and again the lines of the poem will be between these two lines.
Let's look at the script.
isList, in line 7, is a TRUE/FALSE variable. We use it to know if we are reading the list of poems or a poem.
isReading, in line, is also a TRUE/FALSE variable. We set it to TRUE when we are reading a notecard, and we check it in the touch_event, to avoid that the owner can touch again while we are reading.
We look for the text in findText(), lines 17-28, as we did in the previous script.
In lines 25-26, we store the first and last line of the poem in global variables.
We read the list of poems in readLines(), lines 30-43, with the same process than the previous script.
notecardLine already has the line to start, and we read until the line to end.
We say the list of poems, or the poem, in notecardReady(), lines 45-61, that is called after finishing to read the notecard.
In touch_start, lines 71-80, we start reading the list of poems.
In listen, 82-96, we start reading the poem chosen by the owner.
And in dataserver, lines 98-110, we receive the asynchronous request and call again readLines() to read in sync, or notecardReady() if we receive an EOF.
=== Answers to questions ===
Are the synchronous functions faster because they are coming from the region as compared to the central database?
* Yes, that is the reason why they are  faster.
Does llGetNotecardLineSync() tell the central server to send the notecard to the sim server?
* No, we have to make the request, using an asynchronous function: llGetNotecardLine() or llGetNumberOfNotecardLines().
Can we rely on an entire notecard being present just because a part of it is?
* We cannot rely on an entire notecard being present.
* We need to check for NAK each time that we read a line with llGetNotecardLineSync().
So the safest way would be asynchronous and the fastest synchronous?
* No, the best way is try Sync and fall back to Async when it fails.
* Sync is the faster way, and its also safe if we script it properly:
** When we get a NAK we request the line Async and then the notecard will be stored in the region, and we will go on with Sync.
* We will be mostly using the Sync version, but we need to know the Async version, because we need to use Async requests too.
What size is the storage for notecards in the region?
* LL recently changed the caching protocol. For over 14 years, it used to store 45 notecards regardless of size. Now it stores a set size in bytes. (48 * 65536) + 10 or 3145738 bytes, so chances of a notecard getting removed while reading from the cache are significantly reduced, but not impossible.
* Caching didn't come with the llGetNotecardLineSync() function. It's been around for years, but didn't originally exist when the async functions first came. It was added a little later to prevent them from always contacting the dataserver for every line read.
* The async version is slower than the sync version only because the request has to go through the request and return event process.
Is the chunk size such that an entire notecard will be removed or retained as a unit?
* Yes, an entire notecard's data is removed upon cycling, we won't be able to get any line via sync.
* But we cannot trust that all the notecard is there, because it could be removed while we are reading it.
* Our code is basically unchanged because it is always possible for the notecard to be decached between reads.
A notecard is stored in the object, the region and the data server?
* A notecard is stored in the central database, accessible to Async funtion and dataserver.
* The notecard is temporarily copied to the region for a faster access.
* In the object is stored the uuid of the notecard, not a copy of it, the notecard's contents are never stored in the object.
Is the object storage part of the sim storage?
* No, the objects has its own storage, but it only stores uuid's, not the items themselves. The same happens with our inventory, it only has uuid's.
** Object Inventory exists in the Sim only as a collection of named references / UUIDs that point to the actual item's data in the inventory servers.
** Nothing in our inventory, or an objects inventory has presence in the sim, until it is rezzed or other reference causes it to be pulled in.
Do we try to get the notecard on the sim server first to check if there are any lines?
* We request the first line Async, and with this we also get the notecard stored in the region, next lines we will try to read with Sync, if there aren't any lines we will get an EOF.
Do we store the lines in a list?
* Yes, we store all the notecard in a list in the script.
What does it mean NAK?
* Negative AcKowledge.
* ACK and NAK are terms for complementary responses to a request in general.  ACK means got the request and will fulfill it.  NAK means got the request and won't/can't fulfill it.
* Only NAK is used in LSL scripts.
Could we combine two or more lines to print out at a time instead of one at time?
* Yes, up to 1024 bytes, all llSay functions cut the text to 1024 bytes, some characters take more than 1 byte.
** If a multi-byte char is truncated, it is padded with a "?" per truncated byte.
Is there a function that takes a string and returns the byte usage?
* No, there is no ll function for it, we can use this user function:
<pre>
integer UTF8ByteSize(string str) {
    return ((3 * llSubStringIndex(llStringToBase64(str) + "=", "=")) >> 2);
}
</pre>
Does llSay uses UTF-8?
* LSL script memory uses UTF-16, all characters take at least 2 bytes, some take  4.
* llSay functions use UTF-8, all strings in chat/comm are UTF-8, the usual characters take 1 byte, but some characters take 2, 3 or 4 bytes.
* Linkset Data uses UTF-8.
What is the maximum size of a notecard?
* The maximum size of a notecard is 64k
* Lines have a maximum of 1024 characters
* We could save lines longer than 1024 in a notecard, but then we couldn't read them, reading lines are cut to 1024.
Does it matter if we use capitals or not to look for the text?
* Yes, it will only find the text if it is written in the same lettercase.
* llFindNotecardTextSync(), instead of text, can use a regex pattern or regular expression, with wildcards and commands, and this way we can search in a case insensitive way.
What are the parameters in llFindNotecardTextSync()?
* llFindNotecardTextSync( notecardName, text, 0, 2, [] )
** The name of the notecard.
** The text that we are looking for.
** 0 means starting with the first match.
** 2 means returning two matches (the two first ones), because we are only looking for the <> label before and after the poem.
** [] is an empty list for a parameter that is not used, reserved for future versions of the function.
Why we do this?
* if ( data == "" ) data = " ";
** When the data is empty "", we change it to a white space " ".
** We do it because the llSay() functions doesn't say empty lines, if we want them to write a blank line, we need to say a space.
** The same happens with floating texts, empty lines are ignored.

Latest revision as of 10:27, 22 November 2024

Reading Notecards

Monday class

How the information is stored in SL

Today we will use something that we haven't seen in my classes yet: asynchronous function calls.

Before explaining what is that, let's see how information is organized in secondlife.

Please rezz the "Object 1a Saying names" that is in the Class Materials and open the "Script 1a saying names".

The script stats with a list of uuid's of avatars. There is a function that loops in this list and says the displaynames of the uuid's, called from the event touch.

Touch the box to see it working.

There is a strange thing here. It's not saying the name of the second uuid in the list, which is the uuid of my alt.

It happens that llGetDisplayName() only works with uuid's of avatars that are in the region. If the avatar is in another region or offline the functions returns "".

The same happens with llGetUsername() and llKey2Name(). Also with llGetAgentSize(), llGetAgentInfo() and llGetObjectDetails().

Let's see why.

There is a central database with all the information about objects, avatars, script, notecards, textures, everything. All is stored there: when we save a script, or a notecard, or build a new object. It's a huge database with all SL in it.

And each region has its own database with the information that the region needs: objects that are in the region and people who is currently in the region.

When we teleport, our avatar, the attachments and HUDs that we are wearing, the scripts in these attachments and HUDs, are deleted from the previous region and loaded into the new region.

The region database is fast to access, and many of the info that we need is there. We can use a function, for instance llGetDisplayName() or llGetObjectDetails(), and we receive the data inmediately.

But the central database is slow to access. And our script would have to wait for several 1/10 of second, which is a long time from a script point of view.


Synchronous and asynchronous functions

Here is where the asynchronous functions appear.

The asynchronous function to get the display name is:

  • llRequestDisplayName( avatarId )

with the uuid of the avatar as parameter.

When we call one of these asynchronous functions, the function doesn't return us the info that we want but sends a request for it.

Internally, our request is sent to the central database. And our script will go on executing the next commands.

But we want our info. Where is it?

We will receive it in the event "dataserver", that will be triggered when the data arrives from the central database.

The parameters of the event are:

  • dataserver(key queryid, string data)

later we will see what is "queryid"

data is the info requested, always comes as a string, we will typecast it to the right type.

Why this thing with the dataserver event and not just sending us the data?

Because meanwhile the data arrives, our script can be doing other things.

The functions that we have been using until now are all "synchronous", which means "same time", because we get the returned value at the same time.

"asynchronous" means "different time", because the value will arrive later.

Imagine that, in RL, we need to ask something to a friend.

We take the phone and call our friend, if the friend answers we get the information immediately. This is synchronous.

If our friend doesn't answer the phone, we send a message. And we don't wait for the answer, we go on with our lives.

After some time, the phone will warn us that we have a message from our friend. This is asynchronous.

We need to do an important rewritting in our script.

Our function to say the names would not work with llRequestDisplayName().

After calling llRequestDisplayName() we don't have any name to say yet.

We need to split the function in two: the part that requests the name (a function that we call getName() ), and the part that says the name (that we call sayName() ).

It works like this:

  • getName() calls llRequestDisplayName() that sends the request to the central database
  • the event dataserver is triggered and there we call sayName()
  • sayName() says the name

And now the variables are global, because we are using them in several functions or events.

We do this process for each uuid in the list of people.

sayName() calls again getName() to repeat the process for the next uuid in the list, until we arrive to the end of list.

We don't send all the requests together because in the event dataserver would be difficult to know which name goes with which uuid.

Because perhaps we will not receive the data in the same order that we have requested it.

Each request, internally, is an internet connexion between the computer that is managing the region and the computers in the central database. It can take more or less depending on traffic. And perhaps our second request could go faster and arrived before our first request.

Please rezz the "Object 1b Saying names with async" that is in the Class Materials and open the "Script 1b saying names with async".

Touch the object to see that now it says all the names.

Let's look at the script.

In lines 4-8 we have the variables that now has to be global because we are using them in getName(), setName() and the event dataserver.

totalPeople is the total of uuids in the list.

currentPerson is the index in the list that we are working on.

personId is the uuid that we are working on.

And displayName to store the name when we get it.

The process starts in touch_start, lines 30-34.

We initialize totalPeople and currentPerson and call getName() to start the process of the first uuid.

getName() is in lines 10-18.

We try to get the display name in the usual way, in case that the avatar is in the region.

If we get the name, we call sayName() to say it, in line 14.

If not, we request the name in line 16.

We could do other things, because the event dataserver with the answer will take some time to arrive.

But we don't have anything else to do, so the scripts stops and gets idle.

After a long time (from the script point of view), about half a second, the event dataserver is triggered.

dataserver is in lines 36-39.

The information is in the parameter "data".

We set displayName and call setName() to say it.

sayName() is in lines 20-26. It's called by getName() when the avatar is in the region, and by dataserver when not.

We say the name, increase the index, and, if there are more uuid's, we call getName() to repeat all the process for the next one.


More on asynchronous functions

Now let's improve our script. We will say the display name and also the username.

The asynchronous function to get the username is llRequestUsername(). We will request for both the display name and the username.

But, how do we know in the event dataserver if we are receiving the display name or we are receiving the username?

The llRequest*** functions returns a value.

We get a value of type "key", a UUID, unique for this call to this function. Each time that we call the a llRequest*** function we will receive a different value. And we will store this value in a variable named, for instance, myRequestQueryId.

In the event dataserver we receive in the parameter queryid this same UUID that the function returned to us when we called it, and that we stored in the variable myRequestQueryId.

We can be sending other requests, where we will get different UUID's as query id's.

And in the event dataserver we will compare our "id" variables with the parameter queryid to know what information we are receiving.

We compare the dataserver queryid with our stored id in the dataserver event:

	dataserver( key queryid, string data ) {
    		if ( queryid == myRequestQueryId) {
        			llSay( 0, data);  // data has the info that we requested, use it wisely
   	 	}
	}

Please rezz the "Object 1c Saying names with two async" that is in the Class Materials and open the "Script 1c saying names with two async".

Touch the object to see that now it says all the names an usernames.

Let's look at the script.

The structure of the script is the same and the calls work in the same way.

In lines 8-9, we have added the global variables displayNameQueryId and usernameQueryId to store the uuid's of the requests.

And in line 10 the variable username to store the username.

In getName(), lines 14-26, we request for the display name and the username, in lines 23-24, storing the request uuid's returned by the functions.

In the event dataserver, lines 44-53, we compare the parameter queryid with our id variables in lines 45 and 47, to store the information.

In line 50 we check if we have got both names to call sayName().

And in sayName(), lines 28-34, we add the username to the message.


Reading a notecard

And it happens that the notecards are stored in the central database. So we will need to use an asynchronous function to read them. And several times, because we can't read a notecard with one function, we have to read it line by line.

Please rezz the "Object 2 Reading notecard" that is in the Class Materials folder and open the "Script 2 reading notecard" that is in the same folder.

In this example we are only reading the notecard. We don't have anything else to do while we wait for the central database to answer. We would prefer to have a function that returns us the info, and to have our script waiting for it. But no way to do it, so let's get asynchronous :)

Our strategy in this example is:

  • We ask for the first line of the notecard, and stop here.
  • We will receive the event dataserver with the first line, we store this line, and we ask for the next one.
  • We will reveive the next line, store, ask for next... until we arrive to the end of the notecard.

Let's see how it looks in the script:

In line 2 there is the variable that we will use to store the ID of the query: key notecardQueryId;

And in line 4 the list where we will keep the notecard text: list notecardText;

The function readNotecard() in lines 7-17, that we call in the event state_entry, looks for the first notecard in the object contents and asks for the first line. If there isn't any notecard it says a message. We have already seen the inventory functions in a previous class.

The function to ask for a line in the notecard is llGetNotecardLine() and its parameters are the name of the notecard and the line number that we want, starting with line 0. We assign the return value, the query id, to notecardQueryId.

We use llGetNotecardLine() in line 13 to get the first line. And nothing else until we receive the event dataserver.

Let's go to line 34 to see the event dataserver.

In line 35 we compare our variable notecardQueryId with the parameter queryid. It wouldn't be necessary in this example. We request info one by one and there is only a query going on. But it's a good practice to do it. We will see an example of two queries in our Info Board.

When we ask for a notecard line that doesn't exist it returns the constant EOF (meaning "End Of File"). So we check for it in line 36.

If it is not EOF we add the line to our list, increase the line counter, and request the next line. When EOF we go to say the notecard on lines 19-26.

As you probably already know, lsl script is not a perfect language.

The issue with notecards is that they can't be read if they have items included, like landmarks, textures... You can try it later, it returns 0 lines.

And it's not possible to write a notecard with a script, only to read them.

Now drop a notecard (without any items included!) in your box and see how it says its text to you. If it's a long notecard it can take several seconds.


Answers to questions

Do we get the information from the llFunctions from a sim server ?

  • All synchonous llFunctions get their information from the sim the script is in. They can't learn anything that the sim itself doesn't know.
  • Communcation with the data server where the database is kept is asynchronous, meaning the script has to send a request to the remote server and wait for a reply that will come some indefinite time later. In the meantime, while it waits, the script can go on to do other stuff.

Are our textures are stored on the central database?

  • Yes, the textures and everything else.

Is because of the copy of attachments and scripts from one region to another that it takes a while for our clothes to rez when we TP?

  • Yes, because our attachments and scripts are loading in the new region.
  • In that case most of the communication is between the sending and receiving sims though, not the central database.

Woulld that be affected by a huge inventory?

  • No, it depends on what we are wearing, not on the size of our inventory

If we delete a texture other people can still use it, Is a texture stored somehow redundant?

  • A texture is stored only once, with its id, in the central database.
  • This id is also in the object that has the texture in, and in the inventories of the people who have the texture.
  • The texture is not deleted while its id is somewhere, in some object, or in some inventory.
  • Our inventory, internally, is a list of id's to the object, texture, everything that is stored in the central database.
    • If we build a new object, and then delete it, also from the trash folder, the object is deleted.
    • But if we have given it to a friend before deleting it, the object is not deleted, because our friend has an id of the object.
    • The object will be deleted if our friend deletes it too.
  • Internally, in the central database, with each object, texture, etc, there is a count of how many active id's to it there are, if this count goes down to 0, the item is deleted.

So each object has its own memory?

  • An object has a list of id's of its contents, not the contents themselves.

So synchronous will come from direct communication with the same region we are in and we don't need anything in between? and asynchronous will request the info that is not immediately available?

  • Yes, right

Will the responses to our requests arrive in the same order?

  • No. That, in general is a feature of asynchonous commmunication. Unless you wait for a response each time, you cannot depend on replies arriving in the order requests were sent. Much of the time they will be in order, but if you depend on it, it will reach out and bite you. For instance SL's dataservier is actually most likely several servers working together, with a load manager queuing requests to the server with the least number of outstanding requests. But some requests may take more time than others, so two requests arriving close together may be queued to different machines and it becomes a race to see which one is handled first.

Will llList2Key() give us the username, and llGetDisplayName() will transform that username to the display name, but both use the personId?

  • No, llList2Key() get one of the uuid from the list of people and stores it in personId, then we use personId with llGetDisplayName().
  • Don't confuse llList2Key() with llKey2Name() that gives us the legacy name.

So the first time you click the box, it may take some time to respond. But after that, it responds immediately.

  • It takes a few time for each avatar in the list people who is not here, about half a second.
  • Changing the list people with a long lists of avatars (who are not in the region), we will see their names being said one every half a second or so.

Where, if there are more uuids, do we repeat the process?

  • We repeat the process in sayName(), lines 22-25.
  • We add one to the index (currentPerson) and check it with the list length (totalPeople).
  • If we are not at the end of the list we call getName() again in line 24.

So if totalPeople in the list is bigger than currentPerson we call getName() over and over?

  • Yes, we repeat the sequence getName - dataserver - sayName for each uuid in the list.

Are all the aynchronous requests processed?

  • Yes, all the requests are answered, the time of answer depends on how busy are the servers in the central database, usually is tenths of second.

Do we have to request the names separately?

  • Yes, llRequestDisplayName() only allows us to ask for the name of one avatar, so we need to request them one by one.

Does other scripts receive the same response in their events dataserver?

  • Since the dataserver event occurs in all scripts in the same prim, if there are unrelated scripts in te prim, it is possible to receive dataserver events that aren't for our script. These can simply be ignored. It is also possible to have the requests made In one or more scripts and have the responses handled in another script working in cooperation with them.

Could we get an answer that should be for another object?

  • No, the event response is to the same prim, not even another link in the same linkset will see our events.

What is &&?

  • && means "and".
  • || means "or".
  • They are logical operators to check several conditions together, for instance:
    • if (displayName != "" && username != "")
  • The "if" is true if both displayname and username are not empty.

Why do we use && instead of &? Do they have different meanings in scripting?

  • Yes, they are different.
  • && is the logical "and", that we use with conditions that are TRUE or FALSE.
  • & is the bitwise "and", that we use with two integers that are bit values.
  • It makes sense if you look at the bits. For the logical operations such as && all that matters is zero and not zero. For the bitwise operations, each bit position is compared independently of the others, so in binary 1 is 01, 2 is 10, and 3 is 11: 01 & 10 == 00, 01 & 11 == 01.

Can you give us an example of &?

  • Let's look at it in the event changed:
changed(integer change) {
        if (change & (CHANGED_OWNER | CHANGED_INVENTORY))
            llResetScript();
}
  • the parameter "change" is a set of bits, each possible change has a different bit assigned:
    • CHANGED_OWNER has a bit value of 0000 0001.
    • CHANGED_INVENTORY is 1000 0000.
  • | is an "or" bit by bit. If one of the two bits is 1 the result is 1. If the two bits are 0 the result is 0.
        0000 0001        CHANGED_OWNER
      | 1000 0000        CHANGED_INVENTORY
-------------------------------------------------------------
        1000 0001        ( CHANGED_OWNER | CHANGED_INVENTORY)
  • & is an "and" bit by bit. If the two bits are 1 the result is 1. If one of the two bits is 0 the result is 0.
        1000 0001        ( CHANGED_OWNER | CHANGED_INVENTORY)
      & 1000 0000        change, if the inventory has changed
---------------------------------------------------------------------
        1000 0000        change & (CHANGED_OWNER | CHANGED_INVENTORY)
    • the if condition is TRUE because the result is a non-zero value.
  • in case of another change, for instance CHANGED_LINK:
        1000 0001        ( CHANGED_OWNER | CHANGED_INVENTORY)
      & 0010 0000        change, if the linked prims have changed
---------------------------------------------------------------------
        0000 0000        change & (CHANGED_OWNER | CHANGED_INVENTORY)
    • the if condition is FALSE because the result is zero.
  • Using && and || all these operations would have a TRUE result, because && and || only check if the integer is 0 (FALSE) or not 0 (TRUE).

Does llGetNotecardLine() return an key?

  • Yes, it's an asynchronous function, and returns a key to be used in the dataserver event to compare with the parameter queryid.

Could we read only parts of the text?

  • We are reading lines from 0 to the end, we can read only some lines if we know what numbers of lines we want.


Wednesday class

Reading a notecard with llGetNotecardLineSync

We studied on Monday that notecards are stored in the central database.

We need to use asynchronous functions to read them, line by line, and we receive the text of the lines in the event dataserver.

This is the traditional way to read notecards, that we have been using since the old times.

Early this year a new function was added to LSL to read notecards faster:

  • llGetNotecardLineSync( notecardName, notecardLine)

It has the same parameters than llGetNotecardLine(), the function that we used on Monday.

But the new function reads the lines from the region database, not from the central database, and we get the text of the line in return, not a request id.

llGetNotecardLineSync() is a synchronous function, as its name says.

But there is a problem... how can llGetNotecardLineSync() read the notecards from the region if they are stored in the central database?

Regions has some memory reserved to store notecards, as a cache memory.

And we need to get our notecard stored in the region before using llGetNotecardLineSync().

To store the notecard in the region we need to call an asynchronous function on notecards, like llGetNumberOfNotecardLines() or llGetNotecardLine().

The memory in the region to store notecards is limited. When more notecards are stored, the older ones are removed from the region.

So, if other objects in the region are reading notecards, our notecard could be removed while we are reading it.

If llGetNotecardLineSync() doesn't find the notecard stored in the region, it returns the constant NAK.

We need to look for NAK in each answer, and if it happens, use the asynchronous llGetNotecardLine() instead. We will get the text of the line in the event dataserver, and the notecards stored in the region again.

Please rezz the "Object 3 Reading Notecards sync" that is in the Class Materials folder.

Let's look to the script.

In touch_start, lines 15-21, we look for the first notecard in the contents.

If there is one we read the first line of the notecard in the asynchronous way, as we did on Monday.

With this we get the first line and we get the notecard stored in the region.

The new function is in the event dataserver, lines 23-39.

First we check if it is the answer to our request, comparing the parameter "request" with our variable "requestLineId", in line 24.

The reading happens in the loop do ... while in lines 25-34.

do ... while is like a loop while, but with the condition at the end. This loop is always executed at list once.

We use this loop because we want to process the data before checking the condition.

We start checking is the data is EOF, meaning that we are trying to read a line after the end of a notecard.

If is not EOF, we have the text of the line in data.

We add the text to the list "notecard" and we increase the number of line.

Now we read the next line using llGetNotecardLineSync() and we get the answer immediately.

If should be the text of the line, but it could happen that in meanwhile our notecard has been removed from the memory of the region.

In line 30, we check is the answer is NAK.

If it is NAK we go back to the asynchronous method, requesting the line with llGetNotecardLine().

This request will trigger another event dataserver, so we will not do anything else in the current event.

Let's look at the while in line 34. It handles different situations.

  • In case of EOF, we have finishing reading the notecard, we finish the loop.
  • In case of NAK, our notecard is not in the region, we can't use the sync function, and we have requested, in line 31, the line with the async function.
  • In case of NAK, we are finishing the loop too, but the event will be triggered again when we receive the requested line, and the loop will go on again, from the line where it was, which is in the global variable notecardLine.
  • If it is not EOF neither NAK, it means that we have read the line with the sync function, and we stay in the loop to read the next line.

If we have finished the do ... while with EOF, lines 35-37, we call list Notecard().

If we have finishied the do ... while with NAK, we do nothing, the loop will restart with the next line when the event is triggered again when we receive the requested line.

listNotecad() is in lines 5-11.

We loop on the lines of the notecard to say them.

Touch the sphere to see it working.


Finding text in a notecard with llFindNotecardTextSync

This is a vey new function, added some weeks ago.

llFindNotecardTextSync() looks for a text in a notecard and returns a list with the different lines and positions where the text appears in the notecard.

As it name says, it's a synchronous function. So it needs that the notecard is stored in the region before using it.

If has five parameters:

  • the name of the notecard
  • the text that we want to look for
  • usually 0, we can use another value to discard some matches, for instance 3 will start with the 4th match.
  • how many matches we want to get, the maximum is 64.
  • and empty list, it's reserved to add other options in the future

So we call the function like this:

  • lFindNotecardTextSync( notecardName, text, 0, 64, [] );

It returns a strided list, with:

  • Row
  • Column
  • Length

If it finds 64 matches, the maximum, the list will have 192 items.

Please rezz the "Object 4 Finding Text Sync" that is in the Class Materials folder.

Let's look at the script.

In lines 2-5, we have contants for the stride and the items in the stride.

This way we don't need to remember them, and the script is more clear using names instead of numbers.

In touch_start, lines 27-30, we call our function findText() with "or", which is a text that is repeated several times in the example notecard.

All the process is done in findText(), lines 7-23.

We have a do ... while here too, lines 11-16.

We start trying the sync function in line 12.

Usually we will get a NAK the first time. We check for it in line 13, and call an asynchronous function to get the notecard stored in the region.

We don't need the number of lines, so we don't store the request id that it returns, and we don't have an event dataserver.

When the answer to llGetNumberOfNotecardLines() arrives, it will be discarded, because there is no event dataserver to trigger.

In line 16, if it has been a NAK, we loop again, and this time we will get the answer.

Using found == [NAK] is "wrong", but it works well.

The operator == with two lists only compares the length of the lists, not their contents.

It works because if it is a NAK, the length is 1, otherwise the length is never 1, but 0, or 3, or 6...

In lines 17-22 we say the information, in what lines and positions "or" has been found.

Touch the sphere to see it working.


Both together: reading poems

Now we are going to see a more practical use of the llFindNotecardTextSync(), combined with llGetNotecardLineSync().

The idea of the function to find text is to allow us to have plenty of read only data stored in notecards, without need to read the full notecards in memory (or in the linkset data, that we will study next week).

For instance, long configuration notecards, where we can look for the section that we want, and read only the lines in this section.

Or a notecard with many usernames, where we can look if a name is there or not, without reading any line.

Or a book, to read only a chapter. Or a dictionary, to read only the definition of a word.

In this example we have a notecard with some poems, and we will read one, chosen by the owner of the object.

Please rezz the "Object 5 Reading Poems" that is in the Class Materials folder.

Touch the sphere to get a list of the poems. Say the number of poem that you want in channel 1, like /1 5 and it says the poem.

Look at the notecard "Poem" to see how it is organized. All poems written by ChatGPT.

First we have a line with <LIST>, the titles of the poems, and another line with <LIST>.

We will look for the text <LIST> to find the titles, which are in the lines between the two lines <LIST>.

Below we have all the poems, starting with its title between <> and ending with again with the title.

To read a poem we will look for <title_of_the_poem>, and again the lines of the poem will be between these two lines.

Let's look at the script.

isList, in line 7, is a TRUE/FALSE variable. We use it to know if we are reading the list of poems or a poem.

isReading, in line, is also a TRUE/FALSE variable. We set it to TRUE when we are reading a notecard, and we check it in the touch_event, to avoid that the owner can touch again while we are reading.

We look for the text in findText(), lines 17-28, as we did in the previous script.

In lines 25-26, we store the first and last line of the poem in global variables.

We read the list of poems in readLines(), lines 30-43, with the same process than the previous script.

notecardLine already has the line to start, and we read until the line to end.

We say the list of poems, or the poem, in notecardReady(), lines 45-61, that is called after finishing to read the notecard.

In touch_start, lines 71-80, we start reading the list of poems.

In listen, 82-96, we start reading the poem chosen by the owner.

And in dataserver, lines 98-110, we receive the asynchronous request and call again readLines() to read in sync, or notecardReady() if we receive an EOF.


Answers to questions

Are the synchronous functions faster because they are coming from the region as compared to the central database?

  • Yes, that is the reason why they are faster.

Does llGetNotecardLineSync() tell the central server to send the notecard to the sim server?

  • No, we have to make the request, using an asynchronous function: llGetNotecardLine() or llGetNumberOfNotecardLines().

Can we rely on an entire notecard being present just because a part of it is?

  • We cannot rely on an entire notecard being present.
  • We need to check for NAK each time that we read a line with llGetNotecardLineSync().

So the safest way would be asynchronous and the fastest synchronous?

  • No, the best way is try Sync and fall back to Async when it fails.
  • Sync is the faster way, and its also safe if we script it properly:
    • When we get a NAK we request the line Async and then the notecard will be stored in the region, and we will go on with Sync.
  • We will be mostly using the Sync version, but we need to know the Async version, because we need to use Async requests too.

What size is the storage for notecards in the region?

  • LL recently changed the caching protocol. For over 14 years, it used to store 45 notecards regardless of size. Now it stores a set size in bytes. (48 * 65536) + 10 or 3145738 bytes, so chances of a notecard getting removed while reading from the cache are significantly reduced, but not impossible.
  • Caching didn't come with the llGetNotecardLineSync() function. It's been around for years, but didn't originally exist when the async functions first came. It was added a little later to prevent them from always contacting the dataserver for every line read.
  • The async version is slower than the sync version only because the request has to go through the request and return event process.

Is the chunk size such that an entire notecard will be removed or retained as a unit?

  • Yes, an entire notecard's data is removed upon cycling, we won't be able to get any line via sync.
  • But we cannot trust that all the notecard is there, because it could be removed while we are reading it.
  • Our code is basically unchanged because it is always possible for the notecard to be decached between reads.

A notecard is stored in the object, the region and the data server?

  • A notecard is stored in the central database, accessible to Async funtion and dataserver.
  • The notecard is temporarily copied to the region for a faster access.
  • In the object is stored the uuid of the notecard, not a copy of it, the notecard's contents are never stored in the object.

Is the object storage part of the sim storage?

  • No, the objects has its own storage, but it only stores uuid's, not the items themselves. The same happens with our inventory, it only has uuid's.
    • Object Inventory exists in the Sim only as a collection of named references / UUIDs that point to the actual item's data in the inventory servers.
    • Nothing in our inventory, or an objects inventory has presence in the sim, until it is rezzed or other reference causes it to be pulled in.

Do we try to get the notecard on the sim server first to check if there are any lines?

  • We request the first line Async, and with this we also get the notecard stored in the region, next lines we will try to read with Sync, if there aren't any lines we will get an EOF.

Do we store the lines in a list?

  • Yes, we store all the notecard in a list in the script.

What does it mean NAK?

  • Negative AcKowledge.
  • ACK and NAK are terms for complementary responses to a request in general. ACK means got the request and will fulfill it. NAK means got the request and won't/can't fulfill it.
  • Only NAK is used in LSL scripts.

Could we combine two or more lines to print out at a time instead of one at time?

  • Yes, up to 1024 bytes, all llSay functions cut the text to 1024 bytes, some characters take more than 1 byte.
    • If a multi-byte char is truncated, it is padded with a "?" per truncated byte.

Is there a function that takes a string and returns the byte usage?

  • No, there is no ll function for it, we can use this user function:
integer UTF8ByteSize(string str) {
    return ((3 * llSubStringIndex(llStringToBase64(str) + "=", "=")) >> 2);
}

Does llSay uses UTF-8?

  • LSL script memory uses UTF-16, all characters take at least 2 bytes, some take 4.
  • llSay functions use UTF-8, all strings in chat/comm are UTF-8, the usual characters take 1 byte, but some characters take 2, 3 or 4 bytes.
  • Linkset Data uses UTF-8.

What is the maximum size of a notecard?

  • The maximum size of a notecard is 64k
  • Lines have a maximum of 1024 characters
  • We could save lines longer than 1024 in a notecard, but then we couldn't read them, reading lines are cut to 1024.

Does it matter if we use capitals or not to look for the text?

  • Yes, it will only find the text if it is written in the same lettercase.
  • llFindNotecardTextSync(), instead of text, can use a regex pattern or regular expression, with wildcards and commands, and this way we can search in a case insensitive way.

What are the parameters in llFindNotecardTextSync()?

  • llFindNotecardTextSync( notecardName, text, 0, 2, [] )
    • The name of the notecard.
    • The text that we are looking for.
    • 0 means starting with the first match.
    • 2 means returning two matches (the two first ones), because we are only looking for the <> label before and after the poem.
    • [] is an empty list for a parameter that is not used, reserved for future versions of the function.

Why we do this?

  • if ( data == "" ) data = " ";
    • When the data is empty "", we change it to a white space " ".
    • We do it because the llSay() functions doesn't say empty lines, if we want them to write a blank line, we need to say a space.
    • The same happens with floating texts, empty lines are ignored.