StAX - Second Life Wiki

StAX

From Second Life Wiki

Jump to: navigation, search

Created by Kira Komarov of the Wizardry and Steamworks group.

Contents

Introduction

The StAX parser is an XML parser that is able to process tree-like structured data as the data gets streamed-in. One of the features of this script is that it uses a look-back as it parses the XML string. This could be adapted for a real stream, perhaps, for example, in order to listen on some channel for messages and extract the desired information. The current function is at wasStaX_GetNodeValue and its converse function wasStaX_SetNodeValue and can be used independently based on the function parameters.

One of the applications I thought this would be useful for, were the notecard configurations which are hard to standardise and everybody seems to make up their own way of reading-in data. The StAX parser is sufficient to provide that layer of standardisation and relies on very few syntactic restrictions.

The code is designed around the StAX concept. However, we do not implement all the features since we do not consider them useful in the context of Second Life.

Syntactical Restrictions

The following is a list of symbols that are forbidden from being used as data (either as nodes, or content) in the xml-file:

  • The smaller-than sign: <
  • The greater-than sign: >
  • The forward-slash: /

There are no restrictions imposed on the content disposition in the notecard itself. For example,

<xml>
    <about>
        <description>
                            This is a StaX XML stream parser, 
                            with look-back and suitable for streams.</description>
</about>
                  <usage>

    wasStaX_GetNodeValue(string xmlStream, string node)
    
    where xmlStream is a a flat string containing the xml file and,
    node is a node containing the value you wish to extract.
</usage></xml>

Which, although it is badly tabulated and improperly aligned will still parse and will produce no errors when processed with wasStaX_GetNodeValue or wasStaX_SetNodeValue.

The StAX method of parsing an XML file, relies on pushing and pop-ing nodes off a stack (hence the funny name) till the last node in the stack. At the end, the expected result is that the stack will be empty or will contain unmatched nodes, in which case, the XML file is improperly formatted.

Limitations

  1. So far, this implementation does not support attributes for tags and they may be implemented later. However, there is not much need for them at this point given Limits imposed on notecards.
  2. Currently, the parser returns the empty string if the node is not a leaf of the XML tree. This may be changed, to return the sub-tree of that node, however at this stage that is not important.

Practical Applications

wasStaX_GetNodeValue and wasStaX_SetNodeValue provide a nice way of maintaining configurations with multiple layers of nesting and in an organized format. Furthermore, with some changes to those getter and setter functions, it is possible to expand wasStaX_GetNodeValue to immediately return the value once it is found. The advantage over DOM style parsers is that StAX does not need the XML file to be complete and is thus a good solution where the next data chunk is unknown.

Based on those, a few scenarios come to mind:

  • The most trivial, reading and writing configurations from and to notecards.
  • Dynamic configurations: our Wizardry and Steamworks/Population Genetics and Selection article uses more-or-less the same concept by using a typical format for message submission. One could use StaX instead to pass parameters between components and the advantage would be a clear, unified API that leaves no room for mistake. All the message and pattern matching done by the Wizardry and Steamworks/Population Genetics and Selection Simulator is done manually every time in each script. Using an universal function, shared across all the scripts, that additionally parses and verifies the consistency of the messages is a great advantage.
  • Any hierarchical tree-like data which needs to be kept track of, can use wasStaX_GetNodeValue and wasStaX_SetNodeValue to keep track of the nodes and their position in the hierarchy.

Set-Up for Examples

  • Create a notecard called Input and copy and paste the following contents:
<xml>
    <about>
        <description>
                            This is a StaX XML stream parser, 
                            with look-back and suitable for streams.</description>
   </about>
   <usage>

    wasStaX_GetNodeValue(string xmlStream, string node)
    
    where xmlStream is a a flat string containing the xml file and,
    node is a node containing the value you wish to extract.

   </usage>
</xml>

  • Create a script with the contents from the code-section below named Code: Notecard Configuration Reader.
  • Create a new primitive and drop both the script and the Input notecard into the primitive.
  • You may need to reset the scripts in the primitive if the notecard cannot be read properly.
  • Make sure you leave an empty space after the last line in the notecard.

The expected output, should be:

Object: This is a StaX XML stream parser, with look-back and suitable for streams. 
Object: wasStaX_GetNodeValue(string xmlStream, string node) where xmlStream is a a flat string containing the xml file and, node is a node containing the value you wish to extract. 

wasStaX_GetNodeValue

string wasStaX_GetNodeValue(string xmlStream, string node) {
    list stream = llParseString2List(xmlStream, [" "], ["<", ">", "/"]);
    integer size = llGetListLength(stream);
    list StaX = [];
    string value = "";
    integer ptr = 0;
    do {
        string current = llList2String(stream, ptr);
        string lookback = llList2String(stream, ptr-1);
 
        if(current != "/" && lookback == "<") {
            StaX += current;
            jump next_tag;
        }
        if(lookback == "/") {
            StaX = llDeleteSubList(StaX, llGetListLength(StaX)-1, llGetListLength(StaX)-1);
            jump next_tag;
        }
        if(current != ">" && current != "/" && current != "<") 
            if(llList2String(StaX,llGetListLength(StaX)-1) == node)
                value += current + " ";  
@next_tag;
    } while(++ptr<size);
 
    if(llGetListLength(StaX) != 0) {
        llSay(DEBUG_CHANNEL, "The following tags may be unmatched: " + llDumpList2String(StaX, ",") + ". Please check your file.");
    }
    return value;
}

Example: Notecard Reader

Input notecard:

<xml>
    <about>
        <description>
                            This is a StaX XML stream parser, 
                            with look-back and suitable for streams.</description>
   </about>
   <usage>

    wasStaX_GetNodeValue(string xmlStream, string node)
    
    where xmlStream is a a flat string containing the xml file and,
    node is a node containing the value you wish to extract.

   </usage>
</xml>

Output:

Object: This is a StaX XML stream parser, with look-back and suitable for streams. 
Object: wasStaX_GetNodeValue(string xmlStream, string node) where xmlStream is a a flat string containing the xml file and, node is a node containing the value you wish to extract. 

Code

//////////////////////////////////////////////////////////
// [K] Kira Komarov - 2011, License: GPLv3              //
// Please see: http://www.gnu.org/licenses/gpl.html     //
// for legal details, rights of fair usage and          //
// the disclaimer and warranty conditions.              //
//////////////////////////////////////////////////////////
 
string wasStaX_GetNodeValue(string xmlStream, string node) {
    list stream = llParseString2List(xmlStream, [" "], ["<", ">", "/"]);
    integer size = llGetListLength(stream);
    list StaX = [];
    string value = "";
    integer ptr = 0;
    do {
        string current = llList2String(stream, ptr);
        string lookback = llList2String(stream, ptr-1);
 
        if(current != "/" && lookback == "<") {
            StaX += current;
            jump next_tag;
        }
        if(lookback == "/") {
            StaX = llDeleteSubList(StaX, llGetListLength(StaX)-1, llGetListLength(StaX)-1);
            jump next_tag;
        }
        if(current != ">" && current != "/" && current != "<") 
            if(llList2String(StaX,llGetListLength(StaX)-1) == node)
                value += current + " ";  
@next_tag;
    } while(++ptr<size);
 
    if(llGetListLength(StaX) != 0) {
        llSay(DEBUG_CHANNEL, "The following tags may be unmatched: " + llDumpList2String(StaX, ",") + ". Please check your file.");
    }
    return value;
}
 
key nQuery = NULL_KEY;
integer xLine = 0;
list xList = [];
//pragma inline
string xName = "Input";
 
string sCnt = "";
 
default
{
    state_entry()
    {
        integer itra;
        for(itra=0, xList=[], xLine=0; itra<llGetInventoryNumber(INVENTORY_NOTECARD); ++itra) {
            if(llGetInventoryName(INVENTORY_NOTECARD, itra) == xName)
                jump found_notecard;
        }
        llInstantMessage(llGetOwner(), "Failed to find notecard.");
        return;
@found_notecard;
        nQuery = llGetNotecardLine(xName, xLine);
    }
 
    dataserver(key id, string data) {
        if(id != nQuery) return;
        if(data == EOF) {
            llOwnerSay(wasStaX_GetNodeValue(sCnt, "description"));
            llOwnerSay(wasStaX_GetNodeValue(sCnt, "usage"));
            return;
        };
        if(data == "") jump next_line;
        sCnt += data;
@next_line;
        nQuery = llGetNotecardLine(xName, ++xLine);
    }
    changed(integer change) {
        if(change & CHANGED_INVENTORY) llResetScript();
    }
}

wasStaX_SetNodeValue

/////////////////////////////////////////////////////////
// [K] Kira Komarov - 2011, License: GPLv3              //
// Please see: http://www.gnu.org/licenses/gpl.html     //
// for legal details, rights of fair usage and          //
// the disclaimer and warranty conditions.              //
//////////////////////////////////////////////////////////
 
string wasStaX_SetNodeValue(string xmlStream, string node, string value) {
    list stream = llParseString2List(xmlStream, [""], ["<", ">", "/"]);
    integer size = llGetListLength(stream);
    list StaX = [];
    integer ptr = 0;
    integer set = 0;
    do {
        string current = llList2String(stream, ptr);
        string lookback = llList2String(stream, ptr-1);
 
        if(current != "/" && lookback == "<") {
            StaX += current;
            jump next_tag;
        }
        if(lookback == "/") {
            StaX = llDeleteSubList(StaX, llGetListLength(StaX)-1, llGetListLength(StaX)-1);
            jump next_tag;
        }
        if(current != ">" && current != "/" && current != "<") 
            if(llList2String(StaX,llGetListLength(StaX)-1) == node) {
                if(!set) {
                    stream = llListReplaceList(stream, (list)value, ptr, ptr);
                    set = 1;
                    jump next_tag;
                }
                stream = llListReplaceList(stream, (list)"", ptr, ptr);
            }
@next_tag;
    } while(++ptr<size);
 
    if(llGetListLength(StaX) != 0) {
        llSay(DEBUG_CHANNEL, "The following tags may be unmatched: " + llDumpList2String(StaX, ",") + ". Please check your file.");
    }
    return llDumpList2String(stream, "");
}

Example: StaX Writer

Input notecard:

<xml>
    <about>
        <description>
                            This is a StaX XML stream parser, 
                            with look-back and suitable for streams.</description>
   </about>
   <usage>

    wasStaX_GetNodeValue(string xmlStream, string node)
    
    where xmlStream is a a flat string containing the xml file and,
    node is a node containing the value you wish to extract.

   </usage>
</xml>

Output:

Object: It's awesum! Very awezum! 
Object: DUMP: <xml>    <about>        <description>It's awesum! Very awezum!</description>   </about>   <usage>    wasStaX_GetNodeValue(string xmlStream, string node)        where xmlStream is a a flat string containing the xml file and,    node is a node containing the value you wish to extract.   </usage></xml>
Object: wasStaX_GetNodeValue(string xmlStream, string node) where xmlStream is a a flat string containing the xml file and, node is a node containing the value you wish to extract. 

Code

/////////////////////////////////////////////////////////
// [K] Kira Komarov - 2011, License: GPLv3              //
// Please see: http://www.gnu.org/licenses/gpl.html     //
// for legal details, rights of fair usage and          //
// the disclaimer and warranty conditions.              //
//////////////////////////////////////////////////////////
 
string wasStaX_SetNodeValue(string xmlStream, string node, string value) {
    list stream = llParseString2List(xmlStream, [""], ["<", ">", "/"]);
    integer size = llGetListLength(stream);
    list StaX = [];
    integer ptr = 0;
    integer set = 0;
    do {
        string current = llList2String(stream, ptr);
        string lookback = llList2String(stream, ptr-1);
 
        if(current != "/" && lookback == "<") {
            StaX += current;
            jump next_tag;
        }
        if(lookback == "/") {
            StaX = llDeleteSubList(StaX, llGetListLength(StaX)-1, llGetListLength(StaX)-1);
            jump next_tag;
        }
        if(current != ">" && current != "/" && current != "<") 
            if(llList2String(StaX,llGetListLength(StaX)-1) == node) {
                if(!set) {
                    stream = llListReplaceList(stream, (list)value, ptr, ptr);
                    set = 1;
                    jump next_tag;
                }
                stream = llListReplaceList(stream, (list)"", ptr, ptr);
            }
@next_tag;
    } while(++ptr<size);
 
    if(llGetListLength(StaX) != 0) {
        llSay(DEBUG_CHANNEL, "The following tags may be unmatched: " + llDumpList2String(StaX, ",") + ". Please check your file.");
    }
    return llDumpList2String(stream, "");
}
 
key nQuery = NULL_KEY;
integer xLine = 0;
list xList = [];
//pragma inline
string xName = "Input";
 
string sCnt = "";
 
default
{
    state_entry()
    {
        integer itra;
        for(itra=0, xList=[], xLine=0; itra<llGetInventoryNumber(INVENTORY_NOTECARD); ++itra) {
            if(llGetInventoryName(INVENTORY_NOTECARD, itra) == xName)
                jump found_notecard;
        }
        llInstantMessage(llGetOwner(), "Failed to find notecard.");
        return;
@found_notecard;
        nQuery = llGetNotecardLine(xName, xLine);
    }
 
    dataserver(key id, string data) {
        if(id != nQuery) return;
        if(data == EOF) {
            sCnt = wasStaX_SetNodeValue(sCnt, "description", "It's awesum! Very awezum!");
            llOwnerSay(wasStaX_GetNodeValue(sCnt, "description"));
            llOwnerSay("DUMP: " + sCnt);
            llOwnerSay(wasStaX_GetNodeValue(sCnt, "usage"));
            return;
        };
        if(data == "") jump next_line;
        sCnt += data;
@next_line;
        nQuery = llGetNotecardLine(xName, ++xLine);
    }
    changed(integer change) {
        if(change & CHANGED_INVENTORY) llResetScript();
    }
}