Difference between revisions of "Lua FAQ"

From Second Life Wiki
Jump to navigation Jump to search
(Created page with "__TOC__ === Can I keep using LSL? === Yep! LSL should work exactly the same as before, and a lot of testing work has been done (and is being done) to ensure that is the case...")
 
Line 307: Line 307:


In general, C# doesn't aim at producing very small bytecode for async functions, which is important for something like Second Life where you might have thousands of different individual scripts running within a region. Luau does generate small bytecode, while still having decent performance, making it a good choice for Second Life. And hey, strings are UTF-8 in Lua instead of UTF-16 as in Mono, so they use much less memory for most data as well.
In general, C# doesn't aim at producing very small bytecode for async functions, which is important for something like Second Life where you might have thousands of different individual scripts running within a region. Luau does generate small bytecode, while still having decent performance, making it a good choice for Second Life. And hey, strings are UTF-8 in Lua instead of UTF-16 as in Mono, so they use much less memory for most data as well.
=== Are we going to get a higher memory limit in scripts? ===
Not at first, Luau uses less memory than Mono, so you shouldn't really miss it. You're not going to start hitting script memory limits just because you converted your script to Lua. Using Lua proper should allow you to more precisely control memory usage in your scripts, e.g. by being able to handle memory allocation errors in <code>pcall()</code> and access to the mutable <code>buffer</code> class for byte-packing.
There is some interest in giving specific scripts a little more memory if they're designated "high priority", like a main "game manager" object for an experience, but there are no specific plans for this at the moment.
=== Are my existing LSL scripts going to run on Lua Automatically? ===
While we can run LSL in Lua now, and the idea is to eventually phase out Mono, there’s no immediate plans to transparently migrate existing scripts over to Lua. There are ideas about how this could be done safely without breaking existing content, but we want to make sure we have everything right before we start work on this.
=== How does interoperation with LSL work? ===
Everything that you expect from LSL should have some equivalent in Lua. There’s <code>key<code>s, <code>rotation</code>s and <code>vector</code>s and something like <code>some_vec * some_rot</code> in LSL is the exact same in Lua scripts.
All LSL functions will be available in Lua, and something like <code>llSay(0, “hi!”)</code> is just <code>ll.Say(0, “hi!”)</code> now. What about <code>llSetPos(llGetPos() + <0, 0, 1>);</code>? Just <code>ll.SetPos(ll.GetPos() + vector(0, 0, 1))</code>.
No special magic required here, and you can write whatever fancy wrappers you want around the existing LSL functions. We’ll be looking at how residents actually write these wrappers to inform the design of “official” higher-level Lua APIs, so we’re excited to see your ideas!
=== Can I mix LSL and Lua in the same script? ===
No, although most things should be a direct translation. At some point we may make a tool to help you do this, but there are no specific plans at the moment.
If your object has multiple scripts, you can mix LSL and Lua within the same object, though.
=== Can we use all of Lua? Is anything not there? ===
Most everything will be there, except for two notable exceptions:
- <code>getfenv()</code> / <code>setfenv()</code> / <code>_ENV</code> are not accessible because they necessarily make the script harder to optimize. Usually people only use this for mocking in test code, and this can be accomplished by passing in a parameter with a table of function references rather than pretending you have different global variable values for certain functions.
- <code>loadstring()</code> is not supported since it has a huge impact on performance. For those not familiar, this is equivalent to eval() in other languages. Every single time you call <code>loadstring()</code> the compiler has to go over what you passed in, compile it, and then load the generated bytecode. The Lua compiler isn’t particularly fast, and doing this can cause your script to halt in a way that we can’t interrupt to give other scripts time to run. This is a big part of why <code>eval()</code> and <code>loadstring()</code> are generally considered poor programming practice.
The one real use-case we could think of for <code>loadstring()</code> is a debugger, and we’re looking at implementing a “real” debugger that lets you step through your code. Note that this won’t happen with the initial release, but watch this space!
One thing that is planned, but will likely not be in the initial release is the coroutine library. This is one we’d really like to have in there, so you can do fancy things like have multiple “timers” in a script by just having coroutines that sleep every so often, but multiple running coroutines within the same script doesn’t fit well with the existing scripting architecture the Second Life server uses. Rest assured, this will be in there soon™ after Lua is released, but in the meantime you can still use something like the <code>Promise</code> chaining that JavaScript uses to write asynchronous event handlers.
We’ll have code demonstrating "proper" usage of Lua around the time that the public beta starts.

Revision as of 12:01, 2 July 2024

Can I keep using LSL?

Yep! LSL should work exactly the same as before, and a lot of testing work has been done (and is being done) to ensure that is the case. Even if you’re an LSL pro and rely on things like (some_list != []) for getting the list length, or strange things like ((largestring="") + largestring), you’re fully covered. Those still work, and now have optimized implementations.

In fact, LSL will get better than ever before, running faster, and using less memory! The generated bytecode for an LSL script in Lua is generally half the size of a Mono script, and most strings take up half as much memory by virtue of using UTF-8 instead of UTF-16.

We’ve taken a lot of care in writing the new LSL compiler to make the scripting experience for people who prefer LSL better than it was before.

Are you going to stop adding functions / events to LSL?

In the general case, no. A big part of the work we’ve done is making it easy to share functions between LSL and Lua without much extra effort. For events and functions that can be reasonably exposed to LSL (i.e., they only use simple types that LSL has anyway) then LSL should still get those.

For more advanced features, the idea is that there should still be a path for exposing them to LSL. Obviously, with Lua it’s possible to have object-oriented APIs that would not be possible in LSL, but those objects will generally be written in a way that calling a method on a “special” object we introduce in Lua will really boil down to a function call that can be expressed in LSL.

Not a real example, but some_object:giveInventoryList("folder_name", inventory_list) might really boil down to a llGiveInventoryList(some_object_id, "folder_name", "inventory_list"); call. In that way, most additional Lua functionality will be a fancy “easier” wrapper around functions shared with LSL!

The caveat is if functions are added that use new types that aren't in LSL (like functions that take in a key -> value map) likely won't be exposed to LSL, as there's no way to represent them in LSL anyway.

Wait, LSL can run on Lua now? How? Why would you do that?

A reasonable question! It sounds crazy at first, but as it turns out, LSL maps really well to Lua bytecode, and we already have a compiler that can output both Mono and legacy LSO2 bytecode. Writing a compiler that could output Lua bytecode wasn't terribly difficult, and it allows LSL scripts to run faster, and be more memory-efficient. Most importantly, it allows us to test behavior of the new scripting engine against the behavior of the existing Mono scripting engine. This helps us make sure we're doing an apples-to-apples comparison on our benchmarks, and that all the nice things that Mono currently gets us will also work correctly under the new scripting engine.

Additionally, for better or for worse, LSL is here to stay. People have been writing scripts in it for 20 years, and they're going to be running on the servers for years to come. Performance and memory usage issues with LSL scripts will continue to matter for the foreseeable future. The faster the server can run existing LSL scripts, the more time it can dedicate to running newer Lua scripts. Memory usage has been a particular problem with the existing Mono scripting engine, and this would help us address this.

What about running scripts on the viewer-side if I want viewer-side effects? Does this have any interaction with the work adding Lua addons into the viewer?

No, the work on adding Lua addons to the viewer is a separate effort. Lua scripts on SL will strictly run on the region, and not on the people's viewers. We're focused on providing a high-quality replacement to the existing server-side scripts before we look at anything else.

Note that we are all too aware of the current difficulty of scripting HUD UIs for Second Life (we make a lot of our own) and we're thinking of ways we can make things better, but there are no definite plans, and it wouldn't involve running scripts on the viewer in the near-term. The current UI tooling exposed to Lua viewer addons isn't tooled for being called from untrusted code either way, so lack of "client-side" scripts is unlikely to hinder any effort to create a better API for HUDs for in-world systems.

You said Lua runs LSL faster than Mono? It uses less memory? How? I'm suspicious.

A reasonable suspicion! We were also suspicious and suspected that Lua would be slightly slower, but in reality, yes, Lua can run LSL scripts quite a bit faster than the custom-patched Mono 2.6 we currently use. The best LSL on Mono has performed compared to Luau in our internal tests is 10% slower than Luau. Similarly, most LSL on Luau scripts end up being around half the size of Mono scripts. That means much more memory to play with in your LSL scripts.

The reason why isn't intuitive, after all, Mono is a lot more intricate and has a lot of fancy performance optimizations like a JIT optimizer, but it becomes clear when you look at the bytecode that's generated for LSL on each scripting engine.

Consider the following LSL function as an example: 

float loudSubtract(float val, float sub) {
    // subtract, telling the owner what we did
	float new_val = val - sub;
	llOwnerSay("New val is " + (string)new_val);
	return new_val;
}

This compiles to the following Luau bytecode (subject to further optimization): 

Function 0 (_floudSubtract):
SUB R2 R0 R1
GETIMPORT R3 2 [ll.OwnerSay]
GETIMPORT R6 5 [lsl.cast]
MOVE R7 R2
LOADN R8 3
CALL R6 2 1
LOADK R5 K6 ['New val is ']
CONCAT R4 R5 R6
CALL R3 1 0
RETURN R2 1

Okay, that's pretty decent. Some places where things could be more optimized, but no big deal. The equivalent Mono bytecode is (and bear with us, this is long:)

While this might look unreasonably large, this is also pretty similar to how async C# functions that have awaits get compiled. The code required to allow a function to pause in the middle to let other things run is necessarily very large, and our Mono bytecode instrumentation is essentially a special variant of that which optimizes for cases where the function won't have to pause. Try pasting the following code into sharplab.io (while decompiling to IL) and you'll see something very similar: 

using System.Threading.Tasks;
public class Example {
    public async Task llOwnerSay(string val)
    {
        // stub
    }
    
    public async Task<float> loudSubtract(float val, float sub) {
        float new_val = val - sub;
        await llOwnerSay("New val is " + new_val.ToString());
        return new_val;
    }
}

The Lua bytecode ends up being a lot smaller, because the Lua scripting engine itself knows how to handle this case, and doesn't need to add any special code to your scripts to allow them to be paused and resumed in the middle of a function.

Even though Mono has a JIT optimizer, which should in theory make the script run faster, this sort of code is very unusual, and the JIT optimizer isn't able to optimize it very well. That means that even though Luau itself is in theory slower, it's able to run LSL much faster than Mono can.

You might also notice things like call float64 [LslUserScript]LindenLab.SecondLife.LslUserScript::Subtract(float64, float64) in the Mono code. Those are necessary because of LSL's weird operand evaluation order, but are unnecessary in Luau because of the more efficient form of bytecode that Luau uses.

In general, C# doesn't aim at producing very small bytecode for async functions, which is important for something like Second Life where you might have thousands of different individual scripts running within a region. Luau does generate small bytecode, while still having decent performance, making it a good choice for Second Life. And hey, strings are UTF-8 in Lua instead of UTF-16 as in Mono, so they use much less memory for most data as well.

Are we going to get a higher memory limit in scripts?

Not at first, Luau uses less memory than Mono, so you shouldn't really miss it. You're not going to start hitting script memory limits just because you converted your script to Lua. Using Lua proper should allow you to more precisely control memory usage in your scripts, e.g. by being able to handle memory allocation errors in pcall() and access to the mutable buffer class for byte-packing.

There is some interest in giving specific scripts a little more memory if they're designated "high priority", like a main "game manager" object for an experience, but there are no specific plans for this at the moment.

Are my existing LSL scripts going to run on Lua Automatically?

While we can run LSL in Lua now, and the idea is to eventually phase out Mono, there’s no immediate plans to transparently migrate existing scripts over to Lua. There are ideas about how this could be done safely without breaking existing content, but we want to make sure we have everything right before we start work on this.

How does interoperation with LSL work?

Everything that you expect from LSL should have some equivalent in Lua. There’s keys, rotations and vectors and something like some_vec * some_rot in LSL is the exact same in Lua scripts.

All LSL functions will be available in Lua, and something like llSay(0, “hi!”) is just ll.Say(0, “hi!”) now. What about llSetPos(llGetPos() + <0, 0, 1>);? Just ll.SetPos(ll.GetPos() + vector(0, 0, 1)).

No special magic required here, and you can write whatever fancy wrappers you want around the existing LSL functions. We’ll be looking at how residents actually write these wrappers to inform the design of “official” higher-level Lua APIs, so we’re excited to see your ideas!

Can I mix LSL and Lua in the same script?

No, although most things should be a direct translation. At some point we may make a tool to help you do this, but there are no specific plans at the moment.

If your object has multiple scripts, you can mix LSL and Lua within the same object, though.

Can we use all of Lua? Is anything not there?

Most everything will be there, except for two notable exceptions:

- getfenv() / setfenv() / _ENV are not accessible because they necessarily make the script harder to optimize. Usually people only use this for mocking in test code, and this can be accomplished by passing in a parameter with a table of function references rather than pretending you have different global variable values for certain functions. - loadstring() is not supported since it has a huge impact on performance. For those not familiar, this is equivalent to eval() in other languages. Every single time you call loadstring() the compiler has to go over what you passed in, compile it, and then load the generated bytecode. The Lua compiler isn’t particularly fast, and doing this can cause your script to halt in a way that we can’t interrupt to give other scripts time to run. This is a big part of why eval() and loadstring() are generally considered poor programming practice.

The one real use-case we could think of for loadstring() is a debugger, and we’re looking at implementing a “real” debugger that lets you step through your code. Note that this won’t happen with the initial release, but watch this space!

One thing that is planned, but will likely not be in the initial release is the coroutine library. This is one we’d really like to have in there, so you can do fancy things like have multiple “timers” in a script by just having coroutines that sleep every so often, but multiple running coroutines within the same script doesn’t fit well with the existing scripting architecture the Second Life server uses. Rest assured, this will be in there soon™ after Lua is released, but in the meantime you can still use something like the Promise chaining that JavaScript uses to write asynchronous event handlers.

We’ll have code demonstrating "proper" usage of Lua around the time that the public beta starts.

Retrieved from "https://wiki.secondlife.com/w/index.php?title=Lua_FAQ&oldid=1216874"