Talk:Content Creation/Scripting User Group/Transcripts/2011 09 19

'Lua doesn't meet our concurrency requirements'

That's straight from Kelly Linden's mouth, on this transcript, 12 years ago (from the perspective of when I'm commenting).

I wonder what kind of 'concurrency requirements' those might have been that Lua couldn't fulfill.

Taking a look at some benchmarks (compiled afresh every day or so, which means that, so long as GitHub is around, you should be able to see these benchmarks in the future, too) shows that the point where Lua actually excels triumphantly over C#/Mono is... when using coroutines!

On these (vast) selection of benchmarks, the point is not really to discuss 'which language is best'. For instance, a C# implementation will often beat not only Lua (which is often deployed as an embedded interpreter or also as JIT compiler), but even Go (which is statically typed and natively compiled)! The tradeoff? It consumes more memory. A lot more memory. Oodles of memory! A nightmare of memory! And if you don't pay attention, it'll easily gobble up all the memory it can find, leaks all the time, and even if forcing the garbage collector to run periodically (which I often do!) — something that the language itself is supposed to do on its own — well, it still leaks memory, until, eventually, it swallows up everything and there is just a way out: kill the running application and start from zero.

That is one of the reasons (probably not the only one, though) for having to reboot SL simulators once in a while. It's not really because there are caching problems (these are rare nowadays — especially since all assets are being streamed from Amazon's cloud). It's also not really an issue with LL's core simulator code — it is, after all, written in C (which is an excellent choice in terms of speed and a very small footprint — hardly anything else except for machine code can beat well-written C code — but it's hard to maintain, of course). I always suspected that the culprit is, in fact, Mono, and LSL written on top of Mono.

From a purely conceptual point of view, Mono does have one huge advantage. Just like Java VM, in theory at least, you can run a tiny Mono runtime, and just have any programming language (well, 'any' that is supported by Microsoft — a growing number these days, though) compile to Mono's bytecode. This was, after all, what LL sort of promised and/or hinted at, when we made the switch to Mono: the idea would be that, in the near future, we could just use whatever programming language we wanted, and just compile it for the Mono VM.

And, in fact, we know this is conceptually possible, because the OpenSimulator guys did exactly that. While not necessarily popular — because, well, most scripts are written in LSL, after all, and it's far easier to just copy some code from SL and get it running under OpenSimulator (which will be easy enough to do in most cases) than to rewrite the code from scratch using, say, C#. There ought not to be significant speed/memory improvements anyway: it's just a question of having a more palatable programming language available to scripters.

However, after a decade and a half, it's now more than clear that LL does not intend to allow any other scripting language. Whatever plans existed to support C# or other languages were either abandoned, or buried deep, with the lowest possible priority — it's unlikely it'll emerge to the surface again. Ever.

By contrast, making the switch from Mono to Lua would do wonders in terms of improving overall script behaviour and performance, while at the same time keeping the simulators much happier, consuming less memory and resources, and benefitting everybody. It would also allow 'serious' programmers — as opposed to self-taught scripters — to deal with complex programming issues using a 'serious' language; more specifically, the games/virtual world industry is full of knowledgeable Lua programmers — because Lua was designed to be an embeddable (interpreted) language for real-time performance, as well as to be very, very, very, very easily extensible. Oh, and, of course, LL wouldn't have to cram all those gazillion functions on a single package — they could be held in separate libraries, and the Lua core would just happily load whatever is required. It would also give scripters the ability to create their own libraries (with a well-known, well-understood, well-tested, very clear interface), share them freely (using the existing permissions system!) and just let other scripters import their libraries and use them on their own scripts.

Do you know that, every time someone copies some LSL code from this Wiki, or from the forums, or from blogs, slaps it into a 'new script', and compiles it, the simulator will generate a new instance of the script — from scratch? The script may have the same name, exactly the same lines of code, be identical in every regard, and even (internally) be compiled to the exact same bytecode, but... from the perspective of the simulator, they're two different running applications, consuming twice the resources. Now think of your favourite region in SL, where a few dozens of residents happen to have their homes, all of which furnished with furniture such as chairs — all of which, these days, most likely will be running a variant of the famous AVSitter script. Well, the bad news is that even though the exact same script may be used in each case, in practice, the simulator will treat them all as being different, and handle resource management accordingly. The exception to this rule is when a single resident compiles the code just once, and then replicates their own script at will, by duplicating the script (or, rather, duplicating the object that contains the script). In this very strict scenario, the simulator will not run multiple 'independent' copies, but 'know' that it can just (internally) run one instance, and just let the data be different in each case. The stats window for running scripts may be showing two (or more...) instances being active, but, in fact, there will be just one running copy, it's just the way they're being executed that 'tricks' the stats window to show allegedly different scripts — but they're not, they're the same bytecode being run (just with separate memory/stack/heap/whatever LSL uses).

Using Lua would not only avoid that issue, but, thanks to its much more advanced system/memory management capabilities and ease in extensibility, one could just use a common pool of libraries, all dynamically shared among the many running scripts, thus lowering dramatically the amount of memory (and other resources!) being currently wasted.

As for using Lua instead of Mono... what about having both?

— Gwyneth Llewelyn (talk) 15:25, 27 January 2023 (PST)