Mono

From Second Life Wiki
Revision as of 08:27, 31 January 2008 by Babbage Linden (talk | contribs)
Jump to navigation Jump to search

Linden Lab Studio Icehouse is now testing a simulator upgrade which will dramatically speed the running of scripts. The Linden Scripting Language ( LSL ) will not change in any way, so all of your existing scripted objects and attachments will continue to function as before, only faster. The key to this improvement is an open-sourced virtual machine called Mono.

Mono is now in beta. You can download a viewer which will connect you to the beta grid. Then you can teleport to a Mono region and give Mono a try yourself. Instructions at the bottom of this page.

How LSL scripts work

All your LSL scripts are run by the simulator (Linden Lab server program) that also runs the region you are in. When you teleport, or region cross, your new region's simulator takes on the duty of running all your scripted attachments. But the simulators cannot understand LSL directly -- the language was designed for human readability, not machine. So before the script can be executed, it must be turned into a machine readable format. This process is called compilation, and the resulting machine readable version of the script is called bytecode. LSL scripts are compiled when they are created by resident-programmers. The bytecode itself is stored on the Linden Lab asset servers and never needs to be referred to directly by residents. Instead, when you rez a scripted object, the simulator for the region you are in notes the script(s) in the object, and requests the appropriate bytecode from the asset database. The simulator program has several parts, and the part which runs the script bytecode is called the LSL virtual machine.

In today's Second Life, scripts are everywhere in regions: from simple rotating objects to complicated vehicles, vendors, or attachments that respond to your chat commands. For many regions the LSL virtual machine is kept busy trying to execute hundreds of scripts all at once. As the number and complexity of scripts in a region rises, so do the demands upon the simulator. After a certain point the virtual machine starts taking up so much processing time that the rest of the simulator (particularly the physics engine) bogs down, and server-side lag results. Thus anything that can speed up the execution of scripts can push out the point where server lag starts to occur.

Enter Mono

Mono is another kind of virtual machine. It is fully open-sourced and has a proven record of speed and versatility. For over a year now Mono has been considered by Linden Lab as an alternative to the old LSL virtual machine. But there are difficulties with switching virtual machines. The most fundamental problem is that all bytecodes are different. So the LSL bytecode is just gibberish to Mono, and vice versa. Before we could start using a Mono virtual machine, we need to develop a compiler which can take LSL scripts and turn them into Mono bytecode. This is tricky, because the goal is to make scripts running under Mono behave 'exactly' like scripts running under the current virtual machine. It's painstaking work, and requires an extraordinary amount of testing. The final sprint of coding was completed in the third quarter of 2007, and since November Linden Lab QA has been rigorously pounding on the new virtual machine with an assortment of tests both automated and manual.

The Plan

The ultimate test of the new virtual machine will be when residents can start to use Mono for their LSL scripts. Once Mono passes our internal QA we are going to deploy it to our beta grid. That grid is currently being used by Havok 4, the new version of our physics engine. However, through the wonders of Het Grid, we will be able to share the grid with Havok. Het Grid allows us to make some regions on a grid run different simulator software than others. So we can have some regions on the beta grid running Havok, and some running Mono!

We'll use the same Het Grid strategy when Mono is ready for the main grid -- initially it will be deployed only to a few regions, and residents will be able to tp over to them to confirm that their scripts still behave properly under Mono. Eventually we will release Mono gridwide and the LSL compiler will be deprecated.

Early results

We've run some benchmarks to compare the performance of the LSL2 virtual machine and the Mono VM. When we run the tests side by side we found that Mono is up to 220x faster than LSL2. These benchmarks were math intensive scripts usually used to evaluate performance. Once we get the beta program going we hope to get input from residents on Mono performance with more typical SL scripts.

Mono How-to

Suppose you have some scripts you want to test under Mono. You will need to go to a Mono-enabled region and you will need to run a Mono viewer. Step-by-step follows, also check the Mono beta FAQ if you have questions.

  • Download a Mono viewer: beta viewers
    • This viewer will automatically connect you to the beta grid.
    • It also contains the UI to enable compiling your script to Mono
  • Launch the Mono viewer and log in as usual.
  • Teleport to a Mono-enabled region of the beta grid.
    • Most regions are running Havok4, not Mono
    • Mono regions are:
      • Sandbox Cordova MONO
      • Sandbox Goguen MONO
      • Sandbox Newcomb MONO
      • Sandbox Wanderton MONO
  • Rez the object containing the script.
  • Edit the object and open the script.
  • You will see a new checkbox for compiling this script to Mono instead of LSL. Check the box for Mono, then save.
    • If you fail to see the checkbox, you are not running a Mono viewer.
    • If the checkbox is grayed out for Mono, you are not in a Mono enabled region.
  • I suggest you append the word "Mono" to the object/script name. This way you'll be able to identify which objects in your inventory will only run in Mono regions.
  • Confirm that the scripted object runs the same under Mono as it has before.
    • Any differences in behavior should be noted in a JIRA ticket.
    • First check this meta JIRA to see if anyone has reported the issue already
      • This may not be doable unless you've drilled down into your script to locate what is breaking.
    • If a JIRA already exists, and it appears to be the same failure, then simply add your information as a comment and attachment to that issue.
    • If not, open a new SVC ticket for each bug you find. In the ticket summary please use the words "Mono beta" so that we can filter for them.
    • In addition to your description of the problem please attach your script itself, or (ideally) a smaller script which illustrates the difference in behavior between the two virtual machines.
    • Link your JIRA to the meta JIRA.

Testing

During the integration of Mono we have used tests to ensure there are no regressions. For the current version of the regression test see LSL Language Test. This effectively serves as the specification of LSL.

There are tests for library call bindings in LSL Library Call Test 1 and LSL Library Call Test 2. This is split to overcome the memory limitations.

There are several benchmarks to test the performance. They are available here:

FAQ

Section for questions.

When will the Mono Beta start?
The Mono beta is in progress
Do you have more info about the Mono Beta?
Yes, see Mono beta FAQ.
Will the available memory for scripts change? (Currently 16k in LSL2 VM)
For the same LSL script the Mono bytecode and LSL2 bytecode will be of different size. In order to be compatible with all known scripts, we have expanded the size ceiling for Mono to be 64k. This is ok to do for Mono because unlike LSL2, Mono allocates memory dynamically, whereas all LSL2 scripts occupy 16K. Mono scripts only allocate the memory that they need.
64K? Wow, isn't that going to encourage inefficient scripting?
We hope that the change will promote more efficient scripting. Currently programmers have to get around the 16K limit by using multiple scripts, and a lot of cycles get spent on passing data between those scripts. With a single script that would not be necessary.
Will I have to manually convert all my objects to using Mono, or is there an automated tool?
Yes, you must manually invoke compilation to Mono. Though you can make use of the Tools Menu to recompile all scripts in selection to Mono.
Can I keep my scripts running on the old LSL2 VM forever?
We have no plans to eliminate the LSL2 VM. This will be re-examined after Mono has been live on the Main Grid for a while. But right now it is easier to continue to support LSL2 than to migrate all scripts.
Will I be able to write scripts in languages besides LSL since Mono supports lots of languages?
Eventually. Right now our goal is to make Mono completely compatible with LSL2 for LSL scripts.
Why didn't you use <favorite_language> instead of Mono? (ie lisp, python, lua, javascript)
TBA
Will LSL be getting real language features with this change? (ie arrays, references/pointers, includes/imports)
No. The LSL language is not changing with this update.
In LSL2 VM scripts are compiled on the viewer then uploaded, will that change with the Mono VM?
Yes, Mono compilation is done in a distributed fashion on the sim hosts.
Related to the above, I use a "clever trick" to upload my compiled bytecode in LSL2 with out the correct script text. What will happen to scripts I uploaded in this way when converted to Mono? Will I be able to continue to use my "clever trick" for Mono scripts?
The Mono compiler looks only at the script text. The Mono VM will only run bytecode which has been compiled by our Mono compiler. You will not be able to run any uploaded Mono bytecode.
What about scripts whose LSL code has been lost, ie scripts that still run, but result in "Script missing from database." when you try to edit them? Is there any possibility of bytecode translation, or are these scripts stuck in the LSL vm forever?
There are currently no plans to allow byte code translation of LSL scripts, only compiling from source. This may be considered depending on resident demand.