User talk:Tedd Maa/stand-alone script engine
- Although this is interesting Tedd, it doesn't seem to keep in mind one very fundamental issue: computations at the server end have limited scalability, relative to any of the pressure points on scaling, which all derive from population pressure in one way or another. No matter how we partition computing resources server-side, there will always be a key computational workload there for physics etc, so in general there is only limited power left over for ambitious scripting. While we'd all like high-powered in-world scripting, that's just not going to happen as long as the only available computing horsepower for scripting is server-side.
- Although it's not going to happen for a long time, relatively unlimited scripting power is ultimately going to have to come by harnessing the clients, through server-side orchestration of partitioned tasks distributed very dynamically across all connected participants. This is the only source of computing power that both scales with population and offers massive resource per leaf node. What's more, with multicore CPUs everywhere (and with very little work for them to do because desktop apps tend not to be structured for parallelism), this is one resource that has a very bright future.
- So, while you offer some interesting things to think about here, if you want to look ahead to a future of high-powered server-side scripting, you really need to think about workload offloading too. (That's still "server-side scripting", even though the computations are distributed and run client-side, because such computations can never under any circumstances affect the client state; "client-side scripting" is a different matter altogether.) --Morgaine Dinova 04:18, 15 October 2007 (PDT)
Tedd Maa: In reply to the points by Morgaine.
- There are limitations on the server side, yes. But this does not mean we should purposely design the scripting system in a bad way. It is up to the server itself to scale physics well. But the script engine is not limited to controlling physical objects. It can be clouds, smoke or even gameplay rules that don't even exist as an object in the world.
- The whole point of this design is to be able to run scripts from anywhere. For example from the client. But running it from the client breaks persistency in the world. When user logs off, objects disappear. So I imagine people who want to run their own script engine having a shell somewhere like we did back in the 90's with good old bots and bouncers on IRC. But your script can very well be bound to a script engine embedded in your SL viewer for example, no problem. The region server doesn't need to discriminate on who is running the script server if server admin doesn't want it to.
- It is still called server-side scripting. But one single script (or multiple scripts) can be any program in any programming language running on any number of computers on any number of different operating systems and CPUs anywhere in the world. As long as the region has only ONE endpoint to speak to it does not discriminate on implementation, operating system, number of computers involved in computation, etc. This is the whole essence of my plan. I will look through the document and see if maybe I didn't communicate that clearly enough. Its still just a first draft.
Tedd Maa 20071015 16:40 UTC/GMT+1
Object Orientated Program Environment
Most traditional OOP languages fall short of the object orientated design. That is they call themselves a object-oriented program language, but they are not much of a object orientated approach outside of the language itself. Notice how I use "oriented" and "orientated." One leans towards the paradigm from the outside and the other is centered in the entire paradigm and works outward. At first glance of the your plan I heard a familiar tune, and it matches with the object-oriented program languages. Such languages have been quite a step in the industry, but they have been pushed out to market before their maturity.
As you can see from my post about "Client Script & Persistent Data Object," one of the ideas here is similar. In fact, the entire concept you have presented so far can be summed up underneath paravirtual machines. On that point, to call this "Tedds stand-alone script engine" may do some injustice to those who might want to contribute to the development of this project.
Another point, the security has to go much deeper than what I see talked about in the article. To offset the execute state to a different physical area is not enough, for example. We need to be able to run arbitrary code in arbitrary locations.
I stay focused on the object-orientated environments, and I hope you follow if you continue such motives to rewrite the engine. Dzonatas Sol 12:31, 15 October 2007 (PDT)
good point on the "Tedds stand-alone script engine" - wasn't meant to name it at all, was meant as "Tedds proposal on ..." or "Tedds discussion on ...". I will fix that in next revision (hopefully coming in a couple of days).
Security is something I'm still thinking about. I'll try to get something more about that into next revision. (security always comes first, so it will probably dictate some of the details in the design)
Tedd Maa 20071016 21:50 UTC/GMT+1
I've got a few things to say, as someone who's implemented and dealt with scripting under MUDs. What I'm going to say may fly in the face of what you recognize is true, but it's important to recognize some of what has been dealt with.
First, there is literally no difference between the effect of running a script on the server itself and the effect of running it somewhere else. They are functionally and from a security context identical -- which means that you can't state that simply running it on a different server, or in a different process, or anything is going to improve security. The reason why is because the server must trust the script-RPC input enough to allow the state changes that the script requests to occur within its (authoritative) view of the world.
This means that the effects that the ScriptEngine is requesting must be granted privilege if the Script running in the ScriptEngine is to effect any changes in the persistent world.
The only effective difference between running the ScriptEngine on the server and running it on the client (or somewhere else referenced by the client) is that the ScriptEngine running on the server can be vetted as to its possible impact by the server operator. It's MUCH harder to verify a new protocol that submits requested changes than to verify source that implements the protocol. (If the sim is running on a low-powered machine, the external scripting engine could conceivably overwhelm it with events unless it's carefully throttled.)
A local ScriptEngine can enforce requirements, can enforce how quickly a script is allowed to execute its next instructions, and can much more easily manage things. As well, a local ScriptEngine can enforce its own security boundary by preventing access to P/Invoke or any of a number of other dangerous functions and objects. It can also verify the code the same way that the CLR does to ensure that it doesn't call anything that isn't acceptable. A local ScriptEngine could accept CLR bytecode (though it doesn't have to). It could accept things written in LSL1 or anything, compile them to MSIL, and run them.
Technically, a server operator doesn't need to allow any external ScriptEngines -- and I would argue that not allowing it is the most secure option (to minimize attack surface). Thus, I'm going to make a new proposal...
Every grid's operation staff (even if it's a staff of one) is god, within that grid. They rather have to be (by definition, they're the operations staff that implements the grid policy). That means that they must be able to grant or restrict access to any function of the grid's implementing servers, including server scripting, server-remote scripting, and each and every function that could be called. They need to be able to limit the number of instructions that a script can run, they need to be able to implement their own concepts of "ownership" and "permissions"... and as such, they need to be able to have a death-grip on the code that they allow to run. (This means that they must be able to have more granularity than simply saying 'llPushObject not allowed in this parcel'.)
So, I propose a privilege-based and privilege-mandated system. Allowing users to run scripts on the server is fine -- as long as you don't try to retrofit a security policy onto a code base that doesn't support it. The initial design must include the means to grant or deny permissions on an individual function level -- and must also include the means to create "groups" of permissions that can be granted in toto to a user. Another important design requirement is that a single function can belong to more than one group -- so that (for example) functions which are string handling utilities can be assigned to both the 'stringhandling' and the 'remoteHTTP' groups.
All decision on whether to grant the privilege needs to be made before the function does any processing -- which means that every function must, before anything else, check its authority to run that command. (This is the same thing that would be necessary for any implementation of server-remote ScriptEngines, on each and every message.) There are ways to optimize this, of course -- keeping a hash of functions that have been verified as having been given permission to run so the security manager doesn't need to do much more time-consuming privilege lookups on every function call is a good way of doing it -- but it still must be done.
This is not to say that the idea of a remote ScriptServer is bad -- it would be very useful, for those who would otherwise run afoul of (say) XMLRPC connection throttling, they can just do it from their own system and not worry about it. It just has to be done very carefully, and the incoming request validator needs to operate with the same level of rigorous input validation that would otherwise be applied to user scripts running on the server.
(And as for 'persistence' in the world... the only time you can have persistence is if the grid operator agrees to do everything it can to make the world actually persistent, including accept uploads to its asset server and require no state to be saved on the client. I'm a very big fan of allowing (or, in some contexts, requiring) the user to 'pay his own way' in terms of asset storage -- if the user stops participating in a world, eventually the data expires from the grid's asset cache and stops showing up in the world until the next time the user logs in and retransfers the assets from local to gridcache. Because it wouldn't take up assetserver space indefinitely, it would also reduce the need to charge L$ for texture and animation uploads. Unless that's a very large part of LL's reduction of L$ in the economy, it's not necessary and it only provides a barrier to entry. Play for free, but if you want to create? You gotta pay.)
Hunting Hare 18:55, 10 November 2007 (PST)