User:Babbage Linden/Office Hours/2009 04 01
Jump to navigation
Jump to search
Transcript of Babbage Linden's office hours:
Topics * Lots of people are concerned about the reduction in performance of list processing commands. (Caused by Mono scheduler change) LINK * on the one hand it is actually just fixing a bug: lists shouldn't have been able to process that quickly. LINK * the model we want is that objects can tell us what their prim and script usage is going to be upfront LINK
[3:41] | WolfPup Lowenhar: | morning babbage
|
[3:41] | Nock Forager: | I saw something
|
[3:41] | Babbage Linden: | hi everyone
|
[3:41] | Morgaine Dinova: | Hiya Babbage!
|
[3:41] | Babbage Linden: | sorry, i'm late
|
[3:41] | Babbage Linden: | got caught up with things here
|
[3:41] | WolfPup Lowenhar: | that is ok babbage
|
[3:41] | Morgaine Dinova: | How are things?
|
[3:41] | Babbage Linden: | thanks for hanging around
|
[3:41] | Babbage Linden: | good thanks
|
[3:41] | WolfPup Lowenhar: | np i just found a bug in thr rc!!
|
[3:41] | Babbage Linden: | i'd like to talk about the scheduler beta today if that's ok
|
[3:41] | WolfPup Lowenhar: | rc11
|
[3:42] | Babbage Linden: | have any of you tried your scripts out on it yet?
|
[3:42] | Morgaine Dinova: | Cool
|
[3:42] | WolfPup Lowenhar: | i hven not been to the bata grid at all
|
[3:42] | Nock Forager: | Tried on few. No problems.
|
[3:42] | Babbage Linden: | that's good to hear nock
|
[3:43] | Babbage Linden: | here's the public jira feedback:
|
[3:43] | Babbage Linden: | http://jira.secondlife.com/browse/SVC-3997
|
[3:43] | WolfPup Lowenhar: | the only problem is that main scripts i wou want to check i cant because there server is on aginin and not on aditi
|
[3:43] | Babbage Linden: | lots of people are concerned about the reduction in performance of list processing commands
|
[3:44] | Babbage Linden: | that's a good point wolfpup, we should try to get the agni data copied over so people have their new scripts to try
|
[3:44] | WolfPup Lowenhar: | < has a vendor that dose a lot of list proccessing
|
[3:44] | Morgaine Dinova: | Take the opportunity to add O(1) arrays to LSL then .... <grin>
|
[3:44] | WolfPup Lowenhar: | and the vendors server is on agini
|
[3:45] | Babbage Linden: | so, there are 2 things that might be affected: memory, where scripts temporarily use too much memory and speed, where scripts used to get away with calling expensive library calls many times in the same time slice
|
[3:46] | WolfPup Lowenhar: | would there be a way to get a private region coppied to aditi ?
|
[3:46] | Babbage Linden: | it's possible wolfpup, but we'd have to do it manually, so try to avoif it
|
[3:46] | Babbage Linden: | best thing to do is to rez objects from inventory
|
[3:46] | Morgaine Dinova: | Babbage: how about letting people activate their own inventory copies to Aditi, eg. simply by entering there?
|
[3:47] | Babbage Linden: | as we can copy the agni data over to aditi to ensure you have all your inventory available
|
[3:47] | Babbage Linden: | that would be handy morgaine, but it's a manual process unfortunately
|
[3:47] | Babbage Linden: | also, it wipes out any inventory you previously had on aditi
|
[3:48] | Babbage Linden: | which admiitedly shouldn't often be a problem
|
[3:48] | Babbage Linden: | but sometimes you will want to develop something on aditi using a new feature
|
[3:48] | WolfPup Lowenhar: | most of the inventory i have on aditi is the same as on agini
|
[3:48] | Babbage Linden: | over multiple sessions, like with http-in
|
[3:48] | Babbage Linden: | anyway, there is lots of concern about the slow down in list processing speed with the scheduler changed
|
[3:48] | Babbage Linden: | changes
|
[3:49] | Morgaine Dinova: | Is that slow-down inevitable owing to the change, or just not optimized yet?
|
[3:49] | Babbage Linden: | on the one hand it is actually just fixing a bug: lists shouldn't have been able to process that quickly
|
[3:49] | Morgaine Dinova: | Oh dear
|
[3:49] | Babbage Linden: | the slow down is because previously it was possible to call library functions that took a long time several times in a time slice
|
[3:49] | Morgaine Dinova: | Well that's a disaster, if you're implying that "it was too quick before"
|
[3:50] | Babbage Linden: | in the worst case it can take 300us to copy a list from managed code to C++ to have a function operate on it
|
[3:50] | Babbage Linden: | and a time slice is 150us
|
[3:50] | Morgaine Dinova: | In that case add O(1) arrays to LSL. No question.
|
[3:50] | Babbage Linden: | so, you "should" only be able to make one list call per time slice if the scheduler is working properly
|
[3:50] | Babbage Linden: | which it now is
|
[3:51] | Morgaine Dinova: | Well that's a disaster.
|
[3:51] | Babbage Linden: | on the upside list processing is still faster than LSL
|
[3:51] | Babbage Linden: | on the downside, it's now a lot slower than it used to be under mono
|
[3:51] | Babbage Linden: | a relatively easy way to improve the performance is to avoid the copy to unmanged code
|
[3:51] | Babbage Linden: | which causes a lot of marshalling to happen
|
[3:52] | WolfPup Lowenhar: | now i realy need to goto aditi to check my vendors :/
|
[3:52] | Babbage Linden: | and boxing/unboxing as the mono representation is converted to a C++ list representation expected by the library calls
|
[3:52] | Babbage Linden: | one way to avoid that overhead is to do the list processing in managed code
|
[3:52] | Babbage Linden: | reimplement the functions that don't need to access simulator data in C#
|
[3:53] | Babbage Linden: | but that's pretty horrible from a maintenance perspective as we end up with 2 implementations of the library functions
|
[3:53] | Babbage Linden: | which could get out of sync
|
[3:53] | Babbage Linden: | so functions would behave differently when called from LSO or Mono scripts
|
[3:53] | Babbage Linden: | that would be a bug, but it would be possible
|
[3:53] | Babbage Linden: | if there were 2 implementations
|
[3:54] | Morgaine Dinova: | I rather think the point is being missed. List traversal is not the end goal --- data access is. You're treating list traversal as a resource-hungry operation, when it's just a means to an end for people. Provide O(1) arrays that provide element access with zero resource usage and the problem disappears.
|
[3:54] | Babbage Linden: | morgaine, i agree, we should have arrays
|
[3:54] | WolfPup Lowenhar: | true morgaine
|
[3:54] | Babbage Linden: | in fact, i'll go further and say that we should have C# with arrays, objects and other useful stuff
|
[3:54] | WolfPup Lowenhar: | multople elemt arays work much easier
|
[3:55] | Babbage Linden: | i really don't want to have to extend LSL and make it work for LSO and Mono when it's probably less work to add support for C#
|
[3:55] | Babbage Linden: | but anyway, that's getting somewhat away from the subject
|
[3:56] | Babbage Linden: | the other issue with the current scheduler beta is that we have a number of other potential changes in the works
|
[3:56] | Babbage Linden: | the first is the script resource limits project
|
[3:56] | WolfPup Lowenhar: | a vewnodr i use would work much better if it had an aO(1,1,1) aray to handle the data
|
[3:56] | Babbage Linden: | and the other is our planned move to 64 bit
|
[3:56] | Babbage Linden: | which will mean using a 64 bit Mono
|
[3:57] | Babbage Linden: | and having to change the Mono memory limits to accommodate that
|
[3:57] | WolfPup Lowenhar: | im guessing that would improve script performance
|
[3:57] | Babbage Linden: | they are somewhat related
|
[3:57] | Babbage Linden: | as we talked about last week, if we introduce script limits
|
[3:58] | Babbage Linden: | and make Mono scripts cost 64KB and LSO scripts cost 16KB, people will be incentivised to use LSO
|
[3:58] | WolfPup Lowenhar: | how long befor the limits go into efect?
|
[3:58] | Babbage Linden: | which we don't want, because on average Mono scripts use less memory and CPU than LSO scripts
|
[3:58] | WolfPup Lowenhar: | i also rember you talking about having the scripts request memory
|
[3:58] | Babbage Linden: | if we move to 64 bit, the extra size used by Mono references will mean that we will probably need to give even more memory to Mono scripts
|
[3:59] | Babbage Linden: | so, in the worst case Mono scripts end up "costing" 112KB and LSO 16KB
|
[3:59] | Babbage Linden: | the incentives are even more out of whack
|
[3:59] | Morgaine Dinova: | Hmm
|
[3:59] | Babbage Linden: | yes, wolfpup, i did talk about scripts reserving memory
|
[4:00] | Babbage Linden: | so, one way to get things back in sync again would be to add the script memory reservation function
|
[4:00] | WolfPup Lowenhar: | that would give people more incentive to use mono if they could set the mem a script uses
|
[4:00] | Babbage Linden: | so people can change Mono scripts to request a maximum below 16KB
|
[4:00] | Babbage Linden: | which would make them cheaper than LSO scripts
|
[4:01] | Babbage Linden: | in some cases LSO would genuinely be the best choice
|
[4:01] | WolfPup Lowenhar: | because for a simple dor scrit that would only need say 2k 5they could set it to request it
|
[4:01] | Babbage Linden: | but the memory reservation functions would allow you to tell the system about the true cost of your script
|
[4:01] | Babbage Linden: | and it wouldn't have to always use the worst case cost
|
[4:01] | Morgaine Dinova: | Needing to express how much memory a program needs takes us back to the 60's though. Can't that be hidden?
|
[4:02] | WolfPup Lowenhar: | the problem would be knowing the true cost
|
[4:02] | Babbage Linden: | wolfpup, yes, it's hard to work out the true cost at the momemt
|
[4:02] | Babbage Linden: | all you really get is a stack/heap collision telling you that you need to use less memory
|
[4:02] | Babbage Linden: | or add another script
|
[4:02] | Babbage Linden: | and use link messages
|
[4:03] | WolfPup Lowenhar: | morgain being able to request memory is still in ues in some systems today
|
[4:03] | Babbage Linden: | so, we potentially also need better ways to report current memory usage
|
[4:04] | Babbage Linden: | as you can see, we're ending up with a number of things
|
[4:04] | Babbage Linden: | scheduler, 64 bit and script limits
|
[4:04] | Babbage Linden: | that are all related
|
[4:04] | Babbage Linden: | and we don't want to ask everyone to test their scripts 3 times
|
[4:04] | Babbage Linden: | so, the current thinking is that we'll roll the projects together
|
[4:05] | WolfPup Lowenhar: | and that can realy create a migrain for some one trying to keep them from creating bugs in the others :)
|
[4:05] | Morgaine Dinova: | If a Mono script can use less memory than an LSO, why not only allocate 8KB or less to start with, and if it's not enough then restart in a larger VM behind the scenes? It'll only happen on startup, and at most a few times.
|
[4:05] | Babbage Linden: | and release them all together to ensure that you only need to test once
|
[4:06] | Babbage Linden: | morgaine, the model we want is that objects can tell us what their prim and script usage is going to be upfront
|
[4:06] | Babbage Linden: | so we can either say, sorry you can't rez
|
[4:06] | Babbage Linden: | or say, go ahead
|
[4:06] | Babbage Linden: | knowing that the object will work fine
|
[4:06] | Babbage Linden: | we don't wa
|
[4:06] | Babbage Linden: | nt to allow scipts to rez, grow and then get to the point where the sim is swapping
|
[4:07] | Babbage Linden: | this memory model is unlike most modern operating systems
|
[4:07] | WolfPup Lowenhar: | < has seen a lot of sim swaping latly for the sim he comanages
|
[4:07] | Babbage Linden: | which will use virtual memory to allow you to allocate a lot of memory
|
[4:08] | Babbage Linden: | we specifically want to avoid using virtual memory
|
[4:08] | Babbage Linden: | as having the sim swap is really bad news
|
[4:08] | Babbage Linden: | so, we ask scripts to tell us up front how much memory they will use
|
[4:08] | Morgaine Dinova: | I still think it has to be automatic, can't expect users to know.
|
[4:08] | Babbage Linden: | morgaine, agreed
|
[4:08] | Babbage Linden: | we need a model where scripters will set their scripts to request 8KB
|
[4:08] | Babbage Linden: | then put them in a vendor
|
[4:09] | Babbage Linden: | then sell them saying "requires 8KB of memory"
|
[4:09] | Babbage Linden: | and have residents just rez them
|
[4:09] | Babbage Linden: | anyway, I'm going to have to run
|
[4:09] | Babbage Linden: | as i have another meeting
|
[4:10] | WolfPup Lowenhar: | morgaine if coders were able to find out the max their script used when they have a final product ready to put out then they would be able to have the script alocate it
|
[4:10] | Babbage Linden: | but, the news for this week is: there are a few scripting changes coming up, we don't want to ask people to test scripts every 5 minutes, so we'll likely roll them together
|
[4:10] | Babbage Linden: | and none of these changes will get to agni in the near future
|
[4:10] | Babbage Linden: | end of the year is more likely
|
[4:10] | Morgaine Dinova: | Wolf: doesn't look right to me. That's what computers are for, to determine resource usage.
|
[4:10] | WolfPup Lowenhar < still like having things run 'clean'
| |
[4:10] | Babbage Linden: | so, don't panic about testing scripts right now
|
[4:10] | Babbage Linden: | ok?
|
[4:11] | Morgaine Dinova: | KK Babbage :-)
|
[4:11] | Babbage Linden: | ok, thanks for coming
|
[4:11] | Babbage Linden: | got to run
|
[4:11] | Babbage Linden: | see you next time
|
[4:11] | Morgaine Dinova: | Take care B :-)
|
[4:11] | WolfPup Lowenhar: | tc babbage
|
[4:11] | Nock Forager: | Seeya Babbage.
|
[4:11] | WolfPup Lowenhar: | ahhh babbage you got a bear?
|
[4:11] | WolfPup Lowenhar < has started a collection :)
| |
[4:12] | Morgaine Dinova: | Make a bear quest Babbage, like Simon did :-)
|
[4:12] | Babbage Linden gave you Babbage Linden Bear.
| |
[4:12] | Morgaine Dinova: | Oh cheers B :-))
|
[4:12] | Nock Forager: | Got new one :)
|
[4:12] | Babbage Linden: | seeya!
|
[4:12] | WolfPup Lowenhar: | Thank Youuuuuu!! for the tip
|
[4:12] | Morgaine Dinova waves
| |
[4:13] | WolfPup Lowenhar: | grrr
|
[4:13] | Morgaine Dinova: | LOL @ tip Wolf :P
|
[4:13] | Morgaine Dinova: | Better change that gesture :P
|
[4:15] | Morgaine Dinova: | Think I'll go back to bed. Have fun people, cyu later :-)
|
[4:15] | Nock Forager: | ya next week! |