User:Babbage Linden/Office Hours/2009 04 01

From Second Life Wiki
< User:Babbage Linden/Office Hours
Revision as of 07:04, 1 April 2009 by Nock Forager (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Transcript of Babbage Linden's office hours: 

 Topics
 * Lots of people are concerned about the reduction in performance of list processing
   commands. (Caused by Mono scheduler change) LINK
 * on the one hand it is actually just fixing a bug: lists shouldn't
   have been able to process that quickly. LINK
 * the model we want is that objects can tell us what their prim
   and script usage is going to be upfront LINK


[3:41] WolfPup Lowenhar: morning babbage

[3:41] Nock Forager: I saw something

[3:41] Babbage Linden: hi everyone

[3:41] Morgaine Dinova: Hiya Babbage!

[3:41] Babbage Linden: sorry, i'm late

[3:41] Babbage Linden: got caught up with things here

[3:41] WolfPup Lowenhar: that is ok babbage

[3:41] Morgaine Dinova: How are things?

[3:41] Babbage Linden: thanks for hanging around

[3:41] Babbage Linden: good thanks

[3:41] WolfPup Lowenhar: np i just found a bug in thr rc!!

[3:41] Babbage Linden: i'd like to talk about the scheduler beta today if that's ok

[3:41] WolfPup Lowenhar: rc11

[3:42] Babbage Linden: have any of you tried your scripts out on it yet?

[3:42] Morgaine Dinova: Cool

[3:42] WolfPup Lowenhar: i hven not been to the bata grid at all

[3:42] Nock Forager: Tried on few. No problems.

[3:42] Babbage Linden: that's good to hear nock

[3:43] Babbage Linden: here's the public jira feedback:

[3:43] Babbage Linden: http://jira.secondlife.com/browse/SVC-3997

[3:43] WolfPup Lowenhar: the only problem is that main scripts i wou want to check i cant because there server is on aginin and not on aditi

[3:43] Babbage Linden: lots of people are concerned about the reduction in performance of list processing commands

[3:44] Babbage Linden: that's a good point wolfpup, we should try to get the agni data copied over so people have their new scripts to try

[3:44] WolfPup Lowenhar: < has a vendor that dose a lot of list proccessing

[3:44] Morgaine Dinova: Take the opportunity to add O(1) arrays to LSL then .... <grin>

[3:44] WolfPup Lowenhar: and the vendors server is on agini

[3:45] Babbage Linden: so, there are 2 things that might be affected: memory, where scripts temporarily use too much memory and speed, where scripts used to get away with calling expensive library calls many times in the same time slice

[3:46] WolfPup Lowenhar: would there be a way to get a private region coppied to aditi ?

[3:46] Babbage Linden: it's possible wolfpup, but we'd have to do it manually, so try to avoif it

[3:46] Babbage Linden: best thing to do is to rez objects from inventory

[3:46] Morgaine Dinova: Babbage: how about letting people activate their own inventory copies to Aditi, eg. simply by entering there?

[3:47] Babbage Linden: as we can copy the agni data over to aditi to ensure you have all your inventory available

[3:47] Babbage Linden: that would be handy morgaine, but it's a manual process unfortunately

[3:47] Babbage Linden: also, it wipes out any inventory you previously had on aditi

[3:48] Babbage Linden: which admiitedly shouldn't often be a problem

[3:48] Babbage Linden: but sometimes you will want to develop something on aditi using a new feature

[3:48] WolfPup Lowenhar: most of the inventory i have on aditi is the same as on agini

[3:48] Babbage Linden: over multiple sessions, like with http-in

[3:48] Babbage Linden: anyway, there is lots of concern about the slow down in list processing speed with the scheduler changed

[3:48] Babbage Linden: changes

[3:49] Morgaine Dinova: Is that slow-down inevitable owing to the change, or just not optimized yet?

[3:49] Babbage Linden: on the one hand it is actually just fixing a bug: lists shouldn't have been able to process that quickly

[3:49] Morgaine Dinova: Oh dear

[3:49] Babbage Linden: the slow down is because previously it was possible to call library functions that took a long time several times in a time slice

[3:49] Morgaine Dinova: Well that's a disaster, if you're implying that "it was too quick before"

[3:50] Babbage Linden: in the worst case it can take 300us to copy a list from managed code to C++ to have a function operate on it

[3:50] Babbage Linden: and a time slice is 150us

[3:50] Morgaine Dinova: In that case add O(1) arrays to LSL. No question.

[3:50] Babbage Linden: so, you "should" only be able to make one list call per time slice if the scheduler is working properly

[3:50] Babbage Linden: which it now is

[3:51] Morgaine Dinova: Well that's a disaster.

[3:51] Babbage Linden: on the upside list processing is still faster than LSL

[3:51] Babbage Linden: on the downside, it's now a lot slower than it used to be under mono

[3:51] Babbage Linden: a relatively easy way to improve the performance is to avoid the copy to unmanged code

[3:51] Babbage Linden: which causes a lot of marshalling to happen

[3:52] WolfPup Lowenhar: now i realy need to goto aditi to check my vendors :/

[3:52] Babbage Linden: and boxing/unboxing as the mono representation is converted to a C++ list representation expected by the library calls

[3:52] Babbage Linden: one way to avoid that overhead is to do the list processing in managed code

[3:52] Babbage Linden: reimplement the functions that don't need to access simulator data in C#

[3:53] Babbage Linden: but that's pretty horrible from a maintenance perspective as we end up with 2 implementations of the library functions

[3:53] Babbage Linden: which could get out of sync

[3:53] Babbage Linden: so functions would behave differently when called from LSO or Mono scripts

[3:53] Babbage Linden: that would be a bug, but it would be possible

[3:53] Babbage Linden: if there were 2 implementations

[3:54] Morgaine Dinova: I rather think the point is being missed. List traversal is not the end goal --- data access is. You're treating list traversal as a resource-hungry operation, when it's just a means to an end for people. Provide O(1) arrays that provide element access with zero resource usage and the problem disappears.

[3:54] Babbage Linden: morgaine, i agree, we should have arrays

[3:54] WolfPup Lowenhar: true morgaine

[3:54] Babbage Linden: in fact, i'll go further and say that we should have C# with arrays, objects and other useful stuff

[3:54] WolfPup Lowenhar: multople elemt arays work much easier

[3:55] Babbage Linden: i really don't want to have to extend LSL and make it work for LSO and Mono when it's probably less work to add support for C#

[3:55] Babbage Linden: but anyway, that's getting somewhat away from the subject

[3:56] Babbage Linden: the other issue with the current scheduler beta is that we have a number of other potential changes in the works

[3:56] Babbage Linden: the first is the script resource limits project

[3:56] WolfPup Lowenhar: a vewnodr i use would work much better if it had an aO(1,1,1) aray to handle the data

[3:56] Babbage Linden: and the other is our planned move to 64 bit

[3:56] Babbage Linden: which will mean using a 64 bit Mono

[3:57] Babbage Linden: and having to change the Mono memory limits to accommodate that

[3:57] WolfPup Lowenhar: im guessing that would improve script performance

[3:57] Babbage Linden: they are somewhat related

[3:57] Babbage Linden: as we talked about last week, if we introduce script limits

[3:58] Babbage Linden: and make Mono scripts cost 64KB and LSO scripts cost 16KB, people will be incentivised to use LSO

[3:58] WolfPup Lowenhar: how long befor the limits go into efect?

[3:58] Babbage Linden: which we don't want, because on average Mono scripts use less memory and CPU than LSO scripts

[3:58] WolfPup Lowenhar: i also rember you talking about having the scripts request memory

[3:58] Babbage Linden: if we move to 64 bit, the extra size used by Mono references will mean that we will probably need to give even more memory to Mono scripts

[3:59] Babbage Linden: so, in the worst case Mono scripts end up "costing" 112KB and LSO 16KB

[3:59] Babbage Linden: the incentives are even more out of whack

[3:59] Morgaine Dinova: Hmm

[3:59] Babbage Linden: yes, wolfpup, i did talk about scripts reserving memory

[4:00] Babbage Linden: so, one way to get things back in sync again would be to add the script memory reservation function

[4:00] WolfPup Lowenhar: that would give people more incentive to use mono if they could set the mem a script uses

[4:00] Babbage Linden: so people can change Mono scripts to request a maximum below 16KB

[4:00] Babbage Linden: which would make them cheaper than LSO scripts

[4:01] Babbage Linden: in some cases LSO would genuinely be the best choice

[4:01] WolfPup Lowenhar: because for a simple dor scrit that would only need say 2k 5they could set it to request it

[4:01] Babbage Linden: but the memory reservation functions would allow you to tell the system about the true cost of your script

[4:01] Babbage Linden: and it wouldn't have to always use the worst case cost

[4:01] Morgaine Dinova: Needing to express how much memory a program needs takes us back to the 60's though. Can't that be hidden?

[4:02] WolfPup Lowenhar: the problem would be knowing the true cost

[4:02] Babbage Linden: wolfpup, yes, it's hard to work out the true cost at the momemt

[4:02] Babbage Linden: all you really get is a stack/heap collision telling you that you need to use less memory

[4:02] Babbage Linden: or add another script

[4:02] Babbage Linden: and use link messages

[4:03] WolfPup Lowenhar: morgain being able to request memory is still in ues in some systems today

[4:03] Babbage Linden: so, we potentially also need better ways to report current memory usage

[4:04] Babbage Linden: as you can see, we're ending up with a number of things

[4:04] Babbage Linden: scheduler, 64 bit and script limits

[4:04] Babbage Linden: that are all related

[4:04] Babbage Linden: and we don't want to ask everyone to test their scripts 3 times

[4:04] Babbage Linden: so, the current thinking is that we'll roll the projects together

[4:05] WolfPup Lowenhar: and that can realy create a migrain for some one trying to keep them from creating bugs in the others :)

[4:05] Morgaine Dinova: If a Mono script can use less memory than an LSO, why not only allocate 8KB or less to start with, and if it's not enough then restart in a larger VM behind the scenes? It'll only happen on startup, and at most a few times.

[4:05] Babbage Linden: and release them all together to ensure that you only need to test once

[4:06] Babbage Linden: morgaine, the model we want is that objects can tell us what their prim and script usage is going to be upfront

[4:06] Babbage Linden: so we can either say, sorry you can't rez

[4:06] Babbage Linden: or say, go ahead

[4:06] Babbage Linden: knowing that the object will work fine

[4:06] Babbage Linden: we don't wa

[4:06] Babbage Linden: nt to allow scipts to rez, grow and then get to the point where the sim is swapping

[4:07] Babbage Linden: this memory model is unlike most modern operating systems

[4:07] WolfPup Lowenhar: < has seen a lot of sim swaping latly for the sim he comanages

[4:07] Babbage Linden: which will use virtual memory to allow you to allocate a lot of memory

[4:08] Babbage Linden: we specifically want to avoid using virtual memory

[4:08] Babbage Linden: as having the sim swap is really bad news

[4:08] Babbage Linden: so, we ask scripts to tell us up front how much memory they will use

[4:08] Morgaine Dinova: I still think it has to be automatic, can't expect users to know.

[4:08] Babbage Linden: morgaine, agreed

[4:08] Babbage Linden: we need a model where scripters will set their scripts to request 8KB

[4:08] Babbage Linden: then put them in a vendor

[4:09] Babbage Linden: then sell them saying "requires 8KB of memory"

[4:09] Babbage Linden: and have residents just rez them

[4:09] Babbage Linden: anyway, I'm going to have to run

[4:09] Babbage Linden: as i have another meeting

[4:10] WolfPup Lowenhar: morgaine if coders were able to find out the max their script used when they have a final product ready to put out then they would be able to have the script alocate it

[4:10] Babbage Linden: but, the news for this week is: there are a few scripting changes coming up, we don't want to ask people to test scripts every 5 minutes, so we'll likely roll them together

[4:10] Babbage Linden: and none of these changes will get to agni in the near future

[4:10] Babbage Linden: end of the year is more likely

[4:10] Morgaine Dinova: Wolf: doesn't look right to me. That's what computers are for, to determine resource usage.

[4:10] WolfPup Lowenhar < still like having things run 'clean'

[4:10] Babbage Linden: so, don't panic about testing scripts right now

[4:10] Babbage Linden: ok?

[4:11] Morgaine Dinova: KK Babbage :-)

[4:11] Babbage Linden: ok, thanks for coming

[4:11] Babbage Linden: got to run

[4:11] Babbage Linden: see you next time

[4:11] Morgaine Dinova: Take care B :-)

[4:11] WolfPup Lowenhar: tc babbage

[4:11] Nock Forager: Seeya Babbage.

[4:11] WolfPup Lowenhar: ahhh babbage you got a bear?

[4:11] WolfPup Lowenhar < has started a collection :)

[4:12] Morgaine Dinova: Make a bear quest Babbage, like Simon did :-)

[4:12] Babbage Linden gave you Babbage Linden Bear.

[4:12] Morgaine Dinova: Oh cheers B :-))

[4:12] Nock Forager: Got new one :)

[4:12] Babbage Linden: seeya!

[4:12] WolfPup Lowenhar: Thank Youuuuuu!! for the tip

[4:12] Morgaine Dinova waves

[4:13] WolfPup Lowenhar: grrr

[4:13] Morgaine Dinova: LOL @ tip Wolf :P

[4:13] Morgaine Dinova: Better change that gesture :P

[4:15] Morgaine Dinova: Think I'll go back to bed. Have fun people, cyu later :-)

[4:15] Nock Forager: ya next week!
Retrieved from "https://wiki.secondlife.com/w/index.php?title=User:Babbage_Linden/Office_Hours/2009_04_01&oldid=299263"