User:Babbage Linden/Office Hours/2009 04 01

From Second Life Wiki
< User:Babbage Linden/Office Hours
Revision as of 07:04, 1 April 2009 by Nock Forager (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Transcript of Babbage Linden's office hours: 

 Topics
 * Lots of people are concerned about the reduction in performance of list processing
   commands. (Caused by Mono scheduler change) LINK
 * on the one hand it is actually just fixing a bug: lists shouldn't
   have been able to process that quickly. LINK
 * the model we want is that objects can tell us what their prim
   and script usage is going to be upfront LINK


[3:41] WolfPup Lowenhar: morning babbage

[3:41] Nock Forager: I saw something

[3:41] Babbage Linden: hi everyone

[3:41] Morgaine Dinova: Hiya Babbage!

[3:41] Babbage Linden: sorry, i'm late

[3:41] Babbage Linden: got caught up with things here

[3:41] WolfPup Lowenhar: that is ok babbage

[3:41] Morgaine Dinova: How are things?

[3:41] Babbage Linden: thanks for hanging around

[3:41] Babbage Linden: good thanks

[3:41] WolfPup Lowenhar: np i just found a bug in thr rc!!

[3:41] Babbage Linden: i'd like to talk about the scheduler beta today if that's ok

[3:41] WolfPup Lowenhar: rc11

[3:42] Babbage Linden: have any of you tried your scripts out on it yet?

[3:42] Morgaine Dinova: Cool

[3:42] WolfPup Lowenhar: i hven not been to the bata grid at all

[3:42] Nock Forager: Tried on few. No problems.

[3:42] Babbage Linden: that's good to hear nock

[3:43] Babbage Linden: here's the public jira feedback:

[3:43] Babbage Linden: http://jira.secondlife.com/browse/SVC-3997

[3:43] WolfPup Lowenhar: the only problem is that main scripts i wou want to check i cant because there server is on aginin and not on aditi

[3:43] Babbage Linden: lots of people are concerned about the reduction in performance of list processing commands

[3:44] Babbage Linden: that's a good point wolfpup, we should try to get the agni data copied over so people have their new scripts to try

[3:44] WolfPup Lowenhar: < has a vendor that dose a lot of list proccessing

[3:44] Morgaine Dinova: Take the opportunity to add O(1) arrays to LSL then .... <grin>

[3:44] WolfPup Lowenhar: and the vendors server is on agini

[3:45] Babbage Linden: so, there are 2 things that might be affected: memory, where scripts temporarily use too much memory and speed, where scripts used to get away with calling expensive library calls many times in the same time slice

[3:46] WolfPup Lowenhar: would there be a way to get a private region coppied to aditi ?

[3:46] Babbage Linden: it's possible wolfpup, but we'd have to do it manually, so try to avoif it

[3:46] Babbage Linden: best thing to do is to rez objects from inventory

[3:46] Morgaine Dinova: Babbage: how about letting people activate their own inventory copies to Aditi, eg. simply by entering there?

[3:47] Babbage Linden: as we can copy the agni data over to aditi to ensure you have all your inventory available

[3:47] Babbage Linden: that would be handy morgaine, but it's a manual process unfortunately

[3:47] Babbage Linden: also, it wipes out any inventory you previously had on aditi

[3:48] Babbage Linden: which admiitedly shouldn't often be a problem

[3:48] Babbage Linden: but sometimes you will want to develop something on aditi using a new feature

[3:48] WolfPup Lowenhar: most of the inventory i have on aditi is the same as on agini

[3:48] Babbage Linden: over multiple sessions, like with http-in

[3:48] Babbage Linden: anyway, there is lots of concern about the slow down in list processing speed with the scheduler changed

[3:48] Babbage Linden: changes

[3:49] Morgaine Dinova: Is that slow-down inevitable owing to the change, or just not optimized yet?

[3:49] Babbage Linden: on the one hand it is actually just fixing a bug: lists shouldn't have been able to process that quickly

[3:49] Morgaine Dinova: Oh dear

[3:49] Babbage Linden: the slow down is because previously it was possible to call library functions that took a long time several times in a time slice

[3:49] Morgaine Dinova: Well that's a disaster, if you're implying that "it was too quick before"

[3:50] Babbage Linden: in the worst case it can take 300us to copy a list from managed code to C++ to have a function operate on it

[3:50] Babbage Linden: and a time slice is 150us

[3:50] Morgaine Dinova: In that case add O(1) arrays to LSL. No question.

[3:50] Babbage Linden: so, you "should" only be able to make one list call per time slice if the scheduler is working properly

[3:50] Babbage Linden: which it now is

[3:51] Morgaine Dinova: Well that's a disaster.

[3:51] Babbage Linden: on the upside list processing is still faster than LSL

[3:51] Babbage Linden: on the downside, it's now a lot slower than it used to be under mono

[3:51] Babbage Linden: a relatively easy way to improve the performance is to avoid the copy to unmanged code

[3:51] Babbage Linden: which causes a lot of marshalling to happen

[3:52] WolfPup Lowenhar: now i realy need to goto aditi to check my vendors :/

[3:52] Babbage Linden: and boxing/unboxing as the mono representation is converted to a C++ list representation expected by the library calls

[3:52] Babbage Linden: one way to avoid that overhead is to do the list processing in managed code

[3:52] Babbage Linden: reimplement the functions that don't need to access simulator data in C#

[3:53] Babbage Linden: but that's pretty horrible from a maintenance perspective as we end up with 2 implementations of the library functions

[3:53] Babbage Linden: which could get out of sync

[3:53] Babbage Linden: so functions would behave differently when called from LSO or Mono scripts

[3:53] Babbage Linden: that would be a bug, but it would be possible

[3:53] Babbage Linden: if there were 2 implementations

[3:54] Morgaine Dinova: I rather think the point is being missed. List traversal is not the end goal --- data access is. You're treating list traversal as a resource-hungry operation, when it's just a means to an end for people. Provide O(1) arrays that provide element access with zero resource usage and the problem disappears.

[3:54] Babbage Linden: morgaine, i agree, we should have arrays

[3:54] WolfPup Lowenhar: true morgaine

[3:54] Babbage Linden: in fact, i'll go further and say that we should have C# with arrays, objects and other useful stuff

[3:54] WolfPup Lowenhar: multople elemt arays work much easier

[3:55] Babbage Linden: i really don't want to have to extend LSL and make it work for LSO and Mono when it's probably less work to add support for C#

[3:55] Babbage Linden: but anyway, that's getting somewhat away from the subject

[3:56] Babbage Linden: the other issue with the current scheduler beta is that we have a number of other potential changes in the works

[3:56] Babbage Linden: the first is the script resource limits project

[3:56] WolfPup Lowenhar: a vewnodr i use would work much better if it had an aO(1,1,1) aray to handle the data

[3:56] Babbage Linden: and the other is our planned move to 64 bit

[3:56] Babbage Linden: which will mean using a 64 bit Mono

[3:57] Babbage Linden: and having to change the Mono memory limits to accommodate that

[3:57] WolfPup Lowenhar: im guessing that would improve script performance

[3:57] Babbage Linden: they are somewhat related

[3:57] Babbage Linden: as we talked about last week, if we introduce script limits

[3:58] Babbage Linden: and make Mono scripts cost 64KB and LSO scripts cost 16KB, people will be incentivised to use LSO

[3:58] WolfPup Lowenhar: how long befor the limits go into efect?

[3:58] Babbage Linden: which we don't want, because on average Mono scripts use less memory and CPU than LSO scripts

[3:58] WolfPup Lowenhar: i also rember you talking about having the scripts request memory

[3:58] Babbage Linden: if we move to 64 bit, the extra size used by Mono references will mean that we will probably need to give even more memory to Mono scripts

[3:59] Babbage Linden: so, in the worst case Mono scripts end up "costing" 112KB and LSO 16KB

[3:59] Babbage Linden: the incentives are even more out of whack

[3:59] Morgaine Dinova: Hmm

[3:59] Babbage Linden: yes, wolfpup, i did talk about scripts reserving memory

[4:00] Babbage Linden: so, one way to get things back in sync again would be to add the script memory reservation function

[4:00] WolfPup Lowenhar: that would give people more incentive to use mono if they could set the mem a script uses

[4:00] Babbage Linden: so people can change Mono scripts to request a maximum below 16KB

[4:00] Babbage Linden: which would make them cheaper than LSO scripts

[4:01] Babbage Linden: in some cases LSO would genuinely be the best choice

[4:01] WolfPup Lowenhar: because for a simple dor scrit that would only need say 2k 5they could set it to request it

[4:01] Babbage Linden: but the memory reservation functions would allow you to tell the system about the true cost of your script

[4:01] Babbage Linden: and it wouldn't have to always use the worst case cost

[4:01] Morgaine Dinova: Needing to express how much memory a program needs takes us back to the 60's though. Can't that be hidden?

[4:02] WolfPup Lowenhar: the problem would be knowing the true cost

[4:02] Babbage Linden: wolfpup, yes, it's hard to work out the true cost at the momemt

[4:02] Babbage Linden: all you really get is a stack/heap collision telling you that you need to use less memory

[4:02] Babbage Linden: or add another script

[4:02] Babbage Linden: and use link messages

[4:03] WolfPup Lowenhar: morgain being able to request memory is still in ues in some systems today

[4:03] Babbage Linden: so, we potentially also need better ways to report current memory usage

[4:04] Babbage Linden: as you can see, we're ending up with a number of things

[4:04] Babbage Linden: scheduler, 64 bit and script limits

[4:04] Babbage Linden: that are all related

[4:04] Babbage Linden: and we don't want to ask everyone to test their scripts 3 times

[4:04] Babbage Linden: so, the current thinking is that we'll roll the projects together

[4:05] WolfPup Lowenhar: and that can realy create a migrain for some one trying to keep them from creating bugs in the others :)

[4:05] Morgaine Dinova: If a Mono script can use less memory than an LSO, why not only allocate 8KB or less to start with, and if it's not enough then restart in a larger VM behind the scenes? It'll only happen on startup, and at most a few times.

[4:05] Babbage Linden: and release them all together to ensure that you only need to test once

[4:06] Babbage Linden: morgaine, the model we want is that objects can tell us what their prim and script usage is going to be upfront

[4:06] Babbage Linden: so we can either say, sorry you can't rez

[4:06] Babbage Linden: or say, go ahead

[4:06] Babbage Linden: knowing that the object will work fine

[4:06] Babbage Linden: we don't wa

[4:06] Babbage Linden: nt to allow scipts to rez, grow and then get to the point where the sim is swapping

[4:07] Babbage Linden: this memory model is unlike most modern operating systems

[4:07] WolfPup Lowenhar: < has seen a lot of sim swaping latly for the sim he comanages

[4:07] Babbage Linden: which will use virtual memory to allow you to allocate a lot of memory

[4:08] Babbage Linden: we specifically want to avoid using virtual memory

[4:08] Babbage Linden: as having the sim swap is really bad news

[4:08] Babbage Linden: so, we ask scripts to tell us up front how much memory they will use

[4:08] Morgaine Dinova: I still think it has to be automatic, can't expect users to know.

[4:08] Babbage Linden: morgaine, agreed

[4:08] Babbage Linden: we need a model where scripters will set their scripts to request 8KB

[4:08] Babbage Linden: then put them in a vendor

[4:09] Babbage Linden: then sell them saying "requires 8KB of memory"

[4:09] Babbage Linden: and have residents just rez them

[4:09] Babbage Linden: anyway, I'm going to have to run

[4:09] Babbage Linden: as i have another meeting

[4:10] WolfPup Lowenhar: morgaine if coders were able to find out the max their script used when they have a final product ready to put out then they would be able to have the script alocate it

[4:10] Babbage Linden: but, the news for this week is: there are a few scripting changes coming up, we don't want to ask people to test scripts every 5 minutes, so we'll likely roll them together

[4:10] Babbage Linden: and none of these changes will get to agni in the near future

[4:10] Babbage Linden: end of the year is more likely

[4:10] Morgaine Dinova: Wolf: doesn't look right to me. That's what computers are for, to determine resource usage.

[4:10] WolfPup Lowenhar < still like having things run 'clean'

[4:10] Babbage Linden: so, don't panic about testing scripts right now

[4:10] Babbage Linden: ok?

[4:11] Morgaine Dinova: KK Babbage :-)

[4:11] Babbage Linden: ok, thanks for coming

[4:11] Babbage Linden: got to run

[4:11] Babbage Linden: see you next time

[4:11] Morgaine Dinova: Take care B :-)

[4:11] WolfPup Lowenhar: tc babbage

[4:11] Nock Forager: Seeya Babbage.

[4:11] WolfPup Lowenhar: ahhh babbage you got a bear?

[4:11] WolfPup Lowenhar < has started a collection :)

[4:12] Morgaine Dinova: Make a bear quest Babbage, like Simon did :-)

[4:12] Babbage Linden gave you Babbage Linden Bear.

[4:12] Morgaine Dinova: Oh cheers B :-))

[4:12] Nock Forager: Got new one :)

[4:12] Babbage Linden: seeya!

[4:12] WolfPup Lowenhar: Thank Youuuuuu!! for the tip

[4:12] Morgaine Dinova waves

[4:13] WolfPup Lowenhar: grrr

[4:13] Morgaine Dinova: LOL @ tip Wolf :P

[4:13] Morgaine Dinova: Better change that gesture :P

[4:15] Morgaine Dinova: Think I'll go back to bed. Have fun people, cyu later :-)

[4:15] Nock Forager: ya next week!
Retrieved from "https://wiki.secondlife.com/w/index.php?title=User:Babbage_Linden/Office_Hours/2009_04_01&oldid=299263"