User:Babbage Linden/Office Hours/2009 05 13

From Second Life Wiki
Jump to navigation Jump to search

Transcript of Babbage Linden's office hours:

 Topics
 * http texture enabled viewer will be RC around august? LINK
 * Some numbers from script memory analysis. LINK
   - Top memory usage per m^2 is 16 m^2 parcel using 65MB of memory... LINK
   - dividing the attachment script size by 40 gives 3MB per avatar LINK


[3:17] WolfPup Lowenhar: hey babbage

[3:18] Qie Niangao: g'mornin', Babbage. :)

[3:18] Nock Forager: Hi Babbage

[3:18] Babbage Linden: morning all

[3:18] Gunter Vandyke: Hi Babbage

[3:18] Xugu Madison: Morning Babbage, apologies for the nudge

[3:18] Uchi Desmoulins: Morning!

[3:18] Wilma Philbin: morning Babbage

[3:18] Sandling Honey: Morning!

[3:18] Aeron Kohime: Morning Babbage

[3:18] wesley Saunders: morning

[3:18] Babbage Linden: how's everything going?

[3:19] Xugu Madison: Quiet :)

[3:19] Imaze Rhiano: good thank you - how about you?

[3:19] WolfPup Lowenhar: it is going

[3:19] Gunter Vandyke: good thx

[3:19] Nock Forager: RC1 works fine. Anyone tried Adult related scripts?

[3:19] wesley Saunders: fine thx

[3:19] Babbage Linden: so, this week we've mostly been working on http textures

[3:19] Babbage Linden: and have nearly got that finished

[3:20] Gunter Vandyke: great

[3:20] Babbage Linden: we're just wiring up some integration tests for it

[3:20] Aeron Kohime: Doing well, the per pixel lighting in the RC is nice.

[3:20] Babbage Linden: and documenting the interface before we hand it over to the viewer team

[3:20] Xugu Madison: http textures - that's just transfer of in-world textures over HTTP, not using external textures by URL, yet, right?

[3:20] Babbage Linden: so, it should be used by RC viewers around august i think

[3:20] Aeron Kohime: So we will be able to host our own textures soon?

[3:21] Babbage Linden: xugu, no just in world textures at the moment

[3:21] wesley Saunders: nice

[3:21] Babbage Linden: although it may allow off world hosted textures at some point in the future

[3:21] Aeron Kohime nods.

[3:21] Babbage Linden: and also may allow us to host frequently used textures using cdns

[3:22] Xugu Madison was hoping to be able to have textures passed through the university proxy, for caching, for example

[3:22] Uchi Desmoulins: I can't wait for http textures..

[3:23] Aeron Kohime: it'll improve some services such as HippoVend, you can see the textures online

[3:23] Babbage Linden: we've got some more results from the script memory usage analysis too

[3:24] Babbage Linden: it looks as though the maximum memory used by attachments is only about 2MB for an entire region with 40+ avatars connected

[3:24] Babbage Linden: so that's good news

[3:24] Babbage Linden: it probably means we can set the avatar script memory pools at a size which allows all current attachment usage to work fine

[3:24] Uchi Desmoulins: Uhh D:

[3:24] Aeron Kohime: That probably varies in some sims which use things like DCS or Samurai Island

[3:24] Uchi Desmoulins: Sims disappearing

[3:25] Wilma Philbin: Will we be able to see how much memory is used?

[3:25] Imaze Rhiano: only 2mb? that is surprising... there wasn't anyone with 255 prim hair and resize script in SIM? :P

[3:25] Babbage Linden: i'll dig in to the numbers further, but that's the headline from xan

[3:26] Aeron Kohime: Well tis good news atleast

[3:26] Babbage Linden: that may have been 1 avatar with 2MB of scripts attached and 39 avatars with no attachments though

[3:26] Aeron Kohime: Hopefully SL dodged the memory per avatar limitation bullet.

[3:26] Babbage Linden: i don't think our analysis digs in to individual avatar script usage

[3:27] Babbage Linden: just the attachment memory usage for a region

[3:27] Aeron Kohime: So it was just one sim tested?

[3:27] Babbage Linden: no, all the regions on the grid

[3:28] Aeron Kohime: Ahh

[3:28] Gunter Vandyke: i have arround 8000 active scripts in my region when 60 ppl are there

[3:28] Imaze Rhiano: maybe there was 39 traffic bots and one customer :P

[3:28] Babbage Linden: possibly

[3:29] Aeron Kohime: Well fi its for all the regions I am guessing either some kind of average or the max

[3:29] Gunter Vandyke: no bots

[3:29] Babbage Linden: we've also got the per parcel stats now

[3:30] Xugu Madison: how are they looking?

[3:30] Babbage Linden: the top memory usage per m^2 is a 16 m^2 parcel using 65MB of memory...

[3:31] Saijanai Kuhn: have you gotten http texture working in pyogp yet?

[3:31] Babbage Linden: 411KB per m^2

[3:31] Qie Niangao: hehehe... somebody was having fun with that 16

[3:32] Babbage Linden: there are 51000 parcels using more than 1KB / m^2

[3:32] Babbage Linden: 25000 using more than 2KB / m^2

[3:33] Imaze Rhiano: prim count would be better measurement, because some parcels might have double prims

[3:33] Babbage Linden: 12000 using more than 4KB / m^2

[3:34] Babbage Linden: so, now we can start looking at these numbers to work out good script limit values

[3:34] Aeron Kohime: limiting memory on parcel via prim allowance? Sounds interesting, maybe land can have a multiplier for memory like they do prims now.

[3:34] Xugu Madison: Sounding like it would make sense to bring in a cap at 8KB/m^2, and slowly drop it down towards 2-4KB/M^2

[3:34] Babbage Linden: that reduce simulators swapping

[3:34] Babbage Linden: but don't stop reasonable uses of SL working

[3:35] Wilma Philbin: Will we be given a way to see how much memory we use

[3:35] Babbage Linden isn't sure using 65MB of memory for a 16 m^2 parcel is reasonable...

[3:35] Wilma Philbin: ?

[3:35] Babbage Linden: wilma, yes

[3:35] Babbage Linden: the first step is to work out good numbers for the script limits

[3:36] Wilma Philbin: Like the script time - in region Debug window?

[3:36] Babbage Linden: then to add UI to show memory usage and find script using memory

[3:36] Aeron Kohime: Well, is that kb based on m^2 so, about 512 kb a region for that 4kb, or is my math all messed up

[3:36] Babbage Linden: then to add the enforcement

[3:36] Xugu Madison: Babbage, how about putting the UI in, so people can start seeing how much they're using and re-arrange before the limits come in?

[3:36] Wilma Philbin: Sounds like a good plan ㋡

[3:36] Aeron Kohime: my math is messed up

[3:36] Qie Niangao: Babbage, that 16 was a one-of, right? not something that appeared in a lot of sims, over and over?

[3:36] Aeron Kohime: oh wait nevermind

[3:37] Babbage Linden: if we went with 1KB / m^2 a 512 m ^2 parcel would get 512 KB of memory

[3:37] Babbage Linden: no, it's a 1 off

[3:37] Aeron Kohime: I like the idea of having a UI before hand to warn users

[3:37] Qie Niangao: :)

[3:37] Babbage Linden: almost twice the memory usage / m^ 2 of the next highest parcel

[3:37] Xugu Madison: I think a lot of people might self-fix if they knew...

[3:38] Babbage Linden: yes, i think they will xugu

[3:38] Babbage Linden: i think we'll see a big spring clean of SL :-D

[3:38] WolfPup Lowenhar: script mem usage would be nice for land owners to be able to see not just estate managers

[3:38] Nock Forager: +1 to Xugu.

[3:38] Qie Niangao: speaking of self-fixing...

[3:38] Qie Niangao: A naive question: If a script is set to not-running state (as with llSetScriptState(.., FALSE)), does it still use sim memory? (I'm guessing it's still loaded, but hopefully can get paged out and not contribute to further swapping... but not sure)

[3:39] Aeron Kohime: Babbage: Alright I see what you mean, so at 1kb a full region would get about 65 mb.

[3:39] Aeron Kohime: or 64 MiB

[3:39] Babbage Linden: ok, so the attachment number was / m ^ 2, which is wrong

[3:40] Babbage Linden: dividing the attachment script size by 40 gives 3MB per avatar

[3:40] Babbage Linden: which is very high

[3:40] Aeron Kohime: That does sound high

[3:41] Qie Niangao: wow... that's a lot different!

[3:41] Imaze Rhiano: sounds about right :P

[3:41] Babbage Linden: heh

[3:41] Uchi Desmoulins: Woo Draconis

[3:41] Babbage Linden: do, we'll have to look in to a good per avatar number in the same way

[3:41] Aeron Kohime: so about 48 to 192 scripts per avatar

[3:42] Uchi Desmoulins: 200 is pretty common for the people coming and going from my sim

[3:42] Babbage Linden: at the 99th percentile regions are using 560KB per avatar

[3:42] wesley Saunders: but i think this is growing

[3:43] Babbage Linden: at the 95th percentile regions are using 257 KB per avatar

[3:43] Babbage Linden: but i'll have to dig in to this data further to get good numbers

[3:44] Babbage Linden: these are all approximate

[3:45] Babbage Linden: so, hopefully i'll have better numbers to talk about next week

[3:45] Babbage Linden: and i'll spend some time talking them over with jack

[3:46] Qie Niangao: Jack may wonder if there's a difference between Mainland and Estates

[3:46] Babbage Linden: yes, i'm sure he will :-D

[3:48] Babbage Linden: the other thing i spent some time on was playing around with search with SignpostMarv

[3:48] Babbage Linden: the idea was to present it as a hack at open hack day london

[3:48] Babbage Linden: but in the event there wan't any net access at open hack london

[3:49] Babbage Linden: so i put together an iphone orchestra instead: http://jimpurbrick.com/2009/05/12/london-geek-community-iphone-oscestra/

[3:49] Babbage Linden: but, the idea with the search hack is that there are now 180,000 slurls on the web, at least

[3:49] Babbage Linden: embedded in web pages that have a page rank score and can be searched

[3:50] Babbage Linden: so we were looking at using search engines to find pages containing slurls

[3:50] Babbage Linden: based on a search term

[3:50] Babbage Linden: as a way of finding things in SL

[3:50] Saijanai Kuhn: that was you? very kool

[3:50] Babbage Linden: the next thought was that SL is continuous unlike the web

[3:51] Saijanai Kuhn is playing with iphone programming right now. Got the xmlrpc to send commands to a prim

[3:51] Babbage Linden: you might have slurls pointing to points right next to each other

[3:51] Babbage Linden: they are really the same "place" in SL

[3:51] Babbage Linden: but the slurls end up being slightly different

[3:51] Babbage Linden: so, you should be able to use a fuzzy comparison to collapse together search results

[3:52] Babbage Linden: for example, treat all slurls within 10m of each other as the same slurl

[3:52] Xugu Madison: Babbage, well, don't parcel URLs work for this?

[3:52] Xugu Madison: http://world.secondlife.com/place/df0663a6-a0b1-9574-617b-b3d9062a1e7c for example

[3:52] Infinity Gears departs hoping you find the time to find his missing roughly 700 U.S. dollars of inventory that your techs just shrugged off. tata

[3:52] Babbage Linden: then you should be able to combine the page rank scores to get better results

[3:52] Saijanai Kuhn is confused. aren't teh SLURLs based on UUIDs?

[3:53] Aeron Kohime: How about a type of clustering? Instead of purely physical location but also with parcel name and and the like?

[3:53] Babbage Linden: xugu, yes place URLs work like that

[3:53] Babbage Linden: but there are lots of X/Y/Z based slurls on the web waiting to be searched

[3:53] Xugu Madison: Good point

[3:53] Babbage Linden: so, we were looking at doing that

[3:54] Xugu Madison: HOping I can get one last thing in before you wrap up; any news on the C# API docs, you were apparently going to have a meeting Friday?

[3:54] Babbage Linden: now hackday is passed and I'm back to being super busy

[3:54] Aeron Kohime: for crowded shop areas, 10m might to to much so speration for sperate parcels, but if they are in the same parcel more then 10 meters might be acceptable.

[3:54] Qie Niangao: well, the XYZs are within-sim... so there's an assumed (and probably valid) discontinuity at sim borders... if it matters.

[3:54] Saijanai Kuhn: C# as in mono-scripting for prims?

[3:54] Babbage Linden: but if you'd like to work on that I'm sure SignpostMarv would love to hear from you

[3:55] Nock Forager: I asked marv to add delicious search on it too. But it's bit difficult to do so ;-)

[3:55] Xugu Madison nods to Sai

[3:55] Aeron Kohime: Now that is more interesting to talk about C# scriptin in SL would be amazing, but there are security reasons for not alloing it right

[3:55] Babbage Linden: xugu, yes that meeting happened#

[3:55] Babbage Linden: but i wasn't there

[3:55] Saijanai Kuhn: ooh. And that's in the documentation stage already?

[3:55] Babbage Linden: i'll follow up with robla and let you know what the outcome was

[3:55] Xugu Madison: Thanks!

[3:56] Babbage Linden: saijanai, there is a C# API already

[3:56] Saijanai Kuhn is confused

[3:56] Babbage Linden: used by Scripts compiled to Mono

[3:56] Babbage Linden: and it's likely going to be the same API used by C# scripts

[3:56] Babbage Linden: so, we could open source that API now

[3:56] Aeron Kohime: So scripts get the C# compilers optimizations?

[3:56] Saijanai Kuhn: ah, the CLR stuff, but that's compiled server-side and only limited to LSL-like constructs

[3:56] Babbage Linden: to allow opensim and others to be able to build towards compatibility

[3:57] Babbage Linden: at the moment the opensim c# API is different

[3:57] Xugu Madison nods to Sai "But, they're working on bytecode upload, once the Mono bytecode verifier is there"

[3:57] Babbage Linden: mostly because they can't see our API yet

[3:57] Saijanai Kuhn: there's no C#-style string/array processing for example

[3:57] Xugu Madison: Sai, which is why I want them to hurry up :)

[3:57] Babbage Linden: saiganai, no, but there is a ScriptTypes assembly for example

[3:58] Saijanai Kuhn: hmmm. Accesible via LSL?

[3:58] Babbage Linden: which defines the LSL Vector, Quaternion, Key and List types

[3:58] Wilma Philbin: Does that mean we will eventually get proper arrays to work with in scripts?

[3:58] Babbage Linden: those are the types used by LSL scripts compiled to Mono

[3:58] Saijanai Kuhn: and objects and classes?

[3:58] Babbage Linden: and will be used by C# when we make that available

[3:58] Babbage Linden: if we open source those APIs now

[3:58] Saijanai Kuhn: ah, OK, predefined types. but they map only to the LSL functionality.

[3:59] Babbage Linden: then opensim can use them and work towards compatibility

[3:59] Babbage Linden: so that when we allow C#, there is hopefully a shared API

[3:59] Babbage Linden: and your opensim scripts will work in SL

[3:59] Babbage Linden: and vice versa

[3:59] Aeron Kohime: So we wille ventually get C#?

[3:59] Babbage Linden: that's my hope, yes

[3:59] Xugu Madison: Babbage, and means we'll probably see scripts getting pre-tested on OpenSim

[3:59] Saijanai Kuhn: and.. it seems like LSL lists are quite inefficient as they're implemented right now with LSL => C#

[3:59] Qie Niangao: I'm so confused now. Why would there ever be a reason anymore for bytecode upload? or is this for something other than scripts?

[4:00] WolfPup Lowenhar: < is still trying to get an opensim going

[4:00] Xugu Madison: Qie, because LSL is a half-crippled language lacking a lot of features

[4:00] Babbage Linden: qie, bytecode upload allows arbitrary languages

[4:00] Saijanai Kuhn: one assumes that direct compiling will allow more efficient strings/arrays than the LSL emulation mode?

[4:01] Babbage Linden: c# and lsl are likely to be first

[4:01] Babbage Linden: then maybe arbitrary languages

[4:01] Saijanai Kuhn: thought you were hoping for an assembler first thing

[4:01] Babbage Linden: c# would definitely allow arrays, as that's part of the language

[4:02] Imaze Rhiano: Have you had time to follow Open SIM's MRM scripting yet? http://www.maxping.org/technology/platforms/open-simulator/mrm-scripting---coming-soon.aspx

[4:02] Babbage Linden: and lots of the other things that people want in LSL

[4:02] Saijanai Kuhn: Babage, right, so it wouldn't need to be translated into an LSL compatible type, but directly use array functions