User:Which Linden/Office Hours/2009 Feb 5

• [10:59] Morgaine Dinova: You can build your own house in a sandbox sim, but it you want to keep it around in a fixed place of your own, then that costs money in SL, currently.
• [10:59] Saijanai Kuhn: Hey Which
• [11:00] Jabba Aabye: Hi Which
• [11:00] Imaze Rhiano: hi Which
• [11:00] Morgaine Dinova: 'Morning Which :-)
• [11:00] Which Linden: good morning, all!
• [11:00] Melisz Umia: hi
• [11:00] Imaze Rhiano: owning house is not really needed here...
• [11:00] Saijanai Kuhn: Melisz, Which is a Linden Lab employee. that's why he has the last name of Linden
• [11:01] Saijanai Kuhn: he helps design and program the Second Life system
• [11:01] Melisz Umia: so nice
• [11:01] Morgaine Dinova: And this is his Office Hours spot, where we talk technical things about something or other :P
• [11:01] Saijanai Kuhn: this is his office where we are sitting
• [11:02] Saijanai Kuhn: see, not everyone decides to built a house. Which decided to have a bamboo grove with furniture
• [11:02] Which Linden: ...where I spend an inordinate amount of time trying to sit on my own chair
• [11:02] Melisz Umia: cool
• [11:02] Morgaine Dinova: You should make a raised plant bed to sit on Which :-)
• [11:02] Which Linden: I actually want to learn sculpties so I can add more bamboo without blowing out my prim count and/or your viewer
• [11:03] Saijanai Kuhn: talk to Cel. He's got a sculpty program he uses to make elaborate sculpties. Its not commercial yet though
• [11:03] Which Linden: yeah there's a lot of cool stuff I should do
• [11:03] Which Linden: cel who?
• [11:03] Saijanai Kuhn: Cel makes Sculptypaint
• [11:04] Saijanai Kuhn: Cel Edman
• [11:04] Which Linden: cool cool
• [11:04] Morgaine Dinova: Hey Which, are you joining us in the IETF process for interop, MMOX?
• [11:04] Which Linden: Morgaine: I'd like to
• [11:04] Saijanai Kuhn: speaking of which I still haven't gotten anything form the list
• [11:05] Which Linden: already have some comments on the LLSD draft
• [11:05] Morgaine Dinova: Super!
• [11:05] Morgaine Dinova: Sai: I think you have a local bug. I got instant response yesterday.
• [11:05] Which Linden: check your spam filters
• [11:05] Imaze Rhiano: humm... I haven't either received any messages from MMOX list... except confirmation message
• [11:06] Morgaine Dinova: Sure, there's nothing there yet, except Infinity's test. Or wasn't last night
• [11:06] Morgaine Dinova: checks
• [11:06] Saijanai Kuhn: I've sent 3 mesages however
• [11:07] Saijanai Kuhn: I got the registration message, but nothing else. I'm thinking Infinity needs to enble something. Meant to talk to her today about that
• [11:07] Which Linden: heh, melisz, that chair is for tiny avatars, it looks weird with normal-sized avs
• [11:07] Melisz Umia: heh ic
• [11:07] Morgaine Dinova: Still nothing in Archives, except Infinity's test. Until I see something appear in the Archive, I don't really expect to receive the post in email either.
• [11:08] Saijanai Kuhn: tiny avatars are specialized avatars that are half size or smaller
• [11:08] Saijanai Kuhn: dont' think I have one to show you though
• [11:08] Imaze Rhiano: I have one...
• [11:08] Saijanai Kuhn: yeah, well sent 3 messages to the list, so thinking the list isn't fully activqated yet
• [11:09] Morgaine Dinova: Haha, nice kittie
• [11:09] Which Linden: come on man, it's the Internet Engineering Task Force -- they know a little bit about internetting
• [11:09] Imaze Rhiano: oh nice... scripts not allowed...
• [11:09] Which Linden: yeah sorry I have restricrtive perms because I can't be arsed to do the inevitable cleanup
• [11:10] Morgaine Dinova: We were getting griefed, Imaze
• [11:10] Melisz Umia: @@
• [11:10] Which Linden: though as I recall when we were griefed the griefers were dropping stuff in the neighboring parcels
• [11:10] Melisz Umia: so cute
• [11:10] Saijanai Kuhn: Infinty is the list moderator. She sent a test message, but wondering if she needs to set a swich to let others send/receive messages
• [11:11] Morgaine Dinova: Post a message about that in Groupies
• [11:11] Saijanai Kuhn: right. Infinty isn't in groupies right now, was earlier
• [11:11] Morgaine Dinova: She may not realize that she hasn't activated it yet
• [11:11] Saijanai Kuhn: might still be online. Will drop her an email
• [11:11] Saijanai Kuhn: IM
• [11:12] Which Linden: the thing I was curious about in the spec is the conversion from string to boolean -- right now the spec says "true" -> true and "" -> false, but nothing about any other strings
• [11:12] Which Linden: currently the existing llsd implemntations treat any non-empty strings as true
• [11:13] Melisz Umia: i need go , thx for show :P
• [11:13] Morgaine Dinova: Have fun, Mel :-)
• [11:13] Which Linden: see you melisz, thanks for stopping by!
• [11:13] Imaze Rhiano: bye Melisz
• [11:13] Melisz Umia: bye
• [11:13] Saijanai Kuhn: check out the library Melisz
• [11:13] Melisz Umia: i will
• [11:13] Jabba Aabye: See ya Melisz
• [11:13] Morgaine Dinova: Which, that does seem wierd
• [11:14] Morgaine Dinova: Which spec doc specifcally? Linkie?
• [11:14] Imaze Rhiano: I think better would be "true" --> true, anything else --> false
• [11:15] Morgaine Dinova: How about "true" == true, "false" == false, and everything undefined as a boolean?
• [11:15] Morgaine Dinova: everything else*
• [11:15] Which Linden: [1]
• [11:15] Imaze Rhiano: if you want to have 3 value booleans...
• [11:15] Which Linden: yeah, you cant convert booleans to undefined I don't think
• [11:16] Which Linden: since undef is its own type
• [11:17] Imaze Rhiano: where is that "true" / ""?
• [11:18] Imaze Rhiano: I just saw "0" / "1" thingy...
• [11:18] Which Linden: in the link I sent -- look at the subsubsubsection "conversions:"
• [11:18] Which Linden: Boolean The value true is represented as the string "true". The
• value false is represented as the empty string ("")
• [11:18] Imaze Rhiano: oh ya
• [11:19] Imaze Rhiano: we are reading this wrongly now...
• [11:19] Imaze Rhiano: "An empty String is converted to false. Anything else is
• converted to true.
• "
• [11:19] Morgaine Dinova: Nice to see the outer construct of a JSON serialization be the array, rather than the struct ... same approach I'm taking in Imprudence plugins, so that I can be sure to see the commandName appear first on the wire.
• [11:20] Imaze Rhiano: " Boolean The value true is represented as the string "true". The
• value false is represented as the empty string ("").
• "
• [11:21] Imaze Rhiano: but ya - that should definately be "The boolean value false is represented as the string "false".
• [11:21] Which Linden: wait, imaze, where did you get that first quote of yours? the one with "Anything else is
• converted to true."
• [11:22] Which Linden: actually I think that ""->false is better than "false"->false
• [11:22] Jabba Aabye: why not both, and the rest is always true
• [11:22] Imaze Rhiano: 2.1.2.
• [11:23] Morgaine Dinova: Conversions are a separate issue though, The native booleans are true and false, anthing else is a conversion. If you make "" your native false, then you are giving semantics of falsehood to all empty strings, which is not sensible.
• [11:24] Which Linden: I think there are some standards-dweeby notions of why it's a bad idea to have a non-zero-length string represent false
• [11:24] Which Linden: Morgaine: yes, of course, I was only talking about conversions not the serialization
• [11:25] Imaze Rhiano: we also need think about to what value empty string would be then converted
• [11:25] Imaze Rhiano: or should that be "time to throw exception and flash blue screen situation"
• [11:25] Morgaine Dinova: Fair enough, but making "false" convert to Boolean true just because it is non-empty is a recipe for mayhem.
• [11:25] Which Linden: I personally think the correct conversion rule is ""->false, everything else->true
• [11:25] Which Linden: Morgaine: agreed
• [11:26] Imaze Rhiano: humm... ya
• [11:26] Imaze Rhiano: you are right morgaine
• [11:26] Imaze Rhiano: for example if you convert boolean to string in .NET then you get either "True" or "False"
• [11:28] Jabba Aabye: most XML parsers indeed convert boleans to "true"|"false", for string to bolean ""|"false > False, the rest True
• [11:28] Morgaine Dinova: Conversions are just reflections of common practice anyway. So you need to make all the expected conversions work in the way commonsense requires, or there's trouble.
• [11:29] Jabba Aabye: better for debugging too btw..
• [11:29] Imaze Rhiano: I think that XML convention is something that should be followed
• [11:30] Which Linden: I gotta say, when I first started working on this stuff, my common sense was that "false" should convert to boolean false
• [11:30] Which Linden: Now I have been disabused of that notion after reading some PHP code
• [11:30] Morgaine Dinova: Heh, I'd like to see that example.
• [11:31] Which Linden: PHP is useful -- it allows you to quickly demonstrate any number of horrible programming practices
• [11:31] Jabba Aabye: Yes, but PHP and XML are always fighting each other.. although XML is of course broadly used now for passing serialized data...
• [11:32] Jabba Aabye: And php is I think not a very good learner for typecasting or conversion :P
• [11:32] Which Linden: heh nope
• [11:32] Which Linden: changing the subject a bit -- do you all want some SCIENCE?
• [11:32] spud Nirvana: yo
• [11:33] Jabba Aabye: Hey Spud
• [11:33] Which Linden: welcome spud
• [11:33] Imaze Rhiano: hi spud
• [11:33] spud Nirvana: hows it goin
• [11:33] Imaze Rhiano: welcome to SL spud :)
• [11:33] spud Nirvana: new here
• [11:34] Imaze Rhiano: you should visit in that landmark that I gave for you - lot's of info for new residents there :)
• [11:34] Morgaine Dinova: I'm basically a scientist/engineer --- so go for it Which :-)
• [11:34] Which Linden: as in, I have stats on group chat that you may be interested in
• [11:34] spud Nirvana: me tooo
• [11:34] Morgaine Dinova: Hah, stats aren't science :P
• [11:34] Which Linden: a "soft science"
• [11:34] Which Linden: messages sent/second: 8.6
• [11:35] Which Linden: that's pretty low
• [11:35] Which Linden: group churn/second: 23.2
• [11:35] Which Linden: churn = group chats being created or deleted
• [11:35] Which Linden: so we have higher churn than message rate
• [11:35] Morgaine Dinova: 23 per second?
• [11:35] Which Linden: yup
• [11:35] Which Linden: membership churn/second: 260
• [11:35] Which Linden: a membership is an agent listening in to a group
• [11:36] Morgaine Dinova: That's unbelievable
• [11:36] Imaze Rhiano: uh ... are those statistics from all groups?
• [11:36] Which Linden: if you think about it, it makes sense -- the average user is in ~10 groups, so every time they log in, that's 10 membership churn
• [11:36] Which Linden: yup
• [11:36] spud Nirvana: zzzzzzzzzzzzzzzzzzzzzzz huh
• [11:37] Morgaine Dinova: Oh, your meaning of "membership" is different
• [11:37] Morgaine Dinova: You mean joining the fanout list
• [11:37] Which Linden: number of groups 131953
• number of memberships 629637
• [11:37] Which Linden: that's roughly 2 unique groups per concurrent user
• [11:37] Morgaine Dinova: So you're talking about IM fanout membership, not group membership churn
• [11:38] Which Linden: how would you measure the other type of churn you're talking about?
• [11:38] spud Nirvana: well i am off exploring peace all
• [11:38] Morgaine Dinova: Cyu spud
• [11:38] Jabba Aabye: Bye spud!
• [11:38] spud Nirvana: ciao bella
• [11:39] Imaze Rhiano: lot's of newcomers tonight
• [11:39] Which Linden: peace
• [11:39] Morgaine Dinova: Which, your numbers don't match the way IM currently works, because a person isn't added to her IM fanout list until the first message arrives --- that's why we get error messages on the first incoming message
• [11:40] Which Linden: Morgaine: yes, that's true, however, conceptually that distinction should be erased
• [11:40] Morgaine Dinova: You can't measure conceptual throughput ;P
• [11:41] Which Linden: it's not conceptual -- when you join a group before the first message is sent, it still incurs server load
• [11:41] Which Linden: I can't think of an IM system that wouldn't have such server load
• [11:42] Morgaine Dinova: It would do, if you built the exploders on login, but apparently you don't, because the error messages point to your building them each time a message is sent.
• [11:42] Which Linden: so if we discard the churn due to people logging in and being in groups that don't ever get any messages, we'll be discarding a nontrivial source of server load
• [11:42] Morgaine Dinova: Eg. a couple of days ago, I received an error every time that a message was appearing in Gridnauts ... it was telling me that it tried to create the channel and was failing.
• [11:42] Which Linden: yes, the system is made more unreliable by doing it at message send time, so we'd like to change it to do it at login; this will not change the load number though
• [11:43] Morgaine Dinova: Aye
• [11:43] Which Linden: because, right now, even at login it pings the server and gives you an "invite"
• [11:43] Which Linden: and the "invite" basically is the same thing as being in the group except for some fiddly details that suck and cause that message to occur more often than it should
• [11:44] Morgaine Dinova: The way to get this right is to design for the large case, so that there are no surprises ahead. Eg. assume that everyone belongs to millions of groups.
• [11:44] Jabba Aabye: 0, 1 or infinite ;)
• [11:45] Morgaine Dinova: If your design can cope with that, then your implementation will work fine even on small numbers.
• [11:45] Which Linden: Morgaine: yes agreed
• [11:45] Which Linden: more numbers: average group size 4.8
• largest group 785
• stdev group size 13.1
• [11:45] Which Linden: so groups on average are small
• [11:45] Which Linden: I should do a histogram of group size
• [11:46] Morgaine Dinova: Largest group 785? Live Music Enthusiasts has 4,000
• [11:46] Which Linden: 785 concurrent
• [11:46] Morgaine Dinova: Ah, see, you need to break that "conceptual" merging :-)
• [11:46] Which Linden:  ?
• [11:47] Imaze Rhiano: would be nice to get clients participate actively to chat message delivery - develope somekind p2p chat
• [11:47] Morgaine Dinova: Is that group name public knowledge, the 785 conc? Wondering if LME is largest
• [11:47] Which Linden: I have heard of groups with 10000 members; obviously not all are online at the same time
• [11:48] Jabba Aabye: There is a fashion group with more than 10k i remember...
• [11:48] Morgaine Dinova: Imaze: in the fullness of time, P2P is the only thing that scales.
• [11:48] Imaze Rhiano: IM could use p2p easily
• [11:49] Which Linden: group IM cannot use P2P easily though
• [11:49] Which Linden: currently person-to-person IM is already P2P
• [11:49] Morgaine Dinova: I expect distribution of unencumbered VW assets to be entirely P2P in due course. Nothing else can handle the numbers.
• [11:49] Imaze Rhiano: oh... didn't know that
• [11:49] Jabba Aabye: persons can switch being the "group-server"...?
• [11:49] Which Linden: groups are tricky because they're many-to-many
• [11:50] Which Linden: yeah, so if you wanted to decentralize group IM you'd have to give up some property or other, like, individuals might miss messages
• [11:50] Which Linden: or, as in the current system, messages take a long time to reach their destinations
• [11:50] Jabba Aabye: its possible, but indeed hard to make
• [11:51] Morgaine Dinova: Time taken is just a scalability issue though. Put the resources in and time drops.
• [11:51] Imaze Rhiano: would need to make somekind hieararchial p2p network for groups with lot's of redundancy
• [11:51] Morgaine Dinova: Assuming the design is scalable of course, ie. no linear depedencies. etc
• [11:51] Which Linden: Morgaine: yes, that's true, and we should throw more hardware at our current setup IMO
• [11:52] Morgaine Dinova: Which: you have excess hardware already ... you're just not using it :P
• [11:52] Which Linden: yes, well, of course
• [11:52] Imaze Rhiano: ya... most of SIMs are empty :P
• [11:52] Morgaine Dinova: Yeah. Your basic resource model is flawed.
• [11:53] Saijanai Kuhn: thinking that we can start playing with new IM models even at the server level, once Zha gets her AD done
• [11:54] Morgaine Dinova: Yes, hope so Sai
• [11:54] Saijanai Kuhn: and wheee just helpd ENus debug pyogp while we sit here
• [11:54] Which Linden: there are reasons other than poor modeling to not use idle sims for group chat
• [11:54] Morgaine Dinova: But I think Duke Nukem Forever will be released and pigs will fly before Zha gets her AD done.
• [11:54] Imaze Rhiano: wants to play Duke Nukem!
• [11:54] Morgaine Dinova: Hehe
• [11:54] Saijanai Kuhn: nyah. She got approval for OpenSim TP code done, so it should be possible to get approval for AD code also
• [11:55] Morgaine Dinova: Sai: yeah, I'm thinking "this side of heat death of the universe"
• [11:55] Saijanai Kuhn: would be nice if she can get blanket approval for contributions to OpenSIm
• [11:55] Which Linden: one issue is that simhosts die all the time, and we have an incredibly elaborate system for trying to keep regions up in spite of the attrition -- there's no generalized way to spread any given application over N unrelliable nodes, so we have to hand-build that for each application, which gets expensive
• [11:56] Morgaine Dinova: Oh well, we learned our lesson --- pump priming a project should not be done by an IBMer.
• [11:56] Which Linden: zing
• [11:57] Morgaine Dinova: Which: your RabbitMQ direction sounds excellent for that, since Erlang/OTP handles failure very cleanly and efficiently.
• [11:58] Imaze Rhiano: RabbitMQ Erlang/OTP?
• [11:58] Morgaine Dinova: LL are evaluating what to use for their backend messaging system. RabbitMQ is a leading contender, and that's written in Erlang using their OTP framework.
• [11:59] Which Linden: Morgaine: yeah, the one axis of scalability that's not well-understood for rabbit is number of queues
• [11:59] Saijanai Kuhn: snaps fingers. Betch IBM has per cycle approval thing, so she's actually fighting to get any/all OpenSim cotnributions approved not just this one
• [11:59] Morgaine Dinova: Sai: hope so
• [11:59] Which Linden: So we're trying to figure out how many queues per host we can make rabbit do
• [11:59] Saijanai Kuhn: would hate to see her have to go through this more than once/year
• [12:00] Morgaine Dinova: Which: are you getting hopeful numbers back from tests, ie. large?
• [12:01] Which Linden: Right now we're stuck at 65,300 queues on our one host
• [12:02] Which Linden: trying to mess around with upping resource limits to fix that
• [12:02] Morgaine Dinova: OTP gives you an immensely powerful system for such testing. Once you have an Erlang node on each machine, you can distribute pretty much anything anywhere, great joy for designers.
• [12:02] Morgaine Dinova: Very cool!
• [12:02] Which Linden: and, of course, seeing whether clustering two nodes gives us double the number of queues
• [12:03] Which Linden: Morgaine: I'd caution that OTP isn't a magical distribution machine; you still have to be very mindful of how you write your app, just like MapReduce
• [12:03] Morgaine Dinova: Oh sure, there's no magic bullet. But at least the gun has multiple barrels here :P
• [12:04] Morgaine Dinova: And it's far more flexible than MapReduce.
• [12:04] Imaze Rhiano: sighs...
• [12:04] Imaze Rhiano: why everyone need to invent new language?
• [12:05] Morgaine Dinova: Because current ones are not capable or working in a concurrent environement, Imaze.
• [12:05] Morgaine Dinova: I mean current *mainstream* ones. Erlang has been around for ages, but never became mainstream (yet).
• [12:06] Imaze Rhiano: Hi Jessika and welcome to SL
• [12:06] Which Linden: I think it's more of a "the old guard have to die out" kinda thing -- current languages *can* operate in concurrent environment given the proper tweaks, but no one really wants to do those because they'd be backwards-incompatible
• [12:06] Which Linden: so you have to bild new ones that will slowly replace the old ones
• [12:07] Morgaine Dinova: Erlang's not the only one that can though, Imaze. Any functional language works too, eg. Haskel is fine, although they're not fully concurrent yet I think.
• [12:07] Imaze Rhiano: F*
• [12:07] Morgaine Dinova: True about old guard having to die out ... although some of us "got it" right at the start ;-)
• [12:07] Which Linden: so hey, time for me to turn into an orange squash
• [12:07] Imaze Rhiano: that is M$ functional language prototype "F#"
• [12:07] Which Linden: It's been a pleasure discussing things with you all
• [12:08] Saijanai Kuhn: laters which
• [12:08] Which Linden: I hope you have a great day!
• [12:08] Imaze Rhiano: bye
• [12:08] Which Linden:  :-)
• [12:08] Jabba Aabye: Thank you too Which
• [12:08] Morgaine Dinova: Yes, Microsoft realizes it too. It's just unfortunatele that their O/S's are such crap, but some of their research is OK
• Which Linden