User:Which Linden/Office Hours/2009 Aug 20

• [11:06] Which Linden: yo, yo!
• [11:06] Khamon Fate: oh these are nice bamboos
• [11:06] Morgaine Dinova: Who?
• [11:06] Which Linden: sorry I'm late, was wrestling with email
• [11:06] Twisted Laws: hello
• [11:06] Morgaine Dinova: Hiya Which :-)
• [11:07] Aimee Trescothick: h'lo :)
• [11:07] Which Linden: is Skills floating up for anyone else?
• [11:07] Skills Hak: :o
• [11:07] Skills Hak: still?
• [11:07] Twisted Laws: no
• [11:07] Mojito Sorbet: Standing on the ground
• [11:07] Skills Hak: working on clientside AO
• [11:07] Morgaine Dinova: Standing now, junmped a sec ago
• [11:08] Mojito Sorbet: Doing various AO poses
• [11:08] Skills Hak: okies :)
• [11:08] Which Linden: perhaps my viewer dropped a packet, cause you floated right through the sign, on my screen
• [11:08] Which Linden: just snapped back
• [11:08] Which Linden:  :-)
• [11:08] Mojito Sorbet: That happense a lot lately
• [11:08] Morgaine Dinova: Never believe viewers, they lie ;-)
• [11:09] Mojito Sorbet: I might have the interpoplate velocity setting wrong.
• [11:09] Mojito Sorbet: *interpolate
• [11:09] Latif Khalifa: it's the "sim freeze" problem plaguing sl since about 1.25-6 or so
• [11:09] Skills Hak: actually i WAS floating through the sign but then tp'ed back
• [11:09] Skills Hak: x)
• [11:09] Mojito Sorbet: And I am really wearing all grey clothes.
• [11:10] Which Linden: ha ha ha
• [11:10] Latif Khalifa: my sim regulary freezes from anywhere 2 sec to a full minute
• [11:10] Which Linden: reminds me of the trend where everyone was walking around as Ruth
• [11:10] Mojito Sorbet: We could have a costume party and all come as Ruth
• [11:11] Which Linden: Latif: hm, is there a JIRA for the freezing issue?
• [11:11] Latif Khalifa: plenty of them
• [11:11] Skills Hak: i bet it's the host swapping mem to disk
• [11:11] Which Linden: which one has some dates/times on it?
• [11:11] Skills Hak: you should try ask a nice linden to look it up for you
• [11:11] Mojito Sorbet: It would be interesting to know if everyone in the sim sees it at the same time
• [11:11] Latif Khalifa: people made freeze graph makers etc etc
• [11:11] Skills Hak: is it on TPs?
• [11:11] Which Linden: (if you know this off the top of your head, I don't mean to use you as my personal JIRA search engine)
• [11:12] Manuel McWinnie: hello
• [11:12] Morgaine Dinova: Moon regularly displays her graphing tool showing the freeze blips
• [11:12] Which Linden: (Wolfram Alpha eat your heart out)
• [11:12] Latif Khalifa: there is one slightly misfilled in misc
• [11:12] Mojito Sorbet: I try to avoid wearing mono HUDs when TPing
• [11:12] Latif Khalifa: ah jira was down
• [11:12] Skills Hak: 0 scripts here
• [11:12] Latif Khalifa: https://jira.secondlife.com/browse/MISC-2329
• [11:12] Skills Hak: well office hour hud ^^
• [11:12] Mojito Sorbet: Since switching to Emerald I do not need most of them
• [11:12] Daniel Voyager: hey
• [11:13] Latif Khalifa: there are plenty of others
• [11:13] Skills Hak: i just replaced my AO hud with the clientside ao
• [11:13] Nicholas Hawthorne: hey
• [11:13] Which Linden: hey there dragons
• [11:13] Nicholas Hawthorne:  :)
• [11:14] Morgaine Dinova: Here be dragons
• [11:14] Latif Khalifa: for your sake, they better be carnivors lol
• [11:14] Daniel Voyager: lol
• [11:14] Nicholas Hawthorne: xD
• [11:14] Morgaine Dinova: /mews
• [11:14] Morgaine Dinova: purrs
• [11:14] Nicholas Hawthorne: dan, i said a snow sim, not here :D
• [11:14] rfc3692 Static: clientside ao? cool
• [11:14] Mojito Sorbet: Soemthing on TV this morning about how the duckbilled donosaurs could eat redwood trees.
• [11:14] Which Linden: man, I love those graphs on that MISC-2329
• [11:14] Daniel Voyager: we don't eat humans hehe
• [11:14] Nicholas Hawthorne: lol
• [11:15] Latif Khalifa: which is a bamboo
• [11:15] Nicholas Hawthorne: we might smell you but not eat you
• [11:15] Morgaine Dinova: Which is only scared of pandas
• [11:15] Nicholas Hawthorne: xD
• [11:16] Nicholas Hawthorne: wishes his friend was online, he's the panda
• [11:16] rfc3692 Static: lol
• [11:16] Nicholas Hawthorne: HahaAHAHaa!
• [11:16] Nicholas Hawthorne: you should see how we fly lol
• [11:16] Which Linden: show us!
• [11:16] rfc3692 Static: skills, sorry my ignorance: there's somewhere a source to implement clientside ao? or u're writing it yourself? sorry for my english
• [11:17] Which Linden: lollerskates
• [11:17] Nicholas Hawthorne: no scripts sec
• [11:17] Which Linden: oh....heh
• [11:17] Mojito Sorbet: I thought that looked odd
• [11:17] Which Linden: scripts are back on
• [11:18] Mojito Sorbet: That's better
• [11:18] Skills Hak: rfc3692 , sorry i have to run. it will show up in emerald soon :)
• [11:18] Mojito Sorbet: cool
• [11:18] Skills Hak: laters :)
• [11:18] Nicholas Hawthorne: yeah,
• [11:19] Which Linden: see ya
• [11:19] rfc3692 Static: cool, i can't wait for sources :)
• [11:19] Latif Khalifa: whuchm ah found onw with pretty graphs https://jira.secondlife.com/browse/SVC-4196
• [11:19] Which Linden: oh very nice
• [11:20] Nicholas Hawthorne: lol
• [11:20] Latif Khalifa: but in my experience it does not apepar freeze of 2-7 seconds for the whole sim on avatar rez, i get them for avatar derez, also for some other reasons i cannot determine
• [11:20] Latif Khalifa: whole up to 60 sec of total 0 packets from the sim for everybody in the sim
• [11:21] Twisted Laws: removing mono scripts seems to be worse then adding them
• [11:21] Mojito Sorbet: Have you tried this both with and without mono attachements?
• [11:21] Mojito Sorbet: oh
• [11:21] Mojito Sorbet: How about NO scripts?
• [11:22] Which Linden: did you try it without the simulator part?
• [11:22] Which Linden:  :-P
• [11:22] Latif Khalifa: well one thin everybody agrees upon, sim seems to stabilize after restart
• [11:22] Twisted Laws: lots of testing on beta grid... objects with lots of mono scripts cause a big impact when removed
• [11:22] Latif Khalifa: for up to couple of days
• [11:22] Latif Khalifa: then after day 3 freezes gradually get worse and worse
• [11:22] Which Linden: man, histograms, too!
• [11:22] Latif Khalifa: which would indicate some kind of resource leak
• [11:23] Latif Khalifa: Which, its really cripling, people would do everything to help fix this
• [11:23] Mojito Sorbet: Maybe a controlled experiment with Linden techs watching sim resource usage at the same time
• [11:23] Latif Khalifa: imagine dead reckogning fun on a sim that freezes for 30 seconds
• [11:23] Mojito Sorbet: See if memory goes up or something
• [11:23] Latif Khalifa: not to mention chat lag
• [11:23] Twisted Laws: we did that with vektor Moijito
• [11:24] Mojito Sorbet: ok
• [11:24] Latif Khalifa: people drift left and right, they appear to be typing, transactions fail, all sorts of havoc
• [11:24] Morgaine Dinova: Oh FFS, Infinity's off on another strawman in OGPX
• [11:24] Which Linden: heh these are tagged as "hard-to-fix" internally
• [11:24] Latif Khalifa: "we have no clue what's going on here" :P
• [11:25] Mojito Sorbet: Does LL ever do "code reviews"?
• [11:25] Twisted Laws: everyone move back to non-mono scipts and things get better
• [11:25] Nicholas Hawthorne: o.o
• [11:25] Which Linden: Mojito: yes, nearly every commit is reviewed
• [11:25] Morgaine Dinova: Which: one of your people is severely out of control and throwing straw men around with gusto at the IETF
• [11:25] Mojito Sorbet: Or red herrings
• [11:26] Which Linden: damn yo, how that no one gets poked in the eye with a straw finger
• [11:26] Morgaine Dinova: Total red herrings
• [11:26] Which Linden: *hope
• [11:26] Which Linden: Twisted: really? You did the experiment of moving everything to non-mono?
• [11:26] Twisted Laws: yes
• [11:26] Morgaine Dinova: Just unbelievable
• [11:26] rfc3692 Static: sorry, got to go... have a nice day :)
• [11:26] Twisted Laws: same tests as compiled in mono... on the beta grid
• [11:27] Twisted Laws: no impact for removing 200+ LSL scripts, 0.8 sec impact for 200+ mono scripts
• [11:27] Which Linden: see ya rfc, thanks for stopping by
• [11:27] rfc3692 Static: hope to see u soon, this is so interesting :)
• [11:27] Latif Khalifa: mono does seem to be a factor... but its difficult to tell on this end
• [11:29] Twisted Laws: we only see the external effect... no idea what happens on server at that time
• [11:29] Which Linden: we are talking about MISC-2329 here, right?
• [11:29] Twisted Laws: its actually more https://jira.secondlife.com/browse/SVC-4196 ... they are all related tho
• [11:29] Morgaine Dinova: OMG, 3 posts in a row, completely and willfully misrepresententing everything, and avoiding answering Carlo's simple question --- [1]
• [11:29] Latif Khalifa: https://jira.secondlife.com/browse/SVC-4196
• [11:30] Twisted Laws: that is written on entering, but the effect leaving is worse
• [11:30] Which Linden: I see
• [11:30] Twisted Laws: an empty sim with no one entering or leaving doesn't seem to change over time
• [11:30] Latif Khalifa: wonders what DEV-31771 says :P
• [11:30] Morgaine Dinova: Which, are you, personally, in favour of interoperation between virtual worlds of the SL type, using OGP, for example between SL and on or more of the Opensim-based grids?
• [11:31] Morgaine Dinova: one* or more
• [11:31] Which Linden: Morgaine: ... my trap-sense is tingling, so I'm not sure whether to answer or not :-)
• [11:31] Latif Khalifa: lol
• [11:32] Morgaine Dinova: It's quite a simple question. If that's a trap then the question is sensitive, and I'm surprised that it can be sensitive.
• [11:33] Latif Khalifa: well openness seems to be on a retriet in the lab, ogp is going fast nowhere, viewer is no longer synced to exernal svn... the "m linden" effect in full swing, don't get your expectations high :D
• [11:33] Which Linden: Well the, without implying anything else by this answer, then yes I am in favor of interoperation between SL-like virtual worlds
• [11:33] Morgaine Dinova: Cool
• [11:35] Morgaine Dinova: Well it's always been the understanding that that's what OGP was all about, in AWG, in MMOX, and then at the start of OGPX ... that LL was working towards interop between virtual worlds, although they wanted to narrow it down to interop between SL-like one, which is fine. Unfortunately, that's no longer the case.
• [11:35] Morgaine Dinova: So we're currently trying to puzzle out why.
• [11:35] Which Linden: I haven't really been following the ogpx mailing list lately, it seems like there's a lot of inside baseball
• [11:37] Latif Khalifa: its pretty useless to follow, week long back and forth what the word "grid" means and should it be used or not, and similar useless chatter instead of real work on the protocol specification
• [11:37] Which Linden: Looks like lots of dithering about what everyone's intentions ar
• [11:38] Which Linden: I guess this is why standardization efforts never take fewer than a million yers. :-)
• [11:39] Which Linden: Getting back to the bugs that Latif brought up, it looks like we are trying to gather more metrics on them, so as to better detect them from our end
• [11:39] Morgaine Dinova: Easy to summarize: Joshua has written a rather good charter for the group, good in the sense of highly readable. It very clearly defines the scope of the OGPX workgroup to be that of creating a protocol that allow integrating single region servers into a single virtual world. In other words, it's a protocol for growing SL, from LL's perspective. It does not deal at all with communication between more than one such worlds.
• [11:39] Morgaine Dinova: The scope is very precise, so there is no room for doubt, as the charter stands.
• [11:40] Which Linden: I think that could be just a word discussion Morgaine, insofar as we refer to "the web" even though it is not actually a single entity
• [11:40] Morgaine Dinova: Now we're trying to get that stated clearly, and Infinity is throwing red herrings and straw men around all over the place avoiding the question.
• [11:41] Morgaine Dinova: No Which, this is a simple question: will the scope of the OGP that the group defines deal with interop between more than one virtual world. Easy
• [11:41] Which Linden: I guess I'll end my discussion of that topic now, I think getting into it would consume too much of my time
• [11:41] Morgaine Dinova: Can't be fudged.
• [11:41] Twisted Laws: but what is the argument with the direction its going Morgaine? couldn't opensim follow the same protocol and thus work with SL?
• [11:42] Morgaine Dinova: Twisted: there is no argument with the direction. Nobody is arguing with LL's direction, we're letting LL set it. But we want it to be clear.
• [11:42] Latif Khalifa: Which, I certainly do hope soulution is found fast, because its getting worse, and its really crippling popular places (ie. those with high avatar turnaround)
• [11:43] Morgaine Dinova: It's not a matter of shame to decide not to interop between multiple virtual worlds, it's simply a choice. It should not be hidden or obscured.
• [11:44] Morgaine Dinova: Back to Latif's topic.
• [11:44] Latif Khalifa: Morg, my attitude is, enjoy the party while it lasts, M's being bent on SL "business case" is going to be its undoing, I personally plan to have fun while its still afloat :P
• [11:45] Which Linden: Latif: yeah, keep the data coming, exact times and dates, trying to find ways of reproingi t
• [11:45] Morgaine Dinova: Latif: I agree it's all falling apart.
• [11:45] DrFran Babcock: listens.
• [11:45] Latif Khalifa: Which, I can call you to my sim after its being up for 2 weeks or so, so you can observe the effect first hand
• [11:46] Which Linden: Often with bugs like these, there is a period of thrashing while everyone tries just to repro it in a closed environment, followed by a flash of insight in which everything is understood and fixed
• [11:46] Latif Khalifa: two factors convreging seem to bring the worse of it... sim uptime + grid concurrency, ie. popular sim that has been up for 3 weeks + weekend = disaster
• [11:46] Which Linden: Latif: hm well that could be useful, but what would be more useful would be some way of quickly and reliably bringing a sim into that state from scratch
• [11:47] Which Linden: bots may be a way to do that?
• [11:47] Latif Khalifa: could be, i actually wanted to try this on aditi, but never gor around to it
• [11:47] Latif Khalifa: have like 20 bots constatly teleport in and out of a sim, with a lot of prim scripted attachmetns
• [11:48] Youri Ashton: hi hi
• [11:48] Latif Khalifa: (been to busy figuring out how to make pretty bots with libomv) :P
• [11:48] Latif Khalifa: but now that's done
• [11:48] Youri Ashton: late due thunderstorms messing with my net
• [11:50] Which Linden: cool!
• [11:50] Which Linden: I wish we had thunderstorms here
• [11:50] Latif Khalifa: my sim gets about 20 unique visitor per hour, and its performance goes down the tubes in about a week... so its slow
• [11:50] Latif Khalifa: and difficult to reproduce
• [11:51] Twisted Laws: i think you can reproduce it just adding and removing objects with scripts in them Latif
• [11:51] Latif Khalifa: i think prospero had a look once, the sim wasn't too bloated, process size was big but not out of control
• [11:51] Twisted Laws: we see the performance drop quickly in the sandboxes all the time
• [11:52] Morgaine Dinova: I don't think there's a repro problem at all, since the freezes have been demonstrated at numerous office hours. And Periapse has even said he thinks it quite likely that there's a rogue blocking network call that shouldn't be blocking associated with the freezes. What it needs is a dev's attention.
• [11:53] Latif Khalifa: well SVC-4196 has been brought up on many office hours, it got 150 votes, dunno what else to do
• [11:53] Which Linden: We need some sort of speciality which is not exactly a dev per se but more of a "repro engineer"
• [11:53] Which Linden: I guess that's "QA" in a sense
• [11:54] Twisted Laws: Andrew/Simon seem to think it has to do with more happening in frame then theres time for
• [11:54] Latif Khalifa: for 30-60sec?
• [11:54] Which Linden: Man, lots of different hypotheses
• [11:54] Latif Khalifa: that's one long frame
• [11:54] Which Linden: yeah that doesn't really add up
• [11:54] Twisted Laws: yes, but thats where they worked on the max number of objects that can be rez'd as that kills a sim quick
• [11:55] Morgaine Dinova: More hypotheses aren't needed. Just sit a dev down with Latif and Moon and others and all the repros are there --- just needs an LL dev sitting on the sim side with an analyser.
• [11:55] Twisted Laws: someone was testing in cordova with single objects with 600 mono scripts in them...
• [11:56] Twisted Laws: they said they were trying to figure how to keep a region down...
• [11:56] Twisted Laws: was not a good person
• [11:56] Which Linden: imagines a big oscilloscope-looing "sim analyzer"
• [11:56] Twisted Laws: they did succeed a couple time for up to 30 minutes
• [11:57] Latif Khalifa: sim shoud log these events if fram started more than a second ago > log
• [11:57] Latif Khalifa: frame*
• [11:57] Which Linden: I am sure that they do
• [11:58] Which Linden: But....that doesn't tell you why the frame was long
• [11:58] Which Linden: So what we need sim-side is an equivalent of those time things that tell you which subsystem took how much frame time in the viewer
• [11:59] Which Linden: oh we actually kinda do have that alreaduy
• [12:00] Which Linden: but it's not very granular
• [12:01] Latif Khalifa: if a frame can take 30 seconds or more, somehting goes horribly wrong there
• [12:01] Which Linden: totally
• [12:01] Latif Khalifa: finny thing is, sim is not very loaded
• [12:01] Latif Khalifa: my sim show .99 time dilation, and then it goies into freeze for 5 sec
• [12:02] Latif Khalifa: and so on
• [12:02] Which Linden: there are probably a bunch of places where operations that usually are quick are not broken across frame boundaries when they get slow
• [12:03] Latif Khalifa: i suspect some sync call to a central service, as it does sim to get worse when there is a lot of people online
• [12:03] Which Linden: right....could be!
• [12:03] Which Linden: I'll add that as a theory to help the debuggers
• [12:05] Latif Khalifa: i bet simple strace -p on an affected sim while this is happening could be revealing :P
• [12:05] Which Linden: could be overwhelming :-)
• [12:05] Which Linden: So, I should get lunch, thanks all for coming and having an interesting discussion!
• [12:05] Latif Khalifa: if you see it block for a long time.. that would be a clue :P
• [12:06] Twisted Laws: thanks, cya
• [12:06] Latif Khalifa: have a nice day which :D
• [12:06] Which Linden: See you next week!