User:Andrew Linden/Office Hours/2008 05 22

Transcript of Andrew Linden's office hours:

[17:09]	Simon Linden:	We've also found some of the problems that give a momentary slowdown, and it appears to be other back-end software that runs on the same system hogging the CPU. No solution yet, but at least the problem is better understood
[17:09]	Andrew Linden:	Sorry I'm late.
[17:09]	Kitto Flora:	Is the majority of the delay in the simulator, or accessing the ithem from Asset Server?
[17:10]	Simon Linden:	Well, I was working on the simulator side, ignoring the network download
[17:10]	Jessica Kabumpo:	Simon - if that's the case, there's a seemingly simple solution - get another box
[17:10]	Yuu Nakamichi:	other back-end software seems to be causeing the TD?
[17:10]	Simon Linden:	For any object, the first time you rez it the asset server and network are likely to be the bottleneck
[17:11]	Simon Linden:	Once it's been downloaded once, however, it will be cached and that is less likely to be a delay
[17:11]	Kitto Flora:	OK - thats what we see, based on the hit that border crossing took
[17:11]	Simon Linden:	Jessica - yes, another box would make it faster, but we have a lot of boxes :)
[17:11]	Jessica Kabumpo:	Oh, I was just at one of my favorite hangouts ... our border region was .... a gaping void
[17:12]	Jessica Kabumpo:	oy .... wait, is everything open source on that box?
[17:12]	Jessica Kabumpo:	if so, you could try to streamline the daemons on it
[17:13]	Simon Linden:	We'll (probably not me) definitely be looking at it more. It's a matter of getting all the components to play nicely together
[17:14]	Yuu Nakamichi:	is this part of the project sidewinder is working on?
[17:15]	Simon Linden:	Yes, he's involved. He's been helping get others involved as this isn't a physics problem.
[17:15]	Yuu Nakamichi:	Hi Side
[17:15]	Jessica Kabumpo wonders why the Lindens haven't rezzed for her yet
[17:15]	Sidewinder Linden:	hi yuu - sorry was in another meeting
[17:15]	Simon Linden:	We sort of stumbled onto it since we shipped Havok, and then were looking at all problems that cropped up
[17:15]	Sidewinder Linden:	jess we have the slow bits on
[17:15]	Jessica Kabumpo:	ohs, okay
[17:15]	Sidewinder Linden:	/kidding... maybe :)
[17:16]	Jessica Kabumpo:	haha
[17:16]	Jessica Kabumpo:	Lawl ^-^
[17:16]	Simon Linden:	Yeah, Saijanai is still grey for me. Seems slow
[17:16]	Rex Cronon:	hello everybody
[17:16]	Yuu Nakamichi:	hi Rex
[17:16]	Andrew Linden guesses that Saijanai is using a Mac.
[17:16]	Jessica Kabumpo is using her iMa
[17:16]	Yuu Nakamichi:	so
[17:16]	Jessica Kabumpo:	*iMac
[17:16]	Rex Cronon:	hi
[17:16]	Yuu Nakamichi:	since we are talking baout performance at the moment
[17:17]	Yuu Nakamichi:	do you want to elaborate fruther on what you found so far?
[17:17]	Jessica Kabumpo has something about source builds failing
[17:17]	Rex Cronon:	this cloud thing is weird
[17:17]	Yuu Nakamichi:	hehe
[17:17]	Simon Linden:	I'm clueless about the OS viewer issues
[17:18]	Kitto Flora:	Nebutars
[17:18]	Jessica Kabumpo:	we already have release engineering for Windows, Mac, and Linux
[17:18]	Jessica Kabumpo:	but I kinda need another one added to the mix - FreeBSD
[17:19]	Andrew Linden:	Simon, you already talked about some of the performance issues you looked at recently? I think Yuu was asking for more info.
[17:19]	Yuu Nakamichi:	if I understand correctly, you say the cyclical TD was exposed by moving to H4 but it wasn't caused by it?
[17:19]	Andrew Linden:	Oof. Does FreeBSD have accelerated OpenGL support?
[17:19]	Simon Linden:	Yes, I mentioned that I had recently made improvements to the object rez code and an internal function dealing with the land height
[17:20]	Jessica Kabumpo:	Yes, with the proper AMD ATi or nVidia drivers
[17:20]	Simon Linden:	The cyclical TD does appear to be not related to Havok
[17:20]	Jessica Kabumpo has played WoW on her FreeBSD powered laptop
[17:20]	Andrew Linden:	I think we're still using some proprietary or restricted libs on the SL client.... those would be the main blockers for FreeBSD port.
[17:20]	Simon Linden:	It may be related to the changes for the asset system - there have been other people working frantically on that trying to keep it alive :)
[17:20]	Andrew Linden:	However, I'm not totally up to date on the open source viewer builds.
[17:21]	Andrew Linden:	That land height query optimization was a small one... might move the simlator FPS 1 unit or so?
[17:21]	Simon Linden:	It mostly comes down to the other back-end processes that exist on a simulator host. Sometimes they seem to be hogging the CPU and so a region doesn't get the processing power it needs, and slows down
[17:21]	Andrew Linden:	er... 1%?
[17:23]	Simon Linden:	Possibly 1% if there's a lot of collsions going on. It's a baby step, but there doesn't seem to be one big problem, so we have to find all the small ones we can
[17:23]	Jessica Kabumpo:	what all processes do we have on the sim hosts?
[17:23]	Sidewinder Linden:	there are many
[17:23]	Sidewinder Linden:	some of the ones causing issues are background administrative ones
[17:23]	Sidewinder Linden:	things such as simstate saves
[17:24]	Sidewinder Linden:	some things to do with object transport
[17:24]	Andrew Linden:	each host runs some database query processes, apache, squid, and then occasionally handle region backups
[17:25]	Sidewinder Linden:	and the region backups are generally independent from region to region, but on an openspace, that process happens on a cpu that hosts four regions
[17:25]	Sidewinder Linden:	so when one region state backup is in motion it can (andusually does) affect other regions to some degree)
[17:25]	Sidewinder Linden:	these things have been around for a long time, but for various reasons seem to have colletively gotten "heavier" over the last while
[17:25]	Jessica Kabumpo:	sounds like some code cleanup is in order
[17:25]	Sidewinder Linden:	we also switched some messages to xml-based, which adds parse and handling load to the simulator host
[17:26]	Sidewinder Linden:	yes
[17:26]	Sidewinder Linden:	simon is looking at some of these with specific optimization work, and others we (mostly simon) has been characterizing so that we can figure out the best path to getting them cleaned up
[17:26]	Jessica Kabumpo:	yea, there's a reason I don't use XML for certain parts of my website
[17:26]	Andrew Linden:	goodbye fast unreliable UDP packets, hello slower reliable TCP packets carrying XML
[17:27]	Sidewinder Linden:	heh :)
[17:27]	Saijanai Kuhn:	have you guys looked at all at how reverse http might change things?
[17:27]	Sidewinder Linden:	btw sorry if i'm repeating something said - crashed
[17:27]	Simon Linden:	yes, there just was a recent discussion about using that
[17:27]	Sidewinder Linden:	oops
[17:27]	Sidewinder Linden:	not directly w/r/t these issues yet
[17:28]	Andrew Linden:	reverse http is being considered for some external interfaces. You should ask Zero Linden about it.
[17:28]	Jessica Kabumpo:	speaking of HTTP .....
[17:28]	Saijanai Kuhn:	sure. Just wondering if it changed performance for your http in any way.
[17:28]	Sidewinder Linden:	for now, we've been trying to understand the issues and sources first... then see what the right solution path might be
[17:28]	Jessica Kabumpo:	POST support needs to be improved upon for llHTTPRequest()
[17:29]	Andrew Linden:	I think Kelly Linden has moved on to some llHttpRequest() related overhaul or new functionality.
[17:29]	Andrew Linden:	I don't know much about it right now.
[17:29]	Jessica Kabumpo:	hte problem is I have to use php://input to read data .... that doesn't always work because of how my service is set up
[17:29]	Jessica Kabumpo:	$_POST always works, though
[17:31]	Andrew Linden:	Well, since conversation has lulled... I should mention that I'm sarting anothter little freeze on some bug fixes to push through QA for... somme release about two weeks away.
[17:31]	Saijanai Kuhn:	we're looking at using reverese http for the python test harness commands even before it appars in server protocols. Hadn't even considered doing any testing of textures
[17:32]	Andrew Linden:	This week's update has been delayed to next week.
[17:32]	Rex Cronon:	btw. i have a problem, but i don't know if is h4 related
[17:32]	Andrew Linden:	What is the problem Rex?
[17:33]	Rex Cronon:	i have huds that use llsettext, for some reason the text doesn't update. i have to go into edit mode, select object and than the text updates
[17:33]	Andrew Linden:	hrm... sounds related to a bug that CG Linden was telling me about.
[17:33]	Saijanai Kuhn:	does that happen with any viewer?
[17:33]	Andrew Linden:	That sounds like an "interestlist" bug.
[17:33]	Rex Cronon:	actually i have similar problem with huds that have prims that should rotate, but don't until i select them
[17:34]	Andrew Linden:	How reliable do your objects reproduce the problem Rex?
[17:34]	Rex Cronon:	i use the rc
[17:34]	Rex Cronon:	almost every time i tp to a sim it happens
[17:34]	Andrew Linden:	oof... are you talking about llTargetOmega() update bugs?
[17:35]	Rex Cronon:	i don't know if the rotation is related to targetomega
[17:35]	Andrew Linden:	Rex, gimme a copy of the problem objects with instructions on how to use it, and then tell me the Jira #. Interestlist bugs are something I'm willing to work on.
[17:35]	Andrew Linden:	oh... they are just llSetRot() calls on the script? Probably the same thing.
[17:36]	Andrew Linden:	Er... not the same thing as llTargetOmega() but same as llSetText()
[17:36]	Rex Cronon:	for rotation i use llsetlocalrot
[17:36]	Arawn Spitteler:	I have SetText on a sensorrepeat script hud, but I thought it a Sensor Repeat Bug
[17:36]	Andrew Linden:	Arawn, give me a copy too.
[17:37]	Rex Cronon gave you scanner.
[17:37]	Andrew Linden:	Alright, I'll try to actually examine it tomorrow.
[17:37]	Rex Cronon:	just wear it, and type /7 sc ?
[17:38]	Arawn Spitteler gave you Rail follower.
[17:38]	Andrew Linden:	ok thanks
[17:38]	Andrew Linden:	Rex, does the problem only happen when they are attached as HUD's?
[17:38]	Rex Cronon gave you compass_with_points.
[17:38]	Rex Cronon:	the compass. just attach it
[17:38]	Arawn Spitteler:	The Rail Follower, Should be worn, to make you follow the rails, so you only need to turn towards them. At the boarders, it just goes to a faster llSensorRepeat, and stays there.
[17:39]	Rex Cronon:	i don't attach usually outside
[17:39]	Andrew Linden:	ok, just curious. HUD's are special case objects, especially where the interestlist is concerned.
[17:40]	Rex Cronon:	it might all be just lag. but i am not sure
[17:40]	Andrew Linden:	(the "interestlist" stuff is the code on the simulator that figures out which data to send to your SL client)
[17:40]	Arawn Spitteler:	I was using a HUD, before expanding the script into a better rail follower, so I'm new to where the script might be failing.
[17:40]	Jessica Kabumpo:	what's the status on the teleport-on-bump feature?
[17:41]	Andrew Linden:	Jessica, elaborate on that "teleport-on-bump" feature. I'm unfamiliar with it.
[17:41]	Rex Cronon:	oh, and i few days ago i saw somebody sitting on object that i think was using target omega. when somebody else sat on it the whole thing stopped rotating
[17:41]	Jessica Kabumpo:	its where you walk into an object, and it teleports you somewhere
[17:42]	Arawn Spitteler:	That shouldn't happen.
[17:42]	Andrew Linden:	... teleports you somewhere arbitrary? It doesn't teleport you home?
[17:42]	Andrew Linden:	Do you know of an example object that does that? Where is it located?
[17:42]	Kitto Flora:	Is that a bug, or a new feature request?
[17:43]	Jessica Kabumpo:	I remember it being in the system, some of the older stargates require it.
[17:44]	Jessica Kabumpo:	back in 2006, I could dial a stargate, walk through it and bam, I'm teleported
[17:44]	Arawn Spitteler:	Some weeks ago, we discussed llTeleportAgent(), and it seems we haven't got it, due to potential errors.
[17:44]	Andrew Linden:	so... there is an llTeleportAgent() LSL call that doesn't work right?
[17:45]	Andrew Linden:	Does someone have a jira # on that?
[17:45]	Rex Cronon:	i have the same problem with the rotating hud prim right now. after i click on it in selcts made than it updates rotation
[17:45]	Rex Cronon:	select mode*
[17:45]	Jessica Kabumpo:	I think so, I don't see how else the stargate network would work with simply walking through it
[17:46]	Jessica Kabumpo:	but yea, llTeleportAgent would be nice to have work properly. its annoying to have to sit on a stargate and then be sit-glitched on arrival
[17:46]	Kitto Flora:	Maybe trap the av i na cage and warp the cage to destination?
[17:47]	Arawn Spitteler:	The Cage would take off without hte Avie, unless sat on.
[17:47]	Jessica Kabumpo:	I don't think that would work for the stargate network in SL.
[17:47]	Andrew Linden:	No that wouldn't work Kitto. Any cage warp would leave the avatar behind.
[17:47]	Jessica Kabumpo:	besides, not all gates are on the mainland
[17:47]	Kitto Flora dunno then.
[17:48]	Kitto Flora:	AFAIK llTeleportAgen() only works for 'home'.
[17:48]	Arawn Spitteler has only seen stargates that give SLURL
[17:48]	Andrew Linden:	Jessica, where is an example bugged stargate?
[17:48]	Jessica Kabumpo:	no, that's llTeleportAgentHome
[17:48]	Kitto Flora:	(exactly)
[17:48]	Jessica Kabumpo:	I have a landmark for one that's nearby, you have to sit on the gate to go through it
[17:48]	Jessica Kabumpo:	the one in FurNation is probably the same way, but I've only arrived at that one
[17:49]	Jessica Kabumpo gave you Kokoro no Heian, Gingivere (56, 38, 419).
[17:49]	Yuu Nakamichi:	any news on megaprims?
[17:50]	Sidewinder Linden:	heh
[17:50]	Yuu Nakamichi:	I heard you had an internal discussion on it?
[17:50]	Arawn Spitteler:	I was able to rez on a Megaprim, has that changed?
[17:50]	Sidewinder Linden:	interesting... word travels fast... just curious - "where'd ya hear that"?
[17:50]	Sidewinder Linden:	so we had an internal discussion
[17:50]	Simon Linden wasn't involved
[17:50]	Yuu Nakamichi:	:)
[17:50]	Andrew Linden:	Sidewinder, I told them I would be attending that meeting in the last meeting.
[17:50]	Arawn Spitteler recalls the internal discussion was scheduled for 4PM, today.
[17:50]	Sidewinder Linden:	i will be publishing some information once we've done some internal followup
[17:50]	Sidewinder Linden:	ahhh
[17:51]	Yuu Nakamichi:	yeah Side we had a long disucssion here last time
[17:51]	Arawn Spitteler:	Hi, Ella
[17:51]	Andrew Linden:	Well, we discussed my proposal and someone else wanted megaprims re-enabled pronto.
[17:51]	Ellla McMahon:	Hi everyone :)
[17:51]	Yuu Nakamichi:	hehe
[17:51]	Rex Cronon:	hi ella
[17:52]	Yuu Nakamichi:	Hi ellla
[17:52]	Andrew Linden:	I've got a few bits I can share Yuu...
[17:52]	Yuu Nakamichi:	sure
[17:52]	Andrew Linden:	I'm not the only LL dev who has concerns about technical problems with megaprims.
[17:52]	Arawn Spitteler just built Spirit City a new sandbox platform, and is wondering if it has to be paved in the next 35 minutes
[17:52]	Jessica Kabumpo:	megaprims are useful if used properly, one area I frequent is made of almost exclusively megaprims
[17:53]	Andrew Linden:	Steve and Don brought up a bunch of other problems that I haven't been talking about.
[17:54]	Arawn Spitteler heard something about MegaPrims causing Lag, but that could be Urban Legend
[17:54]	Andrew Linden:	In particular, there are some problems with the interestlist itself...
[17:54]	Sidewinder Linden:	arawn... one thing we will be doing is to establish more facts about various angles of this, and that is why i'd said i would publish some more information once we've done more internal work on this... :)
[17:54]	Jessica Kabumpo:	transgender lounge - the dance floor and the structure on it is all made of megaprims - 20x20 I think
[17:54]	Arawn Spitteler:	interestlist is soundijng high on the interest list
[17:55]	Jessica Kabumpo:	doesn't lag me at all
[17:55]	Andrew Linden:	optimizations have been made based on assumptions about max size of prims and it isn't clear what is currently broken in the interestlist regarding megaprims.
[17:55]	Simon Linden:	InterestList is really a core part of the simulator - it figures out what updates to send where
[17:55]	Andrew Linden:	but one item I've mentioned in the past was brought up... should we allow attachment of megaprims? Probably not
[17:56]	Arawn Spitteler wants a 60 meter crown
[17:56]	Rex Cronon:	i think i already saw people doing that
[17:57]	Andrew Linden:	Attachment of megaprims would be an example of content that would probably not be supported in any long-term megaprim solution... there are a lot of technical problems there and probably very little really cool non-griefy mega-attachment content.
[17:57]	Jessica Kabumpo:	I have a megaprim chocolate chip cookie. I think its 20m
[17:57]	Andrew Linden:	But no decisions have been made yet.
[17:58]	Jessica Kabumpo:	attaching it does weird things
[17:58]	Rex Cronon:	but the h4 physics engine is not affected at all by attachemens. right?
[17:59]	Andrew Linden:	Correct Rex, attachments aren't included in the physics engine.
[18:00]	Andrew Linden:	Regardless of what happens on the megaprim front... the prim encroachment and the collisions across region boundaries are projects that have their own merit.
[18:00]	Andrew Linden:	And are projects that I hope to complete this year.
[18:01]	Rex Cronon:	good luck with them
[18:01]	Yuu Nakamichi:	I'm looking forward to 1.22 - many fixes that we have been waiting for
[18:02]	Andrew Linden:	Yes, 1.22 is overdue.
[18:02]	Andrew Linden:	It was delayed for some problem... not sure what it was, but it wasn't a bad Havok4 bug fix that held it up.
[18:03]	Andrew Linden:	Fixed for 1.23, whenever that is: the "avatar drifts up when hovering" problem - fixed
[18:04]	Jessica Kabumpo:	it would also be nice to have another way to recover objects that fly onto no-autoreturn parcels one doesn't have access to
[18:04]	Andrew Linden:	I also fixed a physics engine crash mode relating to some bad attachments...
[18:05]	Sidewinder Linden:	/sry guys... i'm going to have to jump... good to see you...
[18:05]	Andrew Linden:	er woops... not about bad attachments, but about unlinking certain scripted objects.
[18:05]	Rex Cronon:	bye sidewinder
[18:05]	Yuu Nakamichi:	bye side
[18:05]	Saijanai Kuhn:	laters
[18:06]	Andrew Linden:	As I mentioned... SVC-2014 is fixed... should be in 1.23 (hovering vatar slows down nearby dynamic objects)
[18:06]	Yuu Nakamichi:	phew
[18:06]	Yuu Nakamichi:	:)
[18:06]	Andrew Linden:	Hrm... looking back... some of these bugs have been hard to track down. The bug closure rate is slowing a bit.
[18:07]	Saijanai Kuhn:	80-20 rule?
[18:07]	Andrew Linden:	Yeah probably.
[18:07]	Jessica Kabumpo:	could be teh 99-1 rule I've been coming across in my web services
[18:07]	Jessica Kabumpo:	99% of the time is spent tracking down 1% of the bugs
[18:08]	Andrew Linden:	Oh right... I fixed an annoying "avatar goes into 'striding' or 'falling' animation when walking up some stairs or ramps". SVC-1937 is fixed.
[18:08]	Jessica Kabumpo:	there's an issue with stand scripts similar to that
[18:09]	Andrew Linden:	btw, I should mention that bug was only made fixable by a repro that was provided in a comment in the jira item.
[18:09]	Yuu Nakamichi:	heh
[18:09]	Jessica Kabumpo:	go to stand on someone, you're stuck in the falling anim
[18:09]	Andrew Linden:	Perhaps LL should pay out small rewards for very helpful jira comments and hints.
[18:09]	Rex Cronon:	i think that whoever made that comment, would be glad to know it was so usefull:)
[18:10]	Jessica Kabumpo can demonstrate with a victim
[18:10]	Yuu Nakamichi:	good idea!
[18:10]	Andrew Linden:	LL has this tool for showing appreciation to fellow colleagues, and I often wish I could also show appreciation to people who participate in the bug reports.
[18:10]	Yuu Nakamichi:	this could help raise the general quality of comments
[18:11]	Andrew Linden:	BTW, I was having a discussion today with fellow Lindens about the usefullness of jira.
[18:11]	Rex Cronon:	it might increase the nr of comments too:)
[18:11]	Arawn Spitteler:	it makes sense, to consider volunteers part of the team.
[18:11]	Andrew Linden:	Some people don't find it helpful, however it really makes some things easier.
[18:11]	Jessica Kabumpo:	I remember back in 2006, we had this profile voting thing ... like people could give other points for being cute or pretty or whatnot ... what happened to that?
[18:11]	Rex Cronon:	jira is kind of hard to navigate
[18:12]	Rex Cronon:	sometimes people report the same bug but under different names
[18:12]	Andrew Linden:	I neglect it somewhat, and often close bugs late or let bugs linger for a while, but it is invaluable for some issues
[18:12]	Arawn Spitteler can't remember where he Jiraed the difficulties in navigating the Jira.
[18:12]	Kitto Flora:	Jira, as it is, is overloaded by the huge number of 'bugs' being reported.
[18:12]	Ellla McMahon:	unfortunately it is often used when what people really want is support
[18:12]	Andrew Linden:	and I often go back to it to find the next issues I should be working on.
[18:13]	Andrew Linden:	Right, it isn't really a support pipeline.
[18:13]	Saijanai Kuhn:	I made one meta-jira jira. Factor scripting into its own category
[18:13]	Jessica Kabumpo:	!stand Ara
[18:13]	Andrew Linden:	However, if you have a problem, and you provide the jira# to a Linden, it makes it much easier for them to "remember" it. That is, they can write the # down and read it in detail later.
[18:14]	Andrew Linden:	If you just tell them your problem it may get lost in the daily rush and bustle.
[18:14]	Andrew Linden:	jira items on how to change jira...
[18:15]	Andrew Linden:	I think there is some Linden's in charge of wrangling and improving our jira setup... not exactly sure who they are.
[18:15]	Rex Cronon:	maybe what jira needs is one or more librarians, that know all the bugs related to a spcific field, and both lindens and users can go to these person to find out what is new
[18:16]	Kitto Flora:	Would be better to have a well managed jira list of recognized problems and an entry jira list that users can enter bugs on, which are FULLY triaged into responses / filing into the main list.
[18:16]	Andrew Linden:	Well, I'm going to have to run.
[18:16]	Ellla McMahon:	thank you Andrew :)
[18:17]	Arawn Spitteler:	A tree, of some sort, so the more experienced can distinguish where the bug is, but the newer won't have to distinguish VWR from DOC
[18:17]	Rex Cronon:	bye andrew. and hopefully it will be an easy bug with the HUDs:)
[18:17]	Jessica Kabumpo:	!stand And
[18:17]	Saijanai Kuhn:	later all