User:Which Linden/Office Hours/2008 Feb 7

Transcript of Which Linden's office hours:

[11:13]	Which Linden:	Hi Phli
[11:14]	Phli Foxchase:	Wich, you are working on https ?
[11:14]	Which Linden:	Certified HTTP
[11:14]	Phli Foxchase:	what's the difference ?
[11:14]	Which Linden:	Well, https is http tunneled over SSL
[11:14]	Saijanai Kuhn:	its encrypted
[11:15]	Phli Foxchase:	ok
[11:15]	Which Linden:	And Certified HTTP is a spec for adding some additional information to a http message so that it can be idempotent
[11:15]	Which Linden:	so Certified HTTP can be done over https as well
[11:15]	Phli Foxchase:	ok
[11:15]	Which Linden:	I am beginning to see why 'Certified HTTP' is not the awesomest of names
[11:16]	Which Linden:	'
[11:16]	Saijanai Kuhn:	crazy HTTP sounds easier to work with ;-)
[11:16]	Which Linden:	hah
[11:16]	Which Linden:	Certified HTTP does sound very heavy-weight and expensive
[11:17]	Phli Foxchase:	Linden Lab works on certified http and https both ?
[11:17]	Which Linden:	We don't work on https
[11:17]	Phli Foxchase:	ok
[11:17]	Saijanai Kuhn:	up... the CAPs and login are on https...
[11:17]	Which Linden:	I mean, we use it, but we're not responsible for the standard
[11:17]	Which Linden:	If that's what you meant
[11:17]	Which Linden:	If you meant, do we deal with it on a daily basis, then yes
[11:17]	Phli Foxchase:	Yes
[11:18]	Phli Foxchase:	and where do you want to put certified http?
[11:18]	Phli Foxchase:	Ims ?
[11:18]	Which Linden:	Certified HTTP is going to be purely internal, for L$ transactions
[11:18]	Which Linden:	At first
[11:18]	Saijanai Kuhn:	IMS at least for groups, should probably stay UDP
[11:19]	Which Linden:	The agent domain will change a lot of the design decisions we've made so far
[11:20]	Which Linden:	I.e. group chat over udp makes sense now cause there are so many recipients of it
[11:20]	Saijanai Kuhn:	for IM? I hope that just thinking about thingswould lead to redesigns for IM
[11:21]	Which Linden:	when we move to agent hosts, there would be a much higher density of agents per agent host, so it'd make sense to send one tcp message to each host for all the agents hosted thereupon
[11:21]	Saijanai Kuhn:	hmmm. So would the Agent domain be responsible for gorup IM, or just for tracking the destination of group IM?
[11:21]	Which Linden:	well the location of the agent moves to the agent host rather than the sim
[11:22]	Phli Foxchase:	and what protocole do you plan to replace ?
[11:22]	Saijanai Kuhn:	asure, but wouldn't it be better to offload the agent domain as much as possible too?
[11:22]	Phli Foxchase:	with certified http*
[11:22]	Saijanai Kuhn:	http://wiki/secondlife.com/wiki/ImprovedInstantMessage is the current protocol
[11:22]	Which Linden:	Phli: I guess what we'll be replacing is mysql transactions
[11:22]	Saijanai Kuhn:	I've got theat memorized. Ishould make a hot key for it in the client LOL
[11:23]	Which Linden:	Phli: right now we have lots of transactions going through a central database
[11:24]	Phli Foxchase:	asset server ?
[11:24]	Which Linden:	And we'd like to distribute those transactions over lots of agent stores
[11:24]	Which Linden:	not the asset server, the central database
[11:24]	Phli Foxchase:	ok
[11:24]	Which Linden:	eventually the goal is to have every Resident's data live on a particular database cluster
[11:25]	Which Linden:	and have many such clusters
[11:25]	Which Linden:	each with their own distinct population of Residents ("agents")
[11:25]	Which Linden:	We call them agents when we're referring to the code
[11:25]	Which Linden:	Not sure why
[11:26]	Which Linden:	I guess the Resident is the person, and the Agent is the representation in our system of the Resident's will
[11:26]	Saijanai Kuhn:	Zero said that you guys had worked out a glossary for some o fthis stuff
[11:27]	Which Linden:	Hah, we do have a linden glossary
[11:27]	Saijanai Kuhn:	this was a recent meeting. Might not be the same glossary
[11:27]	Which Linden:	"the digital representation of a Second Life Resident who is logging in or logged into a Second Life grid. "
[11:28]	Which Linden:	Mebbe
[11:28]	Which Linden:	Phli: check this out: http://wiki.secondlife.com/wiki/Agent_Domain
[11:28]	Which Linden:	This is what we're talking about
[11:29]	Phli Foxchase:	Do you have a date for when certified http will be ready ?
[11:29]	Which Linden:	This quarter
[11:29]	Which Linden:	Come hell or high water
[11:29]	Which Linden:	It's mostly done, you can start using it now if you want to
[11:29]	Which Linden:	There are just some loose ends
[11:29]	Saijanai Kuhn:	Which you mentioned standards. Will this stuff become part of the standard, or are you working within an existing standard?
[11:30]	Which Linden:	Standards.... um, hmmm.
[11:30]	Which Linden:	Well, we are calling it a "standard" or a "spec" despite the fact that we're the only users
[11:30]	Saijanai Kuhn:	[11:17] Which Linden: I mean, we use it, but we're not responsible for the standard
[11:31]	Which Linden:	Oh, I meant SSL the standard
[11:31]	Which Linden:	Just now I thought you were referring to Certified HTTP
[11:31]	Which Linden:	Certified HTTP is not a standard, but hopefully it will be
[11:31]	Saijanai Kuhn:	I thought you meant that chttp was or was becoming a standard and you guys were working on it as a standard (not just internally)
[11:31]	Which Linden:	Some standards body has to bless it I think before it can be a standard
[11:32]	Which Linden:	Yes, we want to be interoperable in that regard
[11:32]	Saijanai Kuhn:	yeah, total confusion. I thought you meant you were already working with a standards body or were going to soon or something. My bad
[11:32]	Which Linden:	Not any time soon.
[11:32]	Which Linden:	I think the plan is for it to become a de facto standard first
[11:32]	Saijanai Kuhn:	well, sounds closer than the SL protocols LOL
[11:33]	Which Linden:	If you can't even get your shit to be a de facto standard then it probably sucks
[11:33]	Saijanai Kuhn:	...
[11:33]	Which Linden:	Heh, yeah, well, you're helping us turn the sl protocols into something semi-standard
[11:33]	Which Linden:	Bringing us out of proprietary/anarhy
[11:33]	Saijanai Kuhn:	thats my goal (wishes I could get paid for it though)
[11:34]	Which Linden:	I hear ya
[11:34]	Saijanai Kuhn:	Zha says wonce I'm further along with IM, she'll back me in pitching a call to hire me for that
[11:34]	Which Linden:	You would, at that point, probably know more about IM than any individual Linden
[11:34]	Saijanai Kuhn:	gotta show more functiong work to make it have a chance
[11:35]	Saijanai Kuhn:	well, once I get that table filled in, and a chart like you have here created, it should be possible for anyone to "know" as much
[11:35]	Saijanai Kuhn:	IM isn't that complicated in SL. Its just really REALLY obscure
[11:36]	Which Linden:	Yup
[11:36]	Which Linden:	Getting back to how it might be affected by agent domain though
[11:36]	Saijanai Kuhn:	though, just glancing at that table tells me a lot about the kinds of errors we see. LIke me sending a group IM and it showing up as individual IM. Or TPs going bad at the same time as IM
[11:37]	Which Linden:	Those things happen?
[11:37]	Saijanai Kuhn:	yeah. I spam a group with Zero's office hours and I get private calls from folks compliaing about spam
[11:37]	Saijanai Kuhn:	Last week my python script suddenly started showing messages but mostly to inidivuals
[11:38]	Saijanai Kuhn:	I found a bug in the code and suddenly the individuals stopped getting mesages, but group IM showed nothing but my name and no message
[11:38]	Saijanai Kuhn:	so I fixed one bug and uncovered another
[11:38]	Which Linden:	:-/
[11:39]	Saijanai Kuhn:	today I spammed Metaverse wth Zero's office hours and got flamed bigtime for spamming private IM
[11:39]	Saijanai Kuhn:	that was with the main client, not my scirpt
[11:39]	Which Linden:	So it effectively seemed like you mass-sent private ims to the members of teh group?
[11:39]	Saijanai Kuhn:	yep
[11:40]	Saijanai Kuhn:	some people say that they'll get error messages about a group session not starting, which automatically closes the group window... on a bunch of messages they're reading
[11:40]	Which Linden:	My understanding is that group and personal ims go over rather different routes
[11:41]	Saijanai Kuhn:	is all the same packet though
[11:41]	Which Linden:	Once it gets to the viewer, yes
[11:41]	Saijanai Kuhn:	so if there's a bug in the client for packet handling, or the packet sending...
[11:41]	Which Linden:	So the sim can corrupt it
[11:41]	Which Linden:	Or the client can
[11:41]	Which Linden:	Yah
[11:42]	Saijanai Kuhn:	anything that corrupts that packet in the middle would change the value of Dialog byte and boom, you've got strangeness
[11:42]	Which Linden:	With agent domain, I'd expect this code path to be significantly cleaned up
[11:42]	Saijanai Kuhn:	sure hope so ;-)
[11:42]	Which Linden:	It'd be a lot easier to locate the agent since they'd live on a particular agent host
[11:42]	Saijanai Kuhn:	right now, its still confusing to me though. blank messages adie, I still don't understand EvetnQueueGet
[11:43]	Saijanai Kuhn:	blank messages aside*
[11:43]	Which Linden:	Oh, well, um, hmm...do you want more details, or is the general concept eluding you?
[11:43]	Saijanai Kuhn:	...details please
[11:43]	Saijanai Kuhn:	or rather, let me tell you what I'm doing...
[11:44]	Which Linden:	sure
[11:44]	Saijanai Kuhn:	I get the cap for event queue get. I send the ack with an undef/boolean/false
[11:44]	Saijanai Kuhn:	get the first ID. Send that back in the ack with a boolean/true
[11:45]	Saijanai Kuhn:	I would *expect that the next time I do that, the old event would be cleared, but thats not what is happening
[11:45]	Which Linden:	wait you get teh cap over udp?
[11:45]	Saijanai Kuhn:	I'm getting the new event accumjulated in the old event
[11:45]	Saijanai Kuhn:	I'm sending the ack over the EventQUeueGet CAP
[11:45]	Which Linden:	I see
[11:46]	Which Linden:	Ummm.... my event-queue-fu is weak
[11:46]	Saijanai Kuhn:	So it *seems* like I'm never clearing the queue. Each new event is tacked into the same event array
[11:46]	Which Linden:	lemme looks some stuff up
[11:46]	Which Linden:	right, yeah, so perhaps the ack isn't being done properly
[11:46]	Saijanai Kuhn:	might be. I'm getting a new AID at the end of each new event though and its always +1 over what I sent
[11:47]	Saijanai Kuhn:	maybe its a problem with my boolean
[11:47]	Saijanai Kuhn:	so it never closes
[11:48]	Saijanai Kuhn:	the other up, is probably some UDP packet issue which I can just "figure out" eventually
[11:48]	Saijanai Kuhn:	the other prob*
[11:48]	Saijanai Kuhn:	firing up eclipse...
[11:48]	Which Linden:	man, is there any documentation anywhere about the llsd structure of the event queue?
[11:49]	Saijanai Kuhn:	my original packet issue was I said IM_packet = ... position + postion instead of IM_packet == position + UUID...
[11:49]	Saijanai Kuhn:	sigh...
[11:49]	Saijanai Kuhn:	then my next problem was I was sending the wrong length byte for my name... hmmm Maybe I need to check, that again. Name is coming through but not message...
[11:50]	Saijanai Kuhn:	anyway, cclipse is still loading...
[11:50]	Which Linden:	vc still loading for me :-)
[11:50]	Saijanai Kuhn:	afk while waint gof Java VM to start
[11:53]	Which Linden:	the ack seems to be per-message
[11:53]	Which Linden:	not per-message
[11:54]	Which Linden:	per http request
[11:54]	Which Linden:	i.e. you ack the entire queue
[11:55]	Saijanai Kuhn:	that makes sense... but the entire event queu comes in as one LLSD-XML array? DIctinoary?
[11:55]	Saijanai Kuhn:	so if I ack it I would expect the entire thing to clear...
[11:56]	Which Linden:	yah
[11:56]	Saijanai Kuhn:	if thats not the behavior, then its prety wasteful, I think, since you would accumualte every packet ever sent forever
[11:56]	Saijanai Kuhn:	Though I've seen some libsl packets that suggest that that is happening
[11:56]	Which Linden:	yeah so you get a structure that looks like this: {'id': 1234, 'events': [...] }
[11:56]	Saijanai Kuhn:	The *client* seems to be able to clear at least some things
[11:57]	Saijanai Kuhn:	right, I send the undef and get back the packets with id 1234
[11:57]	Which Linden:	ok
[11:57]	Saijanai Kuhn:	I send the 1234 and get back teh same packet with id 12345
[11:57]	Saijanai Kuhn:	1235
[11:58]	Which Linden:	yup
[11:58]	Which Linden:	my reading of the code is that if you POST a structure looking like {'ack':1234} then it should do the right thing
[11:58]	Saijanai Kuhn:	so that accumualtes each new item. is there anyway to clear it?
[11:59]	Which Linden:	I think that it should be clearing it
[11:59]	Saijanai Kuhn:	oh bother... My reinstalling of the OS seems to have reinstalled an ancient version of my source...
[12:00]	Saijanai Kuhn:	wiat wrong file (where)
[12:00]	Saijanai Kuhn:	wherw
[12:01]	Which Linden:	hey, why don't you send me your code?
[12:01]	Which Linden:	it's possible that there's a server bug, who knows
[12:01]	Saijanai Kuhn:	OK. I'd life to take a moment to tidy it up first though. Huge blocks of comments and stray comments and so on
[12:01]	Which Linden:	ok, cool
[12:02]	Which Linden:	it does seem like we're at the point where I'm saying "that should work" and you're saying "but it doesn't" :-)
[12:02]	Which Linden:	Anyway, that was fun digging through that old code
[12:02]	Saijanai Kuhn:	like I said position+ poistion instead of poistion +UUID...
[12:02]	Saijanai Kuhn:	sighs
[12:03]	Which Linden:	I gotta get lunch though
[12:03]	Saijanai Kuhn:	for some reason the server didn't like 3 zero FP values intead of 16 bytes of zero
[12:03]	Saijanai Kuhn:	KK will tidy it up and send it to you
[12:03]	Which Linden:	Awesome
[12:04]	Which Linden:	Take it easy!
[12:04]	Saijanai Kuhn:	byut eyeballing it, i didn't notice 12 bytes instead of 16