User:Zero Linden/Office Hours/2008 Jan 17

Transcript of Zero Linden's office hours:

[8:43]	Harleen Gretzky:	Zero Linden's office hour
[8:43]	Wyn Galbraith:	Hello.
[8:43]	Harleen Gretzky:	Hi Zero
[8:43]	Zha Ewry:	In a Linden office, full of gearheads
[8:43]	Teravus Ousley:	greetings
[8:43]	Zha Ewry:	Hey Zero!
[8:43]	Rex Cronon:	u r where a tech meeting is supposed to take place
[8:43]	Thoys Pan:	hello zero
[8:43]	Rex Cronon:	hi zero
[8:43]	Saijanai Kuhn:	goood morning teacher
[8:43]	Dr Scofield:	hi zero
[8:43]	SignpostMarv Martin:	hi Zero
[8:44]	Wyn Galbraith:	Greetings Zero ;)
[8:44]	Wyn Galbraith:	You could join us on Squirrel's zero :)
[8:44]	katalina Zerbino:	what is an avatar?
[8:44]	Arawn Spitteler rezzed a chair, to save on lag, by sitting on a physical object...
[8:45]	Neas Bade:	morning zero
[8:45]	luckdog Enzo:	the flags is nice,but how about my country,the flag of china.?
[8:45]	Zero Linden:	Sorry all
[8:45]	Zero Linden:	Was in an "engineering directors" meeting that was the only time it could be held this week
[8:45]	Zero Linden:	oy vey
[8:45]	luckdog Enzo:	thnx very much
[8:45]	SignpostMarv Martin:	yw
[8:45]	Kerry Giha:	Ah the life of meetings
[8:46]	Zero Linden:	Welcome to my office hours all
[8:46]	Saijanai Kuhn looks alert (whats up with that?)
[8:46]	luckdog Enzo:	lol~its nice
[8:46]	TomHa Zymurgy:	hav a sit ;)
[8:46]	Teravus Ousley figures it might be good to jump right in because of ~15 minutes less? :D
[8:46]	Zero Linden:	So, I had a wonderful face-to-face meeting with Adam Z. on Tuesday
[8:46]	Zha Ewry:	So.. If we don't have a topic Zero, I was going to pester you, merrciless ly, about the long poll, and how we plan to use it and such in the next round of stuff
[8:47]	Zero Linden:	He was in town - we ate Indian food
[8:47]	Neas Bade:	awesome
[8:47]	Teravus Ousley:	:D
[8:47]	Zha Ewry pricks up her ears at Adam's name
[8:47]	Saijanai Kuhn:	Adam Z...
[8:47]	Zha Ewry wonders if it was the same nice little Indian down the street from Zero's office
[8:47]	Thoys Pan:	Adam azure islands?
[8:47]	Dr Scofield:	LL going to switch to OpenSim... ;-)
[8:47]	Zero Linden:	He told me a bit about the way OpenSim runs the scripts in a separate script server
[8:48]	Zero Linden:	which seems much like the "script interface" we've been talking about at the last few meetings
[8:48]	Zha Ewry nods
[8:48]	Zha Ewry:	Closely related in any case
[8:49]	Thoys Pan goes brb (souptime in holland)
[8:49]	Rex Cronon:	"script interface" talk? i guess reading the transcripts would clear that.
[8:49]	Zero Linden:	As currently defined, it is pretty heavily reliant on .Net structures, and works via RPC, but it might serve as a model for a
[8:49]	Zero Linden:	open protocol there
[8:50]	Zha Ewry nods
[8:50]	Zero Linden ducks after realizing he's been remiss about the transcripts
[8:50]	Zha Ewry:	one would want a set of web service style notifcations for that
[8:50]	Neas Bade:	there is a latency problem with using web services on that interface
[8:50]	Saijanai Kuhn has a pointer to zero's transcripts on teh AW Groupies page. Its loking barren
[8:50]	Zha Ewry:	Not web services, but web services mediated notifcatoin
[8:50]	Neas Bade:	you really want something fast and low latency
[8:50]	Zha Ewry:	pub/sub low level
[8:50]	Zha Ewry:	but, controlled by caps
[8:50]	Neas Bade:	ok, right
[8:51]	Saijanai Kuhn:	TCP?
[8:51]	Zha Ewry:	Which is, in general, going to be an issue with the whle long term design for things like managing any UDP/low level stuff in the protocol
[8:51]	Dr Scofield:	saijanai, this is the HTTP-and-HTTP-only camp
[8:51]	Zha Ewry:	Its all sort of "Well, we just send stuff on the cirucuits"
[8:52]	Zero Linden:	Well, lessee
[8:52]	Zha Ewry:	and the client hopefully swallows it
[8:52]	Zero Linden:	we've often assumed that hte Agent Domain and the Region Doain have to be addresable - that is
[8:52]	Zero Linden:	they are not behind NAT firewalls that don't allow incoming connections
[8:52]	Zha Ewry nods
[8:52]	Zero Linden:	in that case, one can do pub/sub pretty easily with HTTP and caps
[8:52]	Zha Ewry:	But.. at what performance cost
[8:52]	Catherine Pfeffer:	(as long as there's no dependancy towards .net everything is fine... lol)
[8:52]	Zero Linden:	you do it by registring a cap that is the service you want events posted to (the sub)
[8:53]	Dr Scofield grins
[8:53]	Zero Linden:	and then the generator of the event just needs to invoke all the registered caps when the event fires (the pub)
[8:53]	Saijanai Kuhn is a mac user.Everything is .net already... isnt' it? ;-)
[8:53]	Zha Ewry nods at Zero
[8:53]	Zero Linden:	a pub/sub aggrigator could be used
[8:53]	Zha Ewry:	Things like that feel right
[8:53]	Zero Linden:	I think, one might even be able to put a small amount of convention in those calls so that
[8:53]	Catherine Pfeffer:	(sorry didn't mean to troll... not really)
[8:53]	Zero Linden:	(which is always the trick to making pub/sub efficient, if I understand it at all)
[8:53]	Zha Ewry:	where we use the high level caps stuff to control the low level
[8:53]	Zha Ewry:	event flow
[8:54]	Zha Ewry:	And aboslutely, Zero
[8:55]	Zha Ewry:	There's a huge amount of work been done on how to let the low level stuff do a little bit of control. to keep from having to do the expensive operations
[8:55]	Zha Ewry:	What you want to avoid, is letting that get out of hand, tho
[8:56]	Zero Linden:	So - this leads into the event queue and long poll I think
[8:56]	Zha Ewry nods
[8:56]	Zha Ewry:	The event queue, and the circuits, even
[8:57]	Zero Linden:	both of these are mechanisms to get the semantics of REST across a path that can't support it directly
[8:57]	Saijanai Kuhn:	and how that actually works. Serveral lindens have attempted to pour the knowledge into the head but its leaking
[8:57]	Zero Linden:	We could imagine other, lower latency paths, that still support the REST calls, but don't use an HTTP/TCP connection per request
[8:57]	Zero Linden:	surely there must be already something out htere that does this
[8:58]	Zero Linden:	other than, say, Pipelined HTTP, which has a host of problems
[8:58]	Dr Scofield doesn't dare to mumble "xmpp"
[8:59]	Zero Linden:	well- but xmpp doesn't - there isn't a strong request/response correllation is there? and the addressing is - well, Baroque at best
[8:59]	Dr Scofield:	IQ message
[8:59]	Zero Linden:	but I imagine one could cast HTTP semantics into XMPP, and that is perhaps not so bad
[8:59]	Dr Scofield:	IQ is supposed to be suc
[8:59]	Dr Scofield:	sync
[9:00]	Zero Linden:	hmmm google doesn't turn up a link to anything like IQ message
[9:00]	Neas Bade:	it would be nice to not get too far ahead of ourself here, and actually have some sort of REST semantics before trying to cast it into something else :)
[9:00]	Zero Linden:	hehe
[9:00]	Dr Scofield:	you send IQ request and get a result back
[9:00]	Dr Scofield:	XMPP RFC
[9:00]	Dr Scofield:	s
[9:01]	Zero Linden:	well, Neas, one thing I think Zha is trying to be sure of is that all the styles of communication we are going to need can be covered
[9:01]	Zha Ewry:	Also.. I think there is a case to be made for building it as simply as possible, and seeing where we need more than what we can get from base http.
[9:01]	Dr Scofield:	xmpp message is async
[9:01]	Dr Scofield:	xmpp iq sync
[9:01]	Zero Linden:	oh oh - I see on IQ, sorry, thought it was a separate protocol
[9:01]	Zha Ewry:	And, yes, tha is on my list
[9:01]	Neas Bade:	zha, absolutly agreed
[9:01]	Zero Linden:	So, event queue
[9:02]	Zero Linden:	details? It is simple: the viewer can't be contacted via HTTP
[9:02]	Zha Ewry:	Its really easy, to convince yourself you're going to *need* a fancy new widget, when base http will do. Now, we know there will be cases where we want better, but I am far from sure anyuone knows the line between those and the http is good enough cases
[9:02]	Neas Bade:	I'd rather have us prove http won't work first before trying to do something else, instead of guessing that it won't up front for most things
[9:02]	luckdog Enzo:	there are still so many questions for HTTP or ohers .we should have a better way to resoulve it.and the first is the net speed.
[9:02]	Zero Linden:	so instead, we remember a list of everythign we would have liked to send to the viewer via HTTP as a sequence of events
[9:02]	Dr Scofield:	neas, pipelined HTTP is reinventing the wheel, i think
[9:02]	Zero Linden:	and give the viewer a cap for fetching the events
[9:03]	Zero Linden:	when the viewer invokes the cap, it gets a sequence number and a list of events
[9:03]	Dr Scofield:	polling
[9:03]	Zha Ewry:	Right. So, you pull on that instead
[9:03]	Zha Ewry:	and when it has nothing to say, it says 'Try again later"
[9:03]	Zero Linden:	the simulator holds those sent events on the side, since it doesn't know if the viewr actually got 'em
[9:03]	Zero Linden:	the next time the viewer invokes the cap, it includes the sequence number of the last set of events it got
[9:04]	Zero Linden:	the simulator says - if the number matches the events on the side, ditch 'em, 'cause they got there
[9:04]	Zero Linden:	if it doesn't match, it sticks them back on the front of the queue, 'cause they got lost
[9:04]	Zha Ewry nods
[9:04]	Zha Ewry:	That process, then, is sort of lockstep?
[9:04]	Zero Linden:	lather - rinse - repeat
[9:04]	Zero Linden:	and then it bumps the sequence number and sneds the event queu again (moving them all aside)
[9:05]	Zha Ewry:	since the client asks
[9:05]	Zha Ewry:	Assumign a request doesn't get broken, we should see it just rachet up?
[9:05]	Dr Scofield:	hmmm...still think a bi-directional protocol would be better suited for that
[9:05]	Zero Linden:	yes, the sequence number just goes up each time
[9:05]	Zero Linden:	Dr. - so I agree
[9:05]	Teravus Ousley wonders if there's a spec somewhere with the events that get put into EventQueueGet
[9:06]	Zero Linden:	but the barrier to implementing XMPP was high-
[9:06]	Zero Linden:	Teravus
[9:06]	Zero Linden:	the events are just an LLSD version of messages right now
[9:06]	Teravus Ousley:	ah
[9:06]	Zha Ewry:	As if they had been put on the UDP cirtcuit, Zero?
[9:08]	Zha Ewry:	.
[9:08]	Rex Cronon:	zero crashed?
[9:08]	Zha Ewry wonders if Zero has crashed
[9:09]	Zero Linden:	the events have the form { message: m, body: b}
[9:09]	Zero Linden:	where b is the LLSD encoded message body
[9:09]	Zero Linden:	in theory, we were going to implement { request: r, body: b}
[9:09]	Zero Linden:	or rather { request: r, body: b, id: i }
[9:10]	Saijanai Kuhn:	there's something missing from that equation, though, Zero. When I send theack with the last event ID obtained, I get the same event with a new ID
[9:10]	Zero Linden:	which would be a REST semantics request expecting a respond during the poll
[9:10]	Zero Linden:	but that isn't spec'd or implemented
[9:10]	Rex Cronon:	shouldn't id be first?
[9:10]	Saijanai Kuhn:	do I need to close the connection and reopen it to stop getting the same event?
[9:10]	Saijanai Kuhn:	its ack/ID in LLSD format, Rex
[9:10]	Zero Linden:	Sai - I don't think so
[9:11]	Saijanai Kuhn:	OK, so I'm doing something wrong (as usual)
[9:11]	Thoys Pan is back
[9:12]	Zha Ewry wonders exactly how close to REST polling gets.. The event queue is full of events that don't have resources. I'm not suggesting that there is a good REST semantic, actually, just musing that it feels funny
[9:12]	Saijanai Kuhn:	you send the llsd-xml of ack and the id and it sends you back an llsd array of events
[9:12]	Zero Linden:	but the event set is sent as { id: n, events: [ ... ] }
[9:12]	Zero Linden:	the ack must be (body of the poll) { ack: n }
[9:12]	Saijanai Kuhn:	right
[9:12]	Zero Linden:	Zha - no, at present, the event queu does messages
[9:12]	Teravus Ousley listens
[9:12]	Rex Cronon:	until a prototype is made, u can't be sure how bad or good this is
[9:13]	Dr Scofield needs to go earlier today: RL avalanche training
[9:13]	Dr Scofield:	cu all
[9:13]	Saijanai Kuhn:	I send and receive stuff in the right format, but don't get what I expect. I'm either sending the wrong ID or not understanding
[9:13]	Zero Linden:	so, messages can be cast as POSTs to a /messages/<messagename>
[9:13]	Rex Cronon:	bye dr
[9:13]	Zero Linden:	where the requestor doesn't expect anything other than a null body resposne
[9:13]	Zero Linden:	messages are a degenerate REST request :-)
[9:13]	Neas Bade:	you'd still get a proper return code though, right?
[9:14]	Neas Bade:	20x or 40x depending
[9:14]	Zero Linden:	No - the message system didn't have any semantics of return code at all
[9:14]	Saijanai Kuhn:	<llsd><undef /> ?
[9:14]	Neas Bade:	so you have no idea if the message was accepted?
[9:14]	Zero Linden:	some messages have the semantic of the sender knows if it was received or no
[9:14]	Zero Linden:	and the event queue supports that by, when it gets the ack, it tells all the senders that cared that the message was received
[9:14]	Zero Linden:	but that is all
[9:14]	Saijanai Kuhn:	this seemes an abuse of CAP
[9:15]	Zha Ewry:	Not really.
[9:15]	Zha Ewry:	The cap, is just saying "here is how you polll"
[9:15]	Zero Linden:	so it is like a REST request where the sender checks if it got any HTTP return code or not -- but doens't even look a thte code
[9:15]	Thoys Pan:	btw, for intrested people i opensourced the ircchantoslgroupbot -> http://code.google.com/p/sl-group-irc-gateway/
[9:15]	Saijanai Kuhn:	but if you send a message and don't know if it arrived...
[9:15]	Zero Linden:	Sai - no more like <llsd />
[9:15]	Zha Ewry:	Well, the poll, is outside the caps abstraction
[9:15]	Zero Linden:	!
[9:16]	Zha Ewry:	Thus, the comment, about resources
[9:16]	Zha Ewry:	If you wanted to do it all pure rest, you'd die making all those messages into REST resrouces
[9:16]	Saijanai Kuhn:	OK, I cansorta see that
[9:17]	Zero Linden:	well - internlly, in the viewer, we cast all of those events into /external/message/<message-name>
[9:17]	Zha Ewry brushes signposts flag out of her face
[9:17]	Zero Linden:	(or some such), look up those resources and invoke POST on 'em with the body
[9:17]	Saijanai Kuhn:	so you can POST messages to EventQueueGet? or is that only with other caps?
[9:18]	Zero Linden:	there is a wild cared handler at /external/message/* that picks up any message that isn't implemented as an HTTPResponder
[9:18]	Saijanai Kuhn:	ah, ok. So thats the format for all CAPs other then EventQueueGet?
[9:18]	Zero Linden:	and turns around and looks up th emessage in the old message template system, finds the registered C++ function pointer, and calls it
[9:18]	Zero Linden:	doing the translation from LLSD to the old binary version on the fly
[9:19]	Zero Linden would kill for a shared white board right now
[9:19]	Zero Linden or even a non-shared one
[9:19]	Rex Cronon:	doesn't windows have that?
[9:20]	Rex Cronon:	oh, u mean in here
[9:20]	Saijanai Kuhn:	recalls one fo the IM services used ot have one but hasn't seenit lately
[9:20]	Teravus Ousley:	yeh, best way in here at the moment is to set up a vid stream
[9:20]	Zero Linden:	so - here's the path...........
[9:21]	Zero Linden:	Simulator wants to send message FooBar with {x:12, name: Baz} to the viewer
[9:21]	Thoys Pan:	zero, can notecards be created using NewFileAgentInventory?
[9:21]	Arawn Spitteler suggests Zarf might market through Etch-A-Sketch.
[9:21]	Zero Linden:	it could send it via UDP, binary encoded, or it could send it via the event queue
[9:22]	Zero Linden:	if it does the later
[9:22]	Zero Linden:	then { message: FooBar, body: { x: 12, name: Baz } } gets added to the EventQueue for the agent
[9:22]	Zero Linden:	Now -
[9:22]	Zero Linden:	the simulator waits
[9:23]	Zero Linden:	at some point, the viewer invokes the cap for it's event queue that it was given long ago, when it connected to this simualtor
[9:23]	Zero Linden:	the viewer does a POST to the cap w/ { ack: undef }
[9:23]	Thoys Pan:	Zero, the cap NewFileAgentInventory is used for Image uploading and other assets, but would it work for notecards?
[9:24]	Zero Linden:	the simulator responds with a response of { id: 912, events: [ { message: FooBar, body: { x: 12, name: Baz } } ] }
[9:24]	Zha Ewry:	Is the undef everytime, or just the first?
[9:24]	Zero Linden:	The viewer gets that body, pulls it apart and makes the requests that the simulator would have made if it could have
[9:25]	Zero Linden:	(Zha - just the first time)
[9:25]	Zero Linden:	so the viewer does an internal POST to /external/messages/FooBar with a body of { x: 12, name: Baz }
[9:26]	Zero Linden:	that either lands at an HTTPResponder that handles the mssage, or routes to the old message handler with LLSD to binary translation
[9:26]	Zero Linden:	the next time the viewer polls, it invokes the smae cap with { ack: 912 }
[9:26]	Zha Ewry suspects that Undef is what's killing Sai
[9:27]	Zero Linden:	so - there are two levels of REST going on here - the outer, done with caps, to enable getting the inner requests, tothe viewer
[9:27]	Saijanai Kuhn:	but the internal is all within the viewer. Thats how the SL viewer handles things. A new viewer, that doesn't know about the UDP, wouldn't be doing thigns that way
[9:27]	Lena Franciosa:	Hi everyone
[9:28]	Wyn Galbraith:	Hello Lena.
[9:28]	Rex Cronon:	jo
[9:28]	Rex Cronon:	i mean hi
[9:28]	Zero Linden:	Sai - sure it would - it just wouldn't bother implementing the connector back to the old call-back style of message handler, since it wouldn't have that
[9:28]	Neas Bade:	Zero: is there any mock up code to do any of this. I think I'm getting lost in the micro explanations and need some bits to make it all make sense to me
[9:28]	Saijanai Kuhn:	well it wouldn't be POSTing anything
[9:28]	Neas Bade:	like some python mock up or something
[9:29]	Zero Linden:	The new viewer would still have to contend with the fact that the simulator can't make an HTTP connection directly to the viewer
[9:29]	Zha Ewry:	I'd actually like a set of annotated flows, or such, and a simple pyhton that implements it
[9:29]	Zero Linden:	the viewer is behind NAT firewals
[9:29]	Zero Linden:	Neas - alas, this is all in C++ - though it is pretty simply C++
[9:29]	Zha Ewry:	Right. We have to asume the poor viewer is hiding behing a firewall, a NAT router, and a few buggy gateways
[9:29]	Saijanai Kuhn:	I send { ack: undef } to EventQueueGet
[9:30]	Neas Bade:	well, C++ is better than nothing
[9:30]	Zero Linden:	righ - and it doesn't have a stable IP address either!
[9:30]	Saijanai Kuhn:	I get back { message: FooBar, body: { x: 12, name: Baz } }
[9:31]	Wyn Galbraith has to run to work y'll.
[9:31]	Saijanai Kuhn:	rather I get back the { id: 912, events: [ { message: FooBar, body: { x: 12, name: Baz } } ] }
[9:31]	Wyn Galbraith:	B-bye!!! :D
[9:31]	Saijanai Kuhn:	I send {ack:912} and get back
[9:31]	Saijanai Kuhn:	{ id: 913, events: [ { message: FooBar, body: { x: 12, name: Baz } } ] }
[9:32]	Linx Newt:	aoummmmmm
[9:32]	Linx Newt:	hey hey :)
[9:32]	Rex Cronon:	bye wyn
[9:32]	Lena Franciosa:	hey linx
[9:32]	Saijanai Kuhn:	ack 913 gets back { id: 914, events: [ { message: FooBar, body: { x: 12, name: Baz } } ] }
[9:32]	Thoys Pan:	hello
[9:32]	Linx Newt:	c est une secte ?
[9:32]	Saijanai Kuhn:	so obviously I'm missing something
[9:32]	Linx Newt:	lol
[9:33]	Zero Linden:	Sai - here's the code
[9:33]	Lena Franciosa:	oui on va t'enroller
[9:33]	Lena Franciosa:	mdr
[9:33]	Zero Linden:	last
[9:33]	Catherine Pfeffer:	chut lynx
[9:33]	Zero Linden shout: last
[9:34]	Zero Linden:	there
[9:34]	Linx Newt:	c quand le suicide collectif ? :)
[9:34]	Lena Franciosa:	ssh
[9:34]	Catherine Pfeffer:	SILENCE
[9:34]	Zero Linden:	so you can see that there is always an outter map of {id:, evnets:}
[9:34]	Zero Linden:	actually - I need to run too
[9:35]	Zero Linden:	sigh
[9:35]	Zero Linden:	thanks all for coming
[9:35]	Rex Cronon:	bye zero
[9:35]	Zha Ewry:	Zero, if we can ghet this a little more wrtten up