User:Zero Linden/Office Hours/2008 Jan 17

From Second Life Wiki
Jump to navigation Jump to search

Transcript of Zero Linden's office hours:

[8:43] Harleen Gretzky: Zero Linden's office hour
[8:43] Wyn Galbraith: Hello.
[8:43] Harleen Gretzky: Hi Zero
[8:43] Zha Ewry: In a Linden office, full of gearheads
[8:43] Teravus Ousley: greetings
[8:43] Zha Ewry: Hey Zero!
[8:43] Rex Cronon: u r where a tech meeting is supposed to take place
[8:43] Thoys Pan: hello zero
[8:43] Rex Cronon: hi zero
[8:43] Saijanai Kuhn: goood morning teacher
[8:43] Dr Scofield: hi zero
[8:43] SignpostMarv Martin: hi Zero
[8:44] Wyn Galbraith: Greetings Zero ;)
[8:44] Wyn Galbraith: You could join us on Squirrel's zero :)
[8:44] katalina Zerbino: what is an avatar?
[8:44] Arawn Spitteler rezzed a chair, to save on lag, by sitting on a physical object...
[8:45] Neas Bade: morning zero
[8:45] luckdog Enzo: the flags is nice,but how about my country,the flag of china.?
[8:45] Zero Linden: Sorry all
[8:45] Zero Linden: Was in an "engineering directors" meeting that was the only time it could be held this week
[8:45] Zero Linden: oy vey
[8:45] luckdog Enzo: thnx very much
[8:45] SignpostMarv Martin: yw
[8:45] Kerry Giha: Ah the life of meetings
[8:46] Zero Linden: Welcome to my office hours all
[8:46] Saijanai Kuhn looks alert (whats up with that?)
[8:46] luckdog Enzo: lol~its nice
[8:46] TomHa Zymurgy: hav a sit ;)
[8:46] Teravus Ousley figures it might be good to jump right in because of ~15 minutes less? :D
[8:46] Zero Linden: So, I had a wonderful face-to-face meeting with Adam Z. on Tuesday
[8:46] Zha Ewry: So.. If we don't have a topic Zero, I was going to pester you, merrciless ly, about the long poll, and how we plan to use it and such in the next round of stuff
[8:47] Zero Linden: He was in town - we ate Indian food
[8:47] Neas Bade: awesome
[8:47] Teravus Ousley: :D
[8:47] Zha Ewry pricks up her ears at Adam's name
[8:47] Saijanai Kuhn: Adam Z...
[8:47] Zha Ewry wonders if it was the same nice little Indian down the street from Zero's office
[8:47] Thoys Pan: Adam azure islands?
[8:47] Dr Scofield: LL going to switch to OpenSim... ;-)
[8:47] Zero Linden: He told me a bit about the way OpenSim runs the scripts in a separate script server
[8:48] Zero Linden: which seems much like the "script interface" we've been talking about at the last few meetings
[8:48] Zha Ewry nods
[8:48] Zha Ewry: Closely related in any case
[8:49] Thoys Pan goes brb (souptime in holland)
[8:49] Rex Cronon: "script interface" talk? i guess reading the transcripts would clear that.
[8:49] Zero Linden: As currently defined, it is pretty heavily reliant on .Net structures, and works via RPC, but it might serve as a model for a
[8:49] Zero Linden: open protocol there
[8:50] Zha Ewry nods
[8:50] Zero Linden ducks after realizing he's been remiss about the transcripts
[8:50] Zha Ewry: one would want a set of web service style notifcations for that
[8:50] Neas Bade: there is a latency problem with using web services on that interface
[8:50] Saijanai Kuhn has a pointer to zero's transcripts on teh AW Groupies page. Its loking barren
[8:50] Zha Ewry: Not web services, but web services mediated notifcatoin
[8:50] Neas Bade: you really want something fast and low latency
[8:50] Zha Ewry: pub/sub low level
[8:50] Zha Ewry: but, controlled by caps
[8:50] Neas Bade: ok, right
[8:51] Saijanai Kuhn: TCP?
[8:51] Zha Ewry: Which is, in general, going to be an issue with the whle long term design for things like managing any UDP/low level stuff in the protocol
[8:51] Dr Scofield: saijanai, this is the HTTP-and-HTTP-only camp
[8:51] Zha Ewry: Its all sort of "Well, we just send stuff on the cirucuits"
[8:52] Zero Linden: Well, lessee
[8:52] Zha Ewry: and the client hopefully swallows it
[8:52] Zero Linden: we've often assumed that hte Agent Domain and the Region Doain have to be addresable - that is
[8:52] Zero Linden: they are not behind NAT firewalls that don't allow incoming connections
[8:52] Zha Ewry nods
[8:52] Zero Linden: in that case, one can do pub/sub pretty easily with HTTP and caps
[8:52] Zha Ewry: But.. at what performance cost
[8:52] Catherine Pfeffer: (as long as there's no dependancy towards .net everything is fine... lol)
[8:52] Zero Linden: you do it by registring a cap that is the service you want events posted to (the sub)
[8:53] Dr Scofield grins
[8:53] Zero Linden: and then the generator of the event just needs to invoke all the registered caps when the event fires (the pub)
[8:53] Saijanai Kuhn is a mac user.Everything is .net already... isnt' it? ;-)
[8:53] Zha Ewry nods at Zero
[8:53] Zero Linden: a pub/sub aggrigator could be used
[8:53] Zha Ewry: Things like that feel right
[8:53] Zero Linden: I think, one might even be able to put a small amount of convention in those calls so that
[8:53] Catherine Pfeffer: (sorry didn't mean to troll... not really)
[8:53] Zero Linden: (which is always the trick to making pub/sub efficient, if I understand it at all)
[8:53] Zha Ewry: where we use the high level caps stuff to control the low level
[8:53] Zha Ewry: event flow
[8:54] Zha Ewry: And aboslutely, Zero
[8:55] Zha Ewry: There's a huge amount of work been done on how to let the low level stuff do a little bit of control. to keep from having to do the expensive operations
[8:55] Zha Ewry: What you want to avoid, is letting that get out of hand, tho
[8:56] Zero Linden: So - this leads into the event queue and long poll I think
[8:56] Zha Ewry nods
[8:56] Zha Ewry: The event queue, and the circuits, even
[8:57] Zero Linden: both of these are mechanisms to get the semantics of REST across a path that can't support it directly
[8:57] Saijanai Kuhn: and how that actually works. Serveral lindens have attempted to pour the knowledge into the head but its leaking
[8:57] Zero Linden: We could imagine other, lower latency paths, that still support the REST calls, but don't use an HTTP/TCP connection per request
[8:57] Zero Linden: surely there must be already something out htere that does this
[8:58] Zero Linden: other than, say, Pipelined HTTP, which has a host of problems
[8:58] Dr Scofield doesn't dare to mumble "xmpp"
[8:59] Zero Linden: well- but xmpp doesn't - there isn't a strong request/response correllation is there? and the addressing is - well, Baroque at best
[8:59] Dr Scofield: IQ message
[8:59] Zero Linden: but I imagine one could cast HTTP semantics into XMPP, and that is perhaps not so bad
[8:59] Dr Scofield: IQ is supposed to be suc
[8:59] Dr Scofield: sync
[9:00] Zero Linden: hmmm google doesn't turn up a link to anything like IQ message
[9:00] Neas Bade: it would be nice to not get too far ahead of ourself here, and actually have some sort of REST semantics before trying to cast it into something else :)
[9:00] Zero Linden: hehe
[9:00] Dr Scofield: you send IQ request and get a result back
[9:00] Dr Scofield: XMPP RFC
[9:00] Dr Scofield: s
[9:01] Zero Linden: well, Neas, one thing I think Zha is trying to be sure of is that all the styles of communication we are going to need can be covered
[9:01] Zha Ewry: Also.. I think there is a case to be made for building it as simply as possible, and seeing where we need more than what we can get from base http.
[9:01] Dr Scofield: xmpp message is async
[9:01] Dr Scofield: xmpp iq sync
[9:01] Zero Linden: oh oh - I see on IQ, sorry, thought it was a separate protocol
[9:01] Zha Ewry: And, yes, tha is on my list
[9:01] Neas Bade: zha, absolutly agreed
[9:01] Zero Linden: So, event queue
[9:02] Zero Linden: details? It is simple: the viewer can't be contacted via HTTP
[9:02] Zha Ewry: Its really easy, to convince yourself you're going to *need* a fancy new widget, when base http will do. Now, we know there will be cases where we want better, but I am far from sure anyuone knows the line between those and the http is good enough cases
[9:02] Neas Bade: I'd rather have us prove http won't work first before trying to do something else, instead of guessing that it won't up front for most things
[9:02] luckdog Enzo: there are still so many questions for HTTP or ohers .we should have a better way to resoulve it.and the first is the net speed.
[9:02] Zero Linden: so instead, we remember a list of everythign we would have liked to send to the viewer via HTTP as a sequence of events
[9:02] Dr Scofield: neas, pipelined HTTP is reinventing the wheel, i think
[9:02] Zero Linden: and give the viewer a cap for fetching the events
[9:03] Zero Linden: when the viewer invokes the cap, it gets a sequence number and a list of events
[9:03] Dr Scofield: polling
[9:03] Zha Ewry: Right. So, you pull on that instead
[9:03] Zha Ewry: and when it has nothing to say, it says 'Try again later"
[9:03] Zero Linden: the simulator holds those sent events on the side, since it doesn't know if the viewr actually got 'em
[9:03] Zero Linden: the next time the viewer invokes the cap, it includes the sequence number of the last set of events it got
[9:04] Zero Linden: the simulator says - if the number matches the events on the side, ditch 'em, 'cause they got there
[9:04] Zero Linden: if it doesn't match, it sticks them back on the front of the queue, 'cause they got lost
[9:04] Zha Ewry nods
[9:04] Zha Ewry: That process, then, is sort of lockstep?
[9:04] Zero Linden: lather - rinse - repeat
[9:04] Zero Linden: and then it bumps the sequence number and sneds the event queu again (moving them all aside)
[9:05] Zha Ewry: since the client asks
[9:05] Zha Ewry: Assumign a request doesn't get broken, we should see it just rachet up?
[9:05] Dr Scofield: hmmm...still think a bi-directional protocol would be better suited for that
[9:05] Zero Linden: yes, the sequence number just goes up each time
[9:05] Zero Linden: Dr. - so I agree
[9:05] Teravus Ousley wonders if there's a spec somewhere with the events that get put into EventQueueGet
[9:06] Zero Linden: but the barrier to implementing XMPP was high-
[9:06] Zero Linden: Teravus
[9:06] Zero Linden: the events are just an LLSD version of messages right now
[9:06] Teravus Ousley: ah
[9:06] Zha Ewry: As if they had been put on the UDP cirtcuit, Zero?
[9:08] Zha Ewry: .
[9:08] Rex Cronon: zero crashed?
[9:08] Zha Ewry wonders if Zero has crashed
[9:09] Zero Linden: the events have the form { message: m, body: b}
[9:09] Zero Linden: where b is the LLSD encoded message body
[9:09] Zero Linden: in theory, we were going to implement { request: r, body: b}
[9:09] Zero Linden: or rather { request: r, body: b, id: i }
[9:10] Saijanai Kuhn: there's something missing from that equation, though, Zero. When I send theack with the last event ID obtained, I get the same event with a new ID
[9:10] Zero Linden: which would be a REST semantics request expecting a respond during the poll
[9:10] Zero Linden: but that isn't spec'd or implemented
[9:10] Rex Cronon: shouldn't id be first?
[9:10] Saijanai Kuhn: do I need to close the connection and reopen it to stop getting the same event?
[9:10] Saijanai Kuhn: its ack/ID in LLSD format, Rex
[9:10] Zero Linden: Sai - I don't think so
[9:11] Saijanai Kuhn: OK, so I'm doing something wrong (as usual)
[9:11] Thoys Pan is back
[9:12] Zha Ewry wonders exactly how close to REST polling gets.. The event queue is full of events that don't have resources. I'm not suggesting that there is a good REST semantic, actually, just musing that it feels funny
[9:12] Saijanai Kuhn: you send the llsd-xml of ack and the id and it sends you back an llsd array of events
[9:12] Zero Linden: but the event set is sent as { id: n, events: [ ... ] }
[9:12] Zero Linden: the ack must be (body of the poll) { ack: n }
[9:12] Saijanai Kuhn: right
[9:12] Zero Linden: Zha - no, at present, the event queu does messages
[9:12] Teravus Ousley listens
[9:12] Rex Cronon: until a prototype is made, u can't be sure how bad or good this is
[9:13] Dr Scofield needs to go earlier today: RL avalanche training
[9:13] Dr Scofield: cu all
[9:13] Saijanai Kuhn: I send and receive stuff in the right format, but don't get what I expect. I'm either sending the wrong ID or not understanding
[9:13] Zero Linden: so, messages can be cast as POSTs to a /messages/<messagename>
[9:13] Rex Cronon: bye dr
[9:13] Zero Linden: where the requestor doesn't expect anything other than a null body resposne
[9:13] Zero Linden: messages are a degenerate REST request :-)
[9:13] Neas Bade: you'd still get a proper return code though, right?
[9:14] Neas Bade: 20x or 40x depending
[9:14] Zero Linden: No - the message system didn't have any semantics of return code at all
[9:14] Saijanai Kuhn: <llsd><undef /> ?
[9:14] Neas Bade: so you have no idea if the message was accepted?
[9:14] Zero Linden: some messages have the semantic of the sender knows if it was received or no
[9:14] Zero Linden: and the event queue supports that by, when it gets the ack, it tells all the senders that cared that the message was received
[9:14] Zero Linden: but that is all
[9:14] Saijanai Kuhn: this seemes an abuse of CAP
[9:15] Zha Ewry: Not really.
[9:15] Zha Ewry: The cap, is just saying "here is how you polll"
[9:15] Zero Linden: so it is like a REST request where the sender checks if it got any HTTP return code or not -- but doens't even look a thte code
[9:15] Thoys Pan: btw, for intrested people i opensourced the ircchantoslgroupbot ->
[9:15] Saijanai Kuhn: but if you send a message and don't know if it arrived...
[9:15] Zero Linden: Sai - no more like <llsd />
[9:15] Zha Ewry: Well, the poll, is outside the caps abstraction
[9:15] Zero Linden: !
[9:16] Zha Ewry: Thus, the comment, about resources
[9:16] Zha Ewry: If you wanted to do it all pure rest, you'd die making all those messages into REST resrouces
[9:16] Saijanai Kuhn: OK, I cansorta see that
[9:17] Zero Linden: well - internlly, in the viewer, we cast all of those events into /external/message/<message-name>
[9:17] Zha Ewry brushes signposts flag out of her face
[9:17] Zero Linden: (or some such), look up those resources and invoke POST on 'em with the body
[9:17] Saijanai Kuhn: so you can POST messages to EventQueueGet? or is that only with other caps?
[9:18] Zero Linden: there is a wild cared handler at /external/message/* that picks up any message that isn't implemented as an HTTPResponder
[9:18] Saijanai Kuhn: ah, ok. So thats the format for all CAPs other then EventQueueGet?
[9:18] Zero Linden: and turns around and looks up th emessage in the old message template system, finds the registered C++ function pointer, and calls it
[9:18] Zero Linden: doing the translation from LLSD to the old binary version on the fly
[9:19] Zero Linden would kill for a shared white board right now
[9:19] Zero Linden or even a non-shared one
[9:19] Rex Cronon: doesn't windows have that?
[9:20] Rex Cronon: oh, u mean in here
[9:20] Saijanai Kuhn: recalls one fo the IM services used ot have one but hasn't seenit lately
[9:20] Teravus Ousley: yeh, best way in here at the moment is to set up a vid stream
[9:20] Zero Linden: so - here's the path...........
[9:21] Zero Linden: Simulator wants to send message FooBar with {x:12, name: Baz} to the viewer
[9:21] Thoys Pan: zero, can notecards be created using NewFileAgentInventory?
[9:21] Arawn Spitteler suggests Zarf might market through Etch-A-Sketch.
[9:21] Zero Linden: it could send it via UDP, binary encoded, or it could send it via the event queue
[9:22] Zero Linden: if it does the later
[9:22] Zero Linden: then { message: FooBar, body: { x: 12, name: Baz } } gets added to the EventQueue for the agent
[9:22] Zero Linden: Now -
[9:22] Zero Linden: the simulator waits
[9:23] Zero Linden: at some point, the viewer invokes the cap for it's event queue that it was given long ago, when it connected to this simualtor
[9:23] Zero Linden: the viewer does a POST to the cap w/ { ack: undef }
[9:23] Thoys Pan: Zero, the cap NewFileAgentInventory is used for Image uploading and other assets, but would it work for notecards?
[9:24] Zero Linden: the simulator responds with a response of { id: 912, events: [ { message: FooBar, body: { x: 12, name: Baz } } ] }
[9:24] Zha Ewry: Is the undef everytime, or just the first?
[9:24] Zero Linden: The viewer gets that body, pulls it apart and makes the requests that the simulator would have made if it could have
[9:25] Zero Linden: (Zha - just the first time)
[9:25] Zero Linden: so the viewer does an internal POST to /external/messages/FooBar with a body of { x: 12, name: Baz }
[9:26] Zero Linden: that either lands at an HTTPResponder that handles the mssage, or routes to the old message handler with LLSD to binary translation
[9:26] Zero Linden: the next time the viewer polls, it invokes the smae cap with { ack: 912 }
[9:26] Zha Ewry suspects that Undef is what's killing Sai
[9:27] Zero Linden: so - there are two levels of REST going on here - the outer, done with caps, to enable getting the inner requests, tothe viewer
[9:27] Saijanai Kuhn: but the internal is all within the viewer. Thats how the SL viewer handles things. A new viewer, that doesn't know about the UDP, wouldn't be doing thigns that way
[9:27] Lena Franciosa: Hi everyone
[9:28] Wyn Galbraith: Hello Lena.
[9:28] Rex Cronon: jo
[9:28] Rex Cronon: i mean hi
[9:28] Zero Linden: Sai - sure it would - it just wouldn't bother implementing the connector back to the old call-back style of message handler, since it wouldn't have that
[9:28] Neas Bade: Zero: is there any mock up code to do any of this. I think I'm getting lost in the micro explanations and need some bits to make it all make sense to me
[9:28] Saijanai Kuhn: well it wouldn't be POSTing anything
[9:28] Neas Bade: like some python mock up or something
[9:29] Zero Linden: The new viewer would still have to contend with the fact that the simulator can't make an HTTP connection directly to the viewer
[9:29] Zha Ewry: I'd actually like a set of annotated flows, or such, and a simple pyhton that implements it
[9:29] Zero Linden: the viewer is behind NAT firewals
[9:29] Zero Linden: Neas - alas, this is all in C++ - though it is pretty simply C++
[9:29] Zha Ewry: Right. We have to asume the poor viewer is hiding behing a firewall, a NAT router, and a few buggy gateways
[9:29] Saijanai Kuhn: I send { ack: undef } to EventQueueGet
[9:30] Neas Bade: well, C++ is better than nothing
[9:30] Zero Linden: righ - and it doesn't have a stable IP address either!
[9:30] Saijanai Kuhn: I get back { message: FooBar, body: { x: 12, name: Baz } }
[9:31] Wyn Galbraith has to run to work y'll.
[9:31] Saijanai Kuhn: rather I get back the { id: 912, events: [ { message: FooBar, body: { x: 12, name: Baz } } ] }
[9:31] Wyn Galbraith: B-bye!!! :D
[9:31] Saijanai Kuhn: I send {ack:912} and get back
[9:31] Saijanai Kuhn: { id: 913, events: [ { message: FooBar, body: { x: 12, name: Baz } } ] }
[9:32] Linx Newt: aoummmmmm
[9:32] Linx Newt: hey hey :)
[9:32] Rex Cronon: bye wyn
[9:32] Lena Franciosa: hey linx
[9:32] Saijanai Kuhn: ack 913 gets back { id: 914, events: [ { message: FooBar, body: { x: 12, name: Baz } } ] }
[9:32] Thoys Pan: hello
[9:32] Linx Newt: c est une secte ?
[9:32] Saijanai Kuhn: so obviously I'm missing something
[9:32] Linx Newt: lol
[9:33] Zero Linden: Sai - here's the code
[9:33] Lena Franciosa: oui on va t'enroller
[9:33] Lena Franciosa: mdr
[9:33] Zero Linden: last
[9:33] Catherine Pfeffer: chut lynx
[9:33] Zero Linden shout: last
[9:34] Zero Linden: there
[9:34] Linx Newt: c quand le suicide collectif ? :)
[9:34] Lena Franciosa: ssh
[9:34] Catherine Pfeffer: SILENCE
[9:34] Zero Linden: so you can see that there is always an outter map of {id:, evnets:}
[9:34] Zero Linden: actually - I need to run too
[9:35] Zero Linden: sigh
[9:35] Zero Linden: thanks all for coming
[9:35] Rex Cronon: bye zero
[9:35] Zha Ewry: Zero, if we can ghet this a little more wrtten up