User:Which Linden/Office Hours/2007 Nov 15

Transcript of Which Linden's office hours:

[11:00]	Tillie Ariantho:	there we are. :)
[11:01]	Which Linden:	Hey folks!
[11:01]	Masaw Umaga:	hey!
[11:01]	Which Linden:	I invited Donovan to come by since our topic today (suggested by Zha) is to talk about Pantheon and what it's for.
[11:01]	Morgaine Dinova:	'Morning :-)
[11:02]	Which Linden:	Good morning, Morgaine.
[11:02]	Morgaine Dinova:	I don't see Zha
[11:02]	Tillie Ariantho:	She's fetching a diagram or something.
[11:03]	Donovan Linden:	woah, we're going to talk about pantheon? :)
[11:03]	Which Linden:	Yeah, weren't you there?
[11:03]	Saijanai Kuhn:	what ist he topic?
[11:03]	Which Linden:	The cool tool Pantheon
[11:03]	Donovan Linden:	i should have drawn some diagrams or something
[11:04]	Which Linden:	Sorry, yeah, I kinda forgot what the topic was until just like 30 minutes ago
[11:04]	Donovan Linden:	hehe
[11:04]	Tillie Ariantho:	:-p
[11:04]	Which Linden:	Actually now I have to doube check
[11:04]	Saijanai Kuhn:	is OK, we moved our meeting here so we could get feedback from you if you had time
[11:04]	Which Linden:	Whats' your meeting about?
[11:04]	Saijanai Kuhn:	but your meeting comes frist
[11:05]	Which Linden:	Yeah, it was definitely Pantheon
[11:05]	Tillie Ariantho:	Any of the Lindens here have an idea about why the "missing gesture from database" can be something like the default standing animation? ,)
[11:05]	Tillie Ariantho:	Happens to me quite often that after a login it isnt found and I am 'walking' with the 'flying' anim...
[11:06]	Donovan Linden:	ok well let's get started I guess
[11:06]	Tillie Ariantho:	.)
[11:06]	Which Linden:	You know, brokenness.
[11:06]	Which Linden:	Yeah, Donovan, what *is* Pantheon??
[11:06]	Saijanai Kuhn:	This is the first "formal" meeting of the client VAG. https://wiki.secondlife.com/wiki/Multi-Process_Client_VAG_--_draft
[11:06]	Donovan Linden:	saijanai: I saw that on the mailing list, it looks very nice
[11:06]	Morgaine Dinova:	Not this, hehe. After Which's lol
[11:06]	Donovan Linden:	ok, so how many people here have heard of comet, in the same context as ajax
[11:07]	Tillie Ariantho:	not me.
[11:07]	Masaw Umaga:	no
[11:07]	Emma Nowhere:	me
[11:07]	Wyn Galbraith wants to sit by the bush
[11:07]	Tillie Ariantho:	I only know a bit ajax. .)
[11:07]	Wut Burt:	aha
[11:07]	Tao Takashi:	Hi
[11:07]	Wut Burt:	is that not unlike what google chat uses?
[11:07]	Donovan Linden:	ok. comet is like ajax, except instead of polling the server for changes, the client always keeps a connection open to the server
[11:08]	Donovan Linden:	it is probably what google chat uses, I haven't looked
[11:08]	Which Linden:	I think it is
[11:08]	Wut Burt:	blocked socket, basically
[11:08]	Tillie Ariantho:	Is that good? Keeping open connections?
[11:08]	Donovan Linden:	the server keeps these sockets open, but doesn't respond to the http request in them, until there is an event on the server
[11:08]	Morgaine Dinova:	If it's just event messaging, it's good.
[11:08]	Tillie Ariantho:	We sometimes have problems with too many open connections between web and app server... so it sounds not so good...
[11:08]	Emma Nowhere:	depends, some unix kernals have to be recompiled to handle it
[11:08]	Donovan Linden:	when there is an event on the server, the server sends it over the socket
[11:09]	Squirrel Wood:	/ao off
[11:09]	Donovan Linden:	our networking library, eventlet, uses non-blocking io and coroutines to scale to 10,000 connections
[11:09]	Tillie Ariantho:	Sounds a bit like MQSeries. :)
[11:09]	Seeping Blister:	sure is.
[11:09]	Donovan Linden:	any comet server is likely to have to use non-blocking sockets on the server side
[11:10]	Tillie Ariantho:	That network lib is used on a sim server?
[11:10]	Donovan Linden:	but there are http servers that can handle the load for many languages
[11:10]	Which Linden:	10000 threads = THE PAIN
[11:10]	Morgaine Dinova:	Donovan: I like the sound of that. Everything I do is event-driven.
[11:10]	Donovan Linden:	tillie it is
[11:10]	Wut Burt:	Which: :)
[11:10]	Donovan Linden:	ok, so that's what comet is. mulib, my "rest services library", has two comet implementations
[11:10]	Morgaine Dinova:	Not 10,000 threads, hehe, 10,000 coroutines :-)
[11:10]	Tillie Ariantho:	Hope it is not for agent server and such... 10.000 fixed connections there sounds bad. .)
[11:11]	Donovan Linden:	tillie: the agent domain hosts should be able to handle thousands of simultaneous logged in agents without a problem
[11:11]	Donovan Linden:	once we actually write them
[11:11]	Saijanai Kuhn:	login will be an interesting issue though
[11:12]	Tillie Ariantho:	Yes, but if the connections stay open 10.000 will be able to connect and the others... not. That I meant.
[11:12]	Donovan Linden:	anyway, back to mulib. mulib.eventrouter is a comet server I did in about december of 06
[11:12]	Which Linden:	We won't have any central servers any more, so we'll be able to add more servers, Tillie.
[11:12]	Donovan Linden:	tillie: I can't imagine we would have 10,000 simultaneous users on an agent host
[11:12]	Wyn Galbraith:	In the old days when you had to walk 10 miles in 10 feet of snow to get to the computer.
[11:12]	Tillie Ariantho:	Okay. :) I am quiet. .)
[11:12]	otakup0pe Neumann:	the central db is dead long live the central db.
[11:13]	Which Linden:	so, eventrouter, how does that use comet?
[11:13]	Donovan Linden:	eventrouter is an interesting experiment because it is completely asynchronous
[11:13]	Wyn Galbraith:	Wouldn't that be decentral?
[11:13]	Donovan Linden:	eventrouter exposes python methods that can be called from javascript (ajax)
[11:14]	Donovan Linden:	but it also exposes javascript functions that can be called from python
[11:14]	Donovan Linden:	neither of these calls have a return value
[11:14]	Morgaine Dinova:	Donovan: Are you documenting the on-the-wire protocol too?
[11:14]	Tillie Ariantho:	.D
[11:14]	Donovan Linden:	the wire protocol is json
[11:14]	Morgaine Dinova:	Oh, I thought Zero didn't like JSON
[11:15]	Donovan Linden:	one other interesting thing about eventrouter is that it gives you a unique server-side python object, the Target, for each browser which is connected
[11:15]	Emma Nowhere:	any chance you could do like Yahoo does with it's webservices and let you specify the return type as xml or json?
[11:15]	Donovan Linden:	so it is easy to create pages which look completely different to different users, which is slightly more difficult with pantheon
[11:16]	Donovan Linden:	emma: yes, the head of mulib does content-type negotitation, and internally we can switch between json and llsd
[11:16]	Morgaine Dinova:	That's nice
[11:16]	Which Linden:	The javascript client doesn't have an llsd parser though, does it?
[11:16]	Emma Nowhere:	json is awesome for ajax work, but if you're building server to server stuff, or even invoking from LSL, it's not as good
[11:16]	Donovan Linden:	it does negotiation based on the Accept and Content-Type headers
[11:16]	Donovan Linden:	which: right, which is why the server supports generating json
[11:16]	Which Linden:	I see, so a non-javascript client could speak only llsd with the server
[11:17]	Donovan Linden:	yup, or you can plug in your own parsers/generators for whatever format you want, based on mime type
[11:17]	Donovan Linden:	ok, so eventrouter is a very simple asynchronous message bus
[11:17]	Donovan Linden:	pantheon is a little different of a beast
[11:17]	Donovan Linden:	pantheon is also a comet server, but it is also a pure REST server
[11:18]	Donovan Linden:	instead of the server having apis to send events to the clients, the event is generated by performing a PUT or DELETE
[11:19]	Donovan Linden:	clients indicate which subpaths they are "observing" and the server takes care of handing all the events to those subpaths to that client
[11:19]	Donovan Linden:	so in this case, clients can have different views of the same page, but only by maintaining client state
[11:19]	Donovan Linden:	for example, this tree item is revealed, or that "pane" is open
[11:19]	Emma Nowhere:	so, when i connect via rest, i can get all the events since my last connect?
[11:19]	Donovan Linden:	or tab or whatever you want to imagine for us
[11:20]	Donovan Linden:	s/us/ui/
[11:20]	Donovan Linden:	yes, when the pantheon javascript or lsl client connects, it has no etags to give to the server
[11:20]	Donovan Linden:	every time the client sees a new event, it gets the etag for the resource the event occurred on
[11:21]	Donovan Linden:	when the clients ask for new events, they simply say "here are the paths I am interested in, and here are the latest etags I have for them:
[11:21]	Donovan Linden:	"
[11:21]	Donovan Linden:	what this means is that it is very easy to bridge between systems: lsl can get events in one representation,
[11:21]	Donovan Linden:	json can generate and watch events using the json representation
[11:22]	Morgaine Dinova:	So the server is building a forest, and the etags point into somewhere in the trees. How does past history get culled from the forest?
[11:22]	Donovan Linden:	and scripts can use llsd, or insert your favorite format here
[11:22]	Donovan Linden:	for the culling question, let me bring up the code
[11:22]	Donovan Linden:	the etag implementation I just described has been happening in the collaborative-editor branch and hasn't merged into release yet
[11:23]	Which Linden:	So the culling has to do with event eliding right?
[11:23]	otakup0pe Neumann:	i just want to add that the collaborative editor is pretty hot.
[11:23]	Which Linden:	If you're looking at /counter and somebody changed it from 1 to 2 to 3 to 4
[11:23]	Morgaine Dinova:	Yeah, getting rid of old stuff not accessible through etags any longer
[11:23]	Which Linden:	You only need to know that it's currenltly at 4
[11:23]	Donovan Linden:	http://svn.secondlife.com/trac/mulib/browser/branches/collaborative-editor/mulib/pantheon.py
[11:24]	Donovan Linden:	the todo in the docstring needs to be changed, that code is using etags
[11:24]	Donovan Linden:	each path on the server keeps it's current etag, and also every etag that it ever was
[11:25]	Donovan Linden:	so when an old client comes in with an old event, no matter how old, the client gets the "latest" event for that path
[11:25]	Donovan Linden:	because we're using put, and we're assuming idempotency (each put replaces the last completely)
[11:25]	Donovan Linden:	oh, I guess I should mention the difference between resource events and container events
[11:26]	Tillie Ariantho:	Oh ah, I see Donovan is a non-documenter, I dont see comments in there. .)
[11:26]	Donovan Linden:	clients can choose to subscribe to a specific thing, such as "first_name"
[11:26]	Donovan Linden:	well, pantheon has just been a weekend creative project for me, so I tend to stay heads-down until the design is finished
[11:26]	Donovan Linden:	the page I pasted is horribly messy with commented out print statements
[11:27]	Donovan Linden:	ok, back to kinds of events
[11:27]	Donovan Linden:	if a client subscribes to first_name, and someone does a PUT /first_name, the client will get the event
[11:27]	Donovan Linden:	but there are also container events, because sometimes you don't know the names of all the possible resources you are interested in
[11:28]	Donovan Linden:	so a client might subscribe to people/*
[11:28]	Donovan Linden:	and if someone does a PUT to /people/fred or /people/bob, the client will get the event
[11:28]	Which Linden:	How does the client know it's a container event?
[11:28]	Which Linden:	Does it need to know?
[11:29]	Donovan Linden:	which I assume the callback it registers will know
[11:29]	Donovan Linden:	a callback registered for people/* will not currently be triggered by a PUT /people
[11:29]	Which Linden:	Oh, right, because it has to explictly register for container/*
[11:29]	Donovan Linden:	so client can register both callbacks for people/* and people if they want
[11:29]	Morgaine Dinova:	What is the semantic of GET /people/* ? Get all current subscribers, or get a container tag that will explode to all subscribers at time of use?
[11:30]	Donovan Linden:	so pantheon is great because clients get to decide which events they see
[11:30]	Donovan Linden:	morgaine: GET /people/* would 404
[11:30]	Donovan Linden:	GET /people/ would get you a json map
[11:30]	Morgaine Dinova:	kk
[11:30]	Donovan Linden:	with fred and bob keys
[11:30]	Donovan Linden:	with whatever body was associated with the appropriate put
[11:31]	Donovan Linden:	ok, I think I'm about done blabbin
[11:31]	Morgaine Dinova:	So the map is for time of query, not for time of subsequent PUT
[11:31]	Donovan Linden:	morgaine: right, it's pure rest, you get a representation from that time
[11:31]	Donovan Linden:	but then you have the etag, and can use that to request a comet event when that container changes
[11:31]	Morgaine Dinova:	Donovan: it's not blabbin ... you're wonderfully clear :-)))
[11:32]	Donovan Linden:	good :) thanks
[11:32]	Which Linden:	Yeah man, nice exposition
[11:32]	Which Linden:	So how does the client register for events?
[11:32]	Donovan Linden:	http://svn.secondlife.com/trac/mulib/browser/branches/collaborative-editor/mulib/pantheon.js
[11:32]	Which Linden:	(i.e. what's the REST api for that)
[11:32]	Donovan Linden:	the javascript api is observe(path, callback)
[11:33]	Donovan Linden:	the wire encoding can be negotiated as I mentioned
[11:33]	Donovan Linden:	the format of the body is a map with an observe key
[11:33]	Donovan Linden:	wait let me double check
[11:34]	Emma Nowhere:	Donovan, is that JS designed for people to use on their own pages?
[11:34]	Morgaine Dinova:	Well I've understood only part of this, but going to give it a long hard think. The concept sounds great.
[11:34]	Donovan Linden:	a map with client-id and observe keys
[11:34]	Saijanai Kuhn:	so this is kind of a like a baby form of a Tobject (Saijanai is a bit slow)?
[11:34]	Donovan Linden:	emma: sure, if there is a pantheon server running somewhere
[11:34]	Which Linden:	This is? sendContent': serializeJSON({ 96 'client-id': _client_id, 97 'observe': watching})}
[11:34]	Donovan Linden:	saijanai: I don't know what Tobject is
[11:34]	Which Linden:	I think that's teh line that sends it
[11:34]	Which Linden:	(the 96 and 97 are newlines)
[11:34]	Donovan Linden:	the observe key is a list of maps with a path key and an optional etag key
[11:35]	Saijanai Kuhn:	Croquet's base object. All things in a croquet world are TObjects, and broadcast themselves to other Croquet clients
[11:35]	Donovan Linden:	the response is a map with event (PUT or DELETE), path, and body keys
[11:35]	Donovan Linden:	although you may choose to structure your code so it ignores the body and does a GET on the resource, to play well with cachine
[11:35]	Donovan Linden:	caching
[11:36]	Donovan Linden:	saijanai: oh, interesting
[11:36]	Donovan Linden:	so yeah, it's a way of getting event notifications between machines, but it's also REST, which I think is fabulous :)
[11:36]	Donovan Linden:	I'm very happy with the way pantheon turned out, although I still think of it as a research project
[11:37]	Which Linden:	Yeah, it's very cool. Very "pure" in a sense
[11:37]	Donovan Linden:	one of the best things about it is that the clients and servers can speak different languages, and the pantheon server itself doesn't have to be any more complicated than a Panthon() as the root Resource
[11:37]	Which Linden:	I had a question from before: what happens if the client sends down an etag that isn't in the server's list?
[11:37]	Donovan Linden:	as long as you trust everyone equally to make changes
[11:37]	Donovan Linden:	which: I dunno, I think it just gives it the latest
[11:38]	Donovan Linden:	which: that might happen if a client was connected across a server restart
[11:38]	Which Linden:	Right, or if the client is behaving badl
[11:38]	Which Linden:	OR something
[11:38]	Donovan Linden:	it would be interesting to implement retry logic in the errback
[11:38]	Donovan Linden:	pantheon has the advantage over eventrouter that it can actually survive system restarts
[11:38]	Which Linden:	EVentrouter can't?
[11:39]	Donovan Linden:	well, the applications that are developed on it often can't
[11:39]	Morgaine Dinova:	Can the client subscribe to a notification to be received on container growth?
[11:39]	Donovan Linden:	because they assume they have some server-side state for each client
[11:39]	Which Linden:	Oh, right, because each client is assigned a target-id that it never re-requests
[11:39]	Donovan Linden:	morgaine: yes, that is what the container events (people/*) are for
[11:39]	Donovan Linden:	they trigger on growth and shrinkage (PUT and DELETE)
[11:40]	Donovan Linden:	as far as granting different clients different access rights to pantheon...
[11:40]	Which Linden:	So if you have a list [] and it gets a new list inside: [ [ ] ] you get the entire inner list in the event?
[11:40]	Donovan Linden:	which: I htink that is correct
[11:40]	Which Linden:	Might be bad if you PUT a huge list, though I guess that would be a fool thing to do
[11:41]	Donovan Linden:	I haven't quite fleshed out security yet, but I'm pretty sure it's as simple as having a non-generic-pantheon server side service that gives out capabilities
[11:41]	Saijanai Kuhn:	so you can be sure soneone will do it
[11:41]	Donovan Linden:	the "public" pantheon can then be wrapped to not accept PUT and DELETE
[11:41]	Donovan Linden:	and the only PUT and DELETEs that will trigger events will happen through those capabilities
[11:41]	Donovan Linden:	but that part isn't developed yet
[11:42]	Which Linden:	So can you use POST to make changes?
[11:42]	Which Linden:	To the REST structure?
[11:42]	Donovan Linden:	sure, you can implement whatever server-side python you want with standard mulib
[11:42]	Morgaine Dinova:	Good, I just want to be sure that the client can receive the membership notifications automatically ... having to query for subscription list before every container put wouldn't be great.
[11:42]	Donovan Linden:	if you change the pantheon's rest store, you have to call a method to notify clients
[11:43]	Donovan Linden:	morgaine: play with it to see what's going over the wire
[11:43]	Morgaine Dinova:	kk
[11:43]	Donovan Linden:	for exmaple, run mud and try the hello_pantheon example while sniffing the wire
[11:43]	Donovan Linden:	or, firebug is also great for seeing what happens
[11:43]	Donovan Linden:	actually I think firebug is pretty much a necessity :)
[11:44]	Which Linden:	Ha ha, yeah it rules
[11:44]	Which Linden:	Though we did just write some code that only worked if you had Firebug installed :-/
[11:44]	Donovan Linden:	oh, I suppose I should mention the very hacky lsl client
[11:44]	Morgaine Dinova:	lol Which :P
[11:44]	Donovan Linden:	http://svn.secondlife.com/trac/mulib/browser/branches/collaborative-editor/mulib/cube.lsl
[11:44]	Donovan Linden:	here is an lsl client
[11:45]	Donovan Linden:	it uses a very weird format involving newlines and pipes and base 64 encoding
[11:45]	Donovan Linden:	I would love to know if there is a more standard format I could use
[11:45]	Donovan Linden:	because I have to parse/generate this format in mulib
[11:45]	Donovan Linden:	for the content type text/plain! because that's what lsl sends
[11:46]	Donovan Linden:	I suppose I should convert the input format to the same format.. right now it's json for the input, weird format for the output
[11:46]	Which Linden:	It would be cool if we could send llsd+xml natively from within LSL
[11:46]	Donovan Linden:	sure would, but lsl isn't nearly rich enough, type wise
[11:47]	Which Linden:	Right, we only have lists
[11:47]	Wut Burt:	ish :)
[11:47]	Donovan Linden:	homogenous lists?
[11:47]	Donovan Linden:	or are they heterogenous?
[11:48]	Which Linden:	No one seems to know....
[11:48]	Morgaine Dinova:	lol
[11:48]	Donovan Linden:	oh well :)
[11:48]	Which Linden:	I think I heard homogeneous, but I've never uesd one myself (lame, I know)
[11:48]	Morgaine Dinova:	Sai/Wut, looks like we'll need a Pantheon front end too :-)
[11:49]	Donovan Linden:	so for anyone who hasn't seen my blog post about the cube.lsl example:
[11:49]	Donovan Linden:	http://ulaluma.com/pyx/archives/2007/09/rest_to_lsl_via_python_and_comet.html#comments
[11:49]	Which Linden:	So do you envision Pantheon being used for any infrastructure in Second Life?
[11:49]	Donovan Linden:	(warning: plays a movie)
[11:49]	Which Linden:	The movie is quite cool, BTW
[11:49]	Donovan Linden:	all right I'm done blabbin again
[11:50]	Morgaine Dinova:	I can't get flash-encap'd movies on this box, 64-bit browser
[11:50]	Which Linden:	Cool
[11:50]	Donovan Linden:	which: I think we could use it for infrastructure
[11:50]	Donovan Linden:	but currently I have no plans for that
[11:51]	Donovan Linden:	I have been thinking of a new event queue design which works more like pantheon
[11:51]	Morgaine Dinova:	I'll ssh URL over to another box
[11:51]	Donovan Linden:	and allows selecting events from more than the simulator
[11:51]	Which Linden:	Agent presence was listed as a possibility -- i.e. you subscribe to each of your friends. Seems like it would lead to meltdown though
[11:51]	Donovan Linden:	no, I don't think it would lead to meltdown
[11:51]	Donovan Linden:	I think that is perfectly reasonable
[11:51]	Which Linden:	Oh, I guess each viewer/sim would have only one connection per agent to the presence servers
[11:52]	Donovan Linden:	I think in the long run, the viewer will have a comet connection open to both it's agent host and it's simulator
[11:52]	Donovan Linden:	the agent host and the simulator would then be able to select from another set of backend hosts to fulfill the client's request for event notifications
[11:53]	Donovan Linden:	if we end up going with the presence-publish-subscribe design I did a while back,
[11:53]	Donovan Linden:	we will probably have a hybrid architecture, where GETs against a server cause lazy subscriptions to the actual back-end event source
[11:53]	Morgaine Dinova:	Well you don't want to expand distributions lists by querying group membership at messaging time, that scales very badly. Really need the membership-change event to create relatively static lightweight exploders. That's one problem with current IM.
[11:53]	Donovan Linden:	and if there aren't GETs for that resource after a timeout, the subscription is dropped
[11:53]	Which Linden:	Morgaine: True
[11:54]	Donovan Linden:	morgaine: yes, there has been an internal design written up for chat that includes flooding all the agent hosts
[11:54]	Morgaine Dinova:	The exploders can live on the clients, for simple IM. But harder when messaging is server-side.
[11:55]	Which Linden:	So, any more questions?
[11:55]	Donovan Linden:	well, seems like everybody is in awe :)
[11:55]	Which Linden:	Ha ha, yeah. Or jonesing for the Client VAG.
[11:55]	Morgaine Dinova:	I'm in total awe. Now comes the hard work of figuring it out. :-)
[11:56]	Emma Nowhere:	nope, reading all the code you posted url's to
[11:56]	Donovan Linden:	so I'll lurk for the vag meeting but I'll be afk
[11:56]	Morgaine Dinova:	Cool
[11:56]	Which Linden:	OK, let's wrap up the office hours and do the VAG.