User:Which Linden/Office Hours/2008 Jan 10

Transcript of Which Linden's office hours:

[11:00]	Kri Ayakashi:	hey Which
[11:00]	Which Linden:	morning kri
[11:00]	Kri Ayakashi:	what's todays topic?
[11:01]	Kri Ayakashi:	hehehe nice spot  :)
[11:01]	Which Linden:	:-) I blend in here
[11:01]	Which Linden:	I was thinking that I'd talk about versioning of oplogs, but I'm willing to entertain alternate suggestions
[11:01]	Kri Ayakashi:	oplogs?
[11:02]	Gellan Glenelg:	hi
[11:02]	Kri Ayakashi:	hey
[11:02]	Which Linden:	After all, the point of these office hours is for your benefit, not for me to ramble on about my problems.
[11:02]	Which Linden:	Hey Gellan
[11:02]	Kri Ayakashi:	yeah but some have pre determined topics  :)
[11:02]	Which Linden:	The oplogs are a part of certified http.
[11:02]	thekarthic Balbozar:	'mornin
[11:02]	Kri Ayakashi:	hey
[11:03]	Which Linden:	Lemme see if I can rustle up a few more attendees
[11:03]	Kri Ayakashi:	I was wondering how is the eventlet fitting in the regions structure
[11:03]	thekarthic Balbozar:	o wow your a pant
[11:03]	thekarthic Balbozar:	plant
[11:04]	Which Linden:	Kri: Eventlet is essentially a networking libray
[11:04]	Kri Ayakashi:	yes I know
[11:05]	Kri Ayakashi:	I'm familiar with stackless and twisted and recently I found out about eventlet/greenlet
[11:05]	Which Linden:	The way we're currently using it is as sort of a "glue" between simulator hosts and various web services.
[11:05]	Which Linden:	I.e. the simulator wants to use a particular service (e.g. group chat), so it contacts a local eventlet-based process that knows what host to hit.
[11:06]	Which Linden:	These local processes are the "backbones"
[11:06]	Which Linden:	But backbone is just the name for the pluggable architecture we have for making new eventlet/mulib based web services
[11:07]	Which Linden:	So, pretty much every host has a backbone on it, but the role of the backbone changes depending on the role of the host
[11:07]	Kri Ayakashi:	tell me ... are there any works or plans to make xmlrpc more reliable?
[11:07]	Tao Takashi:	Hi there
[11:07]	Tao Takashi:	happy new year  :)
[11:08]	Kri Ayakashi:	hey guys
[11:08]	Which Linden:	Kri: there have been some stopgap measures to prevent xmlrpc from dying completely, but the inbound-http project is considered to be more "correct"
[11:08]	Which Linden:	Hey Tao
[11:09]	Kri Ayakashi:	hmm there is going to be inboud http?
[11:09]	Fumon Kubo:	Can somone point me at the mentioned HTTP project?
[11:09]	Tao Takashi:	yes, give us inbound http and web media please and external textures  :)
[11:09]	Which Linden:	Hopefully, we will.
[11:09]	Kri Ayakashi:	meaning the scripts will be able to serve as HTTP servers?
[11:09]	Tao Takashi:	I meant tomorrow  ;-)
[11:10]	Which Linden:	But Donovan has been making the point that technically, client side http is sufficient
[11:10]	Which Linden:	I mean, think about it this way: if you have a server that accepts messages on one url, say http://myhost.com/incoming
[11:10]	Tao Takashi:	what does client side mean?
[11:11]	Tao Takashi:	in which context?
[11:11]	Which Linden:	Then puts them to be read in another url, http://myhost.com/outgoing, then the lsl client can read the outgoing url
[11:11]	Kri Ayakashi:	but that still means polling
[11:11]	Which Linden:	And you can treat the incoming usl as if it were the http-inbound version of the lsl script
[11:11]	Which Linden:	Tao: we're talking about lslhttprequest as the client
[11:12]	Which Linden:	llhttpreques
[11:12]	Which Linden:	bleh
[11:12]	Kri Ayakashi:	but that still leaves lsl script to poll for changes every so often
[11:12]	Which Linden:	Kri: yeah, it's still polling, but you can minimize the resources consumed by using "long polls"
[11:12]	Which Linden:	or "comet"
[11:12]	Which Linden:	(scare quotes essential  :-)
[11:12]	Kri Ayakashi:	and xmlrpc or incoming http could be used more effectively in live communication
[11:13]	Kri Ayakashi:	yes you see comet is even less defined  :)
[11:13]	Which Linden:	The viewer uses comet for all its http-based communication, which covers a few things that are live.
[11:13]	Goldie Katsu is relieved that scare quotes were invented.
[11:13]	Which Linden:	I think
[11:14]	Charli3 Spitteler shouts: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
[11:14]	Which Linden:	But anyhow, comet is perfectly effective for pretty much all the stuff that goes over http
[11:15]	Charli3 Spitteler shouts: BUG!!!!!
[11:16]	Which Linden:	I think we need to make some easy-to-use comet tools so that people making lsl web apps can take advantage of it more readily
[11:16]	Which Linden:	Pantheon kinda gets there, but it's still somewhat difficult to understand
[11:16]	Which Linden:	And most people have a shared host that can't be running a Python process
[11:16]	Kri Ayakashi:	I understand that with current implementation of XMLRPC in Sl there is a main proxy that resolves the channel then connects to the script that initiated it and transfers the request, but wouldnt it be better if the proxy would only resolve the server and then with 30x header bounce the request to the correct server also informing the client to contact that server directly from now on?
[11:17]	Which Linden:	Kri: yeah, probably that would be a better design
[11:18]	Which Linden:	But honestly, having a proxy at all is unscaleable
[11:18]	Kri Ayakashi:	well the proxyt would be used only with first request
[11:18]	Which Linden:	I believe all inbound-http plans involve the script needing to communicate its own url to the outside world
[11:18]	Kri Ayakashi:	but agreed aside from the channel key there should be some domain or IP information that would allow the client to connect directly
[11:19]	Tao Takashi:	having more comet tools would definitley be good not only for SL
[11:19]	Which Linden:	... but for the world@
[11:19]	Which Linden:	!
[11:19]	Kri Ayakashi:	well with xmlrpc it's the same story, you need to provide the channel key to the external server
[11:19]	Squirrel Wood:	!stand kerry
[11:20]	Which Linden:	Oh? I guess so, forgot about that
[11:20]	Which Linden:	Yeah, so, once your script has some information that it has to get out there, it might as well be a full url.
[11:20]	Fumon Kubo , a complete non-webbie, frantically researches newterm >.>
[11:21]	Which Linden:	Ha ha, yeah, comet doesn't have a lot of official documentation,.
[11:21]	Kri Ayakashi:	would it be possible to have XML parser in LSL sometime in the future? I know all this kind of requests are postponed to after Mono release but still never too early to ask  :)
[11:21]	Fumon Kubo:	I generally get turned on to this kind of thingjust by it being mentioned in passing.
[11:21]	Tao Takashi:	I follow the comet planet
[11:21]	Tao Takashi:	or Comet Daily it is
[11:22]	Tao Takashi:	cometdaily.com
[11:22]	Which Linden:	Kri: anything is possible, especially since I have no idea about how that code works!
[11:22]	Which Linden:	It could probably provide kittens every 20 minutes for all I know
[11:22]	Kri Ayakashi:	heheh yeah the LSL compiler is a funky kung foo for me too  ;)
[11:22]	Tao Takashi:	right, what actually is the state of Mono?
[11:23]	Which Linden:	No idea. It's being worked on
[11:23]	Kri Ayakashi:	good question  :)
[11:23]	Tao Takashi:	let's not hope for the havok timeline  ;-)
[11:23]	Which Linden:	Yeah
[11:23]	Tao Takashi:	for me webmedia would be more important anyway
[11:23]	Fumon Kubo:	Probably being worked on over there > behind the door marked "Beware of the leper"
[11:23]	Tao Takashi:	then I actually could move much stuff out of SL and back on my webserver
[11:24]	Tao Takashi:	if the implementation allows that is
[11:24]	Kri Ayakashi:	sometime ago when Cory presented the Mono idea (there's a videocast) he shown a fibonacci script run on LSL2 based sim and Mono based sim, shortly after I did the same test on one of the regions I visited and the result time was similar to what Cory presented on Mono island
[11:24]	Which Linden:	I also have very little insight into the webmedia project, or even if there is one.
[11:25]	Tao Takashi:	there is one  :)
[11:25]	Which Linden:	Excellent.
[11:25]	Tao Takashi:	they wanted to finish stage 1 at q4/07
[11:25]	Tao Takashi:	But I think Callum should be careful with such announcements  ;-)
[11:25]	Which Linden:	I think people should know by now that we never ever hit our planned release dates.
[11:25]	Tao Takashi:	Qarl also seems to be involved
[11:26]	Tao Takashi:	I did no except it to happen but some people might not know about it
[11:26]	Which Linden:	So, re: the discussions about secondlife: urls on sldev
[11:27]	Which Linden:	I wanted to just put the notion out there that perhaps we should distringuish between real-url usage and "make the viewer do this thing" usage
[11:27]	Which Linden:	Right now those two uses are conflated into the secondlife: protocol, but they
[11:27]	Tao Takashi:	that also seemed to be the topic at Zero's
[11:27]	Which Linden:	Oh, was it
[11:27]	Tao Takashi:	it seems to be all over  ;)
[11:27]	Kri Ayakashi:	yeah
[11:27]	Tao Takashi:	I am just waiting for a fix, I am not too much involved but of course see the problem
[11:28]	Kri Ayakashi:	but it's valid especially when we keep in mind opening the grid to other grids
[11:28]	Kri Ayakashi:	which brings an idea that SL logins could be like OpenID
[11:28]	Tao Takashi:	openid was already discussed
[11:29]	Kri Ayakashi:	having KriAyakashi@agni.secondlife.com would let me in to other grids
[11:29]	Tao Takashi:	this of course should be an option in any case
[11:29]	Which Linden:	We should really move to openid
[11:29]	Tao Takashi:	but it also could be with the weblogin, after all openid is nothing else
[11:29]	Tao Takashi:	openid and yadis
[11:29]	Which Linden:	Once people stop naming projects "yet another. ..." I will stop spinning in my grave.
[11:30]	Kri Ayakashi:	hehe
[11:30]	Tao Takashi:	actually yadis was the initial name for openid  ;-)
[11:31]	Which Linden:	Hah, seriously?
[11:31]	Tao Takashi:	(as I just read  ;) )
[11:31]	Which Linden:	good thing they changed that
[11:31]	Tao Takashi:	but it's back  ;-)
[11:31]	Which Linden:	So, um, ok, moving on to a new topic: oplog versioning.
[11:32]	Which Linden:	Anyone want to hear about that?
[11:32]	Which Linden:	http://wiki.secondlife.com/wiki/Certified_HTTP_Versioning
[11:32]	Tao Takashi:	well, I am not really into the topic (yet)
[11:32]	Goldie Katsu grumbles at herself as she tries to rememeber and find the link to the other ID project
[11:32]	Tao Takashi:	what other? inames and the like?
[11:33]	Which Linden:	Higg--something
[11:33]	Which Linden:	Higgins
[11:33]	Which Linden:	http://www.eclipsecon.com/higgins/
[11:33]	Which Linden:	That what you mean, Goldie?
[11:33]	Which Linden:	Oh, hm, it's not a protocol, it's some code
[11:34]	Goldie Katsu:	no, it is something mentioned in a google discussion. They found problems with open ID and chose another one.
[11:34]	Tao Takashi:	LID and inames seem to be other ones
[11:34]	Goldie Katsu:	I will go dig that name up (mayb e in my delicious)
[11:34]	Which Linden:	There's also OpenAuth which is tangentially related
[11:34]	Tao Takashi:	or was it around authsub and stuff like that
[11:34]	Tao Takashi:	OAuth is maybe what you mean  ;-)
[11:34]	Which Linden:	Er, perhaps
[11:34]	Goldie Katsu:	Open Auth is probably it - I'll go poke my external memory sources.
[11:35]	Tao Takashi:	as OpenAuth is a TM
[11:35]	Which Linden:	Yeah it's oauth
[11:35]	Goldie Katsu:	yep Oauth
[11:35]	Tao Takashi:	OAuth is the new standard which evolved from all the other ones like Google's AuthSub, Yahoo's BBauth and so on
[11:35]	Which Linden:	OpenAuth is for AIM! Ha ha.
[11:35]	Tao Takashi:	as nobody wanted to use the competitors mechanism
[11:35]	Tao Takashi:	so they had to invent a new one to make everybody happy
[11:36]	Tao Takashi:	welcome to politics  ;-)
[11:36]	Tao Takashi:	now it's 1.0 and now we only need to see al those people adopt it  ;-)
[11:36]	Which Linden:	Eh, redesign is the order of the day in software
[11:36]	Goldie Katsu:	ah then it makes sense that that would be used for opensocial
[11:36]	Tao Takashi:	yes, if the software is bad
[11:36]	Tao Takashi:	not if I don't like the guy who wrote it maybe  ;-)
[11:36]	Goldie Katsu:	as an auth option
[11:37]	Tao Takashi:	well, only parts of oauth will be used for opensocial
[11:37]	Tao Takashi:	to sign requests I think
[11:37]	Tao Takashi:	but it makes sense to use that as then libraries could be reused
[11:37]	Tao Takashi:	the whole thing is not comparable to openid though as it's main goal is to give limited access to some service
[11:38]	Tao Takashi:	like flickr does when you want to export photos to 3rd party sites
[11:38]	Which Linden:	Right, OpenID allows you to sign in, and OAuth allows you to give other sites permission to do stuff with your data
[11:38]	Which Linden:	or deny, as the case may be
[11:38]	Tao Takashi:	yes, compared to the bad practices of giving out your gmail password to get the addressbook
[11:39]	Which Linden:	Or sharing your SL account password with your business partner. o.O
[11:39]	Goldie Katsu:	yes
[11:39]	Tao Takashi:	well, not sure if this is necessarily a use case.. here maybe better group controls could help  ;-)
[11:40]	Which Linden:	Yeah, totally.
[11:40]	Tao Takashi:	probably you need full permissions there
[11:40]	Goldie Katsu:	Group controls would be very helpful. I know shared projects are very difficult because of the current perms options.
[11:40]	Tao Takashi:	but for maybe getting some profile details from another agent domain might be a usecase
[11:40]	Which Linden:	But in the context of a multi-hosted grid, you'd probably want to be able to specify "joe's region can: NOT spend my L$"
[11:40]	Tao Takashi:	I recently again had the joy to sell 1000 objects to another avatar to take the build ober
[11:40]	Goldie Katsu nods
[11:41]	Tao Takashi:	over
[11:41]	Which Linden:	Whoa.
[11:41]	Goldie Katsu:	yes that is very hard.
[11:41]	Goldie Katsu:	especially when some are sub ground.
[11:41]	Tao Takashi:	yes, and she had to select all these items to buy them but some where somehow hidden inside the build and very hard to select
[11:41]	Goldie Katsu:	underground (not sure where my brain parser is today)
[11:42]	Tao Takashi:	I think a chmod for regions would be great  ;-)
[11:42]	Which Linden:	Heh, yeah
[11:42]	Tao Takashi:	chown I mean
[11:42]	Tao Takashi:	but chmod too
[11:42]	Goldie Katsu:	yes, or group write execute
[11:42]	Tao Takashi:	or that everything will be set to the group
[11:42]	Goldie Katsu:	chmod thisland 774
[11:42]	Which Linden:	Yeah, you kinda want to use the group as a proxy for the project
[11:43]	Which Linden:	Then you can change the membership of the group as the project changes
[11:43]	Tao Takashi:	yes, just make some filesystem interface for regions  ;-)
[11:43]	Goldie Katsu:	chmod thisland/someparcel 774
[11:43]	Goldie Katsu:	:)
[11:44]	Which Linden:	So, OK, I've been meaning to get this versioning thing off my chest, but I'm a procrastinator
[11:44]	Which Linden:	Lemme run off at the mouth for a bit then you tell me what you think
[11:44]	Which Linden:	The core of chttp is this notion of deterministic replay
[11:45]	Tao Takashi:	today I won't try to understand the code again though  ;-)
[11:45]	Which Linden:	What that means is, we have a way to write a bit of code such that you can run that bit, crash, then pick up again where we left off
[11:45]	Tao Takashi:	btw, whatwg is working on some serverside events for HTMl5
[11:45]	Tao Takashi:	so comet becomes less hacky
[11:46]	Which Linden:	Oplogs implement deterministic replay by periodically saving off pieces of state.
[11:46]	Which Linden:	excellent
[11:46]	Kri Ayakashi:	so it's like a self serializing coroutine?
[11:46]	Which Linden:	Kri: yeah, kinda
[11:47]	Which Linden:	If we could depend on stackless tasklet serialization we'd have done that
[11:47]	Kri Ayakashi:	it's not dependable? :D
[11:47]	Kri Ayakashi:	it works even on different endian architectures over network
[11:47]	Which Linden:	We don't run on stackless.
[11:47]	Kri Ayakashi:	how about Twisted's Perspective Broker?
[11:48]	Which Linden:	Never heard of that
[11:48]	Kri Ayakashi:	it can serialize and send objects over RPC
[11:48]	Kri Ayakashi:	actually I've made once a simple game where the client only deserialized objects sent from the server and used them
[11:48]	Which Linden:	Oh, yeah, we don't need to do this over rpcs, it's a local-state thing
[11:48]	Kri Ayakashi:	so the client was like 20lines or so but could run remote programs locally pretty nicely
[11:49]	Which Linden:	It does seem like an improvement over cpickle though
[11:49]	Which Linden:	So, I guess our current implementation has some limitations inherited from our choice to use bog-standard python 2.3
[11:49]	Kri Ayakashi:	but why do you need versioning? I'm sorry I'm not much into the certified http protocol
[11:50]	Which Linden:	OK, so the assumption of deterministic replay is that you run the same code and just use the oplog to fill in the state from the previous run
[11:50]	Which Linden:	But what if you change the code?
[11:50]	Which Linden:	Then the contents of the oplog don't apply
[11:51]	Which Linden:	This is exacerbated by the fact that the oplog doesn't store any meta information about the execution context, but the problem is pretty pervasive
[11:51]	Kri Ayakashi:	so you mean that this is some kind of low-level binary template or something like that?
[11:51]	Which Linden:	No, it's not low-level
[11:52]	Which Linden:	But think about it this way: you're modeling your app as a sequence of function calls
[11:52]	Kri Ayakashi has to read through the certified HTTP RFC obviously  :)
[11:52]	Which Linden:	If the return values from those function calls start coming back out in different order, you're boned
[11:52]	Which Linden:	The versioning page is probably a better reference for this problem, since it's specific to teh Python implementation
[11:53]	Which Linden:	http://wiki.secondlife.com/wiki/Certified_HTTP_Versioning
[11:53]	Kri Ayakashi:	hmm how about Deferreds? so called chained events
[11:53]	Which Linden:	We want to preserve a sequential programming style, it's kinda the reason that eventlet exists
[11:54]	Kri Ayakashi:	I see
[11:54]	Kri Ayakashi:	what's the driving reason?
[11:54]	Which Linden:	But yeah, Deferreds are kinda the same thing. Right, if you change the order of the deferreds, and then try to start up in the middle of one of them, it might not have the proper prerequisites in place to call the next one
[11:55]	Tao Takashi wanted to make a blogpost about eventlet
[11:55]	Which Linden:	Whether it's a sequential or callback style is actually somewhat immaterial to this problem, now that I think about it
[11:55]	Tao Takashi:	and I tried running Django with that WSGI server of it
[11:56]	Kri Ayakashi:	Tao and how did it go?
[11:56]	Kri Ayakashi:	I'm planning to give Twisted WSGI + Django a try
[11:56]	Which Linden:	I.e. this call sequence might be (state is referred to in <>s) A<a> -> B<a, b> -> C
[11:57]	Which Linden:	If you change the order for whatever reason, you might start with B -> A<a,b> -> C
[11:57]	Which Linden:	In which case your B has to change to no longer assume that <a> is there
[11:57]	Tao Takashi:	well, the server worked but I did not get any data over  ;-)
[11:57]	Tao Takashi:	I think at least
[11:57]	Which Linden:	But you might have some stored state wherein <a> is there!
[11:57]	Tao Takashi:	I also used eventlet for making my XFN parser/scraper a bit faster
[11:58]	Kri Ayakashi:	Which I see your point it can be tricky
[11:58]	Tao Takashi:	I just wasn't sure how to do this in an iterative instead of recursive style
[11:58]	Tao Takashi:	I also wanted to check out repoze which puts WSGI in front of Zope and try eventlet with that
[11:58]	Tao Takashi:	but I usually want too much anyway  ;-)
[11:59]	Which Linden:	Kri: yeah, so what the versioning does is associate a particular number with the code, and you have to keep the old code around so that it can be run on old state dumps
[11:59]	Kri Ayakashi:	Tao it's not too much, it's too little.. time  :)
[11:59]	Tao Takashi:	however you look at it  ;-)
[11:59]	Which Linden:	I'll have to look into stackless tasklets to see how they handle changing of the code.
[12:00]	Tao Takashi:	next week the SnowSprint starts, 1 week in the alps with lots of Plone and Python geeks, we'll see what I might get done there
[12:00]	Which Linden:	OK, I'm done nattering on about the versioning, thanks for listening.
[12:00]	Kri Ayakashi:	it's definetly interesting topic thanks for sheding some light onto it
[12:00]	Kri Ayakashi:	I need to read a bit more re. this.
[12:00]	Tao Takashi:	sorry, I really need to look first a bit deeper into it before I can be helpful there I think
[12:00]	Which Linden:	Lemme know if the wiki page needs editing.
[12:01]	Which Linden:	To be clearer or whatever.
[12:01]	Tao Takashi:	I am still at eventlet  ;-)
[12:01]	Kri Ayakashi:	I'll do my homework and let you know next week  :)
[12:01]	Which Linden:	You had me at "eventlet"  :-)
[12:01]	Which Linden:	Excellent!
[12:02]	Which Linden:	Oh, wow, how time flies. I have to do
[12:02]	Which Linden:	go
[12:02]	Kri Ayakashi:	heheh take care
[12:02]	Harleen Gretzky:	bye Which
[12:02]	Tao Takashi:	me, too.. getting late again at the office
[12:02]	Tao Takashi:	take care Which!
[12:02]	Which Linden:	Thanks for being a great audience, I enjoyed talking with you.