User:Which Linden/Office Hours/2007 Nov 1

Transcript of Which Linden's office hours:

[11:07]	Morgaine Dinova:	Was at a music oncert (as always) ... japanese is about as formal as I get though, lol
[11:08]	Which Linden:	Hah, yeah, it always seems like folks come to the office hours after a party. I guess that's SL for ya
[11:08]	Which Linden:	How'd Zero
[11:08]	Saijanai Kuhn:	was at the after-party of Zero's hours (40 people and some in the wings)
[11:08]	Which Linden:	's office hours go?
[11:08]	Saijanai Kuhn:	during his hours
[11:09]	Saijanai Kuhn:	which was ess than last time (80 people)
[11:09]	Morgaine Dinova:	Nah, this was Cylindrian's concert in Avilion
[11:09]	Wolt Amat:	What do you do, Which?
[11:09]	Morgaine Dinova:	That was at 58, despite 10am. She could sell out 500 at peak time.
[11:10]	Which Linden:	I'm a developer, Wolt
[11:10]	Which Linden:	The topic today will be something about web services
[11:10]	Wolt Amat:	Cool.
[11:10]	JetZep Zabelin:	brb 1 sec
[11:10]	Which Linden:	Either chttp like old, or mulib/eventlet since that was a good topic last time
[11:11]	Which Linden:	Any opinions?
[11:11]	Wolt Amat:	I'll just sit and listen, se if I can catch up.
[11:11]	Morgaine Dinova:	Mulib would be great. Our mulib rep on AWG has just left us (Gareth was poached), so info on that would be good.
[11:11]	Which Linden:	OK!
[11:11]	Wolt Amat waters Which, who is looking a bit dry.
[11:12]	Which Linden:	We definitely agreed last time that the mulib examples pages need some reqorking
[11:12]	JetZep Zabelin:	hehe thx which
[11:12]	Which Linden:	and by reworking I mean add lots more content
[11:12]	Morgaine Dinova:	As long as it's not JetZep who waters Which, I thik Which will be happy :-)
[11:12]	Which Linden:	heh
[11:13]	Tao Takashi:	Hi
[11:13]	Which Linden:	Hi!
[11:13]	Which Linden:	So, the stacked thing is fairly straightforward: put up a python object and it's accessible via REST
[11:14]	Which Linden:	(actually I'm looking through the transcript from last time, seeing if there's anything I should repeat/say)
[11:15]	Morgaine Dinova:	Transcript on your User page?
[11:15]	Which Linden:	Yeah, I'll post a link
[11:15]	Which Linden:	once the dang wiki loads
[11:15]	Morgaine Dinova:	Chhers, no need to repeat, unless you think crucial
[11:16]	Which Linden:	So one thing that is pretty cool is that mulib can automatically negotiate content-types
[11:16]	Morgaine Dinova:	Or just point us to a timestamp in transcript for key points made.
[11:16]	Which Linden:	Wait, I must have not posted last week's!
[11:16]	dibbs Dovgal:	lol
[11:16]	Which Linden:	Sorry all, I am a right fool
[11:16]	Morgaine Dinova:	Ew
[11:16]	Morgaine Dinova:	Hi dibbs
[11:17]	dibbs Dovgal:	Hey Morgaine
[11:17]	Which Linden:	I can't even post them now, they're on my other computer
[11:17]	Morgaine Dinova:	lol
[11:17]	Morgaine Dinova:	Just summarize highlights then :-)
[11:17]	Which Linden:	Well, we talked about stacked and how it's insecure but useful
[11:17]	dibbs Dovgal:	The web page does seem fairly sparse.
[11:17]	Tao Takashi:	wasn't the whole OH a highlight? :)
[11:17]	Which Linden:	It was amazing, you guys really had to be there. :-)
[11:18]	Zha Ewry:	Have you decided if the put is properly idempotent, by the way?
[11:18]	Which Linden:	One thing that we need more examples of is how a stacked object looks over REST
[11:18]	Which Linden:	Zha: no, we haven't
[11:19]	Zha Ewry:	I am pretty sure, that it isn't. But.. it's a little subtle
[11:19]	Which Linden:	or, rather, we haven't gotten any further in that discussion
[11:19]	Tao Takashi:	I should check how the REST part in Zope3 works
[11:19]	Tao Takashi:	would be similar to REST then
[11:19]	Tao Takashi:	err, stacked
[11:19]	Tao Takashi:	but with permissions ;_)
[11:19]	Which Linden:	Yeah, so to recap: if you PUT to /a/b/c an object whose value is 'foo', and then you PUT to /a/b/c an object whose value is 'bar', is that idempotent?
[11:19]	Which Linden:	We say yes, Zha says no
[11:20]	Zha Ewry:	I dont say no
[11:20]	Zha Ewry:	I say.. tha it's problemaic
[11:20]	Which Linden:	Hah.
[11:20]	Saijanai Kuhn:	undeterminate?
[11:20]	Which Linden:	Yeah, that's definitely true
[11:20]	Zha Ewry:	The question is when I put
[11:20]	Tateru Nino:	Do you mean "Is idempotence a requirement"? Or is the _sequence_ idempotent?
[11:20]	Zha Ewry:	foo, then you put bar, and I put foo, what's the expected behavior
[11:21]	Zha Ewry:	Especially if I get
[11:21]	Which Linden:	Idempotence is definitely a requirement for PUT
[11:21]	Zha Ewry:	idemptoent is deeply a equirmeent
[11:21]	Which Linden:	But the interaction with GET is not a part of that requirement
[11:21]	Which Linden:	I guess what we are talking about here is: is it RESTful?
[11:21]	Zha Ewry:	Well, the pattern, the RESTful pattern, is what happens when you try to use it to
[11:22]	Zha Ewry:	(right)
[11:22]	Tao Takashi:	depends on how far the context spans
[11:22]	Zha Ewry:	REST would argue, that the put, creates a new resource, and subsequent puts,. by me,
[11:22]	Tao Takashi:	so is there anything in the REST docs?
[11:22]	Which Linden:	I suppose this is complicated by the fact that there are no hard lines in REST
[11:22]	Zha Ewry:	should not change the state again
[11:23]	Zha Ewry:	Right
[11:23]	Tao Takashi:	so I'd need to use a new URL?
[11:23]	Which Linden:	Actually I've seen lots of discussions where the POST creates the resource and the PUT fills it in
[11:23]	Zha Ewry:	And.. We can, carefully chose how to do it
[11:23]	Zha Ewry:	That style would work too, Which. Well, in fact
[11:23]	Morgaine Dinova:	Idempotency is usually in reference to what happens when you put foo twice, not foo then bar. In a stateless system, there will always be the nondeterminacy of concurrent puts yielding different states depending on who wins.
[11:23]	Morgaine Dinova:	That's not a problem.
[11:23]	Which Linden:	yeah, what you said\
[11:24]	Which Linden:	:-)
[11:24]	Zha Ewry:	Right. But.. REST is deeper than jus idemportent
[11:24]	Zha Ewry:	REST says that you pay attention to more
[11:24]	Which Linden:	What section of the "REST doc" should we be referring to, anyhow?
[11:24]	Zha Ewry:	LOL
[11:24]	Zha Ewry:	I read Roy's thesis
[11:24]	Zha Ewry:	I don't recommend that to most people
[11:24]	Zha Ewry:	Maybe not even me.
[11:25]	Which Linden:	Hah. The wikipedia page ain't bad
[11:25]	Morgaine Dinova:	His thesis isn't very pragmatic, but it's a good read. For those who like theses :-)
[11:26]	Which Linden:	Anyway, my fundamental problem with your stance is that it implies that you cannot change a resource, ever
[11:26]	Which Linden:	Unless there's something more to it
[11:26]	Zha Ewry nods
[11:26]	Zha Ewry:	Its not that you can't..
[11:26]	Zha Ewry:	Its the question of telling what happens when multople people do... and Roy gets squidgy there
[11:26]	dibbs Dovgal:	Ups. Meeting . Sorry just when it was getting interesting!
[11:27]	Which Linden:	seeya dibbs, thanks for stopping by
[11:27]	Which Linden:	In http, you can't really know if more than one person is participating, technically
[11:27]	Which Linden:	theoretically
[11:28]	Morgaine Dinova:	Well happily, the better solution there is the one that happens to scale best, which is that updates are made not to a global state of the resource but to a received past state. This doesn't work when it's a shared global lke a wallet, but it works in a lot of normal situations.
[11:28]	Which Linden:	Morgaine: can you explain more?
[11:28]	Zha Ewry:	Well, right, Morg, but that implies a change in the reousrce on each put
[11:28]	Morgaine Dinova:	So anyone about to make a transaction gets a token for the state to be updated first ... from then on each transaction is separate.
[11:28]	Zha Ewry:	Roy is fond of the 50x fsyle
[11:29]	Zha Ewry:	*style
[11:29]	Zha Ewry:	where you keep invalidating the old state by idemportnet update
[11:29]	Zha Ewry:	In ptactice, that's either vie cookie ish stuff, or resource fiddling
[11:29]	Morgaine Dinova:	It even works for shared global states like a single wallet too, but only if chunks of the wallet are ringfenced first.
[11:29]	Zha Ewry:	(via
[11:29]	Which Linden:	So everyone has their own view of the object?
[11:29]	Tao Takashi:	so wa
[11:30]	Tao Takashi:	oops :)
[11:30]	Zha Ewry:	and. I'm not dogmatic about this.. but.. we need a very crisp agreement about what we expect to happenin our RESTful services, and how we expose them.
[11:30]	Tao Takashi:	so what happens if two people create a ressource at the nearly same time?
[11:30]	Morgaine Dinova:	A short-term view, yes. It's a bit like your temporary hand in the air in chttp
[11:30]	Zha Ewry:	One always is first at the server end
[11:30]	Tao Takashi:	what happens to the second one?
[11:31]	Zha Ewry:	either they get back a "the reousce has moved on, retryr" reply
[11:31]	Zha Ewry:	forwarding style
[11:31]	Zha Ewry:	or, they get back a resource busy, try again, which will cause them to retry when the resource is no longer being updated
[11:31]	Morgaine Dinova:	"I am about to take an interest in L$1000 in your wallet, give me a partial pass for it". The second party gets the same.
[11:32]	Which Linden:	So, effectively you lock the resource
[11:32]	Tao Takashi:	well, I am more talking about PUTing something to an already existing object
[11:32]	Morgaine Dinova:	Zha: yes, that's an alternative, but doesn't scale as well.
[11:32]	Morgaine Dinova:	No locking!!
[11:32]	Morgaine Dinova:	Not global locking anway :-)
[11:32]	Zha Ewry:	Depends on the read/write ratio.. and no it's not a lock
[11:32]	Morgaine Dinova:	Locking is forbidden in any discussions with me :-))))
[11:32]	Zha Ewry:	It's al ocal resource sayign "I can'f deal with you just now"
[11:33]	Zha Ewry:	Nicely, it's only the local resource which has to think about it
[11:33]	Morgaine Dinova:	Just kidding. In practice though, locking scales horribly, or not at all.
[11:33]	Which Linden:	I agree
[11:33]	Zha Ewry:	You always have to risk a fail. so, if someone else is updating the resource, it's no different than a timeout, or a other 50x failure
[11:34]	Which Linden:	So, let me see if I got this
[11:34]	Zha Ewry:	Not sure it's what I would do, but it is not a terrible practice
[11:34]	Zha Ewry:	What we want, is clean semantics, which let multple updaters know when they clash, but not lock
[11:34]	Which Linden:	The sequence for a client is: request modification rights for the resource; use modification rights to update resource
[11:34]	Morgaine Dinova:	Zha is right, ultimately. However, if the resource has substantial amounts available (we're not at the bottom of the barrel yet), then usually all locking and failure and retry semantics can be avoided.
[11:35]	Which Linden:	A client might see: request mod rights; get rejected; wait a bit then retry
[11:35]	Which Linden:	That sound right?
[11:35]	Morgaine Dinova:	Only when at the bottom of the barrel.
[11:36]	Which Linden:	Right, ok, normally it's the first case
[11:36]	Which Linden:	But still, the two-phase nature of it seems like it's adding state
[11:36]	Morgaine Dinova:	Eg. when a sim is full, and the transaction is a teleport.
[11:36]	Morgaine Dinova:	Whereas the normal situation doesn't hit the resource exhaustion.
[11:37]	Zha Ewry:	Depends on what the resource needs to do, and whetheer the updters need to see the intermediate state
[11:37]	Zha Ewry:	If you don't, that's ok.
[11:37]	Morgaine Dinova:	Yeah
[11:37]	Zha Ewry:	If you do it's a problem
[11:37]	Which Linden:	Ok, well, this has been interesting but we have deviated somewhat from our topic
[11:38]	Zha Ewry:	Drag us back, Which. Your meeting
[11:38]	Morgaine Dinova:	For most resources, you have a pool, and it can get exhausted. At the bottom of the pool, Zha's retry semantics kick in.
[11:38]	Which Linden:	So: content-type negotiation!
[11:38]	Zha Ewry:	Depends on the semantics of the underlying resource, tho
[11:38]	Which Linden:	It's so bad.
[11:38]	Morgaine Dinova:	Zha: yep. Some non-scalable resources have a pool size of unity.
[11:39]	Zha Ewry:	(We'll try to avoid them, but sometimes.. ick)
[11:39]	Which Linden:	When you're handling a request with mulib, you send back the response by doing request.write(body)
[11:39]	Which Linden:	But body can be any obkject not just a string
[11:40]	Which Linden:	You register converters for different object types
[11:40]	Wolt Amat:	Thanks, Which. Need to go. See you again.
[11:40]	Which Linden:	See ya wolt!
[11:40]	Morgaine Dinova:	Cyu Wolt
[11:41]	Which Linden:	Effectively you're building up a many-to-many relationship between content-types and object types
[11:41]	Tao Takashi would like to see this done with interfaces and adapters
[11:41]	Tao Takashi:	cya wolt
[11:41]	Unmasked Shepherd pouts.
[11:42]	Which Linden:	Yeah, right now you just call stacked.add_producer(python_type, converter, content-type)
[11:42]	Morgaine Dinova:	Which: many-to-many suggests that an object can be of more than a single type, right?
[11:42]	Which Linden:	Yeah, so, pretty much any python object can be converted into text/plain, say
[11:43]	Which Linden:	Likewise, they can all probably be text/html if you wrote a converter for that
[11:43]	Tao Takashi:	with adapters you'd say ITextPlain(object) and you get the adapter for it
[11:43]	Tao Takashi:	then you might call adapter.output() or something
[11:43]	Morgaine Dinova:	This is very powerful, since it means we'll be able to optimize the transfer representations later.
[11:44]	Which Linden:	Yeah, if you have a client that wants application/binary-llsd you can get that
[11:44]	Which Linden:	(if we wrote that converter)
[11:44]	Which Linden:	Oh, it's application/llsd+binary
[11:44]	Tao Takashi:	do you also have chains which can be resolved automatically?
[11:44]	Morgaine Dinova:	Start off with a basic converter for an on-the-wire representation that works, then hone it.
[11:44]	Which Linden:	Chains.... not sure what that means, Tao
[11:45]	Tao Takashi:	we have a similar approach in the portal_transforms tool in Plone where you also register transforms like that
[11:45]	Tao Takashi:	and if you there is no direct transformer from object to text/plain it might go via text/html automatically
[11:45]	Which Linden:	Ah, right, well that depends on what's in the accept header that the client sends down
[11:45]	Tao Takashi:	if text/html is registered and one for text/html => text/plain is registered
[11:46]	Which Linden:	Oh, that seems weird to me. How could you write a general-purpose text/html to text/plain converter?
[11:46]	Tao Takashi:	you strip out the html tags mostly
[11:47]	Tao Takashi:	it's also mainly used for indexing purposes
[11:47]	Which Linden:	Ah, hmmm..
[11:47]	Morgaine Dinova:	I guess that's handy for debugging, maybe.
[11:47]	Which Linden:	Yeah, actually it would be
[11:47]	Tao Takashi:	but you might think of a text/html to pdf converter
[11:47]	Tao Takashi:	and then you wouldn't need to write a pdf converter for every type
[11:47]	Which Linden:	Since the htmlaxception stuff is pretty hard to read when it gets dumped into the command-line
[11:47]	Tao Takashi:	but we probably have different use cases here
[11:47]	Tao Takashi:	as you are not creating a CMS ;-)
[11:48]	Which Linden:	Yeah
[11:48]	Which Linden:	We're more interested in machine-readable content
[11:48]	pallina60 Loon:	boh
[11:48]	Abu Beerbaum:	mh
[11:48]	Morgaine Dinova:	So all this is in mulib now, Which?
[11:48]	Which Linden:	So it doesn't support chains right now, but it does support the many-to-many mapping
[11:48]	Tao Takashi:	but chains might be useful as well here maybe.. you might only need one converter to a common format and convert from there maybe in different ones
[11:49]	Tao Takashi:	instead writing a converter for each combination
[11:49]	Abu Beerbaum:	do you whant speak with Benjamin about Denis ?
[11:49]	Abu Beerbaum:	thank´s Angie :-))
[11:49]	pallina60 Loon:	no babbler abu
[11:49]	Morgaine Dinova:	No need to extend it as yet ... just as long as the principle is extensible to adding chains later.
[11:49]	Angelica Lean:	it's most important to explain the gebedia questions
[11:49]	Angelica Lean:	ty
[11:49]	pallina60 Loon:	yes
[11:49]	Angelica Lean:	i think...
[11:49]	Which Linden:	Hey Abu and company: we can hear you, do you mind moving over a bit?
[11:50]	pallina60 Loon:	qui non va babbler
[11:50]	pallina60 Loon:	:-(
[11:50]	Tao Takashi:	actually it would be nice now to have more componentized stuff in the python world so that you could actually reuse this transform tool
[11:50]	Abu Beerbaum:	ok , sorry
[11:50]	Angelica Lean:	Hello All happy hallowen:-)
[11:50]	Which Linden:	:-)
[11:50]	Which Linden:	If you want to see the code in mulib now: http://svn.secondlife.com/svn/mulib/trunk/mulib/stacked.py
[11:51]	Which Linden:	search for 'add_producer'
[11:51]	Which Linden:	Just a big 'ol dict of dicts
[11:51]	Tao Takashi:	ok, so it's a module variable
[11:51]	Which Linden:	yep
[11:52]	Tao Takashi:	this is what I'd do with zope3 like interfaces/adapters :)
[11:52]	Morgaine Dinova:	svn://svn.secondlife.com/svn/mulib/trunk ? Or is that first "svn" in front of /mulib superfluous?
[11:52]	Which Linden:	It could possibly be broken into a set of config files or something, we've discussed stuff like that
[11:52]	Morgaine Dinova:	Trying
[11:52]	Which Linden:	No it's necessary Morgaine
[11:52]	Tao Takashi:	well, the nice thing about adapters is that you define new adpaters anywhere
[11:52]	Morgaine Dinova:	kk
[11:52]	Tao Takashi:	so in fact it's a distributed config file
[11:53]	Which Linden:	I think you want http as the protocol if you're checking it out
[11:53]	Tao Takashi:	if you have a new type you can actually define the adapter locally
[11:53]	Which Linden:	You can just hit that url in a browser though
[11:53]	Which Linden:	You can do that with stacked, too Tao
[11:53]	Which Linden:	You just have to import stacked
[11:53]	Which Linden:	(thus introducing a possibly-unwanted dependency)
[11:54]	Which Linden:	(but that's the breaks)
[11:54]	Tao Takashi:	you don't need this with interfaces etc.
[11:54]	Tao Takashi:	I might do an example
[11:54]	Which Linden:	Sure! Be glad to see it.
[11:54]	Tao Takashi:	I really dig this stuff these days :) in fact we converted the whole Plone stuff to use this
[11:54]	Tao Takashi:	we had similar approaches before
[11:55]	Tao Takashi:	but if it grows they might become unflexible
[11:55]	Tao Takashi:	but also depends on the use cases I guess
[11:56]	Tao Takashi:	I will email you an example how it works.. I have 6 hours of train rides tomorrow so I should find some time :)
[11:56]	Which Linden:	Ha ha, yeah
[11:56]	Which Linden:	OK, well, that topic was a lot simpler than I thought it would be
[11:56]	Morgaine Dinova:	Yeah, you were right about the protocol. Cool, got it, will have a play.
[11:56]	Tao Takashi:	I will also try to get through that code and maybe add a docstring here and there ;-)
[11:56]	Which Linden:	We are accepting patches these days. :-)
[11:57]	Morgaine Dinova:	Hehe
[11:57]	Which Linden:	But that is no excuse for us not going through ourselves
[11:57]	Zha Ewry:	Which.. Would you like to schedule an office hours, to chew the idemptoten/rest patterns a ibut more?
[11:57]	Zha Ewry:	That would be very cool
[11:57]	Tao Takashi:	I just should submit a contrib agreement ;-)
[11:57]	Which Linden:	Zha: yeah, why don't I invite donovan for that?
[11:57]	Which Linden:	He's more seasoned in this than I am
[11:58]	Morgaine Dinova:	What's in the contrib agreement? I don't sign NDAs anymore, had enough of that in defence contracts.
[11:58]	Zha Ewry:	Sure. Ahm. Maybe Donovan, and Zero, in fact.
[11:58]	Tao Takashi:	nothing about not being allowed to say anything ;-)
[11:59]	Jai Baxter:	Just your first born child as hostage, Morgaine, nothing serious
[11:59]	Which Linden:	There's no NDA in the contrib agreement, I don't htink
[11:59]	Zha Ewry:	Just something about patents, copyright, blood of your firstborn child. But you can talk about it
[11:59]	Morgaine Dinova:	Hehe
[11:59]	Tao Takashi:	it just makes sure that Linden Lab has the right on your contributions
[11:59]	Jai Baxter:	jinx Zha, you owe me a Coke :P
[11:59]	Tao Takashi:	rights
[11:59]	Which Linden:	The contrib agreement is on the wiki somewhere, can't find it now
[11:59]	Morgaine Dinova:	If it's just ownership transfer, that's fine.
[11:59]	Which Linden:	Zha: We actually rely on the current behavior of mulib already
[12:00]	Jai Baxter:	But seriously, it's a fairly benign one
[12:00]	Which Linden:	So it'd take some doing to convince us to change it. :-)
[12:00]	Which Linden:	Not that it can't be done
[12:00]	Tao Takashi:	Morgaine: basically I think it is
[12:00]	Tao Takashi:	not that I've read it that closely yet
[12:00]	Tao Takashi:	but I guess I will do it before I sign it :)
[12:00]	Zha Ewry:	I'll pass the link if the wiki ever loads
[12:01]	Which Linden:	found it: http://secondlifegrid.net/programs/open_source/submission
[12:01]	Zha Ewry:	And. Which, the right answer may be, 90% case just to make the semanic clear
[12:01]	Which Linden:	Linked from this page: http://wiki.secondlife.com/wiki/Submitting_code
[12:01]	Zha Ewry:	Mostly a matter of makeing it all clear
[12:01]	Which Linden:	Zha: yeah, I like clear things.
[12:01]	Morgaine Dinova:	That services supplier really needs a good kicking. Dunno how Lindens put up with it, lol. Shouldn't be hard to get a wiki to run efficiently :-)
[12:02]	Zha Ewry:	I still say the outsourced it to someone with a ZX81
[12:02]	Morgaine Dinova:	Hehe
[12:02]	Which Linden:	Morgaine: yeah, though to some degree we're putting a burden on them with e.g. our custom auth logic
[12:02]	Tao Takashi:	but reading?
[12:03]	Tao Takashi:	there are clever inventions like Squid or Varnish ;-)
[12:03]	Which Linden:	Yeah, we're kicking them on that
[12:03]	Which Linden:	I'm not, Rob is
[12:04]	Which Linden:	It's OK to bitch about it, though. Best to keep visibility up
[12:04]	Which Linden:	OK, wow, it's over time.
[12:04]	Which Linden:	I should head in and put up my office hours from the past!
[12:04]	Tao Takashi:	indeed, I should go :) need to get some stuff packed for Berlin
[12:04]	Jai Baxter:	Good stuff, though, Which. Thanks.
[12:04]	Which Linden:	Thank you all for stopping by.
[12:05]	Which Linden:	:-)
[12:05]	Tao Takashi:	thanks again for hosting :)
[12:05]	Morgaine Dinova:	Well, weakest link in the chain applies ... once the wiki is fixed, we'll moan about IM slowing down AWG discussion :-)))
[12:06]	Morgaine Dinova:	Yeah, good as always Which.
[12:06]	Saijanai Kuhn:	a reminder to all, if you have an interst in helping with the design of the next (and next next) versions of SL, to IM me or Zha for an invite inot AW Grouipes
[12:06]	Which Linden:	Sheesh, can't tp. Gonna log off :-)