User:Which Linden/Nov 12 PUT Semantics Discussion

Transcript of a short meeting of interested parties on the semantics of PUT:

[13:40]	Zha Ewry:	So.. Was Zero going to try and make it? Or is he still in trransit?
[13:40]	Which Linden:	hey there
[13:40]	Which Linden:	zero lied about being available today
[13:41]	Zha Ewry:	Ah
[13:41]	Which Linden:	he is at a conference
[13:41]	Zha Ewry:	You konw ou're short, when you're shorted than the potted plants ;-)
[13:41]	Which Linden:	my av is actually kinda big
[13:41]	Which Linden:	so, don't feel bad
[13:41]	Which Linden:	I'll come up with a tiny version later
[13:41]	Zha Ewry:	Well, yes, and konw. The Ave.. is tiny. the prims are tall.
[13:42]	Zha Ewry:	*no
[13:42]	Zha Ewry:	Sheesh. So.
[13:42]	Zha Ewry:	We pretty much, I think were on the point that, we'll use very lightweifh idempotent on the low level calls?
[13:42]	Which Linden:	I'm not 100% sure what that means, zha
[13:43]	Morgaine Dinova:	No seats. Set buildable for a few mins so we can res some?
[13:43]	Donovan Linden:	sure
[13:43]	Zha Ewry:	We were chewing, in chat.. about which model for put.. and in general, we seemed to hink that we were OK with the lightest possible models, with on way of telling collisions,a nd such, for most of our calls
[13:43]	Morgaine Dinova:	Not needed ... but for some reason humans seem to like them :-)
[13:44]	Zha Ewry:	Zero, had, laid out his thinking
[13:44]	Which Linden:	Yeh, that is true, we don't want to burden our communications with extensive requirements.
[13:44]	Zha Ewry:	Which, I think is pretty much, only break out the heavy stuff, when we really need it
[13:44]	Zha Ewry:	Right, what you said
[13:44]	Donovan Linden:	k I think I made it a sandbox
[13:45]	Zha Ewry:	Which, rather leads to wanting to enumerate what the heavy stuff is. Where do we need to know aboru collisions, and then, pick out a
[13:45]	Zha Ewry:	strategy for those
[13:45]	Zha Ewry:	c-http, presumably, for most of them
[13:45]	Zha Ewry:	and Escrow, when its n-way.
[13:45]	Morgaine Dinova:	I'd like you to consider splitting burdened and lightweight services, so that we can make the latter more highly scalable (assuming that they're the most common)
[13:45]	Which Linden:	Yeah, though the operations teh escrow performs will largely be POST semantics
[13:46]	Zha Ewry:	Right
[13:46]	Which Linden:	OK, so what's the definition of a heavy service?
[13:46]	Morgaine Dinova:	idempotent ones I guess. But anything that eats resources swerver-side
[13:46]	Zha Ewry:	I only care for this dicussion, about thinking about where we'll need something like c-http, or escrow
[13:47]	Zha Ewry:	And. Yeah, in general, what morg said
[13:47]	Zha Ewry:	we have, I think, a set of stuff, where, we genuinely don't care about over-writes
[13:47]	Which Linden:	Right
[13:47]	Zha Ewry:	and.. a smaller set, where we care about allowing to detect, and re-try
[13:47]	Zha Ewry:	and.. a much smaller set, where we want it to be multi-party ACID
[13:47]	Which Linden:	How would one re-try? With a new uuid?
[13:48]	Zha Ewry:	I would assume, yes
[13:48]	Which Linden:	What's the use case for multi-party ACID?
[13:48]	Zha Ewry:	That's escrow, for htings like "buy land"
[13:48]	Zha Ewry:	Where you want the lindens, the land, and such to either all go, or not go, not in between, right?
[13:49]	Which Linden:	Ah, actually we're planning on that merely being atomic
[13:49]	Zha Ewry:	Really?
[13:49]	Which Linden:	Er, I guess wewant durable too
[13:49]	Which Linden:	But it's certainly not isolated
[13:49]	Zha Ewry:	Ah
[13:49]	Zha Ewry:	Hmm.
[13:49]	Zha Ewry:	Not?
[13:49]	Zha Ewry:	Probably not
[13:50]	Morgaine Dinova:	I read what Zero said on that last time, and agree completely. Overwrite and state-change between your (subjectively) idempotent writes is not a problem, that's how the world works. If you don't want that semantic, don't use that mechanism, use some kind of single-assignment method.
[13:50]	Zha Ewry:	I tend to head straight to ACID, what can I say, DB/2 lurks in my blood
[13:50]	Which Linden:	Not, because e.g. in the case where you're buying nonexistant land, it might take your money, decide the land doesn't exist, then return your money, so for a short period of time you are sans money
[13:50]	Zha Ewry:	Right. Zero's comments, where very considered
[13:50]	Which Linden:	Yeah, I think that should be the "default"
[13:50]	Zha Ewry:	and.. what we came to, I think, was, that, we need to spec the basse idempotency
[13:51]	Zha Ewry:	and.. then figure out where we do care about detecting collisions
[13:51]	Donovan Linden:	what kind of collisions?
[13:52]	Which Linden:	Like assets, perhaps
[13:52]	Donovan Linden:	so what exactly are we talking about when we say something like "spec the base idempotency"
[13:52]	Zha Ewry:	Assets, in the large.. are interesting.
[13:52]	Which Linden:	But in that case the uuid generation virtually guarantees there is no unintentional collision
[13:52]	Donovan Linden:	right which
[13:53]	Zha Ewry:	That depends, a little, if you keep the every mod, makes a new asset stake
[13:53]	Donovan Linden:	in my mind, the PUT is an atomic swap of the old entity for the new entity
[13:53]	Which Linden:	I guess the interesting case is if multiple parties are editing the same asset, and blowing away each other's edits
[13:53]	Donovan Linden:	and they happen in whatever order they get to the server in
[13:53]	Zha Ewry:	Which. is a little less clear, if every asset has an URL, but I assume we keep that model, because it caches well
[13:54]	Which Linden:	Yeah, I think that's reasonable
[13:55]	Which Linden:	I' still having trouble understanding why we need anything more than the simple replacement semantics
[13:55]	Donovan Linden:	replacement semantics are what I was imagining
[13:55]	Morgaine Dinova:	Just in case it feels like we're in dangerous waters, we're not: remember that the whol Unix filestore and processes work that way: anyone can overwrite global state from under you. Yet, it's not a problem, because most things are coded sensibly.
[13:55]	Which Linden:	Yeah, true
[13:55]	Zha Ewry:	Yeah, the only place, it bites you, a bit.. is if you are logging, but then you do append semantics.
[13:55]	Donovan Linden:	yes, all the parts have to be coded defensively, that's just the way it is
[13:56]	Which Linden:	append semantics?
[13:56]	Donovan Linden:	for append semantics I have been using POST
[13:56]	Zha Ewry:	As long as we consistently document it that way, and make it easy to do,
[13:56]	Zha Ewry:	righ.. and post, is assumed heavyier weight
[13:56]	Donovan Linden:	the server knows how big the existing store is, and generates the new url from that
[13:56]	Which Linden:	heavier weight is sort of relative
[13:56]	Zha Ewry:	Sure
[13:57]	Which Linden:	since PUT implies moving a lot of data around, whereas POST could just describe a delta
[13:57]	Zha Ewry:	Hmm
[13:57]	Donovan Linden:	I think PUT moving a lot of data around is inerent in REST... you're transferring representations
[13:57]	Donovan Linden:	you get consistency by shipping a consistent bundle
[13:58]	Zha Ewry:	Well, if its big, yes
[13:58]	Which Linden:	So, if I understand it right, you're thinking about appending to a list, and what happens if two parties think that 5 is the last element and PUT their entry at 6 simultaneously
[13:58]	Zha Ewry:	A lot of what we're going to put or post, shouldn't be all that big
[13:58]	Donovan Linden:	which: that is what I was thinking of, and I have implemented it by having the clients do a POST...
[13:59]	Zha Ewry:	Right.
[13:59]	Donovan Linden:	and the server turns around and does a self-PUT to the new url
[13:59]	Which Linden:	(if the list's representation is like stacked's, where the entries are all at list_name/1 , 2 , 3)
[13:59]	Which Linden:	Yeah, though with stacked you can directly PUT to elements of the list
[13:59]	Zha Ewry:	Well, you hopefully, don't let them see the place in the list, until they get back the result
[13:59]	Which Linden:	Right
[13:59]	Zha Ewry:	No reason to make them know
[13:59]	Donovan Linden:	which: that would be more like list[4] = 'foo' semantic
[13:59]	Which Linden:	Right
[14:00]	Which Linden:	So, I guess the answer is "don't do that
[14:00]	Which Linden:	"
[14:00]	Zha Ewry:	What you don't want, I think is list[x] = "y"
[14:00]	Zha Ewry:	unless you only care that Y get set to some last value
[14:00]	Zha Ewry:	Err, list[x]
[14:00]	Which Linden:	You could PUT to /list_name/last
[14:00]	Which Linden:	and then the server could figure it out, but that sounds like it might as well be a POST
[14:00]	Zha Ewry:	Yes, and then, you get back, where in the list you were, if you care
[14:01]	Donovan Linden:	actually.. yeah... the server can redirect to the new url, I read about that in the spec
[14:01]	Which Linden:	in the location header in the response to the PUT?
[14:01]	Zha Ewry:	Yes, re-direct, in general, is a very good reply in REST.
[14:01]	Donovan Linden:	which yes I think so
[14:01]	Donovan Linden:	should look at the spec again
[14:01]	Zha Ewry:	You did a put to X, and you get back the new URL, which is where it got put
[14:01]	Donovan Linden:	I think so zha
[14:02]	Which Linden:	zha: did you have to go at 2?
[14:02]	Zha Ewry:	That.. is a pretty cleans ememntic
[14:02]	Zha Ewry:	About yes
[14:02]	Zha Ewry:	*semantic
[14:02]	Which Linden:	ok, well, shit, I guess I thought we would be done
[14:02]	Zha Ewry:	So.. we basically, have a very simple semantic here, for 90% of this. What's going to be inetrestign is flagging where
[14:02]	Zha Ewry:	we need more, and the best use cases
[14:03]	Zha Ewry:	But.. the basics, sound like we all agree
[14:03]	Donovan Linden:	it seems like we're close to done
[14:03]	Donovan Linden:	simple replacement for PUT and chttp and more complicated handshakes for more complex things
[14:03]	Zha Ewry:	The big thing, is going to be making it very clear to people what we expect the behairo to be
[14:03]	Zha Ewry:	Exactly donovan
[14:03]	Zha Ewry:	and.. then.. we can talk about subscribing to continiing notification
[14:03]	Which Linden:	Yeah, should we make Stacked handle the /last semantic or whatever?
[14:04]	Which Linden:	for lists?
[14:04]	Morgaine Dinova:	Better think about resourcing on POST. If it's an infinite append, you'd better think about limits.
[14:04]	Donovan Linden:	I already have a publish/subscribe framework for put in pantheon :)
[14:04]	Zha Ewry:	Becaause, with that, and this, afd capabiltiies, I think we have all the low level stuff
[14:04]	Zha Ewry:	Ah
[14:04]	Zha Ewry:	Pantheon?
[14:04]	Donovan Linden:	which: it's sugar, but sure we could do it
[14:04]	Which Linden:	I like sugar
[14:04]	Donovan Linden:	zha: in mulib, mulib/pantheon.py
[14:04]	Zha Ewry:	Sugar, on low level protocols
[14:05]	Zha Ewry:	is really good, because it gets consistent use for the case someone else will bork up if you make them all code it
[14:05]	Which Linden:	Isn't the pub/sub stuff implemented more fundamentally on the Nodes?
[14:05]	Which Linden:	I guess Pantheon is a more robust implementation
[14:05]	Donovan Linden:	yeah. so I'm for implementing PUT to append then
[14:05]	Donovan Linden:	which: we should look at that again, but I think the multiplexing in pantheon is a key improvement
[14:05]	Which Linden:	Cool beans, I am too
[14:06]	Zha Ewry:	So. Which. do we want to talk about pub/sub soon?
[14:06]	Donovan Linden:	sure, but for what use case
[14:06]	Zha Ewry:	I would love to have all the core transport semntics proposed,, because then we can go and begin coding them, and breaking them
[14:06]	Which Linden:	Yeah, we could go over the Pantheon stuff at my office hours, if you want
[14:06]	Donovan Linden:	sure, that sounds like a good idea
[14:06]	Zha Ewry:	Thats sounds like a good statrt, Which.. and..
[14:06]	Zha Ewry:	Ahm.
[14:07]	Donovan Linden:	it would be good to get the collaborative-editor branch merged into trunk soon
[14:07]	Zha Ewry:	How about, a simple use case, like "Where is avatar X"
[14:07]	Morgaine Dinova:	Yeah, and I can go about thinking about their scalability then.
[14:07]	Zha Ewry:	which ought to be subscribable
[14:07]	Donovan Linden:	zha: that's a great one, and I already have a bunch of internal docs on that in the wiki I can move to the public wiki
[14:07]	Zha Ewry:	Uber cool
[14:07]	Which Linden:	Word up
[14:07]	Zha Ewry:	OK, I need to run
[14:08]	Zha Ewry:	If zero thinks we're good here..
[14:08]	Which Linden:	Later! thanks for coming!
[14:08]	Zha Ewry:	I'm psuyched
[14:08]	Which Linden:	Ha ha, yeah, I'll forward this transcript to him
[14:08]	Zha Ewry:	pysched, even
[14:08]	Morgaine Dinova:	Sounds unhealthy
[14:09]	Donovan Linden:	sweet, thanks a lot
[14:09]	Which Linden:	All right, so we gotta get that stuff going on. Donovan, want to pair on that stuff?
[14:09]	Morgaine Dinova:	Well, that was easy, lol
[14:09]	Donovan Linden:	sure
[14:09]	Which Linden:	Ha ha, yeah sometimes you just have to see where you agree
[14:11]	Morgaine Dinova:	I tend to see it as finding the paths towards agreement. Initially there isn't always an intersection, you have to find it :-)
[14:11]	Morgaine Dinova:	Anyway, I'll leave you good folks to it :-)
[14:11]	Which Linden:	OK! Thanks for coming!
[14:11]	Morgaine Dinova:	Thanks Which, see you anon :-)
[14:11]	Which Linden:	:-)
[14:11]	Morgaine Dinova waves to Donovan