User:Zero Linden/Office Hours/2007 May 24

{{{ Jarod Godel: Are you our guest pris^H^H^H^Hhost? Donovan Linden: Hello, I am your guest, erm, host Rex Cronon: hello everybody Jarod Godel: cool Donovan Linden: Hi rex Rex Cronon: for some reason i can't detach my gun. so don't be intimidated by me. although i came bearing arms, i don't intend to use them:) Donovan Linden: hehe Catherine Linden is Online Donovan Linden: it will probably just take a while, they shipped derezloss, a branch which makes moving items back into inventor safer Rex Cronon: i guess to be safer takes longer Donovan Linden: instead of removing the item from the simulator immediately, it waits until the inventory database says it actually saved the object Donovan Linden: we should have a "derezzing" effect, like in tron, then if it takes a long time people will just think it is cool :) Donovan Linden: So how long should I wait for more people to arrive? Rex Cronon: usually they are here around this time +/- 10 minutes Donovan Linden: I'll jus wait 10 minutes Donovan Linden: Any votes on what to talk about? I was thinking I could talk about how message-liberation and capabilities Donovan Linden: could allow us to provide web services easily Rex Cronon: how about sculpties? Donovan Linden: I don't know much about sculpties Donovan Linden: what do you know about them? :) Rex Cronon: why were sculpties used when instead a .obj file format would have been a lot better? Jarod Godel: yes Donovan Linden: hmm. well, can you guess? Donovan Linden: I have a guess Jarod Godel: Message liberation and how that relates to the modularization of second life protocols. Rex Cronon: ok i am listening Donovan Linden: jarod: ok, good, that's what I'll talk about Donovan Linden: rex: ok, I think sculpties were implemented the way they were for reasons of efficiency on many levels Jarod Godel: Rex, I think scupties are preferred because it's still one prim. OBJ files are, I think, multiple primitives. Could be wrong, though. Donovan Linden: one, you can't store that much vector data in a texture, so that limits the complexity of objects nicely Donovan Linden: and two, since it's a texture, all of the code for downloading and parsing it was already written Donovan Linden: and any other things that we can already do with textures can more easily be applied to them Donovan Linden: for example, we could perhaps animate a sculptie texture, or programatically generate one Donovan Linden: because libraries are already available in the client to do so Rex Cronon: ok here is what i think. obj files are text and easier to edit and compress, u can load only objects that have a specific number of vertices, and in a .obj file u can have holes inside objects Donovan Linden: I think sculpties are a great example of how to ship a feature, iteratively Donovan Linden: rex: the client is open source, someone could easily write an .obj to prim/sculptie importer Rex Cronon: oh and an object file can contain only 1 object, doesn't have to contain multiple groups(group is set of vertices and faces) Jarod Godel: ok Donovan Linden: anyway, I think the answer really is, it was easy to do and get right Donovan Linden: we could write a server-side .obj to prim/sculptie converter some day and expose it as a web service, perhaps Jarod Godel: Message liberation... Donovan Linden: or someone else can do it Donovan Linden: ok, let's get started Jarod Godel: Benjamin keeps talking about building a web browser plun-in for SL. Rex Cronon: ok lets go to message liberation:) Donovan Linden: how many people know what message liberation is already Rex Cronon: hi wyn Jarod Godel: Does that relagte to M.L.? Donovan Linden: quick status update on the branch, it is going into qa ASAP but the "surprise" release backed up qa a bit Wyn Galbraith *yawns without coffee* "Good morning, Rex, Jarod, and Greetings Donovan. Jarod Godel: Hi Donovan Linden: jarod: I will talk about how that related to m-l today :) Jarod Godel: woot! Donovan Linden: the "surprise" release was a decision to do two smaller releases rather than wait for m-l again Rex Cronon: m.l = message liberation? Donovan Linden: yes, rex Donovan Linden: but in our informal testing, m-l seems to be very free of bugs Emily Brissot is Offline Donovan Linden: we are on branch message-liberation-8 now, and the last one that shipped into release was m-l-4 Donovan Linden: so it hasn't seen real production use in a while, which always exposes bugs Jarod Godel: fracturing protocol, one release at a time Khamon Fate: hey y'all people Donovan Linden: but we may have shaken most of the bad bugs out with m-l-4, so hopefully the Day of Liberation goes smoothly Khamon Fate: when is the Day of Liberation scheduled? Wyn Galbraith murmurs goodmorning to Khamon. Donovan Linden: still not scheduled, but it should be soon now Jarod Godel: Are you planning on exposing messages to us soon, aside from opening the client? Khamon Fate IMs answers to Wyn Jarod Godel: err, aside from the open client Donovan Linden: ok, and now on to how m-l benefits residents Donovan Linden: initially m-l will only be used to allow us to fine-tune our internet usage Wyn Galbraith: Morning Zha. Donovan Linden: by moving messages over llsdhttp we can ship data to the viewer in a way that is more efficient and plays more nicely with internet providers Zha Ewry: Morning, Wyn Rex Cronon: could there a page on wiki that lists all the benefits that ml will bring? Zha Ewry: all Donovan Linden: so residents will see this as a modest increase in bandwidth utilization Rex Cronon: hi zha Donovan Linden: Rex: Good idea I'll make tasks for icehouse to do that Donovan Linden: but just switching old style asynchronous messages over to llsdhttp isn't really intersting enough Donovan Linden: what would be much more interesting is breaking up the existing message protocol into pieces that can more easily be used by clients other than the sl client Donovan Linden: for example right now to send messages to and get messages from a region you have to go through the login process and then use the message system to negotiate a sessio with the region Donovan Linden: which includes deciding where to stand, and what your avatar shape looks like Rex Cronon: u could just provide a protocol and anybody that wanted to make their own client would just have to use your protocol Donovan Linden: right now the viewer has to tell the server what "shape" the avatar is before the server will give the viewer any object data Donovan Linden: rex, we already do that Donovan Linden: we have open sourced the viewer so anybody can implement the protocol Donovan Linden: in fact libsl did so before we open sourced the viewer Rex Cronon: but right now u r in a transition phase Donovan Linden: but what I am talking about is making the existing api less cumbersome to use Donovan Linden: let's imagine a web browser for example Rex Cronon: easier to use. that doesn't sound bad:) Donovan Linden: can we brainstorm what tasks we would like to do in a browser easily? Donovan Linden: anyone have any ideas? Rex Cronon: download pages and images/scripts/files from those pages Donovan Linden: define a page Rex Cronon: a text file containing html/javascript code Donovan Linden: that's not an api though, that would be the equivalent of linden lab developing our own web client for 6 months and then saying "here it is!" Donovan Linden: what we'd like to do is define a minimal api so others can design web applications in addition to linden Jarod Godel: inventory Donovan Linden: so the obvious first step is login, which gets you your seed capability to your agent domain Zha Ewry nods "In fact, there are two bits there, Donovan, on the wire, which isn't so much and api as a spec" and a library, which is an API Zha Ewry: it gets very confusing to mix the two up" Jarod Godel: with ML you could create a drag'n'drop AJAX app for giving stuff to people Donovan Linden: zha: message liberation takes care of the wire, that's llsd over http Wyn Galbraith: You could then maybe setup other ways to message into SL from without SL without logging in using other languages. Zha Ewry: Half of the on the wire. It tells you how to endode, not what to encode Donovan Linden: right, so that is what I would like to talk about today Jarod Godel: inventory and IM first Zha Ewry: just like http tells you mime type, etc. but there is more that you need to actually get useful stuff done Donovan Linden: what would people like us to encode in an easy to use api first? Jarod Godel: but let me add: make that the first part of ONE apI, NOT THE FIRST API OF MANY Jarod Godel: sorry. didn't mean caps Donovan Linden: so: from the agent domain seed capability you can ask for the root folder inventory capability Rex Cronon: log in/out, object creation/editing, saying and listening to text chat Jarod Godel: right, and have to dump a list of folders so I could, say, view it in a Treelist made from VB.NET or JavaScipt. Donovan Linden: log in/out is weird though, because you're talking about login in the sense of going in-world Jarod Godel: yeah, but wouldn't session states handle that? Donovan Linden: there's a lower level use case which is more like "authenticate me for some services, even though my avatar does not appear in world" Zha Ewry: Well.. one interesting question, is how deeply coupled is "logged in" with "in world" Zha Ewry: (What you said) Donovan Linden: I don't think one's avatar should have to be in-world for an application to examine the avatar's inventory Zha Ewry: And, in fact I'd like it to be possible to be both in-world and use a low level API for some stuff Jarod Godel: Like Amazon's S3 service, we'd fire off a login hash for every request? Zha Ewry: so, I casn get eclipse to upload scripts while I'm logged in Donovan Linden: ok, so once the viewer gets back the root inventory folder capability, it invokes it with a GET, producing the root-level listing of folders/objects Wyn Galbraith: Or Java. What if you were on a system that didn't support the graphic end of SL but you needed to message someone, or check your inventory, or even clean up your inventory, recover a notecard, snapshot, etc. Jarod Godel: Donovan, right. like, to re-use an example, Amazon's service. Jarod Godel: (or Box.net) Donovan Linden: looking like this {'library': 'http://capability-to-library-subdirectory', 'some object': 'http://cap-to-object-data} Rex Cronon: is the above line code? Jarod Godel: JSON, eh? Not XML? Donovan Linden: so I think that pretty much covers inventory, to create you just PUT to a new url with a body indicating either object data or folder metadata Khamon Fate: I don't understand the statement: log in/out is weird though, because you're talking about login in the sense of going in-world. We log into web services now that use the same procedure don't we? Donovan Linden: it's not json, it's llsd which is xml Donovan Linden: but I often use json notation (python object notation actually) Rex Cronon: not familiar with python:( Jarod Godel: I speal JSON with a Perl Data::Dumper accent Donovan Linden: all of these apis may also be available as pure json in addition to llsd Donovan Linden: but whether that happens is up to me writing the code Zha Ewry: Well, right now, login pretty much implies in-world Khamon Fate: Jarod, if Microsoft can randomly rename components, so can Icehouse. Zha Ewry: Which I think we all know is wrong Jarod Godel: that'd be cool, as JSON is much more bandwidth efficient Donovan Linden: khamon: what do you mean? Jarod Godel: Zha, they'd have to have some kind of API authentication, but it would probably beb stateless. Khamon Fate: I'm just telling Jarod to let you call it whatever you please. Jarod Godel: Maybe like this: http://developer.yahoo.com/auth/ Khamon Fate: Or do you mean my question? I'm asking what you meant be saying " log in/out is weird though, because you're talking about login in the sense of going in-world" Zha Ewry nods "The point being, that you need to seperate the notion of authenticating, from going in world" Khamon Fate: in relation to web services built on the API Khamon Fate: using the API Donovan Linden: khamon: I was asking which "it" you were talking about when you were talking to jarod Donovan Linden: I just got confused Jarod Godel: He was poking fun at your calling the JSON notation llsd. Khamon Fate: or poking fun at you for calling it JSON Khamon Fate: ha ha ha Jarod Godel: We have a running joke about Microsoft renaming "domain" to something different than the DNS definition. Donovan Linden: ah. well, I actually use json notation to produce llsd internally, so it is accurate Zha Ewry: Mind you it gets really interesting when you have several parallel request streams running into the server cluster. A normal client, a web client, and maybe a mobile phone client all trying to manipulate the state of the single agent. Khamon Fate: They've redfined loads of terminology over the years Donovan Linden: the python libraries I use to do all this stuff will be open sourced, hopefully in a month or two Donovan Linden: the gating factor is me writing documentation and more tests Jarod Godel: The "backbone" stuff? Donovan Linden: zha: that's fine, since we're modelling all operations as rest Jarod Godel: oh. CRUD. ;-) Donovan Linden: all the different clients will have the same concurrency interactions as other rest architectures like the web Khamon Fate: So Donovan will LLSD require ACKs and NAKs to communicate parameters to the clients Zha Ewry nods "It's fine, but expect people to be surprised." Khamon Fate: but leave texture data and such in a udp streaming mode Donovan Linden: khamon: http is a request/response protocol, so when you invoke the capability in your client, you are guaranteed to get a response Zha Ewry: Well, or a timeout Donovan Linden: in the python libraries we use in icehouse, indra.llsdhttp, that manifests as either getting a result or an exception Khamon Fate: Oh sorry, I sliped into client mode, y'all are talking about web services. Sorry Donovan Linden: so: Donovan Linden: try: Donovan Linden: root_listing = llsdhttp.get(my_inventory_cap) Donovan Linden: except llsdhttp.ConnectionError: Donovan Linden: print "second life is down again? nooooo!!" Khamon Fate: Weren't we talking about this at Benjamin's office hours Tuesday? Khamon Fate: Is he Icehouse too? Jarod Godel: Khamon, Benjamin's handling the what, Donovan's discussing the how. I think. Donovan Linden: khamon: I speak only for myself, if benjamin says similar things it's coincidental :) Donovan Linden: can I get a link to the transcript? Rex Cronon: what is this supposed to return " root_listing = llsdhttp.get(my_inventory_cap)" Khamon Fate: http://wiki.secondlife.com/wiki/User:Benjamin_Linden Zha Ewry: A capabitlty, I assume Donovan Linden: rex: the json notatino folder listing I typed above Jarod Godel: Benjamin's been discussing what we'll do once we have ML: browser plug-ins, cellphone chatting, etc. That's why I wanted to discuss ML, to hear about how we're getting from here to the "there" benjamin is talking about. Donovan Linden: json notation Rex Cronon: what are the types that a capability can be? Donovan Linden: a capability is just an unguessable url a client has that grants access to an internal service Jarod Godel: http://wiki.secondlife.com/wiki/Capabilities Donovan Linden: in this case, the service was the "list my root inventory folder" service Zha Ewry: Well, one "could" have a strongly typed world, in which when one got a capabitlity, it was mmp like, and not only covered the secure access issues, but the object/type behaviro as well. So one could, *only* send an approproiate set of requests to it. Donovan Linden: invoking it with a GET gave back an LLSD map of names of things in that directory to capabilities to those things Jarod Godel: Which would look like: {"root": ["Scripts", "Textures", "etc."]} Zha Ewry: Of course, one might end up redesignign the intel 432, which wouldn't make anyone happy Donovan Linden: zha: you will be able to introspect it to discover it's type and signature Donovan Linden: and there will be autogenerated api documentation, generated by a crawler introspecting the entire tree of a running system Jarod Godel: wsdl? Donovan Linden: no, probably just html documentation Jarod Godel: ok Donovan Linden: the format of the document returned from the introspection api remains to be decided Donovan Linden: I have a simple proof-of-concept that just uses pure llsd Zha Ewry: Jarod, not to be snarky, but would you actually imagine using wasdl, in this context, in a useful way? I mean.. I can see it being easy enough to generate, but a) it isn't documentation for huumans and b) it would be opaque beyond words Rex Cronon: that is a very important part "the format" Jarod Godel: So, ML is (effectively?) going to break SL into two parts: data and real-time. Data will be things we grab via web services, and real-time will be udp streaming "in-world"? Donovan Linden: zha: if it is used at all it should be autogenerated from whatever simple format is used Jarod Godel: Zha, oh gosh no. I hate wsdl. I was just checking. Donovan Linden: jarod: the day of liberation is not going to do that Donovan Linden: but it will enable us to build services which do that Jarod Godel: ah, coolio Donovan Linden: but yes, all of the data available over udp now will be available over llsdhttp Donovan Linden: it just will be very inconvenient to use, because the message system semantics are asynchronous Donovan Linden: how many people here know about the existing message system semantics? Jarod Godel: That's going to be so cool, because then we can start pulling SL down to our desktop. IM, exchange rates, etc. Zha Ewry reluctantly raises her hand "Pretty much" Khamon Fate: we know about asynchronicity provided by the data servers Donovan Linden: zha: are logical request/response pairs of messages grouped together in any way? Khamon Fate: facilitated by the dataservers i mean Donovan Linden: like is it always a field called "transaction id" Donovan Linden: khamon: the sims talk to the dataserver asynchronously, the dataservers talk to the database synchronously in a single thread Zha Ewry thinks there is some of that, but is digging through old transcripts now. IM: raillere Allen: . Donovan Linden: which is why they are being removed and replaced with the sims talking to web services using non-blocking apis Ryan Linden is Online Khamon Fate: what's a non-blocking api? Zha Ewry: And you are rest enough that it is all giong to be idempotent by capability? Donovan Linden: zha: yes, the services you invoke through the caps should be fully rest with idempotency guarantees Donovan Linden: at least the services designed by icehouse will be Spike Linden is Online Donovan Linden: there are already services internally that are not properly rest Zha Ewry: Which lets you get non-blocking, and safe at the same time Rex Cronon: u ask for somthing than u can go do something else, while u have a listener that waits for waht u asked Donovan Linden: khamon: a non-blocking networking api is an api where doing network io does not block other useful work from happening Donovan Linden: yep Ethan Linden is Online Donovan Linden: yep rex, I mean Khamon Fate: i see that in the discussion thanks Zha Ewry: And.. you pretty much want idempotent, and fail, so you can re-issue if lost, and detect state changes that happen between your getting the capabitity and using it. Khamon Fate: so that'll work both ways Jarod Godel: No, non-blocking is essentially an AJAX friendly api (to put it in terms my, poor, webdev brain can grok)? Khamon Fate: yes it'll work sim to database and sim to client and sim to web Donovan Linden: jarod: all io in javascript is continuation-passing style which is one way of doing non-blocking io, yes Jarod Godel: woot Donovan Linden: in the c++ simulator and viewer all io is continuation-passing style non-blocking io as well, but at a much lower level Rex Cronon: but u can have a problem if u have too many listeners Donovan Linden: rex: so that would segue into another design discussion about cache design Zha Ewry: Donovan, when you say there are some REST broken services already, what part is wrong? (Not idempotent?) Jarod Godel: and refresh buttons Khamon Fate: the apis will be processed through web servers talking to the sims right? or will the sims themselves be processing pages and feeding them directly to the browsers? Khamon Fate: ha ha ha Jarod said "refresh" Donovan Linden: zha: a common mistake for newbie developers is to make urls contain verbs, and use POST for every resource Zha Ewry winces. "I assume somone will educate them." Donovan Linden: khamon: that's right, the capability servers the viewers hit will talk to the appropriate internal service Donovan Linden: whether it's an internal cache, another internal web service, or the sim itself running a web service Donovan Linden: zha: because of the linden culture it's all through osmosis. yes, we should have another linden u on REST Khamon Fate: so the sims themselves will be offering web services Donovan Linden: khamon: they already are, but only internal tools are using them Donovan Linden: ok, I think that pretty much wraps it up, great discussion! Zha Ewry: I might politely suggest that any publicly exposed rest services be scrubbed of such stuff, before letting people see them ;-) Donovan Linden: zha: if I can help it, yes :) Rex Cronon: i have a quick question, why is there no trascript for may 15? Zha Ewry: Wonderful stuff, Donovan, thanks for the time, and effort Donovan Linden: I'm sure there will always be cases of slightly funky apis needing refactoring Echo Seigo: thanks Donovan, good stuffs Donovan Linden: rex: who hosted may 15? Rex Cronon: i don't know Donovan Linden: I can either pester them for a transcript or if someone here has it, send it to me Wyn Galbraith: Thanks for the meeting Donovan, Linden hours are always an education :) Rex Cronon: ian linden Jarod Godel: Thanks, Donovan. Zha Ewry nods "Sure, but getting the verb in the wrong place, is likely to get people giggling at you." Zha Ewry: One tries to avoid that. Mia Linden is Online Jarod Godel: It was: Ian, Which, (someonme, I mssed it), and now Donovan. }}}