User:Zero Linden/Office Hours/2007 May 24

Jarod Godel: Are you our guest pris^H^H^H^Hhost?
Donovan Linden: Hello, I am your guest, erm, host
Rex Cronon: hello everybody
Jarod Godel: cool
Donovan Linden: Hi rex
Rex Cronon: for some reason i can't detach my gun. so don't be intimidated by me. although i came bearing arms, i don't intend to use them:)
Donovan Linden: hehe
Catherine Linden is Online
Donovan Linden: it will probably just take a while, they shipped derezloss, a branch which makes moving items back into inventor safer
Rex Cronon: i guess to be safer takes longer
Donovan Linden: instead of removing the item from the simulator immediately, it waits until the inventory database says it actually saved the object
Donovan Linden: we should have a "derezzing" effect, like in tron, then if it takes a long time people will just think it is cool :)
Donovan Linden: So how long should I wait for more people to arrive?
Rex Cronon: usually they are here around this time +/- 10 minutes
Donovan Linden: I'll jus wait 10 minutes
Donovan Linden: Any votes on what to talk about? I was thinking I could talk about how message-liberation and capabilities
Donovan Linden: could allow us to provide web services easily
Rex Cronon: how about sculpties?
Donovan Linden: I don't know much about sculpties
Donovan Linden: what do you know about them? :)
Rex Cronon: why were sculpties used when instead a .obj file format would have been a lot better?
Jarod Godel: yes
Donovan Linden: hmm. well, can you guess?
Donovan Linden: I have a guess
Jarod Godel: Message liberation and how that relates to the modularization of second life protocols.
Rex Cronon: ok i am listening
Donovan Linden: jarod: ok, good, that's what I'll talk about
Donovan Linden: rex: ok, I think sculpties were implemented the way they were for reasons of efficiency on many levels
Jarod Godel: Rex, I think scupties are preferred because it's still one prim. OBJ files are, I think, multiple primitives. Could be wrong, though.
Donovan Linden: one, you can't store that much vector data in a texture, so that limits the complexity of objects nicely
Donovan Linden: and two, since it's a texture, all of the code for downloading and parsing it was already written
Donovan Linden: and any other things that we can already do with textures can more easily be applied to them
Donovan Linden: for example, we could perhaps animate a sculptie texture, or programatically generate one
Donovan Linden: because libraries are already available in the client to do so
Rex Cronon: ok here is what i think. obj files are text and easier to edit and compress, u can load only objects that have a specific number of vertices, and in a .obj file u can have holes inside objects
Donovan Linden: I think sculpties are a great example of how to ship a feature, iteratively
Donovan Linden: rex: the client is open source, someone could easily write an .obj to prim/sculptie importer
Rex Cronon: oh and an object file can contain only 1 object, doesn't have to contain multiple groups(group is set of vertices and faces)
Jarod Godel: ok
Donovan Linden: anyway, I think the answer really is, it was easy to do and get right
Donovan Linden: we could write a server-side .obj to prim/sculptie converter some day and expose it as a web service, perhaps
Jarod Godel: Message liberation...
Donovan Linden: or someone else can do it
Donovan Linden: ok, let's get started
Jarod Godel: Benjamin keeps talking about building a web browser plun-in for SL.
Rex Cronon: ok lets go to message liberation:)
Donovan Linden: how many people know what message liberation is already
Rex Cronon: hi wyn
Jarod Godel: Does that relagte to M.L.?
Donovan Linden: quick status update on the branch, it is going into qa ASAP but the "surprise" release backed up qa a bit
Wyn Galbraith *yawns without coffee* "Good morning, Rex, Jarod, and Greetings Donovan.
Jarod Godel: Hi
Donovan Linden: jarod: I will talk about how that related to m-l today :)
Jarod Godel: woot!
Donovan Linden: the "surprise" release was a decision to do two smaller releases rather than wait for m-l again
Rex Cronon: m.l = message liberation?
Donovan Linden: yes, rex
Donovan Linden: but in our informal testing, m-l seems to be very free of bugs
Emily Brissot is Offline
Donovan Linden: we are on branch message-liberation-8 now, and the last one that shipped into release was m-l-4
Donovan Linden: so it hasn't seen real production use in a while, which always exposes bugs
Jarod Godel: fracturing protocol, one release at a time
Khamon Fate: hey y'all people
Donovan Linden: but we may have shaken most of the bad bugs out with m-l-4, so hopefully the Day of Liberation goes smoothly
Khamon Fate: when is the Day of Liberation scheduled?
Wyn Galbraith murmurs goodmorning to Khamon.
Donovan Linden: still not scheduled, but it should be soon now
Jarod Godel: Are you planning on exposing messages to us soon, aside from opening the client?
Khamon Fate IMs answers to Wyn
Jarod Godel: err, aside from the open client
Donovan Linden: ok, and now on to how m-l benefits residents
Donovan Linden: initially m-l will only be used to allow us to fine-tune our internet usage
Wyn Galbraith: Morning Zha.
Donovan Linden: by moving messages over llsdhttp we can ship data to the viewer in a way that is more efficient and plays more nicely with internet providers
Zha Ewry: Morning, Wyn
Rex Cronon: could there a page on wiki that lists all the benefits that ml will bring?
Zha Ewry: all
Donovan Linden: so residents will see this as a modest increase in bandwidth utilization
Rex Cronon: hi zha
Donovan Linden: Rex: Good idea I'll make tasks for icehouse to do that
Donovan Linden: but just switching old style asynchronous messages over to llsdhttp isn't really intersting enough
Donovan Linden: what would be much more interesting is breaking up the existing message protocol into pieces that can more easily be used by clients other than the sl client
Donovan Linden: for example right now to send messages to and get messages from a region you have to go through the login process and then use the message system to negotiate a sessio with the region
Donovan Linden: which includes deciding where to stand, and what your avatar shape looks like
Rex Cronon: u could just provide a protocol and anybody that wanted to make their own client would just have to use your protocol
Donovan Linden: right now the viewer has to tell the server what "shape" the avatar is before the server will give the viewer any object data
Donovan Linden: rex, we already do that
Donovan Linden: we have open sourced the viewer so anybody can implement the protocol
Donovan Linden: in fact libsl did so before we open sourced the viewer
Rex Cronon: but right now u r in a transition phase
Donovan Linden: but what I am talking about is making the existing api less cumbersome to use
Donovan Linden: let's imagine a web browser for example
Rex Cronon: easier to use. that doesn't sound bad:)
Donovan Linden: can we brainstorm what tasks we would like to do in a browser easily?
Donovan Linden: anyone have any ideas?
Rex Cronon: download pages and images/scripts/files from those pages
Donovan Linden: define a page
Rex Cronon: a text file containing html/javascript code
Donovan Linden: that's not an api though, that would be the equivalent of linden lab developing our own web client for 6 months and then saying "here it is!"
Donovan Linden: what we'd like to do is define a minimal api so others can design web applications in addition to linden
Jarod Godel: inventory
Donovan Linden: so the obvious first step is login, which gets you your seed capability to your agent domain
Zha Ewry nods "In fact, there are two bits there, Donovan, on the wire, which isn't so much and api as a spec" and a library, which is an API
Zha Ewry: it gets very confusing to mix the two up"
Jarod Godel: with ML you could create a drag'n'drop AJAX app for giving stuff to people
Donovan Linden: zha: message liberation takes care of the wire, that's llsd over http
Wyn Galbraith: You could then maybe setup other ways to message into SL from without SL without logging in using other languages.
Zha Ewry: Half of the on the wire. It tells you how to endode, not what to encode
Donovan Linden: right, so that is what I would like to talk about today
Jarod Godel: inventory and IM first
Zha Ewry: just like http tells you mime type, etc. but there is more that you need to actually get useful stuff done
Donovan Linden: what would people like us to encode in an easy to use api first?
Jarod Godel: but let me add: make that the first part of ONE apI, NOT THE FIRST API OF MANY
Jarod Godel: sorry. didn't mean caps
Donovan Linden: so: from the agent domain seed capability you can ask for the root folder inventory capability
Rex Cronon: log in/out, object creation/editing, saying and listening to text chat
Jarod Godel: right, and have to dump a list of folders so I could, say, view it in a Treelist made from VB.NET or JavaScipt.
Donovan Linden: log in/out is weird though, because you're talking about login in the sense of going in-world
Jarod Godel: yeah, but wouldn't session states handle that?
Donovan Linden: there's a lower level use case which is more like "authenticate me for some services, even though my avatar does not appear in world"
Zha Ewry: Well.. one interesting question, is how deeply coupled is "logged in" with "in world"
Zha Ewry: (What you said)
Donovan Linden: I don't think one's avatar should have to be in-world for an application to examine the avatar's inventory
Zha Ewry: And, in fact I'd like it to be possible to be both in-world and use a low level API for some stuff
Jarod Godel: Like Amazon's S3 service, we'd fire off a login hash for every request?
Zha Ewry: so, I casn get eclipse to upload scripts while I'm logged in
Donovan Linden: ok, so once the viewer gets back the root inventory folder capability, it invokes it with a GET, producing the root-level listing of folders/objects
Wyn Galbraith: Or Java. What if you were on a system that didn't support the graphic end of SL but you needed to message someone, or check your inventory, or even clean up your inventory, recover a notecard, snapshot, etc.
Jarod Godel: Donovan, right. like, to re-use an example, Amazon's service.
Jarod Godel: (or Box.net)
Donovan Linden: looking like this {'library': 'http://capability-to-library-subdirectory', 'some object': 'http://cap-to-object-data}
Rex Cronon: is the above line code?
Jarod Godel: JSON, eh? Not XML?
Donovan Linden: so I think that pretty much covers inventory, to create you just PUT to a new url with a body indicating either object data or folder metadata
Khamon Fate: I don't understand the statement: log in/out is weird though, because you're talking about login in the sense of going in-world. We log into web services now that use the same procedure don't we?
Donovan Linden: it's not json, it's llsd which is xml
Donovan Linden: but I often use json notation (python object notation actually)
Rex Cronon: not familiar with python:(
Jarod Godel: I speal JSON with a Perl Data::Dumper accent
Donovan Linden: all of these apis may also be available as pure json in addition to llsd
Donovan Linden: but whether that happens is up to me writing the code
Zha Ewry: Well, right now, login pretty much implies in-world
Khamon Fate: Jarod, if Microsoft can randomly rename components, so can Icehouse.
Zha Ewry: Which I think we all know is wrong
Jarod Godel: that'd be cool, as JSON is much more bandwidth efficient
Donovan Linden: khamon: what do you mean?
Jarod Godel: Zha, they'd have to have some kind of API authentication, but it would probably beb stateless.
Khamon Fate: I'm just telling Jarod to let you call it whatever you please.
Jarod Godel: Maybe like this: http://developer.yahoo.com/auth/
Khamon Fate: Or do you mean my question? I'm asking what you meant be saying " log in/out is weird though, because you're talking about login in the sense of going in-world"
Zha Ewry nods "The point being, that you need to seperate the notion of authenticating, from going in world"
Khamon Fate: in relation to web services built on the API
Khamon Fate: using the API
Donovan Linden: khamon: I was asking which "it" you were talking about when you were talking to jarod
Donovan Linden: I just got confused
Jarod Godel: He was poking fun at your calling the JSON notation llsd.
Khamon Fate: or poking fun at you for calling it JSON
Khamon Fate: ha ha ha
Jarod Godel: We have a running joke about Microsoft renaming "domain" to something different than the DNS definition.
Donovan Linden: ah. well, I actually use json notation to produce llsd internally, so it is accurate
Zha Ewry: Mind you it gets really interesting when you have several parallel request streams running into the server cluster. A normal client, a web client, and maybe a mobile phone client all trying to manipulate the state of the single agent.
Khamon Fate: They've redfined loads of terminology over the years
Donovan Linden: the python libraries I use to do all this stuff will be open sourced, hopefully in a month or two
Donovan Linden: the gating factor is me writing documentation and more tests
Jarod Godel: The "backbone" stuff?
Donovan Linden: zha: that's fine, since we're modelling all operations as rest
Jarod Godel: oh. CRUD. ;-)
Donovan Linden: all the different clients will have the same concurrency interactions as other rest architectures like the web
Khamon Fate: So Donovan will LLSD require ACKs and NAKs to communicate parameters to the clients
Zha Ewry nods "It's fine, but expect people to be surprised."
Khamon Fate: but leave texture data and such in a udp streaming mode
Donovan Linden: khamon: http is a request/response protocol, so when you invoke the capability in your client, you are guaranteed to get a response
Zha Ewry: Well, or a timeout
Donovan Linden: in the python libraries we use in icehouse, indra.llsdhttp, that manifests as either getting a result or an exception
Khamon Fate: Oh sorry, I sliped into client mode, y'all are talking about web services. Sorry
Donovan Linden: so:
Donovan Linden: try:
Donovan Linden:    root_listing = llsdhttp.get(my_inventory_cap)
Donovan Linden: except llsdhttp.ConnectionError:
Donovan Linden:    print "second life is down again? nooooo!!"
Khamon Fate: Weren't we talking about this at Benjamin's office hours Tuesday?
Khamon Fate: Is he Icehouse too?
Jarod Godel: Khamon, Benjamin's handling the what, Donovan's discussing the how. I think.
Donovan Linden: khamon: I speak only for myself, if benjamin says similar things it's coincidental :)
Donovan Linden: can I get a link to the transcript?
Rex Cronon: what is this supposed to return " root_listing = llsdhttp.get(my_inventory_cap)"
Khamon Fate: http://wiki.secondlife.com/wiki/User:Benjamin_Linden
Zha Ewry: A capabitlty, I assume
Donovan Linden: rex: the json notatino folder listing I typed above
Jarod Godel: Benjamin's been discussing what we'll do once we have ML: browser plug-ins, cellphone chatting, etc. That's why I wanted to discuss ML, to hear about how we're getting from here to the "there" benjamin is talking about.
Donovan Linden: json notation
Rex Cronon: what are the types that a capability can be?
Donovan Linden: a capability is just an unguessable url a client has that grants access to an internal service
Jarod Godel: http://wiki.secondlife.com/wiki/Capabilities
Donovan Linden: in this case, the service was the "list my root inventory folder" service
Zha Ewry: Well, one "could" have a strongly typed world, in which when one got a capabitlity, it was mmp like, and not only covered the secure access issues, but the object/type behaviro as well. So one could, *only* send an approproiate set of requests to it.
Donovan Linden: invoking it with a GET gave back an LLSD map of names of things in that directory to capabilities to those things
Jarod Godel: Which would look like: {"root": ["Scripts", "Textures", "etc."]}
Zha Ewry: Of course, one might end up redesignign the intel 432, which wouldn't make anyone happy
Donovan Linden: zha: you will be able to introspect it to discover it's type and signature
Donovan Linden: and there will be autogenerated api documentation, generated by a crawler introspecting the entire tree of a running system
Jarod Godel: wsdl?
Donovan Linden: no, probably just html documentation
Jarod Godel: ok
Donovan Linden: the format of the document returned from the introspection api remains to be decided
Donovan Linden: I have a simple proof-of-concept that just uses pure llsd
Zha Ewry: Jarod, not to be snarky, but would you actually imagine using wasdl, in this context, in a useful way? I mean.. I can see it being easy enough to generate, but a) it isn't documentation for huumans and b) it would be opaque beyond words
Rex Cronon: that is a very important part "the format"
Jarod Godel: So, ML is (effectively?) going to break SL into two parts: data and real-time. Data will be things we grab via web services, and real-time will be udp streaming "in-world"?
Donovan Linden: zha: if it is used at all it should be autogenerated from whatever simple format is used
Jarod Godel: Zha, oh gosh no. I hate wsdl. I was just checking.
Donovan Linden: jarod: the day of liberation is not going to do that
Donovan Linden: but it will enable us to build services which do that
Jarod Godel: ah, coolio
Donovan Linden: but yes, all of the data available over udp now will be available over llsdhttp
Donovan Linden: it just will be very inconvenient to use, because the message system semantics are asynchronous
Donovan Linden: how many people here know about the existing message system semantics?
Jarod Godel: That's going to be so cool, because then we can start pulling SL down to our desktop. IM, exchange rates, etc.
Zha Ewry reluctantly raises her hand "Pretty much"
Khamon Fate: we know about asynchronicity provided by the data servers
Donovan Linden: zha: are logical request/response pairs of messages grouped together in any way?
Khamon Fate: facilitated by the dataservers i mean
Donovan Linden: like is it always a field called "transaction id"
Donovan Linden: khamon: the sims talk to the dataserver asynchronously, the dataservers talk to the database synchronously in a single thread
Zha Ewry thinks there is some of that, but is digging through old transcripts now.
IM: raillere Allen: .
Donovan Linden: which is why they are being removed and replaced with the sims talking to web services using non-blocking apis
Ryan Linden is Online
Khamon Fate: what's a non-blocking api?
Zha Ewry: And you are rest enough that it is all giong to be idempotent by capability?
Donovan Linden: zha: yes, the services you invoke through the caps should be fully rest with idempotency guarantees
Donovan Linden: at least the services designed by icehouse will be
Spike Linden is Online
Donovan Linden: there are already services internally that are not properly rest
Zha Ewry: Which lets you get non-blocking, and safe at the same time
Rex Cronon: u ask for somthing than u can go do something else, while u have a listener that waits for waht u asked
Donovan Linden: khamon: a non-blocking networking api is an api where doing network io does not block other useful work from happening
Donovan Linden: yep
Ethan Linden is Online
Donovan Linden: yep rex, I mean
Khamon Fate: i see that in the discussion thanks
Zha Ewry: And.. you pretty much want idempotent, and fail, so you can re-issue if lost, and detect state changes that happen between your getting the capabitity and using it.
Khamon Fate: so that'll work both ways
Jarod Godel: No, non-blocking is essentially an AJAX friendly api (to put it in terms my, poor, webdev brain can grok)?
Khamon Fate: yes it'll work sim to database and sim to client and sim to web
Donovan Linden: jarod: all io in javascript is continuation-passing style which is one way of doing non-blocking io, yes
Jarod Godel: woot
Donovan Linden: in the c++ simulator and viewer all io is continuation-passing style non-blocking io as well, but at a much lower level
Rex Cronon: but u can have a problem if u have too many listeners
Donovan Linden: rex: so that would segue into another design discussion about cache design
Zha Ewry: Donovan, when you say there are some REST broken services already, what part is wrong? (Not idempotent?)
Jarod Godel: and refresh buttons
Khamon Fate: the apis will be processed through web servers talking to the sims right? or will the sims themselves be processing pages and feeding them directly to the browsers?
Khamon Fate: ha ha ha Jarod said "refresh"
Donovan Linden: zha: a common mistake for newbie developers is to make urls contain verbs, and use POST for every resource
Zha Ewry winces. "I assume somone will educate them."
Donovan Linden: khamon: that's right, the capability servers the viewers hit will talk to the appropriate internal service
Donovan Linden: whether it's an internal cache, another internal web service, or the sim itself running a web service
Donovan Linden: zha: because of the linden culture it's all through osmosis. yes, we should have another linden u on REST
Khamon Fate: so the sims themselves will be offering web services
Donovan Linden: khamon: they already are, but only internal tools are using them
Donovan Linden: ok, I think that pretty much wraps it up, great discussion!
Zha Ewry: I might politely suggest that any publicly exposed rest services be scrubbed of such stuff, before letting people see them ;-)
Donovan Linden: zha: if I can help it, yes :)
Rex Cronon: i have a quick question, why is there no trascript for may 15?
Zha Ewry: Wonderful stuff, Donovan, thanks for the time, and effort
Donovan Linden: I'm sure there will always be cases of slightly funky apis needing refactoring
Echo Seigo: thanks Donovan, good stuffs
Donovan Linden: rex: who hosted may 15?
Rex Cronon: i don't know
Donovan Linden: I can either pester them for a transcript or if someone here has it, send it to me
Wyn Galbraith: Thanks for the meeting Donovan, Linden hours are always an education :)
Rex Cronon: ian linden
Jarod Godel: Thanks, Donovan.
Zha Ewry nods "Sure, but getting the verb in the wrong place, is likely to get people giggling at you."
Zha Ewry: One tries to avoid that.
Mia Linden is Online
Jarod Godel: It was: Ian, Which, (someonme, I mssed it), and now Donovan.