AW Groupies/Chat Logs/AWGroupies-2008-05-20

[9:30] Latha Serevi: Hey, what's the format for today? any voice/audio/video? thx.
[9:30] Zha Ewry: Text, only, as usual, transcript at the end
[9:30] Day Oh: lol hi Zha (:
[9:31] Zha Ewry: Ok
[9:31] Zha Ewry: Plenty of room for more aves, but I may have to rez more chairs
[9:31] SignpostMarv Martin: applications for SL's 5th Birthday end on the 25th of May, though I know that may be a bit short-notice to make adecision as to what would go in it :-P
[9:31] Zha Ewry: Donovan, the little beanbag facinfg us is yours to grab when you're ready ;-)
[9:32] Donovan Linden: which one?
[9:32] Saijanai Kuhn: we need to see about getting a web media board here, Zha
[9:32] Zha Ewry: That one you're on
[9:33] Donovan Linden: ok :)
[9:33] Zha Ewry: Its set up so you face us ;-)
[9:35] Zha Ewry: My, look at this crowd
[9:35] Hamilton Linden: Hi
[9:35] Rex Cronon: hello everybody
[9:35] Zero Linden: hey all!
[9:35] Zha Ewry: /There are some more beanbag over near the Ruth picture
[9:35] Donovan Linden: thanks for coming everybody
[9:35] Saijanai Kuhn: 32 people
[9:35] JayR Cela: heyyy there Rx :_)
[9:36] Jrift Sol: hello to everyone. hey rex!
[9:36] Zha Ewry: Welcome everyone, especially newcomers
[9:36] Tao Takashi: Hi1
[9:36] Tao Takashi: !
[9:36] Rex Cronon: hi
[9:36] Zha Ewry: Donovan Linden's got the floor, for some of the reverse http discussion
[9:36] Saijanai Kuhn: Told you I'd spam advertise.
[9:36] Tao Takashi: indeed :)
[9:36] Tao Takashi: thanks Sai! :)
[9:36] Davina Glitter: Hello Don and thank you for having the session.
[9:36] Tao Takashi: Hi Donovan! :)
[9:37] Donovan Linden: Ok. So just for some background, the SL architecture, when it was pure UDP, allowed for sending messages to the client that the server initiates, without the client making a request
[9:37] Zha Ewry: I think there are enough benbags in the outer row. I bet some are facing the wrong way. if so, sing out and I'll turn you areuond
[9:37] Gareth Ellison: wow, i got here this time
[9:37] Tao Takashi: and Hi Donovan! :)
[9:37] Anaid Daviau: Turn me please!
[9:38] Donovan Linden: hi tao :)
[9:38] Tao Takashi: oh, I said that already ;-)
[9:38] Tao Takashi: will try not to repeat himself too much today ;-)
[9:38] Donovan Linden: When we did the Message Liberation project which allows arbitrary messages to be moved over tcp instead of udp, we needed the same semantics
[9:38] Zero Linden: oh - these are the new beanbags - shift-arrows change your pose!
[9:39] Zha Ewry: smiles
[9:39] Donovan Linden: We also chose to use HTTP as the transport because we like http
[9:39] Zha Ewry: BamBam's excellent work :-)
[9:39] Donovan Linden: HTTP is a request response protocol; the server can only send data to a client in response to a request from the client
[9:39] Gareth Ellison: when you say arbitary does that mean anything from the message template can be sent over HTTP?
[9:40] Saijanai Kuhn: LOL We're full, Zha
[9:40] Donovan Linden: So, we used a technique called "The long poll" where the client is always making a request to the server, but the server does not respond until there is an event to send to the client
[9:40] Gareth Ellison: coughes and points at streamlet
[9:40] Donovan Linden: As soon as the client gets a response from the server, the client makes a new request for the next event
[9:41] Donovan Linden: This is a common COMET technique, and works quite well. It has been in use in the current SL client for almost a year now.
[9:42] Donovan Linden: however, this technique has some undesirable latency issues that are unavoidable in normal http
[9:42] Zha Ewry: / Newp Agent limit here is 100
[9:42] Donovan Linden: the server cannot start sending a new request until the client has received the old response and made a new request
[9:43] Donovan Linden: [1]
[9:43] Xugu Madison: I take it that opening multiple connections from the client to the server wouldn't really be a solution because of the amount of resources (sockets) being tied up?
[9:43] Donovan Linden: The comet long poll technique is great because it requires only a normal HTTP client which should be available in almost any programming environment
[9:43] Gareth Ellison: breaks the HTTP RFC too - which i think we agreed was a bad thing(TM)
[9:44] Zha Ewry: more and 2 connectoins
[9:44] Gareth Ellison: many connections
[9:44] Zha Ewry: / Also violates,http 1.1,
[9:44] Gareth Ellison: >2
[9:44] Donovan Linden: Gareth: What breaks the RFC?
[9:44] Donovan Linden: It does not.
[9:44] Zha Ewry: 1.1?
[9:44] Donovan Linden: The spec recommends not using more than 2 sockets per server.
[9:44] Gareth Ellison: a SHOULD rather than a MUST in IETF speak?
[9:44] Zha Ewry: its not a shall
[9:44] Donovan Linden: the long poll requires 1 connection, and sending client-initiated events requires another
[9:44] flopsie McArdle: hi everyone
[9:44] Zha Ewry: right
[9:44] Donovan Linden: it is not a MUST
[9:44] Gareth Ellison: yay
[9:45] Zha Ewry: We're saying that many connections is basically a bad idea
[9:45] Rex Cronon: hi
[9:45] Gareth Ellison: fires up 20 threads each with their own connection
[9:45] Zha Ewry: because that shoudl.. is largely honored
[9:45] Saijanai Kuhn: Reading list for newcomers: https://lists.secondlife.com/pipermail/sldev/2008-May/009807.html
[9:45] Zero Linden: well, browsers routinely use four
[9:45] Gareth Ellison: for i in Range(50): thread.start_new_thread(url_grabber,()) :P
[9:45] Donovan Linden: but still, the technique is possible even on web browsers which restrict keepalive connections to 2
[9:46] SignpostMarv Martin: Donovan : that the pipelining that Firefox has ?
[9:46] SignpostMarv Martin: ^craming stuff down a single connection vs making multiple ones
[9:46] Donovan Linden: zero: browsers will use 4 non-keepalive connections
[9:46] Donovan Linden: and 2 keepalive connections
[9:47] Zero Linden: ah ... okay keep-alive connection limit, gotcha
[9:47] Donovan Linden: pipelining is different than keepalive, it allows the client to make another request in the same socket before the server has responded to the first one
[9:47] Donovan Linden: Ok! so that's the background.
[9:48] Donovan Linden: For people that don't believe comet works, gmail uses comet. google docs uses comet. meebo uses comet.
[9:48] Saijanai Kuhn: and SL uses a comet-like thing also
[9:48] Gareth Ellison: i doubt we have any heretics here donovan ;)
[9:48] Donovan Linden: But, since we have our own custom client in the Second Life viewer, we don't have to jump through as many hoops as normal web browsers do.
[9:49] Tao Takashi: everybody should use an SL client to browse the web
[9:49] Donovan Linden: If we're willing to use a custom protocol, we can set things up so the server can push messages over a tcp socket without the client requesting it
[9:49] Gareth Ellison: is morgaine here? she had an interesting thought on just using HTTP to setup the connection and then scrapping HTTP semantics once the socket is open
[9:49] Jrift Sol: that sounds like it might cause issues with firewalls though
[9:49] Saijanai Kuhn: thats why the reverse, Jrift
[9:50] Donovan Linden: Firewalls normally only notice which port is being connected to and do not enforce the wire protocol
[9:50] Donovan Linden: does anyone know of a firewall which cares about the actual bytes which go across the socket?
[9:50] Donovan Linden: Gareth: So, that's what we're proposing.
[9:50] Gareth Ellison: pointsto all deep packet inspection firewalls everywhere
[9:50] SignpostMarv Martin: the great firewall of china ? :-P
[9:50] Zha Ewry: Some of the inspecting ones could get pissy
[9:50] Python Morales: someone told me long ago that some firewalls do look at the protocol, the traffic
[9:50] Davina Glitter: Comcast
[9:51] Code Tracer: yes, content security module of the firewalls care
[9:51] Gareth Ellison: my ISP i recently found out shoves traffic through transparent proxies at times
[9:51] Gareth Ellison: they tend to do that at peak times
[9:51] Jn Uriza: SPI..stateful packet inspection firewalls care about the content
[9:51] Somatika Xiao: Comcast....
[9:51] SignpostMarv Martin: my old ISP was doing something that borked SL, and only SL
[9:51] Donovan Linden: HTTP/1.1 includes the Upgrade header which we can use to set up the connection and then scrap HTTP for... reverse HTTP!
[9:51] Tao Takashi: it depends on what level they look into the content
[9:52] Gareth Ellison: well, basically - yes, we have firewalls and transparent proxies which get annoyed at weird protocol screwups
[9:52] Donovan Linden: https://wiki.secondlife.com/wiki/Reverse_HTTP
[9:52] Donovan Linden: The Upgrade and 101 Switching Protocols semantics are part of HTTP/1.1, so anything that follows the spec should be able to handle it
[9:52] Gareth Ellison: Upgrade: PTTH/0.9
[9:52] JayR Cela: I beleave Comcast only looks at the amount of bits transversing the connection / not the actual Data
[9:52] Gareth Ellison: i love that
[9:53] Donovan Linden: Once the server agrees to Upgrade, the server would then start using the socket as if it were a client, and the client would use the socket as if it were a server
[9:53] Donovan Linden: However, we can also add a fallback for safety to this design.
[9:53] Zero Linden: / believes that most consumer level firewalls don't look at the wire protocol (though a few do to block URLs, but that is all) -- whereas firewalls sold to corp. IT often have lots of wire inspection - often to support the high-$
[9:54] Donovan Linden: We can change the Reverse HTTP spec to require that the server send the client a setup message immediately after Upgrade
[9:54] Gareth Ellison: consumer level firewalls come in a few flavours - basic filters/ACLs or proxies with censorware
[9:54] Code Tracer: yeah zero, i know couple of firewalls do content security such as check point... etc
[9:54] Gareth Ellison: hands up anyone who has attended a high school in the last decade and hasn't experienced the censorware firewalls
[9:54] Jrift Sol: i've opened a can of worms here. sorry :)
[9:54] Donovan Linden: If the client does not recieve the message within a timeout, the client can make the request again without the upgrade header, and fall back to the COMET-style protocol.
[9:55] Code Tracer: well actually, costumer, anti-vires applications do sniff ur connection against backdoors and trojans
[9:55] Rex Cronon: try going to a public library gareth:)
[9:55] Code Tracer: they might block reverse http thing
[9:55] Jrift Sol: my university certainly inspects packets. access to the sl website is even blocked there
[9:55] Gareth Ellison: lol rex
[9:55] Code Tracer: anti-virus*
[9:55] Donovan Linden: There are also other options such as multipart streaming COMET which is another way to optimize the latency without falling back all the way to one client request per server push
[9:56] Gareth Ellison: (offtopic in the extreme) my own site is routinely blocked by local libraries, unblocked when i complain and later blocked again in cycles
[9:56] Saijanai Kuhn: Hey morgaine, get over heah
[9:56] Saijanai Kuhn: wrong window
[9:56] Gareth Ellison: donovan - with long poll, can the socket remain idle for a horrendously long amount of time?
[9:56] Donovan Linden: however, I think what is on the Reverse HTTP page, amended with some information about what the client should do after a timeout if the server wants to Upgrade but a proxy or firewall won't let the real PTTH protocol through
[9:56] Gareth Ellison: or do we need keep-alive requests
[9:57] Donovan Linden: Gareth: With the long poll it's generally a good idea to time out after 25-30 seconds to avoid bad NAT implementations which have short idle timeouts
[9:57] Donovan Linden: keep-alive is orthogonal to the long poll; it's a good idea but it's not required
[9:57] Zero Linden: / thinks current implementation in viewer is timinig out at 30 seconds
[9:58] Donovan Linden: the current second life viewer makes a new HTTP/1.0 request every time
[9:58] Zha Ewry: / One thing to do, with things like this, is actually build a tiny test utility and pass it around widely
[9:58] Donovan Linden: zero: yes, that is because squid is killing the connection after 30 seconds of idle
[9:58] Gareth Ellison: gives 50 points to zha
[9:59] Gareth Ellison: an idea - put such a test util up on the secondlife downloads page
[9:59] Code Tracer: donovan; using this reverse http thing is a big over head... why dont u open a simple socket and keep it alive, instead of using all these http servers. proxies..
[9:59] Gareth Ellison: get the unwashed masses to get some realworld data
[9:59] Donovan Linden: there is a working utility here: https://wiki.secondlife.com/wiki/Reverse_HTTP
[9:59] Donovan Linden: Code: I don't buy that HTTP is an onerous overhead.
[9:59] Gareth Ellison: hmm, code has a point - with the reverse HTTP thing, why not just use a simple socket?
[9:59] Code Tracer: yeah ?
[10:00] Gareth Ellison: do we really need the HTTP semantics if we're using it for streaming data
[10:00] Saijanai Kuhn: Libraries are well tested for http
[10:00] Morgaine Dinova: Code: don't talk heresy. Everything has to be HTTP, you know that :-)))
[10:00] Zha Ewry: Because then you get reo redo the full semantics and have your own
[10:00] Zero Linden: the reasons for using HTTP, even beyond libraries
[10:00] Xugu Madison: Saijanai, but if you're sending a message, that's hardly rocket science...
[10:00] Zero Linden: is that we'd have to define wire protocol for almost everything there: status codes,
[10:00] Donovan Linden: the disadvantages of using a custom protocol are even worse, because we'dThe major advantage of Reverse HTTP is that the implementation can be incredibly simple using existing HTTP servers and clients
[10:00] Zero Linden: content type negotiation
[10:00] Code Tracer: since u change the protocol it is not http anymore.. u just using port 80 that's it
[10:00] Zero Linden: payload framing
[10:00] Donovan Linden: whoops, got two thoughts mixed up there
[10:00] Zero Linden: request naming
[10:00] Gareth Ellison: as i see it, we're currently hacking around some defiencies in HTTP - which works great for one-off messages or file transfers, but crap for event streams
[10:01] Zero Linden: retry semantics
[10:01] Morgaine Dinova: Zero: use SCTP, no framing
[10:01] Zero Linden: etc.....
[10:01] Donovan Linden: Code: but it's possible to fall back to pure http
[10:01] Zero Linden: you NEED framing for the messages
[10:01] Tao Takashi: so the code snippet there sets up the rhttp server?
[10:01] Gareth Ellison: so you want the content negotiation and other stuff?
[10:01] Donovan Linden: Tao: It sets up the rhttp server and actually does a client connection across it as well, and a reverse http request
[10:01] Code Tracer: donovan; i see ...
[10:01] Jn Uriza: hacking around deficiencies of HTTP has been going on every since the protocol was first RFC'd
[10:01] Saijanai Kuhn: First use for r-http *I* think is to use it to control test harness bots
[10:01] Donovan Linden: it's what produced the top protocol dump on the Reverse HTTP page
[10:02] Tao Takashi: for me it just says that a WSGI server is running
[10:02] Saijanai Kuhn: just send commands to client from sever via POST
[10:02] Jrift Sol: it must be tempting to just have the client doubling as a server, and for the sl servers to push updates
[10:02] Tao Takashi: but maybe I should actually try to understand the code ;-)
[10:02] Chema Benelli: shouts: I count 3 crossed conversations here
[10:02] Donovan Linden: conn = PtthConnection(CONNECTIONS[0]); print httpc.get('/', connection=conn)
[10:02] Tao Takashi: which involves reading it..
[10:02] Donovan Linden: those two lines at the bottom of the example actually make a requet from the server to the client
[10:02] Saijanai Kuhn: Tao its so simple its confusng.
[10:02] Gareth Ellison: hmm, coming from a background where XML-RPC rules the world (any other current or former opensim guys here?) i can see why HTTP is nice for some kinds of RPC
[10:03] Donovan Linden: it would be better to break the script into client and server peices, so it's clearer and can be used to test real firewalls and proxies
[10:03] Maltos Sosa: Im Opensim :)
[10:03] Gareth Ellison: maltos - so you know why XML-RPC rules the world then? ;)
[10:03] Taarna Welles: Lol Maltos
[10:03] Zero Linden: +1 donovan (I was just going to say tha)
[10:03] Gareth Ellison: has LLSD over REST for RPC stuff
[10:03] Tao Takashi: it's maybe small, not sure it's simple ;-)
[10:04] Donovan Linden: Another disadvantage of using a protocol other than HTTP as the push protocol is that we'd have to design and implement it, and everyone would have to implement it in each language they wanted to use
[10:04] Gareth Ellison: if LL ever patent LLSD somehow i'm completely and utterly screwed
[10:04] Donovan Linden: most languages already have http clients and servers that can be adapted to this
[10:04] Saijanai Kuhn: Creative Commons Gareth
[10:04] Gareth Ellison: ok, settled - HTTP is the winner
[10:04] Tao Takashi: many people would feel more comfortable implementing OGP if they knew it's using somethign they know, which is HTTP
[10:04] Tao Takashi: even if it's HTTP with a twist
[10:05] Zero Linden: Garth - I think LLSD has been documented publically for over a year -- which would mean we couldn't patent it
[10:05] Morgaine Dinova: Zero: a lot of those things you mentioned are severe over-complication. All you really need is a TCP stream and a method of serializing. You don't need retry semantic in the envelope when all you're doing is transferring data structures which themselves contain error indicators.
[10:05] Tao Takashi: so basically what they need to understand is Donovan's code snippet then ;-)
[10:05] Gareth Ellison: zero - true, but stranger things have happened
[10:05] Code Tracer: Gareth - .lol.
[10:05] Donovan Linden: morgaine: but you need a semantic for replies, and for figuring out what to do when a reply is lost
[10:06] Tao Takashi: I think one of out major principley should be not to re-invent anything if possible
[10:06] Gareth Ellison: ok, let's move on here?
[10:06] Gareth Ellison: HTTP is sexy, ok - next.....
[10:06] Tao Takashi: even if it might have slight advantages (but we might not know the disadvantages yet)
[10:06] Donovan Linden: oh, one last thing about Reverse HTTP -- since it is going to be going over HTTPS, I think that seals the deal for all the worries about proxies and firewalls
[10:06] Zero Linden: I strongly disagree - that is practically what the current UDP system is - though even there, there is framing, error indicators, non-delivery, header info....
[10:06] Zero Linden: and without those things, extension is very very difficult
[10:07] Donovan Linden: proxies and firewalls should work just fine since they already have to allow https traffic without knowing about the body
[10:07] Tao Takashi: the whole UDP format stuff prevented me from doing a python libsl
[10:07] Jrift Sol: donovan, you mentioned multi-part streaming comet earlier. So far, that sounds like a safe (as far as packet inspecting firewalls go) method, and not too different from what's currently in use, and goes a long way to solving the latency where there's a lot of updates. What problems do you perceive with that?
[10:07] Code Tracer: donovan - i agree ssl covers
[10:07] Tao Takashi: so +1 for known formats and standards
[10:07] Saijanai Kuhn: Tao, why is that?
[10:07] Zero Linden: This is true - once you've negotiated an end-to-end SSL connection - the firewalls can't look inside
[10:07] Morgaine Dinova: Donovan: but you're still thinking about replies! See, you still have HTTP on the mind. I'm thinking about a data stream. HTTP is just a means to an end, for getting that data across. And 90% of it is unnecessary.
[10:07] Tao Takashi: because nothing was/is documented, UDP in general is more complicated to implement (esp. if you add TCP functionality to it) etc.
[10:08] Tao Takashi: but for me it started with the packet format that the hashes used back then did never to be right
[10:08] Donovan Linden: jrift: that requires defining the semantics for replies, replies can come out of order, and you'd have to decide what to do about dropped messages
[10:08] Tao Takashi: and there was also no library you could use to help it
[10:08] Saijanai Kuhn: well, the packets are all documented in the SL source code. I've been using them
[10:08] Tao Takashi: "documented in source code".. well :)
[10:08] Gareth Ellison: lol
[10:08] Saijanai Kuhn: message listing.
[10:08] Morgaine Dinova: HTTP is a very dangerous "The only tool in my toolkit" instrument.
[10:08] Tao Takashi: I used that, too
[10:08] Donovan Linden: morgaine: You don't think the client will ever have to respond to a server side message?
[10:08] Gareth Ellison: docstrings \_/
[10:09] Donovan Linden: that seems naive
[10:09] Tao Takashi: but back then there als have been these hashes and no explanation of how they got computed
[10:09] Tao Takashi: anyway, every custom format makes implementations harder
[10:09] Tao Takashi: and we want adoption so we should go with known standards where possible
[10:09] Donovan Linden: also, it seems like a huge advantage to have all messages be REST (except for a very few messages which will stay UDP and not have replies)
[10:09] Gareth Ellison: i really want to clarify - we have HTTP transport that is unused in the viewer already, right?
[10:09] Donovan Linden: Gareth: it's not unused, it's heavily used
[10:09] Donovan Linden: every udp message that now goes over tcp uses it
[10:10] Saijanai Kuhn: ah, well, libsl uses its own hash. I just index off of the packet ID num
[10:10] Gareth Ellison: well, yeah - but a whole load still goes over UDP
[10:10] Gareth Ellison: it should be possible to shove it all over HTTP, right?
[10:10] Tao Takashi: Sai: It might have changed, I haven't looked into that for 1.5 years ;-)
[10:10] Donovan Linden: and some of those messages will stay udp. any message which is time-sensitive and an old message can be dropped if a new message has been received should go over udp.
[10:10] Saijanai Kuhn: OK, only started doing it 6 months ago, so major changes I'm sure
[10:11] Donovan Linden: ObjectUpdate for example
[10:11] Tao Takashi: so no messages which require an ACK etc.?
[10:11] Donovan Linden: I think that makes the most sense tao
[10:11] Tao Takashi: (or resending)
[10:11] Zha Ewry: Well, lettign UDP drop messages, puts you at the mercy of it's total lack of semantic understanding of what to drop, we've talked about that
[10:11] Morgaine Dinova: Donovan: the only reason you're making those HTTP calls is to get data back (other than the side effect of changes on the server). If you have another means of getting that data across then the two things can be separated: (i) the perform-action side effect for the server, and (ii) the checking of data (which may be a response) on the client side. No longer any need to couple the two things.
[10:11] Tao Takashi: it definitely makes implementation easier if you don't have to worry about housekeeping
[10:12] Jn Uriza: surely your now talking about encapsultating a higher level protocol or logic within HTTP so why bother, why not skip the whole HTTP overhead altogether? or maybe I'm missing the guts of this
[10:12] Donovan Linden: if you want an ack, you should use rest semantics
[10:12] Zero Linden: Right - pushing all that coordination into the higher levels means that each and every sub-system has to do that work
[10:12] Zero Linden: which is what is going on in our code base now
[10:12] Zero Linden: it is a tremendous duplication of effort
[10:12] Donovan Linden: Jn: because we want request/response semantics because we want atomic messages
[10:12] Zero Linden: becuase the vast majority of interactions in the protocol are request/response
[10:12] Tao Takashi: goes back looking into Donovan's script :)
[10:12] Jn Uriza: ah ok
[10:12] Donovan Linden: so why bother defining all of that when we already have a perfectly good spec?
[10:12] Zero Linden: a very very small percentage of traffic is just a pure message
[10:13] Donovan Linden: ok. is there anything else on the agenda? :)
[10:13] Gareth Ellison: python libsl i heard
[10:13] Gareth Ellison: that's what woke me up
[10:13] Gareth Ellison: in a frenzy, running to the computer.....
[10:13] Python Morales: same here :)
[10:13] Day Oh: :D Thanks Donovan
[10:14] Gareth Ellison: i didn't even bother to change my av out of slut mode........
[10:14] Donovan Linden: ah cool, Tao's python client?
[10:14] Gareth Ellison: we all follow the Tao!
[10:14] Donovan Linden: so enus is also starting work on a Python client for automated testing right now
[10:14] Tao Takashi: ok, me :)
[10:14] Q Linden: and jacob and I are here because we're interested as well
[10:14] Donovan Linden: saijani has also been working on a python client for a while
[10:14] Tao Takashi: well, it's not yet a python libsl, it's mainly the structure for such a thing ;-)
[10:14] Gareth Ellison: has a few misc python modules too
[10:15] Saijanai Kuhn: Just documenting the protocols basically
[10:15] Tao Takashi: here is my blog post about it: [2]
[10:15] Python Morales: i have written bots for the rex project using python, using libsl from ironpython
[10:15] Donovan Linden: so since all these people are doing it independently... we should work together! :)
[10:15] Gareth Ellison: packet encoding/decoding in binary and LLSD
[10:15] Q Linden: bingo
[10:15] Tao Takashi: well, I started it from a discussion with Enus Linden
[10:15] Gareth Ellison: using libsl from ipy is trivial
[10:15] Saijanai Kuhn: https://wiki.secondlife.com/wiki/Example_protocol_code
[10:15] Gareth Ellison: but not exciting
[10:15] Tao Takashi: so I started to setup some structure
[10:15] Day Oh: Sai you made a sweet script that parses the message template right
[10:15] Zha Ewry: looks up from .cs files and listens
[10:16] Tao Takashi: it was coming from the discussion about a test harness for OGP
[10:16] Saijanai Kuhn: only the header, not the body, Day
[10:16] Donovan Linden: zero did a library for parsing the message template
[10:16] Gareth Ellison: there's python code in the viewer that parses the message template
[10:16] Saijanai Kuhn: https://wiki.secondlife.com/wiki/Template_dictionary_python
[10:16] Tao Takashi: and for that to work we thought some library to test server components might not be bad
[10:16] Donovan Linden: I think it's in the open source python indra library
[10:16] Gareth Ellison: has some code that can serialise and deserialise arbitary packets
[10:16] Donovan Linden: it should be pretty easy to implement the udp protocol in python using the struct module
[10:17] Saijanai Kuhn: Yeah, I was lazy and just rolled the half dozen or so packets by hand for group IM testing
[10:17] Tao Takashi: Here is the source code for pyogp with some documentation
[10:17] Tao Takashi: [3]
[10:17] Gareth Ellison: donovan - do you recall the streamlet code i sent you?
[10:17] Tao Takashi: I used the struct lib in my python libsl experiments back in Jan 2007
[10:18] Donovan Linden: gareth: hmm, I do not
[10:18] Gareth Ellison: check your old emails
[10:18] Gareth Ellison: i dropped you and i think saijanai a tarball
[10:18] Q Linden: [4]
[10:18] Tao Takashi: like here: [5]
[10:18] Saijanai Kuhn: vaguely recalls tarball.
[10:18] Donovan Linden: mail didn't find anything with the word "streamlet" in it
[10:18] Gareth Ellison: that's the one Q - unsure if the SVN repo has a public browser up anywhere
[10:18] Q Linden: whoops, links are dead
[10:19] Gareth Ellison: "old archWG code"
[10:19] Saijanai Kuhn: doesnt' get too many tarballs in email. Penis enlargement messages yes, tarballs, no
[10:19] Gareth Ellison: that was the subject
[10:19] Morgaine Dinova: It would be nice if AWG had its own SVN repo for all these things. We're scattered everywhere because of the lack
[10:20] Tao Takashi: I also encapsulated LL's capsserver in pyogp
[10:20] Tao Takashi: so you can simply instantiate it from the command line
[10:20] Gareth Ellison: we did have on openmv.org
[10:20] Donovan Linden: gareth: ok I found it
[10:20] Gareth Ellison: but nobody used it
[10:20] Gareth Ellison: i think only me and one other person whose name i don't recall bothered to setup an account
[10:20] Tao Takashi: more information here: [6]
[10:20] Day Oh: Do you people working on python stuff have an IRC channel or anything? (:
[10:21] Morgaine Dinova: Gareth: same for all the others, there's no buyin. That's why it would be good if LL provided one for the project.
[10:21] Zha Ewry: and AWG svn would be handy
[10:21] Tao Takashi: well, there is #opensl
[10:21] Gareth Ellison: #opensl on EFNet
[10:21] Gareth Ellison: heh
[10:21] Tao Takashi: but not too much happening there regarding AWG
[10:21] Tao Takashi: I think ;-)
[10:21] Gareth Ellison: it would be good if we had a linden-hosted repo
[10:21] Saijanai Kuhn: choopa is more reliable for me thandirect efnet
[10:21] Gareth Ellison: any chance of that happening?
[10:21] Tao Takashi: somehow must stuff is happening in-world which maybe also isn't too good for transparency
[10:21] Enus Linden: morgaine, we'd talked about a LL hosted svn repo for the ogp test harness. i need to make that happen asap
[10:22] Tao Takashi: why not simply use Google Code? ;-)
[10:22] Zha Ewry: In world is a mixed blessing
[10:22] Saijanai Kuhn: thinks that test harness will be first use of reverse http: command sending via post from control script/web page to test harness
[10:22] Zha Ewry: I'm a huge fan of eatting our own dogfood
[10:23] Zha Ewry: But.. it does make it necessary to be very good about posting transcripts and publisziing meetinfgs
[10:23] Tao Takashi: well, I am not against meetings in here but IMHO it would also be nice to do things more via mailing lists like inbetween meetings
[10:23] Tao Takashi: and transcripts honestly are a pain to read
[10:23] Gareth Ellison: i'd prefer IRC personally - way way easier to discuss things without the avatars
[10:23] Gareth Ellison: but then i'm weird
[10:23] Donovan Linden: gareth: streamlet looks good
[10:23] Morgaine Dinova: Too many kiddies on efnet, freenode is a lot more serious
[10:23] Gareth Ellison: efnet is already the unofficial home of SL-related stuff
[10:24] Tao Takashi: but opensim moved ;-)
[10:24] Gareth Ellison: opensim moved, but libsl, opensl and #secondlife remained
[10:24] Gareth Ellison: admittedly #secondlife is quite a bit more trollish
[10:25] Tao Takashi: so the question is now if anybody is interested on working on pyogp
[10:25] Gareth Ellison: ooh ooh
[10:25] Gareth Ellison: me me
[10:26] Saijanai Kuhn: moi
[10:26] Morgaine Dinova: I'll take a look Tao
[10:26] Zha Ewry: looks up, and waves "After Memorial day"
[10:26] Tao Takashi: [7] is the docs so far and [8] is the code
[10:26] Saijanai Kuhn: still hasn't gotten group IM workign :-(
[10:26] Python Morales: Tao, I am busy with another thing for at least several weeks, but perhaps later
[10:26] Gareth Ellison: lost the "AWG Python Nympho" tag :(
[10:26] Tao Takashi: right now it's not too much but at least I setup eggs for all the components
[10:27] Saijanai Kuhn: use it or lose it Gareth. Everyt Nompho knows that
[10:27] Morgaine Dinova: Yummy, eggs
[10:27] Tao Takashi: like one for the library, one for the agentdomain, one for the capsserver
[10:27] Gareth Ellison: :(
[10:27] Tao Takashi: and one for some common classes, like Agent, Region etc.
[10:27] Gareth Ellison: i still have code-sex with python every night
[10:27] Gareth Ellison: in a dark room, me and the REPL
[10:27] Tao Takashi: I guess they can be reused.. at least the interfaces
[10:27] Tao Takashi: if not we can delete it ;-)
[10:27] Gareth Ellison: the REPL being in the context of a live server daemon with connected users
[10:27] Donovan Linden: Gareth: It should be easy to use streamlet to implement the new comet fallback from the Reverse HTTP page
[10:27] Gareth Ellison: ultimate kinky
[10:27] Tao Takashi: it's all in the pyogp folder
[10:27] Enus Linden: Tao, i replied to sldev with agent domain loginuri info (like, the right endpoint for testing :)
[10:28] Tao Takashi: Enus: Yes, I saw that, thanks :)
[10:28] Donovan Linden: I have a server-side implementation inside the indra package, but it's not yet open source
[10:28] Tao Takashi: I already used that location though
[10:28] Gareth Ellison: donovan - streamlet was designed for that longpoll method
[10:28] Tao Takashi: as Which was telling me about it
[10:28] Saijanai Kuhn: and once its working, you gotta put a version inthe current protocols page
[10:28] Saijanai Kuhn: link to a version
[10:28] Morgaine Dinova: Tao: want to hold a meeting sometime specifically about that code, and where it can be taken?
[10:28] Tao Takashi: sure, maybe I can also make a little screencast
[10:28] Zha Ewry: Anyone who wants to hold those, is welcome to use this space
[10:29] Tao Takashi: I am just not sure when, this week is a bit full (and there's a holiday) and next week I have to give a Plone training and I am speaking about OGP at the Metaverse08
[10:30] Gareth Ellison: why do these sexy conferences never happen in north wales? :(
[10:30] Morgaine Dinova: Too cold and wet
[10:30] Tao Takashi: well, this one is in Germany.. I am not sure how sexy it will get though ;-)
[10:30] Gareth Ellison: no it isn't - boiling hot
[10:30] Tao Takashi: but we have at least Linden Lab attendance :)
[10:30] Tao Takashi: but not for talking about AWG/OGP unfortunately
[10:31] Tao Takashi: and if somebody wants to come to EuroPython, registration is open and I will do a sprint about OGP if somebody shows up who is interested
[10:31] Tao Takashi: [9]
[10:31] Saijanai Kuhn: afk
[10:31] Python Morales: Tao, I am considering europy
[10:31] Tao Takashi: cool! :)
[10:32] Tao Takashi: if you want to give a talk, deadline is in 2 days ;-)
[10:32] Tao Takashi: (for the topic not the contents)
[10:32] Python Morales: Tao, browsed thru your ogp code now .. not much meat yet (or did i mess something?) but interesting .. and i guess zope.interfaces may well be good for this kind of stuff
[10:32] Tao Takashi: I will also give a talk about pyogp there
[10:32] Tao Takashi: no, not much yet.
[10:32] Gareth Ellison: oh, is the pyogp code using a web framework Tao?
[10:32] Tao Takashi: there is a login class based on my tests I did
[10:32] Python Morales: oh, perhaps i could patch together something about using py for live rapid protoyping of game AIs, or something :)
[10:33] Tao Takashi: no, it's not using a web framework
[10:33] Gareth Ellison: yay
[10:33] Tao Takashi: zope.interfaces is only a small part of zope which is good for registering components
[10:33] Gareth Ellison: the only decent web framework is web.py
[10:33] Tao Takashi: so that you can loosely couple those bits together
[10:33] Q Linden: runs from rwars
[10:33] Gareth Ellison: web.py running on lighttpd with load-balanced circular replicating MySQL backend
[10:33] Gareth Ellison: that's the best setup ever
[10:34] Donovan Linden: boo mysql
[10:34] Morgaine Dinova: flat files ftw
[10:34] Tao Takashi: so here is the login code: [10]
[10:34] Python Morales: zope.interfaces kinda gives you a way to make interface definitions in py, like you can do in c# or java with interface declarations
[10:34] Gareth Ellison: has roughly 2GB per day of new data shoved into MySQL
[10:35] Gareth Ellison: it hasn't crashed down painfully yet
[10:35] Tao Takashi: csv file FTW! ;-)
[10:35] Tao Takashi: nobody needs mysql
[10:35] Gareth Ellison: tell me - how do you do replication with CSV?
[10:35] Gareth Ellison: anyway.....
[10:35] Morgaine Dinova: Gareth: it's waiting for you to put more eggs into that basket before crashing ;-)
[10:35] Gareth Ellison: ah, but the beauty is that i have 4 circular-replicated hosts with auto failover
[10:35] Tao Takashi: this is the agent definition so far: [11]
[10:36] Tao Takashi: but looking at it I guess it probaly should not be in model but in the lib
[10:36] Donovan Linden: you do replication by having multiple rest backends, and not using csv :)
[10:36] Q Linden: bye, all...got next mtg...thanks!
[10:36] Gareth Ellison: [12] < is this design still relevant today btw?
[10:36] Rex Cronon: by q
[10:37] Tao Takashi: and this is the capsserver entrypoint which sets up a mulib capsserver: [13]
[10:37] Gareth Ellison: old patent application for SL that was declined
[10:37] Morgaine Dinova: Bah, the only way to do backups is to send stuff out on the net, and let everyone else back it up for you. Linus said so. ;-)
[10:37] Gareth Ellison: seems to have some vastly superior algorithms compared to opensim - only sends what's visible and has changed etc
[10:37] Tao Takashi: I send it to my wifi interface
[10:37] Tao Takashi: I only have to compute the storage capacity of air waves
[10:37] Gareth Ellison: backups via spam
[10:38] Gareth Ellison: encode your data into email addresses, post on your website
[10:38] Morgaine Dinova: backupfs only stores diffs I think
[10:38] Tao Takashi: more so via p2p networks
[10:38] Gareth Ellison: spammers will send it back to you
[10:38] Tao Takashi: right but your webspace would need to store it before that happens
[10:38] Day Oh: Hmm Google Code lets you set up a group, right
[10:38] Tao Takashi: what do you mean by group?
[10:39] Day Oh: Some sort of mailing list type thing? I think I'm subscribed to sleek's
[10:39] Gareth Ellison: being serious..... that old SL patent application - is it still relevant today?
[10:39] Tao Takashi: yes
[10:39] Gareth Ellison: i mean the actual design
[10:39] Tao Takashi: you can add a Google Group to it
[10:39] Tao Takashi: or multiple ones
[10:39] Tao Takashi: like one for svn-commit messages
[10:39] Morgaine Dinova: Being serious and software patents shouldn't be in the same sentence.
[10:39] Tao Takashi: I might do that later for pyogp specific things to discuss
[10:40] Gareth Ellison: yeah - note i said it was declined
[10:40] Gareth Ellison: philip filed it back in 2003
[10:40] Tao Takashi: general AWG stuff should probably best be discussed on sldev as long as it's not that much
[10:40] Tao Takashi: I hope it will get more though
[10:41] Gareth Ellison: mmmmm host code on google code, get on the google groups for SVN commits sent to my google apps hosted gmail
[10:42] Gareth Ellison: google own my life
[10:42] Tao Takashi: was there any doubt?
[10:43] Enus Linden: Q and I are going to chat about an hq repo vs LL svn fwiw. we can talk about Google Code too... and it has been suggested that Apache v2 license would be appropriate for this effort.
[10:43] Enus Linden: hg*
[10:43] Gareth Ellison: svn
[10:43] Morgaine Dinova: I'm surprised that wikis don't get used for collaborative design and coding. They're almost ideal, with an attached SVN. Far better than MLs, which are hard to focus.
[10:43] Gareth Ellison: svn is pretty much the defacto standard these days
[10:44] Tao Takashi: I am not sure, wiki's also tend to grow into various direction and then it's hard to find stuff again
[10:44] Gareth Ellison: though i can remember when i called a friend a heretic because he'd recently imported his CVS code into SVN
[10:44] Donovan Linden: hg is a million times better than svn
[10:45] Gareth Ellison: SVN == largest deployment, everyone and his dog has a client
[10:45] Tao Takashi: I don't care what it is, as long as a) there is something b) the decision process does not take 2 months ;-)
[10:45] Enus Linden: amen tao
[10:45] Gareth Ellison: lol
[10:45] Zha Ewry: looks up and amens
[10:45] Gareth Ellison: perhaps we should all vote or something?
[10:45] Tao Takashi: as for the license I don't know anyway what all the differences between those 1000 licenses are ;-)
[10:46] Donovan Linden: have you ever tried hg?
[10:46] Gareth Ellison: places 1000 votes for svn
[10:46] Tao Takashi: I tried it the first time today to get the eventlet code
[10:46] Python Morales: gareth, svn is basically cvs with basic annoyances fixed - but the same model. distributed systems like hq and bzr and darcs etc. are a new generation in design and have benefits. i think it comes down to needs .. some projects need distributed more, some are fine with centralized
[10:46] Gareth Ellison: admittedly no, i haven't
[10:46] Donovan Linden: trust me, once you've used a distributed revision control system you'll never want to go back to svn
[10:46] Tao Takashi: Donovan: any plans to increase it to 0.3 btw? :)
[10:47] Donovan Linden: Tao: increase what? the hg on my personal site?
[10:47] Donovan Linden: sure
[10:47] Tao Takashi: the version number of eventlet
[10:47] Zha Ewry: The main virtue of svn, is everyone has it
[10:47] Donovan Linden: ah, yes, I do plan on doing an eventlet beta this week if I can
[10:47] Tao Takashi: well, it was "easy_install mercurial" and I had it, too
[10:47] Tao Takashi: cool
[10:47] Morgaine Dinova: I don't think the key problem here is *which* svn.
[10:47] Enus Linden: I spoke with Robla about licenses, as he is far far more familiar with them that I, and he suggested Apache v2. Zha had mentioned the same... [14]
[10:47] Gareth Ellison: hmm, i'll admit that my current scheme of mounting all my SVN repos from central location via sshfs is a bit of a pain
[10:47] Tao Takashi: then I will wait with my pypi upgrade until that's around
[10:48] Donovan Linden: Tao: ok, cool
[10:48] Tao Takashi: so far I only know "hg clone", I guess I should read more about it soon ;-)
[10:48] Tao Takashi: maybe you should then get rid of the .svn dirs in it ;-)
[10:48] Donovan Linden: hg clone/hg commit/hg push, that's all you need to know
[10:48] Donovan Linden: yeah I should get rid of the .svn heh
[10:49] Gareth Ellison: hg commit and hg push are seperate commands?
[10:49] Gareth Ellison: i've entered alien territory here
[10:49] Donovan Linden: commit commits to the local repo, push moves the patch to the shared repo
[10:49] Donovan Linden: or any other repo
[10:49] Donovan Linden: all hg repos are peers
[10:49] Donovan Linden: you can push patches from any to any other
[10:50] Donovan Linden: branches are just cloned repos
[10:50] Zha Ewry: Apache v2 is nicely compatible with most of what we're doing
[10:50] Tao Takashi: Zha: what are the "features"? :)
[10:50] Gareth Ellison: from a brief reading it seems to be a copyleft - am i correct?
[10:50] Zha Ewry: Its BSD with a bit of the patent torpedo loophole closed
[10:50] Morgaine Dinova: Sold. Get hg installed. And then stick the client sources in there too, and let the community make its own client, which you build and release at same time as yours :-)
[10:51] Tao Takashi: CG Linden mentioned that this is the plan
[10:51] Donovan Linden: the only problem with hg is hosting
[10:51] Tao Takashi: to move to HG
[10:51] Zha Ewry: Its a bsd style license
[10:51] Donovan Linden: brian o'sullivan (a linden) uses a tool to bridge between svn and hg
[10:51] Zha Ewry: But. it does have a few lines which prevent you from shoving hiden IPR content
[10:51] Gareth Ellison: moving from svn to hg feels oddly like moving from my male avatar to my female avatar fulltime
[10:52] Tao Takashi: ok, thanks Zha
[10:52] Zha Ewry: Its also the one Which is using for all the c-hhtp stuff
[10:52] Zha Ewry: which limits the number of licenses people have to think about, which is a good thing
[10:52] Tao Takashi: I now put pyogp to be LGPL
[10:53] Zha Ewry: checks, and thinks she has the Ws all in the right cases
[10:53] Tao Takashi: need to think about what's appropriate ;-)
[10:53] Gareth Ellison: my inner stallman is screaming "GPL THE LOT"
[10:53] Zha Ewry: The OpenSim communnity wimpers when they see GPL in the mix
[10:53] Gareth Ellison: but i'll refrain from giving into the bearded urges
[10:54] Zha Ewry: His comments on SLDEV where actually pretty rational and consdered
[10:54] Tao Takashi: indeed
[10:54] Tao Takashi: I am not sure if pyogp will be integrated into OpenSim though ;-)
[10:54] Tao Takashi: I mean we are able to build quite separate components now anyway
[10:55] Zha Ewry: nods
[10:55] Tao Takashi: my first project should be "Copy Domain" to get some press ;-)
[10:55] Gareth Ellison: pyogp WILL be headed into litesim at some point - i can't wait to run screaming from C# once and for all
[10:55] Zha Ewry: I am sort of handed my thoughts on licenses by people with legal degrees who work for my IPR department
[10:56] Tao Takashi: I might be missing such a department ;-)
[10:56] Morgaine Dinova: Too much focus on licenses, IMO. It has split the developer community apart, and nobody benefits, other than lawyers.
[10:56] Zha Ewry: What, you don't like yet another algolish object oriented language, Gareth?
[10:56] Gareth Ellison: i started my own pylibsl for that purpose
[10:56] Tao Takashi: Donovan: btw. your script still does nothing for me ;-)
[10:56] Gareth Ellison: zha - i've fallen in love with python
[10:56] Tao Takashi: Donovan: it says 2328 wsgi starting up on ('127.0.0.1', 9999)
[10:56] Python Morales: blushes
[10:56] Tao Takashi: and that's it
[10:57] Morgaine Dinova: LOL
[10:57] Gareth Ellison: nothing like sitting in a dark room with a python REPL on my terminal, an unhealthy pile of cans of red bull and said REPL running on a live server daemon with 10 users connected
[10:57] Donovan Linden: tao: hmm, I'll try it again
[10:57] Donovan Linden: ok, I have to go to our huddle now, thanks everybody
[10:57] Gareth Ellison: when will we get word on hg hosting then?
[10:57] Jrift Sol: thanks Donovan.
[10:57] Gareth Ellison: before you go off to strange places?
[10:57] Rex Cronon: bye donovan
[10:58] Code Tracer: Bye for now.
[10:58] Gareth Ellison: sir donovan! a quick reply!
[10:58] Donovan Linden: k gareth
[10:58] Day Oh: Thanks Donovan, and thanks everyone, I'm gonna tp out as well
[10:58] Rex Cronon: so the meeting is over? right?
[10:58] Enus Linden: Gareth, I'll be speaking with Q about hg this afternoon and will get to AWG asap
[10:58] Rex Cronon: bye day
[10:58] Day Oh: Best wishes <3
[10:58] Goldie Katsu: Thanks Donovan.
[10:58] Zha Ewry: Super meetiing people
[10:58] Donovan Linden: gareth: reply to what?
[10:59] Gareth Ellison: when we get hg
[10:59] Gareth Ellison: but enus already replied
[10:59] Donovan Linden: ah. ok. later!
[10:59] Enus Linden: tata4now
[10:59] Rex Cronon: by everybody, i am going to h4 office hour @: [15]
[10:59] Rex Cronon: tc]

AW Groupies/Chat Logs/AWGroupies-2008-05-20

Navigation menu

Search