User:Enus Linden/Office Hours/2010 February 05

[13:02] Latif Khalifa: perhaps enus will grace us with his presence this year
[13:02] Latif Khalifa: :P
[13:02] Dedric Mauriac: if anyone would like a free bear, let me know and i'll send you mine.
[13:02] Ardy Lay: Nice lag-walk there Enus.
[13:02] Enus Linden: it's punishment latif
[13:03] Latif Khalifa: speaking of the...
[13:03] Latif Khalifa: lol
[13:03] Enus Linden: tz ardy, i've been working on perfecting it!
[13:03] Enus Linden: tx*
[13:03] Latif Khalifa: judging by how often you come, you sure must feel like it's punishment :P
[13:04] Enus Linden: tbh, i've been wrestling with the idea of 1. stopping my OH or 2. sharing the mc responsibilities
[13:04] Enus Linden: its not getting any easier to make time, and that sucks
[13:04] Enus Linden: i dont want to stop em
[13:04] Enus Linden: but do want them to be mutually useful
[13:05] Enus Linden: nice to see that folks havent given up yet!
[13:05] Latif Khalifa: i was advertising the OH xD
[13:05] Arcane Laval: :)
[13:05] Dedric Mauriac: we are all just bots
[13:05] Latif Khalifa: long time no see :)
[13:06] Enus Linden: have you an agenda in advertising today latif?
[13:06] Latif Khalifa: erm, no
[13:06] Enus Linden: crap, so i need to have one
[13:06] Enus Linden: ;)
[13:06] Latif Khalifa: i could've made one had i known it was required
[13:06] Enus Linden: i've a couple of things on the mind...
[13:06] Ardy Lay: Ah.. well, if you are not getting anything out of them then I don't see there is much point. I know I don't have much to offer you. I was just hoping to learn a little more about what's going on in SL.
[13:07] Enus Linden: one is pyogp, which should start moving again soon
[13:07] Enus Linden: another is in regards to the automation team
[13:07] Enus Linden: and we'll see where things go from there
[13:07] Latif Khalifa: perhaps some would be interested to hear about automated load testing using Amazon EC2 cloud
[13:07] Enus Linden: aye latif
[13:08] Latif Khalifa: i know i would xD
[13:08] Enus Linden: i'll cover that
[13:08] Morgaine Dinova: Me too
[13:08] Dedric Mauriac: interesting
[13:08] Enus Linden: awesome
[13:08] Enus Linden: well, thanks all for coming
[13:08] Enus Linden: hi rakesh!
[13:08] Dedric Mauriac: will there be a white paper about it?
[13:08] Rakesh Linden: hi Enus
[13:08] Enus Linden: dedric, maybe?
[13:08] Rakesh Linden: there is no voice
[13:08] Latif Khalifa: hello Rakesh
[13:08] Enus Linden: OH is in text
[13:08] Enus Linden: to many languages
[13:08] Enus Linden: to listen clearly
[13:08] Rakesh Linden: i c.
[13:08] Youri Ashton: friendly hello's!
[13:09] Rakesh Linden: hi everyone
[13:09] Enus Linden: plus text requires minimal moderation
[13:09] Enus Linden: hey youri
[13:09] Morgaine Dinova: Enus, what would you like to see on your part, that would make these meetings more useful to you, beyond just "community contact"?
[13:09] Enus Linden: so pyogp lib work update really quicly
[13:09] Enus Linden: good ? morgaine
[13:09] Enus Linden: i'll take a note to discuss it
[13:09] Arcane Laval: I'll be right back!
[13:10] Enus Linden: alright: pyogp
[13:10] Enus Linden: i have done nil with this for a couple of months
[13:10] Enus Linden: i ported to hg
[13:10] Enus Linden: did some dev, moved indra.base to llbase after pushing that to pypi
[13:10] Enus Linden: and it stalled
[13:11] Enus Linden: I'm too busy to continue regularly, but...
[13:11] Enus Linden: i'll be handing off maintenance and further extension of the libs to some folks in QA here
[13:11] Enus Linden: OS folks are of course encouraged to collab as well
[13:12] Enus Linden: but that's been quiet for some time and i assume not much there
[13:12] Morgaine Dinova: Since it's the basis of your test platform (isn't it?), do you mean it's stalled because your current test stuff works and you have no incentive to touch it?
[13:12] Enus Linden: i'd love to see an ecosystem pop up around it again
[13:12] Eddi Decosta: hey Enus ㋡
[13:12] Enus Linden: should prolly start engageing in irc and the mailing list more agin
[13:12] Enus Linden: again
[13:12] Enus Linden: morgaine: it's still ramping up as the basis
[13:13] Enus Linden: Rakesh (and Ali, Vektor, Fisher, and others as time allows) will be driving it's evolution primarily now
[13:13] Youri Ashton: hey fisher!
[13:13] Eddi Decosta: heya Fisher! ㋡
[13:13] Enus Linden: there is MUCh incentive to touch it: it's incomplete, amd not much testing is in place using it now
[13:14] Fisher Linden: Hey
[13:14] Enus Linden: that's going to cahnge this quarter
[13:14] Morgaine Dinova: Well I still hope to poke VWRAP bits into PyOGP, once we know what the hell to poke in. ;-) I'm not in the slightest bit happy with Pixel's addition of OGP code into Snowglobe --- that's no way to test out protocol ideas, by hardwiring them into C++ code within a nightmare codebase.
[13:14] Eddi Decosta: welcome to sl Rakesh lol ㋡
[13:14] Enus Linden: VWRAP is highly encouraged to leverage PyOGP
[13:14] Morgaine Dinova: :-)
[13:14] Enus Linden: maybe a pyogp-vwrap fork is in order
[13:15] Enus Linden: in fact, we should make it so
[13:15] Rakesh Linden: thx Eddi
[13:15] Morgaine Dinova: Could happen. Although with Hg, a "fork" isn't necessary, just another repository, or several.
[13:15] Ardy Lay: Ugh. Some uninitiated noob is closing my JIRA issues as "expected bahaviour". I gotta go beat on them. Bye.
[13:15] Enus Linden: i'd suggest a fork of the lib packages
[13:15] Rakesh Linden: is VWRAP a standard yet?
[13:15] Enus Linden: slightly more formal
[13:15] Morgaine Dinova: Lord no
[13:16] Latif Khalifa: there is nothing to fork over for yet
[13:16] Enus Linden: it's a theory
[13:16] Rakesh Linden: I thought it is still a proposal
[13:16] Latif Khalifa: no implementation exists yet
[13:16] Latif Khalifa: so talking about vwrap fork is premature imhp
[13:16] Latif Khalifa: o even
[13:16] Enus Linden: yep
[13:16] Morgaine Dinova: VWRAP is currently at the 5% reached initial draft stage, if that. And not even that 5% is well examined yet, let alone agreed. Early days.
[13:17] Enus Linden: i'll support pyogp as a test bed when timing dictates it
[13:17] Rakesh Linden: i agree with Latif
[13:17] Enus Linden: so sometime next week i'll set up a roadmap and walkthrough for qa to grab pyogp and make it go as a server side test source
[13:18] Enus Linden: they've been tinkering, but we're all busy
[13:18] Morgaine Dinova: That's cool
[13:18] Sea Urchin beanbag: Going to next texture.
[13:18] Sea Urchin beanbag: Going to next texture.
[13:18] Sea Urchin beanbag: Going to next texture.
[13:18] Enus Linden: their work will complement the work of the automation team as it spins up
[13:19] Enus Linden: the automation team is Spider, Ray, Xandix, and the not yet started but hired Scobu
[13:19] Enus Linden: Cassandra = PM
[13:19] Enus Linden: i'm in the mix in some ways, mostly high level
[13:19] Morgaine Dinova: Project Manager>
[13:19] Morgaine Dinova: ?
[13:19] Enus Linden: yep Morgaine
[13:19] Morgaine Dinova: kk
[13:19] Enus Linden: Spider = Lead
[13:19] Latif Khalifa: is automatio team all about services, sims, and the backend?
[13:20] Enus Linden: not onlt latif
[13:20] Enus Linden: will cover Viewer and Web as well
[13:20] Latif Khalifa: interesting
[13:20] Morgaine Dinova: Viewer? In what respect?
[13:20] Dedric Mauriac: click button, verify something happens
[13:20] Enus Linden: cannot share details w/o consent at this time
[13:21] Enus Linden: GUI automation would be great, but isnt part of the picture right now
[13:21] Enus Linden: pixel based testing isn't our idea of fun yet
[13:21] Morgaine Dinova: But that's GUI QA.
[13:21] Dedric Mauriac: unit testing
[13:21] Morgaine Dinova: Hmm. Can you explain the diff btn automation and QA teams?
[13:21] Enus Linden: functional automation against the viewer is being worked on now though
[13:21] Enus Linden: yeah morgaine
[13:22] Enus Linden: Automation team's charter is to build frameworks and environments to support testing
[13:22] Dedric Mauriac: qa is people with eyes verifying manually
[13:22] Morgaine Dinova: Jeez no Dedric, or I hope not
[13:22] Enus Linden: their work will initially be setting up a harness that handles code and enironments to test in, running existing unit tests and functional tests, and enabling more to be run via enrionemnts and such
[13:22] Dedric Mauriac: automation supports qa
[13:23] Enus Linden: QA = test engineers
[13:23] Enus Linden: test planning, risk assessmnet, test execution,
[13:23] Enus Linden: QA and Dev write autoamted test cases
[13:23] Enus Linden: automation team makes sure they are run and reported on
[13:24] Enus Linden: once there is sufficient framework coverage, the automation team can certainly help with test writing
[13:24] Enus Linden: but that is not their responsibility atm
[13:24] Morgaine Dinova: You mean QA team "makes sure they are run and reported on", right?
[13:25] Widget Whiteberry slips away
[13:25] Enus Linden: in a way morgaine, it's the auto team facilitating the running and reporting. QA uses the data as part of test plan validation
[13:25] Morgaine Dinova chuckles
[13:26] Enus Linden: so the 30000 foot view of the automatin team roadmap as we know it:
[13:26] Enus Linden: provide a test harness
[13:26] Morgaine Dinova: I'm confused, but don't worry about it. The way I make sense of it is Autom == make test infrastructure, QA == use it. But I guess that's wrong.
[13:26] Dedric Mauriac: that's a lot of feet
[13:26] Enus Linden: hook existing automated tests in (ours, third party libs, w/e)
[13:26] Enus Linden: morgaine, that's right
[13:27] Morgaine Dinova: Uh, OK ^_^
[13:27] Enus Linden: 3000 ft cont: extend the harness with test beds in the form of dedicated hardware and EC2 virtual environments
[13:27] Enus Linden: missed a 0 there ;)
[13:27] Morgaine Dinova: UDP loss
[13:28] Enus Linden: work on facilitating load tests in real and virtual envs
[13:28] Enus Linden: report on all of it
[13:28] Enus Linden: build libs to support test case state setting and execution
[13:28] Enus Linden: etc etc etc
[13:28] Enus Linden: we've sketched out plans for the year, given info we know, as of this week
[13:28] Enus Linden: we start real work monday
[13:29] Enus Linden: wheeeee
[13:29] Sea Urchin beanbag: Going to next texture.
[13:29] Morgaine Dinova: Wheee indeed :-)
[13:29] Latif Khalifa: sounds some awesome QA juice on the way :D
[13:29] Fisher Linden: Wheeee indeed
[13:29] Sea Urchin beanbag: Going to next texture.
[13:29] Morgaine Dinova: About time. You can't afford to do the grunt work manually, really can't.
[13:29] Sea Urchin beanbag: Going to next texture.
[13:29] Sea Urchin beanbag: Going to next texture.
[13:29] Sea Urchin beanbag: Going to next texture.
[13:29] Enus Linden: agreed morgaine
[13:30] Dedric Mauriac: oh, i think they can afford it, but wouldn't make sense though
[13:30] Enus Linden: QA folks are going to start getting more direction toward automation in many ways
[13:30] Enus Linden: for some, it's a natrual inclination
[13:30] Enus Linden: but as a team, we've got to start heading that way more and more
[13:31] Morgaine Dinova: Will be brill to be able to say "1.23.9 is ready for RC1 release, passes all regression tests for 1.23.8".
[13:31] Dedric Mauriac: with the use of automation, problems can be found quicker, as they happen in development, or newly discovered
[13:31] Enus Linden: test plans need to assert things like "100% unit tests coverage and 80% functinal test coverage"
[13:31] Enus Linden: we need to start budgeting time, and building up skill sets to enable it
[13:31] Latif Khalifa: hopefully with amazon and there diverse locations you could add some higher latency tests like connecting from europe etc. some network timing issues are known to slip by if you test from the same country :)
[13:31] Enus Linden: so will improve over the upcoming months
[13:31] Fisher Linden: Entrance and exit criteria, Enus
[13:32] Dedric Mauriac: are you using things like nunit/nmock/nhibernate/log4net and such?
[13:32] Morgaine Dinova: Oooh, coverage analysis, need. Even if the coverage assignment is largely handwaved, it's still helpful.
[13:32] Morgaine Dinova: neat*
[13:32] Enus Linden: Fisher will do a lot of the grunt work :P
[13:32] Fisher Linden: lol...yummy!
[13:32] Fisher Linden: We love coverage analysis too
[13:32] Enus Linden: latif: latency tests YES
[13:32] Morgaine Dinova gives Fisher a cookie
[13:32] Fisher Linden: nomnomnom
[13:33] Morgaine Dinova: lol
[13:33] Enus Linden: so perf/load/benchmark testing....
[13:33] Enus Linden: all part of the same equation
[13:33] Enus Linden: we've talked about my intent to use pyogp on ec2 against a test env before
[13:33] Dedric Mauriac: don't the methods in theLSL language already have automated testing in them wiht each build?
[13:33] Enus Linden: that's on the automation team's roadmap to facilitate
[13:34] Enus Linden: and i need to plug it into QA's
[13:34] Enus Linden: tbh, I need to hire someone to drive it. I can only drive it so much
[13:34] Enus Linden: i've started. I have test pyogp debs for our system image
[13:34] Latif Khalifa: you can do some serious ddos with ec2. i mean load testing :P
[13:34] Morgaine Dinova: haha
[13:34] Enus Linden: latif: yeah
[13:34] Enus Linden: 1st goal: simulate peak concurrency on a grid
[13:34] Fisher Linden: Really looking forward to ec2 testing
[13:35] Rakesh Linden: ec2 will definitely play a big part in future at Linden
[13:35] Dedric Mauriac: I would attempt to simulate more than the peak. mabe 150% peak
[13:35] Enus Linden: 2nd goal: 2x peak
[13:35] Enus Linden: then 4x
[13:35] Dedric Mauriac: ah, good goals
[13:35] Enus Linden: and then til stuff falls apart and i break the whole grid
[13:35] Latif Khalifa: with libomv you could do prolly 200-300 clients per instance
[13:35] Enus Linden: latif: on windows yeas
[13:35] Latif Khalifa: don't know how much pyogp can scale
[13:35] Fisher Linden: We will see!
[13:36] Enus Linden: pyogp can handle 80
[13:36] Enus Linden: becaus eof eventlet
[13:36] Dedric Mauriac: try simulating 1500 people in one sim. lol
[13:36] Enus Linden: eventlet grabs 75% of a cpu
[13:36] Enus Linden: and that's it
[13:36] Latif Khalifa: ah
[13:36] Enus Linden: could easily run on multiple cpus
[13:36] Rakesh Linden: so libomv doesn't work w Mono?
[13:36] Latif Khalifa: Rakesh it sure does
[13:36] Enus Linden: libomv does work well under mono
[13:36] Enus Linden: but mono adds resource overhead
[13:36] Enus Linden: at least on our eniacs i can only run 100 clients
[13:37] Enus Linden: on my windows desktop i easily run 400
[13:37] Rakesh Linden: i c
[13:37] Enus Linden: 9eanic = dev box w/ system image like that of a prod host)
[13:37] Latif Khalifa: Enus mono 2.6 got a lot better. some memleaks and cpu usage problems we had disappeared
[13:37] Enus Linden: ooohhh
[13:37] Enus Linden: we still only have 1.9
[13:37] Enus Linden: will re-investigate
[13:37] Latif Khalifa: if you ever want to test it, go with 2.6
[13:38] Latif Khalifa: i've seen significant improvements
[13:38] Enus Linden: easy to add a new version to a host, but wont update the sys image package anytime soon
[13:38] Enus Linden: thanks for the tip
[13:38] Rakesh Linden: esp since we use Mono for LSL interpreter
[13:38] Dedric Mauriac: so besides havok, we now need to upgrade mono. heh.
[13:38] Morgaine Dinova: That's extremely bad news if eventlet only scales that far. Stackless scales massively according to EVE, but I don't know what "massively" translates to for SL work. Hmmmm ... :-(
[13:38] Latif Khalifa: my test with 20 clients went from 60% cpu to 7%
[13:38] Enus Linden: woah
[13:38] Enus Linden: eventlet is odd in that way morgaine
[13:38] Dedric Mauriac looks for a JITA region
[13:38] Latif Khalifa: and memory usage was flat
[13:39] Enus Linden: so, back to distributed load testing
[13:39] Morgaine Dinova: Latif, can't the Mono thread pool be extended to hundreds and thousands?
[13:39] Enus Linden: vision: test automation harness (we're going to use TeamCity, will get url in a sec) to spawn a bajillion test agent hosts on ECS, distributed in various hosting facilities in Europe, Aisie, NA, etc
[13:40] Morgaine Dinova: s/can't/can/
[13:40] Enus Linden: those agents can be driven via a web hook to run specific tests using specific accounts against a staging env that very closely matches agni
[13:40] Dedric Mauriac: http://www.jetbrains.com/teamcity/
[13:40] Enus Linden: this staging env is also in the works, but is a separate team's deliverable
[13:41] Enus Linden: the agents shoudl act very very similar to real agents in world
[13:41] Dedric Mauriac: real scripted agents
[13:41] Enus Linden: and we should see cimilar metrics of various kinds to what we see in production
[13:41] Enus Linden: similar*
[13:41] Enus Linden: ty for the link dedric
[13:42] Enus Linden: once we are happy that we ahve a useful test config, we ramp it up
[13:42] Dedric Mauriac: i've used ccnet in the past, but have had someone else recommend team city to me as well.
[13:42] Latif Khalifa: we also use teamcity for automated testing in libomv (on a much lower scale of course) :)
[13:42] Enus Linden: and watch things fall over
[13:42] Enus Linden: fix
[13:42] Enus Linden: repeat
[13:43] Enus Linden: it's not just this sort of testing we're targetting tho
[13:43] Enus Linden: it's far simpler to enable testing susystems
[13:43] Enus Linden: sub*
[13:43] Enus Linden: web services, util.agni services, web apps, etc
[13:44] Enus Linden: they will come first. it's kind of nice, seeing the roadmap and it's building blocks
[13:44] Dedric Mauriac: most awesome app - http://www.avatarsunited.com/apps/helloworld
[13:44] Latif Khalifa: i could imagine. it woudl still be helpful to have a way to test things on production level loads. things have a nasty habit of breaeking at high load that work well under subsystem test
[13:45] Enus Linden: latif: absolutely
[13:45] Enus Linden: case in point: I laod testing avatars unites
[13:45] Enus Linden: we all know how well that worked out
[13:45] Latif Khalifa: hehehe
[13:45] Enus Linden: due to timing, could only test from a handful of angles
[13:45] Enus Linden: we *did* see the breaking point
[13:45] Enus Linden: called it out
[13:45] Latif Khalifa: is it RoR?
[13:46] Enus Linden: and simply didnt expect the uptake to be what it was
[13:46] Dedric Mauriac: I log in and see about 100 bithdays every day on the home page now
[13:47] Enus Linden: being able to test comprehensively is what this year will be all about
[13:47] Dedric Mauriac: Enus, i'm proud that you are moving in that direction.
[13:47] Morgaine Dinova: Dedric: AU's webite work pretty poorly. Your URL is another that fails -- I just get the general app/ directory
[13:47] Sea Urchin beanbag: Going to next texture.
[13:47] Enus Linden: unit, functional, subsystem, end to end, at scale, for every product and app we handle
[13:47] Dedric Mauriac: ah, i didn't publish the app yet. maybe that is why
[13:47] Enus Linden: such a cynic morgaine ;)
[13:47] Dedric Mauriac: it's just me testing a very simple "hello world" app
[13:47] Morgaine Dinova: Me? Never!
[13:47] Enus Linden: well, 70 should have been 60
[13:48] Enus Linden: oops, ww
[13:48] Al Sonic: Hello, United Avatars!
[13:48] Dedric Mauriac: ah yea, i should upgrade it to say that instead. heh.
[13:48] Enus Linden: i wonder if this OH should shift to a more general QA and Automation orientation
[13:49] Dedric Mauriac: :)
[13:49] Enus Linden: getting back to morgaine's question about it's usefulness
[13:49] Enus Linden: it was a vehicle for pyogp convos and ogp convos for a wihle
[13:49] Enus Linden: while
[13:49] Enus Linden: perhaps it's time to re-orient things
[13:49] Dedric Mauriac: Has anything became apparent using automated testing that you would have never found on your own?
[13:50] Enus Linden: some bottlenecks in load testing i've done is surprising
[13:50] Enus Linden: the way sims work (or dont) is often surprising
[13:50] Enus Linden: libomv knows all about that
[13:50] Enus Linden: ;)
[13:50] Dedric Mauriac: the nameless delagger is Enus
[13:50] Latif Khalifa: ;)
[13:51] Morgaine Dinova: Resi's are beyond surprise. ;-) [Working on that cynicism here :P]
[13:51] Dedric Mauriac: or is it, delagulator
[13:51] Latif Khalifa: i actually often wonder what makes sim choke with over 50-60 agents. is it cpu, io saturation, etc
[13:51] Enus Linden: right now, the focus w/in automation will be building regression test suites, and automation coverage on new projects
[13:52] Enus Linden: latif: io and then cpu (script time often)
[13:52] Dedric Mauriac: with automated testing, i'm sure you can make charts to help identify what happens
[13:52] Fisher Linden: hmm....charts ....
[13:52] Enus Linden: dedric: we'll have thousands of pretty charts
[13:52] Enus Linden: literally
[13:52] Latif Khalifa: Enus, script time should be throttled
[13:52] Enus Linden: maybe tens of them
[13:52] Enus Linden: latif, should be, is, still sucks
[13:52] Enus Linden: WIP
[13:52] Dedric Mauriac: you sould let us see some - even if they are really old. us geeks need eye candy.
[13:52] Latif Khalifa: i mean mono gets it's time slice and that's it
[13:53] Enus Linden: dedric, in progress.
[13:53] Enus Linden: I plan to publish publicly
[13:53] Enus Linden: at least some things
[13:53] Latif Khalifa: but i could imagine doing high bandwidth network with 60+ clients can be resource intensive on the sim
[13:53] Dedric Mauriac: The article about the MySQL upgrade was the most intriging thing I saw this year on the blogs.
[13:53] Enus Linden: in fact, will start to talk to the auto team abuot building OS style collab on test case authoring and input
[13:54] Enus Linden: derric: that was rad. loved it
[13:54] Enus Linden: it was clearly well received
[13:54] Morgaine Dinova: Did you see this Enus? Was very interesting reading -- http://arstechnica.com/business/data-centers/2010/02/what-second-life-can-teach-all-companies-about-scaling-web-apps.ars
[13:54] Enus Linden: i glanced at page 2 morgaine XD
[13:54] Enus Linden: it's open and pending a read
[13:54] Morgaine Dinova: Hehe
[13:55] Enus Linden: (first link i got to it was to page 2)
[13:55] Youri Ashton: thanks for the meeting enus!
[13:55] Youri Ashton: bye bye all
[13:55] Enus Linden: my pleasure!
[13:55] Enus Linden: ciao youri
[13:55] Enus Linden: see ya laters
[13:55] Morgaine Dinova: Cyu Youri :-)
[13:55] Latif Khalifa: it was interesting to read how people scale up things
[13:56] Latif Khalifa: like handling enourmous read loads
[13:56] Enus Linden: scaling is a focal point here atm. last year we spent a lot of time re-architecting things, with good impact
[13:56] Latif Khalifa: a lot of it seems to be, use db's as key value store
[13:56] Latif Khalifa: and memcache the hell out of it
[13:56] Latif Khalifa: joins will kill you
[13:56] Enus Linden: amen latif, is a good model in many cases
[13:57] Enus Linden: (AU would benefit there)
[13:57] Morgaine Dinova: Hmmmm ... not "with good impact" from the resi's PoV, Enus. And that's not cynicism I'm afraid.
[13:57] Latif Khalifa: read a but about facebook infrastructure
[13:57] Enus Linden: Morgaine: think back to last january
[13:57] Enus Linden: and the months leading up
[13:57] Latif Khalifa: they handle 150k pageviews a sec
[13:57] Latif Khalifa: and they have 35TB memcached cluster
[13:57] Morgaine Dinova: Yeah, Facebook is great with articles and presentations on their internals.
[13:57] Latif Khalifa: which they have hacked to use UDP
[13:57] Enus Linden: we fell over at 3pm PSt 3 out of every 7 days
[13:58] Latif Khalifa: since it's all iternal net, and ultra low latency
[13:58] Latif Khalifa: they claim to get <1ms latency from memcached over udp on local net
[13:58] Latif Khalifa: which is pretty awesome
[13:58] Morgaine Dinova: Latif: notice that though they went back to UDP, they had to create their own reliability layer on top of it, and they said that was utter murder.
[13:59] Enus Linden: eep, almost 2. i gotta run.
[13:59] Enus Linden: THANKS everyone
[13:59] Morgaine Dinova: KK, Cyu Enus, take care :-)
[13:59] Rakesh Linden: ty Enus
[13:59] Fisher Linden: Thanks, Enus. :-)
[13:59] Dedric Mauriac: Thanks enus.
[13:59] Enus Linden: note: next week I will miss this time slot
[13:59] Latif Khalifa: take care enus
[13:59] Enus Linden: i hope to have a stand in
[13:59] Enus Linden: this was a nice caht
[13:59] Fisher Linden: Spider!
[13:59] Enus Linden: maybe i'll get the auto team to front someone to talk more
[14:00] Morgaine Dinova: woot!
[14:00] Enus Linden: exactly fisher
[14:00] Dedric Mauriac: have an ajenda next time
[14:00] Enus Linden: this went well w/o
[14:00] Enus Linden: ;)
[14:00] Enus Linden: will try
[14:00] Enus Linden: ciao!