User:Which Linden/Office Hours/2010 Feb 4
Enkidu hosted my office hours this week because I was unexpectedly sick. Here is a transmission that we found wedged into the knothole of a tree near the last coordinates transmitted by the starship's emergency beacon.
The wikifier failed, not really sure why, too lazy to figure it out. Still legible.
- [11:03 ] Xugu Madison: Enkidu!
- [11:03 ] Enkidu Linden: hello!
- [11:04 ] Xugu Madison: How are you doing?
- [11:04 ] Les Linden: hey enkidu -- RDW OH?
- [11:04 ] Enkidu Linden: i'm well
- [11:04 ] Enkidu Linden: yeah, les
- [11:04 ] Les Linden: word... i'll ping
- [11:04 ] Enkidu Linden: now we just need which
- [11:04 ] Tillie Ariantho: Hello! :)
- [11:04 ] Xugu Madison: Xugu Madison points Enkidu at the sign "He's ill :("
- [11:05 ] Les Linden: ahhhh
- [11:05 ] Les Linden: indeed
- [11:05 ] Les Linden: poor which
- [11:05 ] Xugu Madison: Well, wow, we've got three Lindens. Les,
haven't met you before, what do you do at the lab?
- [11:06 ] Enkidu Linden: oh, it was behind me
- [11:06 ] Les Linden: hi xugu - i'm a program mgr in global tech!
- [11:06 ] Tillie Ariantho: Nothing to discuss without Which? ^^
- [11:06 ] Les Linden: not on my side... actually frees me up for
another mtg. see you guys!!!
- [11:06 ] Enkidu Linden: bye les
- [11:06 ] Tillie Ariantho: What' global tech? Like... uh... loading
windlight settings? ,D VWR-16769
- [11:06 ] JIRA-helper: http://jira.secondlife.com/browse/VWR-16769
- [#VWR-16769] Windlight: Settings get loaded at startup... all of them
- [11:06 ] Xugu Madison: Oh dear. See you Les!
- [11:07] Xugu Madison: Enkidu, how are you settling into the lab?
- [11:07] Sasy Scarborough: hi all
- [11:07] Tillie Ariantho: Hello Sasy. :)
- [11:07 ] Enkidu Linden: very well. it's been over a year now
- [11:07 ] Tillie Ariantho: Which is ill, but we have Enkidu here. =)
- [11:08 ] Mojito Sorbet: Could also do "Which is on first" jokes
- [11:08 ] Enkidu Linden: yeah, i'm here! we don't need that guy
- [11:08 ] Tillie Ariantho: So what are you working on usually, Enkidu?
Dont say 'secret projects' now. :P
- [11:09 ] Enkidu Linden: well, i work in the web group
- [11:09 ] Xugu Madison: Web design, applications, services, some
combination of all, or something else?
- [11:09 ] Tillie Ariantho: Ah =)
- [11:10 ] Tillie Ariantho: So xStreet, migration of the forums, blog,
profiles, all that?
- [11:10 ] Enkidu Linden: actually
- [11:10 ] Tillie Ariantho: Hello Aimee, bye Aimee. :P
- [11:10 ] Enkidu Linden: none of that at all
- [11:10 ] Aimee Linden: lol
- [11:10 ] Tillie Ariantho: Enkidu: ah.
- [11:11 ] Enkidu Linden: over the past year most of my work has been on secondlife.com
- [11:11 ] Tillie Ariantho: Ah darn chatrange thingy...
- [11:11 ] Tillie Ariantho: Enikdu: so you did the fully animated
homepage with all the flash bits? ^^
- [11:11 ] Aimee Linden: Aimee Linden grins, I'm winding it up
- [11:11 ] Sasy Scarborough: the new page is pretty, about becoming a premmy
- [11:11 ] Enkidu Linden: haha
- [11:11 ] Enkidu Linden: no i'm afraid i had nothing to do with that either
- [11:12 ] Tillie Ariantho: ugh. ^^
- [11:12 ] Enkidu Linden: my projects tend to be a little farther back
- [11:12 ] Tillie Ariantho: so you configure the apache to still work
even when Evil Linden puts up flash bits? :P
- [11:12 ] Sasy Scarborough: Sasy Scarborough still wonders what it
means that they will have special build access on mainland
- [11:12 ] Xugu Madison: Don't suppose you're at all involved with the
single sign-on stuff Enkidu?
- [11:12 ] Enkidu Linden: for example, a few months ago which and i worked
together to improve the transactions history page
- [11:13 ] Tillie Ariantho: ah =)
- [11:13 ] Tillie Ariantho: Okay, that part of the web. :)
- [11:13 ] Xugu Madison: Love the new transaction history page
- [11:13 ] Mojito Sorbet: Reducing the unneeded white space between enties?
- [11:13 ] Aimee Linden: I don't think Lindens are allowed to Flash, at
least not in a PG area
- [11:13 ] Xugu Madison: Still needs to be 3 months, not 1, but what can you do..
- [11:13 ] Enkidu Linden: thanks. i can do very little about that
- [11:13 ] Xugu Madison: Flash? Wait, what?
- [11:13 ] Enkidu Linden: although i do agree
- [11:13 ] Tillie Ariantho: So bringing the backend to the front stuff. ^^
- [11:14 ] Xugu Madison: Enkidu, tell them I'll go premium if they make
it three months
- [11:14 ] Tillie Ariantho: Xugu: the main page with all the animated
boxes some time ago I meant. ,)
- [11:14 ] Enkidu Linden: haha, awesome
- [11:14 ] Enkidu Linden: i will pass that along
- [11:14 ] Saijanai Kuhn: tremenouds lag can't even type let me relog
- [11:14 ] Sasy Scarborough: paying for a year is the best way to go :)
they pay you in the end
- [11:15 ] Xugu Madison: I'll admit, I can't see them going "Xugu will
go premium? Make it so!", but I live in hope :)
- [11:15 ] Tillie Ariantho: haha
- [11:15 ] Enkidu Linden: i don't mind saying that
- [11:16 ] Xugu Madison: Will it help, though? :)
- [11:16 ] Xugu Madison: but go for it, it'll do my ego good
- [11:16 ] Enkidu Linden: "get your hardhats, people! it's go time!"
- [11:16 ] Xugu Madison: :-D
- [11:16 ] Xugu Madison: Reminded randomly... anyone heard more about
the creative conference LL are arranging? This question goes to the Lindens too
- [11:16 ] Xugu Madison: ...the Linden, Aimee fled :(
- [11:16 ] Mojito Sorbet: Premium accounts get special no-lag acees; you
didnt know that?
- [11:17 ] Enkidu Linden: i don't know anything about it, unfortunately
- [11:17 ] Aimee Linden: was just relogging :)
- [11:17 ] Xugu Madison: Yay, Aimee back!
- [11:17 ] Sasy Scarborough: Sasy Scarborough thinks her premmy account
is broken then :P
- [11:17 ] Sasy Scarborough: welcome back Aimee
- [11:18 ] Tillie Ariantho: Mojito: I have 2 premiums, and I dont feel
like I am especially unlagged. :P
- [11:18 ] Xugu Madison: Sasy, it's a toggle. I think it's
ctrl-alt-opt-shift-a while hopping on your left foot...
- [11:18 ] Sasy Scarborough: lol
- [11:18 ] Enkidu Linden: so, did you guys want to talk about web stuff?
- [11:18 ] Sasy Scarborough: make it a gesture :P
- [11:18 ] Xugu Madison: Enkidu, sure!
- [11:19 ] Xugu Madison: Not.. entirely sure where I'd start, but web is good!
- [11:19 ] Mojito Sorbet: You must have missed the email where they tell
you the secret access URL...
- [11:19 ] Saijanai Kuhn: this is still laggy, even for which's place
- [11:19 ] Tillie Ariantho: Sure, why not talk about web stuff. :)
- [11:20 ] Xugu Madison: Done anything excitingly webby this week Enkidu?
- [11:20 ] Xugu Madison: Anything you can tease us by not being able to
tell us? :)
- [11:20 ] Enkidu Linden: totally!
- [11:20 ] Enkidu Linden: i love teasing you guys
- [11:20 ] Tillie Ariantho: :D
- [11:21 ] Sasy Scarborough: avatars united is webby i think are you
involved in the behind the scenes of that too ?
- [11:21 ] Enkidu Linden: lots of stuff going on in our group
- [11:21 ] Enkidu Linden: yes, AU is ours now
- [11:21 ] Sasy Scarborough: I like it :)
- [11:21 ] Sasy Scarborough: Sasy Scarborough hates Plurk loves AU
- [11:21 ] Enkidu Linden: i'll be very loosely involved
- [11:22 ] Xugu Madison: Plurk always seemed like a giant heap of
- [11:22 ] Enkidu Linden: i got to meet several of the former enemy unknown staff,
now lindens, and they seem really great
- [11:22 ] Sasy Scarborough: is it going to be widget heavy so you can
shout in one network and have it yell in all ?
- [11:22 ] Aimee Linden: I like the time line idea with plurk though
- [11:22 ] Tillie Ariantho: AU is nice and clean, only it needs a bit of
useful stuff now. :)
- [11:23 ] Sasy Scarborough: yes its a great format Aimee just
unfortunatly doesnt stop people being trolls or incredibly cruel to one another
- [11:23 ] Mojito Sorbet: Watch out for scalability though.
- [11:23 ] Tillie Ariantho: Just connecting to everyone and grouping
with everyone is just a start for bigger things.
- [11:25 ] Tillie Ariantho: Yah, I've seen. Any nice new stuff in there? ^^
- [11:25 ] Enkidu Linden: yeah, focus on performance
- [11:26 ] Enkidu Linden: honestly, the performance stuff is the whole cake for me
- [11:26 ] Xugu Madison: We're finally accepting that we're going to
- [11:26 ] Enkidu Linden: i'm really excited to see the impact upgrading has on
our web properties' user experience
- [11:27 ] Xugu Madison: Xugu Madison nods "Latency is a nightmare"
- [11:27 ] Xugu Madison: (in general terms, I mean, not SL-specific)
- [11:28 ] Enkidu Linden: not that we don't have room to improve there
- [11:28 ] Xugu Madison: It feels a bit like you're trying to do too
much on some pages. I don't know... again, bit of a web luddite, so might be biased
- [11:29 ] Enkidu Linden: what pages are you thinking of?
- [11:29 ] Tillie Ariantho: Xugu, today everyone tries to do a lot on
the web front... with Web 2.0 features on that many pages some might just feel as if they lose the race. ^^
- [11:30 ] Sasy Scarborough: Sasy Scarborough has to dash , have fun all
- [11:30 ] Tillie Ariantho: So trying to add shiny web2.0 to get to the
front row again, but forgetting their actual content. ^^
- [11:30 ] Enkidu Linden: bye, sasy
- [11:30 ] Mojito Sorbet: I was just reading a paper titled something
like "Social networking killed the database"
- [11:31 ] Mojito Sorbet: It was about how tradional LAMP ways of
looking the problems, and SQL, could never scale up to what is needed for social networking
- [11:31 ] Enkidu Linden: indeed
- [11:31 ] Tillie Ariantho: Some websites just feel like a web2.0
showcase... inplace-edit, drag'n'drop, animated stuff... but nothing useful behind it. ^^
- [11:31 ] Tillie Ariantho: I am not talking about the SL pages now.
- [11:32 ] Mojito Sorbet: And more AJAX than is really needed.
- [11:32 ] Xugu Madison: Mojito, which parts wouldn't scale?
- [11:32 ] Mojito Sorbet: The database itself, mostly. Wait, I look up the URL
- [11:32 ] Enkidu Linden: yeah, database becomes a huge bottleneck
- [11:33 ] Tillie Ariantho: How does google do it with their million servers...?
- [11:33 ] Mojito Sorbet:
- [11:33 ] Enkidu Linden: interestingly, we ran into this relatively early in our
history (as i understand it), not because of social networking, but because running a virtual world is also db-expensive
- [11:33 ] Mojito Sorbet: Google does NOT use relational databases, for one thing
- [11:34 ] Xugu Madison: Is it a case of too much data, or just patterns
not well suited to relational databases?
- [11:34 ] Mojito Sorbet: Both
- [11:34 ] Enkidu Linden: yeah, rdbms basically fails at scale
- [11:34 ] Mojito Sorbet: Let me give you another one that hits closer to home...
- [11:34 ] Mojito Sorbet: A paper at Ars Technica, "What Second Life can
teach your datacenter about scaling Web apps"
- [11:35 ] Tillie Ariantho: Isnt there some description somewhere how
the general infrastructure of google looks like?
- [11:35 ] Enkidu Linden: ooo i should read that
- [11:35 ] Tillie Ariantho: ^^
- [11:35 ] Mojito Sorbet: Writeen buy a guy who USED to work for Linden Labs
- [11:35 ] Xugu Madison: It was really interesting to realise SL is
moving to web services, actually. Not because they're the fastest option, but because you can actually do things like scaling, practically.
- [11:36 ] Mojito Sorbet:
- [11:36 ] Xugu Madison: Xugu Madison tries not to think too hard about
the infrastructure of Google, and is mostly waiting until it breaches the self-awareness threshhold
- [11:36 ] Enkidu Linden: oh yeah, ian
- [11:36 ] Tillie Ariantho: =)
- [11:37 ] Mojito Sorbet: If you google the term "nosql" you will find
all sorts of interesting papers
- [11:37 ] Tillie Ariantho: Xugu: :P Maybe at some time SL will migrate
from a virtual world to virtual life, too. :P
- [11:37 ] Tillie Ariantho: or better, evolve.
- [11:37 ] Enkidu Linden: i was talking to a friend who is a facebook engineer,
and he was telling me they've run into exactly the same problems
- [11:37 ] Xugu Madison: Great. The SL overmind and the Google megabrain
fighting over the burning remains of the planet. Just what we need :)
- [11:37 ] Tillie Ariantho: And how did they fix it? ^^
- [11:38 ] Xugu Madison: Xugu Madison is rather glad they've got only
about 7,000 users.
- [11:38 ] Tillie Ariantho: Dont say "We started to kill all the virtual
people from facebook and halved our userbase by this." :P
- [11:38 ] Enkidu Linden: by switching to services and using db as little more
than a name-value store
- [11:39 ] Tillie Ariantho: Hm, is there a way for LL to do that, too?
- [11:39 ] Enkidu Linden: i bet if you read ian's ars technica paper, you'll see
the same recommendation
- [11:39 ] Enkidu Linden: sure! we have done that quite a lot
- [11:40 ] Xugu Madison: How are databases doing with Moore's law, BTW?
Do they scale well with hardware?
- [11:40 ] Tillie Ariantho: Means you would just drop all assets from DB
to organized filesystem and only store access keys in the DB?
- [11:41 ] Mojito Sorbet: Databases scale with hardware if they are built right.
- [11:41 ] Mojito Sorbet: For example, Oracle scales quite a lot, but
you end up buying very expensive hardware.
- [11:41 ] Enkidu Linden: i can't speak to that example because i don't know
enough about our asset system
- [11:42 ] Mojito Sorbet: A key/value store scales much better, using
cheap hardware, according to all the papers I have read
- [11:42 ] Enkidu Linden: yep
- [11:42 ] Mojito Sorbet: Look up names like "hadoop" and "couchDB"
- [11:42 ] Tillie Ariantho: But I guess it is something like that.
Getting out the big data blogs and only store the way smaller keys, which are way better cacheable then, too.
- [11:42 ] Xugu Madison: Mojito, was more thinking if our user base
scales as about Moore's law (it does), we shouldn't need significantly more expensive hardware to keep up... right?
- [11:42 ] Mojito Sorbet: And work hard to avoid JOINs
- [11:43 ] Xugu Madison: Xugu Madison likes JOINs....
- [11:43 ] Mojito Sorbet: Xugu, only oif there are no bottlenecks
- [11:43 ] Enkidu Linden: i also love joins
- [11:43 ] Enkidu Linden: but that's the price of them
- [11:43 ] Mojito Sorbet: Read that first paper, about social media
killing the RDBMS, and follow links
- [11:44 ] Mojito Sorbet: There is a technique called "map reduce" that
google uses a lot. A way of distributing query load
- [11:44 ] Enkidu Linden: if you join in a service instead, you can move the service around
- [11:44 ] Tillie Ariantho: Are there any databases done in hardware?
... hm... uh... harddisc are behaving like databases, right? ^^ In a way.
- [11:44 ] Mojito Sorbet: Not these days.
- [11:44 ] Xugu Madison: Tillie, I think Oracle has an option to store
directly into a partition...
- [11:44 ] Mojito Sorbet: These days you need all sorts of replication,
dynamic failover, and things that are far beyond the capabilities of those old DB-Hardware ideas of 20 years ago
- [11:44 ] Tillie Ariantho: I mean you have a key to access stuff, and
it has a cache for often used entries ...
- [11:45 ] Mojito Sorbet: Oracle has a bare-disk support feature you can
use. Bypasses the OS filesystem
- [11:45 ] Mojito Sorbet: But Oracle's pricing model puts it out of
range of anyone without deep pocketys.
- [11:46 ] Xugu Madison: We primarily still suffer from the code
architecture being horrendously inefficient with number of queries, so that's where we're focusing for speedups
- [11:46 ] Enkidu Linden: yes
- [11:46 ] Mojito Sorbet: On the other hand, Oracle also owns BerkeleyDB
(nee SleepyCat), a key/value DB
- [11:46 ] Tillie Ariantho: Xugu: who is we, what are you working on? :)
- [11:46 ] Enkidu Linden: nothing ruins a perfectly good computer like software
- [11:47 ] Tillie Ariantho: haha
- [11:47 ] Xugu Madison: Tillie; distributed systems group at St Andrews
university. Internal web application for adminstrating courses. We're selling if anyone's buying :)
- [11:47 ] Tillie Ariantho: ah =)
- [11:47 ] Enkidu Linden: is that a competitor of blackboard?
- [11:48 ] Enkidu Linden: shoot, brb
- [11:48 ] Xugu Madison: It's... similar. Except no-one's nicknamed our
stuff "Blackhole" yet
- [11:48 ] Tillie Ariantho: haha
- [11:48 ] Tillie Ariantho: 'yet' :P
- [11:49 ] Xugu Madison: It's primarily focused on the admin side,
though, rather than content delivery. Storing, tracking, and managing marks. Particularly is responsible for reporting final marks up to the institution now. Finished our first semester where we're used university wide, on Wednesday
- [11:49 ] Xugu Madison: Tillie; the project is called MMS (Module
Management System). We did here it called Mess a few times
- [11:50 ] Tillie Ariantho: ,)
- [11:50 ] Enkidu Linden: k
- [11:50 ] Xugu Madison: got a survey out to the users tonight, waiting
on responses. So far, mostly people seem really impressed
- [11:50 ] Xugu Madison: Also, distressingly shocked that the university
brought in new procedures and they actually helped
- [11:50 ] Tillie Ariantho: Enkidu probably missed a meeting and just
made a coffee for M as excuse? :D Ah, Enkidu back. ;)
- [11:51 ] Enkidu Linden: hey now!
- [11:51 ] Tillie Ariantho: ^^
- [11:51 ] Xugu Madison: ...Enkidu makes coffee for NO-ONE!?
- [11:51 ] Tillie Ariantho: I don't know how it is working over there at
Linden Labs, Enkidu, you tell us! :P
- [11:51 ] Enkidu Linden: i'm working at home today and my daughter had a big fall
- [11:51 ] Xugu Madison: Yikes! She okay?
- [11:52 ] Xugu Madison: (Although, I'd hope so if you're back at keyboard)
- [11:52 ] Enkidu Linden: yeah, she's made of rubber
- [11:52 ] Xugu Madison: Useful
- [11:52 ] Enkidu Linden: well, we have five minutes, so i can talk about working here
- [11:52 ] Enkidu Linden: or more web stuff
- [11:52 ] Xugu Madison: About working there sounds good
- [11:53 ] Aimee Linden: we're meant to work? DAMN didn't see that on
the job description
- [11:53 ] Xugu Madison: Xugu Madison just opened that Ars Technica
article "Wait, LL data centres are managed by floating robots? Oh no..."
- [11:53 ] Enkidu Linden: ok!
- [11:53 ] Tillie Ariantho: me thought about working at a company like
LL for a second but they a) probably wouldnt pay what I get now, b) I hate to move and c) my current job isnt that bad right now. ^^
- [11:54 ] Enkidu Linden: let's talk about your C
- [11:55 ] Tillie Ariantho: oh, and D) I probably have nothing to offer
what they would need. :)
- [11:55 ] Enkidu Linden: i've never had that thought working here
- [11:55 ] Tillie Ariantho: I'm working in the IT of a bank... building
stuff together that our devs write, and deplying it to WebSphere servers ... banking portals with some 2 million customers. ^^
- [11:56 ] Tillie Ariantho: That was by c). ,)
- [11:56 ] Mojito Sorbet: What concurrency?
- [11:56 ] Enkidu Linden: it's a cool feeling knowing what you do impacts so many
people, isn't it?
- [11:56 ] Tillie Ariantho: Mojito: 30,000 session in the high time.
- [11:56 ] Tillie Ariantho: sessions
- [11:57 ] Tillie Ariantho: It's 8 websphere instances on 2 Linux
machines, 128GB ram each.
- [11:57 ] Tillie Ariantho: But we can handle that with 1 instance on 1
machine, we tried. :)
- [11:57 ] Enkidu Linden: pretty good for 2 machines!
- [11:57 ] Tillie Ariantho: So 7 are more or less to distribute the load a bit. ^^
- [11:58 ] Tillie Ariantho: That's Intel Quadcores, some XEONs, I don't
know the details, 2,6GHz or something.
- [11:59 ] Mojito Sorbet: http://www.sics.se/~joe/apachevsyaws.html
- [11:59 ] Mojito Sorbet: That has an interesting chart showing how
various web servers handle high concurrency
- [11:59 ] Tillie Ariantho: We were testing with a Sun Niagara with 64
threads, but that wasnt better, the Linux systems rulored them. :P
- [11:59 ] Xugu Madison: It's always really cool when you get everything
onto 1-2 machines. So much less distribution work
- [11:59 ] Mojito Sorbet: "rulored"?
- [11:59 ] Tillie Ariantho: Before we were on 8 Sun Fire 480Rs ,)
- [12:00 ] Tillie Ariantho: with Solaris 8
- [12:00 ] Tillie Ariantho: Mojito: were way better. ,)
- [12:00 ] Xugu Madison: We've apparently got... part of a Sun-Fire V440
shared with the main web server
- [12:00 ] Tillie Ariantho: The 64 threads of the niagara didnt help,
but the higher core speed.
- [12:00 ] Tillie Ariantho: of the linux machines
- [12:00 ] Mojito Sorbet: To take advantage of lots of cores, you have
to program properly for that environment
- [12:01 ] Mojito Sorbet: http://www.sics.se/~joe/images/apachevsyaws.jpg
- [12:01 ] Enkidu Linden: wow, i just read that
- [12:01 ] Tillie Ariantho: Our web servers are on 2 machines like that, too. ^^
- [12:01 ] Tillie Ariantho: 16 instances of apache.
- [12:01 ] Tillie Ariantho: 8 on each machine.
- [12:02 ] Mojito Sorbet: Apache dies at about 4,000 threads in that
picture. YAWS keeps on goingat 80,000
- [12:02 ] Mojito Sorbet: Linux threads are too high overhead.
- [12:03 ] Tillie Ariantho: I dont have any numbers/diagrams at hand,
sorry, all bank internal stuff.
- [12:03 ] Mojito Sorbet: YAWS is written in Erlang, which has extremely
- [12:04 ] Enkidu Linden: i need to learn erlang
- [12:04 ] Mojito Sorbet: It great fun, once the shock-horror wears off
- [12:04 ] Tillie Ariantho: Our application servers are pretty bored
most of the time. :) Only the database (informix 9 on solaris) has like 60% load.
- [12:04 ] Enkidu Linden: we've been using tsung here for some load testing, and
it's in erlang
- [12:04 ] Xugu Madison: Yeah, I hear about Erlang a lot...
- [12:04 ] Mojito Sorbet: No loops, no variables, no shared anything
- [12:05 ] Enkidu Linden: hey guys, i'm afraid i need to go work on something
- [12:05 ] Xugu Madison: Take care Enkidu, thanks for coming!
- [12:05 ] Tillie Ariantho: bah, work is overrated. :P
- [12:05 ] Enkidu Linden: thanks for sticking around though!
- [12:05 ] Xugu Madison: Same to you Aimee!
- [12:05 ] Aimee Linden: :)
- [12:05 ] Xugu Madison: and of course, everyone else
- [12:05 ] Enkidu Linden: see you next week!