User:Zero Linden/Office Hours/2007 Mar 22

Transcript of Zero Linden's office hours:

[7:32]	Xanshin Paz:	woot!
[7:32]	Zha Ewry:	I see has high as 40, sometimes, but... Not here. Weird
[7:32]	Zha Ewry:	Hey, Zero!
[7:32]	Zero Linden:	Good morning all
[7:32]	Xanshin Paz:	my 7600 does 40 whne the grid isn't sucking.....
[7:32]	Tree Kyomoon:	hello zero!
[7:32]	Xanshin Paz:	ahem, hi Zero!
[7:33]	Zha Ewry:	The hard core group here, today :-)
[7:33]	Zero Linden:	don't worry -- I complain the grid sucks all the time!
[7:33]	Xanshin Paz:	and me :)
[7:33]	Zha Ewry:	Well, Zero, I'm sure I can send someone over to try and sell you a large Z series box for it :-)
[7:34]	Zero Linden:	Ah - yes, I'm sure that's the problem - the old 486 box we run the grid on isn't really big enough.
[7:34]	Zha Ewry:	Yep
[7:34]	Xanshin Paz:	:)
[7:34]	Tree Kyomoon:	lol
[7:34]	Zha Ewry:	Well, that and 40,000 peple trying to touch assets
[7:35]	Zero Linden:	Silly - people - isn't plywood the only texture you need?!?!
[7:35]	Tree Kyomoon:	exactly!
[7:35]	Xanshin Paz:	have we hit 40K?
[7:35]	Zha Ewry contemplates plywood skirts with a shudder
[7:35]	Zha Ewry:	Not quite.
[7:35]	Zha Ewry:	39 and small change is the top, I think
[7:35]	Xanshin Paz:	I've seen 38....wow
[7:36]	Zero Linden:	The live monitor behind me in the S.F. office is off - so I can't check
[7:36]	Zero Linden:	We have this big display on the wall that flashes a series of live stats from the grid and historical charts
[7:36]	Zha Ewry:	25K at the moment. Quiet...
[7:36]	Tree Kyomoon:	Im surprised you guys dont push that in SL
[7:37]	Zero Linden:	The stats?
[7:37]	Zha Ewry:	I'd actually love to be able to pull up the basic health stats of the core boxes
[7:37]	Zha Ewry:	How many assets served, how the Isolan is doing... Traffic between colos, sort of know
[7:38]	Zha Ewry:	when I should go and read a good paper on DNS scaling, and not try to rez anything
[7:38]	Zero Linden:	The "grid tool" would be fun to put out in read only form
[7:38]	Zero Linden:	Is there a good paper on DNS scaling? I thought it was all magicians in black robes...
[7:38]	Xanshin Paz:	unless it adds to the load :)
[7:38]	Tree Kyomoon:	oh how much better these chairs look with antialiasing on!
[7:38]	Zha Ewry:	It got bad enough earlier this week, that several friends and i went and tested suff on the beta grid, where things were fast
[7:39]	Zero Linden:	Sigh...
[7:39]	Zero Linden:	Welcome Fate
[7:39]	Xanshin Paz:	do you think concurrency is getting self limiting?
[7:39]	Fate Garden:	Hi Zero
[7:39]	Zero Linden:	No, I don't think it is.
[7:39]	Xanshin Paz:	people get frustrated, and just give up....
[7:39]	Zero Linden:	True, but not as fast the pile on to the grid as a whole
[7:40]	Zero Linden:	*they
[7:40]	Xanshin Paz:	yeah
[7:40]	Zero Linden:	Okay - usual disclaimer: Everything said here will get posted to the wiki
[7:40]	Xanshin Paz:	kk
[7:40]	Zha Ewry:	I think, there must be some bottlenecks, at this point. You get odd effects, like can't rez, when local sims are still ok, otherwise
[7:40]	Zero Linden:	First, a little news
[7:40]	Zha Ewry perks up! News!
[7:40]	Zero Linden:	The first phase of the "message liberation" project passed QA
[7:41]	Zero Linden:	slide 2
[7:41]	Fate Garden claps
[7:41]	Zero Linden:	slide 1
[7:41]	Zero Linden:	this one
[7:41]	Zha Ewry:	So, Wednesday?
[7:41]	Zero Linden:	and has been merged into release-candidate, which means it will be shipping with the next full-downtime release
[7:41]	Tree Kyomoon:	great!
[7:42]	Dr Scofield:	hi there
[7:42]	Zero Linden:	The team has about a weeks worth of work left on phase 2
[7:42]	Zero Linden:	next
[7:42]	Zero Linden:	And then that will be going into QA and then release
[7:42]	Zero Linden:	So - less than a month away from "Dia de la Liberacion"
[7:42]	Sifu Moraga:	Sorry I'm late.
[7:42]	Zha Ewry:	So, you'll roll the stuff out, turned off, in the next big down.
[7:42]	Fate Garden:	are these slides on the wiki?
[7:42]	Zero Linden:	Yes and Yes
[7:43]	Fate Garden:	thankye
[7:43]	Zha Ewry:	How do we learn what you are turning on, and where, in terms of the newe messgaes
[7:43]	Zero Linden:	After phase two ships
[7:43]	Zero Linden:	we'll be able to move messages from UDP to TCP at will, live, without downtime
[7:43]	Zero Linden:	and back again if there is a problem
[7:44]	Frank Bogomil:	any latent thoughts about documentation of the full range of messages accepted by the server
[7:44]	Zero Linden:	But most importantly, the dreaded message template checksum will be gone and we'll never have to do a full down time just to add a message!
[7:44]	Frank Bogomil cheers
[7:44]	Zero Linden:	We will, for those following sldev, be fixing the block and variable ordering problem
[7:44]	Zha Ewry:	Are you going to post the current set of wha's mapped which way (UDP vs HTP) anywhere, so we can knwo what's running as you migrate?
[7:45]	Zero Linden:	Zha - that's an interesting idea -
[7:45]	Zha Ewry smiles
[7:45]	Zero Linden:	I can try to keep updates on the wiki....
[7:45]	Zero Linden:	alas, we have no automated way ....
[7:45]	Sifu Moraga:	Zero could make it a web service :-)
[7:45]	Zero Linden:	but it is an interesting idea, the control file is just XML serialized LLSD
[7:45]	Zero Linden:	would be really easy to stick on a web server!
[7:45]	Zha Ewry:	Well, just trying to see if your savy users can help, by being aware...
[7:46]	Zero Linden:	Some other feedback
[7:46]	Zero Linden:	After Tuesday's office hours, I sent out a number of e-mails to the developer team
[7:46]	Zero Linden:	letting them know what has been going on here
[7:47]	Zero Linden:	The whole company has been generally very excited that these office hours have been so well received (I assume that is why you keep coming back!)
[7:47]	Zha Ewry:	Yes, yes.
[7:47]	Xanshin Paz:	:)
[7:47]	Dr Scofield:	;-)
[7:47]	Tree Kyomoon:	yes very exciting to be here
[7:47]	Zero Linden:	In particular, the HTML/JPEG/Browser/??? on a prim discussion has been getting some traction
[7:47]	Wyn Galbraith:	Good morning. It's the only way to learn the secrets.
[7:47]	Sifu Moraga:	Yes, it is good to keep up a conversation
[7:48]	Zha Ewry:	Its very interesting to see, both the tech side, community side, and how it works in terms of collaboration
[7:48]	Zero Linden:	as you might have seen,I re-org'd the wiki pages and at least one other Linden, Kelly LInden, has begun to contribute
[7:48]	Zha Ewry nos. Noticed that.
[7:48]	Zha Ewry:	Now, if you can get the client team... really intersted.
[7:48]	Zero Linden:	I think use cases are going to be an eye opener for dev. team on this
[7:49]	Sifu Moraga feels guilt I've been posting much
[7:49]	Zero Linden:	They are interested, though the project has always seemed like a second tier one --
[7:49]	Zha Ewry nods, and tries to remember to post more of them
[7:49]	Sifu Moraga has NOT been posting much. oops
[7:49]	Zero Linden:	but I think that is partially due to not knowing exactly what it should be, and as developers do,
[7:49]	Tree Kyomoon:	you guys certainly listen to your customers more than most software companies
[7:49]	Xanshin Paz:	I think *.on aprim will be much bigger than voice
[7:49]	Zero Linden:	imaginging a much bigger temple than what needs to get built
[7:49]	Dr Scofield:	lol
[7:49]	Wyn Galbraith won't have time to catch up with wiki for a few more weeks.
[7:49]	Tree Kyomoon:	not to name names but cough ADOBE cough Macromedia COUGH
[7:50]	Xanshin Paz:	:)
[7:50]	Zha Ewry snickers
[7:50]	Dr Scofield likes that you are sharing those beihind the scenes details and are interested in our feedback
[7:50]	Tree Kyomoon:	excuse me....some bad customer service in my throat there
[7:50]	Zero Linden:	I've entered a task for the byte-range stuff with details - though I don't think it has been taken up yet - but the maintence team at least now has it on a list
[7:51]	Zero Linden:	and the in bound HTTP is getting a little more notice now that you all indicated that a) you didn't need global unique for all time URLs, and
[7:52]	Zero Linden:	b) you all are comfortable with a sunset date for XML-RPC and inbound e-mail....(or at least deprecation)
[7:52]	Dr Scofield:	is there a one-to-one mapping of SIM to IP address?
[7:52]	Zero Linden:	No, there is not for two reasons
[7:52]	Zero Linden:	1) We run upto 16 regions on a host
[7:53]	Sifu Moraga:	really!
[7:53]	Dr Scofield:	ah, ok, there goes the dyn DNS idea ;-)
[7:53]	Zha Ewry:	Low prim ones?
[7:53]	Zero Linden:	2) Regions migrate from host to host if they go down or are restarted
[7:53]	Sifu Moraga:	yup
[7:53]	Zero Linden:	well 16 iff it is a dual-core, dual-CPU machine and they are all void sims
[7:53]	Zero Linden:	normally ust 4 to a box
[7:53]	Zha Ewry:	Low prim.. yes
[7:53]	Zero Linden:	*just
[7:53]	Sifu Moraga:	ok, we had speculated whether you did that or not
[7:53]	Tree Kyomoon:	whew
[7:54]	Zero Linden:	Well, actually, the dynamic DNS has lots to reccomend it: and internally there are folks who'd like to do
[7:54]	Zero Linden:	things like
[7:54]	Sifu Moraga:	do you use virtuailzatoin?
[7:54]	Zero Linden:	grasmere.region.secondlife.com
[7:54]	Zero Linden:	and
[7:54]	Zero Linden:	zero.linden.resident.secondlife.com
[7:54]	Tree Kyomoon:	wayyy cool
[7:54]	Zha Ewry:	Oooh.
[7:54]	Zero Linden:	and have those map to web services about those entities
[7:54]	Zero Linden:	BUT, here's the catch that you can help with
[7:54]	Zha Ewry:	You could alais, that, couldn
[7:54]	Tree Kyomoon:	schweet
[7:54]	Zha Ewry:	't't you?
[7:55]	Zero Linden:	Doing dynamic dns for that sort of thing scares us a bit. Do you have pointers to good
[7:55]	Zero Linden:	dynamic DNS implementations with reasonable interfaces?
[7:55]	Dr Scofield:	hmm...
[7:55]	Sifu Moraga:	it does sound very DNS intensive
[7:55]	Zero Linden:	Obviously we need stuff that scales very well...
[7:55]	Dr Scofield:	right
[7:56]	Zha Ewry:	How much motion are you talking about? Normally, not much, I assume. Moves when machines crash or you need to rebalance?
[7:56]	Tree Kyomoon:	actually our company could use that as well but on a smaller scale
[7:56]	Dr Scofield:	there are services out there that offer this as part of their DNS serivce
[7:56]	Zero Linden:	In my IETF work I got very very scared by the DNS gurus who ripped any proposal that used DNS apart
[7:56]	Zha Ewry:	But you don't move stuff around a lot
[7:56]	Sifu Moraga:	Though DNS is heirarchical, so you might find that it works
[7:56]	Zero Linden:	Zha - we don't move things much - BUT - we would have a heavy access load, and when we do move something, we need it to react quickly
[7:57]	Zha Ewry:	Yes.
[7:57]	Frank Bogomil:	dyndns.com just uses bind and they handle a lot of traffic
[7:57]	Zero Linden:	so normal DNS timeouts, like 1 hour, aren't going to work
[7:57]	Zha Ewry:	The ratio is what i'd be looking at
[7:57]	Zha Ewry:	No, not at all
[7:57]	Zha Ewry:	You'ld want fast
[7:57]	Zero Linden:	Really? just building config files and having it reload?
[7:57]	Dr Scofield:	my private DNS provider allows to set the timeout to as low as 5min
[7:57]	Zha Ewry:	But, fast with low update, isn't bad
[7:57]	Tree Kyomoon:	why not just use ip addresses?
[7:57]	Frank Bogomil:	lots of web sites use 5 second timeouts so they can load balance on the fly, cnn used to use 1 sec
[7:57]	Zha Ewry:	Because moves kill you
[7:57]	Zero Linden:	Tree - we do - buy do you konw the IP address of this region?
[7:57]	Tree Kyomoon:	ah right
[7:58]	Zero Linden:	Exactly
[7:58]	Zha Ewry:	69.35...
[7:58]	Zha Ewry:	Jmm 25
[7:58]	Zha Ewry:	Let me look
[7:58]	Zha Ewry:	I knw your on the 25 side
[7:58]	Zero Linden:	Remember, we'd have well over 4M records in the resident.secondlife.com domain
[7:58]	Tree Kyomoon:	wouldnt internet propegation slow you down a lot anyway?
[7:58]	Zha Ewry:	104.251 :-)
[7:58]	Zero Linden:	and 7k (this week) in the region.secondlife.com domain
[7:58]	Sifu Moraga:	and countin....
[7:58]	Tree Kyomoon:	for us it takes up to 48 hours ...sometime longer
[7:59]	Tree Kyomoon:	when we do a new somthing.trainone.com
[7:59]	Dr Scofield:	tree, that is all configurable
[7:59]	Sifu Moraga:	You could ask serious DNS hackers like Dan Kaminsky
[7:59]	Dr Scofield:	you can set it up such that it propagates much faster
[7:59]	Zero Linden:	Well, DNS is a great, well designed distributed caching system for what it is intended to do.... I'm just worried that we might be trying to stress it in a way that isn't intended
[7:59]	Tree Kyomoon:	man sounds like we really need this too!
[8:00]	Sifu Moraga:	Dan has a lot of experience stressing the DNS system
[8:00]	Sifu Moraga:	ww.doxpara.com
[8:00]	Zha Ewry:	Well, th root question, for you, is do you need to fully expose the inside the datacenter details to the outside. There are a lot of load blancing hack on levels 4-7
[8:00]	Dr Scofield:	plus you could spread the load: objects could register with non-LL dyn DNS services
[8:01]	Zha Ewry:	which let you leave the outside world thinkign things are static, and only worry about updates inside the datacenters
[8:01]	Zha Ewry:	You'd have to deal with the colos very carefully, tho.
[8:01]	Dr Scofield:	home.secondlife.mydomain.org
[8:01]	Tree Kyomoon:	what about going the other way, that is, www.secondlife.com/domains/mydomain?
[8:02]	Sifu Moraga:	harder to scale
[8:02]	Zha Ewry:	Quite a bit harder
[8:02]	Sifu Moraga:	THat would be one DNS lookup for everyone
[8:02]	Zero Linden:	The other way is better understood
[8:02]	Sifu Moraga:	also way cooler
[8:02]	Zero Linden:	so www.sl.com/region/grasmere is clear how to implement that
[8:03]	Zero Linden:	internally, our tree of web services that support the grid do this sort of thing
[8:03]	Zero Linden:	with internal, transparent, routing to the proper service
[8:03]	Zero Linden:	we have a hand built system for this now, and are about to roll out a generic one soon
[8:03]	Zha Ewry:	Zero...Do you edge route through any hard core stuff, or just expose it to the pip providers as is?
[8:03]	Zha Ewry:	pipe
[8:04]	Zero Linden:	we'll be able to configure, from a single file, which web services in the "Indra Global Namespace" are run where
[8:04]	Zero Linden:	and any client can query any machine and have the request proxied to the right host
[8:04]	Frank Bogomil:	doing it via http is more expensive than dns (in theory) simply because of TCP vs UDP
[8:04]	Sifu Moraga:	Hello, Akamai
[8:04]	Sifu Moraga:	:-)
[8:05]	Zha Ewry:	Only for statics, Sifu, in general Fairly static
[8:05]	Zero Linden:	Frank - well on the other hand, you are going to end up doing HTTP to get the data in the end, the DNS just tells you where
[8:05]	Sifu Moraga:	Hey, just joking
[8:05]	Zha Ewry:	Push textures out to akami, tho...
[8:05]	Zero Linden:	so in theory, doing it once, to your local services proxy will eliminate teh DNS call
[8:05]	Zha Ewry:	Base stuff
[8:05]	Frank Bogomil:	true, so long as you don't have to redirect to get the data
[8:06]	Zero Linden:	Zha - I don't know much about our routing strategy --
[8:06]	Frank Bogomil:	(redirect between colos)
[8:06]	Zero Linden:	Frank - well, no, you always will, but the local proxy has basically cached all the DNS queires for all the service provier hosts...
[8:06]	Zha Ewry:	That's the other big lverage point Zero. There are soem cool things you can do with the high end level 3-7 style switches
[8:07]	Zha Ewry:	2-7. sorry, typo there
[8:07]	Zero Linden:	The thing about doing it in the URL path, is that we've found that load ballencers and round robin DNS don't work too well in the internal services architecture
[8:07]	Tree Kyomoon:	there used to be a service where your local computer would keep telling your name server what your current IP was and if it changed the service would rerout the iP
[8:07]	Frank Bogomil:	with dyn dns you could get to the correct server directly.... maybe not worth it since it is only 2x slower at worst to just go HTTP
[8:07]	Zha Ewry:	So that the edge switches konw the they expose a stable external DNS, and then route it to alocal internal dns
[8:07]	Zero Linden:	so this one, where basically every machine can derive the partitioning map from the same live file, seems to work well for now
[8:08]	Zero Linden:	Ah, Zha, I see, no we don't do things like that now
[8:08]	Zha Ewry:	Might be worth keeping in mind
[8:08]	Zha Ewry:	Cisco, Nortell, F5, all do alot of stuff in that space, for example
[8:08]	Zha Ewry:	(Not an endorsement, just three venders...)
[8:09]	Zero Linden:	Zha - I will admit to not knowing enough about how to apply those things to our situation..... but yes, I keep a bee in my bonnet that at some point doing work at another level in the stack is going to be the way to achieve some of this
[8:10]	Zha Ewry:	I'll consider this an exercide in planting a seed
[8:10]	Tree Kyomoon:	is it dependant on separating the avatars from the regions as well?
[8:10]	Zha Ewry:	Now you'll think about it :-)
[8:10]	Zero Linden:	"it"? I think *everything* is dependent on splitting the agents from the regions
[8:11]	Tree Kyomoon:	by it I mean this DNS idea
[8:11]	Zha Ewry:	Has anyone modleed the change in internal traffic you get from that?
[8:11]	Zha Ewry:	You're adding a lot of local traffic
[8:11]	Sifu Moraga:	Sorry guys, need to leave. cu.
[8:12]	Zero Linden:	The split? no, most of the architectural modeling and analysis will happen next quarter (which is almost upon us)
[8:12]	Tree Kyomoon:	bye sifu
[8:12]	Zha Ewry nods
[8:12]	Zero Linden:	We are doing traffic analysis of the UDP to TCP change right now
[8:12]	Tree Kyomoon:	one step at a time...
[8:13]	Zero Linden:	In fact, my big task in the next week is to come up with the basic way to attack that larger architectural split in goals that are achievable quarter by quarter
[8:13]	Zha Ewry:	hmm. When you split off the agents, you're taking on two new types of workload for the sims
[8:13]	Zha Ewry:	More network traffic
[8:14]	Zero Linden:	Zha - no no - less! less!
[8:14]	Zha Ewry:	and something, rcp like, to get the messages to the parts
[8:14]	Zha Ewry:	Well.. Different
[8:14]	Zero Linden:	True - total traffic will increase
[8:14]	Zha Ewry:	You're shfting an interaction
[8:14]	Zero Linden:	BUT, the flow of that traffic will be must more regular
[8:14]	Zha Ewry:	form local to network mediated
[8:15]	Zero Linden:	So instead of every simulator having to be able to talk with every agent's inventory (one of 9 servers at present)
[8:15]	Zha Ewry:	and.. yes, you push some traffic off the sim, to the agent box (a lot one hopes)
[8:15]	Zero Linden:	we will have a much more restricted data path for simulators
[8:15]	Zha Ewry:	yes.
[8:15]	Zha Ewry:	But that path is carying a lot of new traffic
[8:16]	Zha Ewry:	Smaller packets, tho, one suspects
[8:16]	Zero Linden:	And a huge host of non-region specific traffic moves to something that has a close association with the agent's data --
[8:16]	Zero Linden:	like inventory, groups,
[8:16]	Zha Ewry:	How are you binding the network calls onto Havoc and simialr servcies?
[8:17]	Zero Linden:	We still think the amount of traffic connecting arbitrary components will go down considerbaly
[8:17]	Zha Ewry:	where you call locally today
[8:17]	Zero Linden:	Well, for Havoc, we probably won't.. that will stay within the simulator
[8:17]	Zha Ewry nods
[8:17]	Zero Linden:	but for other parts, the simulator just exposes web services for most things
[8:17]	Zha Ewry:	hmm.
[8:17]	Zha Ewry:	You're debian?
[8:18]	Zero Linden:	Yes
[8:18]	Frank Bogomil:	will the agent servers be handling all the UDP traffic (ObjectUpdate) as well?
[8:18]	Zero Linden:	But our webservices framework is custom: a C++ based non-threaded, non-blocking server (see the open source code base), and a
[8:19]	Zero Linden:	python non-threaded, non-blocking server (also built from open source compontry)
[8:19]	Zero Linden:	Frank - no the viewer will get object updates directly from the sims still ...
[8:19]	Zha Ewry:	Oh, tight. So, will this change the tcp level traffic much? i d't think not. Make the chunks a bit different tho
[8:20]	Zha Ewry:	How much tuning do you do on the debian netywork buffer management stuff (Nor, my area, but always important in getting linux to do good network stuff)
[8:20]	Zero Linden:	Oh, which reminds me.... textures over HTTP will be coming soon
[8:20]	Dr Scofield:	nice
[8:20]	Frank Bogomil:	excellent
[8:21]	Zero Linden:	Zha - we have a little bit of tuning - but I suspect we should be doing alot more
[8:21]	Frank Bogomil:	could we use external URLs?
[8:21]	Dr Scofield:	how will that work?
[8:21]	Zero Linden:	Frank - no this means that texture fetches from our servers will go over HTTP rather than via the UDP system
[8:21]	Dr Scofield:	will the client do the pull from the URL?
[8:21]	Dr Scofield:	ah :-(
[8:21]	Frank Bogomil:	that would be great
[8:21]	Frank Bogomil:	then they could be dynamically updated for example
[8:21]	Frank Bogomil:	take load off linden as well
[8:21]	Zero Linden:	It is a start for other things - since it means we have a HTTP path for texture fetching in the viewer
[8:22]	Zha Ewry:	Will it be cachable, in the sense that a http level cache would catch it?
[8:22]	Zero Linden:	and it seriously off loads the simulator which right now is relaying texture data
[8:22]	Zha Ewry:	Or will it be too far inside the stack youre running
[8:22]	Frank Bogomil:	since textures are static you could offload to say akami
[8:22]	Zero Linden:	At first, no - the textures are fetched via a capability, and hence over SSL, and so - not cachable
[8:22]	Dr Scofield:	so there's hope that one of these days we'll be able to specify non-LL URLs for textures
[8:22]	Zero Linden:	other than at the receiving end, of course
[8:23]	Zero Linden:	Dr. S. - yes - PLEASE add real life use cases to the wiki
[8:23]	Dr Scofield:	ok, will do
[8:23]	Frank Bogomil:	do you know how much % of traffic textures are?
[8:23]	Frank Bogomil:	(could help the argument)
[8:23]	Zero Linden:	Gee I should know that number off the top of my head - but I don't
[8:23]	Zero Linden:	it is VERY high
[8:23]	Zha Ewry:	I think we're all on the same page, here... If you can get the textures, especially the huge number of them which are static, into things like akami cache, and people's local caches...
[8:24]	Zha Ewry:	wow.
[8:24]	Frank Bogomil:	yah wow
[8:24]	Tree Kyomoon:	amen!
[8:24]	Fate Garden:	ctrl-alt-2 will give you some idea frank
[8:24]	Tree Kyomoon:	no more 10L uploads
[8:24]	Fate Garden:	sorry ctrl-shift-2
[8:24]	Zero Linden:	Getting textures cachable out in the internet is another matter
[8:24]	Frank Bogomil:	most of us have huge down bandwidth and could handle massive concurrency to outside servers
[8:25]	Zha Ewry:	its a huge win for you, tho
[8:25]	Zero Linden:	right now the architecture is well tuned: you fetch texturs from the region you are on, which will locally (on the sim host) cache those textures used by the region
[8:25]	Tree Kyomoon:	thats right, even me on satellight intenet--would prefer concurrent
[8:25]	Zero Linden:	hence the locality is just right
[8:25]	Zero Linden:	BUT, to make them cachable on the internet as a whole, we'd have to give them generic URLs: http://texture.asset.agni.lindenlab.com/1234 or some such
[8:26]	Dr Scofield:	yep
[8:26]	Tree Kyomoon:	but now we have to get all the content from one IP right?
[8:26]	Frank Bogomil:	yes, use the LLUUID
[8:26]	Zero Linden:	but that means our front end squid caches would have terrible (none!) locality
[8:26]	Zha Ewry:	Has anyone looked to see how well the client is really cachign textures. I don't get the sense that it does as well as I expect.
[8:26]	Tree Kyomoon:	one connection
[8:26]	Zha Ewry:	It seems like gong between two spaces, both with am oderate number of textures, but well beflow you disk cahce size, still yeidfls tons of downloads from the sims
[8:27]	Zero Linden:	Tree - yes, but in general, everyone connecting to that IP are in the same four regions, and hence need the same textures - so good, excellent, locality
[8:27]	Frank Bogomil:	if you could ship the traffic off to an outside service which was cheaper though, or to *us* who would like to have more control of the textures we use
[8:27]	Tree Kyomoon:	the problem is, if you have an object on two different sims but it has the same texture, you have to redownload it as if it were new right?
[8:27]	Zha Ewry:	It would be nice to have some test spaces built, which would say "Walk from point X to point Y, then back, no textures should load if you have more than 512MB of disk cache"
[8:27]	Zero Linden:	Zha - there has been discussion about the texture caching on local disk in SLDev - Steve Linden has posted there with some good details
[8:27]	Zero Linden:	and there some folks playing around with the caching tuning parameters
[8:28]	Zero Linden:	on that list
[8:28]	Zha Ewry nods.
[8:28]	Tree Kyomoon:	but with http textures, you could have one texture SL wide
[8:28]	Fate Garden:	yeah the very latest new new new firstlook is caching far better
[8:28]	Tree Kyomoon:	so popular objects would load really fast everywhere
[8:28]	Zha Ewry:	Only when it doesn't crash on every rebake
[8:28]	Fate Garden:	i've been popping back and forth and NOT rerequesting every little thing
[8:28]	Fate Garden:	much even
[8:28]	Zero Linden:	Indeed - for one, it just uses the local file system as a cache index - rather than our own VFS system
[8:29]	Zha Ewry:	I find adding the minute to re-log sort of negates the benefits :-(
[8:29]	Zero Linden:	Oh look the suns coming up... and it set when we got here....
[8:30]	Zero Linden:	I'm reminded of "Singing in the Rain": "...We've talked the whole night through. So good morning, good moring to you...."
[8:30]	Fate Garden:	thanks for hosting zero, gonna pop over to robin's
[8:30]	Wyn Galbraith found out the other day that SL cache on her local harddisk filled it up, 72G worth.
[8:30]	Zha Ewry:	That sounds like a hint..
[8:30]	Fate Garden waves
[8:30]	Frank Bogomil:	thanx again
[8:30]	Tree Kyomoon:	yes, the sun is erratic lately
[8:30]	Fate Garden:	ow
[8:30]	Zha Ewry:	Thanks, as always
[8:30]	Zero Linden:	well - it is 8:30 and now there are other folks here in the SF office.... time to start my day!
[8:30]	Zha Ewry:	Good stuff
[8:30]	Tree Kyomoon:	lol
[8:30]	Zero Linden:	Thank you all!
[8:30]	Wyn Galbraith:	I like the early hours. I use to get to work at 6:30
[8:30]	Frank Bogomil:	thank you!
[8:31]	Xanshin Paz:	thanks Zero!
[8:31]	Zha Ewry:	/well, it works for me, but it's 10:40 :-)
[8:31]	Zha Ewry:	10:40...
[8:31]	Zha Ewry:	10:40...
[8:31]	Zha Ewry:	30
[8:31]	Wyn Galbraith:	Thanks Zero, I always learn things at the feet of the masters.
[8:31]	Zha Ewry:	east coast, it si a good time. I've gotten coffee, but now snowed under
[8:31]	Wyn Galbraith is on LL time ;)
[8:32]	Zha Ewry:	The tuesdays, I need topry out of meetings for
[8:32]	Tree Kyomoon:	snow? I am in Canada east coast and we have no snow today!
[8:32]	Wyn Galbraith:	I started out on the east coast, Philly born, grewup in Orlando.
[8:32]	Zha Ewry:	Ahh.
[8:32]	Zha Ewry:	Work snow.
[8:32]	Zha Ewry:	e-mails, people banging on my door
[8:32]	Dr Scofield:	got to go, cu all!
[8:33]	Wyn Galbraith:	One of the good things about being currently unemployed, no one banging on my door no email to answer:)
[8:33]	Zha Ewry:	Zero, encourtage other studios to join in ont he office hours :-)
[8:33]	Zero Linden:	I will try...
[8:33]	Tree Kyomoon:	blue linden is here every day I think
[8:33]	Zha Ewry:	Thx
[8:33]	Wyn Galbraith:	I have to start coming to the Tue afternoon ones too. I just get distracted and forget.
[8:33]	Zero Linden:	see ya
[8:33]	Tree Kyomoon:	he's great fun!