User talk:Zero Linden/Office Hours/2007 Mar 06
Jump to navigation
Jump to search
i found the original transcript a tad hard to read & slapped together this little perl script to reformat into a wiki table:
#!/usr/bin/perl
# Placed into the public domain by IBM Research.
use strict;
use English;
sub eject_header ( );
sub eject_line ( $$$$ );
sub eject_footer ( );
eject_header;
while (<>) {
if (m/( # $1: captures whole lead
(\[\d{1,2}:\d{2}\]\s+)? # $2: captures optional timestamp [13:00]
(\S+\s+[^\s:]+) # $3: captures Firstname Lastname
([\s:]) # $4: captures (:| )
)
\s* # gobble up white space
(.*) # $5 captures actual text
/x) {
my $timestamp = $2 || " ";
my $who = $3;
my $sep = $4;
my $text = $5;
eject_line $timestamp, $who, $sep, $text;
next;
}
print "??? ", $_;
}
eject_footer;
exit 0;
sub eject_header() {
printf STDOUT "{|\n";
}
my $last_who = undef;
my $cidx = 0;
sub eject_line($$$$) {
my $timestamp = shift;
my $who = shift;
my $sep = shift;
my $text = shift;
my $bg = "";
if (defined $last_who and $last_who ne $who) {
$cidx = ($cidx + 1) % 2;
}
$last_who = $who;
if (0 == $cidx) {
$bg="background-color:#FFFFFF;";
} else {
$bg="background-color:#F0F0F0;";
}
$text =~ s/%/%%/g;
$who =~ s/\s+/ /g;
printf STDOUT "|- style=\"vertical-align:top;$bg\"\n";
printf STDOUT "| $timestamp\n";
if ($sep =~ /:/) {
printf STDOUT "| $who:\n";
printf STDOUT "| style=\"white-space:normal;\"|" . $text . "\n";
} else {
printf STDOUT "| colspan=\"2\"|''$who " . $text . "''\n";
}
}
sub eject_footer() {
printf STDOUT "|}";
}
with this as the result:
| [13:01] | Maxx Duell accepted your inventory offer. | |
| [13:01] | Zha Ewry: | Hello Zero! |
| [13:01] | Dale Innis: | Hi hi! |
| [13:01] | Jason Ayakashi: | hi, zero |
| [13:01] | Zero Linden: | hello hello |
| [13:02] | Hiro Market: | hi zero |
| [13:02] | Dale Innis apologized for suddenly running off in a few minutes. :) | |
| [13:02] | Dale Innis: | (apologizes) |
| [13:02] | Zero Linden: | Welcome all |
| [13:02] | Zha Ewry: | Hey Rex, welcome back |
| [13:02] | Rex Cronon: | hello everybody |
| [13:03] | Zero Linden: | Everyone had a good weekend? |
| [13:03] | Dale Innis: | Even slept some. :) |
| [13:03] | Zero Linden: | I spent far too much of mine thinking about cross-protocol endpoints. |
| [13:03] | Dale Innis: | lol |
| [13:03] | Zha Ewry: | eewe |
| [13:04] | Zha Ewry dislkies m*n problems | |
| [13:04] | Zha Ewry: | Hello Vitis |
| [13:04] | Rex Cronon: | the things just started here right? |
| [13:04] | Vitis Obviate: | Heya |
| [13:04] | Rex Cronon: | hi |
| [13:04] | Zha Ewry: | Zero has even done his "ground rules" yet |
| [13:04] | Zero Linden: | Yes - and who would have thought that a simple concept like "the sender of this message" could have such a property?!?!? |
| [13:04] | Zha Ewry would :-) | |
| [13:04] | Zero Linden: | Yes, things are just started, Rez |
| [13:05] | Zero Linden: | wow - big turn out today! |
| [13:05] | flying ball whispers: I am ALIVE! | |
| [13:05] | Sifu Moraga: | Gotta load my cache.... |
| [13:05] | flying ball: | , you can't sit on me![cd] |
| [13:05] | flying ball whispers: I am ALIVE! | |
| [13:06] | Dale Innis: | :) yeah some of us will be running of momentarily. |
| [13:06] | Dale Innis: | (off) |
| [13:06] | Sifu Moraga: | I'll be falling asleep in a moment |
| [13:06] | Zero Linden: | Well - as most of you know, but protocol dictate I say.... |
| [13:06] | Zero Linden: | This is Zero's Techno-Geek-Fest office hours |
| [13:06] | Zero Linden: | And it the transcripts will be posted to the SL wiki |
| [13:07] | Sifu Moraga: | Oops, a lot of IBMers here |
| [13:07] | Zero Linden: | Unlike last session - I have nothing in particular on the agenda to talk about..... though I'll give a brief update |
| [13:08] | Zha Ewry: | Good, then I can pester you about in / out blobs |
| [13:08] | C4 (say /5 boom): Ka-Boom | |
| [13:08] | Zero Linden: | Last Thursday we were walking about a sim where almost all the sim traffice to the viewer went over LLSD, via the event queue, via a capability |
| [13:09] | Zero Linden: | That was a good milestone. We hope this week to have a grid up where the only traffic that is UDP are some messages like ObjectUpdate and the rest |
| [13:09] | Zero Linden: | of the servers are all talking LLSD over some TCP based transport, as well as the viewer to sim communications |
| [13:09] | josserand Jacobus: | Hi very nice place, Zha ! May I sit down ? |
| [13:09] | Zero Linden: | That's all from me |
| [13:09] | Zero Linden: | So, I'm happy to talk blobs |
| [13:10] | Zero Linden: | For those that don't know, there is a Zero Office Hour Discussion page in the wiki |
| [13:10] | Hiro Market: | can i just request the wiki for the last session is updated with the diagrams |
| [13:10] | Zero Linden: | https://wiki.secondlife.com/wiki/Discussion_Page_for_Zero_Linden%27s_Office_Hours |
| [13:10] | Hiro Market: | missing atm |
| [13:10] | Zha Ewry: | Good. How much pain would it cause to have bigger than 2K objects come in as blobs? |
| [13:10] | Vitis Obviate: | ahh..images are up...ty |
| [13:10] | Zero Linden: | Hiro - what? https://wiki.secondlife.com/wiki/ZeroTranscript2007Mar01 has images |
| [13:10] | Sifu Moraga: | (and then you'd have to decide how to store them) |
| [13:10] | Zha Ewry: | I'm especially keen on the idea, as that gets us out of the LSL data space problems |
| [13:11] | Zha Ewry: | As to where Sifu, in inventory, inside the script's prim, is one easy thought |
| [13:11] | Dale Innis: | Yeah; get everyone to store the big stuff on their own external srevers! I like that. :) |
| [13:11] | josserand Jacobus: | No answer so I sit down |
| [13:11] | Zha Ewry: | Because, you don't want them to be live, without some effort on the part of the scripter |
| [13:11] | Sifu Moraga: | I currently use notecards for blobs |
| [13:11] | Hiro Market: | no, nothing for me, tried opera and ie |
| [13:12] | Zero Linden: | So, for others, if you haven't read it, you might want to bop on over to the wiki discussion page and read Zha's proposal |
| [13:12] | Zero Linden: | Sifu - right, but notecards arent r/w and aren't likely to be.... ever. |
| [13:12] | Sifu Moraga: | yes, notecards are a real pain |
| [13:12] | josserand Jacobus: | Zero, what mean LindenVillager please |
| [13:12] | Sifu Moraga: | but they were all we could get to work |
| [13:12] | Dale Innis: | Hence the desire for more general blobs. :) |
| [13:12] | Zha Ewry: | Hmm. Before we dive to far, why aren't notecards writable? |
| [13:12] | Rex Cronon: | a link to zha proposal is on you page zero? |
| [13:12] | Sifu Moraga: | exactly |
| [13:12] | Zero Linden: | Hiro - does this work for you: https://wiki.secondlife.com/wiki/Image:ZeroOfficeHours-20070301-slide1.png ?? |
| [13:13] | Zha Ewry: | https://wiki.secondlife.com/wiki/Discussion_Page_for_Zero_Linden%27s_Office_Hours |
| [13:13] | Zero Linden: | https://wiki.secondlife.com/wiki/Discussion_Page_for_Zero_Linden%27s_Office_Hours#Binary_Blob_in.2Fout_from_SL |
| [13:13] | Zha Ewry: | If you use mine, scroll down, Zero's is more precise |
| [13:13] | Zero Linden: | So |
| [13:13] | Zero Linden: | there are two aspects about blobs I'd like to discuss |
| [13:14] | Zero Linden: | First - mechanics of what blobs would look like: |
| [13:15] | Zero Linden: | Right now, the inventory of an object is essentially a list of assets and access rights information |
| [13:15] | Zero Linden: | A binary blob, especially a r/w one could not be stored in the asset server becuase the asset server |
| [13:15] | Zero Linden: | (not a single machine, but logically so), was designed, and is cached, on the assumption that |
| [13:15] | Zero Linden: | assets are data that never change once written |
| [13:16] | Zha Ewry: | I'd assume you'd take it into a native SL type inbound |
| [13:16] | Sifu Moraga: | Hmmm, I see the problem |
| [13:16] | Zha Ewry: | So, the mime type of the blob would tell the inbound process how to turn it into an SL object, you'ddump it into the asset server |
| [13:16] | Zero Linden: | but nonetheless, we'd have to extend the concept of inventory to include objects that aren't assets. |
| [13:16] | Zha Ewry: | and the pass the uuid to the invenotry of the prim running the script |
| [13:16] | Sifu Moraga: | yes |
| [13:16] | Zero Linden: | Not impossible, but definitely different |
| [13:17] | Zero Linden: | And once one wants all the inventory like operations to work on them - you see that we'd be effectively trying to |
| [13:17] | Zha Ewry: | thus it becomes a fairly normal SL object, before it gets into the asset server |
| [13:17] | Zero Linden: | create a sub-class of an abstract class (inventory item) that doesn't currently exist |
| [13:17] | Sifu Moraga: | Since an object is local to a sim, could transitory blobs be stored by the sim server? |
| [13:17] | Zero Linden: | Sifu - it would have to be -- so |
| [13:18] | Zero Linden: | right now, when we move an inventory item (or copy it), there is no data movement at all, becuase the assent doesn't change or move |
| [13:18] | Sifu Moraga: | Ko, so the problem is that an object can reference anything that is not in the asset server |
| [13:18] | Zha Ewry: | right, you just move a uuid |
| [13:18] | Zero Linden: | for a blob, we'd have to hook all those and do copying and/or moving and/or reference counting.... etc... |
| [13:18] | Zero Linden: | none of this is impossible, just more difficult and involves a refactoring of a part of the system |
| [13:19] | Zha Ewry: | Well, you actually looked at amore general solution... II won't complain if you do that... mind you |
| [13:19] | Zha Ewry: | I'd be happy to have a mim driven way of getting blobbed assets into the asset server, and into the inventory of a local object |
| [13:19] | Zero Linden: | So now we have to store that data too... and as Sifu said, it would be on the sim... some of the time |
| [13:19] | Sifu Moraga thinks out loud, could an object store a reference to a transient object? | |
| [13:19] | Zero Linden: | so - when you take an object into inventory (take or take copy), we are basically saving the state of the object (and scripts), and the inventory list, and writing that into a new asset |
| [13:20] | Zha Ewry: | I'd asume you'd be reluctant to do to much of that, as you now have state which is in a bad way, if the sim crashes |
| [13:20] | Zero Linden: | So in this case, we'd have to bundle all the data of the blob up into the inventory entry and store that too |
| [13:20] | Zha Ewry: | right now, assets either are, or don't exist. Not much inbetween |
| [13:20] | Zero Linden: | again, not difficult, but more work to do.... and more data to move |
| [13:20] | Vitis Obviate reflects that we should discuss the "why can't we write notecards" problem also - as Zha suggested - before we dive too deep | |
| [13:20] | Zha Ewry: | In the asset server, with a uuid, or non-existent |
| [13:21] | Zero Linden: | Internally, if the object is rezz'd we'd have to have the data hanging off the inventory item... again not hard |
| [13:21] | Zero Linden: | and when the sim state saves, we have to write out all that data with objects in the same way as when sent to the asset server during Take |
| [13:21] | Zero Linden: | So these changes wouldn't be that hard.. easier than the inventory operation refactoring, actually |
| [13:22] | Zero Linden: | But - then there is the issue of the data and the cost of storing it and moving it around |
| [13:22] | Zha Ewry: | Vitis, I think you just got a big chunk of your answer to read/write notecard, btw |
| [13:22] | Zha Ewry: | Since the same issiues, roughtly obtain |
| [13:22] | Zha Ewry: | (lots of state, and the question of the UUIDs of the notecard in each state) |
| [13:22] | Zero Linden: | If... imagine, a region with 1,000 objects that used blobs. And, say, 2 blobs per object (pulling numbers out of thin air) |
| [13:23] | Rex Cronon: | but what if i am not interested to save the state of the scripts that in an linked set inventory, if i can use scripts to write to notecard, the script can save its onw state |
| [13:23] | Zero Linden: | and say 20k of data per blob |
| [13:23] | Zero Linden: | that's 20k x 1k x 2 = 40M |
| [13:23] | Hiro Market: | somewhere in the forum there's a reference to LL saying they'd open up a database for scripts to read/write data |
| [13:24] | Hiro Market: | seperatly to other mechanisms |
| [13:24] | Zero Linden: | and, since we store objects in XML -- you'd have to base 64 this when stored |
| [13:24] | Zha Ewry: | So, zero,suppose we assume that we use blobs, just to get objects (current set of SL native ones) into/outof the asset server |
| [13:24] | Zero Linden: | Since a typical region runs in 200M of RAM, and we try to keep it under 512M |
| [13:25] | Sifu Moraga thinks that the fact that my blobs are already XML doesn't help | |
| [13:25] | Zha Ewry: | Which scopes things down a lot |
| [13:25] | Zero Linden: | you can see that this would start to put a strain on memory |
| [13:25] | Sifu Moraga: | But you could still have a local DB, though |
| [13:25] | Zha Ewry: | I'd love the more general solution, mind you, but I'm looking for ways for use to unload the storage issue (and the assembly issues) from SL and the scripters |
| [13:26] | Sifu Moraga: | not everything hast too be in memory, does it |
| [13:26] | Zero Linden: | Hiro - I don't know what forum post you're referring to -- but perhaps you are thinking about silo.php -- which is my script for storing arbitrary amounts of data on your own web server from LSL |
| [13:26] | Zero Linden: | ? |
| [13:26] | Zha Ewry: | and let us (the developers who own the binary data) get it to the asset server as neatly as possible |
| [13:26] | Zha Ewry: | (and the converse, to get it out) |
| [13:27] | Zero Linden: | Zha- if your goal is to be able to add textures / sounds / notecards or even (gasp!) scripts to the asset system under LSL control..... |
| [13:27] | Hiro Market: | no, not silo, this is a reference to LL agreeing that not writing to notcards was an issue, and supplying a seperate database mechanism accessible by LSL |
| [13:27] | Zero Linden: | or similarly, to read that data out.... |
| [13:27] | Hiro Market: | quite old, looking for it, caught my eye when I saw it |
| [13:27] | Zero Linden: | then perhaps we should talk about issuing capabilities to that information so you can just do it directly |
| [13:27] | Zha Ewry: | (and follow the DRM model, as we do it...) |
| [13:27] | Sifu Moraga: | Certainly that is not what I need blob transfer for |
| [13:28] | Zero Linden: | Oh, Hiro, don't know about there - there isn't a current plan internally that I know of |
| [13:28] | Zha Ewry: | Capabitlies, with the right strucfture, linked ot scripting, would get us most of the way. |
| [13:28] | Zha Ewry: | to |
| [13:28] | Sifu Moraga: | you think so, Zha? |
| [13:28] | Zero Linden: | Mind you - web server space that runs PHP is so cheep - seems like a much better solution to just have people store their own data in data stores they manage and pay for (and backup!) |
| [13:29] | Zha Ewry: | The question is mostly how to get the data from those servers into SL |
| [13:29] | Zero Linden: | Rather than the scaling issue of us managing a big writable store for everyeone... |
| [13:29] | Zero Linden: | Zha - this brings up my second question... which is - what do you want to do with the blobs |
| [13:29] | Sifu Moraga: | Unfortunatly, I'd rather have a standard-like solution like REST or SOAP (gad, ack) |
| [13:29] | Zha Ewry: | Now, to an extent, I'd be happy to bypass an asset server, but I'm stuck on how to do that such that it all works when the script moves across sim lines |
| [13:30] | Zha Ewry: | REST, not SOAP, please :-) and that's a personal not corporate preference :-) |
| [13:30] | Zero Linden: | and perhaps also notecards and sounds |
| [13:30] | Zero Linden: | and I'm sensing it is you want to import lots of textures into SL for autmated builds or catalogs or knowledge bases or some such |
| [13:30] | Zero Linden: | Ditto here, Zha --- personally, I don't like the taste of SOAP... |
| [13:30] | Sifu Moraga: | I know, I know, Zha, but I don't always decide.... |
| [13:30] | Zha Ewry: | Well also to drive larger data driven scripts. |
| [13:31] | Zha Ewry: | We have lots of use cases where we don't (can't) do the compute in SL, nor do you want us to |
| [13:31] | Sifu Moraga: | So, what I;m doing with my notecards is to do some light processing on them and then sending them to an instrumented client |
| [13:31] | Zha Ewry: | But we need efficient ways of getting the data in, well beyond the 2048 bytes at a time solutions, and with less load on both the scripting workload |
| [13:31] | Zero Linden: | Right - but driving data driven scripts is easy - because since the scripts can't compute that much anyway, you just have the scripts request the data in chunks it can handle, which are going to be small |
| [13:32] | Zha Ewry: | and the http requests pounding the sims |
| [13:32] | Sifu Moraga: | And then, the instrumented client gives a notecard back and gives it to an object via the AV |
| [13:32] | Zero Linden: | Really? Surely you can process 2k worth of, say geometry instructions, in LSL in the 1/2 second it takes to fetch the next 2k |
| [13:33] | Zha Ewry: | If I want to texture, as I go, it gets worse, and I can drive a nice class five sim down to a time dialation of oh, .4 or ..3 with just geometry and scripts |
| [13:33] | Sifu Moraga: | I don't need to do much processing, except a bit of string replacement |
| [13:33] | Zha Ewry: | not even loading texutres. Just rezzing prims, and moving them into place in large numbers |
| [13:33] | Zha Ewry: | If I want to load textures, as I rez... its gets painful |
| [13:34] | Zero Linden: | Right - but I bet the load from the llHTTPRequest isn't really contributing to that problem |
| [13:34] | Sifu Moraga has seen what Zha is up to :-) | |
| [13:34] | Zero Linden: | We tested doing several 100 llHTTPRequests per second in a sim with no noticable lag at all. |
| [13:35] | Zero Linden: | So, for driving scripts, I doubt getting the data in bigger chunks would be any better |
| [13:35] | Rex Cronon: | so zha, u want to build something off grid and upload those objects(or linked sets) into a sim? |
| [13:35] | Zero Linden: | Now, notecards and textures are two totally other problems |
| [13:35] | Hiro Market: | slightly tangental, but has support for procedural textures ever been considered? |
| [13:35] | Zha Ewry: | Hmm. Say, a 5500 prim molecular model? Yeah. |
| [13:35] | Sifu Moraga: | The problem I have is that I can't transfer the entire notecard in one piece, so I need to do about 4 transfers |
| [13:36] | Zha Ewry: | And then stich them together, and then run out of LSL memory |
| [13:36] | Zero Linden: | Righy - but doing the, I dunno, 100 2k data requests for the data for that molecule isn't nearly so much the load on the sim as rezzing and moving 'em |
| [13:36] | Sifu Moraga thinks the molecure is awsome | |
| [13:36] | Zero Linden: | especially since I bet each of those has a script! |
| [13:36] | Zha Ewry: | One problem I see, is we end up with lots of scripts to deal with dataspace |
| [13:36] | Zha Ewry: | Not for long. We kill 'em as fast as we can. |
| [13:37] | Zha Ewry: | If I leave 5500 scripts listening, the sim is an unhappy place for 2 agents |
| [13:37] | Zero Linden: | none the less, the cost of starting a script far outweights the cost of doing one llHTTPRequest |
| [13:37] | Zero Linden: | Let's talk notecards - |
| [13:37] | Zero Linden: | Sifu - you seem to be wanting to do two things with them.... |
| [13:37] | Zha Ewry: | Hmm. I'm not surprised. Scripts require lots of context |
| [13:37] | Zero Linden: | One is transmit coded data to a specially instrumented browser |
| [13:38] | Sifu Moraga nods | |
| [13:38] | Vitis Obviate would use that also | |
| [13:38] | Zero Linden: | You could, of course, use specially coded IM messages -- (what is that crypto product that works over AIM???) |
| [13:38] | Zha Ewry as well | |
| [13:38] | Zero Linden: | but i imagine that is slow.... |
| [13:39] | Zero Linden: | So - why not put the data via llHTTPRequest onto your server, and have the client retrieve it? |
| [13:39] | Sifu Moraga: | We thought of that, but still have to get it out to a web service at one point |
| [13:39] | Zero Linden: | granted, you have to take up the store of it |
| [13:39] | Zero Linden: | right - but the instrumented data is for the viewer, not the human, right? |
| [13:39] | Sifu Moraga: | that too |
| [13:39] | Zero Linden: | is it larger than 2k? |
| [13:39] | Sifu Moraga: | It also needs a bit or processing in-world |
| [13:39] | Sifu Moraga: | The data is for a cpu, its all XML, but unreadable |
| [13:40] | Sifu Moraga: | it is about 16kb in it's least cryptographicaaly secure form |
| [13:40] | Zha Ewry: | A fair bit of crypto/cert stuff ends up bigger than 2048... |
| [13:40] | Zero Linden: | We've had requests to have a version fo llHTTPRequest that is issued by the viewer, not the sim |
| [13:40] | Sifu Moraga: | easily |
| [13:40] | Zha Ewry: | Ah. There's a nice hard number |
| [13:41] | Sifu Moraga: | not the same problem, but still of interest for other stuff |
| [13:41] | Zero Linden: | so a script on the sim makes the request, the request is sent to the viewer, the viewer makes the call, and teh results piped back up to the |
| [13:41] | Zha Ewry: | Thus inside the client's firewall, and access to the local machine? |
| [13:41] | Zero Linden: | sim |
| [13:41] | Sifu Moraga: | yup |
| [13:41] | Zero Linden: | Right - the desire is to allow HUD objects that control things like MP3 players, and other, er, "add-ons" |
| [13:41] | Sifu Moraga: | more or less, though the firewall is incidental to what we are doing |
| [13:41] | Zha Ewry: | That helps although if its 2048, only a little, as you're still in the data stitching game |
| [13:42] | Zero Linden: | This strikes me as more of what you are after in this case, Sifu? |
| [13:42] | Sifu Moraga: | important is that we have two completely separate zone we need to access |
| [13:42] | Zero Linden: | Well, Zha, if the data has to go through LSL, while we can relax the size a bit (we intend to make it configuratble at some point) |
| [13:42] | Zero Linden: | it still won't be useful beyond, say 6k |
| [13:43] | Zha Ewry: | Right. Thus the desire to bypass LSL |
| [13:43] | Zero Linden: | since LSL execution space is going to be limited to 16k for quite some time |
| [13:43] | Sifu Moraga: | 16 kb is the minimum |
| [13:43] | Zha Ewry: | I'm looking at how I can move bigger chunks without having them have to land in LSL at all |
| [13:43] | Zero Linden: | But - the only thing I think I hear that you'd entirely bypass LSL for is: Textures, and Notecards for users to read |
| [13:43] | Sifu Moraga: | It did occure that if we could just send the notecard directly from llHTTPRequest.... |
| [13:43] | Zha Ewry: | Well, notecards for programs as well |
| [13:43] | Vitis Obviate: | yes |
| [13:43] | Sifu Moraga: | ...that would bypass the LSL memory restrictions |
| [13:44] | Zero Linden: | Right - but where? If not a texture or a human readable notecard, then all other destinations in SL require LSL, yes? |
| [13:44] | Sifu Moraga: | I do think the use ofr notecards to be just wrong, somehow |
| [13:44] | Zha Ewry: | I'm happy to loop across a long notecard, one line at a time, never getting big chunks into the LSL data space |
| [13:44] | Vitis Obviate nods | |
| [13:44] | Zero Linden: | Then why not read the data from your web server one line at a time? It is almost as fast, if not faster! |
| [13:44] | Sifu Moraga: | umm, you'l be happy until you have tried it like me |
| [13:45] | Zha Ewry: | I'm muchg less happoy trying to coordinate that across http gets with all the logic to deal with dropped /oiur of order stuff |
| [13:45] | Sifu Moraga: | I get some delay that the users don't like |
| [13:45] | Sifu Moraga: | yes, that is worse |
| [13:45] | Zero Linden: | Indeed, the notary reads notecards one line at a time (needs to for hashing) and it is a bear |
| [13:45] | Zha Ewry: | I also see about 50% of my code space go into the parse/manage packet order problem |
| [13:46] | Zero Linden: | Really? Why not just issue your HTTP requests serially, like you'd do for notecards? |
| [13:46] | Sifu Moraga: | Also, I can't get llHTTPrequest to add chunking headers |
| [13:46] | Zha Ewry: | I understand the 16K problem, so I'm looking for reaosnale ways to bypass it |
| [13:46] | Zha Ewry: | I can issue them serially, but I have to have chunking/retry logic and I do end up exercising ti |
| [13:46] | Zha Ewry: | it |
| [13:47] | Zero Linden: | By the way - I'm not trying to be resistant - just trying to really get at what the user needs are here -- alas, with our highly constrained system, and our opaque implementation, it is hard to know what the crux of a user problem is |
| [13:47] | Sifu Moraga totally understands | |
| [13:47] | Zha Ewry: | Yes, I get that And having the discusssion is the right way to do it |
| [13:47] | Zha Ewry: | I'm totally happy witrh this process, at the moemnt |
| [13:48] | Zero Linden: | ah- I think I'm now seeing the bigger issue - here |
| [13:48] | Zha Ewry: | Point a number of people at the problem, and get to a good technical answer works for me |
| [13:48] | Zero Linden: | I'm wondering then if instead we supported byte-ranges, |
| [13:48] | Sifu Moraga: | If I was travelling to SF any time soon, I'd love to have a RL discussion of it |
| [13:48] | Sifu Moraga: | (unfrotunatly, I'm not) |
| [13:48] | Zero Linden: | and since our requests are squid cached |
| [13:48] | Zero Linden: | if that would work - because squid requests the whole thing when you do a byte range request |
| [13:49] | Zero Linden: | so once the frist range worked |
| [13:49] | Zero Linden: | you'd be getting the rest out of the cache |
| [13:49] | Zha Ewry: | Oh, allow us to load a bigger chunk into the squid, and then grab it 2048 at a time? |
| [13:49] | Sifu Moraga thinks | |
| [13:49] | Zero Linden: | Exactly- if everyone played right by the headers and all.... |
| [13:50] | Sifu Moraga doesn't get it | |
| [13:50] | Zero Linden: | If you do a GET to say http://example.com/foo/bar |
| [13:50] | Zero Linden: | but include a byte range header asking for bytes 0-2047 |
| [13:50] | Zha Ewry willing to be a very obedient header wreiter | |
| [13:50] | Zero Linden: | Squid cache will instead ask for the whole thing |
| [13:50] | Zero Linden: | and cache the whole resource at that URL |
| [13:50] | Zero Linden: | and deliver only the requested bytes |
| [13:50] | Zero Linden: | when you then make a second request to the same URL |
| [13:51] | Zero Linden: | but with byte range 2048-4095 |
| [13:51] | Zha Ewry: | with the byterange... |
| [13:51] | Zha Ewry: | Yes.. |
| [13:51] | Zero Linden: | squid just answers out of the cache |
| [13:51] | Vitis Obviate will use whatever incontonation is required | |
| [13:51] | Zha Ewry: | How big a chunk could the squid handle? |
| [13:51] | Zero Linden: | Now, mind you, it is all REST and stateless... so |
| [13:51] | Zha Ewry likes REST and stateless... | |
| [13:51] | Zero Linden: | in theory, you could have squid drop the resource in the middle and have to refetch |
| [13:51] | Vitis Obviate: | fine with me |
| [13:51] | Sifu Moraga thinks REST is good | |
| [13:51] | Zha Ewry: | Yes, just delay on random fetches if it does |
| [13:52] | Zero Linden: | Squid? Oh, I don't know what our maximum cached object size is set to.... |
| [13:52] | Zha Ewry: | as long as the URL is idempotent, we're ok |
| [13:52] | Zha Ewry smiles | |
| [13:52] | Zero Linden: | right now it is probably pretty large... 1M or so? |
| [13:52] | Zero Linden: | Exactly, Zha |
| [13:52] | Zha Ewry: | I can deal with that :-) |
| [13:52] | Zha Ewry: | Doesn't help with textures or sounds, but... it would help a lot. |
| [13:52] | Zha Ewry: | Be harder to do in the other direction, I'd imagine |
| [13:53] | Zero Linden: | Yes - but there there is the much more natural solution - just let you texture an object with a URL to your own image on your own server... |
| [13:53] | Zero Linden: | Not that we are doing that right now.... |
| [13:53] | Zha Ewry: | Yes. |
| [13:53] | Zero Linden: | We do have to be careful about opening the flood gates to textures |
| [13:53] | Hiro Market: | like the sound of that |
| [13:54] | Zha Ewry: | That'd be fine, within reason. How exactly we get the proggramitrc control is interesting, but I can see ways. |
| [13:54] | Zha Ewry: | Yes, I'm wiling to hear lots of stories about throttle on textures |
| [13:54] | Zero Linden: | Obviously if everyone just uploaded a texture for every surface they needed, under LSL command... we'd see things like |
| [13:54] | Hiro Market: | from my perspective one of the biggest restrictions on LSL is not being able to programatically manipulate textures |
| [13:54] | Zero Linden: | billboards that uploaded new textures every 15 seconds |
| [13:54] | Rex Cronon: | it would be ok to me if instead of html i can use svg |
| [13:54] | Hiro Market: | at pixel level |
| [13:54] | Zero Linden: | which would be cool from the SL experience point of view |
| [13:54] | Zha Ewry: | Yes... throttleing is an issue, of course. |
| [13:54] | Zero Linden: | but would really kill the asset server |
| [13:54] | Zha Ewry: | Althoug.. if you can have "transient" textures |
| [13:55] | Sifu Moraga: | It does sound like you will need the concept of transient blobs stored by the sims |
| [13:55] | Zha Ewry: | locally cached, I can come up with a story that only pounds the sim |
| [13:55] | Zha Ewry: | And not even that bad. |
| [13:55] | Zero Linden: | Hiro - you aren't likely to see things like paramatic textures or pixel level texture control from LSL |
| [13:55] | Zha Ewry: | The sim has to fetch/cache the tyextures today |
| [13:55] | Zero Linden: | These are memory and process intensive operations that the sim doesn't have the resoruces to do |
| [13:55] | Zha Ewry: | Nor should it |
| [13:55] | Zero Linden: | what is more likely, what scales much better, is |
| [13:55] | Zha Ewry: | Keep as much rendering on the client as possible |
| [13:55] | Hiro Market: | again I wonder about procedural textures on the client with shader code on server |
| [13:56] | Zha Ewry nods at Hiro | |
| [13:56] | Zero Linden: | do the pixel level processing on your own computer or server - and then have a way to put that texture on an object |
| [13:56] | Zero Linden: | The servers, the sims, have no graphics hardware at all... so no OpenGL going on there |
| [13:56] | Zha Ewry: | I think mosgt of us who have loooked at the scale issues want a smuch rendering as possible on the client |
| [13:56] | Zha Ewry: | I don't want the SIMs even thinking about rendering |
| [13:56] | Hiro Market: | don't need it, just the shader code on server, rendered on client |
| [13:57] | Zero Linden: | don't expect the sims to be doing graphic manipulation or rendering (which is a hint, by the way, as to why the map is such a pain point for us!) |
| [13:57] | Zha Ewry: | Among other horrors, you end up with the sim needing to know things like the resolutoin of every client they are talking to |
| [13:57] | Zero Linden: | Ah - but Hiro, can you imagine what it would be like if an object could down load arbitrary shader code to your viewer?!?!? |
| [13:57] | Hiro Market: | but procedural shader code is simply code, no rendering on server required at all |
| [13:57] | Zero Linden: | Talk about griefing! |
| [13:58] | Hiro Market: | hmm, point taken, but the advantages must be worth exploring surerly? |
| [13:58] | Zha Ewry: | He. Just because sooner or later we'd have all the pron spam in the shader :-) |
| [13:58] | Zero Linden: | Alas, OpenGL doesn't have virtualization or sandboxing facilities to make that reasonable |
| [13:58] | Zero Linden: | Nor does anyone have shader verifiers as far as I know.... |
| [13:58] | Zero Linden: | But - one could imagine in |
| [13:58] | Zero Linden: | the future |
| [13:58] | Sifu Moraga thinks there might be a case for somre rendering on the server, but it would probably need a different architecture | |
| [13:59] | Hiro Market: | 3dlabs have one |
| [13:59] | Zha Ewry shudders at what you could get the shader to do. | |
| [13:59] | Hiro Market: | a verifier that is |
| [13:59] | Zero Linden: | private sims that said "If you come here, I will use spiffy special purpose shaders.... allow?" |
| [13:59] | Rex Cronon: | talking about griefing, what exactly does the physics engine do to attachements, when the one that wears them is blitzed? |
| [13:59] | Zha Ewry: | Hm. That'd be interesting |
| [13:59] | Zha Ewry: | In general, the question of private sim behavioir is interesting |
| [13:59] | Zero Linden: | Hiro - curious - good to know - though again I wonder if it worries about hostile environments or just makeing sure things won't crash... |
| [14:00] | Sifu Moraga: | My cranial CPU is shutting down, and my Powerbook is out of juice, so I'm off early |
| [14:00] | Sifu Moraga: | cu all |
| [14:00] | Zero Linden: | Yes, it is.... short term, once we start letting estate owners control upgrade times.... we are going to have to deal with |
| [14:01] | Zero Linden: | viewers and regions with different feature sets.... |
| [14:01] | Zha Ewry: | Yes |
| [14:01] | Zero Linden: | Well, Sifu - actually it is 2pm so I'm signing off soon too |
| [14:01] | Zero Linden: | Shorter term - |
| [14:01] | Zha Ewry: | we talked about tadeoffs, between number of scirpts and data spaces and such |
| [14:01] | Zha Ewry: | last week. |
| [14:01] | Zha Ewry: | So lots of things I'd love to be able to contrrol on the sim :-) |
| [14:02] | Zero Linden: | one could easily imagine custome viewers that have special shaders in 'em and then some way for objects |
| [14:02] | Zero Linden: | to invoke that shader behavior if running on viewer that has 'em |
| [14:02] | Zha Ewry: | Wow. This was incredbly useful Zero. Hope it was for you. I'll try and update my blob comments a bit. and think about some of your comments |
| [14:02] | Zero Linden: | Zha - yes - though realize, like all server tuning, the more parameters you have, the better performance you can achieve, but the more compicated it is for people to manage it |
| [14:03] | Zha Ewry: | Hey I race saliboats. More strings, more fast. or more slow. |
| [14:03] | Zha Ewry: | so, yes. |
| [14:03] | Zero Linden: | and - one has to realize that you can only do so much, even with so many knobs |
| [14:03] | Zero Linden: | (thinks of tuning squid, or apache, or mysql....) |
| [14:03] | Zha Ewry: | Yep. 1 cpu, 512M of memory, etc. |
| [14:03] | Zha Ewry: | But, there are cases where we can play those off well |
| [14:04] | Zero Linden: | It is great - I'm very happy with my office hours experiences - and have been talking up how good they are inside of LInden |
| [14:04] | Zha Ewry: | Thanks again Zero, as always. Really focused discussion today |
| [14:04] | Zero Linden: | Thanks all for coming |
| [14:04] | Zero Linden: | See you next time. |
| [14:04] | Kooky Jetaime: | Thank you Zero |
| [14:04] | Vitis Obviate: | very valuable info - ty |
| [14:04] | Drewan Keats: | Thanks Zero. |
| [14:04] | Hiro Market: | ty zero, aoppreciated |
| [14:04] | Zha Ewry: | Thanks everyone, actually, very good discussion today |
--Dr Scofield 01:57, 20 March 2007 (PDT)