User:Which Linden/Office Hours/2009 May 7

• [10:59] Which Linden: hi techwolf
• [11:00] Techwolf Lupindo: Hi tyhrer.
• [11:00] Which Linden: how are ya?
• [11:00] Techwolf Lupindo: I'me at two places at once again.
• [11:00] Techwolf Lupindo: My alt is at Jack Linden oh.
• [11:00] Techwolf Lupindo: doing ok.
• [11:00] Which Linden: nice
• [11:01] Techwolf Lupindo: Last time there was a memorizable quote JL that was spread out and hopefully he got a ton of Notecards.
• [11:01] Which Linden: a memorizable quote?
• [11:02] Techwolf Lupindo: Yea, stated he recieved about zero IM or NC agaist the AC stuff.
• [11:02] Which Linden: d'oh
• [11:03] Techwolf Lupindo: Any agenda for today?
• [11:03] Which Linden: well I have two ongoing conversations with Sai and Morgaine -- we could do either
• [11:03] Morgaine Dinova: Hiya all :-)
• [11:04] Which Linden: sai's topic is about collaboration tools
• [11:04] Which Linden: morgaine's is about architecture of group chat
• [11:04] Which Linden: hi morgaine :-)
• [11:04] Morgaine Dinova: I have one? Don't recall it, hehe
• [11:04] Techwolf Lupindo: Phillip got an earfull yesteday about the IM and group chat troubles.
• [11:04] Which Linden: did he?
• [11:04] Morgaine Dinova: A nice earful
• [11:05] Techwolf Lupindo: Yes.
• [11:05] Aimee Trescothick: stops wondering why she can't move and plugs the space navigator into the right computer
• [11:05] Which Linden: how is group chat post-1.26?
• [11:05] Morgaine Dinova: Just information gathering, and we supplied it.
• [11:05] Which Linden: anecdotally
• [11:05] Techwolf Lupindo: [1]
• [11:05] Techwolf Lupindo: Last comment.
• [11:06] Which Linden: it sucks that this conflates notices with chat
• [11:07] Which Linden: I wonder if we've been ignoring problems with notices in favor of trying to fix the chat
• [11:07] Morgaine Dinova: They both suffer from the same problems, so I think there's been an assumption that they travel on the same infrastruture.
• [11:07] Which Linden: completely different infrastructure
• [11:07] Morgaine Dinova: Eeek
• [11:07] Morgaine Dinova: Makes you wonder why they both suffer it then!
• [11:07] Techwolf Lupindo: IM and groups chat different?
• [11:08] Which Linden: P2P IM vs group chat is a third architecture
• [11:08] Which Linden: I'd caution y'all to not make assumptions about the way our architcture works when filing bug reports -- it often makes it harder to fix them
• [11:09] Morgaine Dinova: Welll ....
• [11:09] Which Linden: you can ask me how it works :-)
• [11:09] Morgaine Dinova: Assumptions are inevitable when there is no knowledge :-)
• [11:10] Morgaine Dinova: So, how do group IM and group notifications differ?
• [11:10] Techwolf Lupindo: Lets start with IM. Client send the IM to where? Sim or decated externial server?
• [11:11] Morgaine Dinova: I think Eddy was trying to log in here, but failed.
• [11:12] Morgaine Dinova: Hiya Sai!
• [11:12] Which Linden: hi sai
• [11:12] Saijanai Kuhn: hey
• [11:12] Which Linden: as a gross overview, p2p im is a sim to sim thing
• [11:12] Morgaine Dinova: Hang on, what's "p2p im"?
• [11:13] Which Linden: when I IM you
• [11:13] Morgaine Dinova: You mean non-group?
• [11:13] Morgaine Dinova: Aha, kk
• [11:13] Which Linden: when it's two people involved
• [11:13] Which Linden: doing a conference im using calling cards makes it group again
• [11:13] Morgaine Dinova: Yeah, very few people have noticed any problems with person-to-person IM. No lag nor loss/reordering/rejections, etc
• [11:14] Morgaine Dinova: Aha. So contacts conferences and 1:1 IM are both "p2p"
• [11:14] Which Linden: yeah, so the way that works is, the viewer sends a message to the sim saying "I want to talk to person B"
• [11:14] Which Linden: morgaine you've got it a bit wrong: 1:1 IM is "p2p", contacts conferences are not p2p
• [11:15] Morgaine Dinova: Oh dear, so we have 4 categories?
• [11:15] Morgaine Dinova: All on different infrastructure? That's madness :P
• [11:15] Which Linden: categories: 1) 1:1 IM, 2) group/conference IM, 3) notices
• [11:16] Which Linden: is that clearer?
• [11:16] Morgaine Dinova: Ah, so both contacts-conf and group-conf on same infrastructure, kk
• [11:16] Morgaine Dinova: So, you started describing how 1) works.
• [11:16] Which Linden: when the sim gets the message, it looks up B's presence entry using a web service; this presence entry contains the simhost where B is
• [11:17] Which Linden: I believe that if B is on the same sim as you, this lookup step can be omitted
• [11:17] Which Linden: but in any case now the sime has the address of B's sim, and it just sends messages to B's sim
• [11:18] Which Linden: this is a bit of a gross overview because I believe the dataserver acts as a middleman in some respects, but, I believe that detail can be safely elided
• [11:18] Tammy Nowotny: hello
• [11:18] Morgaine Dinova: OK, so 1) is pretty simple p2p store and forward.
• [11:18] Which Linden: yes
• [11:18] Which Linden: that's why it works so well :-)
• [11:19] Morgaine Dinova: Hi Tammy :-)
• [11:19] Morgaine Dinova: Hehe
• [11:19] Tammy Nowotny: hello Morgaine
• [11:19] Which Linden: and I should apologize in advance that this is not the most authoritative because I'm going from memory here and may have to do errata next time
• [11:19] Morgaine Dinova: KK, hehe
• [11:19] Which Linden: "hey everyone, forget everything I told you last time LOL" :-P
• [11:19] Which Linden: hopefully not
• [11:19] Morgaine Dinova: Do 3) is that's next simplest :-)
• [11:19] Morgaine Dinova: I know 2) is hard :P
• [11:20] Saijanai Kuhn: BTW, Which, feel free to help fill in that table in the IMprovedInstantMessages page
• [11:20] Which Linden: ok so I gotta confess that I know next to nothing about how notices work
• [11:20] Saijanai Kuhn: [2]
• [11:20] Morgaine Dinova: But you know that they are different?
• [11:20] Which Linden: they are indeed different
• [11:20] Tammy Nowotny: me neither :-)
• [11:21] Morgaine Dinova: Do they travel over UDP?
• [11:21] Which Linden: actually let's poke through the message template for notice-related messages to find out
• [11:21] Morgaine Dinova: Cool!
• [11:21] Saijanai Kuhn: check the key of that message. anything listed is sent via that packet
• [11:22] Saijanai Kuhn: referring to handy blank table that jhurliman and I set up last year..
• [11:22] Which Linden: here's a message template : [3]
• [11:22] Which Linden: yeah I'm sorry Sai I don't have those kinds of details
• [11:23] Morgaine Dinova: Eddy *nearly* managed to log in here Sai, I think just before you arrived. But his cloud never materialized
• [11:23] Which Linden: ok so looking at the template I see GroupNoticeListRequest and GroupNoticesListReply
• [11:23] Which Linden: those look like they may be for getting the list of notices in the groups UI
• [11:24] Which Linden: there's also GroupNoticeRequest and GroupNoticeAdd... not too much detail there
• [11:25] Morgaine Dinova: Hi Dahlia :-)
• [11:25] Saijanai Kuhn: the notice box is flag 1. Would that be the same thing?
• [11:25] Morgaine Dinova: Sai's getting an eyeful ;-)
• [11:25] Dahlia Trimble: hi :)
• [11:25] Saijanai Kuhn: erh, hi Dahlia :-)
• [11:25] Which Linden: sai I'd guess it was more a flag for the viewer's concept of "notices" which are the blue boxes
• [11:25] Dahlia Trimble: is waiting for all to rez...
• [11:26] Tammy Nowotny: has been waiting since October 2006 for that to happen, Dahlia
• [11:26] Viv Trafalgar: is still waiting Dahlia :)
• [11:26] Morgaine Dinova: lol
• [11:26] Dahlia Trimble: lol
• [11:28] Saijanai Kuhn's: heart slows
• [11:28] Which Linden: so maybe I'll have to get back to y'all on how group notices work
• [11:28] Which Linden: for now it'll have to suffice that it's different from the other two
• [11:28] Which Linden: sorry I don't know this already
• [11:28] Dahlia Trimble: they are IIM packets arent they?
• [11:28] Saijanai Kuhn: dahia they don't seem to be
• [11:29] Which Linden: they might end up that way, but what happens on the backend to make those packets get to the viewer... we don't know
• [11:29] Saijanai Kuhn: [4]
• [11:29] Dahlia Trimble: lol cant help you there ;)
• [11:29] Morgaine Dinova: No problem. What do we know about 3) that *IS* definitely different to 2) ?
• [11:30] Which Linden: well I'm kinda familiar with the servers used to implement 2) (group/conference chat) and I'm fairly confident that that architecture wouldn't support group notices
• [11:30] Morgaine Dinova: kk
• [11:30] Which Linden: for one thing, the group chat stuff is very session-based
• [11:31] Which Linden: when you log out, it notices and removes references to you
• [11:31] Saijanai Kuhn: unless its considered part of inventory offer...?
• [11:31] Which Linden: notices get delivered whenever
• [11:31] Morgaine Dinova: Or never
• [11:31] Morgaine Dinova: That's what destroyed SL live music
• [11:31] Which Linden: sai: maybe it's #32? "IM from a group officer to all group members" ??
• [11:31] Morgaine Dinova: The notices weren't getting through
• [11:32] Which Linden: notices are closer to p2p im semantically, right?
• [11:32] Saijanai Kuhn: or one of the "unknowns" :-/
• [11:32] Which Linden: you get a blue box if you're online, but if you're offline you get an email
• [11:32] Which Linden: I wonder if group notices are implemented as kind of a "mass p2p IM"
• [11:33] Dahlia Trimble: seems when I log in I sometimes get a slew of missed group notices
• [11:33] Which Linden: ah.... so this offline business reminds me of why the dataserver is involved as a middleman with respect to 1:1 im
• [11:34] Tammy Nowotny: well... supposedly you get an email... it is hit or miss, especially when the grpup has many members
• [11:35] Which Linden: yeah -- so what the simulator actually does is it tells the dataserver "hey here's an IM for person B, deliver it"
• [11:35] Which Linden: and the dataserver looks up the presence for B and if they're online it just forwards teh packet to the sim, however if they're not, it inserts a row into the database containing the contents, and then triggers the email
• [11:36] Which Linden: so ... semantically that's pretty similar to the mechanics of group notices, but, then again same semantices != same architecture
• [11:36] Morgaine Dinova: Well mail delivery is completely session-based and resilient, so if notifications by email are getting lost, it's *before* the email system takes charge.
• [11:36] Morgaine Dinova: Assuming that you use a standard MTA of course :P
• [11:37] Morgaine Dinova: If you rolled your own, all bets are off :P
• [11:37] Which Linden: heh sure
• [11:37] Tammy Nowotny: well, not necessarily... spam filters and the like are involved once the mail goes out
• [11:37] Morgaine Dinova: Tammy: not by MTAs
• [11:37] Which Linden: I assume the complaints about group notices go deeper than "my spambox ate it"
• [11:37] Tammy Nowotny: also , ifd email bounces, Linden's mailserver doesn't send out subsequent ones
• [11:37] Morgaine Dinova: Tammy: only at the final end-delivery point,
• [11:39] Morgaine Dinova: I hope we're blaming MTAs for message loss ... would get farcical. :-) Yes, MTAs can lose messages, but it's a completely different scenario and behaviour.
• [11:39] Morgaine Dinova: hope we're NOT :-)
• [11:40] Which Linden: it's a little hard to parse VWR-2950 -- what is a good summary of the problematic notice behavior?
• [11:40] Dahlia Trimble: hasnt there been problems of LL mail servers being blacklisted due to script generated spam?
• [11:41] Morgaine Dinova: Summary is: everything is broken, except for message content ... ie if it's delivered, it's delivered fine.
• [11:42] Morgaine Dinova: In other words, all of the UDP semantics are reflected in group IM behaviour: message loss, message duplication, out of order delivery.
• [11:42] Morgaine Dinova: Plus other behaviours on top of that:
• [11:42] Tammy Nowotny: some groups are so large they probably look like spam to some email providers
• [11:42] Which Linden: ok....gimme more detail though, do you mean that you don't get emails when you're offline, do you not get the blue boxes when you're online, do you not see the entries in the notices tab of the group UI?
• [11:42] Which Linden: all of the above?
• [11:42] Storm Thunders: The first two.
• [11:43] Viv Trafalgar: a and b
• [11:43] Viv Trafalgar: but not all the time
• [11:43] Techwolf Lupindo: Is there a way to have all notices sent to e-mail? Sence the client doen't have copy/paste on notice nor any logging of them, It wouold be nice to have a record of them somewhere.
• [11:43] Which Linden: techwolf you mean beyond the historical record in teh group ui?
• [11:44] Which Linden: which is probably not saveable to local disk
• [11:44] Which Linden: and I guess they expire from the group notices ui
• [11:44] Dahlia Trimble: I dont get them when I'm offline as I have IM 2 email turned off in preferences. But I do get a few missed group notices in the blue boxes when I log in after being offline for a while (if thats what you're asking)
• [11:44] Storm Thunders: Within what, 2 weeks now?
• [11:45] Which Linden: Dahlia : well that sounds like the system is working then
• [11:45] Techwolf Lupindo: I think they expire and there isn't a way to save them in the client...unless somone found a hack.
• [11:45] Tammy Nowotny: a and b, not c... there is also an iusse where accumulted messages of all types fail to show up when you log on (although this is usually fixed if you Mute something... anything, actually.)
• [11:45] Dahlia Trimble: as far as I know ;)
• [11:45] MRENG Treves: nice group
• [11:45] Which Linden:  :-) yeah it's busy today
• [11:46] Morgaine Dinova: I can't comment on blue boxes nor emails, it's others who have experienced those. But I know all the group IM symptoms. In addition to the UDP-type problems I mentioned above, there is also posting lag (I've experienced up to 15 mins lag, others have said sometimes a message appears HOURS later), and there is message rejection, plus there is message rejection that nevertheless sends the message successfully. And finally, there is group IM tab silent disabling/disconnectio --- you can sit there with the group tab open, and nothing comes through, despite seeing the messages arrive on an alt's viewer.
• [11:46] Which Linden: Tammy: O_o that's weird
• [11:46] Media Hax: hmmm... nice bamboo you got here Which....
• [11:46] Which Linden: thx!
• [11:46] Dahlia Trimble: I would think that IIM packets have the reliable flag set, but I'm not sure
• [11:46] Morgaine Dinova: So basically everything you can imagine is broken ... except for data integrity.
• [11:46] Tammy Nowotny: I shd probably work up a Jira about that
• [11:46] Aimee Trescothick: that's his family
• [11:46] Which Linden: Morgaine: yeah I am pretty familiar with the group IM woes as well, good summary
• [11:47] Which Linden: data integrity ain't bad,I'll take that
• [11:47] Morgaine Dinova: Hehe
• [11:48] Dahlia Trimble: isnt there an event queue equivilent for group IM IIM packets?
• [11:48] Tammy Nowotny: the act of muting/unmuting evidently pings some server which doesn't always get pinged when you log on
• [11:48] Which Linden: so it does sound like the notice issues would be explainable if it was implemented as a "mass 1:1 IM", though I don't actually know whether it's implemented that way or not
• [11:48] Which Linden: Dahlia: well every message in the message template can be sent over teh event queue
• [11:49] Morgaine Dinova: Which: I've just realized you missed a category 4) -- vicinity chat :-))) Does that fall into any of 1)-3) ?
• [11:50] Which Linden: Morgaine good point
• [11:50] Which Linden: I don't know how that works.....
• [11:51] Which Linden: so a possible 4) there
• [11:51] Morgaine Dinova: I think Sai needs to go around all teh Lindens with his blank IIM matrix and see what can be filled in :-))))
• [11:52] Morgaine Dinova: Wow, an actual typoed "teh" :P
• [11:52] Dahlia Trimble: RL calls... bye all :)
• [11:52] Morgaine Dinova: Cyu Dahlia :-)
• [11:52] Which Linden: heh well no linden is gonna know what those blanks are off teh top of their head -- filling that in will be an exercise in code spelunking
• [11:53] Morgaine Dinova: grins
• [11:53] Which Linden: we're all "teh"-ing here
• [11:53] Morgaine Dinova: Good term :P
• [11:53] Media Hax: <---Thinks to himself..."talking to Which is like talkin to a potted plant..."
• [11:54] Tammy Nowotny: LOL
• [11:54] Which Linden: theraputic and refreshing?
• [11:54] Morgaine Dinova: Careful ..... Which has bamboo sentries posted all around this throne room, you're living dangerously :P
• [11:54] Tammy Nowotny: those are his guard bamboos
• [11:54] Storm Thunders: So I shouldn't try to take a cutting home?
• [11:54] Tammy Nowotny: the other bamboos are his posse
• [11:55] Tammy Nowotny: his peeps, his homies
• [11:55] Media Hax: my dinner
• [11:55] Which Linden: heh...
• [11:55] Morgaine Dinova: Awww, that's cute, a panda on the bamboo :-)
• [11:56] Morgaine Dinova: Anyway, what else can we learn about IM implementaion?
• [11:56] Which Linden: so it seems as though a bit more research is required
• [11:56] Morgaine Dinova: Sai: since pyogp now does IM, presumably you know all about it, right? ;PPPP
• [11:56] Tammy Nowotny: nods
• [11:57] Viv Trafalgar: are we still talking group notices or are we also talking group IM - like text cutoffs and message fails?
• [11:57] Tammy Nowotny: bambooms and a panda are a dangerous mix
• [11:57] Morgaine Dinova: Anything. Filling in some blanks I think
• [11:58] Morgaine Dinova: So what are the salient elements in ordinary group IM? It goes from where to where to where ...?
• [11:59] Which Linden: there's a cluster of machines, each group chat resides on a single machine
• [11:59] Morgaine Dinova: Aha
• [11:59] Which Linden: when you login you tell the machines which host the groups you're a member of that you're online, and they add you to their internal lists of who's online in each group
• [11:59] Morgaine Dinova: And viewers talk directly to each machine in that cluster, or is it proxied via sim?
• [12:00] Which Linden: oh proxied by the sim of course
• [12:00] Morgaine Dinova: kk
• [12:00] Viv Trafalgar: which is then viewable - your online status - from the group info, gotcha.
• [12:00] Which Linden: right, so that list is pretty much just the internal data structure
• [12:01] Morgaine Dinova: Is that a direct description of the physical structure, or just the logical one? (Trying to get a handle on possible failure points)
• [12:01] Which Linden: whenever something happens (a message, someone joining or leaving), the machines iterate over the list of online members and look up their presence to find the simhost to talk to, then they tell the simhost to relay whatever the change is on to the viewer
• [12:02] Which Linden: Morgaine: it's a computer, the physical and logical structures are the same :-)
• [12:02] Storm Thunders: Makes me wonder if there's timeouts or something. I can stay in the same sim, use a group chat a few times, then have it fail a few minutes later.
• [12:02] Techwolf Lupindo: I just got this in a group chat. "[12:01
• [12:02] Which Linden: that looks sorta familiar but I can't recall what that means
• [12:03] Which Linden: there is great possibliity for screwups at the sim end because the sim is mediating all 4 of these different architectures which get dispatched to some other subset of sim-> viewer communications, so it can get its wires crossed
• [12:04] Morgaine Dinova: Which: well the group IM cluster machines are not the same machines as the sim, so that's 2 physical hops minimum, not inside of any computer, unless you've implemented the "cluster" as virtualized guests on a large box.
• [12:04] Which Linden: yes that is correct, the cluster adds a hop
• [12:04] Squirrel Wood: [12:01
• [12:05] Which Linden: what group was that techwolf?
• [12:05] Techwolf Lupindo: Ah, I did have that on when I was on my home sim. I'me on staff, but occually nofly/noscrip/etc are truned on as griefter come and go.
• [12:06] Which Linden: and the viewer also is a source of crossed wires because it gets all these messages and then stick them in the UI in various places
• [12:06] Morgaine Dinova: There's well in excess of 16 bit's worth of groups, right? (Massively more I assume)
• [12:06] Which Linden: yes
• [12:07] Which Linden: 131953 online as of a few months ago
• [12:07] Storm Thunders: We should make a group to keep track of that ;)
• [12:07] Viv Trafalgar:  ::coughs::
• [12:08] Morgaine Dinova: Oh, fewer than I expected, but beyond 64k worth of ports, kk
• [12:08] Morgaine Dinova: Good]
• [12:08] Which Linden: so we're running a bit over, I should declare this day done
• [12:08] Viv Trafalgar: Thank you Which
• [12:08] Which Linden: thanks all for your time and discussion
• [12:08] Morgaine Dinova: The "good" was because since it's over 64k you won't be tempted to make any shortcuts :P
• [12:08] Which Linden: it has been a pleasure
• [12:09] Morgaine Dinova: Thanks Which :-)
• [12:09] Storm Thunders: Thanks Which
• [12:09] Which Linden:  :-)