User:Callum Linden/Media Transcripts/2007-10-02

Introduction

• * Glenn: Good Morning/afternoon everyone. Thank you very much for joining us today.
• * Glenn: Callum has been focusing on media in SL for quite some time, and we're happy to have him here today to describe what he's working on and to get your input - the best is through use cases, examples of what you would do using the technology. With that, I'll turn it over to Callum to start. Thanks for joining us today, Callum!
• * Callum Linden: Thanks Glenn
• Callum Linden: Hello everyone and thanks all for coming - we're all excited about being able to improve the media support and especially excited about the different kinds of media that will become available in the future. I'm especially looking for lots of input on this - I know everyone has lots of great ideas and I want to make sure we do the right things. I wanted to start by giving a brief overview of what we're planning to do, point you to a public wiki page that has some more details then open it up for questions. The wiki page can be found here: http://wiki.secondlife.com/wiki/Media_on_a_parcel

Current Plans

• Callum Linden: Ok - so stage 1 which I just started working on involves a re-factoring of the current media library and adding support for non-interactive Web content as a media type - static HTML on a parcel I guess you'd call it. The estimated time scale is around 2 months by the time we've done the work, cleared it through QA and pushed it into a release - really pushing to get this out by the end of the year

• Then after than is complete, stage 2 (tentative) will add support for more than one URL per parcel (up to a handful perhaps) and perhaps, support for media on a HUD.

• Stage 3 is where we address interactivity issues (and I'm really looking for input here)... The current thinking is a 3 pronged approach: first off, single user interactivity - you click, you see the changes - no one else does - kind of like using a browser on your own computer. Next, what we refer to as 'poor man's shared browsing' where we pass URLs or (mouse clicks and keyboard input) around between users on a sim. Finally true shared browsing where the browser itself lives on the server and sends pages to everyone at the same time - obviously this won't be as useful for highly dynamic pages like Flash or javascript. My guess is that the final solution will probably end up being a combination of all 3 - perhaps parcel owners will choose the type of interactivity.

• Getting the browser running on the server is one problem to overcome - not sure how to do that yet - but the biggest design challenge is that of permissions and how we allow/disallow residents to interact with pages. etc. Again, use case or input/insight will be much appreciated when we get to this bit

• Ok - next and final stage. Stage 4 - what some have referred to as the holy grail (it's only a model) :) is full support for web on a prim. Since there are lots of other systems that have to be improved / added before we can even start work on this and it's not going to happen for quite some time, I'd like to concentrate on the earlier stages - we can do another session on that closer to the time

• Some of the other issues I'm considering are: security!; access to the media stream (both for control and feedback) from LSL; and finally, using media and notecards both for media source urls / content and the notion of embedding media in notecards.

Media Types

• Callum Linden: Question about text on a prim. The media library design allows for us (and you :-) ) to add new media types easily and I hope one of those will be some kind of light weight text renderer that people can use for rendering just text.

Javascript

• Callum Linden: Another question: interaction between the client and javascript. I think I know how to do that - it's just a case of doing the work and turning it on - that'll be quite a useful feature.

Browser, Flash & Quicktime

• Tao Takashi: Ok, so here is what I want: a) some HTML in a HUD so we can get rid of all those annoying prim interfaces for search agents, etc., b) some sort of easy FLV integration, then ustream.tv and blogTV would work and then c) all of the stuff the others want.. ;-) or simply allowing HTMl floaters like the webtab.
• Nat Mandelbrot: Perhaps hooking into their Apple/H.264 feed could also work, instead of waiting for FLV. Either way...
• Callum Linden:Tao's question was about having a simple browser up in the client - we can do that - there are still some things the embedded browser can't do or does badly though which we'll have to manage.
• Tao Takashi: Basically yes, just having a webtab sort of windows but controllable from LSL would be nice.
• Callum Linden: Ah - ok, yes, that's on the list then.
• Tao Takashi: I mean as an alternative to a true HUD, I thought it might be simpler to implement as the code should be in the webtab already and would not needed to be projected on a prim. Then I guess this is a solved problem by now.
• Tao Takashi: I guess with ubrowser basically ;-)
• Count Bayliss: Like a framed page?
• Callum Linden: Question about QuickTime being the core media support.
• Tao Takashi: These days Flash would definitely be a lot easier to handle user-wise
• Callum Linden: yep - it's (Flash) going to be the backbone of our 'movie' support for now since it's (easy) x-platform and along with GStreamer on Linux gives us a wide range of media support.
• Joel Greenberg: Is that Flash on a prim, or in a window?
• Callum Linden: We'd really like to have Flash working - both for things like watching movies and of course, interactive applications like games, and uh - other interactive stuff ... Flash on a parcel, then Flash on a prim eventually. I spent a long time trying to make this work and made some progress on Windows but not on any of the other platforms.
• Marissa Bergbahn: Flash on a prim would be a big improvement on many fields.
• Callum Linden: Flash playback works now (with some caveats) - it's the interactivity that's really hard. So you'll probably be able to get videos working for example on a parcel basis after stage 1. You just won't be able to interact
• Tao Takashi: FLV would be a start for streaming scenarios. You would not need an extra QT server.
• Nat Mandelbrot: @Tao: you don't need QT streaming server to stream non-live Quicktime video
• Callum Linden: Yep - there is a lot of FLV content out there and great tools for authoring it so it's something high on the list.
• Callum Linden: I'm looking at other alternatives - for example, figuring out how to make Flash work as a native media type rather than embedding it in the browser. If anyone knows anything about that and wants to help - please get in touch.
• Tao Takashi: I am talking about live streaming :-)
• Callum Linden: Question: the new media types suggested will be linked to a parcel until the last stage when we enable it for a prim.

Autostart

• Joel Greenberg: Will landowners be able to set streaming media to autostart instead of requiring avatars to press a play button?
• Nat Mandelbrot: Second Joel's question - need to streamline media integration into experience so we can stop telling users "now see that little play button - click that!"
• Callum Linden: Joel: good question - the reason we haven't done that is that people flying over the parcel will experience lag / stutter if we auto play.
• Joel Greenberg: How about continuous streaming?
• Callum Linden:Nat: agreed - other than by writing code to make it not stutter, I'm not sure how to address that.
• Nat Mandelbrot: Perhaps a autostart distance range?
• Callum Linden:Nat: yep - some kind of hysteresis perhaps
• Tao Takashi: But maybe it would be sufficient if everybody has their own web page but some script controls what can be seen by simply setting the URL and reacting on clicks by either LSL or via an http gateway
• Callum Linden: Joel: so when you leave the parcel, your stream keeps going you mean ?
• Joel Greenberg: Yes, the parcel keeps going. Here are couple of use cases: a jumbotron, like in Times Square; HD screens in clubs streaming the action in another room; Sporting events /w/ TV screen in scoreboard
• Callum Linden: Yep - all those would be cool but if the client had to load them when you get close enough to see them, there is going to be a hitch.

Priveleges & Permissions

• Callum Linden: Question about privileges on media
• Callum Linden: The hardest part about interactivity is coming up with a permissions / privilege system that will allow owners to specify who gets to do what on their parcel.
• Nat Mandelbrot: I think a user should have the ability to override their personal media stream and/or scripts running under that user's permissions
• Callum Linden: Yes, that's probably true
• Callum Linden: Question about using LSL to play external wav files. that's not something i've considered - there is a separate project that addresses some of the issues with our audio playback - I think that might fall into that rather than this one. Interactivity is obviously very important - we want to get it right though and there are some interesting problems to solve first - some have been mentioned already - permissions for example.
• Callum Linden: Others include making sure that people see the same page - share cookies and authentication for example.
• Tao Takashi: Shared cookies sounds a little scary.

Security

• Callum Linden: One of the big things to think with auto play media is security - you probably don't want your client to visit a web page automatically when you get close to a parcel. if every there was an exploit in Firefox for example and visiting a page let someone own your machine, you wouldn't want to go to that page.
• Callum Linden: Security will be a big concern I think - both locally in terms of giving people access to your media stream and the sorts of security concerns with the media types I mentioned earlier.
• Nat Mandelbrot: I think at some point though, people just start click "yes" all the time, similar to what happens with Windows security/firewalls, etc.
• Callum Linden: That may be true. Perhaps one solution is to offer it as an option - 'always show me media' for people who are comfortable with it and 'always ask me' for people who aren't.
• Eloise Pasteur: It would carry over well similarly to always play audio/video in preferences.
• Tao Takashi: There will also be the group of people who are afraid of giving out their IP address to whoever owns that parcel and stream.
• Callum Linden: That was raised as an issue with the Quicktime media - other than proxying through our servers first, I'm not sure of a way around that.
• Tao Takashi: Well, give them the option to switch all these things off and I think this should be sufficient.

Parcels By Height

• Xu Caliber: I have a question about media parcels and elevation -- are there any plans to enable separate media parcels at the same xy coordinates but different z coordinates? Ie, if I have a building with different floors in it, will I be able to have different media parcels on each floor?
• Tao Takashi: Great question, 3d parcels in general might be nice to have.
• Callum Linden: Stage 2 will address multiple URLs per parcel. Perhaps just a handful - so yes, you'll be able to place them at different heights; also put them on a HUD which I think will open up all sorts of other interesting possibilities.

HUDs

• Joel Greenberg: Will media in HUDS behave the same way as media in parcels? Or, will individuals be able to view different streams?
• Callum Linden: Some of the issues with media go away when we place it on the HUD - you don't have to share it of course so that makes things easier - I hope we'll be able to have multiple URLs per HUD since the number you can view will depend on the capabilities of your own machine.
• Tao Takashi: I'd think that I have my personal stream in my HUD.
• Callum Linden: and be able to carry it around between parcels you mean ?
• Tao Takashi: Yes, like I could do an ipod sort of thing.
• Nat Mandelbrot: Yeah; I think we are all imaging some similar apps :)
• Callum Linden: That's been mentioned (a lot) and i'll bear it in mind :). it won't be part of stage one for sure since that's purely parcel based - might be able to make it part of stage 2 though -- some kind of media player that lives in your HUD and follows you around.
• Nicolas Broome: I agree this would be fine, have the it in the HUD
• Tao Takashi: Second Life radios will love it ;-), if it's also about mp3 streaming.
• Nat Mandelbrot: media/html-rich HUDs open up a huge rage of possibilities for web integration, but perhaps we are also just resting on our 2D laurels.

Notecards

• Eloise Pasteur: If you're going to use notecards as a source for media content, will you be changing the dataserver to return more than 255 characters per line too?
• Callum Linden: Yes, that is one consideration we'll have to address - do people have use cases for this kind of thing?
• Tao Takashi: A notecard should be some sort of HTML editor with embeddable media URLs and playback ;-)
• Eloise Pasteur: Well I'd love to be able to import, say, an essay from word into SL, paste it into a notecard and put it up for display to discuss in class. Or a passage from a book.
• Eloise Pasteur: You seem to be suggesting other ways I can do that...
• Callum Linden: No, I'm just trying to get an idea of how people would like to use it
• Nat Mandelbrot: @Eloise perhaps someone could use Open Office to write a custom media handler?
• Eloise Pasteur: But sharing a notecard and having it read the whole paragraph properly would be wonderful. Oh, yes, there are ways around it, certainly.

Forms

• Nicolas Broome: Forms for interaction would be nice. e.g. for opinion polls
• Callum Linden: The idea is to unify all the media support so that media on a parcel, media on a prim and media in the UI will become the same thing. - for example, we have an LLWebBrowserCtrl widget right now - I'm hoping to make that an LLMediaControlWidget and let you put any kind of media into a widget.
• Callum Linden: Understood though - it would be great not to have to go via your own web server to store that on.
• Nicolas Broome: Payment is another use case.
• Callum Linden: Can you elaborate Nicholas?
• Nicolas Broome: E.g. pay by credit card like users are used to in the web, so users can enter their payment details to a form and submit it to a shop owner.
• Callum Linden: Oh, I see - yes, that's a possibility and probably involves a lot of discussion on the security aspects I mentioned earlier.
• Tao Takashi: And some in-world interactivity needs to be in place.
• Callum Linden: The embedded browser supports ssl so there is no reason why that can't work though.
• Tao Takashi: Well, maybe not, but nevertheless it would make things easier.
• Nicolas Broome: Ssl sounds good.
• Nat Mandelbrot: Just don't tell Citibank you are running a shared browser on a server! :)
• Marissa Bergbahn: Snything planed yet to enable forms, users can fill out in-world?
• Tao Takashi: Those search HUDs I mentioned definitely would need a search term box ;-)
• Callum Linden: Forms won't come until we have interactive HTML. ok, noted - forms are important - thank you.

Proxy

• Callum Linden: Question about proxy support: that's on the cards (but only for web content right now).

HTTP

• Tao Takashi: Btw, is http textures a related topic?
• Tao Takashi: Probably when you are able to set a web address per prim this would enable this unless it's a limited amount.
• Callum Linden: Not really - that's a separate project - there might be some crossover afterwards though.
• Tao Takashi: Would be cool to have that.. should be quite static compared to web pages and would solve some of my use cases.
• Joel Greenberg: Will the limits on http communication with Flash and browser support be similar to those in LSL?
• Callum Linden:Joel: I think so - do those limits feel ok now ?
• Joel Greenberg: No :(
• Joel Greenberg: We're doing a lot of world-to-web communications.

Providing Use Cases

• Callum Linden: We have some already but if people could add use cases for the sorts of things they want to do, that would help me enormously when trying to figure out what to do and what order to do things in.
• Glenn: Post your use cases in the Discussion page
• Callum Linden: Please add to the discussion page - I have a watch set up on it so I should get notified of changes.
• Tao Takashi: I will try to add some. this pages looks quite cool actually. Always good to see use case in a more formal way :-)
• Callum Linden: Don't worry about formality :) just a sense of what's important would really help.
• Glenn: What would help Callum, and us, most at this point is if each of you could give one example of your most important use for media in SL.
• Eloise Pasteur: I've got two instant ones... that are related...: Nicely formattable text display and ability to make forms to fill out For simple text display.
• Callum Linden: Understood. have you played with the text formatting that quicktime supports ? it's actually pretty good for just text.
• Tao Takashi: The problem with QT is probably that only a few people have an idea how that works.
• Callum Linden: And works now.
• Nicholas: I have a few examples: forms (e.g. opinion polls in SL), payment (credit card etc) , product configurator (for consultants e.g. automobile), collaborative working (see google docs).
• Tao Takashi: I think this wiki page should list the requirements first. I feel like jumping in the middle of something there, esp. as it becomes quite technical very quickly
• Callum Linden: understood Tao - that was an internal document but I felt it was important to get it public so people could see what we were up to - but you have a good point.
• Tao Takashi: Actually I would move some of the stuff down to implementation, like the components diagram.
• Callum Linden: Yep - I think we'll have a pass over it.

Closing

• Glenn: Please take some time in the next few days to post your use cases to the wiki, since that will help a lot.
• Nicolas Broome: Dream of the result in 2/3months :)
• Callum Linden: I'm going to do everything I can to get stage 1 complete before the end of the year.
• Glenn: Thank you very much,Callum, for taking time to join us today. Thank you all for coming, and for your interest and questions.
• Callum Linden: Tthanks everyone - I already have a good sense of what's important, and really appreciate you all coming and for the questions.
• Tao Takashi: I hope you will also embrace the open source community with this. some post to sldev might be cool; maybe they can even help you, as many things seem to be mostly client side.