Brainstorming

This page is all about brainstorming about the upcoming architecture. Add your thoughts here in no particular format. Can be use cases, requirements, scenarios. Maybe shouldn't be too long but long enough to get your idea across. Can also be implementation details maybe but in the lower section.

Usage examples and requirements

General architecture

• always keep the scary numbers of Project_Motivation in mind, ie. design for scalability.
• allow to run a small grid on my laptop.
• allow big-iron sites (eg. LL) to run as proxy cache for small but authoritative private grid.
• allow plug-ins as a general principle for extensibility by attachment; this should apply at all levels.
• Talk to other online virtual worlds to discuss interconnectivity, at least to some degree. For instance, with one online ID and perhaps registration with each virtual world, residents could inhabit seamlessly Second Life then possibly World of Warcraft or any other virtual world, in the same way online surfers can go from one website to another. In short, let there eventually be one huge interconnected 3d environment, with each world as distinctive as a website on the net.
• Integrate within Second Life a platform to allow residents to create their own virtual world which can then be connected via a personal server to the grid. If this is not entirely possible then create a download so a world can be created offline to be later uploaded to a personal server and then be added to the grid. The development of such software could be a challenge to present to the many resident coders. People with technical knowledge should not be the only ones able to add their own sim to the grid. This should be developed in a way to enable everyone to do so.
  • Enable short-term sandboxes launched from within a viewer - think like a traditional LAN party for a multiplayer game, this would do for short social events etc, for 24/7 sims allow hosting by 3rd parties and those 3rd parties (like web hosts) charge for their server space and administration.

Interoperability

• allow different formats for objects or assets in general. You might need different viewers for that (e.g. WoW and SL or EVE are quite different in their structure) but you should at least be able to share an identity. You might need different agents though as e.g. in games you want to store game information in the profile and the profile is attached to an agent. (See Viewer Refactoring below to help with this.)
• for interoperability, objects at the protocol level must be self-describing to allow transformation to the local format.
• allow different types of regions and region formats. Again as an example, EVE and WoW have different understanding on how to partition the world and how to implement it.
• look at things like Multiverse or Metaplace to see how these can be interconnected.
• Use some common IM format to IM between agents. Jabber comes to mind but should probably also be pluggable. Maybe integrating existing multi-format clients (of open source) is an option.

Commerce

• If Second Life is going to live up to its name it should incorporate as many real world activities as possible, and one important area is business. Rather than companies use of Second Life being pretty much restricted to meetings, perhaps this could be expanded to include as many aspects of real world commerce as possible.

• Could Second Life eventually incorporate real world stores on the grid where residents could buy items to be delivered to their homes? Or perhaps service industries could be allowed to function on the grid, so for example real world banking transactions etc could take place. It is not a big leap from this to see how online businesses could be developed within Second Life following the models of for example, Amazon and Ebay.

• Could it be possible to also develop areas such as real world education/training within Second Life with recognised qualifications? I believe a university (or a number of places of learning) within Second Life where recognised qualifications could be gained would attract and help a lot of people.

Objects and Assets

• allow objects to only be allowed to be rezzed on certain regions (adds to the agent's restrictions)
• AND/OR limit object rezzing to specific groups of objects (give region owner ability to control what content is allowed in their regions)
• allow assets to be transferred between agent domains
• allow assets to be accessed from multiple agent domains
• allow truly distributed asset storage
• Think beyond the limitations of prims and sculpties by allowing items made in 3d modelling software to be directly imported onto the grid. Other online virtual 3D environments allow this.
• allow licenses to be attached to objects such as CC licenses etc. Could be some license field in the protocol.
• possibility to add proxies for assets. People running their own servers might want to reduce disk access on the sim machines by adding a second machine with a proxy. Proxies improve response time on webservers with heavy access a lot, so it should work for sims with high traffic, too.

Permissions

• What does a creator need to define? It should be possible for the creator to e.g. define that an object is only valid within a certain grid. The question here might be what makes up a grid. Probably it means a certain agent domain. What does that mean to the user though? If the user bought that object I'd think that he/she should really own it and be able to copy it to other agent domains and other grids. The problem might be what happens if these have lesser security policies and it got copied? (of course in general there is no way to prevent it anyway, I am talking here about different policies for different grids). Should it be some new permission which say "copy to other grids"? When buying such an object you can decide then whether it's worth it or not.
• We already discussed DRM in general and the opinion was that it should be not hardwired into the protocol. Of course it should be possible to plug it in if people think it makes sense.
• What about being able to link from objects. E.g. it is annoying right now that an object cannot be mod,copy,trans but "trans" meaning here that all the copied will be deleted once you give it to another person. Because of that one permission is usually missing. Either you cannot copy it or you cannot transfer it. So here is how it might be

   * Agent buys object which is full perm but it should not be allowed to sell multiple copies of it, only the one you bought.
   * Agent rezzes multiple copies and eventually modifies them
   * Agent give the object to another agent (or sells it). When he does this all the copies should be deleted.
It could be implemented as links to the original object or the object itself is more or less the group of all rezzed items. It makes sure the user can do whatever he wants but cannot just make money from selling multiple copies if not bought.

• Make derivations possible, like e.g. IMVU has it. You buy some object and modify it. You then can resell it but the original creator still gets some fraction for each sold copy.
• For all the mentioned permissions the protocol should provide a means of specifying these capabilities. If you want to sell derivations the other agent domain also needs to support it and needs to be trusted. Possible new permissions could be 'sell copies' and 'franchise' with additional minimum percentage the creator defines (modifiable upwards by the next owner, but not downwards). If object is copy and transfer, seller's copies get deleted with a transfer, if object is set to sell copies, seller's copies are not deleted. Maybe the definition of a minimum price might be interesting, too.
• Access controls beyond the walls of a single managed protection domain like the Linden grid are actually doomed to failure, for numerous reasons detailed by security and cryptography heavyweights who have explored DRM. Nevertheless, extending existing SL object permissions to "trusted" external sites is still perfectly feasible ... but it would be a mistake to believe that such access restrictions would be anything but temporary. This fact will have to be made clear to asset creators to avoid legal problems and discontent later on, as a social measure. As a limited technical measure, a Local_Distribution_Only permissions bit will probably be required as well.

Identity

• identity should be pluggable
  • [Please build a list of desired identity verification systems]
  • OpenID
• various grades of verification should be possible
  • RL Identity Verification:
    • "Is the user exactly who he/she claims he/she is?"
    • Very strong verification. Permanently links ID of account to real-world ID of user.
  • Age Verification:
    • "Is the user old enough to be on this system?"
    • Weak verification. Minimum amount needed to maintain compliance with child online access laws.
  • Unique Verification:
    • "Is this user unique, or is it an Alt?"
    • Weak verification. Minimum needed to enforce bans due to TOS violations.
    • Very difficult to enforce due to ease of changing commonly-used identifiers: IP address, MAC address, hardware serial numbers/profiles, etc.
  • Virtual Identity Verification:
    • In a multi-site, multi-grid, multi-world and multi-national distributed architecture, RL verification is not feasible.
    • RL identity must be safeguarded for residents living under conditions where human rights are not protected.
    • Relevant question: "Is this person XXXX the same XXXX I was talking to yesterday on a different world?"
• Verification must not require sensitive data to pass through insecure systems or require storage of sensitive data (link virtual identity <--> RL identity is sensitive data, too)
• It should be possible to attach an unlimited amount of agent from different domains to one identity

Viewer

• allow all sorts of viewers, from 3D to cellphone to web sites
  • Create a viewer with the absolute minimum functionality possible (i.e. core client functions) and then make it optionally extensible based on client capability.
    • Put core protocol functionality (login/logout, movement etc) into a library
    • Make the server only send client data it has asked for (client sends "i want prims" message and get prims etc)
    • Should be trivial to integrate with other systems if this is done correctly - IRC gateways for IMs for example - note: there should be thoughts about SPAM prevention from the very beginning. A "We can think about that later" wont work here!
  • Allow the client to reassign controls and common functions to a Human Interface Device (gamepad, joystick, customized keyboard, a Bluetooth-connected Wii Remote, etc.)
• Client-side Script runtime
  • In keeping with the idea of the core protocol functionality being a mere library, it could be possible to make the rest of the viewer very modular in this sense and use a scripting language as the glue, or to enable the scripts to execute library calls directly depending on configuration
  • scriptable chat & movements avatar (bot)
  • advanced graphical hud
    • This needs clarification - it seems a good idea to allow 2D sprites rather than only 3D objects built on prims to be mapped onto the HUD, it also seems a good idea to empower the scripts with better capabilities for drawing on such a 2D plane
• Functional refactoring of viewer
  • The current SL viewer implements functions from a large number of domains, many of them concerned with maintaining avatar presence (login, avatar state and behaviour, communications/protocol and event handling, etc), and many others concerned with the actual viewing (managing and rendering a graphic 2D UI and 3D world representation). It is large, monolithic and growing, and quite inflexible in this state, because new clients can be supported only by full code replication, and only one can run at a time per presence because of this. Large-scale refactoring is required here if we desire something better.

• To cater for the stated requirement of allowing all sorts of viewers and running client-side scripting in a flexible manner, then at least the presence-related functions need to be extracted from current graphics code. This would then permit the following scenarios, for example:

[Use cases]

• An intermittent mobile viewer could periodically manipulate or monitor an avatar presence currently logged in from home.
• Within the home, secondary screens running on home theatre equipment would offer a very appealing view of the world.
• A scripted presence could run without any 3D viewer at all, for example a shop transactions handler.
• An audio-only mobile viewer could attend an SL live music concert and still pay the musician's tip jar.
• An event-only viewer could send events to the presence corresponding to RL events, eg. from musical instruments.
• The machinima community could run multiple cameras observing a single presence for great flexibility.
• Under crowd conditions (eg. most live music events), a low-detail viewer could be switched in without relogging.
• SL-based games and sports would be able to run their own bespoke viewer and UI, without reworking presence code.
• A client could be used for remote changing/editing 'information'. Think of a 'blog editor' for SL, for text displayed by hovertext, prim displays and future SVG or HTML displays.

• To achieve this, the "viewer" needs to be redesigned as a view attached to a presence (more precisely, to the client-side presence handler, because the real presence is more properly considered to be server-side), in an N:1 relationship. Client-side scripting would then also attach to the presence handler in a similar manner, for UI control, as a proxy for human input, and also for distributed computation at the request of servers.

• Or, as another way of looking at it, the presence handler (currently a part of the monolithic viewer) needs to become an independent multiplexer process, with graphics viewers and other applications and scripts attaching to it as needed, or not at all if no view is required. The development of many other diverse and quite minimalist viewers then becomes significantly easier.

Agents

• The envisaged massive scaling requires new forms of item organization, search, and access to cope with vast expansion. The current inventory mechanism is already stretched right now, so this issue needs early design attention.
• it should be possible to manage inventories of all your agents under one identity. The idea is to e.g. being able to move items from one inventory to another
• some organisation providing agents should be able to define on which regions an agent is allowed to connect, what he/she can do with object in their inventory
• an agent domain should be able to have certain inventory pools which might be shared among agents (like the library in SL today)
• it should be possible to import and export friends lists or groups of friends
• it should be possible to maintain a friends list among various agent domains (related to import/export maybe but as protocol function)
• it should be possible to copy inventory from one agent domain from another if the object structure is compatible.
• an agent needs to provide various methods of verification
• an agent domain needs to define presence information among an agent's friends list. This should even be possible among various agent domains.
• Each agent needs to be able to define certain permissions on what other agents or groups of agents are allowed to see about him (online status, identity information, etc.)

possible restrictions between regions and agents

• An agent domain should be able to define on which regions an agent is allowed to connect
• An agent domain should be able to define which objects are allowed to be rezzed on which region (e.g. company objects only on company regions etc. This might be related to object pools as described above).
• A region might allow only certain agent domains to connect
• A region might allow anybody to connect but the rights of that agent might be limited depending from where it comes and e.g. which verification mechanism the agent provides.
• Standard specification of rights/permissions in a serialisable form that can be edited by the viewer and uploaded to an authorative per-domain server?

Regions

• allow regions of arbitrary size and form
• allow portals and landmass-style connections between regions or a mix of both
• region should be able to decide which agents to let in depending on the grade of verification (age, RL identity, financial, ... )
• User defined topology. Like bookmarks, but in 2D. Crossing a custom boundary would result in a "walking teleport"
• Multiple instances of a region within a topology. Everyone can have a front row seat for the show.
• Allow as many avatars as will fit in the virtual space of a region. Don't limit the population of a region based on architectural limitations.
• Inactive land acreage and passive prims cost only the price of disk storage. Scripts cost the price of CPU power. Avatars cost the price of bandwidth. In the LL grid, these issues are all coupled together, but this will change in a distributed world.
• Beyond the confines of the Linden grid, land, prim, script and avatar limits should not have a-priori restrictions that are relevant only to the static Linden grid. In the distributed world, local resourcing will determine which limits and costs are appropriate. One fallout from this will be that inactive land acreage will cost next to nothing, since the price of storage is in freefall. Morgaine Dinova 00:59, 24 September 2007 (PDT)

Implementation thoughts

Decentralized assets

• How does a locally-run, disconnected grid provide texture/animation uploads or anything of use without the centralized system to accept the upload fee, read and update the user's inventory, or receive the new asset information?
  • Local database backend stores all the local content, any content on the remote asset servers with appropriate permissions (perhaps a "can download" flag?) can be downloaded for use on a local stand-alone grid
  • Any new assets, or locally modified assets can be re-uploaded
  • "Can download" must imply "Can modify" for practical reasons (DRM will not work!)

Regions

• if we have different region domains will each of them have their own map or would it somehow work to connect certain region domains together while it would still be possible to grow one domains space? Maybe a simplified map will show connected region domains, think of a tag cloud,with the current region+domain largest in the middle, around it the connected domains with only the crossing region(s) displayed.
• Can we see into neighboring regions using a low LOD mesh dynamically created to represent the land and objects in a region?
• Virtualize the regions. If no one is in or near a region, don't waste hardware on it. If a region gets too busy, dynamically split it onto two servers, each taking half of the area.
• Allow arbitrary assignment of geography to processing resources - including dynamic migration.
  • As someone who's developed 3D virtual world simulations before (and always wanted to create something like SL), the first thing that hit me when I entered SL for the first time was that sims were visible to residents (and scripts).
    • A sim is an implementation detail and residents should never need to know they exist. Making that particular initial implementation choice (to divide processing power into a regular grid and distribute it statically over separate servers) has now locked it in for the future - as removing the concept of regions would break a lot of scripted content.
    • However, it is still possible to lay a foundation that can deal with arbitrary assignment of geographic simulation to processing units (including dynamically) transparently and then add a 'backward-compatibility' layer over it which simulates the familiar square regions for legacy content (virtualizes them as suggested above).
    • Such a system could also be implemented to allow non-flat and non-contiguous geography (e.g. like planets, for example). Current continents could be mapped into small surface patches in a 'legacy' area.
    • It would also allow the possibility of distributing the various aspects of simulation of a geographic area differently - such a physics, collision, scripts, occlusion etc. (for example, if there are few physical objects over a large area, one processing unit could compute the physics for the whole area, while a larger number of processing units execute the area's scripts if required)

Currency

• how will virtual currency be handled in a distributed grid architecture?
  • Will LL still support L$ in future, or will it be phased out? (perhaps a virtual currency should have no special place in the grid at all - just as there is no special currency on the web ?)
  • L$ exists as "a limited license right" within Second Life and therefore only makes sense within the official grid(s) owned by LL.
    • Privately-owned grids could be responsible for issuing their own licenses, to throttle their clients' use of those private resources.
      • Allowing private grids to issue their own limited license rights for use of their hardware makes it possible to disconnect them from many if not all centralized systems.
      • Such disconnections may allow us to implement a fully localized grid for use on private intranets or on a standalone system.
    • By their nature, private licenses would be mutually incompatible with the official "L$" licenses.
      • However it may be possible to setup automated gateways to convert between a 3rd-party currency and L$ or any other arbitary currency - something like a cross between paypal and the current popups for when one's L$ balance is too low
    • Alternative servers (or local grids) with accounts linked to LL servers might be able to purchase L$ to be issued to its members for use on LL servers.
• allow for secure transactions other than in L$ via PayPal, credit cards, etc.
• It is possible to maintain a single payment mechanism across multiple grids in a decentralized way using a social-network credit, or Hawala. Actual payment systems using this method exist (for example Ripple) and allow transparent convertibility between different currencies, some of them also allow transfers from and to Paypal and other online payment systems.
• Replace currency with cryptographically signed documents with the electronic equivalent of an IOU?

Assets

• It is possible to implement asset storage in a completely distributed way
  • Assets need not be stored in fixed locations (such as in the 'home' grid of the creator, for example)
  • A completely Peer-to-Peer (P2P) storage protocol is possible

It's worth pointing out that P2P is the only topology that scales with the number of participants in the world. Other topologies have useful properties and get us part of the way (eg. I've been proposing private distributed authoritative world servers fronted by non-authoritative commercial big-iron caches for years on the SL forums), but ultimately P2P has no real challanger. Morgaine Dinova 11:51, 23 September 2007 (PDT)

• • To be successful it would have to retain many of the properties of the current 'fixed' storage schemes
    • Available: Since the individual physical storage providers in a P2P network can't be trusted, a great deal of redundancy would be required to make it unlikely an asset would ever become unavailable
    • Secure: Assets would need to be encrypted using strong PKI where appropriate (for example, so you can be sure nobody but the primary key holder (/ capability holder) can view your script source or edit your object). For the same reason, no asset should be stored in-whole at any single physical location (hence requiring the collusion of a large number of parties to even assemble the encrypted form of an asset in order to mount a cryptographic attack, unless the 'directory' of storage addresses for the asset has also been compromised). Obviously this wouldn't apply to the 'public' form of an asset (e.g. script bytecode, pre-optimized object mesh etc.).
    • Persistent: Some mechanism to control the lifetime of assets may be needed
      • Given the pace of storage technology progress, it may be possible to just keep every asset ever created (ride the wave of progress)
      • If limited asset lifetimes are needed, some kind of distributed garbage-collection algorithm could be employed
      • What would happen to assets that remain accessible but never 'accessed' for long periods of time? (We don't want the 'virtual data archaeologists' who research the 22nd century in one thousand years from now to keep running into cases of assets that are no longer available just because a long time has passed without 'access'!)
      • Will it be possible to delete assets/accounts someday?
  • This may be a hard problem to solve properly now, but just designing an architecture with it in mind and then implementing something similar to the fixed scheme utilized now would leave the possibility open of implementing the more general case in the future.
    • Lets avoid a repeat of the mistake made in the 'design' of the www, where public pages are often stored on servers owned/leased by their creators and routinely lost for the future (save the efforts of the way-back-machine etc).
• Assets should support being signed (and notarized)
  • Perhaps they should even allow arbitrary meta-data to be attached to them by their creators (or anyone with perms) and accessed via scripts
• Assets should be raw data.
  • Asset type, permissions, creator, watermark, etc would be meta-data.
  • Any kind of meta-data could be added and used or ignored as needed.
• How will a region be able to accept an alien object from another system's inventory, or an inventory to accept an object from a foreign region?
  • Would an upload fee make sense for transferring objects to different grids? Who will pay it? The creator? The first one who bought it and travels to the other grid?
  • If so, how would that fee be determined for a completed object?
  • Would some grids (or domains if you'd prefer) be able to reject items from other domains on a permission basis?

Protocols and interaction patterns

I've added this bucket to discuss the related topics of protocols and interaction patterns.

We have several building blocks proposed from Linden Labs, in the form of [certified http], REST, and [capabilties].

We do not have much described from Linden in two other major areas, namely how to choreograph calls into sets of calls which achieve an end (interaction patterns) and how to manage situations where we wish to setup an ongoing stream of interaction between multiple components. This last case is especially cogent when we discuss how region servers interact to provide the illusion of seamlass land.

Zha Ewry - 9/25/07

Capabilities

The capabilities of a domain, region or client do not have to mesh perfectly, but we should set about making sure it doesn't choke.

3D Web

3D Web simply put is forcing the multidimensional flat data of the internet into a single continuous three dimensional space. In the past it has largely failed because of a lack of hardware and attainable social pressure. SL has the potential to fill the 3D Web niche if it can at the region & domain level be integrated with traditional web applications.

Limited Capability Clients

When a client such as a cellphone or any limited capability client (LCC) connects, many of the more advanced features could be turned off for them but if there were a way to automate some of them, then the LCC could still participate.

There are a number of features that would enable supporting LCCs.

• Scriptable capability deficiency handler
  • This could allow the world to automate features not supported by the client so the client could still participate.
    • A cellphone user goes to a show to listen to the audio stream, when they connect to the audio stream the region automatically seats them in a vacant chair.
• Scriptable Auto-navigation waypoint system (choices would be present as menus, etc)
  • This could allow a user who can't see the world navigate it by menu.
• Embedded web pages from prims made the primary focus so they can broken out and interacted with.

There is no requirement stated here that they be scripted in LSL. I don't think LSL would necessarily be appropriate -- Strife Onizuka 17:07, 27 September 2007 (PDT)

Capabilities: The path to 3D Web

With a framework in place to negotiate and script deficiencies in supported features, the ability to support an LCC that is in fact a web browser becomes possible. More importantly, the transition from Web to full 3D Web can be transitioned one feature at a time.

A website could be described as something like an art gallery. Information is organized into rooms and it typically line the walls. The passages between the rooms allow traversal from one page to another. This analogy is nowhere from perfect.

With a dynamic capabilities system, the client could drop down to dumb mode and browse the art gallery like a webpage or it could explore it as a 3D space with all the features turned on.

(Managing conversation in Wikis is tricky, but.. this next bit is mine -- Zha One interesting question for a migration/incremental move between static 3d content and a fully immersive environment, is when you can, and how you can, bring the space from static traditional content to a space where multiple avatars can interact. When we look at this space, it is interesting to notice that we're really doing several things at once in a space like SecondLife. There is the presentation of 3d content, there is the melding of multiple presences with the 3d content and the presentation of that content, the shared space to the viewers interacting in the space. Note that a 3d web, that is to say our current web with 3d content, would be a very static place, there isn't much notion of shared presence in the 2d web. (End of Zha's comment)