Second Life Wiki - User contributions [en]

User:Finrod Meriman

2015-02-16T18:46:40Z

Finrod Meriman: Blanked the page

User:Finrod Meriman/Scalability

2015-02-16T18:42:37Z

Finrod Meriman: Blanked the page

User:Finrod Meriman/AWG Feedback

2015-02-16T18:42:13Z

Finrod Meriman: Blanked the page

User:Finrod Meriman

2015-02-16T18:41:44Z

Finrod Meriman:

* [[User:Finrod Meriman/AWG Feedback]]
* [[User:Finrod Meriman/Scalability]]

Scalability VAG

2007-11-18T00:53:58Z

Finrod Meriman:

: This is an early draft so scope and focus are still fairly open. Please add comments to the [[Talk:Scalability VAG]] if you have slightly different concerns so that we can try to converge on a common viewpoint. Where appropriate, this discussion could expose the need for related sub-VAGs in this area.

= Purpose =

The '''Scalability''' [[Viewpoint Advocacy Groups|'''V'''iewpoint '''A'''dvocacy '''G'''roup]] exists to provide input for architectural design that is focused on the expressed [[Project Motivation|Project Motivation]] of the project and the banner motto of AWG: "Making the Grid a Scalable Place".

More specifically, the Scalability VAG is concerned with identifying all important dimensions of scalability, determining the relevant scaling pressures as numerically as possible, establishing both the end limits and realistic near-term goals for scalability and scaling, and ensuring that these concerns are addressed in the evolving architectural design.

The Scalability VAG is a technical VAG.

==== The Scalability VAG is an umbrella VAG ====
In view of the magnitude of the task described above, it is expected that the Scalability VAG will devolve into separate VAGs each concerned with one dimension of scalability, as well as other specialized VAGs. To this end:
# This Scalability VAG will seek to avoid specialization, to avoid overlap with sub-VAGs.
# This Scalability VAG will provide references, analysis and tools as a resource for sub-VAGs.
# Where other VAGs have viewpoints that ''imply'' the need for scalability but lack technical expertise or interest:
#* This Scalability VAG will adopt the relevant scalability concerns, possibly in a sub-VAG.
#* This Scalability VAG will assist towards conformance of the architecture with those concerns of that VAG.

See the [[Architecture Working Group]] and the [[Viewpoint Advocacy Groups]] for more information.

Known sub-VAGs:
:* [[Event Scalability VAG]]

=== Areas addressed by this viewpoint ===
* Identification of all [[Scalability_VAG#Dimensions of scalability|dimensions of scalability]] relevant to this platform.
* Scalability of the new system architecture in all dimensions of population growth:
:* For each dimension identified:
:::* determination of the relevant growth metrics
:::* assessment of current constraints on scalability
:::* consideration of new designs with improved scalability
:::* detailed scalability analysis of all design proposals
:::* measurement and presentation of levels of conformance
* Specific elements of maintenance scalability, especially in regard to graceful degradation:
:* Dynamic scalability of system architecture in response to:
:::* anomalous growth of total population over short periods of time
:::* variation of concurrent online users over the daily and weekly cycle
:::* anomalous population dynamics resulting from media events and special holiday periods
:::* unplanned resource malfunction and equipment failure
:::* withdrawal of services or resources for maintenance.
* Tools and methods by which scalability of designs and systems can be evaluated and documented.

=== Areas not addressed by this viewpoint ===
* Scalability of interoperation is not addressed at this time, namely the rate and ease or otherwise with which worlds and grids can interconnect with the SL grid and vice versa.
* Functional scalability is not addressed at this time, for instance the rate and ease or otherwise with which worlds and grids can add new service definitions while maintaining interoperability.

= Current State of Play =

* It's hard to know exactly what to analyze for scalability in the new architecture until we know how the grid resources are partitioned into REST services. Candidate partitionings have not been proposed yet, so this is on hold while work continues in the [[Core Grid Services, Protocols, and Structures VAG|REST Services VAG]] and through discussion with Zero at his Office Hours.
* Some tools were deemed appropriate for applying actual workloads to the reference implementation of REST services so that we could spot descaling trends early on. That work is in progress in the [[Multi-Process Client VAG -- draft]], and despite that still being in draft form and at a very early design stage, there is reasonable progress there. (Feel free to join. :P)
* The analysis of mitigating factors which reduce the arithmetic scalability requirements of regions to below the numbers calculated in [[Project_Motivation]] was started but not completed. Completion of this work is a priority.

= Scalability VAG Glossary =
The following terms and abbreviations are defined for this viewpoint with the purpose of reducing repetition and providing joint terms of reference for stakeholders of this VAG. A small handful of terms overlap with those in the [[Architecture_Working_Group_Glossary|main AWG glossary]], but this is only to localise this VAG's lookups and to avoid cluttering up the main glossary with detailed scalability examples and explanations.

==== Multi-Factor Scalability Limit (MFSL) ====
: A scalability limit which results from the combination of two or more partially disjoint or simpler scalability limits, ''ie.'' '''PPSL'''s. It is of high practical value in specifying project scalability goals in a concise manner and without engaging further in-situ scalability analysis or discussion.

: Example: under a (hypothetical) premise that dislike of crowds reduces event attendance by half when events attract more than 1,000 participants, then a '''PPSL''' scalability end-limit of 20,000 would be reduced to an '''MFSL''' of 10,000 participants.

==== Population-Proportional Scalability Limit (PPSL) ====
: A scalability limit which results from direct arithmetic scaling of a current value along a given dimension in proportion to growth in a related dimension. This is a ''raw'' scalability limit, in the sense that it is ''as far as possible'' unaffected by any other up-scaling nor down-scaling factors. As such, it provides a well-defined element for use in scalability analysis rather than an actual target for scalability.

: Example: the end limit of scalability in the number of avatars at an event, calculated from the projected growth in user population (2 bn) versus the current level of interest in attending a given event (100) given the current population (10m), yields a '''PPSL''' of (2000*100/10) = 20,000 people interested in attending that event.

==== Resource ====
: Any finite architectural component or property or behaviour that is subject to exhaustion or which can become a bottleneck to system performance. Resources are the primary focus in designing for scalability. The resource-based approach to highly scalable design involves removal of barriers to massive replication of resources, in conjunction with providing a massively scalable access mechanism for concurrent acquisition of those resources. The potential scalability of a system is however limited by additional factors, especially interdependency of resources, secondary resource exhaustion, etc.

: Examples: client-server bandwidth, agent pool depth, database access mechanism, locks, transaction times, utilities, services.

==== Scalability ====
: The ability of a system to grow effectively ''in a given dimension'' in proportion to the amount of resource or capacity provided. The given dimension is often associated with a related dimension. A specific system may attempt to be ''scaled'' by adding resources, but this is effective only if the system is ''scalable''. If a system is not scalable then adding more resources will not scale it, and may even descale it further. Example: scalability in the number of avatars at an event, viewed against the size of the user population, in proportion to the hardware allocated.

= Source of Viewpoint =
No existing sources for this viewpoint have (yet) been sought. This viewpoint is however highly reusable, and therefore existing sources are very likely to exist and should be investigated.

Sub-VAGs that wish to reuse this viewpoint are required to specify the extent of reuse through inclusion and exclusion.

= General concerns addressed by this viewpoint =

The foregoing scalability viewpoint is founded on a few broad concerns:

:* That all important dimensions of scalability be identified, to avoid costly surprises further downstream.
:* That identified dimensions of scalability be grounded in fact and basic arithmetic as far as possible.
:* That end limits of scalability be accompanied by well-reasoned near-term and mid-term projections.
:* That scalability of the architectural design be accompanied by concrete proposals for achieving it.
:* That all proposals are subject to effective scalability analysis and ceiling estimation.
:* That design costs and impact on other VAGs and aspects of the project be assessed.
:* That the concerns expressed in this viewpoint be addressed in a conformant system architecture.

= Dimensions of scalability =
Like an incoming tide, the pressure of population growth is unstoppable (short of business throttling or termination), and therefore most discussion of scalability centers around population-related issues. However, these are not the only ones that matter, so here we also list other significant dimensions of scalability of relevance to Second Life and other virtual worlds, for subsequent examination.

== Scalability with respect to population growth ==
The following 4 population-related dimensions of scalability are captured from [[Project Motivation]] as an initial basis for this VAG. The estimates in dimensions 1-3 and the '''PPSL/MFSL''' (see [[Scalability_VAG#Scalability VAG Glossary|our glossary]]) numbers are likewise only an initial basis:

# Scalability of world extent:
#:* 60 million regions (probably more) (60 was mentioned [http://wiki.secondlife.com/wiki/User:Zero_Linden/Office_Hours/2007_Sep_18 here] by Zero Linden )
# Scalability of world population:
#:* 2 billion users (''rationale for initial estimate required'')
# Scalability of concurrent users:
#:* 50 - 100 million concurrency across all client types (''rationale for initial estimate required'')
# Scalability for events (concurrent users in a single region):
#:* PPSL: 20 thousand per region ([[Talk:Project_Motivation|calculated proportional to world population]])
#:* PPSL: 100 thousand per region ([[Talk:Project_Motivation|calculated proportional to concurrent users]])
#:* MFSL: ''analysis to be done''

== Scalability of object population ==

== Scalability of object complexity ==

== Scalability of object dynamics ==

== Scalability of physics ==

== Scalability of interoperation ==

= Specific concerns within this viewpoint =
Here we define the specific concerns relevant to each dimension of scalability:

= Use Cases =

= Proposals and Analysis =

= Organization =

== Joining ==

Anyone with an interest in this Viewpoint is welcome to join. You should join the [[AW_Groupies]] group in Second Life.

== In world meetings ==

We meet once a week in-world and more if people are available.

Also members are active on the wiki and in the SLDEV mailing list.

Meetings Schedule:

=== Meeting Agendas ===

* TBD

=== Chat Logs ===

* TBD

= Architectural Descriptions/Views used to express this viewpoint =
This section identifies:
# the general form or representation of [[Architecture_Working_Group_Glossary#Architectural descriptions and views (ADV)|ADV]]s required to express the viewpoint
# the elements within such ADVs which will be used to express the viewpoint
# how these elements within such ADVs map to the concerns of this viewpoint
# the traceability to viewpoint concerns required for conformance with the viewpoint.

None decided.

= Tools employed by this VAG =

* Normal wiki textual and graphic representations are expected to be sufficient for this VAG.
* Programmed tools are expected to be developed for scalability testing and measurement. See [[Multi-Process Client VAG -- draft]].

= External Links =

= VAG wiki structure =
There is a balance to be found between excessive length of pages and excessively fine partitioning. This VAG is a technical one, which means that cohesiveness of subject is especially important, and partitioning should be driven mainly by the need for clarity rather than by simple length. Suggestions for improving partitioning under pressure of growth (highly on topic here) are very welcome, including but not limited to sub-VAG devolution.

= Members (Stakeholders) =
''Please note that the Scalability VAG is a non-hierarchical VAG in every respect, without exception. Any tags supplied with names are purely informational. Stakeholder ordering is alphabetical.''

:[[User:Jesrad Seraph|Jesrad Seraph]] 04:07, 12 November 2007 (PST) - per-resident sim scale approach advocate
:[[User:Morgaine Dinova|Morgaine Dinova]] 02:21, 18 October 2007 (PDT) - Founder, analyst
: [[User:Zha Ewry|Zha Ewry]] 07:57, 25 October 2007 (PDT) - Middleware and core architecture scalability, WEB scale approach advocate
:[[User:Finrod Meriman|Finrod Meriman]] 16:42, 17 November 2007 (PST) - Distributed systems researcher

[[Category: AW Groupies]]

User:Finrod Meriman/Scalability

2007-11-18T00:47:03Z

Finrod Meriman:

== Some thoughts on scalability ==

In the kickoff meeting of the Architecture Working Group Zero presented
several scalability objectives: a platform that could support 60 million
regions, 2 billion total users, and 50 million concurrently connected
users. These objectives reflect Zero's frequently stated intention to
"instill a deep sense of platform" in the AWG.

One way to achieve the desired scalability objectives is to create a
large number of independently hosted regions. As proof, consider how the
current Web scales: any organization (or individual for that matter) can
run an HTTP server and publish content. As of October 2007 Netcraft
reports more than 140 million web servers surveyed on the public
Internet (with countless additional servers deployed inside corporate
firewalls). Scalability of the Web occurs largely because each server is
independent of others.

Although Second Life could scale to meet Zero's targets by creating a
number of independent regions (or a large number of independent copies
of Second Life itself), there are implicit objectives that affect
scalability as well.

=== Scale Identity ===

As the number of organizations hosting regions increases, the complexity
of managing a common identity increases. It becomes necessary to
establish trust relationships between different organizations, an issue
that his historically been limited by complexity of management and
competing business models (there is a perception that the organization
that owns the identity owns the customer).

In the Web each organization typically implements an identity service to
track its customers; e.g. I have distinct accounts at web sites hosted
by Amazon, Ebay, ESPN, and CNN. Multiple attempts have been made to
federate identity across multiple sites with relatively little success.
The wireless carriers have had some success with identity federation:
carriers establish roaming agreements that tie customer billing records
together across multiple domains; my identity as stored in my SIM card
enables the carrier hosting my roaming cell phone to bill my primary
carrier for the minutes I use.

The extent to which my identity scales (that is, the number of regions
over which my identity is known) will determine the extent to which my
avatar can move transparently among regions. (And by the way, this is
really hard.)

=== Scale Per Region Concurrency ===

The number of concurrent users in a single region (sim) right now is
less than 100 (much less for practical use). Scaling the number of
concurrent users opens the potential for many new applications including
concerts, conferences, business meetings and large sporting events.

If we generalize the notion of "concurrent users in a region" to mean
the number of users that can interact with each other at a single event,
then it becomes a bit easier to define the expected behavior. If
fifty-thousand people attend a baseball game, they all watch the same
game, they see each other, they share a common interest in the outcome
of the game, and operate within the same space. The shared social
experience is what differentiates interactions in a virtual world from a
simple webcast. Ideally it should be possible to host a large event
(like the Super Bowl) where the fans have sufficient fidelity of
interaction to enable a "virtual wave" through the stadium. However,
rich social interactions generally occur with others who are nearby.
Scaling the number of users at an event can leverage user expectations
to maintain a reasonable experience.

Early "large" Web events like coverage of the 1998 Winter Olympics in
Nagano enabled literally millions of people worldwide to remotely
"participate" in the event (just a reminder: that was only five years
after the Mosaic was launched). What would it take to scale a virtual
world to handle that kind of traffic?

=== Scale the Number of Objects ===

=== Scale the Complexity of Objects ===

=== Scale the Complexity of Object Behavior ===

How many objects can have scripts? What can scripts control about
objects? How difficult is it to define new behaviors?

=== Scale Geographies ===

The Second Life community includes users from all across the world. The
geographical distribution of users means that network latency and jitter
will be highly variable for the users in a region of the virtual
world. With the current architecture a region where most of the users
are from Europe may be co-located with a server where most users are
from Asia or North America.

Consistent network behavior is extremely important for MMOG's where
users with low latency connections have a decided advantage over those
with a high latency connection. Although many interactions between users in
Second Life do not appear to have such tight latency bounds, the impact
of network behavior on end-user experience requires additional
study.

Some background material on the impact of network behavior is available
from [http://www.thefengs.com/wuchang/work/ Wu-chang Feng] at Portland
State University.

User:Finrod Meriman/Scalability

2007-11-15T18:49:48Z

Finrod Meriman:

User:Finrod Meriman/Scalability

2007-11-14T00:12:26Z

Finrod Meriman: /* Some thoughts on scalability */

User:Finrod Meriman/AWG Feedback

2007-11-13T21:13:13Z

Finrod Meriman:

== Some initial reactions to the architecture... ==

'''Disclaimer''': For those who don't know, I work for Intel. Although the comments are generally personal, I am participating in the working group as Intel's representative. I don't speak formally for Intel, but my affiliation certainly influences my position and interests.

=== 1) Architecting the grid is not a once-and-done task. ===

In the meeting we discussed two primary motivations. The first is fairly
short-term: make it possible to scale performance of the second life
grid. The second is more long-term: lay an architectural foundation on
which virtual worlds (in general) could scale to millions of users
supporting a very diverse set of interactions (everything from WoW to
SL). Both motivations are very reasonable. But there is an obvious
tension between the two. In the short-term it is necessary to consider
the transition from the existing SL Grid though long-term scalability
might be best served by a more complete overhaul. It seems pretty
obvious that re-architecting the grid needs to be viewed as a multi-step
process not a once-and-done event. There will be lots of good ideas to
support a fully general, scalable grid that need to be put off for later
consideration (or we'll never make progress).

=== 2) Make our architectural "views" explicit. ===

There is tremendous value in having multiple "views" of an architecture
(for those who care, you can look for IEEE 1471 in wikipedia to get an
idea how to define views formally). In the meeting I heard three
different views presented. The "communication view" described the
architecture in terms Certified HTTP using RESTful interactions and
LLSD. Most of the detail that was presented focused on the "interface
view" or "component view". Each of the agent & region domains were
broken into services (with standardized interfaces) based on stateful &
stateless interactions. And we discussed a "transactional view" where we
hinted at how to sequence the invocation of various services to
accomplish a higher level task (eg the login example cascaded through
several different invocations... all of which were mandatory to set up
the correct state).

One view that was missing from the discussion was the "data" or
"document" view. What I mean is that we didn't talk about what the data
looked like, where it moved around the system, and how the various
functions transformed the data. I'm sure some of this will be derived
from the current grid architecture, but it still should be thought
through & documented. And of course there is the question (for the
long-term discussion) as to whether there are other standard 3D object
formats that should be adopted to encourage long term stability
(e.g. collada, obj, ...).

Just a couple more comments on the communication piece... Putting aside
my biases about REST... One question I wanted to ask is how the
architecture supports caching & replication. How hard would it be to
take the appropriate pieces and distribute them through something like
Akamai (or some other distributed web cache/content distribution
system)? Should the architecture explicitly include a separate CDN to
improve performance? This question is, in some sense, a question of both
the communication and document views. And it might impact how the
services are decomposed.

=== 3) We're missing a good functional decomposition. ===

Most of the architecture drawings that were presented focused on the
separation into region & agent domains. Clearly this is an important
separation for enabling multiple administrative domains. However, I
think the first structural partitioning should be along functional lines
(yes, the agent & region domains already capture an implicit functional
breakdown). Just to be concrete... I've been asked the question, by
several residents, why IM is so tightly bound to SL. There are many good
reasons to bind them (managability for one), but there are also good
reasons to separate them. In the case of IM, there are already a number
of very good and extensible IM systems such as XMPP (jabber & google
talk).

A more thorough examination of the functions might help to figure out
which services belong in which boxes.

=== 4) Managing trust is going to be really hard. ===

It appears that transactions/patterns (such as login) involve services
in multiple domain. In the case of login, for example, the agent that is
created on behalf of a particular user executes services in a region
domain where the avatar is located. The agent domain proxies trust for
the user potentially across administrative domains. Proxying trust also
creates potential problems for tracking/controling where bits are
sent. This has impact in managing intellectual property. (Not that the
architecture needs to support DRM, but it shouldn't make it impossible
to provide some limits on distribution of IP.)

User:Finrod Meriman/

2007-11-13T21:10:23Z

Finrod Meriman: User:Finrod Meriman/ moved to User:Finrod Meriman: Remove slash

#REDIRECT [[User:Finrod Meriman]]

User:Finrod Meriman

2007-11-13T21:10:23Z

Finrod Meriman: User:Finrod Meriman/ moved to User:Finrod Meriman: Remove slash

For those who don't know, I work for Intel. Although the comments are generally personal, I am participating in the architecture working group as Intel's representative. I don't speak formally for Intel, but my affiliation certainly influences my position and interests.

Feel free to contact me in world or send email to mic.bowman@intel.com.

* [[User:Finrod Meriman/AWG Feedback]]
* [[User:Finrod Meriman/Scalability]]

User:Finrod Meriman/Scalability

2007-11-11T05:10:04Z

Finrod Meriman: New page: == Some thoughts on scalability == ''DRAFT'' In the kickoff meeting of the Architecture Working Group Zero presented several scalability objectives: a platform that could support 60 mill...

== Some thoughts on scalability ==

''DRAFT''

In the kickoff meeting of the Architecture Working Group Zero presented
several scalability objectives: a platform that could support 60 million
regions, 2 billion total users, and 50 million concurrently connected
users. These objectives reflect Zero's frequently stated intention to
"instill a deep sense of platform" in the AWG.

One way to achieve the desired scalability objectives is to create a
large number of independently hosted regions. As proof, consider how the
current Web scales: any organization (or individual for that matter) can
run an HTTP server and publish content. As of October 2007 Netcraft
reports more than 140 million web servers surveyed on the public
Internet (with countless additional servers deployed inside corporate
firewalls). Scalability of the Web occurs largely because each server is
independent of others.

Although Second Life could scale to meet Zero's targets by creating a
number of independent regions (or a large number of independent copies
of Second Life itself), there are implicit objectives that affect
scalability as well.

* The number of regions that share a common set of user identities

In the Web every organization typically implements an identity service
to track its customers; e.g. I have distinct accounts at web sites
hosted by Amazon, Ebay, ESPN, and CNN. Multiple attempts have been made
to federate identity across multiple sites with relatively little
success. In contrast, wireless carriers have established roaming
agreements that tie customer billing records together across multiple
domains; my identity as stored in my SIM card enables the carrier
hosting my roaming cell phone to bill my primary carrier for the minutes
I use.

The extent to which my identity scales (that is, the number of regions
over which my identity is known) will determine the extent to which my
avatar can move transparently among regions. (And by the way, this is
really hard.)

* The number of users that can concurrently share a single region

The number of concurrent users in a single region (sim) right now is
less than 100 (much less for practical use). Scaling the number of
concurrent users opens the potential for many new applications including
concerts, conferences, business meetings and large sporting events.

If we generalize the notion of "concurrent users in a region" to mean
the number of users that can interact with each other at a single event,
then it becomes a bit easier to define the expected behavior. If
fifty-thousand people attend a baseball game, they all watch the same
game, they see each other, they share a common interest in the outcome
of the game, and operate within the same space. The shared social
experience is what differentiates interactions in a virtual world from a
simple webcast. However, rich social interactions generally only occur
with others who are nearby. Scaling the number of users at an event can
leverage user expectations to maintain a reasonable experience.

Early "large" Web events like coverage of the 1998 Winter Olympics in
Nagano enabled literally millions of people worldwide to remotely
"participate" in the event (just a reminder: that was only five years
after the Mosaic was launched). What would it take to scale a virtual
world to handle that kind of traffic?

* The number and complexity of objects in a region

* The complexity of behavior of objects

How many objects can have scripts? What can scripts control about
objects? How difficult is it to define new behaviors?

User:Finrod Meriman/AWG Feedback

2007-11-11T04:54:50Z

Finrod Meriman:

== Some initial reactions to the architecture... ==

'''Disclaimer''': For those who don't know, I work for Intel. Although the comments are generally personal, I am participating in the working group as Intel's representative. I don't speak formally for Intel, but my affiliation certainly influences my position and interests. [[User:Finrod Meriman/]]

=== 1) Architecting the grid is not a once-and-done task. ===

In the meeting we discussed two primary motivations. The first is fairly
short-term: make it possible to scale performance of the second life
grid. The second is more long-term: lay an architectural foundation on
which virtual worlds (in general) could scale to millions of users
supporting a very diverse set of interactions (everything from WoW to
SL). Both motivations are very reasonable. But there is an obvious
tension between the two. In the short-term it is necessary to consider
the transition from the existing SL Grid though long-term scalability
might be best served by a more complete overhaul. It seems pretty
obvious that re-architecting the grid needs to be viewed as a multi-step
process not a once-and-done event. There will be lots of good ideas to
support a fully general, scalable grid that need to be put off for later
consideration (or we'll never make progress).

=== 2) Make our architectural "views" explicit. ===

There is tremendous value in having multiple "views" of an architecture
(for those who care, you can look for IEEE 1471 in wikipedia to get an
idea how to define views formally). In the meeting I heard three
different views presented. The "communication view" described the
architecture in terms Certified HTTP using RESTful interactions and
LLSD. Most of the detail that was presented focused on the "interface
view" or "component view". Each of the agent & region domains were
broken into services (with standardized interfaces) based on stateful &
stateless interactions. And we discussed a "transactional view" where we
hinted at how to sequence the invocation of various services to
accomplish a higher level task (eg the login example cascaded through
several different invocations... all of which were mandatory to set up
the correct state).

One view that was missing from the discussion was the "data" or
"document" view. What I mean is that we didn't talk about what the data
looked like, where it moved around the system, and how the various
functions transformed the data. I'm sure some of this will be derived
from the current grid architecture, but it still should be thought
through & documented. And of course there is the question (for the
long-term discussion) as to whether there are other standard 3D object
formats that should be adopted to encourage long term stability
(e.g. collada, obj, ...).

Just a couple more comments on the communication piece... Putting aside
my biases about REST... One question I wanted to ask is how the
architecture supports caching & replication. How hard would it be to
take the appropriate pieces and distribute them through something like
Akamai (or some other distributed web cache/content distribution
system)? Should the architecture explicitly include a separate CDN to
improve performance? This question is, in some sense, a question of both
the communication and document views. And it might impact how the
services are decomposed.

=== 3) We're missing a good functional decomposition. ===

Most of the architecture drawings that were presented focused on the
separation into region & agent domains. Clearly this is an important
separation for enabling multiple administrative domains. However, I
think the first structural partitioning should be along functional lines
(yes, the agent & region domains already capture an implicit functional
breakdown). Just to be concrete... I've been asked the question, by
several residents, why IM is so tightly bound to SL. There are many good
reasons to bind them (managability for one), but there are also good
reasons to separate them. In the case of IM, there are already a number
of very good and extensible IM systems such as XMPP (jabber & google
talk).

A more thorough examination of the functions might help to figure out
which services belong in which boxes.

=== 4) Managing trust is going to be really hard. ===

It appears that transactions/patterns (such as login) involve services
in multiple domain. In the case of login, for example, the agent that is
created on behalf of a particular user executes services in a region
domain where the avatar is located. The agent domain proxies trust for
the user potentially across administrative domains. Proxying trust also
creates potential problems for tracking/controling where bits are
sent. This has impact in managing intellectual property. (Not that the
architecture needs to support DRM, but it shouldn't make it impossible
to provide some limits on distribution of IP.)

User:Finrod Meriman/AWG Feedback

2007-11-11T04:54:10Z

Finrod Meriman:

== Some initial reactions to the architecture... ==

'''Disclaimer''': For those who don't know, I work for Intel. Although the comments are generally personal, I am participating in the working group as Intel's representative. I don't speak formally for Intel, but my affiliation certainly influences my position and interests. [[User:Finrod Meriman]]

=== 1) Architecting the grid is not a once-and-done task. ===

In the meeting we discussed two primary motivations. The first is fairly
short-term: make it possible to scale performance of the second life
grid. The second is more long-term: lay an architectural foundation on
which virtual worlds (in general) could scale to millions of users
supporting a very diverse set of interactions (everything from WoW to
SL). Both motivations are very reasonable. But there is an obvious
tension between the two. In the short-term it is necessary to consider
the transition from the existing SL Grid though long-term scalability
might be best served by a more complete overhaul. It seems pretty
obvious that re-architecting the grid needs to be viewed as a multi-step
process not a once-and-done event. There will be lots of good ideas to
support a fully general, scalable grid that need to be put off for later
consideration (or we'll never make progress).

=== 2) Make our architectural "views" explicit. ===

There is tremendous value in having multiple "views" of an architecture
(for those who care, you can look for IEEE 1471 in wikipedia to get an
idea how to define views formally). In the meeting I heard three
different views presented. The "communication view" described the
architecture in terms Certified HTTP using RESTful interactions and
LLSD. Most of the detail that was presented focused on the "interface
view" or "component view". Each of the agent & region domains were
broken into services (with standardized interfaces) based on stateful &
stateless interactions. And we discussed a "transactional view" where we
hinted at how to sequence the invocation of various services to
accomplish a higher level task (eg the login example cascaded through
several different invocations... all of which were mandatory to set up
the correct state).

One view that was missing from the discussion was the "data" or
"document" view. What I mean is that we didn't talk about what the data
looked like, where it moved around the system, and how the various
functions transformed the data. I'm sure some of this will be derived
from the current grid architecture, but it still should be thought
through & documented. And of course there is the question (for the
long-term discussion) as to whether there are other standard 3D object
formats that should be adopted to encourage long term stability
(e.g. collada, obj, ...).

Just a couple more comments on the communication piece... Putting aside
my biases about REST... One question I wanted to ask is how the
architecture supports caching & replication. How hard would it be to
take the appropriate pieces and distribute them through something like
Akamai (or some other distributed web cache/content distribution
system)? Should the architecture explicitly include a separate CDN to
improve performance? This question is, in some sense, a question of both
the communication and document views. And it might impact how the
services are decomposed.

=== 3) We're missing a good functional decomposition. ===

Most of the architecture drawings that were presented focused on the
separation into region & agent domains. Clearly this is an important
separation for enabling multiple administrative domains. However, I
think the first structural partitioning should be along functional lines
(yes, the agent & region domains already capture an implicit functional
breakdown). Just to be concrete... I've been asked the question, by
several residents, why IM is so tightly bound to SL. There are many good
reasons to bind them (managability for one), but there are also good
reasons to separate them. In the case of IM, there are already a number
of very good and extensible IM systems such as XMPP (jabber & google
talk).

A more thorough examination of the functions might help to figure out
which services belong in which boxes.

=== 4) Managing trust is going to be really hard. ===

It appears that transactions/patterns (such as login) involve services
in multiple domain. In the case of login, for example, the agent that is
created on behalf of a particular user executes services in a region
domain where the avatar is located. The agent domain proxies trust for
the user potentially across administrative domains. Proxying trust also
creates potential problems for tracking/controling where bits are
sent. This has impact in managing intellectual property. (Not that the
architecture needs to support DRM, but it shouldn't make it impossible
to provide some limits on distribution of IP.)

User:Finrod Meriman

2007-11-11T04:52:53Z

Finrod Meriman:

User:Finrod Meriman

2007-11-11T04:51:15Z

Finrod Meriman: New page: For those who don't know, I work for Intel. Although the comments are generally personal, I am participating in the architecture working group as Intel's representative. I don't speak for...

User:Finrod Meriman/AWG Feedback

2007-09-25T22:38:43Z

Finrod Meriman:

Architecture Working Group

2007-09-25T22:33:58Z

Finrod Meriman: /* Individual Reviews and Feedback */

[[Image:slarch.jpg]]

==Goal==

Discuss and implement the architecture of the future Second Life Grid.

==Materials==

[[Project Motivation]]

[[Proposed Architecture|Architecture proposed by Linden Lab]]

[[Components and Roles]]

[[Use Cases]]

[[Brainstorming]]

[[AWG: Zha's Desiderata for evaluating the proposed design]]

==Meetings==

[[ArchWG_Mtg_1_Agenda|September 13th 2007, Meeting 1]]

[[Workitems for Meeting 1]]

[[ Zha's comments on meeting 1]]

[[ Tree's comments on meeting 1]]

[http://taotakashi.wordpress.com/2007/09/24/second-life-grid-architecture-meetup-transcript/ Transcript and Slides from Tao Takashi's informal meetup on 2007/09/23]

==Individual Reviews and Feedback==

[[Diva Canto's Review]]

[[Mic's Feedback]]

==Discussions==

[[DRM, IP and permissions]] (from the mailing list)

==IRC Chat Logs==

[[Chatlog from 2007/09/16]] (Gigs, otakup0pe and Tao_T talk about possible forms of regions etc.)

<noinclude>
[[Category:Architecture Working Group]]</noinclude>

User:Finrod Meriman/AWG Feedback

2007-09-25T22:33:34Z

Finrod Meriman: New page: Just some initial reactions to the architecture... 1) Architecting the Grid is not a once-and-done task. In the meeting we discussed two primary motivations. The first is fairly short-...

Just some initial reactions to the architecture...

1) Architecting the Grid is not a once-and-done task.

In the meeting we discussed two primary motivations. The first is fairly
short-term: make it possible to scale performance of the second life
grid. The second is more long-term: lay an architectural foundation on
which virtual worlds (in general) could scale to millions of users
supporting a very diverse set of interactions (everything from WoW to
SL). Both motivations are very reasonable. But there is an obvious
tension between the two. In the short-term it is necessary to consider
the transition from the existing SL Grid though long-term scalability
might be best served by a more complete overhaul. It seems pretty
obvious that re-architecting the grid needs to be viewed as a multi-step
process not a once-and-done event. There will be lots of good ideas to
support a fully general, scalable grid that need to be put off for later
consideration (or we'll never make progress).

2) Make our architectural "views" explicit.

There is tremendous value in having multiple "views" of an architecture
(for those who care, you can look for IEEE 1471 in wikipedia to get an
idea how to define views formally). In the meeting I heard three
different views presented. The "communication view" described the
architecture in terms Certified HTTP using RESTful interactions and
LLSD. Most of the detail that was presented focused on the "interface
view" or "component view". Each of the agent & region domains were
broken into services (with standardized interfaces) based on stateful &
stateless interactions. And we discussed a "transactional view" where we
hinted at how to sequence the invocation of various services to
accomplish a higher level task (eg the login example cascaded through
several different invocations... all of which were mandatory to set up
the correct state).

One view that was missing from the discussion was the "data" or
"document" view. What I mean is that we didn't talk about what the data
looked like, where it moved around the system, and how the various
functions transformed the data. I'm sure some of this will be derived
from the current grid architecture, but it still should be thought
through & documented. And of course there is the question (for the
long-term discussion) as to whether there are other standard 3D object
formats that should be adopted to encourage long term stability
(e.g. collada, obj, ...).

Just a couple more comments on the communication piece... Putting aside
my biases about REST... One question I wanted to ask is how the
architecture supports caching & replication. How hard would it be to
take the appropriate pieces and distribute them through something like
Akamai (or some other distributed web cache/content distribution
system)? Should the architecture explicitly include a separate CDN to
improve performance? This question is, in some sense, a question of both
the communication and document views. And it might impact how the
services are decomposed.

3) We're missing a good functional decomposition.

Most of the architecture drawings that were presented focused on the
separation into region & agent domains. Clearly this is an important
separation for enabling multiple administrative domains. However, I
think the first structural partitioning should be along functional lines
(yes, the agent & region domains already capture an implicit functional
breakdown). Just to be concrete... I've been asked the question, by
several residents, why IM is so tightly bound to SL. There are many good
reasons to bind them (managability for one), but there are also good
reasons to separate them. In the case of IM, there are already a number
of very good and extensible IM systems such as XMPP (jabber & google
talk).

A more thorough examination of the functions might help to figure out
which services belong in which boxes.

4) Managing trust is going to be really hard.

It appears that transactions/patterns (such as login) involve services
in multiple domain. In the case of login, for example, the agent that is
created on behalf of a particular user executes services in a region
domain where the avatar is located. The agent domain proxies trust for
the user potentially across administrative domains. Proxying trust also
creates potential problems for tracking/controling where bits are
sent. This has impact in managing intellectual property. (Not that the
architecture needs to support DRM, but it shouldn't make it impossible
to provide some limits on distribution of IP.)

Architecture Working Group

2007-09-25T22:32:52Z

Finrod Meriman: /* Individual Reviews and Feedback */

[[Image:slarch.jpg]]

==Goal==

Discuss and implement the architecture of the future Second Life Grid.

==Materials==

[[Project Motivation]]

[[Proposed Architecture|Architecture proposed by Linden Lab]]

[[Components and Roles]]

[[Use Cases]]

[[Brainstorming]]

[[AWG: Zha's Desiderata for evaluating the proposed design]]

==Meetings==

[[ArchWG_Mtg_1_Agenda|September 13th 2007, Meeting 1]]

[[Workitems for Meeting 1]]

[[ Zha's comments on meeting 1]]

[[ Tree's comments on meeting 1]]

[http://taotakashi.wordpress.com/2007/09/24/second-life-grid-architecture-meetup-transcript/ Transcript and Slides from Tao Takashi's informal meetup on 2007/09/23]

==Individual Reviews and Feedback==

[[Diva Canto's Review]]
[[Mic's Feedback]]

==Discussions==

[[DRM, IP and permissions]] (from the mailing list)

==IRC Chat Logs==

[[Chatlog from 2007/09/16]] (Gigs, otakup0pe and Tao_T talk about possible forms of regions etc.)

<noinclude>
[[Category:Architecture Working Group]]</noinclude>