Difference between revisions of "Incremental Installer Design"

From Second Life Wiki
Jump to navigation Jump to search
Line 52: Line 52:
The installer has two components:
The installer has two components:
; The indexer: Takes a model runtime tree produced by our build system, and generate the appropriate index file and upload the required opaque objects to S3.
; The indexer: Takes a model runtime tree produced by our build system, and generate the appropriate index file and upload the required opaque objects to S3.
* In the patch extension, it will use the index file of a previous version to generate the patch files for those opaque objects which changed in the new version.
:* In the patch extension, it will use the index file of a previous version to generate the patch files for those opaque objects which changed in the new version.
; The updater: Polls for new index files applicable to the selected vendors, products, channel and the architecture of the platform running the installer. Depending on metadata provided in the index file and user preferences, one of the following actions will be performed:
; The updater: Polls for new index files applicable to the selected vendors, products, channel and the architecture of the platform running the installer. Depending on metadata provided in the index file and user preferences, one of the following actions will be performed:
* The index file is ignored
:* The index file is ignored
* The index file is processed and the required opaque object files are downloaded or constructed and made available as an optional upgrade on start of the next session
:* The index file is processed and the required opaque object files are downloaded or constructed and made available as an optional upgrade on start of the next session
* as above, but no choice is given
:* as above, but no choice is given
* as above, but no choice is given and currently running processes are terminated.
:* as above, but no choice is given and currently running processes are terminated.
;: The new runtime is constructed at startup time of the process by copying the opaque objects to their intended location accorting to the index file.
: The new runtime is constructed at startup time of the process by copying the opaque objects to their intended location accorting to the index file.
* In the Link extension, the opaque files are moved to their intended location and replaced by a text file with the same name as the opaque file, but with a .link extension appended. If an existing .link file exists, and the new runtime location differes from the location listed in the .link file, the new location is appended to the .link file.
:* In the Link extension, the opaque files are moved to their intended location and replaced by a text file with the same name as the opaque file, but with a .link extension appended. If an existing .link file exists, and the new runtime location differes from the location listed in the .link file, the new location is appended to the .link file.


== Indexer Details ==
== Indexer Details ==

Revision as of 17:57, 10 June 2008

Linden Lab Improved Installer/Updater

The goal of the installer is to maintain, upgrade or downgrade a copy of a specific runtime tree on local disk using the smallest number of downloads and least amount of local storage.

Runtime trees are hierarchically classified by:

 vendor/
   |_product/
      |_channel/

Only one active runtime tree per channel may exist. The creation of a new runtime should use the least amount of downloads and copy operations as possible.

Runtime trees are also implicitly classified by architecture. A seperate runtime tree exists for every supported architecture.

The benefits of this new architecture are:

  • Faster updates
  • Background updates
  • Simplified re-branding, all version information can be removed from the actual files and specified in the index files instead.
  • Architecture independent process

Basic Idea

To store all files as opaque objects in a flat directory tree called the "staging area". Released runtime trees are represented by index files which map the opaque file to a relative runtime tree locations.

The staging area is just a flat collection of files, named according to they hexified SHA1 signature. The authoritative staging area lives on S3, with all files ever published. The client installer keeps a subset of the S3 staging area on local disk, with all files ever downloaded.

Patch Extension

Add patch files to the staging area which can be used to create one opaque object from an existing opaque object. Patch files are named <src-SHA1-sig>.<dst-SHA1-sig>. The index files will list all availabel patch files at the time the index file was created. The installer will then attempt to construct a requested opaque object using the available patch files.

Link Extension

Add link files which refer to the locations in the runtime trees where the original opaque object was copied. This reduces storage requirements for the staging area.


Components

The installer has two components:

The indexer
Takes a model runtime tree produced by our build system, and generate the appropriate index file and upload the required opaque objects to S3.
  • In the patch extension, it will use the index file of a previous version to generate the patch files for those opaque objects which changed in the new version.
The updater
Polls for new index files applicable to the selected vendors, products, channel and the architecture of the platform running the installer. Depending on metadata provided in the index file and user preferences, one of the following actions will be performed:
  • The index file is ignored
  • The index file is processed and the required opaque object files are downloaded or constructed and made available as an optional upgrade on start of the next session
  • as above, but no choice is given
  • as above, but no choice is given and currently running processes are terminated.
The new runtime is constructed at startup time of the process by copying the opaque objects to their intended location accorting to the index file.
  • In the Link extension, the opaque files are moved to their intended location and replaced by a text file with the same name as the opaque file, but with a .link extension appended. If an existing .link file exists, and the new runtime location differes from the location listed in the .link file, the new location is appended to the .link file.

Indexer Details

The indexer is run as part of the post build / packaging process. It will take as input:

  • The vendor, product and channel name of the runtime
  • A version id
  • A path to the runtime tree
  • In the patch extension, a path to a previous runtime tree for which the patch should be generated.

The indexer will produce:

  • An index file mapping SHA1 hash signatures to relative runtime tree locations
  • A set of opaque files to upload to S3.
  • In the patch extension, a patch file for those opaque files which were in the same location in the previous runtime.
  • The appropriate version manager entries.

Updater Details