Difference between revisions of "Viewer Manifest"

From Second Life Wiki
Jump to: navigation, search
 
(Construct)
Line 21: Line 21:
 
=== Construct===
 
=== Construct===
  
The construct method is the 'manifest' part of the script.  At its conclusion, every file included in the installer should have been enumerated.
+
The construct method is the 'manifest' part of the script.  At its conclusion, every file included in the installer should have been enumerated, and any modifications to those files (e.g. stripping binaries) must have been made.
  
 
The construct method assumes that you are primarily copying files from an arbitrary number of locations into a single destination directory, with most files having roughly the same names and hierarchy in the destination, but with some changes.  To make this task easier, it keeps around a pair of prefixes, one for the source, and one for the destination.  These prefixes are prepended to each path specified, so that you don't have to retype them over and over again.  
 
The construct method assumes that you are primarily copying files from an arbitrary number of locations into a single destination directory, with most files having roughly the same names and hierarchy in the destination, but with some changes.  To make this task easier, it keeps around a pair of prefixes, one for the source, and one for the destination.  These prefixes are prepended to each path specified, so that you don't have to retype them over and over again.  

Revision as of 17:11, 27 February 2007

How to add a file

  • Find the appropriate manifest class for your file. If the file will be common across all platforms, use the CommonManifest.
  • Find the construct method (it starts with the line def construct(self))
  • Add a self.path call at the appropriate place. If your new file is alongside existing files, use their entries in the manifest as a template.

Purpose

The installer manifest (viewer_manifest.py) is a file explicitly enumerating each file that is distributed as part of the installer for each platform. Consolidating knowledge about what files we deliver in the installer will tend to reduce mistakes, like new files not being added to the Windows uninstall section or the Mac updater accidentally deleting critical files.

Using the manifest in conjunction with a compiled build, it is possible to create a directory that contains exactly the same files as the distributed installer would create on the destination computer (modulo runtime files like xpti.dat and the uninstaller on Windows). Since the manifest is the single source of truth for all files that are to be included in the installer, it is also used for packaging up the installers for each platform.

The manifest assumes that all installed files are contained within a single directory on the recipient machine. User-modifiable files (such as preferences) are generated at runtime and are not stored under the directory hierarchy of the application.

Format

The manifest is a Python script that has a very simple declarative mini-DSL. This way, hand-editing of the manifest is supported, and it retains all the power of a scripting language.

Unfortunately the 'manifesty' parts (the construct methods) are intermingled with the packaging part (the package_finish methods). This can be changed as long as the methods are kept strictly separate.

Construct

The construct method is the 'manifest' part of the script. At its conclusion, every file included in the installer should have been enumerated, and any modifications to those files (e.g. stripping binaries) must have been made.

The construct method assumes that you are primarily copying files from an arbitrary number of locations into a single destination directory, with most files having roughly the same names and hierarchy in the destination, but with some changes. To make this task easier, it keeps around a pair of prefixes, one for the source, and one for the destination. These prefixes are prepended to each path specified, so that you don't have to retype them over and over again.

The base source and destination directories are implicit; the default source directory is the directory containing the manifest itself (i.e. newview), and the default destination directory is "newview/packaged".

Code talks louder than words, so here's an example manifest:

class DemoManifest(LLManifest):                     # You have to have LLManifest as an ancestor
    def construct(self):
        super(DemoManifest, self).construct()       # Call superclass, that's where common files are specified.  The 
                                                    # stupid syntax here is entirely Python's fault.
        if self.prefix("dir_1"):                    # Prefixes save you typing -- they simply add their text to any paths
                                                    # prior to the end_prefix() call.  Wrap them in an if so you 
                                                    # get nice indentation.
          self.path("test_a")                       # Copy file "newview/dir_1/test_a" to "dst/dir_1/test_a"
          self.path(src="test_b", dst="test_dst_b") # Copy file "newview/dir_a/test_b" to "dst/dir_1/test_dst_b"
          self.path("*.test")                       # Use wildcards to get an entire group of files.  Only single * works.
          self.path("*.tex", "*.jpg")               # You can rename files en masse if the number of *'s match 
                                                    # in the source and destination paths.
          if self.prefix("nested", dst=""):         # Here's something tricky: use prefixes to bring files from a 
                                                    # subdirectory up into the parent.
            self.path("deep")                       # Copy "newview/dir_1/nested/deep" to "dst/dir_1/deep"
            self.end_prefix()                       # End prefixes this way
          self.end_prefix("dir_1")                  # end_prefix() takes an optional argument to make the nesting 
                                                    # clearer.  It raises an exception if the argument doesn't match the
                                                    # prefixes that it was using, since that would indicate a nesting error.
        self.path("outside_of_prefix")              # Don't have to have a prefix surrounding a path.

One of the philosophies of this format is that it is include-only. It's a lot easier to notice that you've made a mistake if you forget to include a file with the installer, since the generated application won't work right, than it is to notice that you've accidentally included extra files.

Platform

One of the magical things about the LLManifest is that it automatically determines the correct manifest for the current platform based on its class name. E.g. derive from LLManifest and name your class "HP-UXManifest", and that will be selected for use on HP-UX.

Because each platform is a class, we can inherit common behavior. E.g. platform-nonspecific files are listed in CommonManifest, and the common Linux files and packaging are in LinuxManifest, while architecture-specific stuff is in Linux_i686Manifest and Linux_x86_64.

Actions

The manifest builds up a list of files, then performs a set of actions on them. Each action corresponds to two methods:

action_action (src, dst)
Called for every file that is enumerated in the manifest, with the source path, and destination path.
action_finish ()
Called at the conclusion of each run, after each file has had actions called upon it.

Subclasses of LLManifest override the methods for the actions that they need custom behavior for. In practice, the copy_action is standardized, and the package_finish method is overridden for each platform.

The sequence of steps is:

  • Locate the proper class for the platform/arch combo.
  • Call 'construct' on an instance of the class
    • Each call to 'path' calls each action_action method in order for each file (including files specified by globs or via recursive directory copy).
  • Call action_finish in order.

Command Line

The command line looks like:

<script> [<options>] [destination_dir]

The source directory is implicitly set to the directory containing the script itself, and the destination directory is defaulted to 'packaged'.

Here are the options:

Option Default
--actions= "copy package" This argument specifies the actions that are to be taken when the script is run. Pass in --actions="copy" to skip the packaging step and just copy the manifest files to a destination directory.
--arch= "" This argument is appended to the platform string for determining which manifest class to run. E.g. --arch=i686 on Linux would try to use Linux_i686Manifest.
--configuration= "Universal" The build configuration used. Only used on OS X for now, but it could be used for other platforms as well.
--grid= "" (agni) Which grid the client will try to connect to. Even though it's not strictly a grid, 'firstlook' is an acceptable value for this parameter.
--installer_name= Platform-specific The name of the file that the installer should be packaged up into. Only used on Linux at the moment.
--login_url= Derived from grid argument The url that the login screen displays in the client.
--platform= Parsed from python runtime The current platform, to be used for looking up which manifest class to run.
--version= Parsed from llversion.h This specifies the version of Second Life that is being packaged up.

Goals

The manifest file represents the following information:

  • Files that are included in the installer
  • Files and directories whose installed names are different from the names in the build tree
  • Files which are generated by Second Life and need to be uninstalled Pretty much exclusive to Windows
  • Files which are not supposed to be overwritten during an upgrade, because they are user-editable I spoke to Richard, and he agrees that no such files should exist in the application directory. Any that we have now will be moved to the user profile directory. Moreover, none of the existing installers or updaters respect user-modifiable files in the application directory, so we don't even have to worry about backwards compatibility.