Autobuild/How To

Autobuild is an in-house framework for maintaining and building libraries. (Warning: Our tool is distinct from GNU Autobuild but they are similar enough to cause confusion.) It acts as director providing a common interface to build and package libraries, but it is not a build system like make or cmake. You will still need platform specific make, cmake, or project files to actually configure and build your library. Autobuild will, however, allow you invoke these commands and package the product with a common interface. Linden Lab Autobuild is designed as a replacement for the old lindelib policies, doing the right thing so you don't have to.

Getting Autobuild

Auotobuild is available as Mercurial repository which you may check out from http://hg.lindenlab.com/brad/autobuild-trunk. You can either run it directly from the checkout or install it as a normal python package using 'setup.py install'.

Using autobuild to build a library

In this section we discuss how to configure autobuild to build a source package. We will assume that you have a source distribution with a build system like make or cmake up and working.

Before you start

Before you start it will be helpful if you familiarize yourself with the autobuild class model. This page describes the structure of an autobuild configuration file explaining the function of the various elements it contains. The commands that follow in this document will follow the structure of the class model, so they may be easier to understand if you refer to it while going through this page.

Basic build configuration

The following steps will help you configure autobuild to run your package's configure and build commands by invoking nothing more than

 autobuild build

To start you need to create an autobuild.xml configuration file. Change directories to the root of your package (or wherever you would like the configuration file to live) and invoke

 autobuild edit package

This will start an interactive session in which you may fill in the general package details. Minimally you should provide a name and a version. If you intend to package your build into an archive, you will also want to specify a license and license file. We will ignore the other fields for now. As an alternative to the interactive session, you may add or change package fields all from the command line by passing field=value arguments on the command line. For all edit commands, the interactive mode will be used when no attribute arguments are passed and the non-interactive mode otherwise. For example to create our configuration file non-interactively you can invoke

 autobuild edit package name=test license=MIT license_file=LICENSES/test.txt version=1.0

to create the test package version 1.0 using an MIT license. Now that we have defined some of the basic attributes for the package we need to configure the platforms on which we intend for the library to be built. At this level we can really only configure the build directory because we may define multiple build configurations (see below). The build directory is the directory in which autobuild will run the build and configure commands and is presumed to be where any build product will be generated. You can, for example, configure the MacOS X platform with

 autobuild edit platform name=darwin build_directory=build

Note that for relative paths, the root is always the location of the autobuild configuration file. Now you need to add a build configuration which includes the build and configure commands which autobuild will invoke to make the package. Assuming you are dealing with a UNIX style package, you typically just call configure and make from the root directory to make the package. You can define the Release configuration to do just that using

 autobuild edit configure platform=darwin name=Release command=../configure
 autobuild edit build platform=darwin name=Release command=make options='--directory=..' default=True

Note that these commands are run in the build directory so you need to set the path or add an option as appropriate to work select the root directory. The configure and build commands may take three attributes: command, arguments, and options. These are concatenated to produce the actual command invoked in the shell. The build configuration also supports the default option which flags a build configuration to be build by default if no configuration is specified when the autobuild build command is invoked. You may repeat these steps for other build configurations like Debug with different make and configure options to, for example, build a debug version of the library. You now have enough autobuild configuration to build a simple package for the Darwin platform. Simply run

 autobuild build

to configure and make the package. To support other platform you need only repeat the package, config, and build configuration steps for the platform.

In some cases you may find that your different platforms share a common command, arguments, or options. For example, a library configured by cmake may use the same configuration command on all platforms because it is a cross-platform tool. In that case you can define the special platform common to define the commonalities. Attributes from build configurations in the common platform are inherited by those for the working platform. If a command is defined for a platform for a build configuration like Release for both a working platform like darwin that platform will inherit from common. This means that if the arguments or command is not specified in the working platform build configuration for either the build or configure command then value from the common will be used. Conversely, if the arguments or command attribute is defined in the working platform, it will supersede what is specified in common. However, the options for the commands from common will be concatenated with those specified in the working platform configuration with the common options preceding the working. Specifying a common platform can allow you to specify commands once per build configuration enabling a single point of truth for cross platform tools.

Adding dependencies

If your package depends on other archives, you can instuct autobuild to download and install those archives. This feature is useful for binary archives bundled as compressed tar files dowloadable from a url (typically generated by autobuild). First you will need to provide information in the autobuild configuration to identify and locate any package dependencies. You can add a package description of the installable archive using the installables command like the following

  autobuild installables add GL license=GPL license_file=LICENSES/GL.txt platform=darwin 
 url=http://s3.amazonaws.com/viewer-source-downloads/install_pkgs/GL-darwin-20101004.tar.bz2 hash=0b7c1d43dc2b39301fef6c05948fb826

This creates a new entry in the installables dictionary that includes the information needed to download and verify the GL package for the darwin platform. You can add downloads for other platforms using the installables edit command

 autobuild installables edit GL platform=windows 
 url=http://s3.amazonaws.com/viewer-source-downloads/install_pkgs/GL-windows-20101001a.tar.bz2 hash=a94538d064cd0a235b2a95389e7e8ee8

Once an package has been added to the installables configuration for the platform you are working on, you can download it using the autobuild install command

 autobuild install GL

The package contents will be downloaded and extracted into the packages directory inside the build directory configured for the working platform. Autobuild will conveniently track which packages are installed and what files were extracted only installing or updating files as needed. A package may be uninstalled with

autobuild uninstall GL

if it is no longer needed. If you update a package by changing its configuration in the installables configuration, autobuild will automatically clean up the older version so typically you will not need to uninstall packages.

Using autobuild to construct an archive

Constructing the manifest and packaging

Autobuild has the ability to construct a compressed tar archives with the autobuild package command, but to do that you must tell autobuild what to include in the archive. Typically you will be packaging an archive of build product generated by the autobuild build command, but that is not a requirement. Minimally you will need an autobuild configuration with the name, license, license_file, and version attributes set and a platform configuration for the platform for which you wish to build and archive. You may then use the autobuild manifest command to add files which should be included in the archive. For example

autobuild manifest add include/*.h

will add the pattern 'include/*.h' to the manifest for the platform you are currently running (use the -p option to select another platform). Autoubuild supports simple glob patterns, so this pattern instructs autobuild to add all .h header files in the include directory to the archive. The root directory for all paths is assumed to be the build directory configured for the platform. You are strongly encouraged to use only relative paths in your manifest. Additional files may be added by calling manifest add again as needed (note that the add instruction can accept more than one pattern argument). Note that license file will automatically be included in the archive even if it is not explicitly listed in the manifest. If a file is to be included in every archive regardless of platform, you may add it to the manifest of the common platform.

Once you have added all the files required by the archive listed in the manifest, you can generate the archive using

 autobuild package

This command will generate an archive in the directory where the autobuild configuration file resides. Unless set using the --archive-name option, the archive will be named using the convention <name>-<version>-<platform>-<date>.tar.bz2, e.g. test-1.0-darwin-20101020.tar.bz2 for the package described in this tutorial.

Laying out build product

Autobuild places no constraints on the layout of build product bundled into an archive, but there are some recommended conventions. The following table shows the recommended directory under which to place files of different types.

File type	Directory
license	LICENSES
header	include
libraries	lib
executables	bin
scripts	scripts

For more details, see Autobuild/Package_Layout. By following these conventions you will help package users easily configure their build systems to find your files. For types not listed here it is recommended that you install files following well known conventions.