User:Rob Linden/Build Tool Improvement Spec - 2008

From Second Life Wiki
Revision as of 14:57, 2 September 2008 by Rob Linden (talk | contribs) (more cleanup)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Stub article created during Open Source Meeting on 2008-07-31. The idea is to simplify the current CMake-centric instructions by making the process itself much easier.

Please add related items to VWR-8545


Dependency checking and helpful errors

Running ./develop.py should print out as helpful of an error as possible if the necessary components are not available. To the greatest extent possible, as many missing dependencies should be listed in a single pass rather than forcing people to rerun to find the next missing dependency. Here are the dependencies in stages:

  • Stage 1
    • Python
      • Obviously, there's no way to print a friendly error if python is missing
  • Stage 2
    • cmake
      Since CMake may be used to check for all of the other dependencies, it's ok if further dependency checking stops here.
  • Stage 3
    • All platforms
      • svn (to get latest release branch)
      • openssl SDK
      • FMOD
      • LLKDU/OpenJPEG
    • Windows
      • Some version of MSVC or cygwin or mingw32
      • bison and flex (from cygwin)
      • DirectX SDK (Microsoft DirectX SDK (November 2007))
      • Quicktime SDK
    • Linux (Debian package names, other platforms may differ)
      Some header/development packages are also required
      • build-essential (gcc/ld/make etc)
      • libgl1-mesa-dev
      • libzip-dev
      • libfreetype6-dev
      • libxft2-dev
      • libxrandr-dev
      • libxcursor-dev
      • libglu1-mesa-dev
      • libxt-dev
      • unzip

In all cases, there should be URLs provided for each missing option. For maintainability, these will probably be links to wiki.secondlife.com rather than direct links to the download site.

On Linux distros, it may be more helpful to just work it like a missing dependency, providing the command line and perhaps even offering to run it.

Error URLs and helpful error text

Currently, there's not much in the way of exception handling in develop.py. Most exceptions are for missing dependencies and misconfiguration problems. A generic exception handler should be implemented, with a --debug option (or similar) implemented to override with full information, and an help URL for all exceptions should be provided. There may be helpful nuggets in the exceptions that can be used to construct a wiki URL that won't exist when the error is first encountered, but could be quickly populated by people who actually encounter the error. The discussion page associated with the wiki URL could be used as a mini-forum for that problem.

Problems with PATH variable

Error text should suggest when PATH problems are a problem. If there's any easy way to actually diagnose and fix/suggest fix for this, it should be pursued.

Things to make sure are in the path:

  • CMake
  • bison
  • flex

Rumor has it that on Windows, the CMake and/or Python installer can silently fail when "install for all users" is selected. This should be investigated, bug filed, and even better, patch submitted.


Version detection

The build process is finicky about versions. Ensure explicit version checking for:

  • cmake version
  • python version

Warnings rather than errors should be printed for versions newer than tested, but those warnings should be reiterated should the build fail.

Automatic artwork and lib fetching

Currently, artwork and lib fetching is still partially manual.

Here's a shell snippet that works on Linux. This should be integrated into develop.py and done in a cross platform fashion.

chmod +x release/doc/asset_urls.txt
. release/doc/asset_urls.txt
wget $SLASSET_ART
wget $SLASSET_LIBS_WIN32
unzip slviewer-artwork*
unzip slviewer-*-libs*
cp linden/* release/ -r


On some branches, win32 asset_urls.txt are in linden/doc

For mac and linux you should use $SLASSET_LIBS_LINUXI386 or $SLASSET_LIBS_DARWIN and for linux its not unzip for the libs its tar :-

tar -xvzf slviewer-*-libs*

In fact the differences between the different libs packages now are trivial as it only contains some fonts and other trivial bits. But failure to download it (or the wrong one) can cause packaging later to fail.


More progress information

Currently, develop.py silently toils in silence, leaving the developer to wonder if there's actually something happening. At a minimum, steps that often take a while (fetching tarballs) should have textual description indicating to the developer that "this may take a while...", and timeout prompts should be implemented for steps that often stall (e.g. on a download that should take no longer than 5 minutes, echo every 10 minutes "Attempting to download XXX has taken YY minutes. Continue (y/n)?". There's no point in actually stopping activity, but giving a strong indication that aborting is probably the best idea for the developer would be very helpful.


message.xml checksumming

If you have many versions of the viewer on your machine, on Win32 it can be possible for the wrong messages.xml to get used sometimes if it isn't explicitly copied. Second Life will tend to run, but using the wrong one can cause subtle hard to trace errors. Proposed fix: cp linden/etc/messages.xml over to linden/indra/build-VC80/newview/app-settings.

Library download speed optimizations

An investigation into why some developers get very slow downloads needs to occur.

Remove VSTool requirement (make express editions of Visual C++ work) ==

Express editions do not have some of the extra features that VSTool.exe requires, VSTool is a little application that sets a few default project options, like which project is the startup and which build configuration to use.

The develop.py option --unattended currently supports this. It would be nice to have develop.py detect Visual Studio Express editions automatically, via registry lookup.


Make LLKDU optional in a rational way

This is not present in the automated libs download but is required at least on windows by the copy_win_scrips script. The file is not actually needed for the viewer to run (if present it will be used for jpeg2000 decoding in preference to Openjpeg. The current solution is to remove the lines from newview/copy_win_scripts that reference llkdu and this should then allow this custom build step to run. This should be part of the dependency check, and if missing, OpenJPEG should be assumed (with perhaps a well-placed warning).

Fix libGL.a problem on Linux

ligGL.a is supplied in the linden library package. There needs to be a determination whether this should be. To avoid linker errors on some Linux distros, remove indra/libraries/i686-linux/release/libGL.a and indra/libraries/i686-linux/debug/libGL.a