User:Rob Linden/Build Tool Improvement Spec - 2008

Stub article created during Open Source Meeting on 2008-07-31. The idea is to simplify the current CMake-centric instructions by making the process itself much easier.

Please add related items to VWR-8545

Dependency checking and helpful errors

Running ./develop.py should print out as helpful of an error as possible if the necessary components are not available. To the greatest extent possible, as many missing dependencies should be listed in a single pass rather than forcing people to rerun to find the next missing dependency. Here are the dependencies in stages:

• Stage 1
  • Python
    • Obviously, there's no way to print a friendly error if python is missing
• Stage 2
  • cmake

    Since CMake may be used to check for all of the other dependencies, it's ok if further dependency checking stops here.

• Stage 3
  • All platforms
    • svn (to get latest release branch)
    • openssl SDK
  • Windows
    • Some version of MSVC or cygwin or mingw32
    • bison and flex (from cygwin)
    • DirectX SDK (Microsoft DirectX SDK (November 2007))
    • Quicktime SDK
  • Linux (Debian package names, other platforms may differ)

    Some header/development packages are also required

    • build-essential (gcc/ld/make etc)
    • libgl1-mesa-dev
    • libzip-dev
    • libfreetype6-dev
    • libxft2-dev
    • libxrandr-dev
    • libxcursor-dev
    • libglu1-mesa-dev
    • libxt-dev
    • unzip

In all cases, there should be URLs provided for each missing option. For maintainability, these will probably be links to wiki.secondlife.com rather than direct links to the download site.

On Linux distros, it may be more helpful to just work it like a missing dependency, providing the command line and perhaps even offering to run it.

Error URLs and helpful error text

Currently, there's not much in the way of exception handling in develop.py. Most exceptions are for missing dependencies and misconfiguration problems. A generic exception handler should be implemented, with a --debug option (or similar) implemented to override with full information, and an help URL for all exceptions should be provided. There may be helpful nuggets in the exceptions that can be used to construct a wiki URL that won't exist when the error is first encountered, but could be quickly populated by people who actually encounter the error.

Problems with PATH variable

Error text should suggest when PATH problems are a problem. If there's any easy way to actually diagnose and fix/suggest fix for this, it should be pursued.

Rumor has it that on Windows, the CMake and/or Python installer can silent fails when "install for all users" is selected. This should be investigated, bug filed, and even better, patch submitted.

Automatic artwork and lib fetching

Currently, artwork and lib fetching is still partially manual.

Here's a shell snippet that works on Linux. This should be integrated into develop.py and done in a cross platform fashion.

chmod +x release/doc/asset_urls.txt
. release/doc/asset_urls.txt
wget $SLASSET_ART
wget $SLASSET_LIBS_WIN32
unzip slviewer-artwork*
unzip slviewer-*-libs*
cp linden/* release/ -r

On some branches, win32 asset_urls.txt are in linden/doc

For mac and linux you should use $SLASSET_LIBS_LINUXI386 or $SLASSET_LIBS_DARWIN and for linux its not unzip for the libs its tar :-

tar -xvzf slviewer-*-libs*

In fact the differences between the different libs packages now are trivial as it only contains some fonts and other trivial bits. But failure to download it (or the wrong one) can cause packaging later to fail.

message.xml checksumming

If you have many versions of the viewer on your machine, on Win32 it can be possible for the wrong messages.xml to get used sometimes if it isn't explicitly copied. Second Life will tend to run, but using the wrong one can cause subtle hard to trace errors. Proposed fix: cp linden/etc/messages.xml over to linden/indra/build-VC80/newview/app-settings.

Install FMOD API

At the time of writing fmod still needs to be manually downloaded and installed within the linden source tree. This is assumed you are still working from the cygwin shell or a linux terminal.

(linux)

wget http://www.fmod.org/index.php/release/version/fmodapi375linux.tar.gz
tar -xvzf fmodapi375linux.tar.gz
cp fmodapi375linux/api/libfmod-3.75.so release/libraries/i686-linux/lib_debug/
cp fmodapi375linux/api/libfmod-3.75.so release/libraries/i686-linux/lib_release/
cp fmodapi375linux/api/libfmod-3.75.so release/libraries/i686-linux/lib_release_client/
cp fmodapi375linux/api/inc/* release/libraries/include/
 

(windows)

wget http://www.fmod.org/index.php/release/version/fmodapi375win.zip
unzip fmodapi385win.zip
cp fmodapi375linux/api/inc/* release/libraries/include/
<NOT FINISHED>

basically for windows find the fmod.dll and copy it into the windows lib_debug and windows lib_release folders.

Generate the project files

This is where the real fun and grace of cmake starts to take shape. Open a terminal window and go to the folder with the viewer source code. Enter the folder linden/indra/ now run the develop.py script. Probably on windows if python installed correctly you can just type "develop.py".

(It seems possible to stay with in your cygwin prompt but if this breaks on windows, open a command prompt instead)

cd release/indra
./develop.py configure

it may be necessary to add a -G option which selects which version of visual studio to target, possible options are :-

• VC71
• VC2003 (default)
• VC80 (VS2005)
• VC90 (VS2008)

for example

develop.py configure -G VC90

will generate project files for Visual C++ 2008

Library download speed optimizations

An investigation into why some developers get very slow downloads needs to occur.

STOP READING HERE

I'm currently adapting Michelle2's build instructions into an initial spec, and stopping and saving here. -- Rob Linden 17:28, 7 August 2008 (PDT)

Build the viewer

Now the previous cmake process will have generated some build project files, either Unix makefiles, xcode project files or Visual Studio project files.

Visual Studio files (and xcode?) can be found under linden/indra/build-vc71 (in fact the exact path name depends on which target you have generated for). Just open the solution file .sln in that folder and build the solution in the standard way. Unix make files are found in the same location and a build can be started with a make in that directory as well. And i would guess that xcode projects are also created in the same way.

If you don't have the VSTool.exe issues below, just opening up your Visual Studio c++ will show secondlife at the top of the recent projects, ready for you to open.

It is also possible to start the build directly from the command line/console with :-

./develop.py build

On Windows (and mac?) this starts the build without using the Visual Studio/XCode environment but only using the compiler tool chain.

Problems

Auto package download

Last SVN commit broke it, so we all just have to wait a little while for the export scripts to be updated. Someone must have been hand merging the script and now as release with cmake approaches a real release, little issues with the automated scripts are turning up.

Express editions of Visual C++

Express editions do not have some of the extra features that VSTool.exe requires, VSTool is a little application that sets a few default project options, like which project is the startup and which build configuration to use.

FMOD

FMOD headers and libs need to be copied into the viewer build directory as stated above BUT on Windows the manifests are not currently moving the libs to the correct location post build so the application will fail to run, to resolve copy fmod.dll to indra/build-vc71/newview/debug and indra/build-vc71/newview/relwithdebinfo and indra/build-vc71/newview/release

LLKDU

This is not present in the automated libs download but is required at least on windows by the copy_win_scrips script. The file is not actually needed for the viewer to run (if present it will be used for jpeg2000 decoding in preference to Openjpeg. The solution is to remove the lines from newview/copy_win_scripts that reference llkdu and this should then allow this custom build step to run without issue

Linux

ligGL.a is supplied in the linden library package and it probably should not be. To avoid linker errors please remove indra/libraries/i686-linux/release/libGL.a and indra/libraries/i686-linux/debug/libGL.a

cmake 2.6

cmake 2.6 will fail to produce a package on linux and fail with a Make error. This does not happen with cmake 2.4

Custom builds

If you wish to do custom builds or things that develop.py just was not intended for then please have a look at User:Michelle2_Zenovka/cmake-flags