User:Henri Beauchamp/Automated Linux Build Script (1.20 and earlier)
Jump to navigation
Jump to search
Return to Compiling the viewer (Linux)
Automated libraries and headers adjustments, compilation and packaging
Here is a bash scripts (for v1.20 and earlier viewers: for v1.21 and later, please see cmake-SL) that could save you a lot of time... It basically does all what is needed to automatically build and package a viewer in Linux.
IMPORTANT NOTE:
This script includes instructions which have the potential to remove MANY files on your system should you modify it without understanding exactly what it does (i.e. rm -rf $VARIABLE/ where entire portions of your filesystem *will* be deleted without warning).
Although some basic checks are done to avoid removing an entire home directory, or even the root tree, this script is not meant to be tweaked without a very good knowledge of bash scripting. It is therefore highly recommended that you use it without modifying the PATH_TO_SOURCES and TEMP_DIR variables (modifying the USE_* and TUNE_FLAGS variables is safe).
#!/bin/bash
# make-SL v1.94 (c)2007-2010 Henri Beauchamp. Released under GPL license v2:
# http://www.gnu.org/licenses/gpl.txt
###############################################################################
######## THIS IS QUICK'N DIRTY ALPHA SOFTWARE. USE AT YOUR OWN RISKS ! ########
###############################################################################
###############################################################################
# BEWARE: this script is meant to compile a -personal- SL client. It is -NOT- #
# suitable to build client versions meant for public release, because #
# non-open source code is packaged by this script (e.g. libkdu). #
###############################################################################
# This bash script is aimed at easying up the build process of a SL client.
# It does not cover building it with custom/system openjpeg, elfio or fmodapi
# (we use the provided libraries).
# You may enable or disable the use of your system's library by editing
# the USE_SYSTEM_* variable ("yes" --> use the system library, "no" --> use
# LL's provided ones).
# The script also takes care of updating properly the viewer_manifest.py script
# accordingly, so that you (should) end up with a properly packaged client.
# To use this script, simply make it executable (chmod +x make-SL) and
# put it into /usr/local/bin (or any other directory in your PATH).
# Then, download the slviewer-src-*.tar.gz, slviewer-linux-libs-*.tar.gz,
# slviewer-artwork-*.zip and fmodapi*.tar.gz archives, and finally, invoke
# make-SL as follow:
# make-SL path_to_archives (example: make-SL ~/downloads)
# or simply:
# make-SL
# when invoking from the directory where the archives are.
# The sources will be installed into the PATH_TO_SOURCES directory,
# and temporary files stored into the TEMP_DIR directory.
# The packaged build will be moved to your home directory.
# If you want to retry a compilation after fixing something manually and
# don't want make-SL to start all over again, overwriting everything,
# you may invoke it with the --retry option, like this:
# make-SL --retry
# This script has been tested by the author, on a (very customized)
# Mandrake 10.2 distro, and on (less customized but yet not standard)
# Mandriva 2007.1, 2009.0 and 2010.0 distros.
# Tested with SL v1.17, v1.18, v1.19 & v1.20 sources.
# Where the sources of the client will be held (defaults to "./linden"):
PATH_TO_SOURCES="/usr/src/SL"
# Where to store temporary files:
TEMP_DIR="/usr/tmp/$USER"
USE_SYSTEM_GTK="no"
USE_SYSTEM_SDL="no"
USE_SYSTEM_SSL="no"
# Beware: libdb4 makes use of libapr, and libapr makes use of libexpat... so
# you should keep USE_SYSTEM_APR, USE_SYSTEM_DB4 and USE_SYSTEM_EXPAT in sync.
USE_SYSTEM_APR="no"
USE_SYSTEM_DB4="no"
USE_SYSTEM_EXPAT="no"
USE_SYSTEM_OGG="no"
USE_SYSTEM_ZLIB="no"
USE_SYSTEM_UUID="no"
# If your system libcurl does not use c-ares (non-blocking DNS calls), better
# using LL's provided library...
USE_SYSTEM_CURL="no"
USE_SYSTEM_VORBIS="no"
USE_SYSTEM_XMLRPC="no"
USE_SYSTEM_JPEGLIB="no"
# Enabling the system freetype2 prevents the client from using the SL fonts,
# or at least, this is what happens on my system (Mandrake 10.2 and Mandriva
# 2007.1 with PLF's freetype2).
USE_SYSTEM_FREETYPE2="no"
USE_SYSTEM_PERFTOOLS="no"
USE_SYSTEM_ELFIO="no"
# You may add tune flags here, to optimize the code for your processor.
# Example, for an Athlon XP:
# TUNE_FLAGS="-march=athlon-xp"
# BEWARE: because of a bug in the v1.17.2.0 SConstruct file, you MUST keep at
# least an -O2 flag in the TUNE_FLAGS, to avoid getting a very slooooow client...
# This problem was apparently fixed in v1.17.3.0.
TUNE_FLAGS="-fomit-frame-pointer -frename-registers -ftree-vectorize -fweb -fexpensive-optimizations -march=i686 -msse -mfpmath=sse"
# Set this to "yes" to force vectorization use in the viewer code (only for
# processors with SSE or Altivec units, and only if you did enable them via
# the TUNE_FLAGS.
FORCE_VECTORIZE="yes"
# Whether to build with the browser login screen or not:
WITH_MOZILLA="yes"
# When using gcc v4.2 or later, do set this to yes...
ALLOW_WARNINGS="no"
########### functions definitions ###########
function check() {
if [ "$1" == "yes" ] ; then
if [ -f $2 ] ; then
echo "Using the system $3..."
return 0
else
echo "WARNING: system $3 requested but not available..."
fi
fi
return 1
}
function update_manifest() {
grep -v $1 $PATH_TO_SOURCES/indra/newview/viewer_manifest.py >$TEMP_DIR/viewer_manifest.py
mv -f $TEMP_DIR/viewer_manifest.py $PATH_TO_SOURCES/indra/newview/viewer_manifest.py
chmod +x $PATH_TO_SOURCES/indra/newview/viewer_manifest.py
}
function compile() {
cd $PATH_TO_SOURCES/indra
echo "Compiling the client into $TEMP_DIR..."
scons -j6 DISTCC=no BTARGET=client OPENSOURCE=no BUILD=releasefordownload CHANNEL=Release MOZLIB=$WITH_MOZILLA MOZLIB2=$WITH_MOZILLA
if (($? == 0)) ; then
mv $PATH_TO_SOURCES/indra/newview/SecondLife*.tar.bz2 $HOME/
fi
}
########### end of functions ###########
if [ "$TEMP_DIR" == "" ] ; then
export TEMP_DIR=/tmp/$USER/SL
fi
# Check to see if we simply want to retry a compilation:
if [ "$1" == "--retry" ] ; then
compile
exit $?
fi
# Check to see if we want to build a Windows source tree:
WINDOWS="no"
if [ "$1" == "--win" ] ; then
WINDOWS="yes"
shift
fi
# Make a clean build:
if [ -d $TEMP_DIR ] && [ "$TEMP_DIR" != "" ] && [ "$TEMP_DIR" != "/" ] && [ "$TEMP_DIR" != "." ] && [ "$TEMP_DIR" != ".." ] && [ "$TEMP_DIR" != "$HOME" ] ; then
rm -rf $TEMP_DIR/
fi
mkdir -p $TEMP_DIR
cd $TEMP_DIR
# Make sure we don't unpack over an old source tree:
if [ -d linden ] ; then
rm -rf linden/
fi
if [ -d $PATH_TO_SOURCES ] && [ "$PATH_TO_SOURCES" != "" ] && [ "$PATH_TO_SOURCES" != "/" ] && [ "$PATH_TO_SOURCES" != "." ] && [ "$PATH_TO_SOURCES" != ".." ] && [ "$PATH_TO_SOURCES" != "$HOME" ] ; then
rm -rf $PATH_TO_SOURCES/
fi
# Use the parameter (if any) as the path to the archives:
PATH_TO_ARCHIVES=`pwd`
if [ "$1" != "" ] && [ "$1" != "--prep" ]; then
if [ -d $1 ] ; then
PATH_TO_ARCHIVES=$1
shift
fi
fi
# Let's first unpack everything:
if ! [ -f $PATH_TO_ARCHIVES/slviewer-src-*.tar.gz ] ; then
echo "You need slviewer-src !"
exit 1
fi
if ! [ -f $PATH_TO_ARCHIVES/slviewer-linux-libs-*.tar.gz ] ; then
echo "You need slviewer-linux-libs !"
exit 1
fi
if ! [ -f $PATH_TO_ARCHIVES/slviewer-artwork-*.zip ] ; then
echo "You need slviewer-artwork !"
exit 1
fi
if ! [ -f $PATH_TO_ARCHIVES/fmodapi*.tar.gz ] ; then
echo "You need fmodapi !"
exit 1
fi
echo "Extracting the files from the archives..."
tar --no-same-owner -xzf $PATH_TO_ARCHIVES/slviewer-src-*.tar.gz
if [ "$WINDOWS" == "yes" ] ; then
unzip -oq $PATH_TO_ARCHIVES/slviewer-win32-libs-*.zip
else
tar --no-same-owner -xzf $PATH_TO_ARCHIVES/slviewer-linux-libs-*.tar.gz
fi
unzip -oq $PATH_TO_ARCHIVES/slviewer-artwork-*.zip
# You may add lib or artwork patches named in the form:
# slviewer-whatever-patch.zip
# Example: slviewer-v11900-statusbar-artwork-patch.zip
ARTWORK_PATCHES=`/bin/ls $PATH_TO_ARCHIVES/slviewer-*-patch.zip`
if [ "$ARTWORK_PATCHES" != "" ] ; then
for f in $ARTWORK_PATCHES; do
echo "Extracting lib/artwork patch $f..."
unzip -oq $f
done
fi
# fmodapi:
echo "Copying fmodapi files..."
tar --no-same-owner -xzf $PATH_TO_ARCHIVES/fmodapi*.tar.gz
cp -a fmodapi*/api/inc/* linden/libraries/i686-linux/include/
cp fmodapi*/api/libfmod-*.so linden/libraries/i686-linux/lib_release_client/
rm -rf fmodapi*/
# Move the sources to where we want to hold them:
if [ "$PATH_TO_SOURCES" != "linden" ] && [ "$PATH_TO_SOURCES" != "" ] ; then
echo "Moving the sources to $PATH_TO_SOURCES..."
mv -f linden $PATH_TO_SOURCES
fi
# Let's own the files:
chown -R $USER: $PATH_TO_SOURCES/
# Check for patches to apply. The names of the patches must be in the form
# slviewer-*.patch* (Example: slviewer-v117-EmbbededNotecard.patch.bz2).
# They must be applicable from inside the source directory with the -p1
# option, i.e. they have been built from outside the source directory
# with a diff command such as:
# diff -urN linden/ linden-patched/ >slviewer-whatever.patch
# And they may be gzipped or bzipped.
PATCHES=`/bin/ls $PATH_TO_ARCHIVES/slviewer-*.patch* 2>/dev/null`
if [ "$PATCHES" != "" ] ; then
echo "Applying patches..."
cd $PATH_TO_SOURCES
for i in $PATCHES; do
echo "Patch: $i"
if echo $i | grep ".gz" &>/dev/null ; then
gunzip -c $i | patch -p1 -s
elif echo $i | grep ".bz2" &>/dev/null ; then
bzcat $i | patch -p1 -s
else
patch -p1 -s <$i
fi
done
find . -name "*.orig" | xargs rm -f
fi
# In v1.18.4.0, they use c-ares but fail to provide it in the library package...
if [ -f $PATH_TO_SOURCES/indra/llcommon/llares.cpp ] ; then
if ! [ -f $PATH_TO_SOURCES/libraries/i686-linux/include/ares/ares_dns.h ] && ! [ -f $PATH_TO_SOURCES/libraries/include/ares/ares_dns.h ] ; then
if [ -f /usr/include/ares_dns.h ] && [ -f /usr/lib/libcares.a ] ; then
echo "Copying missing c-ares files from the system..."
cd $PATH_TO_SOURCES/libraries/i686-linux
mkdir include/ares
cp -a /usr/include/ares* include/ares/
cp -a /usr/lib/libcares* lib_release_client/
else
echo "Sorry... You need c-ares installed in your system to compile this version of the client."
echo "Try: http://daniel.haxx.se/projects/c-ares/"
exit 1
fi
fi
fi
# Let's use the system GTK+ if available:
if check $USE_SYSTEM_GTK /usr/include/atk-1.0/atk/atk.h "GTK+" ; then
cd $PATH_TO_SOURCES/libraries/i686-linux/lib_release_client
rm -f libgtk* libgdk* libglib* libgmodule* libgobject* libgthread* libpango* libatk*
cd ../include
rm -rf atk-1.0/ gtk-2.0/ glib-2.0/ pango-1.0/
cp -a /usr/include/atk-1.0 .
cp -a /usr/include/gtk-2.0 .
cp -a /usr/lib/gtk-2.0/include/* gtk-2.0/
cp -a /usr/include/glib-2.0 .
cp -a /usr/lib/glib-2.0/include/* glib-2.0/
cp -a /usr/include/pango-1.0 .
if [ -d /usr/include/cairo ] ; then
cp -a /usr/include/cairo/* .
fi
fi
# Let's use the system freetype2 if available:
if check $USE_SYSTEM_FREETYPE2 /usr/include/ft2build.h "freetype2" ; then
cd $PATH_TO_SOURCES/libraries
rm -rf include/freetype/ i686-linux/include/llfreetype2/*
rm -f include/ft2build.h i686-linux/lib_release_client/libfreetype.a
cp -a /usr/include/freetype2/freetype/ i686-linux/include/llfreetype2/
cp -a /usr/include/ft2build.h i686-linux/include/llfreetype2/freetype/
fi
# Let's use the system zlib if available:
if check $USE_SYSTEM_ZLIB /usr/include/zlib.h "zlib" ; then
cd $PATH_TO_SOURCES/libraries
rm -rf include/zlib/
mkdir -p i686-linux/include/zlib
cp -a /usr/include/zlib*.h i686-linux/include/zlib/
fi
# Let's use the system jpeglib if available:
if check $USE_SYSTEM_JPEGLIB /usr/include/jpeglib.h "jpeglib" ; then
cd $PATH_TO_SOURCES/libraries
rm -rf include/jpeglib/ i686-linux/lib_release_client/libjpeg.a
mkdir -p i686-linux/include/jpeglib
cp -a /usr/include/j*.h i686-linux/include/jpeglib/
touch i686-linux/include/jpeglib/jinclude.h
fi
# Let's use the system xmlrpc-epi if available:
if [ -d /usr/include/xmlrpc-epi ] ; then
XMLRPCDIR=/usr/include/xmlrpc-epi
else
XMLRPCDIR=/usr/include
fi
if check $USE_SYSTEM_XMLRPC $XMLRPCDIR/xmlrpc.h "xmlrpc-epi" ; then
cd $PATH_TO_SOURCES/libraries
rm -rf include/xmlrpc-epi/ i686-linux/lib_release_client/libxmlrpc.a
mkdir -p include/xmlrpc-epi
if [ -d /usr/include/xmlrpc-epi ] ; then
cp -a $XMLRPCDIR/*.h include/xmlrpc-epi/
else
cp -a /usr/include/xmlrpc*.h include/xmlrpc-epi/
fi
fi
# Let's use the system ogg if available:
if check $USE_SYSTEM_OGG /usr/include/ogg/ogg.h "ogg" ; then
cd $PATH_TO_SOURCES/libraries
rm -rf include/ogg/ i686-linux/lib_release_client/libogg*
update_manifest libogg
fi
# Let's use the system vorbis if available:
if check $USE_SYSTEM_VORBIS /usr/include/vorbis/vorbisenc.h "vorbis" ; then
cd $PATH_TO_SOURCES/libraries
rm -rf include/vorbis/ i686-linux/lib_release_client/libvorbis*
update_manifest libvorbis
fi
# Let's use the system SDL if available:
if check $USE_SYSTEM_SDL /usr/include/SDL/SDL.h "SDL" ; then
cd $PATH_TO_SOURCES/libraries/i686-linux
rm -rf include/SDL/ lib_release_client/libSDL*
update_manifest libSDL
fi
# Let's use the system openssl if available:
if check $USE_SYSTEM_SSL /usr/include/openssl/opensslconf.h "openssl" ; then
cd $PATH_TO_SOURCES/libraries/i686-linux/lib_release_client
rm -f libssl.* libcrypto.*
update_manifest libssl
update_manifest libcrypto
fi
# Let's use the system apr if available:
if check $USE_SYSTEM_APR /usr/include/apr*/apr_base64.h "apr" ; then
cd $PATH_TO_SOURCES/libraries/i686-linux
rm -rf include/apr-1/*
rm -f lib_release_client/libapr*
cp -a /usr/include/apr*/* include/apr-1/
update_manifest libapr
fi
# Let's use the system db4 if available:
if check $USE_SYSTEM_DB4 /usr/include/db4/db.h "db4" ; then
rm -f $PATH_TO_SOURCES/libraries/i686-linux/lib_release_client/libdb*.so
update_manifest libdb
fi
# Let's use the system expat if available:
if check $USE_SYSTEM_EXPAT /usr/include/expat.h "expat" ; then
cd $PATH_TO_SOURCES/libraries
rm -rf include/expat/
rm -f i686-linux/lib_release_client/libexpat*
mkdir -p i686-linux/include/expat
cp -a /usr/include/expat*.h i686-linux/include/expat/
update_manifest libexpat
fi
# Let's use the system curl if available:
if check $USE_SYSTEM_CURL /usr/include/curl/curl.h "curl" ; then
cd $PATH_TO_SOURCES/libraries
rm -rf include/curl/
rm -f i686-linux/lib_release_client/libcurl.*
update_manifest libcurl
fi
# Let's use the system uuid if available:
if check $USE_SYSTEM_UUID /usr/include/uuid/uuid.h "libuuid" ; then
rm -f $PATH_TO_SOURCES/libraries/i686-linux/lib_release_client/libuuid.*
update_manifest libuuid
fi
if grep tcmalloc $PATH_TO_SOURCES/indra/SConstruct &>/dev/null ; then
if check $USE_SYSTEM_PERFTOOLS /usr/include/google/malloc_hook.h "google-perftools" ; then
if [ -f /usr/lib/libstacktrace.so ] ; then
cd $PATH_TO_SOURCES/libraries/i686-linux
rm -f lib_release_client/libtcmalloc.* lib_release_client/libstacktrace.*
rm -rf include/google/
cp -a /usr/lib/libstacktrace.* /usr/lib/libtcmalloc.so* lib_release_client/
cp -a /usr/include/google include/
update_manifest tcmalloc
update_manifest stacktrace
else
echo "WARNING: bad google-perftools version (libstacktrace not available), skipping."
fi
else
# Remove the google-perftools stuff if not present in the libraries package
# (problem seen in SL v1.17):
if ! [ -f $PATH_TO_SOURCES/libraries/i686-linux/lib_release_client/libtcmalloc.so ] ; then
update_manifest tcmalloc
update_manifest stacktrace
# The following lines remove the
# "external_libs += ['tcmalloc', 'stacktrace']"
# line from SConstruct... and many other lines which renders SConstruct
# unusable to build anything else than the client... this is a quick'n
# dirty patch. :-P
grep -v tcmalloc $PATH_TO_SOURCES/indra/SConstruct >$TEMP_DIR/SConstruct
mv -f $TEMP_DIR/SConstruct $PATH_TO_SOURCES/indra/SConstruct
fi
fi
fi
# Let's use the system libELFIO if available:
if check $USE_SYSTEM_ELFIO /usr/include/ELFIO.h "libELFIO" ; then
cd $PATH_TO_SOURCES/libraries/i686-linux
rm -f include/ELFIO/*
rm -f lib_release_client/libELFIO.*
cp -af /usr/include/ELF* include/ELFIO/
cp -af /usr/lib/libELFIO.so lib_release_client/
update_manifest ELFIO
fi
# Fix a bug in v1.17.1 and later SConstruct file, where some libraries are
# compiled as shared instead of static, and are yet not packaged, and where
# fmod can't be enabled for OPENSOURCE:
sed -e "s/create_dynamic_module(module=module, module_libs=module_libs)/create_static_module(module)/" $PATH_TO_SOURCES/indra/SConstruct >$TEMP_DIR/SConstruct
sed -e "s/enable_fmod = not opensource and/enable_fmod =/" $TEMP_DIR/SConstruct >$PATH_TO_SOURCES/indra/SConstruct
rm -f $TEMP_DIR/SConstruct
# Force the vectorization use if we chose so.
if [ "$FORCE_VECTORIZE" == "yes" ] ; then
TUNE_FLAGS="$TUNE_FLAGS -DLL_VECTORIZE=1"
fi
# Add tune flags, if any:
if [ "$TUNE_FLAGS" != "" ] ; then
sed -e "s/-pipe/$TUNE_FLAGS -pipe/" $PATH_TO_SOURCES/indra/SConstruct >$TEMP_DIR/SConstruct
mv -f $TEMP_DIR/SConstruct $PATH_TO_SOURCES/indra/SConstruct
fi
# Use whatever gcc version we got.
sed -e "s/g++-3.4/g++/" $PATH_TO_SOURCES/indra/SConstruct >$TEMP_DIR/SConstruct
sed -e "s/g++-4.1/g++/" $TEMP_DIR/SConstruct >$PATH_TO_SOURCES/indra/SConstruct
rm -f $TEMP_DIR/SConstruct
# Fix boost library linking problems in v1.20
sed -e "s/-gcc34-mt//g" $PATH_TO_SOURCES/indra/SConstruct >$TEMP_DIR/SConstruct
mv -f $TEMP_DIR/SConstruct $PATH_TO_SOURCES/indra/SConstruct
# Newer GCCs (4.2 and later) will issue many warnings: let's disable -Werror
if [ "$ALLOW_WARNINGS" == "yes" ] ; then
sed -e "s/-Werror //" $PATH_TO_SOURCES/indra/SConstruct >$TEMP_DIR/SConstruct
mv -f $TEMP_DIR/SConstruct $PATH_TO_SOURCES/indra/SConstruct
fi
# Update the manifest file and enable libkdu packaging. I'm too lazy to use
# awk or perl, here, but that's indeed what should be done...
# The following (very dirty) code may break at some point, depending on what
# LL will put in comments in the manifest file...
# First, remove the lines with libstdc++.so.6 and the crash logger:
update_manifest libstdc
update_manifest crash_logger
# Now, any line with a '#' followed with several spaces _should_ be dealing
# with the libkdu stuff... So, we simply remove the '#"...
sed -e "s/# them/# them/" $PATH_TO_SOURCES/indra/newview/viewer_manifest.py >$TEMP_DIR/viewer_manifest.py
sed -e "s/# / /" $TEMP_DIR/viewer_manifest.py >$PATH_TO_SOURCES/indra/newview/viewer_manifest.py
rm -f $TEMP_DIR/viewer_manifest.py
chmod +x $PATH_TO_SOURCES/indra/newview/viewer_manifest.py
# Comment out "export LL_GL_BASICEXT=x" in the wrapper script:
sed -e "s/export LL_GL_BASICEXT=x/#export LL_GL_BASICEXT=x/" $PATH_TO_SOURCES/indra/newview/linux_tools/wrapper.sh >$TEMP_DIR/wrapper.sh
mv -f $TEMP_DIR/wrapper.sh $PATH_TO_SOURCES/indra/newview/linux_tools/wrapper.sh
chmod +x $PATH_TO_SOURCES/indra/newview/linux_tools/wrapper.sh
# Missing file... at least in v1.17.0.0
if grep "secondlife-i686.supp" $PATH_TO_SOURCES/indra/newview/viewer_manifest.py &>/dev/null ; then
if ! [ -f $PATH_TO_SOURCES/indra/newview/secondlife-i686.supp ] ; then
touch $PATH_TO_SOURCES/indra/newview/secondlife-i686.supp
fi
fi
# LL makes the assumption that a certain unicode true type font is installed
# on all Linux systems... which is of course wrong ! Let's check, and when
# this font is absent, replace the broken soft link with an empty file: this
# at least will prevent the viewer to issue many warnings about not being able
# to load the unicode.ttf font.
if ! [ -r $PATH_TO_SOURCES/indra/newview/linux_tools/unicode.ttf ] ; then
rm -f $PATH_TO_SOURCES/indra/newview/linux_tools/unicode.ttf
touch $PATH_TO_SOURCES/indra/newview/linux_tools/unicode.ttf
fi
# Check to see if we simply want to unpack the sources:
if [ "$1" == "--prep" ] || [ "$WINDOWS" == "yes" ] ; then
echo "Prepared sources available in: $PATH_TO_SOURCES"
exit 0
fi
compile