Difference between revisions of "Third Party Libraries"

From Second Life Wiki
Jump to navigation Jump to search
(72 intermediate revisions by 24 users not shown)
Line 1: Line 1:
{{OSWikiLearnBox}}
__NOTOC__
== Third Party Libraries ==
The following table lists libraries and other third party (3p) dependencies used in constructing the Second Life Viewer.


=== apache portable runtime ===
Most of these (all the open source ones) should be available as prebuilds packaged by and for use with [[Autobuild]].
* http://apr.apache.org/
The apache portable runtime is a set of c functions which provide a portable and consistent interface into operating system - or simply widely useful - functionality. We primarily use it for threading and socket i/o.


=== boost ===
==== Generic Layout (Single-branch, 2024 Style) ====
* http://www.boost.org/
Boost is a set of portable c++ libraries which provide a wide set of functionality. We primarily use it for tokenization.


=== TUT ===
3p packages should vendor upstream contents using [https://git-scm.com/book/en/v2/Git-Tools-Submodules git submodules] unless a source git repository for the source dependency is not available.
* http://tut-framework.sourceforge.net/
A typical 3p package layout looks like this:
The TUT framework is a unit test framework based on the use of c++ meta template programming. We use it to test our libraries.


=== quicktime ===
<pre>
* http://developer.apple.com/quicktime/
.
We use this to play in-world video clips on a prim.
├── [vendor submodule]/ - Git submodule containing upstream source
├── .github/           - CI/CD and repository configuration
├── autobuild.xml      - Autobuild manifest
├── build-cmd.sh        - Script used to build the repository
├── LICENSE            - Repository licensing
└── README.md          - Repository readme
</pre>


=== libcurl ===
===== Contributor Instructions =====
* http://curl.haxx.se/libcurl/
The libcurl library handles moving data across the net in many different protocols. We use it to GET/POST/PUT/DELETE web resources.


=== zlib ===
The general workflow for contributing changes to a 3p library looks like this:
* http://www.zlib.net/
zlib is a lossless data-compression library which is used for a variety of network i/o and file compression.


=== mozilla ===
# Fork the repository and make changes. Be sure to test your work locally.
* http://www.mozilla.org/developer/
# Create a PR (Be sure to follow [https://gist.github.com/mikepea/863f63d6e37281e329f8 PR Etiquette] and our [https://github.com/secondlife/.github/blob/main/CODE_OF_CONDUCT.md Code of Conduct])
The mozilla library handles rendering HTML in the viewer.
# Have this PR reviewed and merged
# Done! The maintainer should cut a new version as appropriate using the workflow described below.


=== cg ===
===== Maintainer Instructions =====
* http://developer.nvidia.com/page/cg_main.html
cg controls some visual effects


=== gl ===
[[File:Screenshot 2024-08-10 at 11-48-23 Releases · secondlife 3p-zlib-ng.png|thumb|top|right|A properly versioned and promoted release of 3p-zlib-ng]]
* http://www.opengl.org/
OpenGL renders Second Life.


=== Expat ===
If you are a 3p package maintainer you will want to follow these instructions to review and release new versions:
* http://expat.sourceforge.net/
Expat is a XML parser.


=== FreeType ===
# Review PRs quickly, and merge them directly into the default branch once approved
* http://www.freetype.org/
# Cut a new release by using Github's Releases feature:
FreeType is a font engine.
## Navigate to '''Repository -> Releases''' and click '''"Draft new Release"'''
## Click '''Choose a tag''' and create a new tag using the <code>vUPSTREAM-rRELEASE</code> format described below
## Click '''Generate release notes''' and edit them as appropriate. Keep the title of the release equal to the version being used.
## Click ''Publish release''


=== jpeglib ===
After the release is published, CI/CD should be triggered by the addition of the new tag, and provide a nice set of packages and instructions for consumers to use.
* http://www.ijg.org/
JPEGlib is JPEG decoder library.  It is an open alternative to KDU.


=== KDU ===
===== Version format =====
* http://www.kakadusoftware.com/
Kakadu (KDU) is a JPEG decoder library.


=== Ogg Vorbis ===
3p packages should be released using the following version schema:
* http://xiph.org/ogg/
* http://xiph.org/vorbis/
The Ogg & the Vorbis libraries control audio in Second Life.


=== SDL ===
<pre>
* http://www.libsdl.org/index.php
Format
The Simple DirectMedia Layer libraries are used for handling input and basic window/GL setup on the Linux client.


=== smrtheap ===
  vUPSTREAM-rRELEASE
* http://www.microquill.com/smartheap/index.html
Smartheap is a memory allocation library.


=== speedtree ===
  Example: v1.0.0-r2 (Second vendored release of v1.0.0 of this library)
* http://www.speedtree.com/
Speedtree controls rendering of trees that are not yet implemented in SL.


=== xmlrpc-epi ===
  UPSTREAM - Upstream package version. ex. 1.0.0 or 6.7
* http://xmlrpc-epi.sourceforge.net/
  RELEASE - Vendored release number, ex. 1, 2
Controls the XML-RPC protocol in SL.
</pre>


=== OpenJPEG ===
===== Patches =====


=== GTK+ ===
By convention, patches should be kept in a <code>patches/*</code> directory and applied to the upstream source during build. You can use the following utility function, declared in your '''build-cmd.sh''' script to help:
* http://www.gtk.org/
Used for 'native' UI components on the Linux client


=== ELFIO ===
<syntaxhighlight lang="bash">
* http://sourceforge.net/projects/elfio/
apply_patch()
Used for enhanced stacktrace information on the Linux client
{
    local patch="$1"
    local path="$2"
    echo "Applying $patch..."
    git apply --check --reverse --directory="$path" "$patch" || git apply --directory="$path" "$patch"
}
</syntaxhighlight>
''Patch helper from [https://github.com/secondlife/3p-boost 3p-boost]''


=== Version Information ===
===== Differences from old style (vendor branch) =====


This is an initial list from memory. This needs to be changed.
Originally, 3p libraries were maintained using a separate '''vendor''' and default branch. Repositories using this style should be migrated to new single-branch format. Benefits of using a single-branch are:


* apr-util-1.1.2
* Simpler PR process (Make PR against default, merge it. Done.)
* boost-1.32.0
* Faster release process, which means quicker integration for package consumers
* curl-7.15.1
* Easier integration with GitHub functionality such as Automatic Release Notes
* expat-1.95.8
* More explicit maintenance of patches makes delta from upstreams clearer and simpler to understand
* freetype-2.1.5
 
* glib-2.0.0
=== List of third party libraries ===
* jpeglib-6b
 
* kdu-5.2.1
{| border="1" cellpadding="3"
* Mesa-6.2.1
|- style="background:#ffdead;"
* mozilla-1.8.0.9
!Library
* openjpeg-1.0
!Repository
* openssl-0.9.7c
!License
* SDL-1.2.7
!Upstream Home
* tut-2006-06-22
!Description
* vorbis-1.1.2
 
* xmlrpc-epi-0.51
|-
* vorbis-1.0.3
|apr_suite
* zlib-1.2.1
|[https://github.com/secondlife/3p-apr_suite/ 3p-apr]
|[https://github.com/secondlife/3p-apr_suite/blob/main/apr/LICENSE Apache License]
|http://apr.apache.org/
|"A set of C functions which provide a portable and consistent interface into operating system - or simply widely useful - functionality. We primarily use it for threading and socket i/o. This is actually three packages, apr, apr-util and apr-iconv."
|-
|boost
|[https://github.com/secondlife/3p-boost 3p-boost]
|[https://github.com/boostorg/boost/blob/master/LICENSE_1_0.txt Boost License]
|http://www.boost.org/
|A set of portable C++ libraries which provide a wide set of functionality. Used primarily for tokenization.
|-
|curl
|[https://github.com/secondlife/3p-curl 3p-curl]
|''[https://github.com/secondlife/3p-curl/blob/master/curl/COPYING BSD-style]''<sup>[[#BSDstyle|B]]</sup>
| http://curl.haxx.se/libcurl/
|Handles moving data across the net in many different protocols. Used to GET/POST/PUT/DELETE web resources.
|-
|expat
|[https://github.com/secondlife/3p-expat 3p-expat]
|[https://github.com/libexpat/libexpat/blob/master/COPYING MIT License]
|https://github.com/libexpat/libexpat
|XML parser.
|-
|freetype
|[https://github.com/secondlife/3p-freetype 3p-freetype]
|[https://github.com/freetype/freetype/blob/master/LICENSE.TXT Freetype License and Others]
|http://www.freetype.org/
|Font engine.
|-
|glh_linear
|[https://github.com/secondlife/3p-glh_linear 3p-glh_linear]
|
|
|nVidia NVParse SDK: platform-independent C++ Apple OpenGL helper library
|-
|gstreamer
|[https://bitbucket.org/lindenlab/3p-gstreamer 3p-gstreamer]
|Open
|http://gstreamer.freedesktop.org
|
|-
|havok
|
|Commercial
|
|
|-
|jpeglib
|[https://bitbucket.org/lindenlab/3p-jpeglib 3p-jpeglib]
|Open
|http://www.ijg.org/
|JPEG decoder library.
|-
|kdu
|[https://hg.lindenlab.com/lindenlab/3p-kdu-private 3p-kdu-private]<sup>[[#PrivateRepo|LL]]</sup>
|Commercial
|http://www.kakadusoftware.com
|Kakadu (KDU) JPEG-2000 decoder library.
|-
|libpng
|[https://bitbucket.org/lindenlab/3p-libpng 3p-libpng]
|Open
| http://www.libpng.org/pub/png/libpng.html
|PNG image library.
|-
|libxml2
|[https://github.com/secondlife/3p-libxml2 3p-libxml2]
|[https://gitlab.gnome.org/GNOME/libxml2/-/blob/master/Copyright?ref_type=heads MIT License]
|
|
|-
|ndofdev
|[https://github.com/secondlife/3p-libndofdev 3p-libndofdev]
|''[https://github.com/secondlife/3p-libndofdev/blob/master/libndofdev/COPYING BSD-style]''<sup>[[#BSDstyle|B]]</sup>
|
|Provides joystick driver support (mac and win) for the 3DConnexion SpaceNavigator.
|-
|ogg_vorbis
|[https://github.com/secondlife/3p-ogg_vorbis 3p-ogvorbis]
|''[https://github.com/xiph/ogg/blob/master/COPYING BSD-style]''<sup>[[#BSDstyle|B]]</sup>
|http://xiph.org/ogg/
|Audio codec and playback.
|-
|openal
|[https://github.com/secondlife/3p-openal-soft 3p-openal-soft]
|[https://github.com/secondlife/3p-openal-soft/blob/main/openal-soft/COPYING GPL]
|http://connect.creativelabs.com/openal/default.aspx
|3D Spatial audio
|-
|openjpeg
|[https://github.com/secondlife/3p-openjpeg 3p-openjpeg]
|''[https://github.com/uclouvain/openjpeg/blob/master/LICENSE BSD-style]''<sup>[[#BSDstyle|B]]</sup>
|http://www.openjpeg.org/
|An open-source JPEG-2000 library; a slower alternative to Kadaku. Used in the open source release
|-
|openssl
|[https://github.com/secondlife/3p-openssl 3p-openssl]
|[https://github.com/openssl/openssl/blob/master/LICENSE.txt Apache License]
|http://www.openssl.org/
|Provides encryption for sensitive actions such as user login
|-
|sdl
|[https://github.com/secondlife/3p-sdl2 3p-sdl2]
|[https://github.com/libsdl-org/SDL/blob/main/LICENSE.txt zlib License]
|http://www.libsdl.org/index.php
|The Simple DirectMedia Layer libraries are used for handling input and basic window/GL setup on the Linux client.
|-
|slvoice
|
|Commercial
|
|This is the Vivox provided executable (renamed SLVoice)
|-
|tut
|[https://github.com/secondlife/3p-tut 3p-tut]
|''[https://github.com/secondlife/3p-tut/blob/master/LICENSE.txt BSD-style]''<sup>[[#BSDstyle|B]]</sup>
|http://tut-framework.sourceforge.net/
|Unit test framework based on the use of c++ meta template programming. Used to test LL libraries.
|-
|xmlrpc_epi
|[https://github.com/secondlife/3p-xmlrpc-epi 3p-xmlrpc-epi]
|[https://github.com/secondlife/3p-xmlrpc-epi/blob/master/xmlrpc-epi/COPYING Epinions, Inc License]
|http://xmlrpc-epi.sourceforge.net/
|XML-RPC protocol utility
|-
|zlib-ng
|[https://github.com/secondlife/3p-zlib-ng 3p-zlib-ng]
|[https://github.com/zlib-ng/zlib-ng/blob/develop/LICENSE.md zlib License]
|http://www.zlib.net/
|zlib is a lossless data-compression library which is used for a variety of network i/o and file compression.
|}
 
= Redistribution =
Some files in the SL installation can't be freely redistributed. This means it's not possible to simply package and offer for download a modified viewer: some files '''must not''' be redistributed.
 
This page documents the licenses and redistribution requirements of various files used by the viewer.
 
== Windows ==
These are the files that need to be shipped with the Windows version
 
=== Runtime ===
Microsoft components. It's not yet known whether they're redistributable.
 
* dbghelp.dll: [http://msdn2.microsoft.com/EN-US/library/ms679294.aspx Redistributable]
* msvcp71.dll: C++ Runtime Library, redistributable    -viewer now using msvcp80.dll when compiled with VC2005
* msvcr71.dll: C Runtime Library, [http://msdn2.microsoft.com/en-us/library/abx4dbyh(vs.71).aspx Redistributable]  -viewer now using msvcr80.dll when compiled with VC2005
 
== FONTS ==
Meta font software files
 
From the Readme.txt
The Meta font software files contained in this folder are the
copyrighted property of FSI FontShop International ("FSI") and are
licensed by FSI solely for use by Linden Research, Inc. and by
residents or users of Second Life in the Second Life environment,
subject to the Second Life Terms of Service.  These Meta font software
files may not be copied by residents or developers of Second Life or
used by them for any other purpose whatsoever.
 
*  MtBkLfRg.ttf
*  MtBdLfRg.ttf
 
NOTE:  These files must be replaced for an true opensource build
 
=== Kakadu ===
JPEG2000 library. '''Not redistributable'''.
OpenJPEG is an alternative library that is redistributable
 
* llkdu.dll
 
Starting with Viewer 2.5, llkdu.dll has been phased out. The libkdu.a and kdu.lib are still not redistributable but can be built from the toolkit provided by Kakadu under a license. How to build a viewer with such a library is documented in [[KDU Implementation]].
 
=== OpenSSL ===
SSL library. Redistributable.
 
* libeay32.dll
* ssleay32.dll
 
=== Vivox ===
Vivox is the component that provides the voice functionality.
 
* SLVoice.exe: Voice daemon. '''Not redistributable'''
* SLVoiceAgent: Exact purpose unknown. '''Not redistributable'''
* alut.dll: OpenAL, by Creative Labs. Redistributable.
* ortp.dll: oRTP project, LGPL. Redistributable.
* srtp.dll: Secure RTP?
* vivoxsdk.dll: Vivox Communications API. '''Not redistributable'''
* wrap_oal.dll: OpenAL32 by Creative Labs. Redistributable.
 
=== Unknown Libraries ===
The purpose and licensing of these files is not known:
 
* tntk.dll: Probably Vivox related  -see [https://wiki.secondlife.com/wiki/Voice/Technical Voice/Technical] for other info on ViVox files
 
[[Category:Compiling viewer]]

Revision as of 21:40, 10 August 2024

The following table lists libraries and other third party (3p) dependencies used in constructing the Second Life Viewer.

Most of these (all the open source ones) should be available as prebuilds packaged by and for use with Autobuild.

Generic Layout (Single-branch, 2024 Style)

3p packages should vendor upstream contents using git submodules unless a source git repository for the source dependency is not available. A typical 3p package layout looks like this: 

.
├── [vendor submodule]/ - Git submodule containing upstream source
├── .github/            - CI/CD and repository configuration
├── autobuild.xml       - Autobuild manifest
├── build-cmd.sh        - Script used to build the repository
├── LICENSE             - Repository licensing
└── README.md           - Repository readme
Contributor Instructions

The general workflow for contributing changes to a 3p library looks like this:

  1. Fork the repository and make changes. Be sure to test your work locally.
  2. Create a PR (Be sure to follow PR Etiquette and our Code of Conduct)
  3. Have this PR reviewed and merged
  4. Done! The maintainer should cut a new version as appropriate using the workflow described below.
Maintainer Instructions
A properly versioned and promoted release of 3p-zlib-ng

If you are a 3p package maintainer you will want to follow these instructions to review and release new versions:

  1. Review PRs quickly, and merge them directly into the default branch once approved
  2. Cut a new release by using Github's Releases feature:
    1. Navigate to Repository -> Releases and click "Draft new Release"
    2. Click Choose a tag and create a new tag using the vUPSTREAM-rRELEASE format described below
    3. Click Generate release notes and edit them as appropriate. Keep the title of the release equal to the version being used.
    4. Click Publish release

After the release is published, CI/CD should be triggered by the addition of the new tag, and provide a nice set of packages and instructions for consumers to use.

Version format

3p packages should be released using the following version schema: 

Format

  vUPSTREAM-rRELEASE

  Example: v1.0.0-r2 (Second vendored release of v1.0.0 of this library)

  UPSTREAM - Upstream package version. ex. 1.0.0 or 6.7
  RELEASE - Vendored release number, ex. 1, 2
Patches

By convention, patches should be kept in a patches/* directory and applied to the upstream source during build. You can use the following utility function, declared in your build-cmd.sh script to help: 

apply_patch()
{
    local patch="$1"
    local path="$2"
    echo "Applying $patch..."
    git apply --check --reverse --directory="$path" "$patch" || git apply --directory="$path" "$patch"
}

Patch helper from 3p-boost

Differences from old style (vendor branch)

Originally, 3p libraries were maintained using a separate vendor and default branch. Repositories using this style should be migrated to new single-branch format. Benefits of using a single-branch are:

  • Simpler PR process (Make PR against default, merge it. Done.)
  • Faster release process, which means quicker integration for package consumers
  • Easier integration with GitHub functionality such as Automatic Release Notes
  • More explicit maintenance of patches makes delta from upstreams clearer and simpler to understand

List of third party libraries

Library Repository License Upstream Home Description
apr_suite 3p-apr Apache License http://apr.apache.org/ "A set of C functions which provide a portable and consistent interface into operating system - or simply widely useful - functionality. We primarily use it for threading and socket i/o. This is actually three packages, apr, apr-util and apr-iconv."
boost 3p-boost Boost License http://www.boost.org/ A set of portable C++ libraries which provide a wide set of functionality. Used primarily for tokenization.
curl 3p-curl BSD-styleB http://curl.haxx.se/libcurl/ Handles moving data across the net in many different protocols. Used to GET/POST/PUT/DELETE web resources.
expat 3p-expat MIT License https://github.com/libexpat/libexpat XML parser.
freetype 3p-freetype Freetype License and Others http://www.freetype.org/ Font engine.
glh_linear 3p-glh_linear nVidia NVParse SDK: platform-independent C++ Apple OpenGL helper library
gstreamer 3p-gstreamer Open http://gstreamer.freedesktop.org
havok Commercial
jpeglib 3p-jpeglib Open http://www.ijg.org/ JPEG decoder library.
kdu 3p-kdu-privateLL Commercial http://www.kakadusoftware.com Kakadu (KDU) JPEG-2000 decoder library.
libpng 3p-libpng Open http://www.libpng.org/pub/png/libpng.html PNG image library.
libxml2 3p-libxml2 MIT License
ndofdev 3p-libndofdev BSD-styleB Provides joystick driver support (mac and win) for the 3DConnexion SpaceNavigator.
ogg_vorbis 3p-ogvorbis BSD-styleB http://xiph.org/ogg/ Audio codec and playback.
openal 3p-openal-soft GPL http://connect.creativelabs.com/openal/default.aspx 3D Spatial audio
openjpeg 3p-openjpeg BSD-styleB http://www.openjpeg.org/ An open-source JPEG-2000 library; a slower alternative to Kadaku. Used in the open source release
openssl 3p-openssl Apache License http://www.openssl.org/ Provides encryption for sensitive actions such as user login
sdl 3p-sdl2 zlib License http://www.libsdl.org/index.php The Simple DirectMedia Layer libraries are used for handling input and basic window/GL setup on the Linux client.
slvoice Commercial This is the Vivox provided executable (renamed SLVoice)
tut 3p-tut BSD-styleB http://tut-framework.sourceforge.net/ Unit test framework based on the use of c++ meta template programming. Used to test LL libraries.
xmlrpc_epi 3p-xmlrpc-epi Epinions, Inc License http://xmlrpc-epi.sourceforge.net/ XML-RPC protocol utility
zlib-ng 3p-zlib-ng zlib License http://www.zlib.net/ zlib is a lossless data-compression library which is used for a variety of network i/o and file compression.

Redistribution

Some files in the SL installation can't be freely redistributed. This means it's not possible to simply package and offer for download a modified viewer: some files must not be redistributed.

This page documents the licenses and redistribution requirements of various files used by the viewer.

Windows

These are the files that need to be shipped with the Windows version

Runtime

Microsoft components. It's not yet known whether they're redistributable.

  • dbghelp.dll: Redistributable
  • msvcp71.dll: C++ Runtime Library, redistributable -viewer now using msvcp80.dll when compiled with VC2005
  • msvcr71.dll: C Runtime Library, Redistributable -viewer now using msvcr80.dll when compiled with VC2005

FONTS

Meta font software files

From the Readme.txt 

The Meta font software files contained in this folder are the
copyrighted property of FSI FontShop International ("FSI") and are
licensed by FSI solely for use by Linden Research, Inc. and by
residents or users of Second Life in the Second Life environment,
subject to the Second Life Terms of Service.  These Meta font software
files may not be copied by residents or developers of Second Life or
used by them for any other purpose whatsoever.
  • MtBkLfRg.ttf
  • MtBdLfRg.ttf

NOTE: These files must be replaced for an true opensource build

Kakadu

JPEG2000 library. Not redistributable. OpenJPEG is an alternative library that is redistributable

  • llkdu.dll

Starting with Viewer 2.5, llkdu.dll has been phased out. The libkdu.a and kdu.lib are still not redistributable but can be built from the toolkit provided by Kakadu under a license. How to build a viewer with such a library is documented in KDU Implementation.

OpenSSL

SSL library. Redistributable.

  • libeay32.dll
  • ssleay32.dll

Vivox

Vivox is the component that provides the voice functionality.

  • SLVoice.exe: Voice daemon. Not redistributable
  • SLVoiceAgent: Exact purpose unknown. Not redistributable
  • alut.dll: OpenAL, by Creative Labs. Redistributable.
  • ortp.dll: oRTP project, LGPL. Redistributable.
  • srtp.dll: Secure RTP?
  • vivoxsdk.dll: Vivox Communications API. Not redistributable
  • wrap_oal.dll: OpenAL32 by Creative Labs. Redistributable.

Unknown Libraries

The purpose and licensing of these files is not known:

  • tntk.dll: Probably Vivox related -see Voice/Technical for other info on ViVox files
Retrieved from "https://wiki.secondlife.com/w/index.php?title=Third_Party_Libraries&oldid=1217100"