Difference between revisions of "Third Party Libraries"
Tofu Linden (talk | contribs) m (fix xmlrpc-epi typo) |
(→FONTS: Added note on LL font change to opensource ones) |
||
(57 intermediate revisions by 20 users not shown) | |||
Line 1: | Line 1: | ||
{ | __NOTOC__ | ||
The following table lists libraries and other third party (3p) dependencies used in constructing the Second Life Viewer. | |||
|- | |||
Most of these (all the open source ones) should be available as prebuilds packaged by and for use with [[Autobuild]]. | |||
==== Generic Layout (Single-branch, 2024 Style) ==== | |||
3p packages should vendor upstream contents using [https://git-scm.com/book/en/v2/Git-Tools-Submodules git submodules] unless a source git repository for the source dependency is not available. | |||
A typical 3p package layout looks like this: | |||
<pre> | |||
. | |||
├── [vendor submodule]/ - Git submodule containing upstream source | |||
├── .github/ - CI/CD and repository configuration | |||
├── autobuild.xml - Autobuild manifest | |||
├── build-cmd.sh - Script used to build the repository | |||
├── LICENSE - Repository licensing | |||
└── README.md - Repository readme | |||
</pre> | |||
===== Contributor Instructions ===== | |||
The general workflow for contributing changes to a 3p library looks like this: | |||
# Fork the repository and make changes. Be sure to test your work locally. | |||
# Create a PR (Be sure to follow [https://gist.github.com/mikepea/863f63d6e37281e329f8 PR Etiquette] and our [https://github.com/secondlife/.github/blob/main/CODE_OF_CONDUCT.md Code of Conduct]) | |||
# Have this PR reviewed and merged | |||
# Done! The maintainer should cut a new version as appropriate using the workflow described below. | |||
===== Maintainer Instructions ===== | |||
[[File:Screenshot 2024-08-10 at 11-48-23 Releases · secondlife 3p-zlib-ng.png|thumb|top|right|A properly versioned and promoted release of 3p-zlib-ng]] | |||
If you are a 3p package maintainer you will want to follow these instructions to review and release new versions: | |||
# Review PRs quickly, and merge them directly into the default branch once approved | |||
# Cut a new release by using Github's Releases feature: | |||
## Navigate to '''Repository -> Releases''' and click '''"Draft new Release"''' | |||
## Click '''Choose a tag''' and create a new tag using the <code>vUPSTREAM-rRELEASE</code> format described below | |||
## Click '''Generate release notes''' and edit them as appropriate. Keep the title of the release equal to the version being used. | |||
## Click ''Publish release'' | |||
After the release is published, CI/CD should be triggered by the addition of the new tag, and provide a nice set of packages and instructions for consumers to use. | |||
===== Version format ===== | |||
3p packages should be released using the following version schema: | |||
<pre> | |||
Format | |||
vUPSTREAM-rRELEASE | |||
Example: v1.0.0-r2 (Second vendored release of v1.0.0 of this library) | |||
UPSTREAM - Upstream package version. ex. 1.0.0 or 6.7 | |||
RELEASE - Vendored release number, ex. 1, 2 | |||
</pre> | |||
===== Patches ===== | |||
By convention, patches should be kept in a <code>patches/*</code> directory and applied to the upstream source during build. You can use the following utility function, declared in your '''build-cmd.sh''' script to help: | |||
<syntaxhighlight lang="bash"> | |||
apply_patch() | |||
{ | |||
local patch="$1" | |||
local path="$2" | |||
echo "Applying $patch..." | |||
git apply --check --reverse --directory="$path" "$patch" || git apply --directory="$path" "$patch" | |||
} | |||
</syntaxhighlight> | |||
''Patch helper from [https://github.com/secondlife/3p-boost 3p-boost]'' | |||
===== Differences from old style (vendor branch) ===== | |||
Originally, 3p libraries were maintained using a separate '''vendor''' and default branch. Repositories using this style should be migrated to new single-branch format. Benefits of using a single-branch are: | |||
* Simpler PR process (Make PR against default, merge it. Done.) | |||
* Faster release process, which means quicker integration for package consumers | |||
* Easier integration with GitHub functionality such as Automatic Release Notes | |||
* More explicit maintenance of patches makes delta from upstreams clearer and simpler to understand | |||
=== List of third party libraries === | |||
{| border="1" cellpadding="3" | |||
|- | |- style="background:#ffdead;" | ||
! | !Library | ||
!Repository | |||
!License | |||
!Upstream Home | |||
!Description | |||
|- | |||
|- | |apr_suite | ||
|[https://github.com/secondlife/3p-apr_suite/ 3p-apr] | |||
|[https://github.com/secondlife/3p-apr_suite/blob/main/apr/LICENSE Apache License] | |||
|http://apr.apache.org/ | |||
|"A set of C functions which provide a portable and consistent interface into operating system - or simply widely useful - functionality. We primarily use it for threading and socket i/o. This is actually three packages, apr, apr-util and apr-iconv." | |||
|- | |||
|boost | |||
|[https://github.com/secondlife/3p-boost 3p-boost] | |||
|[https://github.com/boostorg/boost/blob/master/LICENSE_1_0.txt Boost License] | |||
|http://www.boost.org/ | |||
|A set of portable C++ libraries which provide a wide set of functionality. Used primarily for tokenization. | |||
|- | |||
|curl | |||
|[https://github.com/secondlife/3p-curl 3p-curl] | |||
|''[https://github.com/secondlife/3p-curl/blob/master/curl/COPYING BSD-style]''<sup>[[#BSDstyle|B]]</sup> | |||
| http://curl.haxx.se/libcurl/ | |||
|Handles moving data across the net in many different protocols. Used to GET/POST/PUT/DELETE web resources. | |Handles moving data across the net in many different protocols. Used to GET/POST/PUT/DELETE web resources. | ||
|- | |||
|expat | |||
|[https://github.com/secondlife/3p-expat 3p-expat] | |||
|[https://github.com/libexpat/libexpat/blob/master/COPYING MIT License] | |||
|https://github.com/libexpat/libexpat | |||
|XML parser. | |||
|- | |||
|freetype | |||
|[https://github.com/secondlife/3p-freetype 3p-freetype] | |||
|[https://github.com/freetype/freetype/blob/master/LICENSE.TXT Freetype License and Others] | |||
|http://www.freetype.org/ | |||
|Font engine. | |||
|- | |||
|glh_linear | |||
|[https://github.com/secondlife/3p-glh_linear 3p-glh_linear] | |||
| | |||
| | |||
|nVidia NVParse SDK: platform-independent C++ Apple OpenGL helper library | |||
|- | |||
|gstreamer | |||
|[https://bitbucket.org/lindenlab/3p-gstreamer 3p-gstreamer] | |||
|Open | |||
|http://gstreamer.freedesktop.org | |||
| | |||
|- | |||
|havok | |||
| | |||
|Commercial | |||
| | |||
| | |||
|- | |||
|jpeglib | |||
|[https://bitbucket.org/lindenlab/3p-jpeglib 3p-jpeglib] | |||
|Open | |||
|http://www.ijg.org/ | |||
|JPEG decoder library. | |||
|- | |||
|kdu | |||
|[https://hg.lindenlab.com/lindenlab/3p-kdu-private 3p-kdu-private]<sup>[[#PrivateRepo|LL]]</sup> | |||
|Commercial | |||
|http://www.kakadusoftware.com | |||
|Kakadu (KDU) JPEG-2000 decoder library. | |||
|- | |||
|libpng | |||
|[https://bitbucket.org/lindenlab/3p-libpng 3p-libpng] | |||
|Open | |||
| http://www.libpng.org/pub/png/libpng.html | |||
|PNG image library. | |||
|- | |||
|libxml2 | |||
|[https://github.com/secondlife/3p-libxml2 3p-libxml2] | |||
|[https://gitlab.gnome.org/GNOME/libxml2/-/blob/master/Copyright?ref_type=heads MIT License] | |||
| | |||
| | |||
|- | |||
|ndofdev | |||
|[https://github.com/secondlife/3p-libndofdev 3p-libndofdev] | |||
|''[https://github.com/secondlife/3p-libndofdev/blob/master/libndofdev/COPYING BSD-style]''<sup>[[#BSDstyle|B]]</sup> | |||
| | |||
|Provides joystick driver support (mac and win) for the 3DConnexion SpaceNavigator. | |||
|- | |||
|ogg_vorbis | |||
|[https://github.com/secondlife/3p-ogg_vorbis 3p-ogvorbis] | |||
|''[https://github.com/xiph/ogg/blob/master/COPYING BSD-style]''<sup>[[#BSDstyle|B]]</sup> | |||
|http://xiph.org/ogg/ | |||
|Audio codec and playback. | |||
|- | |||
|openal | |||
|[https://github.com/secondlife/3p-openal-soft 3p-openal-soft] | |||
|[https://github.com/secondlife/3p-openal-soft/blob/main/openal-soft/COPYING GPL] | |||
|http://connect.creativelabs.com/openal/default.aspx | |||
|3D Spatial audio | |||
|- | |||
|openjpeg | |||
|[https://github.com/secondlife/3p-openjpeg 3p-openjpeg] | |||
|''[https://github.com/uclouvain/openjpeg/blob/master/LICENSE BSD-style]''<sup>[[#BSDstyle|B]]</sup> | |||
|http://www.openjpeg.org/ | |||
|An open-source JPEG-2000 library; a slower alternative to Kadaku. Used in the open source release | |||
|- | |||
|openssl | |||
|[https://github.com/secondlife/3p-openssl 3p-openssl] | |||
|[https://github.com/openssl/openssl/blob/master/LICENSE.txt Apache License] | |||
|http://www.openssl.org/ | |||
|Provides encryption for sensitive actions such as user login | |||
|- | |||
|sdl | |||
|[https://github.com/secondlife/3p-sdl2 3p-sdl2] | |||
|[https://github.com/libsdl-org/SDL/blob/main/LICENSE.txt zlib License] | |||
|http://www.libsdl.org/index.php | |||
|The Simple DirectMedia Layer libraries are used for handling input and basic window/GL setup on the Linux client. | |||
|- | |||
|slvoice | |||
| | |||
|Commercial | |||
| | |||
|This is the Vivox provided executable (renamed SLVoice) | |||
|- | |||
|tut | |||
|[https://github.com/secondlife/3p-tut 3p-tut] | |||
|''[https://github.com/secondlife/3p-tut/blob/master/LICENSE.txt BSD-style]''<sup>[[#BSDstyle|B]]</sup> | |||
|http://tut-framework.sourceforge.net/ | |||
|Unit test framework based on the use of c++ meta template programming. Used to test LL libraries. | |||
|- | |||
|xmlrpc_epi | |||
|[https://github.com/secondlife/3p-xmlrpc-epi 3p-xmlrpc-epi] | |||
|[https://github.com/secondlife/3p-xmlrpc-epi/blob/master/xmlrpc-epi/COPYING Epinions, Inc License] | |||
|http://xmlrpc-epi.sourceforge.net/ | |||
|XML-RPC protocol utility | |||
|- | |||
|zlib-ng | |||
|[https://github.com/secondlife/3p-zlib-ng 3p-zlib-ng] | |||
|[https://github.com/zlib-ng/zlib-ng/blob/develop/LICENSE.md zlib License] | |||
|http://www.zlib.net/ | |||
|zlib is a lossless data-compression library which is used for a variety of network i/o and file compression. | |||
|} | |||
= Redistribution = | |||
Some files in the SL installation can't be freely redistributed. This means it's not possible to simply package and offer for download a modified viewer: some files '''must not''' be redistributed. | |||
This page documents the licenses and redistribution requirements of various files used by the viewer. | |||
== Windows == | |||
These are the files that need to be shipped with the Windows version | |||
=== Runtime === | |||
Microsoft components. It's not yet known whether they're redistributable. | |||
* dbghelp.dll: [http://msdn2.microsoft.com/EN-US/library/ms679294.aspx Redistributable] | |||
* msvcp71.dll: C++ Runtime Library, redistributable -viewer now using msvcp80.dll when compiled with VC2005 | |||
* msvcr71.dll: C Runtime Library, [http://msdn2.microsoft.com/en-us/library/abx4dbyh(vs.71).aspx Redistributable] -viewer now using msvcr80.dll when compiled with VC2005 | |||
== FONTS == | |||
Meta font software files | |||
From the Readme.txt | |||
The Meta font software files contained in this folder are the | |||
copyrighted property of FSI FontShop International ("FSI") and are | |||
licensed by FSI solely for use by Linden Research, Inc. and by | |||
residents or users of Second Life in the Second Life environment, | |||
subject to the Second Life Terms of Service. These Meta font software | |||
files may not be copied by residents or developers of Second Life or | |||
used by them for any other purpose whatsoever. | |||
* MtBkLfRg.ttf | |||
* MtBdLfRg.ttf | |||
'''NOTE 1:''' These files must be replaced for an true open source build. | |||
'''NOTE 2:''' At least starting with Viewer 6, Linden Lab has replaced the 'Meta' font with Bitstream's [https://dejavu-fonts.github.io/ DejaVu], which is released under a very permissible license; Viewer 7 additionally added {{code|TwemojiSVG.ttf}}, which is presumably the (huge) [https://github.com/13rac1/twemoji-color-font Twitter Unicode emoji color OpenType-SVG font for Linux/MacOS/Windows] (where SVG images are encoded as if they were font glyphs), released under a Creative Commons 4.0 Attrbution license, and which, in turn, is based on the Twitter Emoji font, also released under the same license. | |||
| | |||
AFAIK, the only viewer which still contains the two Meta fonts is the Cool VL Viewer — since it still pulls some code from the original SL Viewer 1 — but these are never used on the compiled application itself. | |||
=== Kakadu === | |||
JPEG2000 library. '''Not redistributable'''. | |||
OpenJPEG is an alternative library that is redistributable | |||
* llkdu.dll | |||
Starting with Viewer 2.5, llkdu.dll has been phased out. The libkdu.a and kdu.lib are still not redistributable but can be built from the toolkit provided by Kakadu under a license. How to build a viewer with such a library is documented in [[KDU Implementation]]. | |||
=== OpenSSL === | |||
SSL library. Redistributable. | |||
* libeay32.dll | |||
* ssleay32.dll | |||
=== Vivox === | |||
Vivox is the component that provides the voice functionality. | |||
* SLVoice.exe: Voice daemon. '''Not redistributable''' | |||
* SLVoiceAgent: Exact purpose unknown. '''Not redistributable''' | |||
* alut.dll: OpenAL, by Creative Labs. Redistributable. | |||
* ortp.dll: oRTP project, LGPL. Redistributable. | |||
* srtp.dll: Secure RTP? | |||
* vivoxsdk.dll: Vivox Communications API. '''Not redistributable''' | |||
* wrap_oal.dll: OpenAL32 by Creative Labs. Redistributable. | |||
=== Unknown Libraries === | |||
The purpose and licensing of these files is not known: | |||
* tntk.dll: Probably Vivox related -see [https://wiki.secondlife.com/wiki/Voice/Technical Voice/Technical] for other info on ViVox files | |||
[[Category:Compiling viewer]] |
Latest revision as of 10:54, 28 September 2024
The following table lists libraries and other third party (3p) dependencies used in constructing the Second Life Viewer.
Most of these (all the open source ones) should be available as prebuilds packaged by and for use with Autobuild.
Generic Layout (Single-branch, 2024 Style)
3p packages should vendor upstream contents using git submodules unless a source git repository for the source dependency is not available. A typical 3p package layout looks like this:
. ├── [vendor submodule]/ - Git submodule containing upstream source ├── .github/ - CI/CD and repository configuration ├── autobuild.xml - Autobuild manifest ├── build-cmd.sh - Script used to build the repository ├── LICENSE - Repository licensing └── README.md - Repository readme
Contributor Instructions
The general workflow for contributing changes to a 3p library looks like this:
- Fork the repository and make changes. Be sure to test your work locally.
- Create a PR (Be sure to follow PR Etiquette and our Code of Conduct)
- Have this PR reviewed and merged
- Done! The maintainer should cut a new version as appropriate using the workflow described below.
Maintainer Instructions
If you are a 3p package maintainer you will want to follow these instructions to review and release new versions:
- Review PRs quickly, and merge them directly into the default branch once approved
- Cut a new release by using Github's Releases feature:
- Navigate to Repository -> Releases and click "Draft new Release"
- Click Choose a tag and create a new tag using the
vUPSTREAM-rRELEASE
format described below - Click Generate release notes and edit them as appropriate. Keep the title of the release equal to the version being used.
- Click Publish release
After the release is published, CI/CD should be triggered by the addition of the new tag, and provide a nice set of packages and instructions for consumers to use.
Version format
3p packages should be released using the following version schema:
Format vUPSTREAM-rRELEASE Example: v1.0.0-r2 (Second vendored release of v1.0.0 of this library) UPSTREAM - Upstream package version. ex. 1.0.0 or 6.7 RELEASE - Vendored release number, ex. 1, 2
Patches
By convention, patches should be kept in a patches/*
directory and applied to the upstream source during build. You can use the following utility function, declared in your build-cmd.sh script to help:
apply_patch()
{
local patch="$1"
local path="$2"
echo "Applying $patch..."
git apply --check --reverse --directory="$path" "$patch" || git apply --directory="$path" "$patch"
}
Patch helper from 3p-boost
Differences from old style (vendor branch)
Originally, 3p libraries were maintained using a separate vendor and default branch. Repositories using this style should be migrated to new single-branch format. Benefits of using a single-branch are:
- Simpler PR process (Make PR against default, merge it. Done.)
- Faster release process, which means quicker integration for package consumers
- Easier integration with GitHub functionality such as Automatic Release Notes
- More explicit maintenance of patches makes delta from upstreams clearer and simpler to understand
List of third party libraries
Library | Repository | License | Upstream Home | Description |
---|---|---|---|---|
apr_suite | 3p-apr | Apache License | http://apr.apache.org/ | "A set of C functions which provide a portable and consistent interface into operating system - or simply widely useful - functionality. We primarily use it for threading and socket i/o. This is actually three packages, apr, apr-util and apr-iconv." |
boost | 3p-boost | Boost License | http://www.boost.org/ | A set of portable C++ libraries which provide a wide set of functionality. Used primarily for tokenization. |
curl | 3p-curl | BSD-styleB | http://curl.haxx.se/libcurl/ | Handles moving data across the net in many different protocols. Used to GET/POST/PUT/DELETE web resources. |
expat | 3p-expat | MIT License | https://github.com/libexpat/libexpat | XML parser. |
freetype | 3p-freetype | Freetype License and Others | http://www.freetype.org/ | Font engine. |
glh_linear | 3p-glh_linear | nVidia NVParse SDK: platform-independent C++ Apple OpenGL helper library | ||
gstreamer | 3p-gstreamer | Open | http://gstreamer.freedesktop.org | |
havok | Commercial | |||
jpeglib | 3p-jpeglib | Open | http://www.ijg.org/ | JPEG decoder library. |
kdu | 3p-kdu-privateLL | Commercial | http://www.kakadusoftware.com | Kakadu (KDU) JPEG-2000 decoder library. |
libpng | 3p-libpng | Open | http://www.libpng.org/pub/png/libpng.html | PNG image library. |
libxml2 | 3p-libxml2 | MIT License | ||
ndofdev | 3p-libndofdev | BSD-styleB | Provides joystick driver support (mac and win) for the 3DConnexion SpaceNavigator. | |
ogg_vorbis | 3p-ogvorbis | BSD-styleB | http://xiph.org/ogg/ | Audio codec and playback. |
openal | 3p-openal-soft | GPL | http://connect.creativelabs.com/openal/default.aspx | 3D Spatial audio |
openjpeg | 3p-openjpeg | BSD-styleB | http://www.openjpeg.org/ | An open-source JPEG-2000 library; a slower alternative to Kadaku. Used in the open source release |
openssl | 3p-openssl | Apache License | http://www.openssl.org/ | Provides encryption for sensitive actions such as user login |
sdl | 3p-sdl2 | zlib License | http://www.libsdl.org/index.php | The Simple DirectMedia Layer libraries are used for handling input and basic window/GL setup on the Linux client. |
slvoice | Commercial | This is the Vivox provided executable (renamed SLVoice) | ||
tut | 3p-tut | BSD-styleB | http://tut-framework.sourceforge.net/ | Unit test framework based on the use of c++ meta template programming. Used to test LL libraries. |
xmlrpc_epi | 3p-xmlrpc-epi | Epinions, Inc License | http://xmlrpc-epi.sourceforge.net/ | XML-RPC protocol utility |
zlib-ng | 3p-zlib-ng | zlib License | http://www.zlib.net/ | zlib is a lossless data-compression library which is used for a variety of network i/o and file compression. |
Redistribution
Some files in the SL installation can't be freely redistributed. This means it's not possible to simply package and offer for download a modified viewer: some files must not be redistributed.
This page documents the licenses and redistribution requirements of various files used by the viewer.
Windows
These are the files that need to be shipped with the Windows version
Runtime
Microsoft components. It's not yet known whether they're redistributable.
- dbghelp.dll: Redistributable
- msvcp71.dll: C++ Runtime Library, redistributable -viewer now using msvcp80.dll when compiled with VC2005
- msvcr71.dll: C Runtime Library, Redistributable -viewer now using msvcr80.dll when compiled with VC2005
FONTS
Meta font software files
From the Readme.txt
The Meta font software files contained in this folder are the copyrighted property of FSI FontShop International ("FSI") and are licensed by FSI solely for use by Linden Research, Inc. and by residents or users of Second Life in the Second Life environment, subject to the Second Life Terms of Service. These Meta font software files may not be copied by residents or developers of Second Life or used by them for any other purpose whatsoever.
- MtBkLfRg.ttf
- MtBdLfRg.ttf
NOTE 1: These files must be replaced for an true open source build.
NOTE 2: At least starting with Viewer 6, Linden Lab has replaced the 'Meta' font with Bitstream's DejaVu, which is released under a very permissible license; Viewer 7 additionally added TwemojiSVG.ttf
, which is presumably the (huge) Twitter Unicode emoji color OpenType-SVG font for Linux/MacOS/Windows (where SVG images are encoded as if they were font glyphs), released under a Creative Commons 4.0 Attrbution license, and which, in turn, is based on the Twitter Emoji font, also released under the same license.
AFAIK, the only viewer which still contains the two Meta fonts is the Cool VL Viewer — since it still pulls some code from the original SL Viewer 1 — but these are never used on the compiled application itself.
Kakadu
JPEG2000 library. Not redistributable. OpenJPEG is an alternative library that is redistributable
- llkdu.dll
Starting with Viewer 2.5, llkdu.dll has been phased out. The libkdu.a and kdu.lib are still not redistributable but can be built from the toolkit provided by Kakadu under a license. How to build a viewer with such a library is documented in KDU Implementation.
OpenSSL
SSL library. Redistributable.
- libeay32.dll
- ssleay32.dll
Vivox
Vivox is the component that provides the voice functionality.
- SLVoice.exe: Voice daemon. Not redistributable
- SLVoiceAgent: Exact purpose unknown. Not redistributable
- alut.dll: OpenAL, by Creative Labs. Redistributable.
- ortp.dll: oRTP project, LGPL. Redistributable.
- srtp.dll: Secure RTP?
- vivoxsdk.dll: Vivox Communications API. Not redistributable
- wrap_oal.dll: OpenAL32 by Creative Labs. Redistributable.
Unknown Libraries
The purpose and licensing of these files is not known:
- tntk.dll: Probably Vivox related -see Voice/Technical for other info on ViVox files