Difference between revisions of "Voice/Technical"

Revision as of 02:45, 16 March 2009

Voice in Second Life uses Vivox technology that is based on RTP (using the oRTP library), SIP (using the amsip library from Antisip), OpenAL, TinyXPath, OpenSSL, and libcurl for the transmission of voice data. The positional 3D audio is achieved with technology from DiamondWare. These technologies are contained in external daemon software that is started and stopped by the Second Life client.

You can find the official ViVox documentation on the JIRA.

The SecondLife viewer handles configuration, control, and display functions, but the voice streams (from the microphone and from the Vivox voice server) do not enter the viewer. Unfortunately the source code for the external SLVoice daemon is not available due to contractual obligations between Linden Lab and Vivox.

Executables

The executables can be found in the Program Files\SecondLife directory of the runtime and in the linden\indra\newview\vivox-runtime\(platform) directory of the source distribution.

SLVoice.exe
- Thin wrapper around the Vivox SDK
- Launched when voice is enabled in SL
- Runs a TCP server that listens and replies to commands and queries using an XML format
SLVoiceAgent.exe
- Actually called the DiamondWare Thin Voice Client
- Launched by SLVoice, but don't know under what conditions
- Unclear how this fits in the communication chain
- Not launched for normal proximity voice chat; don't know about group chat or P2P yet
vivoxsdk.dll
- The Vivox SDK
- Used by SLVoice.exe
- Uses a solution from Antisip for SIP, RTP, and G.711
- Uses OpenAL for playing and recording audio
- Additionally implements the G.722.1/C (aka Siren14) codec
ortp.dll
- Real-time Transport Protocol (RFC3550) stack under LGPL
- Used by vivoxsdk.dll
- Originally from Linphone
- Distributed as part of amsip by Antisip
wrap_oal.dll
- OpenAL, a cross-platform 3D audio API
- Used by vivoxsdk.dll
alut.dll
- The OpenAL Utility Toolkit (ALUT)
- Used by vivoxsdk.dll
ssleay32.dll and libeay32
- From the OpenSSL Project, a collaborative effort to develop a robust, commercial-grade, full-featured, and Open Source toolkit implementing the Secure Sockets Layer (SSL v2/v3) and Transport Layer Security (TLS v1) protocols as well as a full-strength general purpose cryptography library.
- Used by SLVoiceAgent.dll
tntk.dll and srtp.dll
- Part of the DiamondWare distribution
- Apparently not used

SLVoice parameters

SLVoice is started from the viewer in SLVoiceClient::stateMachine() using:

SLVoice.exe -p tcp -h -c -ll <VivoxDebugLevel, default -1>

usage: SLVoice [flags]

valid flags are:

[-ll <log-level>] 0-5, or -1 for no logging

[-lf <log-folder>]

[-lp <log-file prefix>]

[-ls <log-file suffix>]

[-p tcp]

[-i <address>] application sends requests to this address

[-o <address>] according to the executable, the gateway sends responses and events to this address, but according to the official documentation, the gateway both sends and receives to/from this address. Please update this if you determine which is correct.

[-vo] vivox only functionality

[-ov] (undocumented and unused)

[-c] cleanup orphaned SLVoice processes. If this is the only argument to SLVoice, then it cleans up and exits; if there are other arguments, it cleans up any old instances and launches a new one.

[-h] hide the window associated with SLVoice. On MS Windows, this appears to be the default, because there is no window whether this parameter is set ot not.

[-r] (undocumented and unused)

SLVoice.exe was called vivox-gw.exe in the beta release, so you may still see some references to the old name.

IP Addresses

By default, the SecondLife viewer uses IP address 127.0.0.1 (localhost), port 44124 to communicate with SLVoice, the voice client gateway.

It is possible to connect to a client gateway running on another host. This is useful for testing. According to comments in the source code: to do this, launch the gateway on a nearby host,

SLVoice.exe -p tcp -i 0.0.0.0:44124

and modify the Settings for VoiceHost and VoicePort to set the host IP address.

The following ports are used for external communication.

Port 21002 - TCP - for voice control signals
Ports 12000-13000 - UDP - for voice media
Port 80/443 - TCP - for Web server
Ports 5060 or 5062 - UDP - for voice control signals

If the SIP connection on the default port 5062 fails (as it might if the router is designed to provide VoIP services of its own) the viewer will automatically retry and fall back on port 5060. This means that port 5060 may need to be opened on some restrictive firewalls in rare instances.

Logging

The logging for SLVoice is controlled by a viewer variable, VivoxDebugLevel. You can set it directly in your Documents and Settings\<user>\Application Data\SecondLife\user_settings\settings.xml file

or you can set it from the viewer. To set it from the viewer, you have to enable the advanced menu if you haven't already using Ctrl-Alt-⇧ Shift-D (not to be confused with Ctrl-Alt-Del). Then select

Advanced -> Debug Settings

to get the Debug Settings window. In the selection box, hit the triangle to bring up the list of debug variables and select VivoxDebugLevel. In the text entry box, you'll see the current value. Above it is the description, "Logging level to use when launching the Vivox daemon." You can set the level as you like. If you want the output to go somewhere other than the default location, you can enter it in the box as well, for example,

5 -lf myDirectory -lp myPrefix -ls .txt

If <log-folder> is a relative path, it is relative to the directory that SecondLife is running in, ususally C:\Program Files\SecondLife

The default folder is c:/Documents and Settings/<user>/Local Settings/Application Data/Vivox/logs. The default prefix is SLVoice and the default suffix is .txt.

The log file is named <log-file prefix>-<yyyy>-<mm>-<dd><log-file suffix>

The default on September 3, 2007, for example, would be: SLVoice-2007-09-03.txt

The <log-level> must be at least 2 to see INFO messages and 4 to see TRACE messages.

Whenever log level is other than "-1", there is also a log file written to LL_PATH_LOGS, which is usually set to c:/Documents and Settings/<user>/Application Data/SecondLife/logs, with the <log-file prefix> "Connector," that contains log information for the Connector. This logging comes from vivoxsdk.dll which is used by SLVoice.exe. The content does not depend on the log level for SLVoice; the Connector.Create.1 request hardcodes the Connector log level at 10.

There is apparently no Response sent to the viewer for Session.Set3DPosition.1
SLVoice does not log ParticipantPropertiesEvent
SLVoice does not log Account.Logout.1
SLVoice may not log the Response to Connector.InitiateShutdown.1 but when it does it tends to spew tens of thousands (about ten per millisecond) of messages to the log.

17:11:38.988, 0x00001414, INFO , disconnectHandler Client Connection Closed - shutting down

17:11:38.988, 0x00001414, INFO , shutdownGateway Terminating Gateway Process

SLVoice logs commands with added default values for fields not sent by the viewer, for example,

for Account.Login.1 the log includes fields

<EnableBuddiesAndPresence>false</EnableBuddiesAndPresence>

for Connector.Create.1

<AttemptStun>AttemptStunUnspecified</AttemptStun>

for Session.Create.1

<JoinText>false</JoinText>

<PasswordHashAlgorithm>ClearText</PasswordHashAlgorithm>

for Session.Set3DPosition.1

<OrientationType>legacy</OrientationType>

Message Protocol

<---- is a request from the viewer to SLVoice

----> is a response from SLVoice to the viewer

====> is an event notice from SLVoice to the viewer

Login

<---- Aux.GetCaptureDevices.1
<---- Aux.GetRenderDevices.1
<---- Connector.Create.1
----> Aux.GetCaptureDevices.1
----> Aux.GetRenderDevices.1
----> Connector.Create.1
<---- Account.Login.1
----> Account.Login.1
====> LoginStateChangeEvent
<---- Session.Create.1
----> Session.Create.1
====> SessionStateChangeEvent
====> ParticipantStateChangeEvent
====> ParticipantPropertiesEvent
<---- Session.Set3DPosition.1
<---- Connector.MuteLocalMic.1
<---- Connector.MuteLocalSpeaker.1
<---- Connector.SetLocalSpeakerVolume.1
<---- Connector.SetLocalMicVolume.1
<---- Aux.SetCaptureDevice.1
<---- Aux.SetRenderDevice.1
====> ParticipantPropertiesEvent
----> Connector.MuteLocalMic.1
----> Connector.MuteLocalSpeaker.1
----> Connector.SetLocalSpeakerVolume.1
----> Connector.SetLocalMicVolume.1
----> Aux.SetCaptureDevice.1
----> Aux.SetRenderDevice.1

If someone else is already there

====> ParticipantStateChangeEvent
====> ParticipantPropertiesEvent

for each participant

Movement

<---- Session.Set3DPosition.1

many times per second

and if anyone is there

====> ParticipantStateChangeEvent
====> ParticipantPropertiesEvent

for each participant

many times per second

Mute your microphone

<---- Connector.MuteLocalMic.1
----> Connector.MuteLocalMic.1

Adjust the volume of your microphone

<---- Connector.SetLocalSpeakerVolume.1
----> Connector.SetLocalSpeakerVolume.1

many times per second

Mute or adjust participant volume

This first command is internal to SLVoice

<.... Session.SetParticipantMuteForMe.1
<---- Session.SetParticipantVolumeForMe.1
----> Session.SetParticipantVolumeForMe.1

many times per second

Audio Wizard

<---- Aux.GetCaptureDevices.1
<---- Aux.GetRenderDevices.1
----> Aux.GetCaptureDevices.1
<---- Session.Terminate.1
----> Aux.GetRenderDevices.1
====> SessionStateChangeEvent
----> Session.Terminate.1
====> SessionMediaEvent
====> SessionMediaEvent
<---- Aux.CaptureAudioStart.1
----> Aux.CaptureAudioStart.1
<---- Aux.SetMicLevel.1
<---- Aux.SetSpeakerLevel.1
----> Aux.SetMicLevel.1
----> Aux.SetSpeakerLevel.1
====> AuxAudioPropertiesEvent

many times per second

<---- Aux.CaptureAudioStop.1
----> Aux.CaptureAudioStop.1
====> AuxAudioPropertiesEvent
<---- Session.Create.1
----> Session.Create.1
====> SessionStateChangeEvent
====> ParticipantStateChangeEvent
====> ParticipantPropertiesEvent
<---- Connector.MuteLocalMic.1
<---- Connector.SetLocalSpeakerVolume.1
----> Connector.MuteLocalMic.1
----> Connector.SetLocalSpeakerVolume.1
====> ParticipantStateChangeEvent
====> ParticipantPropertiesEvent
<---- Aux.SetRenderDevice.1
----> Aux.SetRenderDevice.1

Logout

<---- Session.Terminate.1
====> SessionStateChangeEvent
----> Session.Terminate.1
<---- Account.Logout.1
<---- Connector.InitiateShutdown.1

We usually disappear before getting the rest

----> Account.Logout.1
====> LoginStateChangeEvent
----> Connector.InitiateShutdown.1

Message Format

Events

These are examples of the Vivox SDK event messages sent to the SecondLife viewer.

AuxAudioPropertiesEvent

</Event>

LoginStateChangeEvent

<AccountHandle>c1_m1000abcdefghijklmnopqrstuvwxy</AccountHandle>

</Event>

ParticipantPropertiesEvent

<SessionHandle>c1_m1000abcdefghijklmnopqrstuvwxy2</SessionHandle>

<ParticipantURI>aBcDeFgHiJkLmNoPqRsTuVwXy</ParticipantURI>

<IsLocallyMuted>false</IsLocallyMuted>

<IsModeratorMuted>false</IsModeratorMuted>

</Event>

ParticipantStateChangeEvent

<SessionHandle>c1_m1000abcdefghijklmnopqrstuvwxy2</SessionHandle>

<StatusString>'</StatusString>

<ParticipantURI>aBcDeFgHiJkLmNoPqRsTuVwXy</ParticipantURI>

<AccountName>aBcDeFgHiJkLmNoPqRsTuVwXy</AccountName>

</Event>

SessionMediaEvent

Ignored by the viewer

<SessionHandle>c1_m1000abcdefghijklmnopqrstuvwxy1</SessionHandle>

<HasText>false</HasText>

<HasAudio>false</HasAudio>

<HasVideo>false</HasVideo>

</Event>

SessionNewEvent

Used by p2p

SessionStateChangeEvent

<SessionHandle>c1_m1000abcdefghijklmnopqrstuvwxy0</SessionHandle>

<URI>sip:confctl-505189@bhr.vivox.com</URI>

</Event>

Actions

These are examples of the Vivox SDK commands used by the SecondLife viewer.

The requestId must be unique for each request, but other than that there doesn't appear to be any restrictions on its value.

Each request is followed by three newline characters "\n\n\n" to indicate the end of the message.

Account.ChannelGetList.1

Coded in llvoiceclient.cpp but not used

Account.Login.1

AudioSessionAnswerMode may be either "AutoAnswer" or "VerifyAnswer" but the viewer always uses "VerifyAnswer."

The ParticipantPropertyFrequency sets the rate at which ParticipantPropertiesEvent events are sent to the viewer, but its value is quite counterintuitive. According to the official ViVox documentation, the valid values and their interpretations are:

0 to disable event messages
5 to get 10 messages per second
10 to get 5 messages per second
50 to get one message per second
100 to get an event message only when the participant starts/stops talking or is muted/unmuted. According to the official documentation, this is the default, but the viewer does not set it and it gets a response indicating that it is set to 10 (5 per sec).

<AccountName>abcdefghijklmnopqrstuvwxy</AccountName>

<AccountPassword>ABCDEFGHIJKLMNOPQRSTUVW</AccountPassword>

<AudioSessionAnswerMode>VerifyAnswer</AudioSessionAnswerMode>

<EnableBuddiesAndPresence>false</EnableBuddiesAndPresence>

</Request>

<AccountHandle>c1_m1000abcdefghijklmnopqrstuvwxy</AccountHandle>

</Results>

<AccountName>abcdefghijklmnopqrstuvwxy<AccountName>

<AccountPassword>ABCDEFGHIJKLMNOPQRSTUVW</AccountPassword>

<AudioSessionAnswerMode>VerifyAnswer</AudioSessionAnswerMode>

<EnableBuddiesAndPresence>false</EnableBuddiesAndPresence>

</Request>

</InputXml>

</Response>

Account.Logout.1

<AccountHandle>c1_m1000abcdefghijklmnopqrstuvwxy</AccountHandle>

</Request>

The viewer doesn't get the response before the Connector shuts down

</Results>

<AccountHandle>c1_m1000abcdefghijklmnopqrstuvwxy</AccountHandle>

</Request>

</InputXml>

</Response>

Aux.CaptureAudioStart.1

</Request>

</Results>

</Request>

</InputXml>

</Response>

Aux.CaptureAudioStop.1

</Results>

</InputXml>

</Response>

Aux.GetCaptureDevices.1

<Device>Realtek HD Front Pink Jack</Device>

</CaptureDevice>

<Device>Realtek HD Digital input</Device>

</CaptureDevice>

<Device>Realtek HD Front Green Jack</Device>

</CaptureDevice>

<Device>Back Line in/Mic, Front Line in</Device>

</CaptureDevice>

</CaptureDevices>

<Device>Realtek HD Front Pink Jack</Device>

</CurrentCaptureDevice>

</Results>

</InputXml>

</Response>

Aux.GetRenderDevices.1

<Device>Realtek HD Audio rear output</Device>

</RenderDevice>

</RenderDevices>

<Device>Realtek HD Audio rear output</Device>

</CurrentRenderDevice>

</Results>

</InputXml>

</Response>

Aux.RenderAudioStart.1

Coded in llvoiceclient.cpp but not used

Aux.RenderAudioStop.1

Coded in llvoiceclient.cpp but not used

Aux.SetCaptureDevice.1

<CaptureDeviceSpecifier>Realtek HD Front Green Jack</CaptureDeviceSpecifier>

</Request>

</Results>

<CaptureDeviceSpecifier>Realtek HD Front Green Jack</CaptureDeviceSpecifier>

</Request>

</InputXml>

</Response>

Aux.SetMicLevel.1

The range for Value is -100 to 100. This only sets the microphone level while running the audio wizard. A Connector.SetLocalMicVolume.1 must be used to set the level for chat.

</Request>

</Results>

</Request>

</InputXml>

</Response>

Aux.SetRenderDevice.1

<RenderDeviceSpecifier>Realtek HD Audio rear output</RenderDeviceSpecifier>

</Request>

</Results>

<RenderDeviceSpecifier>Realtek HD Audio rear output</RenderDeviceSpecifier>

</Request>

</InputXml>

</Response>

Aux.SetSpeakerLevel.1

The range for Value is -100 to 100. This only sets the speaker level while running the audio wizard. A Connector.SetLocalSpeakerVolume.1 must be used to set the level for chat.

</Request>

</Results>

</Request>

</InputXml>

</Response>

Connector.Create.1

<AttemptStun>AttemptStunUnspecified</AttemptStun>

<AccountManagementServer>https://www.bhr.vivox.com/api2/</AccountManagementServer>

<Enabled>false</Enabled>

<Folder>C:\Documents and Settings\HP_Administrator\Application Data\SecondLife\logs</Folder>

<FileNamePrefix>Connector</FileNamePrefix>

</Logging>

</Request>

</Results>

<AttemptStun>AttemptStunUnspecified</AttemptStun>

<AccountManagementServer>https://www.bhr.vivox.com/api2/</AccountManagementServer>

<Enabled>false</Enabled>

<Folder>C:\Documents and Settings\HP_Administrator\Application Data\SecondLife\logs</Folder>

<FileNamePrefix>Connector</FileNamePrefix>

</Logging>

</Request>

</InputXml>

</Response>

Connector.InitiateShutdown.1

</Request>

The viewer doesn't wait for a response

</Results>

</Request>

</InputXml>

</Response>

Connector.MuteLocalMic.1

</Request>

</Results>

</Request>

</InputXml>

</Response>

Connector.MuteLocalSpeaker.1

Used only if AudioLevelVoice is 0 at startup

</Request>

</Results>

</Request>

</InputXml>

</Response>

Connector.SetLocalMicVolume.1

The range for Value is -100 to 100.

</Request>

</Results>

</Request>

</InputXml>

</Response>

Connector.SetLocalSpeakerVolume.1

The range for Value is -100 to 100.

</Request>

</Results>

</Request>

</InputXml>

</Response>

Session.Connect.1

Used by p2p

Session.Create.1

<AccountHandle>c1_m1000abcdefghijklmnopqrstuvwxy</AccountHandle>

<URI>sip:confctl-505189@bhr.vivox.com</URI>

<JoinText>false</JoinText>

<PasswordHashAlgorithm>ClearText</PasswordHashAlgorithm>

</Request>

<SessionHandle>c1_m1000abcdefghijklmnopqrstuvwxy0</SessionHandle>

</Results>

<AccountHandle>c1_m1000abcdefghijklmnopqrstuvwxy</AccountHandle>

<URI>sip:confctl-505189@bhr.vivox.com</URI>

<JoinText>false</JoinText>

<PasswordHashAlgorithm>ClearText</PasswordHashAlgorithm>

</Request>

</InputXml>

</Response>

Session.Set3DPosition.1

When sending a 3D position for speaker and listener, the "listener" is either the avatar position or the camera position. That is, it describes the speaking position and the listening position for this avatar.

<SessionHandle>c1_m1000abcdefghijklmnopqrstuvwxy1</SessionHandle>

<OrientationType>legacy</OrientationType>

</Position>

</Velocity>

</AtOrientation>

</UpOrientation>

</LeftOrientation>

</SpeakerPosition>

</Position>

</Velocity>

</AtOrientation>

</UpOrientation>

</LeftOrientation>

</ListenerPosition>

</Request>

The viewer doesn't get a response

Session.SetParticipantMuteForMe.1

This message is additionally processed internally by SLVoice whenever a Session.SetParticipantVolumeForMe.1 is sent from the viewer.

<SessionHandle>c1_m1000abcdefghijklmnopqrstuvwxy1</SessionHandle>

<ParticipantURI>sip:aBcDeFgHiJkLmNoPqRsTuVwXy@bhr.vivox.com</ParticipantURI>

</Request>

No response is sent to the viewer.

Session.SetParticipantVolumeForMe.1

Used by IM panel and Active Speakers volume control

<SessionHandle>c1_m1000abcdefghijklmnopqrstuvwxy1</SessionHandle>

<ParticipantURI>sip:aBcDeFgHiJkLmNoPqRsTuVwXy@bhr.vivox.com</ParticipantURI>

</Request>

</Results>

<SessionHandle>c1_m1000abcdefghijklmnopqrstuvwxy1</SessionHandle>

<ParticipantURI>sip:aBcDeFgHiJkLmNoPqRsTuVwXy@bhr.vivox.com</ParticipantURI>

</Request>

</InputXml>

</Response>

Session.Terminate.1

<SessionHandle>c1_m1000abcdefghijklmnopqrstuvwxy0</SessionHandle>

</Request>

</Results>

<SessionHandle>c1_m1000abcdefghijklmnopqrstuvwxy0</SessionHandle>

</Request>

</InputXml>

</Response>

@@ Line 97: / Line 97: @@
 or you can set it from the viewer.  To set it from the viewer, you have to enable the [[advanced menu]] if you haven't already using {{KeyCombo|ctrl=*|alt=*|shift=*|D}} (not to be confused with {{KeyCombo|ctrl=*|alt=*|Del}}).  Then select
-: Client -> Debug Settings
+: Advanced -> Debug Settings
 to get the Debug Settings window.  In the selection box, hit the triangle to bring up the list of debug variables and select VivoxDebugLevel.  In the text entry box, you'll see the current value.  Above it is the description, "Logging level to use when launching the Vivox daemon."  You can set the level as you like.  If you want the output to go somewhere other than the default location, you can enter it in the box as well, for example,

Difference between revisions of "Voice/Technical"

Revision as of 02:45, 16 March 2009

Executables

SLVoice parameters

IP Addresses

Logging

Message Protocol

Login

Movement

Teleporting

Participant enters the area

Participant leaves the area

Speech

Mute your microphone

Adjust the volume of your microphone

Mute or adjust participant volume

Audio Wizard

Logout

Message Format

Events

AuxAudioPropertiesEvent

LoginStateChangeEvent

ParticipantPropertiesEvent

ParticipantStateChangeEvent

SessionMediaEvent

SessionNewEvent

SessionStateChangeEvent

Actions

Account.ChannelGetList.1

Account.Login.1

Account.Logout.1

Aux.CaptureAudioStart.1

Aux.CaptureAudioStop.1

Aux.GetCaptureDevices.1

Aux.GetRenderDevices.1

Aux.RenderAudioStart.1

Aux.RenderAudioStop.1

Aux.SetCaptureDevice.1

Aux.SetMicLevel.1

Aux.SetRenderDevice.1

Aux.SetSpeakerLevel.1

Connector.Create.1

Connector.InitiateShutdown.1

Connector.MuteLocalMic.1

Connector.MuteLocalSpeaker.1

Connector.SetLocalMicVolume.1

Connector.SetLocalSpeakerVolume.1

Session.Connect.1

Session.Create.1

Session.Set3DPosition.1

Session.SetParticipantMuteForMe.1

Session.SetParticipantVolumeForMe.1

Session.Terminate.1

Navigation menu

Search