Dragon
| Help Portal: |
Avatar | Bug Fixes | Communication | Community | Glossary | Land & Sim | Multimedia | Navigation | Object | Video Tutorials | Viewer | Wiki | Misc |
Background
Introduction
This wiki page refers to the use of Dragon speech recognition software, such as: Dragon NaturallySpeaking Preferred, Dragon NaturallySpeaking Professional, Dragon NaturallySpeaking Legal (if it's available for some), Dragon Medical, and once upon a time referred to as Dragon Dictate. The umbrella term "Dragon" will be primarily used to refer to this family of PC platform speech recognition software.
While Dragon is primarily considered useful for accessibility in Second Life, the software may be useful to anyone wishing to prevent computer related repetitive strain injuries as well as for the sake of efficiency.
However, at this time, Dragon does not function well with Second Life, and its usefulness is therefore limited until Linden Lab make some changes to their software. Until then, perhaps the following information will be useful.
If you have mobility concerns, please also view this Accessibility page in the Wiki.
Editions of Dragon
- Dragon NaturallySpeaking Standard -- not recommended as it will not play back audio which is essential for effective correction/teaching
- Dragon NaturallySpeaking Preferred -- bare bones minimum suggested, allowing voice to text, and text commands
- Dragon NaturallySpeaking Professional -- Section 508 Certified, and suggested for those with mobility limitations as this version allows for complete customization and computer control command creation
Minimum PC Specifications
Nuance's minimum specifications are a bare minimum and usually not sufficient for most users.
The absolute realistic minimum PC desktop hardware requirements for Dragon Version 10 are:
- For a Windows XP OS, DuoCore 2.0 GHz and 2 GB RAM.
- If the OS is Vista, a higher processor speed and definitely more RAM will yield increased performance from Dragon.
- For a Windows 7 OS, DuoCore 2.0 GHz and 4 GB RAM.
The Basics of Using Dragon in Second Life
A Personal Statement of Experience
Can Dragon be used with Second Life? Yes. Can it be used well? Sadly, no, not yet. However, this has not stopped me from using it on occasion. Do I use it all the time? No, but that’s because I have some upper limb operability.
My main use is voice to text; to reduce the strain on my tendonitis inflicted hands and arms. I take a certain pleasure in being able to chat in the Local Chat bar while using my Logitech Dual Action gamepad to move my avatar around physically (yes, you can walk and talk in Second Life!). Mind, the chat result is littered with errors (dropped letters and spaces from words), but I can either cite “typonese” or explain that I’m using Dragon. For longer chat sessions, I will use Dragon’s Dictation Box feature -- and then not even the fastest typists can usually keep up with me (unless Dragon makes numerous mistakes because I’m dead tired).
I do like to build and create scenes. This is not easy to accomplish with Dragon, although when I tire of clicking with the mouse, I simply say “mouse click” or “mouse right click” wherever I hover the mouse using my Kensington Expert Mouse trackball.
While it is possible to move your avatar about by voice, it can be cumbersome to say “move up 20” over and over to move forward great distances. However, I can use the Go Here function in Second Life by using Dragon to right click a location on my viewer and go there!
Having Dragon NaturallySpeaking Professional (Legal and Medical versions as well) means that I can create custom commands that correspond to actions, gestures, changing the environment settings or anything else that everyone else has to click click click for under the menu system or use keyboard shortcuts for.
I am grateful that I do not have to depend on Dragon 100%, but one thing people have taught me is that those with physical limitations often have incredible patience and perseverance. And hopefully in time, Linden Lab will see what needs to be done to make our world, our Second Life, more accessible.
Elric Anatine
Introduction
Much of the advice below is given under the pretext that the user will have limited to no upper limb functionality. I will provide a section at the end in regards to alternate input devices for those who have some hand use.
The information below is designed for existing users of Dragon, or those wishing some guidance during their initial learning of Dragon and the software's extent of functionality within Second Life.
Emulating Keyboard Shortcuts by Voice
Unfortunately, Second Life has become less friendly with Dragon over time, and emulating keyboard shortcuts by voice do not always function reliably. For instance, to toggle flying, you may be able to say either "press Home" or "press Home key" -- one or the other, or neither will work at any given time. This is not Dragon, but simply the lack of compatibility between Dragon and Second Life.
If you are in possession of Dragon NaturallySpeaking Professional, I suggest creating numerous step-by-step commands (i.e. create a command called "fly" that simply presses the Home key with a keystroke step) for the features you most frequently use. These tend to be more reliable than pressing specific keyboard shortcuts by voice (i.e. "press Home key").
Main Menu Items
The main menu items (File, Edit, View etc.) are, unfortunately, inaccessible by voice. However, there are a number of keyboard shortcuts associated with various menu items that one can affect by saying "press <name of key>" (i.e. "press Control Shift H" to teleport home).
If you are a Dragon NaturallySpeaking Professional user, then it might be prudent to make intuitive step-by-step commands so that you can simply say "teleport home", and Dragon will press the keyboard shortcut sequence that is associated with the action.
Moreover, if you have Dragon NaturallySpeaking Professional, and are so inclined, you may wish to make macro recorder commands that will click on the appropriate menu header item, and subsequent submenu items. Since mimicking those commands with the macro recorder can be time-consuming, I would suggest using a mouse click on the main menu level item, and follow subsequent steps with the appropriate keystrokes (i.e. arrow keys or keyboard keys).
Please note that the submenu does take advantage of Alt key underlines (i.e. once the File menu is open, you can press the < U > and then < B > key to initiate a Bulk Upload).
I did begin creating a Second Life voice command pack, but I've found that Linden Lab frequently changes the submenu items, thwarting the commands on a regular basis. Such is easy enough for me to maintain myself, with regards to the features that I use, but constant updating of everything is quite time-consuming. As such, it is suggested that you create voice commands for those features that you use.
It is my hope that one day Linden Lab will provide a keyboard shortcut for accessing the main menu level items as this would allow for the creation of voice commands to control all functions under the menu.
Lower Screen Button Items
The lower screen button items (Communicate, Fly, Snapshot, Search etc.) are likewise not accessible by voice. The implementation of macro recorder commands would be advisable for those with Dragon NaturallySpeaking Professional. However, be aware that the precise location of these buttons will vary from one person's screen resolution to another. The sharing of these commands will need to be treated with caution and may need reworking.
And again, do watch for changes that Linden Lab makes with their new viewers.
Screen Clicking
To click about the screen (within the search or map windows etc.), you can use the MouseGrid command, coupled with "mouse <left/right/double> click" commands.
If you can keep Second Life windows in consistent locations, then you may wish to make a series of macro recorder voice commands that will click various parts of the screen.
Text Chatting
You will not be able to dictate with full Select-and-Say in the chat bars/windows. As you dictate, words will not appear completely (letters will be missing, and the cursor may jump around). Attempts at correction will yield poor results, as Dragon will shift corrected text around with uncorrected text. This is typical behaviour for any interface that does not follow standard window classes for edit controls.
Using the command "show dictation box" will enable you to dictate into the Dragon dictation box with full Select-and-Say, and then transfer the text into the Second Life chat bar. Please ensure that your cursor is appropriately positioned prior to initiating the dictation box, or you risk spilling the text out into nowhere (possibly with detrimental but hopefully benign or amusing results). In the Dictation Box Settings, choose "Use Windows Copy command" under Copy and "Simulate keystrokes" under Paste. Dead Ghost has a quicker workaround than the Dictation Box, but you must install another piece of software and set this up. Please contact Dead Ghost in world for more information.
Commands such as "delete line", will not function (you will fly instead, which can be most amusing). Understand what these commands actually do on the keyboard, and you will quickly realize that Second Life has keyboard shortcuts that interfere with a number of standard Windows keyboard shortcuts (i.e. deleting a line begins with pressing the <Shift + Home> keys, and in Second Life, the <Home> key is responsible for flying). If you do need to delete something in a text bar, try using the "backspace (2 to 20)" command.
Say "press Enter" or "press Enter key" to open the chat bar. Say "press escape" or "press escape key" to close the chat bar.
Actions and Gestures
Your own actions in chat are often preceded with "/me". You may wish to create vocabulary phrases that will type out commonly used actions rather than you having to say "slash me" prior to the action.
For instance, in the vocabulary editor you may wish to place the following in the written form: /me jumps for joy. In the spoken form you will wish to enter: jumps for joy. Dragon will then enter "/me jumps for joy" when you say "jumps for joy". Ever remember that you can change the spoken form (i.e. make it shorter or use an acronym as convenient -- "JFJ" in the spoken form).
If you wish to take this further, create an actual step-by-step command that will type "/me jumps for joy", and then follow this with pressing the <Enter> key to push the action through Second Life.
Use the above for your gestures as well. For instance, in the vocabulary editor you may wish to place the following in the written form: /yes!. In the spoken form you wish to enter: SL yes. Dragon will then enter "/yes!" when you say "SL yes". Again, feel free to be creative with the spoken form, or even take this further and create an actual step-by-step command.
Text & Voice Chatting
If you wish to enable voice chat in Second Life, you will discover that your microphone will be on the same channel as Dragon. This is good news, because then you can use voice chat and Dragon at the same time or interchangeably very easily.
First, you will need to enable voice chat. You may also wish to assign a keystroke for the "push to talk" action. Alternatively, you can use a mouse for "push to talk" or locking the voice toggle on/off. And finally, you can always create a macro recorder command to click these parts of the screen for you by voice.
Possible scenarios are as follows:
(1) Dictate with Dragon into the chat bar (be mindful of the text chatting limitations mentioned above) and have voice on. Others will be able to see your text and hear you if they have Second Life voice enabled.
(2) Have Dragon's microphone off or asleep while you use Second Life voice. Others will be able to hear your voice (if they have Second Life voice enabled), and of course, Dragon will not transcribe your voice into any text.
(3) Dictate with Dragon into the chat bar (be mindful of the text chatting limitations mentioned above) with voice off. Others will see your text but, of course, not be able to hear you.
Once the cursor focus is in the inventory list (not the inventory search text field), you need only affect the pressing of the first letter of the inventory item or folder to select that item/folder (i.e. "press C" will select Calling Cards, unless you have another item or folder that begins with the letter C).
Unfortunately you cannot press the < C > key twice to select the Clothing folder. Use, instead, the spell command to spell out the first few letters of the folder you wish to navigate to (i.e. you will need to say "spell C L" to navigate to the Clothing folder).
To expand or collapse folders, effect the pressing of the <Enter> key. As a side note, one of the first commands I ever made was a step-by-step command that presses the <Enter> key when I simply say "Enter".
When all else fails, you can use the cursor keys for navigating inventory. More specifically, use the <Up> and <Down> arrow keys, where you say "move <up/down> (1 to 20)".
Moving Around
Use the "move <up/down> (1 to 20)" commands to move your avatar forward and backward respectively.
Use the "move <left/right> (1 to 20)" commands to rotate your avatar left and right respectively. You may need to say "move right 20" two times plus "move right 10" to rotate 90° to your right.
Use the "page <up/down>" commands to jump up and down respectively.
Use the "press Home key" command to toggle the flying movement.
Saying "press Enter" while a landmark is selected from your inventory will teleport you to that destination.
Right click a specific location that you wish to go to. This will provide the pie menu. Choose Go Here from the resulting pie menu. For example, if you wish to walk your avatar to a tree in the distance, hover the mouse over the base of the tree (with a mouse or using Dragon ’s MouseGrid), say “mouse right click”. Nudge the mouse up a bit by saying “mouse up 10” and then “mouse click”.
Alternate User Input Devices
For those of us who cannot make use of a conventional mouse, finding an alternate input device (if possible) can enrich your Second Life experience.
Logitech's Dual Action game pad can work well for some individuals. However, the newer version of this unit can be very stiff and more difficult to use. Moreover, wireless units and those with rumblepacks are much heavier and not advised -- as of this moment, Logitech is no longer producing the basic corded version. If you can, find the older units on eBay. The older units have all black buttons on the right hand side, and the numbers are etched into the buttons. The newer units have the numbers painted on the buttons it in white. The game pad is excellent for moving your avatar around, as well as building. Please also see the Logitech_Gamepad Wiki page for more information.
A Kensington Expert Mouse trackball is great for moving the mouse cursor around the screen. You can either continue to use the Kensington trackball to mouse click, or you can use Dragon ("mouse <left/right/double> click") as needed.
A Wacom pen tablet is very useful for any necessary fine mousing control. However, if your hands are as unreliable as mine, you can inadvertently click on the wrong button from a Second Life dialog box -- this could result in an action you did not want, so for anything critical, I recommend using the Kensington trackball and voice clicking.
The space navigator is thus far unimpressive.
A Slow Dragon
If you find that Dragon is slow, it is very likely due to your PC's system specifications. Despite the minimum requirements cited on the box, the following specifications are a more realistic minimum, providing the best experience possible with Dragon.
Pentium IV, 3.0 GHz, 2 GB RAM or DuoCore 2.0 GHz, 2 GB RAM
Windows XP is faster than Windows Vista with respect to performance of software applications under this operating system. If you are using Windows Vista, increase both your processor and RAM in excess of those listed above. Windows 7 would best be used with 4 GB RAM.
Some other tips are as follows:
- Use a quality USB microphone (Andrea or Sennheiser, although some Plantronics units perform adequately). Make sure that your microphone or system is not picking up any interference (listen to your dictation for clarity).
- Defragment your hard drive every month to maintain smooth operation.
- Do not use a software firewall as they slow your system down considerably. Use a hardware firewall (most Internet routers have them built in – verify this).
- Do not use Norton or Macafee Anti Virus or any other antivirus that taxes your system. NOD32 by ESET is low-impact on a system's resources. Most free antivirus software programs do not allow you to exclude directories from your PC from being actively scanned. Dragon's main and user directories are being written to constantly, and if your antivirus cannot exclude these directories, then Dragon's performance will slow down.
- Quality systems. Business class PCs are not loaded with resource demanding "fluff" software and are preferable to consumer class PCs. You do get what you pay for, and quality components, and fast hard drives all make a difference and impact system performance.
- Watch your downloads. Limewire and similar programs are recipes for disaster and leave you vulnerable to spyware and malware. Run a spyware and malware detector regularly.
Accessing SL with Dragon - The Technical Details
Introduction
In addition to devices such as trackballs, and pen tablets, Dragon can be an integral part of an adaptive solution. Unfortunately, not all interfaces or software applications are Dragon friendly, and Second Life is one of those. However, this is not to say that it does not function, it simply does not function well.
Why Dragon Doesn't Work in Second Life
The short answer is that the Second Life client is not compatible with Dragon . Dragon is similarly not compatible with Microsoft PowerPoint and a plethora of other software programs and interfaces. Out of the box, Dragon is compatible with Internet Explorer, Microsoft Word, Microsoft Outlook, Microsoft Excel, Corel WordPerfect and sundry other popular applications -- and even then only the most recent versions.
There are two levels of compatibility desired: (1) navigation (computer command and control), and (2) dictation (speech to text).
Navigation compatibility allows Dragon users to voice the labels of application menus and option buttons, which Dragon will in turn click (in lieu of mousing). This capability is called Say What You See. For Say What You See to function in an application, the application must use standard window classes for controls, such as "Button", and label both the control and the visual text the same.
For example, in MS Outlook, if one says "Tools", the Tools menu will drop down. This is because the software code (the control label) behind the word "Tools" is in fact "Tools" (fancy that! -- but don't be too excited as some MS programs don't even follow these rules). Second Life does not use these standard window labelling classes for controls, and therefore is not Say What You See compliant. You can say “File” until you are blue in the face and the File menu will never be clicked upon by Dragon .
Dictation compatibility allows Dragon users to edit and correct previously dictated text -- this really is vital as dictators/users and Dragon are not 100%. This capability is called Select-and-Say. For Select-and-Say to function in an application, the application must use standard window classes for edit controls such as Edit, RichEdit, RichEdit20A, RichEdit20W, RichEdit50W. Second Life does not use these standard window classes, and therefore is not Select-and-Say compliant.
For example, if one dictates an email in MS Outlook and says "Hi John" but Dragon types out "High John", one can say "correct high" to correct the incorrect word. Dragon selects "High", and provides you with several choices, one of which will be "Hi" (usually choice 2). You say "choose 2", and the correction is made and Dragon improves (usually). We are in our happy place.
Within MS PowerPoint, we can dictate into a blank slide and even correct text after immediately dictating it, but the moment we leave the PowerPoint window and return to the text box in PowerPoint, we can no longer select or correct. At least the menus and buttons are all clickable by voice!
When in Second Life, we can dictate into the chat bar, but we cannot select or correct and we also find that some of our words are missing letters! In fact, throughout 2009, the case of the missing letters has increasingly worsened in Second Life, to the point that dictating directly into chat windows is a lesson in frustration. Menu items and buttons definitely cannot be clicked.
Here is the reality -- not all applications are written with accessibility in mind, nor in compliance with what Dragon needs in order for compliance with its own code. It's a fact of life. But Second Life is such an incredible experience for so many people, it is my personal fondest hope that one day the Second Life client WILL be compliant with Dragon . Until then, the best we can do is work within its confines, and really understand how Dragon works.
What Linden Lab Can Do
As much as I would like to advocate to Linden Lab for accessibility, I sadly do not have the time (although I would be more than happy to speak directly with anyone from Linden Lab should they be seriously wishing to work towards adapting Second Life for use with Dragon). However, I would point them in these directions and encourage you to ask them to do the same:
Knowledge Base Tech Note # 5176 (Guidelines for Developing Windows Applications Compatible with Dragon NaturallySpeaking): This link contains a downloadable PDF.
Knowledge Base Tech Note # 3067 (Coding HTML documents that can be navigated by voice): This link contains a downloadable PDF.
Dragon NaturallySpeaking Software Developer Kits
Dragon and Viewer 2 Beta
Summary
Unfortunately Dragon's performance in Second Life is not improved in the Viewer 2 Beta. Viewer 2 is therefore not Section 508 compliant, nor is there any indication at this time that it will be.
Moreover, Viewer 2 often requires numerous additional mouse clicks to perform the same tasks as in Viewer 1, making Viewer 2 less friendly for accessiblity. There is also a loss of some keyboard shortcuts in Viewer 2.
In order for Dragon to function effectively in Viewer 2, Linden Lab will still need to review the information provided above.
Customizing Dragon for Second Life
Word List
The following is a text file containing Second Life specific terminology.
Dragon SL Word List version 1.0
Instructions
- Download the above text file to your computer (i.e. your desktop)
- Start Dragon
- Say "import words"
- Follow the on screen instructions, browsing to the above downloaded text file
- Complete the process
- Delete the original text file from your saved location if you wish, as it's no longer necessary
Updates
Please contact Elric Anatine in world if you have additions to this list (please send a notecard). Updates will be posted here.
Quick Tips for Using Dragon in Second Life
Clicking on SL's Main Menu with Mouse Grid
You need not pause between MouseGrid resizing stages. Once you know the numeric sequence, you can specify this sequence without a pause.
Say "mouse grid one one one", pause and then say "mouse click".
This will drop down the File menu, and from here you may use the "move <left/right/up/down> (1-20)" commands to navigate further through the menus.
Once you have highlighted/selected the menu item you wish, say "press enter".
Voicing Second Life Keyboard Shortcuts
You can find a list of Second Life's keyboard shortcut keys here: All_keyboard_shortcut_keys
Say "press (keyboard shortcut)" to activate these shortcuts. For instance, say "press control shift Y" to set the world view to midday.