User:Torley Linden/Bug reporting

Relevant stuff from this should be folded into main Bug Tracker info.

Watch a video tutorial on "How to report a bug"!

If your eyes are glazing over already and you'd prefer me to show rather than tell you, WATCH THIS!

Torley's bug reporting best practices

The following owes a lot to Linden QA Engineers and helpful Residents, in particular Brent Linden the repro format, and I'm writing it down because I know it'll be useful to others. These aren't the rules for bug-reporting, but merely how I, Torley Linden, do it. Feel free to better 'em!

What makes a good bug repro?

In case you aren't yet familiar, "repro" is short for "reproduction", which means: a way to reliably make a bug happen. Why? So the bug can be investigated and fixed — which is a very good thing (TM). Think of modern medicine, where symptoms may point to a cause, or consider detectives & CSI, who look for clues to solve crimes. Finding a good repro is analogous and essential.

Second Life is unique in that, more than other technology-based experiences, we have a strong emphasis in getting our community's help in fixing bugs, especially through our public Issue Tracker. I know, JIRA (software made by Atlassian that powers the Issue Tracker) may look confusing at first, but if you've ever filled out a web survey (even one of those fun questionnaires on MySpace or FaceBook), the process is basically the same. You'll feel a lot more comfortable actually using the Issue Tracker a few times, so don't let anxiety cloud your mind, and take it easy at your own pace. If you're smart enough to use Second Life, you're smart enough to report bugs — it need not be a technical, complicated process, as you'll see.

The recipe (parts of a repro)

-

A dash is used to at the beginning of a general step.

- Verify:

I don't use these very often, but they're useful to include in longer, detailed test plans to guide someone along to ensure they're either following the steps correctly, or found a problem with the repro.

* Observe:

A special step that tells the person following the repro what goes wrong.

= Expected:

A special step describing the correct behavior. This is what needs to be fixed.

> Workaround:

If there's a way around the bug until a better fix is in place, share it here. This can help mitigate the pain a bug causes, so I recommend workarounds to be included whenever applicable.

Cook 'em together and you get a dish like this...

Example bug repro

This bug is fictional and solely provided for educational purposes.

Summary: WindLight: Changing Light color and right-clicking Color box crashes viewer

- Right-click an attachment on your avatar and select Edit.

- In the Edit tools, check "Edit linked parts".

- Click the Features tab and check "Light".

- Click the Color box.

- Verify: The Color Picker appears.

- Choose a new color (any works) and click Select.

- Verify: The Color box changes color.

- Right-click the Color box.

* Observe: The viewer crashes.

= Expected: The viewer shouldn't crash.

> Workaround: Disabling Water Reflections in Preferences > Graphics tab (with Custom checked) stops the crash from happening.

• See what this issue looks like on the actual Issue Tracker!
• See some actual issues filed by Torley Linden!

Additional notes

• "Viewer" refers to the Second Life application you run on your computer, not to be confused with "Second Life" as an online world in its entirety. If a bug caused all of Second Life to crash, that would be very bad indeed! I generally prefer "viewer" instead of "Second Life" because it's more clear what I'm referring to. If a bug crashes a region (also somewhat-incorrectly known as a "sim"), then "crashes region" would be appropriate.
• If the bug is specific to a certain viewer or feature-set, e.g., "WindLight", "Havok4", "Search", etc. I find it useful to include that version at the beginning of the summary. Here's another example: "Search: Looking for avatars with last names ending in 'x' fails". This is largely up to personal preference and isn't required, but does make it easier to sort through issues at-a-glance.
• You can include other info before or after the repro. Remember, this isn't a strict format, but a simple outline to base your own repros on! Improvise and have fun. :)

The meal (essentials)

• A clear summary - A summary — the title, or subject line — should tell us what the bug is about in generally 80 characters or less. "Second Life crashes" is a sucky summary, "Viewer crashes after right-clicking on avatar" is much better. "OMG INVENTORY IZ TOO BIG" is crappy, "Inventory window can't be resized smaller like in previous versions" is a clear improvement. Get the idea?
• Easy-to-follow steps - A repro is worthless without these, just like poor driving directions will get you lost and frustrated. Keep your steps as terse as possible (e.g., not big paragraphs of eye-hurting text), and if a step looks like it's getting too long to read in a single breath, split it into two steps, each on a new line. Universal rule-of-thumb: if two trusted friends can't follow your repro, it's likely not helpful.
• Exact wording of UI (User Interface) elements - If a window is labeled "Communicate", don't refer to it as the "Talk window". If a checkbox says "Edit linked parts", don't call it "Edit linked things". Simply refer to things as they appear. And if you don't know the difference between a button and a menu, or the different types of windows, learn more about elements of graphical user interfaces. If you use a computer, which you do, these are good-to-know. It's normal and OK to feel a little overwhelmed at first if these are new terms — give yourself time to get used to using them, and use them often for good practice.
• Be specific - This reinforces the above three: if someone told you to "click on the thing", would you have a clue which "thing" they're talking about? Of course not. Details can make or break a good repro. If you're really not sure how to address something, ask for help, or post a screenshot (keep reading for more on that).

Sauces & spices (make your repro even better)

• You can be terse! - Don't feel pressured to spell everything out. Shorter sentences will often suffice. You can write "Click OK" instead of "Click the OK button". However, if there's likely to be any reasonable confusion, a few extra words will go a long way towards understanding.
• Use a standard convention for writing about menus - I'm not telling you to boring. Some folks use "-->" or another indicator, but when I refer to nested menus (and other sorts of hierarchies), I simply use ">". It's nice 'n' simple. For example:

Go to World menu > Environment Settings > Environment Editor".

• Use quotation marks to kill ambiguity - Not a hard rule, but very useful when UI element labels have lowercase or more than one word. For example:

With the Build tools open, under the General tab, uncheck "Allow anyone to copy".

"In Preferences > Graphics tab, click "All Avatars and Objects".

• Communicate visually - A wonderful thing about our Issue Tracker is that you can attach screenshots and even videos using the "Attach file" and "Attach screenshot" links on the left. In both cases, you'll see an comment field, which is useful for saying something like, "See the above image I just attached for more info".
  • Attach file - Clicking on this brings you to a new screen where you can click the "Browse..." button and select a file from your hard drive to upload. Don't upload BMP images, they're too big and don't automatically get a thumbnail preview. I recommend using JPGs in most instances, and PNGs when you need lossless quality (for example, you're trying to show the subtle graphical differences between two screenshots and don't want JPG compression to degrade how things look). And if you're uploading videos, make sure they're viewable with QuickTime Player because Second Life uses QT and many Residents already have it installed. Don't use an obscure codec that others can't easily view.
  • Attach screenshot - Clicking on this pops up a new window (make sure your popup blocker is off or allows this), where you can easily select an image in another application (even in Photoshop), copy it, then paste it in here. You'll see a preview before it goes through, and you can give it a descriptive name if needed.

Desserts (what to do next)

• Reporting a bug isn't the end of the story — it still has to be fixed. And before that, garnering community support helps get our attention. There are two basic ways to do this:
  1. Voting - Do you want something fixed? Vote for it by clicking the "Vote for it" link at the left of every issue. I know some people say, "This bug is so obvious that LL should fix it right away", and that may be true, but taking a few seconds to login to the Issue Tracker and click a link and show you are affected helps us more accurately determine the scope and extent of a problem. A "Critical" issue with 50 votes and a good repro is much more likely to be fixed than another with 2 votes and no repro. Search the Issue Tracker for historical evidence.
  2. Commenting - Know more about a bug and what causes it? Have your say! This is really helpful when a bug doesn't have a good repro, but an issue may exist for people on similar systems. For instance, see the comments in VWR-925. Advanced Residents can also edit and add info to an issue to make it clearer to understand.
• If you think the Issue Tracker is too hard to use, let me know and give specific details why. It's fundamental that criticism which helps us improve is actionable, meaning "we can use it to fuel our progress" as opposed to "I hear what you're saying, but can't go further with it". Vague complaints don't do anyone any good. Also if possible, point to bug-reporting systems out there you prefer.
• Not all bugs have repros. But when they do, that definitely should be included.

Doggie bag (further help)

• Issue Tracker - The main info-page for "how to use the Issue Tracker".
• Knowledge Base Article of the Week #6: The Public Issue Tracker - General and specific info which elaborates on some of the above important points, and links to a closely-related article I wrote, "How to report bugs better".
• QA Portal - The foremost gateway to Quality Assurance-related knowledge on this here public wiki. I'm not a QA Engineer, but I do learn from 'em. :)
• Debug_help - Help with the following — if you don't know why it matters, read on:
  • How do I send crash logs?
  • How do I find my system information?
  • Where do I find my SecondLife.log file?
  • How do I get my Fast Timers results?

"I understand that not all bugs have repros. But when they do, that definitely should be included." -Torley