Difference between revisions of "Linden Lab Official:What is Linden Lab's Quality Assurance process?"

From Second Life Wiki
Jump to: navigation, search
m (1 revision)
m (1 revision)
(No difference)

Revision as of 19:21, 5 October 2009

Ll color vert 100.gif
Official Linden Lab® Information: You may access and link to this page, but you may not copy, distribute, modify, adapt, or translate any content on this page. This content is subject to the Terms of Service and is not available under the Creative Commons or any other license.

Have a suggestion to improve this page? Contact us.

This article is part of the Extended Second Life Knowledge Base that includes advanced and specialized information. This information was originally provided by Linden Lab, but is not actively maintained nor guaranteed to be accurate. Linden Lab does not certify nor assume any responsibility for this information.

See the official Second Life Knowledge Base for the most current information.

Ever wondered how Second Life bugs are handled behind-the-scenes? In response to an extensive question asked by Ricky Lucero, Kelly Linden shared insights into our QA methodology:

We do indeed have a QA department.

Our process is roughly thus:

  • Project lead creates a branch from the trunk for a new project
  • Developers working on that project check their code into that branch
  • Developers write test plans for the code they wrote
  • When the project is "finished", the test plans and the branch/revision is sent to QA
  • QA tests the branch
    • They run the given test plans
    • They run related test plans
    • They run "smoke tests"
    • They send bugs back to the Project Lead
  • Developers fix the bugs and send back to QA, rinse, repeat
  • The branch is merged back into trunk
  • Trunk goes to QA and bugs are sent back to the developers (of the many projects that are in at that point)
  • Developers fix the bugs and send back to QA, rinse, repeat
  • A "Beta" gets released (either viewer like the Focus Beta or full grid)
  • Residents and QA send bugs to developers who fix the bugs, rinse repeat
  • New version gets released to the main grid

This is a little basic, and a little bit idealized. Sometimes emergency bug fixes short circuit some of the steps above (such as public beta). There can be more merging and branching of code involved depending on the length of the project and what is happening in the trunk. The summary is that we have a very complex system involving lots of hardware and lots of different software that all must work together, not to mention all the parts of each process internally. These systems are also being used in many, many different ways by many people; sometimes we just don't realize things are being done a certain way inworld until we 'break' it. Are we doing a good job of keeping the bugs out? No, not really. We do however have a QA department, and the process above is almost constantly under review and changing. I think we are doing a better job now than we have in the past. We are also dedicating a large portion of development towards fixing bugs, a lot of which includes future proofing against scaling issues and the like, and replacing core systems that were designed with what is now evident as faulty assumptions. These are often invisible to the user unless they break or we don't do it soon enough. As a developer I QA my own work before sending it to QA. It doesn't make sense to just write code, compile and send off. I check to make sure it does what I think it does first before I send to QA. We do have a larger development team than we do a QA team and doing it any other way I don't really think makes sense. The ability to do this is probably what the job description refers to.

When asked to elaborate on developers handling bugs and issue priorities, Kelly added:

As I said that was a simplified view. Bug reports from residents do not go straight to developers. They are read and repro'd by QA first, but the actual bugs found by the residents are fixed by developers.

Different definitions and different levels of critical, plus the 'behind the scenes' changes which are often far more critical than their hidden nature may imply. There are a large number of issues that are constantly being prioritized, and with a finite development team that means some issues can continually get filtered to the bottom, though we do try to prevent that when we can. Please do not take this as an invitation to post "what about bug X". Please submit bug reports for those when they happen.

Relatedly, we have a Known Issues page.