Viewer Crash Reporting

Objectives

• Report all crashes properly (opt-out available)
• Make crash report processing automatic and immediate
• Make crash report data useful

Motivation

Studio Shiny has committed to making the Second Life viewer more stable. This involves finding and fixing viewer crashes. We have found that the existing crash reporter doesn't help us as much as it should. It doesn't always give us the data we need to investigate crash patterns, doesn't scale to SL's current size, doesn't always report a crash, in fact, sometimes the crash reporter itself crashes.

Tasks

Phase 1

• Create central library of crash reporter functionality
• Get near 100% of all crashes reported with the call stack included, which pinpoints the function or method in which the crash occurred.

Phase 2

• Refinements to the crash reporter functionality
  • properly capture crashes before login or viewer initialization

Viewer Crash Hunters

• A team of engineers will utilize the new Viewer Crash Reporter to identify the highest reported crashes and investigate the call stacks to pinpoint bugs.
• The project is gated on the adoption of
  • the 1.19.0.5 viewer which utilizes the new crash reporter.
  • the 1.19.1 viewer which also provides stack info on "deadlocks" (in which a crash occurs, but in a different thread.)
  • implementation of a thread watchdog process (in 1.21 viewer) to detect deadlock situation and force a "normal" crash that reports the crashing thread
• Crash Hunters will analyze crashes and make fixes into the Release Candidate (RC) branch
  • This is a more aggressive introduction of code into the RC branch cycle, which may in the short term increase the "beta" nature of the Release Candidate during its early iterations (RC1, RC2)
  • the benefit will be a longer term goal of reducing the crash rate dramatically for all Residents by the time that a Release Candidate becomes the final candidate (RC4, RC5).
  • Most crashes currently appear in the rendering code, so many fixes (and possible new bugs!) will appear in rendering options.

Some examples of results:

Fixes in Branch_1-19-1-Viewer

• VWR-4959: WindLight: "Sculpted Prim" preview makes a huge distorted mess of polygons (and possible crash)
• VWR-5351: Crash when opening the expanded media control
• VWR-5499: Release candidate crashes on Mac when looking with option-E
• crash in LLViewerRegion::getCapability
• Crash in apr_file_lock
• Crash on LLVertexBuffer::destroyGLBuffer
• Crash: 'Error parsing skeleton XML file'
• Crash in LLImageBase::allocateData
• LLViewerPartGroup::updateParticles
• crash when quitting from login screen
• Crash in gunzip_file
• Crash in LLTabContainer::draw() if selected tab is invalid
• Notecards crash users when passed from person to person
• Opening a full screen window on some ati cards causes a crash on login
• Crash in LLObjectSelection::addNodeAtEnd
• Crash rendering an area with a lot of sculpties
• RC crashes on the 4th attempt to connect during DNS failure.
• Elminate crash-prone LLLinkedList, LLDoubleLinkedList, LLSkipList, LLPtrSkipLost, and LLAssocList from newview
• Apparent fix: Crash VLLVolume::unref (RC 1.19.1 build 0)
• Apparent fix: crash in LLVOVolume::processUpdateMessage

Fixes in Branch_1-20-Viewer

These appeared in 1.20 Release Candidates RC1 - RC6

• Fixed: VWR-5382: Constant crashing using inventory in 1.19.1 RC0
• Fixed: VWR-3822: ATI crashes on startup when in 16 bit color mode
• Fixed: VWR-6603: Immediate crash when upload image .jpg
• Fixed: VWR-6343: 1.20 causes fatal driver instability with some nVidia drivers
• Fixed: VBO related crash on ATI R100 cards
• Fixed: Occlusion crash on SIS cards
• Fixed: Get rid of a sculptie assert
• Fixed: 1.20 RC crashes with attached obj using Avatar Rendering Cost
• Fixed: Crash on startup (1.19.1 RC)
• Fixed: crash in LLViewerImageList::updateImagesFetchTextures
• Fixed: Crash in LLViewerJointMesh::updateFaceData
• Fixed: crash in LLViewerRegion::setCapability
• Fixed: crash on LLVertexBuffer::destroyGLBuffer
• Fixed: crash on LLVOVolume::sculpt - newview/llvovolume.cpp
• Fixed: crash in LLImageBase::generateMip
• Fixed: crash in LLImageRaw::copyLineScaled
• Fixed: crash in LLFloaterIMPanel when starting conference
• Fixed: crash in LLFace::renderSelected.
• Fixed: Crash on error: mapBuffer returns 0
• Fixed: crash on LLUIImageList::onUIImageLoaded
• Fixed: crash on LLViewerJointMesh::updateFaceData
• Fixed: crash from glMapBuffer returning 0
• Fixed: 1.20 consistently crashing with ctrl-alt-T in ReleaseNoOpt
• Possible fix: VWR-6299: In Stanlee at ~5,247,750 I crash crash crash.
• Possible fix: VWR-6343: 1.20 causes fatal driver instability with some nVidia drivers

Fixes in Branch_1-20-Viewer-2

These appeared in 1.20 Release Candidates RC7 - RC10

• Fixed: Crash on LLSpeakerMgr::findSpeaker
• Fixed: crash on LLAudioData - llaudio/audioengine.cpp
• Fixed: crash on LLGroupNotifyBox::moveToBack
• Fixed: crash on LLPieMenu::show
• Fixed: crash on LLFloaterWorldMap::buildLandmarkIDLists
• Fixed: crash in LLFolderViewFolder::setSelection
• Fixed: crash on LLSurface::resolvePatchRegion()
• Fixed: crash on LLViewerRegion::setCapability
• Fixed: Infinite loop in wearables (LLWearableList::processGetAssetReply) causes a crash
• Fixed: crash in LLMediaImplQuickTime::load
• Fixed: crash on LLViewerParcelMedia::update

See Also

• Viewer Roadmap