Difference between revisions of "Finding leaks"

From Second Life Wiki
Jump to navigation Jump to search
m
 
(11 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{OSWikiContribBox}}
= Overview =
= Overview =


This page describes a method how to identify memory leaks within the SecondLife viewer using Microsoft's debug memory routines.
This page describes a method how to identify memory leaks within the SecondLife viewer using Microsoft's debug memory routines.


'''Note:''' Currently it seems that this will only work for VS2003 because Debug builds fail under VS2005 with /MTd (see the ''comiling'' section below).  If you are using VS2005 and find a solution, please update this Wiki page.
'''Note:''' If you have questions, please use the discussion page on this wiki topic. ([[User:Nicholaz Beresford|Nicholaz]] 10:29, 29 June 2007 (PDT))
 
'''Note:''' If you have questions, please use the discussion page on this wiki topic.


<br/>
<br/>
<br/>
<br/>


= Identify Leaks =
= Identifying Leaks =
<br/>
<br/>
== Modifying the source ==
== Modifying the source ==
Download the memory leak patch from [[http://www.blueflash.cc/users/nicholaz/libs/leakdebug.patch leakdebug.patch]] and apply it to the source tree via  
Download the memory leak patch from [http://www.blueflash.cc/users/nicholaz/~libs/leakdebug.patch leakdebug.patch] and apply it to the source tree via  
* in a command shell go to the source directory on the level where you see the 'linden' folder when you type ''dir''
* in a command shell go to the source directory on the level where you see the 'linden' folder when you type ''dir''
* '''c:\cygwin\bin\patch -p 0 -i leakdebug.patch'''
* '''c:\cygwin\bin\patch -p 0 -i leakdebug.patch'''
Line 19: Line 20:


== Compiling ==
== Compiling ==
For general compiling instructions, see [[Compiling the viewer (MSVS2003)|Microsoft Windows - Visual Studio .NET 2003]] or [[Compiling the viewer (MSVS2005)|Microsoft Windows - Visual Studio 2005 & Visual C++ 2005 Express]].


The patch activates the leak debugging routines through the '''linden_common.h''' files and is only activated in the '''Debug''' configuration.  It should be completely transparent in other configurations like ReleaseNoOpt or ReleaseForDownload.
The patch activates the leak debugging routines through the '''linden_common.h''' files and is only activated in the '''Debug''' configuration.  It should be completely transparent in other configurations like ReleaseNoOpt or ReleaseForDownload.
Line 26: Line 29:
*# select the '''Debug''' configuration from the toolbar
*# select the '''Debug''' configuration from the toolbar
*# mark all projects in the source tree, '''except''' script_compile_fb, test, win_crash_logger and win_updater
*# mark all projects in the source tree, '''except''' script_compile_fb, test, win_crash_logger and win_updater
*# select Properties and make sure the '''Debug''' configuration is selected in the upper right corner
*# select Properties and make sure the '''Debug''' configuration is selected in the upper left corner
*# select C/C++, Language and set Runtime Library to '''Multithreaded Debug /MTd'''
*# select C/C++ > Language and set Runtime Library to '''Multithreaded Debug /MTd'''
*# close the dialog
*# close the dialog


Line 36: Line 39:
* Linker options for newview
* Linker options for newview
*# select '''newview''' alone
*# select '''newview''' alone
*# in Properties, Linker, Input set Ignore Specific Library to '''msvcrt.lib;msvcrtd.lib;libc.lib;libcd.lib;libcmt.lib'''
*# select Properties and make sure the '''Debug''' configuration is selected in the upper left corner
*# select Linker > Input and set Ignore Specific Library to '''msvcrt.lib;msvcrtd.lib;libc.lib;libcd.lib;libcmt.lib'''
*# close the dialog
 
* Remove LLMOZLIB (optional for VS2003, required for VS2005)
*# select '''newview''' alone
*# select Properties and make sure the '''Debug''' configuration is selected in the upper left corner
*# select Linker > Input
*# remove '''llmozlib-vc80.lib''' from the list of input libraries
*# close the dialog
*# close the dialog
*# edit '''llpreprocessor.h''' in the '''llcommon''' project and change
#define LL_LIBXUL_ENABLED  1 (line 59)
to
#ifndef _DEBUG
#define LL_LIBXUL_ENABLED 1
#else
#define LL_LIBXUL_ENABLED 0
#endif


* Compile
* Compile
*# Simply compile the Debug configuration
*# select compile for the Debug configuration
*# have some coffee and/or cigarettes, the build (especially the link step) is excruciatingly slow
<br/>
<br/>


Line 46: Line 66:
* Run the program under the debugger (F5) or standalone (Ctrl+F5)
* Run the program under the debugger (F5) or standalone (Ctrl+F5)
* Running in the debugger during startup you will notice a couple of exceptions when freeing memory.  Just continue (these seem to come from the mozilla DLLs).
* Running in the debugger during startup you will notice a couple of exceptions when freeing memory.  Just continue (these seem to come from the mozilla DLLs).
* If you encounter crash_and_loop situations (when you hit these, you'll know what it means), consider to delete the contents from the crash_and_loop function and replace it with a single '''__asm int 3;''' command.  This way the debugger will stop with an exception but you will be able to continue.
* If you encounter crashAndLoop situations (when you hit these, you'll know what it means), consider to delete the contents from the crashAndLoop function and replace it with a single '''__asm int 3;''' command.  This way the debugger will stop with an exception but you will be able to continue.
* Finish running the program
* Finish running the program
* Look at the '''leaks.log''' file in the program folder.
* Look at the '''leaks.log''' file in the program folder.


<br/>
<br/>
== Identifying leaking source through leaks.log ==
== Identifying the source of the leak through leaks.log ==
Finding leaks is a nice task.  Like post mortem debugging in many cases it is puzzle and detective work and the leaks.log file is your forensic evidence.
Finding leaks is a nice task.  Like post mortem debugging in many cases it is puzzle and detective work and the leaks.log file is your forensic evidence.
* '''client blocks''' were allocated through '''new''' operations inside the source.  Source file and line number will be identified and usually it is not too hard to figure out a program path that bypasses the deletion.
* '''client blocks''' were allocated through '''new''' operations inside the source.  Source file and line number will be identified and usually it is not too hard to figure out a program path that bypasses the deletion.
* '''normal memory blocks''' in the leak dump are coming from 3rd party libraries.  You'll need creativity to find these (I do it mostly through custom code in '''crt_alloc_hook''' in crtdebug.cpp ... see the example there).
* '''normal memory blocks''' in the leak dump are coming from 3rd party libraries.  You'll need creativity to find these (I do it mostly through custom code in '''crt_alloc_hook''' in crtdebug.cpp ... see the example there).
* currently there is a small amount of "unfixable" leaks as normal blocks when the program starts. 
** one is from a technique the Lindens are using in llerror.cpp (see [[https://jira.secondlife.com/browse/VWR-849 VWR-849]]), which Zero Linden does not want to fix, although I have a patch which I deem safe
** another is from code in llmozlib which I could not compile yet (these can be avoided by compiling without mozlib, see compile instructions above)
** I have also found a few that seem to trace down into fmod.lib, but fmod doesn't seem like the cleanest code on the planet anyway.
** unless you want to be religious about the issue, you can ignore normal blocks with alloc numbers under 1,000,000

Latest revision as of 09:03, 16 August 2007


Overview

This page describes a method how to identify memory leaks within the SecondLife viewer using Microsoft's debug memory routines.

Note: If you have questions, please use the discussion page on this wiki topic. (Nicholaz 10:29, 29 June 2007 (PDT))



Identifying Leaks


Modifying the source

Download the memory leak patch from leakdebug.patch and apply it to the source tree via

  • in a command shell go to the source directory on the level where you see the 'linden' folder when you type dir
  • c:\cygwin\bin\patch -p 0 -i leakdebug.patch


Compiling

For general compiling instructions, see Microsoft Windows - Visual Studio .NET 2003 or Microsoft Windows - Visual Studio 2005 & Visual C++ 2005 Express.

The patch activates the leak debugging routines through the linden_common.h files and is only activated in the Debug configuration. It should be completely transparent in other configurations like ReleaseNoOpt or ReleaseForDownload.

When compiling, make sure you compile all projects with the /MTd flag

  • Set all projects to /MTd
    1. select the Debug configuration from the toolbar
    2. mark all projects in the source tree, except script_compile_fb, test, win_crash_logger and win_updater
    3. select Properties and make sure the Debug configuration is selected in the upper left corner
    4. select C/C++ > Language and set Runtime Library to Multithreaded Debug /MTd
    5. close the dialog
  • Add crtdebug.cpp to newview
    1. in the source tree under newview right click Source, add existing
    2. crtdebug.cpp
  • Linker options for newview
    1. select newview alone
    2. select Properties and make sure the Debug configuration is selected in the upper left corner
    3. select Linker > Input and set Ignore Specific Library to msvcrt.lib;msvcrtd.lib;libc.lib;libcd.lib;libcmt.lib
    4. close the dialog
  • Remove LLMOZLIB (optional for VS2003, required for VS2005)
    1. select newview alone
    2. select Properties and make sure the Debug configuration is selected in the upper left corner
    3. select Linker > Input
    4. remove llmozlib-vc80.lib from the list of input libraries
    5. close the dialog
    6. edit llpreprocessor.h in the llcommon project and change
#define LL_LIBXUL_ENABLED  1 (line 59) 

to

#ifndef _DEBUG
	#define LL_LIBXUL_ENABLED		1
#else
	#define LL_LIBXUL_ENABLED		0
#endif
  • Compile
    1. select compile for the Debug configuration
    2. have some coffee and/or cigarettes, the build (especially the link step) is excruciatingly slow


Running

  • Run the program under the debugger (F5) or standalone (Ctrl+F5)
  • Running in the debugger during startup you will notice a couple of exceptions when freeing memory. Just continue (these seem to come from the mozilla DLLs).
  • If you encounter crashAndLoop situations (when you hit these, you'll know what it means), consider to delete the contents from the crashAndLoop function and replace it with a single __asm int 3; command. This way the debugger will stop with an exception but you will be able to continue.
  • Finish running the program
  • Look at the leaks.log file in the program folder.


Identifying the source of the leak through leaks.log

Finding leaks is a nice task. Like post mortem debugging in many cases it is puzzle and detective work and the leaks.log file is your forensic evidence.

  • client blocks were allocated through new operations inside the source. Source file and line number will be identified and usually it is not too hard to figure out a program path that bypasses the deletion.
  • normal memory blocks in the leak dump are coming from 3rd party libraries. You'll need creativity to find these (I do it mostly through custom code in crt_alloc_hook in crtdebug.cpp ... see the example there).
  • currently there is a small amount of "unfixable" leaks as normal blocks when the program starts.
    • one is from a technique the Lindens are using in llerror.cpp (see [VWR-849]), which Zero Linden does not want to fix, although I have a patch which I deem safe
    • another is from code in llmozlib which I could not compile yet (these can be avoided by compiling without mozlib, see compile instructions above)
    • I have also found a few that seem to trace down into fmod.lib, but fmod doesn't seem like the cleanest code on the planet anyway.
    • unless you want to be religious about the issue, you can ignore normal blocks with alloc numbers under 1,000,000