Difference between revisions of "How to add unit tests to C++ code"

From Second Life Wiki
Jump to navigation Jump to search
 
(13 intermediate revisions by 2 users not shown)
Line 2: Line 2:


== Overview ==
== Overview ==
Tests go in a tests/ subdir of the project with the specific naming convention codefilename_test.cpp. The test code itself should use our basic tut template (which as of 2009-04 is somewhat in flux). Add a testing target to the bottom of the project using the cmake command LL_ADD_PROJECT_UNIT_TESTS(project sourcelist).
Tests for a file in a project go into the tests/ subdirectory of that project with the specific naming convention <code>''code_filename''_test.cpp</code>. The test code itself should use our basic tut template (which as of 2009-04 is somewhat in flux). Add a testing target to the bottom of the project using the cmake command LL_ADD_PROJECT_UNIT_TESTS(project sourcelist).


DO NOT add test code that:
DO NOT add test code that:
Line 12: Line 12:


== The unit test template ==
== The unit test template ==
Copy this to a file indra/project/tests/codefile_test.cpp and follow the next section to make sure the build runs with it before you begin writing your test.
Copy this to a file <code>indra/''project_name''/tests/''code_filename''_test.cpp</code> and follow the next section to make sure the build runs with it before you begin writing your test. (You should expect link errors to appear once you've successfully started building the test.)


<pre>
<pre>
Line 22: Line 22:
namespace tut
namespace tut
{
{
// Main Setup
struct LLClassNameFixture
struct LLClassNameFixture
{
{
Line 30: Line 31:
typedef test_group<LLClassNameFixture> factory;
typedef test_group<LLClassNameFixture> factory;
typedef factory::object object;
typedef factory::object object;
}
factory tf("LLClassName");
namespace
{
tut::factory tf("LLClassName test");
}


namespace tut
// Tests
{
template<> template<>
template<> template<>
void object::test<1>()
void object::test<1>()
Line 61: Line 57:
LL_ADD_PROJECT_UNIT_TESTS(chewbacca "${chewbacca_TESTED_SOURCE_FILES}")
LL_ADD_PROJECT_UNIT_TESTS(chewbacca "${chewbacca_TESTED_SOURCE_FILES}")
</pre>
</pre>
=== Extended example ===
The "Chewbacca! What a Wookie!" project code is available on bitbucket ( http://bitbucket.org/poppy_linden/unit-testing-infrastructure-test/ ) and instructions for using it are available on the [[How to test unit test infrastructure changes]] page. Download and run that example, the code will demonstrate the bare basics if the newview code proves too dense to understand. Notably, there are no actual tests, just the scaffolding to get the unit test into the build.


== How to unit test your indra code ==
== How to unit test your indra code ==
Insert Merov's doc here
We've covered the basics of setup, but not the process of writing the tests themselves.
* [[C++ Unit Testing - How It Works]] - how the test framework generates and builds test projects.
* [[C++ Unit Testing - Case Study]] - a walkthrough of the whole process (from setup to fixing link errors to writing tests), by example.  Note that this predates Google Mock ([http://code.google.com/p/googlemock/wiki/ForDummies tutorial here]), which provides another way of fixing up link errors.
* [[C++ Unit Testing - FAQ]]
 
The following is still relevant, but more focused on overall structure of code than what to write.
 
== To add a new test ==
The framework uses template meta-programming to do automatic registration of test functions. Add a new function as a method of the local tut::test_group<local_data> class with a incremented number as the template parameter. In general, you should always append tests, and try to have a limited number of calls to any of the ensure* functions inside the test function.
<code>
<pre>
namespace tut
{
    ... Test group stuff
 
    ... Other tests
 
    template<> template<>
    void math_object::test<7>()
    {
        ...
        ensure("new test", (...));
    }
}
</pre>
</code>
 
== To add a new test group ==
Inside the namespace declaration, instantiate a test_group<test_data> object where test_data is a class which will have necessary information for most of the test calls, eg, an open file handle. The constructor for the test_data object will initialize (ala setUp()) and the destructor will free that data as necessary (ala tearDown()). You can provide an empty struct if you have no data. All instance members of the test_data will be available on the stack as newly generated objects in every call to test(). Each test group is limited to 50 actual test methods unless you make a special declaration. For example:
<code>
<pre>
namespace tut
{
    struct uuid_data
    {
        LLUUID id;
    };
    typedef test_group<uuid_data> uuid_test;
    typedef uuid_test::object uuid_object;
    tut::uuid_test tu("uuid");
 
    template<> template<>
    void uuid_object::test<1>()
    {
        ensure("uuid null", id.isNull());
        id.generate();
        ensure("generate not null", id.notNull());
        id.setNull();
        ensure("set null", id.isNull());
    }
}
</pre>
</code>
 
== To add new test suites ==
You need to simply create a new file for the test suites and add a group and then tests. For example, math.cpp tests the llmath library and currently has 2 test groups, each with a few test() functions.
== Needed Improvements ==
* I added an ensure_not_equals() function in lltut.h since I felt that was necessary. More ensure functions should be written:
** ensure_approximately_equals()
** ensure_memory_matches()
** ensure_equals<A,B,Compare_fn> () { if(compare(a,b))...}
* More tests! I only wrote a few to make sure it worked and was fairly easy to use.
* The test runner needs to have a few more commands and options since it is possible to only run certain test groups. Eg, `./test --group=uuid` could be wired to run the uuid tests.
* The test runner needs more optional controls on the output. The output is generated through callbacks, so an enterprising programmer that loves GUIs could even write a progress bar output.
 
 
[[Category:Automated Testing]]

Latest revision as of 22:26, 24 August 2010

The indra C++ codebase is fraught with much peril. To reduce the amount of risk associated with refactoring legacy code, use unit tests. Here's how to use the new LL_ADD_PROJECT_UNIT_TESTS cmake macro and the existing tut test infrastructure to add a test to the build.

Overview

Tests for a file in a project go into the tests/ subdirectory of that project with the specific naming convention code_filename_test.cpp. The test code itself should use our basic tut template (which as of 2009-04 is somewhat in flux). Add a testing target to the bottom of the project using the cmake command LL_ADD_PROJECT_UNIT_TESTS(project sourcelist).

DO NOT add test code that:

  • talks to a database.
  • communicates across a network.
  • touches the file system.
  • requires doing special things to the environment (such as editing configuration files) to run it.
  • takes longer than about ~.1s to run on a modern computer.

The unit test template

Copy this to a file indra/project_name/tests/code_filename_test.cpp and follow the next section to make sure the build runs with it before you begin writing your test. (You should expect link errors to appear once you've successfully started building the test.)

#include "linden_common.h"
#include "lltut.h"

#include "llclassname.h"

namespace tut
{
	// Main Setup
	struct LLClassNameFixture
	{
		LLClassNameFixture()
		{
		}
	};
	typedef test_group<LLClassNameFixture> factory;
	typedef factory::object object;
	factory tf("LLClassName");

	// Tests
	template<> template<>
	void object::test<1>()
	{
		
	}
}

Code to make the unit test build

There is a macro that takes care of adding the proper testing targets to the build, you merely need to supply source files and a project name.

Basic example

This would go at the bottom of CMakeLists.txt for a project called "chewbacca". The exact quoting is important! CMake is very particular about list variables.

INCLUDE(LLAddBuildTest)
# NOTE: this is different from project_SOURCE_FILES because not all source has tests.
set(chewbacca_TESTED_SOURCE_FILES
  chewbacca.cpp
  person.cpp
  )
LL_ADD_PROJECT_UNIT_TESTS(chewbacca "${chewbacca_TESTED_SOURCE_FILES}")

Extended example

The "Chewbacca! What a Wookie!" project code is available on bitbucket ( http://bitbucket.org/poppy_linden/unit-testing-infrastructure-test/ ) and instructions for using it are available on the How to test unit test infrastructure changes page. Download and run that example, the code will demonstrate the bare basics if the newview code proves too dense to understand. Notably, there are no actual tests, just the scaffolding to get the unit test into the build.

How to unit test your indra code

We've covered the basics of setup, but not the process of writing the tests themselves.

The following is still relevant, but more focused on overall structure of code than what to write.

To add a new test

The framework uses template meta-programming to do automatic registration of test functions. Add a new function as a method of the local tut::test_group<local_data> class with a incremented number as the template parameter. In general, you should always append tests, and try to have a limited number of calls to any of the ensure* functions inside the test function.

namespace tut
{
    ... Test group stuff

    ... Other tests

    template<> template<>
    void math_object::test<7>()
    {
        ...
        ensure("new test", (...));
    }
}

To add a new test group

Inside the namespace declaration, instantiate a test_group<test_data> object where test_data is a class which will have necessary information for most of the test calls, eg, an open file handle. The constructor for the test_data object will initialize (ala setUp()) and the destructor will free that data as necessary (ala tearDown()). You can provide an empty struct if you have no data. All instance members of the test_data will be available on the stack as newly generated objects in every call to test(). Each test group is limited to 50 actual test methods unless you make a special declaration. For example:

namespace tut
{
    struct uuid_data
    {
        LLUUID id;
    };
    typedef test_group<uuid_data> uuid_test;
    typedef uuid_test::object uuid_object;
    tut::uuid_test tu("uuid");

    template<> template<>
    void uuid_object::test<1>()
    {
        ensure("uuid null", id.isNull());
        id.generate();
        ensure("generate not null", id.notNull());
        id.setNull();
        ensure("set null", id.isNull());
    }
}

To add new test suites

You need to simply create a new file for the test suites and add a group and then tests. For example, math.cpp tests the llmath library and currently has 2 test groups, each with a few test() functions.

Needed Improvements

  • I added an ensure_not_equals() function in lltut.h since I felt that was necessary. More ensure functions should be written:
    • ensure_approximately_equals()
    • ensure_memory_matches()
    • ensure_equals<A,B,Compare_fn> () { if(compare(a,b))...}
  • More tests! I only wrote a few to make sure it worked and was fairly easy to use.
  • The test runner needs to have a few more commands and options since it is possible to only run certain test groups. Eg, `./test --group=uuid` could be wired to run the uuid tests.
  • The test runner needs more optional controls on the output. The output is generated through callbacks, so an enterprising programmer that loves GUIs could even write a progress bar output.