Difference between revisions of "Unit tests"

Latest revision as of 17:04, 25 June 2009

Why unit test?

To reduce the amount of risk associated with refactoring legacy code, and to prove the functionality and interface of new code, use unit tests.

In an environment where the code base is highly interdependent, rapidly changing, and expected to provide consistent functionality across years of use, unit tests are one of the few reliable and scalable ways to ensure the code continues to function across time and platforms.

Unit tests facilitate and provide:

Confidence when changing a component that the end behavior is consistent or changes are detected and addressed appropriately.
A wide variety of code examples for using our classes and functions.
Confidence that the code continues to function across third party library, system library, or system image upgrades.

Our Frameworks

We have several types of unit tests, each with their own mechanisms.

Project-level C++ unit tests
System-level C++ unit tests
Project-level python unit tests
System-level python unit tests
(more?)

Details on how to run, write, and build each are as follows.

Project-level C++ unit tests

Please see

How to add unit tests to C++ code

for the basic tutorial. For a step-by-step narration, please see

System-level C++ unit tests

This method of unit testing is deprecated. Please migrate any tests you have in indra/test/ using the project-level C++ test method. If you can't do that, consider making the test into an integration test.

We have a set of tests located in indra/test which are built and run automatically as the tests_ok target in the cmake build. The test framework is currently only set up to test the library functionality. We use the TUT Framework since it requires no runtime library, and was incredibly simple to integrate. There are special linden enhancements to the tut framework available in indra/test/lltut.h.

Project-level python unit tests

Documentation forthcoming.

System-level python unit tests

These are started by indra/test/test.py.

When to write unit tests

Write tests under any or all of these conditions:

After writing your functionality declaration but before implementation. At this point, all tests will fail, and you know you are done when all tests pass.
After you believe a class or library is complete. This is a good time to explore the edge cases.
When a bug has been reported, and you need to make the smallest possible piece of code which exposes the bug.

Thanks for testing!

Need to change the framework?

There's a testplan and code to help you test your changes at How to test unit test infrastructure changes.

Useful Links

The LL Unit Test page
The tut framework doc
Exploring the C++ Unit Testing Framework Jungle
Working Effectively with Legacy Code by Michael Feathers: Much of that book discusses techniques for minimizing the risk of teasing apart difficult classes in order to test them.

@@ Line 1: / Line 1: @@
-= Unit Tests =
+== Why unit test? ==
-== Reasoning  ==
+To reduce the amount of risk associated with refactoring legacy code, and to prove the functionality and interface of new code, use unit tests.
-Many unit test advocates will attempt to convince you that unit tests will always save you time. This is not strictly true, and not really the reason you want to use unit tests. In an environment where the code base is highly interdependent, rapidly changing, and expected to provide consistent functionality across years of use, unit tests are one of the few reliable and scalable ways to ensure the code continues to function across time and platforms.
-Unit tests in Linden will facilitate and provide:
+In an environment where the code base is highly interdependent, rapidly changing, and expected to provide consistent functionality across years of use, unit tests are one of the few reliable and scalable ways to ensure the code continues to function across time and platforms.
-* Confidence when changing a core component, the the end behavior is consistent or changes are detected and addressed appropriately.
+Unit tests facilitate and provide:
+* Confidence when changing a component that the end behavior is consistent or changes are detected and addressed appropriately.
 * A wide variety of code examples for using our classes and functions.
 * Confidence that the code continues to function across third party library, system library, or system image upgrades.
-== Our Framework ==
-We have a set of tests located in indra/test which are built and run automatically on linux and windows and are available in the MacTester project on the mac.
+== Our Frameworks ==
+We have several types of unit tests, each with their own mechanisms.
+* Project-level C++ unit tests
+* System-level C++ unit tests
+* Project-level python unit tests
+* System-level python unit tests
+* (more?)
+Details on how to run, write, and build each are as follows.
+=== Project-level C++ unit tests ===
+Please see
+* [[How to add unit tests to C++ code]]
+for the basic tutorial. For a step-by-step narration, please see
+* [[C++ Unit Testing - How It Works]]
+* [[C++ Unit Testing - Case Study]]
+* [[C++ Unit Testing - FAQ]]
+=== System-level C++ unit tests ===
+This method of unit testing is deprecated. Please migrate any tests you have in indra/test/ using the project-level C++ test method. If you can't do that, consider making the test into an integration test.
+We have a set of tests located in indra/test which are built and run automatically as the tests_ok target in the cmake build.
 The test framework is currently only set up to test the library functionality.
 We use the [http://tut-framework.sourceforge.net/ TUT Framework] since it requires no runtime library, and was incredibly simple to integrate.
 There are special linden enhancements to the tut framework available in indra/test/lltut.h.
-== When to write them ==
+=== Project-level python unit tests ===
+Documentation forthcoming.
+=== System-level python unit tests ===
+These are started by indra/test/test.py.
+== When to write unit tests ==
 Write tests under any or all of these conditions:
 * After writing your functionality declaration but before implementation. At this point, all tests will fail, and you know you are done when all tests pass.
 * After you believe a class or library is complete. This is a good time to explore the edge cases.
 * When a bug has been reported, and you need to make the smallest possible piece of code which exposes the bug.
-== To add a new test ==
-The framework uses template meta-programming to do automatic registration of test functions. Add a new function as a method of the local tut::test_group<local_data> class with a incremented number as the template parameter. In general, you should always append tests, and try to have a limited number of calls to any of the ensure* functions inside the test function.
-<code>
-<pre>
-namespace tut
-{
-    ... Test group stuff
-    ... Other tests
+Thanks for testing!
+== Need to change the framework? ==
+There's a testplan and code to help you test your changes at [[How to test unit test infrastructure changes]].
-    template<> template<>
+== Useful Links ==
-    void math_object::test<7>()
+* [http://wiki.secondlife.com/wiki/Unit_tests The LL Unit Test page]
-    {
+* [http://tut-framework.sourceforge.net/ The tut framework doc]
-        ...
+* [http://gamesfromwithin.com/?p=29 Exploring the C++ Unit Testing Framework Jungle]
-        ensure("new test", (...));
+* [http://www.amazon.com/Working-Effectively-Legacy-Robert-Martin/dp/0131177052/ref=sr_1_1?ie=UTF8&s=books&qid=1234193739&sr=1-1 Working Effectively with Legacy Code] by Michael Feathers: Much of that book discusses techniques for minimizing the risk of teasing apart difficult classes in order to test them.
-    }
-}
-</pre>
-</code>
-== To add a new test group ==
-Inside the namespace declaration, instantiate a test_group<test_data> object where test_data is a class which will have necessary information for most of the test calls, eg, an open file handle. The constructor for the test_data object will initialize (ala setUp()) and the destructor will free that data as necessary (ala tearDown()). You can provide an empty struct if you have no data. All instance members of the test_data will be available on the stack as newly generated objects in every call to test(). Each test group is limited to 50 actual test methods unless you make a special declaration. For example:
-<code>
-<pre>
-namespace tut
-{
-    struct uuid_data
-    {
-        LLUUID id;
-    };
-    typedef test_group<uuid_data> uuid_test;
-    typedef uuid_test::object uuid_object;
-    tut::uuid_test tu("uuid");
-    template<> template<>
+[[Category:Automated Testing]]
-    void uuid_object::test<1>()
-    {
-        ensure("uuid null", id.isNull());
-        id.generate();
-        ensure("generate not null", id.notNull());
-        id.setNull();
-        ensure("set null", id.isNull());
-    }
-}
-</pre>
-</code>
-== To add new test suites ==
-You need to simply create a new file for the test suites and add a group and then tests. For example, math.cpp tests the llmath library and currently has 2 test groups, each with a few test() functions.
-== Needed Improvements ==
-* I added an ensure_not_equals() function in lltut.h since I felt that was necessary. More ensure functions should be written:
-** ensure_approximately_equals()
-** ensure_memory_matches()
-** ensure_equals<A,B,Compare_fn> () { if(compare(a,b))...}
-* More tests! I only wrote a few to make sure it worked and was fairly easy to use.
-* The test runner needs to have a few more commands and options since it is possible to only run certain test groups. Eg, `./test --group=uuid` could be wired to run the uuid tests.
-* The test runner needs more optional controls on the output. The output is generated through callbacks, so an enterprising programmer that loves GUIs could even write a progress bar output.