Unit Tests

Reasoning

Many unit test advocates will attempt to convince you that unit tests will always save you time. This is not strictly true, and not really the reason you want to use unit tests. In an environment where the code base is highly interdependent, rapidly changing, and expected to provide consistent functionality across years of use, unit tests are one of the few reliable and scalable ways to ensure the code continues to function across time and platforms. Unit tests in Linden will facilitate and provide:

Confidence when changing a core component, the the end behavior is consistent or changes are detected and addressed appropriately.
A wide variety of code examples for using our classes and functions.
Confidence that the code continues to function across third party library, system library, or system image upgrades.

Our Framework

We have a set of tests located in indra/test which are built and run automatically as the tests_ok target in the cmake build. The test framework is currently only set up to test the library functionality. We use the TUT Framework since it requires no runtime library, and was incredibly simple to integrate. There are special linden enhancements to the tut framework available in indra/test/lltut.h.

When to write them

Write tests under any or all of these conditions:

After writing your functionality declaration but before implementation. At this point, all tests will fail, and you know you are done when all tests pass.
After you believe a class or library is complete. This is a good time to explore the edge cases.
When a bug has been reported, and you need to make the smallest possible piece of code which exposes the bug.

To add a new test

The framework uses template meta-programming to do automatic registration of test functions. Add a new function as a method of the local tut::test_group<local_data> class with a incremented number as the template parameter. In general, you should always append tests, and try to have a limited number of calls to any of the ensure* functions inside the test function.

namespace tut
{
    ... Test group stuff

    ... Other tests

    template<> template<>
    void math_object::test<7>()
    {
        ...
        ensure("new test", (...));
    }
}

To add a new test group

Inside the namespace declaration, instantiate a test_group<test_data> object where test_data is a class which will have necessary information for most of the test calls, eg, an open file handle. The constructor for the test_data object will initialize (ala setUp()) and the destructor will free that data as necessary (ala tearDown()). You can provide an empty struct if you have no data. All instance members of the test_data will be available on the stack as newly generated objects in every call to test(). Each test group is limited to 50 actual test methods unless you make a special declaration. For example:

namespace tut
{
    struct uuid_data
    {
        LLUUID id;
    };
    typedef test_group<uuid_data> uuid_test;
    typedef uuid_test::object uuid_object;
    tut::uuid_test tu("uuid");

    template<> template<>
    void uuid_object::test<1>()
    {
        ensure("uuid null", id.isNull());
        id.generate();
        ensure("generate not null", id.notNull());
        id.setNull();
        ensure("set null", id.isNull());
    }
}

To add new test suites

You need to simply create a new file for the test suites and add a group and then tests. For example, math.cpp tests the llmath library and currently has 2 test groups, each with a few test() functions.

Needed Improvements

I added an ensure_not_equals() function in lltut.h since I felt that was necessary. More ensure functions should be written:
- ensure_approximately_equals()
- ensure_memory_matches()
- ensure_equals<A,B,Compare_fn> () { if(compare(a,b))...}
More tests! I only wrote a few to make sure it worked and was fairly easy to use.
The test runner needs to have a few more commands and options since it is possible to only run certain test groups. Eg, `./test --group=uuid` could be wired to run the uuid tests.
The test runner needs more optional controls on the output. The output is generated through callbacks, so an enterprising programmer that loves GUIs could even write a progress bar output.

Unit tests

Contents