C++ Unit Testing - How It Works

From Second Life Wiki
Revision as of 17:58, 25 June 2009 by Poppy Linden (Talk | contribs)

Jump to: navigation, search

There is nothing more practical than a good theory Attributed to James Maxwell

It's worth knowing the machinery that compiles and executes unit tests in our build system. This involves traversing CMake and the resulting build files (Visual Studio Solution, Makefile, Xcode project, etc) and discovering assumptions made by the various parts of the build system. Understanding these mechanisms will help point you in the right direction if something unusual happens (as it will) or if you need to diverge from the path detailed in the Indra Unit Testing - Case Study (which is likely).

Suppose you checkout a fresh new branch from svn. You're ready to build the whole thing (we assume you know how to do that). What happens with regards to unit tests?

CMake

indra/develop.py runs cmake to create all the build targets we have. To do its magic, cmake gathers all the CMakeList.txt files spread throughout the source code tree that describe what to build, uses a bunch of rules stored in cmake/*.cmake files and creates an indra/build-platform folder (where platform is vc80, darwin-i386, etc) that contains the build files you need to compile and link.

From a unit tests standpoint, this is when the unit test target projects are being created and inserted in the solution. If something goes wrong here, you're already screwed as far as unit tests are concerned so it's better to understand what happens.

Each unit test is added thanks to a macro called LL_ADD_PROJECT_UNIT_TESTS (or something similar as we'll see later) located at the end of the CMakeList.txt of each project. Something looking like this:

set(project_TESTED_SOURCE_FILES
  someclass.cpp
  foo.cpp
  )
LL_ADD_PROJECT_UNIT_TESTS(project "${project_TESTED_SOURCE_FILES}")

It's short and sweet and some would say it's all you need to know. But, for this to work, the macro makes a lot of assumptions on how tests are put together.

The LL_ADD_PROJECT_UNIT_TESTS cmake macro (found in cmake/LLAddBuildTest.cmake) does the following:

  • takes the source files argument and picks up the cpp files (which contain the classes to test), then adds a "_test.cpp" suffix to them. Thus, it picks up a tests/foo_test.cpp file (the file that contains the unit test code for foo.cpp).
  • creates new targets in the solution file:
    • a PROJECT_project_TEST_foo target that builds the test executable using foo.cpp, tests/foo_test.cpp and some tut files as source code
    • a project_tests project that calls and executes all the listed tests when building the project target
  • adds a dependency between the tests and the project passed as argument so that those tests gets built and executed when the project is built

As you can see, this machinery add constraints on how you need to organize and implement your tests:

  • You pass LL_ADD_PROJECT_UNIT_TESTS the cpp file for the class you want to test. In someproj/CMakeLists.txt, LL_ADD_PROJECT_UNIT_TESTS(someproj foo.cpp) specifies a source file someproj/foo.cpp.
  • The cpp file containing your tests must be called tests/foo_test.cpp. This is hard coded in the LL_ADD_PROJECT_UNIT_TESTS macro. Deviations (like "tests.cpp", "_tut.cpp", "_test1.cpp", etc...) will fail.
  • The foo_test.cpp file must be stored in the someproj/tests subdirectory. So, if you're the first person in your branch to create a test in newview, you'll have to create a newview/tests folder.
  • Your unit test must be associated with a project target, typically the project identified at the top of the CMakeLists.txt file. For the SL viewer for instance, in newview/CMakeLists.txt, this target is viewer: project(viewer).
    • If CMake complains of a circular dependency, please examine it closely. It is currently known that llmath and llcommon cannot be tested because of lltut dependencies, which are being removed in the near future. Otherwise, bring it up on SLDev.

Build System

Once the CMake step is successful, you build the project using your system's build tools. (We suppose here again you know how to do that.) What does that mean for unit tests?

As mentioned above, CMake creates 2 subprojects in the project file:

  • PROJECT_project_TEST_foo
  • project_tests

When building the whole solution, PROJECT_project_TEST_foo gets compiled and linked and project_tests does some magic to get the test executed and the output (fail/pass) passed to the overall report.

You can compile and run the tests individually which is handy when writing your own or trying to find out why your code is failing tests. (You should always do a full build to run all the tests before committing code.) Several methods:

  • In your build system, set the PROJECT_project_TEST_foo target and build it and run it
  • Select the project_tests project, and build it. Building that target actually executes the test code.
  • Under the command line, cd to indra and use develop.py to "build" the PROJECT_project_TEST_foo target, i.e.:
./develop.py build PROJECT_project_TEST_foo

We used only the second solution while implementing the test case.

For example, in windows you get this result in your VS Output tab, something looking like:

1>------ Build started: Project: foo_test_ok, Configuration: RelWithDebInfo Win32 ------
1>Generating foo_test_ok.txt
1>Total Tests:   6
1>Passed Tests: 6
1>Total Tests:   6
1>Passed Tests: 6
1>Build log was saved at "file://c:\Develop\http-texture-7\indra\build-vc80\newview\foo_test_ok.dir\RelWithDebInfo\BuildLog.htm"
1>foo_test_ok - 0 error(s), 0 warning(s)
========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========

That's when everything goes right of course. It just tells you that 6 unit tests were found and passed in the foo_test.cpp file.

Sounds like a good time now to write your own unit tests. Please go to Indra Unit Testing - Case Study.