JUL
23
2004

Testing your code

This is another blog entry from the series of "how to improve my code". I was very happy with the response I got to the "delayed initialization" entry. Today I'll show you how to prove your code is working or at least doing what's expected. I'll talk a little bit about unit testing. If you hate "extreme programming" or "test driven development" bare with me as I'll show you how to very quickly and easily write tests.

There's quite a few frameworks that were designed to ease testing. If you're into testing you probably have your favorite one. If you're researching those framework there are two things which you should look at foremost:

  1. How easy it is to add new tests. This one is very important because in reality no one wants to write tests and if it takes a lot of code no one will. So the less code to write the better.
  2. How flexible is the testing functionality there. Many frameworks allow you to compare only strings. You don't want that, you want to be comparing many different types, dependingly on the test.



Personally I didn't find a framework that would nicely integrate with KDE and C++ so I wrote my own. Enter KUnitTest. For now I licensed it under BSD license but I might consider putting it under public domain since it looks like we'll be using it in some commercial products.

KUnitTest is a very, very small library consisting of the following parts:

  • kunittest.{h,cpp} - it's the test factory, holds all tests and runs
    them,
  • tester.h - which holds the base of a pure test object. "Pure test object" in the KUnitTest is one that doesn't require Qt event loop to run.
  • qtester.{h,cpp} - which holds the base of a test object which requires Qt event loop to perform its tests.


Now lets see how you would add a new test to KUnitTest. You do that by writting a Tester derived class which has the "allTests()" method. Like so:

class SampleTest : public Tester
{
public:
SampleTest();

public:
void allTests();
};


Now in the allTests method we implement our tests. Which for example look as follows:

void SampleTest::allTests()
{
CHECK( 3+3, 6 );
CHECK( QString( "hello%1" ).arg( " world not" ),
QString( "hello world" ) );
}

the check method is template based so as long as the variables which you pass to it have an equals ('==' ) operator you will get a type safe comparison.

Now that you did that the only other thing to do is in the kunittest.cpp add

ADD_TEST( SampleTest );

call in the "void registerTests()" method.

You're done. KUnitTest will do the rest. It will tell you which tests failed, how, what was the expected result, what was the result it got, what was the code that failed and so on. For example for the code above it would output:

tests: SampleTest errors:
tests:  sampletest.cpp[38]: failed on "QString( "hello%1" ).arg( " world not" )"
tests:           result = 'hello world not', expected = 'hello world'


Finally, how to integrate kunittest with your app/lib? What you do is copy the following files: main.cpp, tester.h, kunittest.h, kunittest.cpp, qtester.h, qtester.cpp and Makefile.am to your test directory. Then you start writting tests and run 'make check' to see what changed. Have fun.

If you have ideas on how to make it even more simpler let me know! (I will not respond to anyone who will want testing units to not be classes though :) )

Comments

... inside of KDevelop. As a part the test could be automaticly executed as a target, so developers wouldnt have to dork with their build system. The hardest thing about testing in OSS is that in OSS developers tend to do what they think is fun. Messing with CPP unit is NOT fun, but messing around in GDB is even MORE NOT fun.

With small easy to implement unit tests like this while the tests are not fun, writing 15 lines of code is painless enough that the gain is worth it.

kudos zack ;)


By Ian Reinhart Geiser at Fri, 07/23/2004 - 01:10

geiseri did you look at this?
http://www.informatik.uni-oldenburg.de/~rischwa/

and Zack nice work


By Mathieu Chouinard at Fri, 07/23/2004 - 12:45

It is good to know that KDE finally has a unit test library. I was planning to write one for some time but I really lack spare time.

I have been using cppunit for some time, with some of my own extensions.

Cppunit is quite good. But some things are not perferct. Your thing looks quite good. The most annoying thing with CppUnit is that you have to declare a the Test class, the test method and then explicitely add them.

What rocks with the equivalent code in python or java is that the introspection features are used to construct the test suite. All you have to do is to declare a class whose name begins with Test and the magic will happen, you get the Test suite constructed automatically. I was thinking of doing something similar using Qt's introspection features.

I strongly advise you to look at cppunit to improve your test library. The cool things that cppunit has that you are missing are:
- the program flow is interrupted when a test fails: this is important because if you check that a pointer is not null in some place, you want the program to stop immediately if the pointer is null. They do that by raising an exception.
- automatic aggregating of test classes into a test suite. The have a special macro that you can can use to declare your test class in a .cpp file. Then, a global test suite will be automatically constructed with all your declared classes. This is quite cool for lazy people like me.
- setup / teardown : this is fundamental as soon as you start running multiple tests
- multiple types of output possible

What rocks is when all your tests are compiled into a DLL. You can then have a Graphical test runner, run, fix, recompile, re-run without quitting your graphical runner.

Here is my latest use of cppunit. You can see what it looks like.

The important thing with a unit test library is that it must scale. Currently in my projects, I have around 100 tests in 10 classes. I can test very advanced stuff. It is important that I can run all the tests at once, and get a report of all the failing tests.

Do not hesitate to contact me if you want some input or feedback on your stuff.

I would really love to see KDE moving on to unit testing. For information, I have very very low bug rate since I use unit testing everywhere in my projects. The development time has not changed but I have been able to catch far many tricky bugs than before.

After unit testing, the next step is mock. But one thing at a time...


By pfremy at Fri, 07/23/2004 - 09:30

Hi!

I just wrote my own unittesting "framework" and a kdevelop plugin, to automatically create "testtemplates" for classes.
I'm planning to extend the plugin to support other frameworks, too.
The framework itself isn't yet finished, the next thing i'll implement a way to load "outputplugins" for different outputs.
You can take a look at it(there is even a very small handbook) at
www.informatik.uni-oldenburg.de/~rischwa

Jonas


By rischwa at Fri, 07/23/2004 - 13:31

I like the KDevelop plugin, it's a very nice idea :) I'm not a big fan of the rest of it though. Furthermore you licensed it under GPL so it limits its use.


By zack rusin at Sun, 07/25/2004 - 00:59

I wonder if a few extensions could be made to this to make it even cooler? I'm thinking of a way to make an assertion about the value of a property of a QObject, and maybe even the possibility of dynamically loading tests defined in KJSEmbed. Do you think these would be possible?


By Richard Moore at Fri, 07/23/2004 - 20:35

I guess you didn't find Tim Jansen's work in kdenonbeta. Or was there something you didn't like?


By Mark Bucciarelli at Sat, 07/24/2004 - 23:24

I knew about it. It's very nice but I'm far from ever wanting to include kdeutil to just get a testing framework.


By zack rusin at Sun, 07/25/2004 - 00:54