Back when I was programming C++, I worked with a great test tool: Parasoft C++ Test. It was great for auto-generating unit tests, which in my opinion is the best place to start. Even if they need to be modified by a Human to get proper coverage and real-world test results, the ability to generate tests that automatically check for things like NULLs and ridiculous value ranges is a good start to build from.
Even better is the savings in time. Manually typing out all of the tags, code, and various other “boilerplate” pieces is a pain. With automated test generation, all you have to do is go in afterward and add rows for the tests you want to run.
This is especially helpful for larger projects (more than 100K lines of code), such as Basternae.
After creating a test project, I told Visual Studio 2008 to auto-generate tests for me. It took about twenty minutes with the processor thinking at full speed. I can’t imagine how long it would have taken a human, but since the Human brain functions at about 300Hz it would have taken a mighty long while.
After auto-generating 1,966 tests, I decided to run them. It got just over a quarter of the way through with *LOTS* of failures (approximately none of this new code has ever been tested) and Visual Studio crashed.
One Visual Studio restart later and I was running tests again, with a massive amount of failures.
In the “CharData” class alone, 212 tests were generated and 199 of them threw exceptions on the first run. The rest were “Inconclusive”, which is the default result for auto-generated tests.
Already, Mr. Pex has made it obvious that the application has some architectural flaws, chief among which is the random number generator. The generator lives in an instance of the “SystemData” class when it should be in the “RandomNumber” class, which should be a singleton. When running without the controlled startup environment of the entire codebase, things break horribly. That’s one of the things a test framework is supposed to point out, right?
While automated test generation is not all-inclusive, Pex is extremely helpful for getting to the point where your tests are at least “Inconclusive” and ready to edit by a Human. If you get exceptions without even getting that far, you have some work to do (as I have).
I have seen no better tool for getting a quick and easy start on unit testing. Writing meaningful tests? Well, that’s something you have to figure out on your own, but at least the boilerplate is done for you.
Here’s where you can get more info on Pex, including a video demonstration:
http://research.microsoft.com/en-us/projects/Pex/
It is worth noting that this Pex is extremely beta. Every time I run it, I get the “VSTestHost.exe has stopped working” dialog box. Even with that, all of the tests do manage to run.
