Open Testware Reviews

Unit Test Tool Survey

Reviewed: 2003-Apr-24
Testingfaqs.org category: Unit Test Tools

I have a big list for you this time around. So big, I've put it at the bottom of this survey so you don't wear our your scroll bar looking for the commentary. This is also a monumental event - only the second time I've added a new category to the tool list on testingfaqs.org. First, I found all the existing entries on testingfaqs.org that were best categorized as unit test tools but were languishing elsewhere as misfits. That was 14 tools, most of them commercial (the freeware tools that were already listed on testingfaqs.org were JUnit, JSUnit, and MinUnit). Then I started down my long hit list of freeware unit test tools, and after the dust cleared, I had 43 freeware tools to report to you.

Scope

The tools I list below don't represent nearly all of the unit test tools on my hit list. I had to find a way to make this survey a feasible task, so I decided to filter the list. My criteria for inclusion in this survey were:

The license for the tool is clearly specified (it turns out that all of them allow access to the source code),
The language supported by the tool is fairly common.
The tool has a web page home, not just a tar file download.
The tool focuses on low-level functional testing.
The tool hasn't been clearly superseded by a newer tool.

Many of the tools I excluded are listed on the Software Downloads on Ron Jeffries' xprogramming.com, and in fact that's where I found a majority of the tools on my hit list. This page is recognized in the Extreme Programming community as the authoritative list of unit test tools. Tools often get added here shortly after they first appear in alpha form. However, I have found several additional tools that aren't listed on xprogramming.com, and of course I have shared my list with Ron. Note that I did not check the maturity of the tools that I include in this survey - many are alpha or beta quality.

The field of available unit test tools is growing very rapidly, and I'm sure I don't have on my hit list all of the ones that are out there already.

About unit testing

Before I go on, I should explain what I mean by "unit test tool." On testingfaqs.org, I introduce the category like so: "These tools, frameworks, and libraries support unit testing, which is usually performed by the developer, usually using interfaces below the public interfaces of the software under test." The freeware unit test tools are typically written by developers to support their own testing efforts, and they're intended to be used by programmers.

A unit test tool will usually only support one language. I'm not sure why no one has written a unit test library that's intended to be used from a large number of different languages and platforms. Instead, the usual case is that a developer will write a new unit test library for their chosen language that borrows the design from an established tool for another language. In some cases, the motivation is to have an interface that fits naturally for the language at hand. The result is a long list of tools, with different tools for each language.

It bears mentioniong that the tools for Microsoft .NET work for all .NET-enabled languages. There's also the CUT tool that claims to cover C, C++, and Objective-C, and beyond that, it's supposed to work well for embedded systems as well as standard applications. I haven't checked to see how well it stands up to those claims. Note that there are four tools listed that focus on data rather than programs, listed under "content."

Comments on what you'll find here

Many of the tools on the list are patterned after JUnit for Java, which itself was adapted from SUnit. These tools are often collectively referred to as "xUnit" tools. Java still has the broadest unit test support of all programming languages, but others are starting to catch up. I still have hope that I'll find a freeware FortranUnit, too. :-)

One more phenomenon I'd like to point out is the various tools that support "mock objects," a concept that's only a few years old but seems to be spreading rapidly. These tools are all identified in the Notes column below. According to the MockObject wiki page, "mock objects take stubs to an extreme." I encourage you to read more about them if you're doing unit testing.

Rather than blather on further, I'll let you peruse the list and see what's there for yourself. The table below is sorted first by target language, and then by tool name, so you can see all the tools for each language in one block.

And finally, the list

Tool	Target Language	Notes
Aunit	Ada
Check	C
cUnit	C
CUT	C, C++, Objective-C	works in embedded environments
MinUnit	C	designed for embedded systems
CppUnit	C++
PalmUnit	C++	for PalmOS applications
QtUnit	C++	for applications using the Qt library
unit++	C++
HtmlUnit	content - web sites	Java API
HttpUnit	content - web sites	Java API
XMLUnit	content - XML	Java API
XSLTunit	content - XSLT
EasyMock	Java	mock objects
GrandTestAuto	Java
JTestCase	Java	JUnit add-on
JUnit	Java	the parent of most xUnit frameworks
JUnitEE	Java	JUnit add-on
JUnitX	Java	JUnit add-on
Mock Creator	Java	mock objects
Mock Objects	Java	mock objects
MockMaker	Java	mock objects
Mockry	Java	mock objects
jsAsserUnit	JavaScript
JsUnit (Hieatt)	JavaScript	author is Edward Hieatt, note two tools with the same name
JsUnit (Schaible)	JavaScript	author is Jörg Schaible, note two tools with the same name
TagUnit	JSP	custom tags within Java Server Pages
LingoUnit	Macromedia Director
csUnit	.NET
dotunit	.NET
NUnit	.NET
ObjcUnit	Objective-C
OCUnit	Objective-C
Perl Test::MockObject	Perl	mock objects
PerlUnit	Perl
phpAsserUnit	PHP
PhpUnit	PHP
PBUnit	PowerBuilder
PyUnit	Python
Ruby/Mock	Ruby	mock objects
Ruby Test::Unit	Ruby
OTF	Smalltalk
SUnit	Smalltalk	the first xUnit framework, and the parent of JUnit
vbUnit3 Basic	Visual Basic	also has a commercial version