Open Testware Reviews

Unit Test Tool Survey

Copyright 2003 by Tejas Software Consulting - All rights reserved.

Reviewed: 2003-Apr-24
Testingfaqs.org category: Unit Test Tools

I have a big list for you this time around. So big, I've put it at the bottom of this survey so you don't wear our your scroll bar looking for the commentary. This is also a monumental event - only the second time I've added a new category to the tool list on testingfaqs.org. First, I found all the existing entries on testingfaqs.org that were best categorized as unit test tools but were languishing elsewhere as misfits. That was 14 tools, most of them commercial (the freeware tools that were already listed on testingfaqs.org were JUnit, JSUnit, and MinUnit). Then I started down my long hit list of freeware unit test tools, and after the dust cleared, I had 43 freeware tools to report to you.

Scope

The tools I list below don't represent nearly all of the unit test tools on my hit list. I had to find a way to make this survey a feasible task, so I decided to filter the list. My criteria for inclusion in this survey were:
Many of the tools I excluded are listed on the Software Downloads on Ron Jeffries' xprogramming.com, and in fact that's where I found a majority of the tools on my hit list. This page is recognized in the Extreme Programming community as the authoritative list of unit test tools. Tools often get added here shortly after they first appear in alpha form. However, I have found several additional tools that aren't listed on xprogramming.com, and of course I have shared my list with Ron. Note that I did not check the maturity of the tools that I include in this survey - many are alpha or beta quality.

The field of available unit test tools is growing very rapidly, and I'm sure I don't have on my hit list all of the ones that are out there already.

About unit testing

Before I go on, I should explain what I mean by "unit test tool." On testingfaqs.org, I introduce the category like so: "These tools, frameworks, and libraries support unit testing, which is usually performed by the developer, usually using interfaces below the public interfaces of the software under test." The freeware unit test tools are typically written by developers to support their own testing efforts, and they're intended to be used by programmers.

A unit test tool will usually only support one language. I'm not sure why no one has written a unit test library that's intended to be used from a large number of different languages and platforms. Instead, the usual case is that a developer will write a new unit test library for their chosen language that borrows the design from an established tool for another language. In some cases, the motivation is to have an interface that fits naturally for the language at hand. The result is a long list of tools, with different tools for each language.

It bears mentioniong that the tools for Microsoft .NET work for all .NET-enabled languages. There's also the CUT tool that claims to cover C, C++, and Objective-C, and beyond that, it's supposed to work well for embedded systems as well as standard applications. I haven't checked to see how well it stands up to those claims. Note that there are four tools listed that focus on data rather than programs, listed under "content."

Comments on what you'll find here

Many of the tools on the list are patterned after JUnit for Java, which itself was adapted from SUnit. These tools are often collectively referred to as "xUnit" tools. Java still has the broadest unit test support of all programming languages, but others are starting to catch up. I still have hope that I'll find a freeware FortranUnit, too. :-)

One more phenomenon I'd like to point out is the various tools that support "mock objects," a concept that's only a few years old but seems to be spreading rapidly. These tools are all identified in the Notes column below. According to the MockObject wiki page, "mock objects take stubs to an extreme." I encourage you to read more about them if you're doing unit testing.

Rather than blather on further, I'll let you peruse the list and see what's there for yourself. The table below is sorted first by target language, and then by tool name, so you can see all the tools for each language in one block.

And finally, the list 

Tool
Target Language
Notes
Aunit Ada

Check
C

cUnit
C

CUT
C, C++, Objective-C
works in embedded environments
MinUnit
C
designed for embedded systems
CppUnit
C++

PalmUnit
C++
for PalmOS applications
QtUnit
C++
for applications using the Qt library
unit++
C++

HtmlUnit
content - web sites Java API
HttpUnit
content - web sites
Java API
XMLUnit
content - XML
Java API
XSLTunit
content - XSLT

EasyMock
Java
mock objects
GrandTestAuto
Java

JTestCase
Java
JUnit add-on
JUnit
Java
the parent of most xUnit frameworks
JUnitEE
Java
JUnit add-on
JUnitX
Java
JUnit add-on
Mock Creator
Java
mock objects
Mock Objects
Java
mock objects
MockMaker
Java
mock objects
Mockry
Java
mock objects
jsAsserUnit
JavaScript

JsUnit (Hieatt)
JavaScript
author is Edward Hieatt, note two tools with the same name
JsUnit (Schaible)
JavaScript
author is Jörg Schaible, note two tools with the same name
TagUnit
JSP
custom tags within Java Server Pages
LingoUnit
Macromedia Director

csUnit .NET

dotunit
.NET

NUnit
.NET

ObjcUnit
Objective-C

OCUnit
Objective-C

Perl Test::MockObject
Perl
mock objects
PerlUnit
Perl

phpAsserUnit
PHP

PhpUnit
PHP

PBUnit
PowerBuilder

PyUnit
Python

Ruby/Mock
Ruby
mock objects
Ruby Test::Unit
Ruby

OTF
Smalltalk

SUnit
Smalltalk
the first xUnit framework, and the parent of JUnit
vbUnit3 Basic
Visual Basic
also has a commercial version