Software Quality Notes
Test Tools for Free

by Danny Faught
published in the Dallas/Fort Worth Unix User's Group newsletter, June 2001. Copyright 2001 Danny Faught. For more on freeware test tools, see Open Testware Reviews. Note that in the years since this article was written, many more open source and freeware tools have been released. Many of them are posted on testingfaqs.org.

I often hear from people who are looking for cheap test tools. If you've ever priced commercial test tools, you're probably thinking that anyone looking for a cheap test tool is on a wild goose chase. Besides, if an organization hasn't budgeted properly for the basic tools that its test group needs, they really don't deserve any sympathy, right? Well, possibly. But there are some legitimate factors that may drive an organization to want to limit its tools budget.

Perhaps the test group has made a major tool purchase in the past. It's not hard to spend more than $100,000 on license fees, with annual support costs added on top. Perhaps the tool turned out to be a real stinker - full of bugs, badly documented, or not living up to the promises of the sales pitch. They may have abandoned the tool, leaving their huge investment on the shelf to collect dust. An organization that has been through that is going to be very reluctant to make that same level of investment in another tool.

Perhaps an individual is wanting to learn how to use test tools in their spare time in order to become qualified for a job in test automation. Or maybe a high school or college course needs a tool to use for a project. Or, consider a test group that needs to have a test tool ported to a unique platform that no existing tool supports - getting a vendor to port their tool, or getting a source license so they can port it themselves, can be extremely expensive.

In each of these situations, it would be nice to have a freeware tool available. The good news is that freeware test tools do exist. There is at least one freeware tool available in every category of my Testing Tools Supplier List at http://www.testingfaqs.org/. The bad news is that the quality and functionality varies widely. Some of the freeware tools are the product of a research project that has long since finished, and the tool is no longer maintained. You'll probably need to supply your own elbow grease to install and support the tool. Nonetheless, it would be wise to be aware of what kinds of freeware tools you can get, because some of them can be very useful.

Before I go on, I should explain what I mean by "freeware." I'll use the definition offered by Matt Pierce, a freeware tools author - "You may freely use and distribute copies of this application." I include a broad class of licenses in this definition, for example, the Microsoft Web Application Stress Tool, for which only the executable is freely available, "expect," which is in the public domain courtesy of NIST, and GNATS, which uses the GNU General Public License. Richard Stallman would object that not all of these are free in the philosophical sense, but I haven't found a better word that covers all of these cases. Certainly, the tools for which the source code is available are potentially more useful.

The category with the most freeware options is defect tracking tools. I've found six different freeware defect tracking tools - Buggit, Bugzilla, Debian Bug Tracking System, GNATS, JitterBug, and OpenTrack [OpenTrack is no longer available. -ed.]. There are test case management tools - DejaGnu and Test Case Manager (TCM), plus the unit test framework JUnit. There are two memory management debugging tools - dmalloc and mpatrol. Rounding out the list are ASSIST, a tool to assist with software inspections, GCT, a C code coverage analyzer, ccount, a C readability analyzer, DGL, a test generator, expect, a scripting language that can be used to test command-line programs (also includes a simple text-based capture/replay capability). Also added just a few days ago is Android, a simple analog GUI test tool.

I'm hot on the trail of several other freeware test tools. If you know of any that I haven't listed here, please let me know. And if you'd like to hear of new tools that are added to testingfaqs.org, use the form on the front page of the site to sign up for the monthly Testing FAQ News mailing list.

The current offering of freeware tools isn't rich enough to render commercial test tools obsolete. But we can start to build a critical mass of awareness and interest in freeware tools.
--
Danny R. Faught is a software quality consultant and proprietor of Tejas Software Consulting. He can be reached at faught@tejasconsulting.com and http://www.tejasconsulting.com.