Tejas Software Consulting Newsletter
v3 #5, October/November 2003
This is the sound of the
rumble strip.
Our quote this month comes from James Bach, speaking at the 13th
International Conference on
Software Quality in Dallas. James was talking about the clues that a
tester gets when a software bug is nearby. They may be subtle or
invisible to most people, but to an experienced tester, something like
a brief flicker on the screen is as noticeable as driving over a rumble
strip on the highway.
So here we go again with my bimonthly update of what I've been up to,
what
I'm thinking, and what you think about it. Make sure you're getting the
email edition by signing up at http://tejasconsulting.com/newsletter/.
-Danny R. Faught, Software Alchemist
faught@tejasconsulting.com
-- http://tejasconsulting.com/
-- +1-817-294-3998
Contents
Tejas Newswire
Since we met here last, I've attended two conferences - the Software Test Automation
Conference in Boston (STAC for short) and the 13th International Conference on Software
Quality (ICSQ) in Dallas. I gave my Perl scripting tutorial again
at STAC, and it went well. I think what I learned at the Experiential
Design
Workshop helped me to make it a richer experience. My tutorial at ICSQ
was canceled because there weren't enough registrations, the first time
that's happened to me. But it was just down the road, so I attended
anyway, and I met several people on my "need to meet someday" list. I
especially appreciated the keynote from fellow Fort Worth resident Bill
Curtis, whose performance on the subject of cowboy coders would have
fit right in at a living history presentation.
In the September 2003 issue of Open Testware
Reviews,
I took the QMTest functional test tool through its paces, and I
surveyed
25 data comparator tools, including the venerable "diff" utility. In
the October issue, I introduced an additional review format with
"Technology Bulletin: System Call Hijacking Tools," covering a few
tools that perform a similar job as Holodeck, but on Linux. I also
changed from a magazine-like packaging to a web log format for
announcements of new features, news, and feedback. The value you get
from Open Testware Reviews
increases as each new feature is added to the archives, but you don't
get that value if you don't subscribe. Follow the URL and see for
yourself.
Real soon now, I'll have a "Last Word" article published in STQE
magazine, soon to be called Better Software. In this article, I
root for the test tool underdogs. Also in the pipeline is an article
about resources for software testers.
I'm always looking for new clients who would like to improve their
testing efforts. Let me know if I can answer any questions for you.
Feature Article
What Is This "Testing" Thing?
In response to my last newsletter, Fern Herring said -
Please
explain what this "testing" is that you do, in lay language so I can
understand it.
Thanks for asking - many of my non-technical readers have
probably wanted the same thing. For my technical readers - I'm curious
how you
would answer the question. How do you describe what you do to your
friends, family, and non-technical business contacts? It's not easy to
do! Here's
my attempt.
I like to steal from a BASF slogan - "I don't make the software you
use.
I help make the software you use better." The purpose of most software
testing efforts is to find problems (also
called "bugs" or "defects") with the software before it is given to the
customer. Note that testers rarely contribute directly to the code the
comprises the software product. Software teams don't have foolproof
methods for making software
that works, so the problems are inevitable. It's important to find as
many of the problems as
possible, though we can never find all of them. That's why I call
myself
a "software alchemist" - it seems that the only way out is to have a
philosopher's stone that can magically turn our software lead into
gold.
You've likely run into many bugs in the software you use. Even if you
blamed yourself
for the problems, I bet many of them were caused by bugs in the
software. The fact that you saw them means that they slipped
through the fingers of both the programmers and the testers. It would
take a tremendous amount of effort for the
testers to be able to find most of the bugs, and that would make the
software you buy cost a lot more, perhaps ten times as much or
more. For NASA, it's worth it to do that much testing, but you're
probably not willing to pay much more for software than you do now.
What do testers actually do? It starts out with planning, of course,
though that part isn't much fun. Software projects change direction so
often, I always have this feeling that our plans are for naught. Once a
project is underway, testers may participate in reviewing the early
documents produced by the software designers. Testers might not be
experts in all the various technologies that the designers are linking
together, but we're still good at finding mistakes and missing elements
that could cause major problems if not discovered until late in the
project. Oddly enough, testers often have a better big-picture grasp of
the implementation of the software system than the programmers who each
focus on the details of a small part of it. Some testers also take on a
"quality assurance" role, where they audit and improve the overall
development process as well as the product.
Once the programmers have produced at least part of the software, it's
time for the testers to start running it. The testers may have produced
very detailed plans for how to test the software, especially if a flaw
in the software could cause a safety risk. Sometimes the testing is
"exploratory," with no detailed planning in advance. Usually there will
be some combination between these two extremes.
Testers will use the software's features, trying to find any behavior
that differs from the official documentation or that just doesn't seem
right. Sometimes we write our own programs to automate the process.
When we find a bug, we report it so designers and programmers can fix
it. Writing an effective description of a bug is an art that gets
better with experience. Testing starts by looking at individual
features, or maybe internal functions that the end user can't see. The
programmers may do some of the testing at this level, though there's
wide variance in the industry as to how much the programmers test
their own code.
Testing later moves to a higher level, looking at the software system
as a whole. This requires more specialized skill than lower-level
testing. At this point, we examine things like whether the software
runs fast enough, whether it can run for a long time without failing,
and whether it fails if it encounters unexpected situations.
It's hard to grasp the complexity of creating software and verifying
that it works. One testing sage has estimated that the engineering
effort that goes into producing a word processing program eclipses the
engineering that goes into a new supertanker and a 100-floor building
combined. That's a lot of complexity crammed into that computer disk!
While many people who aren't computer experts are very intimidated by
computers, you should realize that those of us who have some
understanding of what makes them tick also haven't really tamed the
technology.
Testers often disagree about the best way to approach our craft. So I'm
sure that even this brief description will invoke the ire of some of my
colleagues. As always, I invite your feedback, and I promise to give a
forum to alternative ideas.
For further reading, see my article, "Yes
Virginia, We Do Want Our Software to
Work", and for more details specifically about my business, see the
Frequently Asked
Questions about Tejas Software Consulting page.
Feedback on the August/September 2003 issue
Several people responded to my "Trip Report
from a USENIX Moocher" article. Alan Koh wrote:
I
commend you for writing a humorous trip report to the Usenix
conference. It was funny. Great job.
Thanks, Alan. I didn't set out to make it funny, though I do try
hard not to bore you. :-)
Jerry Weinberg wrote:
I
loved the article on working the
conference. It's what I do, but I never saw it written down before.
Thanks. Here's another one I used recently at the International
Conference
on Software Quality. I saw a name badge that said "Taz Daughtrey," and
I realized that I knew of him but hadn't met him before. So I stood
near him, waiting for a break in his conversation with someone else. I
had trouble positioning myself so he could see me. He shifted a few
steps away from me, and I shifted, too. Someone else came up and
swapped pleasantries with him. Finally I announced, "Taz, I'm going to
follow you around the room until you shake my hand." I got the
handshake, and a great new networking contact.
Brian Marick responded to my assertion that I had only met one person
who had heard of both Convex and Cigital, my two previous employers:
Oh
yeah?
Okay, Brian, you got me. Brian and I first met when I was at Convex,
and we both worked together briefly at Cigital.
Besides inspiring this month's feature article, Fern Herring wrote:
Even
though I don't
know what you are talking about most of the time, I really did enjoy
your account of "How I attended a conference on the cheap." It did have
some very good points in it.
It's good to hear there are more holistic thinkers out there.
My anonymous commentator who shared his bad experience at a Convex
interview in the last issue wanted to make another point. Though he
doesn't do well on coding tests, he emphasized that he was and is an
accomplished test tool developer and tester. "Frankly, I think I would
have done a lot of good there, if the interview had turned out better,"
he said.
Copyright 2003 by Tejas Software
Consulting
#####