"So now I know everything anyone knows
"From beginning to end. From the start to the close.
"Because Z is as far as the alphabet goes."
Then he almost fell flat on his face on the floor
When I picked up the chalk and drew one letter more!
A letter he never had dreamed of before!
And I said, "You can stop, if you want, with the Z
"Because most people stop with the Z
"But not me!"
-Dr. Seuss, On Beyond Zebra
Does your alphabet stop at Z? I'm looking beyond, expanding the scope of the raw materials that I use to form my thoughts. Welcome to my newsletter! This time I announce a new service that I would call long-awaited except that it's not exactly what you've been waiting for, and I'll look back over the past year as I celebrate two years in business.
If you're not getting this via email, please permit me to deliver future issues hot off the presses and straight to you. Surf over to http://tejasconsulting.com/newsletter/, plug in your email address, and you're done. See something here you want to share? Feel free to redistribute this newsletter, just please keep it all in one piece.
-Danny R. Faught
faught@tejasconsulting.com
http://tejasconsulting.com/
817-294-3998
My second year in business as Tejas Software Consulting was my best year ever. To my clients, students, mentors, and partners - thank you for your support! It truly takes a community to grow a consultant. I feel like I have to apologize every time I say that business is going well, however. So many of my friends are underemployed right now, I have to consider myself lucky that my gamble of founding a company during an economic downturn has succeeded.
I was happy to be able to help a few friends to get work and to be able to work with them again. I'm encouraged by the increasing trickle of job ads that I've seen. Somehow, I also managed to convince some people to volunteer their time to help me on a few tasks, and I thank John Oppenheimer, Jon Allen, and Dave Liebreich for their help.
I'm in the middle of a transition of my own right now. I had to make some
difficult decisions about my business model, and the result is that I'm now
actively looking for additional clients. But I'm excited about the launch
of Open Testware Reviews, which will let me spend more time working
with open source tools and also will hopefully allow me to make some big
improvements to testingfaqs.org.
I finally got a logo
this year. I worked with the folks at Echoes
Media to try to design a logo that represented software, and quality,
the development process, expert assistance, and being based in Texas but
also traveling elsewhere. Needless to say, we never got all of those concepts
together in one picture, so we punted and chose a longhorn. It turned out
looking like a skull rather than a silhouette of a live cow, which I like
because it looks different from the many other longhorn logos around here,
and because it's a good metaphor for the state of the software most of the
times when I start a project. :-)
My web traffic continues to grow on tejasconsulting.com and testingfaqs.org,
despite the fact that I'm woefully behind on updating testingfaqs.org. Just
shy of 9 gigabytes served this past year and 316,000 page views (compared
to 5 gigs and 200,000 page views in my first year). I more than doubled my
newsletter subscribership - wow, that was a surprise when I checked the numbers!
There are now more than 700 people on the swtest-discuss mailing
list, and we rose above 1400 subscribers for Testing FAQ News. My local networking
group, metro-sqa, is slowly
growing toward critical mass at 35 subscribers. My web audience includes
people in about the same number of countries this year (95), but there were
no web visitors from Kyrgyzstan this year. If you're reading this in Kyrgyzstan,
write and let me know you're okay.
I became a bit more of an activist this year. I've been using recycled
paper in my office ever since it became readily available. I also finally
had real business cards and letterhead printed, and was very happy to find
many paper options with significant recycled content. But I also tried to
influence an environmental decision outside my direct control. I asked SQE
to print my conference tutorial handouts on recycled paper, but I was disappointed
to hear that their printer demanded ridiculous fees for this. So I'm going
to have to learn more about the printing industry to see whether other printers
are more interested in increasing demand for recycled paper.
I tried to organize a tribute to the anniversary of the 9/11/01 attacks, and while I didn't get the broad response I had wanted, I was happy to see three sites in addition to my two that displayed my makeshift image and linked to a tribute page. That page continues to get hits now. I reflected on this very recently as I was raising the flag again at half-mast, in honor of the crew of the space shuttle Columbia. It's disquieting knowing it broke up practically right over me, and that one of the shuttle disaster field offices is right here in Fort Worth.
I ran across some fascinating greeting cards when I was returning from
Change Shop in New Mexico - Tree Free
Greetings made from the kenaf plant. Besides being beautiful --
even the envelopes are printed -- they're not made from wood pulp. The design
I chose for my holiday greetings meshes well with my 9/11 picture, though
you have to look closely to notice that. The profits from the cards
I bought benefited the Fort Worth Zoo. In case in my frantic scribbling I
didn't get one out to you last season, I'll send one to the first three people
who send feedback for my next newsletter issue if you didn't get one already.
Also environmentally speaking, I bought a hybrid electric car, the Toyota Prius. I love it! I'm buying less gas, and spewing much less pollution. I've been asked to write an article about my experiences with it - remind me to share it if you'd like to see it.
So what's ahead? I'm starting to find
my voice, and what I stand for. I've realized that I like getting things
done, even when the environment is chaotic. The extreme programming (XP)
movement has really affected me, though I haven't worked on a full XP project.
I like being able to show progress in the short term, even when working with
an always ambiguous and shifting long-term outlook. And as always, I love
writing, and I'll continue to find excuses to write.
Great newsletter! I read Whittaker's How to Break Software and thought it was excellent, and fun to read relative to software-testing books.
In your feature article, I laughed out loud (I'm hoping you also saw the humor in it). Especially this phrase: "check the available documentation, the knowledge base on the vendor's web site, and independent sources of help on the Internet to verify that the software is misbehaving."
That is why consumers are at a loss about what to do about the poor quality of software (as opposed to other kinds of products) -- they can't even be sure WHETHER it's working.
By qualifying it as the "available documentation," I did gloss over the
issue of poor documentation, though I was also careful to point out that you
might have to look elsewhere for help. In any case, if you can't tell whether
it's working, then in my opinion, it's broken.
A couple of additional points about James Whittaker's stuff. He outdid himself at STAR West, giving a preview Wednesday of what he calls "testing aerobics", crashing software in sync with a very loud rock song called something like "What a Piece of Crap." In his Friday keynote, he showed the same thing but as a video featuring his students as well.
Your reflections on James' book mirror mine. As well as it may be written, it just can't even approach the excitement of James in person. Also, although it may be considered heretical, 99 percent of what James is showing involves bugs that -- while spectacular in their effects -- seldom would afflict normal users; and it seems highly questionable whether James' techniques would be likely to detect most of the more important bugs that real users actually encounter -- especially those that impact functional effectiveness without causing a crash. Before pillorying me, let me qualify this latter point of agreement with Danny's comments.
First, it's my understanding that James' starting point was with 10,000 documented bugs, i.e., bugs found by other people who were not using any particular bug revelation technique, so they in fact were found by people in some sort of normal usage, at least for them. The other side of this is that there's no saying how productive James' techniques would have been in discovering these 10,000 bugs if they had not first known of the bugs' existence. I certainly sense that his methods work a lot more spectacularly explaining the discovery of an already-spotted bug than forcing revelation of bugs we don't know about yet. One could probably say that about any testing technique.
Second, another keynote at STAR West was given by Jason Taylor of Microsoft who credited James for much of the methodology that helped them dramatically cut bugs in upcoming Internet Explorer versions (I believe I've described this accurately). While Jason didn't get into the specifics, it was pretty clear that the methods they had learned from James were not merely the rock video razzle-dazzle and in fact do detect a lot of bugs in mainstream functionality before users find them.
James Whittaker responds to my review of his book How to Break Software
and to Robin's comments:
Every talk I ever give I begin by saying the bugs I am presenting are given for the value of the lessons they teach and not their severity. The preface of my book says the same thing. I would be sued for showing the really good ones.
My critics are quick to criticize the bugs I demo, despite my caveats I give before each talk. Yet few ever demo bugs of their own. We have each chosen the best way to teach our methods and I respect the methods they choose. If they fail to respect mine, so be it. But a teacher must teach, regardless of the critics.
I am not going to defend my bug finding record. We have many happy customers who return again and again for our testing services. My track record is one that I am very proud of.
I do believe that the patterns in the book are very useful for robustness
testing, and the sample bugs are effective at illustrating the techniques.
My main criticism is that I believe that in the scope of all the things that
testers need to do to test a software product, they need to apply several
additional techniques beyond robustness testing, and the book doesn't explain
that. When put into context, How to Break Software is a great resource.
Travis Reed (MCP) takes us back to the certification debate that started in the August/September 2002 issue:
When I first read your article concerning certification, I considered writing, then stopped and said no, I would let this pass. I often read your notes, as with other folks in the business to get an idea where we are as an industry, and to help me keep my bearing. You'll notice at the end of this email a tagline for certification. By trade my job is really in software development and process improvement, but before too long I will be fully certified as a DBA/MCSE and probably shortly after that an MCT from Microsoft.
While all of the arguments you heard not to certify are generally accurate and probably completely true, you never really stated the arguments why you should certify. At best you leave the reader with a sense that they should not really bother to certify.
Whether you certify or not, will the next company you want to work for require it? Will it ever affect your pay? If you consider yourself a software tester, what is your degree in, and should an employer require/hire someone who does or does not have a degree? (AS, BS, MBA)
Now consider again the Microsoft certs. They tie together the common pieces of the networking puzzle, and teach how to analyze for the custom pieces, and then tell how those products can deal with the options that are left.... I believe what you set out to find was the holy grail, when you only needed a dixie cup.
Certification should talk to the commonalities of software testing or whatever, rather than the uniqueness. Unless your average tester is going to take a position as a Manager, they will not be deciding most of the policies and procedures of the testing for several reasons. But they may be asked to do unit testing, is unit testing a standard? What about white box vs. black box? Come to think of it, what is regression testing? Isn't all testing really regression in nature?
Again, the ties that bond, we have a set of standards that government and industries have agreed to. Every day we agree to more and more standards for coding and development. These are the things that make us common to each other, and allow us to communicate to everyone on the same level. Seek the commonalities, understand the differences and why the two exist.
I haven't thought much about certifications like the Microsoft certifications that Travis mentions. For a software quality professional who wants to establish herself as an expert for a particular technology, perhaps this would be beneficial. Personally, I want to emphasize my ability to apply testing techniques across a broad set of technologies, so I don't plan to pursue technology-specific certifications.
I think that broad swaths of the bodies of knowledge for the various testing
certifications are not yet governed by standards. And I don't think that
the standards that exist are universally applicable. For example, I have
advised a client not to follow IEEE 829 to the letter. Perhaps we should
focus more on the details, and certify people in test design, or metrics,
or exploratory testing, etc. Let's tackle smaller problems before trying
to cover everything at once.
I only included an excerpt of Travis' comments here. See the addendum to this issue for his unedited comments.
| Back to the newsletter index | Back to the home page |
