If your organization could benefit from improvements in the way you measure and assure software quality, contact me at faught@tejasconsulting.com or 817-294-3998.
In the section talking about load testing, I enjoyed reading about some different types of loads to place a system under. One thing that wasn't really discussed was when you are really doing performance "analysis," as opposed to load.Yes, Steve didn't really cover performance analysis in his performance testing slides. There's quite a bit more science to it than looking at load tests, and the techniques for performance tuning that are probably outside the scope of a software tester's job. I had to blow the dust off the book High Performance Computing (by Kevin Dowd) when researching the article. This book goes into great detail about performance optimization, and it discusses some of the benchmarks that are used to measure performance.
Jeff also provided some great motivation for performance testing:
By the way, re: performance testing, the most common question that is asked by small start-ups (in direct response to investors) is "can the system scale according to our growth projections".I greatly enjoy your feedback. Please keep sending your comments in to faught@tejasconsulting.com.
"Managing the Testing Process" was developed by Rex Black, and I offer the course in cooperation with Rex Black Consulting Services. Rex has refined this course after many presentations of the material.
The other courses are original. All of these courses are taught at your
location, at a time that is convenient to you. I'd also be glad to discuss
customized training, tailored specifically to your organization's needs.
I also recently published a newspaper article, "Consumer
protection efforts threatened by UCITA," in the April 30 edition of
Dallas-Fort Worth TechBiz. This is a gentle introduction to the
UCITA draft law, written from the consumer advocacy point of view. I researched
the law's status in Texas, and it looks like it's unlikely to pass in this
session, but efforts to get it passed will continue during the upcoming
legislative break.
Software Quality Notes
I recently had the privilege of attending a talk by Watts Humphrey, a Fellow
of the Software Engineering Institute
(SEI). This was his second visit to the Dallas SPIN group, the Association
for Software Engineering Excellence, since his visit shortly after
the group's founding about 9 years ago. Talking with the people milling
about before the talk, I found out that many people in organizations that
aren't interested in the Capability Maturity Model (CMM) generally don't
recognize Watts Humphrey's name. During the talk, though, I realized that
many companies that aren't ready to embrace the CMM and usually run the
opposite direction when they hear the word "process" could benefit from
some of the lower-level methods from the SEI.
Watts Humphrey on Teams
In his talk, Humphrey focused on the Personal Software Process (PSP) and the Team Software Process (TSP). Oh boy, I just wrote "process" twice in one sentence - don't run away yet!
The PSP is the better-known of the two. It applies to the processes that an individual software developer follows, and it doesn't necessarily require support from any higher level processes like the CMM. The PSP involves a very disciplined approach to software development. It focuses on low-level design and coding, and thus is applicable even in organizations that don't have a ream of specifications ready by the time the coding is supposed to begin.
The TSP is newer and not as well-known. While the CMM does not include any specific guidance about how to implement particular processes, the TSP includes details such as forms, scripts, roles, and a support system. Teams following the TSP start off with a "launch." Then after each major project milestone, there is a "relaunch" to kick off the next phase. The process wraps up with a postmortem.
My ears perked up when Humphrey listed the three levels of planning under the TSP - 1) Overall project, 2) Detailed plan for the next phase, and 3) Personal plans for the next phase. He asked "When can you make the most accurate plan?" The answer was of course "At delivery, when you need it the least." When you need the plan the most, before you have requirements, Humphrey said you simply have to guess. When I first learned project management, the course I took did not adequately address the fact that you don't have perfect information about all of the tasks when you first try to put together a schedule. It was nice to see Humphrey break it down into the high-level plan (with lots of guesswork), and the short-term low-level plan that's more detailed.
Humphrey discussed the schedule negotiations that are so common between a project team and management. He said that most teams following the TSP come back with a plan that is quite a bit longer than what management wanted. But because the team has the data to back up their estimate, he hasn't seen a single case of management rejecting the plan. He said "If the engineers tell you it's going to take 18 months, your chances of getting it sooner are zilch."
Some great questions came up during the Q&A session. When asked about the "casualty rate" for teams implementing the TSP, he said that 5-10% of the engineers are likely to balk and leave. However, once the TSP is implemented, staff turnover is very low, such as for one particular team in San Jose that had no turnover, where the average turnover for that region was 27% per year.
With all of these processes to choose from, which to start with? Humphrey said that most of the organizations that he has worked with on the PSP and TSP have been CMM level 1 organizations. He recommends training the team members on the PSP and then launching the TSP, which can be done in 8 months. After that, he recommends getting a Software Engineering Process Group (SEPG) in place to start down the CMM path. The SEPG should define processes for a specific team that actually needs them, not just to stuff a big honking binder (a great term I borrowed from Dilbert, not Humphrey). Humphrey says it takes 2-3 years to move up one CMM level, so starting with the PSP/TSP can yield much faster results. The irony of this is that I started to study the PSP several years ago, only to find a recommendation near the front of Humphrey's book An Introduction to the Personal Software Process that recommended learning about the CMM first.
Humphrey's recommendations have a distinctly practical air to them, while at the same time claiming significant improvements when we pay attention to process. His ideas incorporate many of the best practices that I've seen over the last few years. It was a nice wakeup call from the assumption that process people don't care about practicality. Humphrey's slides are available temporarily on the ASEE web page.
I have a few questions for my readers. Drop me an email and let me know what your response is. I'm curious how many people have these things on their radar -
#####
| Back to the newsletter index | Back to the home page |