Quality Assurance
The Project Leader
"Why didn't I think of this before?" Susan asked herself. "If you don't want to watch your lunch-mate eat pizza, take them somewhere that doesn't serve it." The Dragon Kimono, a Chinese-Japanese restaurant, was the perfect place for lunch: lo-cal, delicious food and not a french fry or burrito to be found on the menu. Besides which, Susan's lunch companion Greg, the new Quality Assurance guy on the project, appeared to have some Asian background and might appreciate the gesture. The waiter arrived and Susan ordered the tuna sashimi from the Japanese side of the menu and looked with interest to see what her companion would select.
"I really appreciate your taking me here for lunch," said Greg. "I see something on the menu that my grandmother used to make for me when I was a kid." Greg pointed a finger at an item on the Chinese side of the menu and said to the waiter, "I'd like the steamed Chicken Feet, please."
Chicken feet? An interesting name. Susan knew that Bird's Nest Soup wasn't really made from bird's nests and that Thousand Year Old Eggs really weren't, so she looked forward to seeing what Chicken Feet might be. But first, time to do some biz.
"Greg, you know we don't have too much time for testing," said Susan, "The programmers finished about 25% behind schedule so our production date is only a month away. Do you think you can handle it?" Susan felt a guilty twinge as she asked this question because she knew it wasn't enough time, but the project had already been delayed once and management was on her case not to let it happen again.
"Well, you can test anything in any amount of time," Greg replied. "If you've only got 10 seconds you click on an icon and if a screen comes up the system passes. If you've got 6 months you test every detail of the application; you simulate system crashes, missing and duplicate input files, and anything else you can think of. If you're somewhere in between, you test as much as you can
and keep your fingers crossed about the rest.
On the one hand, Susan wasn't crazy about the "keep your fingers crossed" part of the answer she had just received, but on the other hand, it did seem as though she wasn't going to have to go back to the well again and ask for more time.
When the food arrived, Susan's tuna sashimi was a work of art. One section of the tuna appeared to be a rose resting on a little patch of greenery and the the others were in fanciful geometric shapes. A gob of acid green Wasabi sauce was the snake in this garden of Eden. Greg's chicken feet really did look like a small herd of chickens had just been lopped off at the knee, and their poor, severed feet fallen into a pool of black blood on Greg's plate. "Er, Greg, what are those chicken feet made from? They look so real."
"They are real," said Greg. "Quite a delicacy in China. They remove the claws, steam them, and serve them up in this delicious black bean sauce." He selected a foot from this little graveyard and began nibbling on an ankle.
Real? He's eating feet! Susan couldn't watch. But she couldn't not watch either and ended up with her head cocked at an odd angle looking out at the other diners and sneaking an occasional horrified peek at her lunch companion's plate. She was sure Greg would be terribly offended if she said anything about this display of foot fetishism, so she resolved to continue talking about the project. "What do you need to get started?" she croaked. Because she was no longer looking at her lunch companion, her question was actually aimed at a passing waiter but she hoped it would curve around in mid-air and eventually reach Greg.
Greg didn't seem to notice that his lunch companion had developed a permanent crick in her neck that caused her to stare fixedly at a spot about three feet over his head and four to the right. "The first thing I need is some spare space to set up a testing environment. It should look just like the development area, source code directories, data base directories, object file directories, libraries, etc."
"Can do," said Susan to a passing plate of General Tso's Chicken. "We've got plenty of disk space." She risked a peek across the table at the satanic ritual going on amid the chop sticks and napery. Greg was happily sucking on a toe covered with dried blood...er, bean sauce, so she resumed her examination of that little piece of air up near the ceiling.
"Then I'll need to get a database from somewhere," Greg continued. "Maybe I can borrow the one Joe's using, and then I'll set up some forms to use for moving programs into QA..." For the first time Greg noticed that there was something a bit peculiar about his lunch companion. She seemed to be staring at something in back of him and up near the ceiling. Was there an NBA center standing behind him? "Susan, are you ok? I notice you've hardly touched your lunch. Was the tuna a little off?"
* * * *
The Quality Assurance (QA) testing process has a tendency to be squeezed into a small time period. On one side the programmers and analysts apply half of the squeeze by finishing their tasks well behind schedule, and management applies the other half by not wanting to move the target date back. Greg is right that you can do QA testing in any amount of time, but you can't do a good job on a big system in not very much time.
The QA people usually view their job as finding bugs in computer systems, which, of course, it is, but it's also similar to the job of an insurance underwriter, accepting responsibility for future problems. Consider the system that goes into production and has a major bug on the first or second day. If you want to assign blame for that occurrence -- as organizations often do -- who is at fault? Is it the programmer that mis-coded the program that caused the problem? Not really. Bugs are viewed as a normal part of coding and no one expects a set of software that's just been delivered to arrive totally bug free. What is expected, however, is that most bugs, especially the large, hairy ones with lots of legs, will be caught by the testing process. So the big first day bug is generally viewed as a QA problem.
The time crunch and the underwriting aspect of QA catches the QA person in a vise. He's not given enough time to do his job correctly, but he is blamed for errors that occur because of that. A good way for Greg to defend himself from this is to make a list of the things he's not going to be able to test in the time available and send that up the line for management approval. The effect of this is to transfer responsibility for as least some first day bugs from the QA tester to the manager who approved bypassing part of the testing. Managers, like politicians, want to take credit for anything good that happens but not responsibility or blame for anything bad, so the result of this is most likely to be the target date being pushed back a bit to allow for better testing.
Greg is about to make a mistake that is made in a surprising number of projects -- using a development database for QA. Clearly using a copy of a production database or an extract from production is a better idea. If you don't do that, a mistake in the database can be disguised by a program written specifically for the flawed database and the error won't be detected. So why does this happen so often? One reason is that testers don't understand exactly what their job is. They think their job is testing programs, but that's only a part of it -- their real job is testing systems. That broadens the testing horizon by quite a bit. Testing operations procedures to see if they work or make sense is important. Making sure that outputs from the system that go to other systems are correctly formatted is important. Reports need to be checked. But the most important aspect of this expanded horizon is testing the database, making sure that it matches and is in sync with the programs and that each data field contains what the programs expect.
Sometimes a development database is used for QA because the production database can't be used. The new system likely involved database changes -- anywhere from minor to massive -- so a straight copy from production won't do the trick, and the conversion program that takes the existing production database and creates the new production database may not have been written yet. The conversion program should really be the first program that's written on a new project, but often it's the last. When the development phase first begins, the design is still soft and subject to changes so developers don't want to write a conversion program and then have to change it multiple times while the system design squirms around under it, so they dive into writing their application programs and then gin up little databases of their own to test them. Months later they emerge from their underground coding caverns with a system ready to test and still no conversion program written.
Doing the conversion program first really pays big dividends that more than offset the minor annoyance of having to make minor changes to it in the early stages of the project. The obvious benefit is that the developers can each have their own little test database containing real data from the production system, but there's another advantage that's a bit more subtle: the conversion program is a way of testing the system design. If the conversion program programmer comes across a data field in the existing production database that has no corresponding field in the new system, it may be a sign that a piece of the application has been overlooked in the design.