Jerry Weinberg, (private communication) 1996
At first glance, ATAMS--an automated tracking and monitoring system that allows the NORAD Command Center at Cheyenne Mountain, Colorado to control the software systems that watch for missile attacks--appeared to be a typical software disaster. It was built on a very short schedule. The contractors
thought they would need two years, the Air Force specified one. Snafus were obvious from the beginning--the planned Sun workstations were delayed and the software had to be written for IBM hardware and then converted to Sun later. Scope creep was a problem, users demanded 10 times more functions than were
originally planned. Tests found unexpected bugs in the systems that ATAMS keeps track of. And several bugs were found in ATAMS itself just before the system was finished.
ATAMS was completed in April 1996, on time and within budget. It worked out of the box. In the months after it was turned on, users identified only two, easily repaired, bugs.
Buford D. Tackett, the team leader responsible, does not think it was a fluke. He 'combined several techniques that were shown years ago to produce better software faster yet are still rarely used.' Conspicuous in his list, 'peer reviews that caught more than 200 major design errors while they were still easy to fix.' (see the August 1997 issue of Scientific American for more details)
Informal technical reviews were first mentioned in the literature more than 25 years ago, when Jerry Weinberg described 'egoless programming' in his book The Psychology of Computer Programming. There are many variations on the basic technical review, ranging from informal reviews where a programmer
walks across the hall to show something to a friend, all the way through walkthroughs and formal inspections, with set schedules, detailed paperwork, and assigned roles for each participant. Each type of review has a different place in the project schedule, a different purpose, and a different process. Reviews aren't just for code. Reviews have proven their worth for any product that must be relied upon or understood by more than one person. In our industry, reviews have proven useful for code, design, analysis, and requirements documents, user documentation, test plans...
Reviews will help you eliminate errors and omissions from your code. But reviews do more than just help you produce a quality product. On a meta-level, reviews can give you the information you need to improve the process that created the errors. Over time, you should find yourself inserting fewer errors in the first place.
The more formal the review, the more demands it will make on management for support, resources, and staff training. And the more value it will return, both to the organization and to the individual workers. In this article, I will confine myself to discussing the less formal variants, since I believe that it is easier for most people to start out with 'baby steps' before committing themselves to full-scale formal inspections. I won't tell you how to conduct reviews. The literature is extensive and easily available, and I've listed some resources that should get you started. Instead, I want to talk about why reviews are valuable and some of the preconditions that must exist, or be created, before reviews will be successful in your organization.
Like all worthwhile things in life, peer reviews are hard. They're easy to get wrong--especially at the beginning, before good habits are established. The available literature talks a lot about process, and the steps to take and the things to do as you conduct reviews. Unfortunately, very little of the literature I've seen talks about the psychology behind successful (or unsuccessful) reviews. And I believe that the review process is fundamentally about people. Your reviews will succeed, or fail, based on the psychological
processes and the patterns of communication that are active in your workplace. The solution is not to avoid reviews, the solution is to learn how to do good reviews!
As a single programmer, all you really need is a friend or coworker who is willing to trade comments with you. However, if you are hoping to start more formal reviews in your workplace, you'll need the support of a key manager, someone high enough in the power structure to deflect blame and interference from outside the group.
Training is important. It will help you set the proper expectations, and will get you off on the right foot. At the very least, everybody should be able to spot, and deflect, blaming behavior before it takes over a situation. The formal structure of a properly run review program can, strange as it may seem, actually add to the sense of safety of everybody involved. And this sense of safety and empowerment will help you bootstrap yourself into a less-blaming organization.
Properly run reviews add value many different ways, both to the individual programmer and to the organization. First and foremost is what I call The Sieve Effect. The more minds--the more different personalities and life experiences--that you can engage in the search for error, the more chance you have of catching problems early, before they become serious (and expensive to fix.)
This leads directly into The Safety Net Effect. Like a circus acrobat on the high-wire, as you get used to well-run reviews you can let yourself be more creative, more free to take risks and to try new things. Your peers act as a safety net underneath you, since if something doesn't work out they will be there, available to catch the problems before you splatter yourself all over the ground. And in today's competitive environment, creativity does pay off!
The Blind Spot Effect--Properly run reviews will teach you to separate the work from the worker. When nobody equates a flaw in the work with a flaw in the worker, you can set aside your defensiveness and free your entire mind for the task of finding flaws. This, of course, makes it easier to find the flaws that do exist. And fewer errors will make it into finished code.
The Designer Effect--You can't review a bowl of spaghetti! Before something can be reviewed, it must be able to stand alone as a separate unit. The very existence of reviews will help enforce good planning and design, low coupling and high cohesion.
The Training Effect should never be underestimated. People will quickly pick up and spread the good
ideas that they see in somebody else's work. In fact, Jerry Weinberg goes so far as to say that, in his experience, this is the most important result of a review program.
The Blender Effect works to increase consistency across teams and projects. Officially or unofficially, group norms will quickly become established and the code will be easier for everybody to read and to maintain. It's always easier to understand a coworker, when you're speaking the same dialect!
The Spotlight Effect--Reviews will shine a light into dark corners, so management always knows what's in the code (and the requirements, analysis, design, test plan, etc.). Jerry Weinberg tells a story (in Quality Software Management, vol. 4, page 285) about a project that discovered one of their programmers had spent a lot of time writing his own spell-checker routine. Needless to say, this routine had not been included in the requirements, and in fact was not even useful since the project was designed to interface with a commercial spell-checker. A colleague of mine tells a similar story about a coworker who spent several months writing his own language using the macro facility of their compiler, instead of working on
his assigned tasks. The company did not conduct code reviews, and, as a result, was unaware of the problem until the integration and test phase of the system.
Like all powerful tools, reviews are dangerous when they are misused. It's very tempting for naive management to try to use reviews to evaluate the people doing the work. Resist the temptation! Reviews provide information about the product, they must never be used to judge the person doing the work. Not because it's 'wrong' to do so, but because it just doesn't work. Reviews allow management to see inside a project, to know what has been completed, what needs more attention, and what has yet to be tackled. If management tries to evaluate the people and the work at the same time, their sources of information about the project will dry up or become badly distorted as soon as people become concerned about 'looking good.' As Freedman and Weinberg explain (in Handbook, page 148), don't evaluate the producers for error, reward them for making their errors easy to detect!
Another excellent way to kill a review program is to let your reviews be used as a way to attack other people. If people don't feel safe in a review, you will not be able to rely on the information that comes out of the review. A strong, competent moderator will keep an eye on the emotional 'feel' of the meeting, and will either head off the blaming behavior before it becomes endemic, or as a last resort, will stop the meeting before irreparable damage is done.
Formal reviews will require a lot of support from management, if they are to succeed. Time needs to be set aside in the schedule. Some organizations report they spend up to 20% of their time in reviews. People need to be trained in review techniques. Physical resources, such as conference rooms, need to be reserved. The reviewers will need to spend time going over the material before the actual review meeting, this preparation time must be allowed for in their schedules. Managers should track the results of reviews as they are completed, and use them to adjust future plans and schedules. Managers need to make sure that the faults identified in reviews are resolved before the work product (code, analysis, design or whatever) is
considered complete.
The Bottom-Line Effect--Everyone will make errors, that is the Nature Of The Universe. But something magical happens when it becomes safe to see an error! Paradoxically, the safer it is to see and to
acknowledge errors, the easier it is to locate and to remove them.
If you spend money and time up front, on reviews and repairs, you will save many times that later, in testing, maintenance, and customer-support. The initial investment can be substantial, and scary. But organizations that have instituted effective review programs believe that the eventual payoff is more than worth it.
Tom Gilb:
I haven't been able to find a home page or central site for Tom Gilb. However, a search did turn up this site, which appears to contain some interesting files and documents.
http://www.stsc.hill.af.mil/swtesting/gilb.html
Software Inspections and Review Organization:
From their Home Page--'The Software Inspection and Review Organization is a voluntary organization devoted to the exchange of information about group-based examination of software work products, including the state of practice, emerging techniques, support resources, and industry use and metrics.' This site appears badly out of date (the last update is listed as Sept. 27, 1996) but it does have a couple of interesting links, including one to the 'Formal Technical Review Archive'.
http://www.ics.hawaii.edu/~siro/
Software Program Managers Network
This site appears to be run by the US Department of Defense. Their Home Page says 'The Mission of the Software Program Managers Network is to enable managers of large-scale, software intensive development or maintenance projects to more effectively manage and succeed by identifying and
conveying to them management Best Practices, lessons-learned, and direct support. ' It looks good, even if you're not doing large scale government development. I spotted several links that I didn't have time to follow, that also looked good. Check this one out!
www.spmn.com
Formal Technical Review Archive:
Another site that needs dusting off (last updated July 31, 1996), this page contains an extensive bibliography, many papers that can be downloaded from the web, and links to organizations that use inspections in their everyday work, such as Motorola and NASA. Out of date or not, it's well worth
bookmarking!
http://www.ics.hawaii.edu/~johnson/FTR/
Gerald M. Weinberg's home page
www.geraldmweinberg.com
'Design and Code Inspections to Reduce Errors in Program Development'
by M.E. Fagan
This classic article was published in 1976, in The IBM Systems Journal. 'Fagan inspections' are commonly considered the most structured and formal variant. They are extremely efficient at removing errors, but they require the highest level of management commitment, training and resources.
Handbook of Walkthroughs, Inspections, and Technical Reviews, 3rd edition
by Daniel Freedman and Jerry Weinberg
Dorset House
ISBN 0-932633-19-6
First published in 1977 as the Ethnotechnical Review Handbook, and now in it's 3rd edition, Handbook is written in an informal, question and answer format that is unusual in a technical book. Because technical reviews and egoless programming are a 'people problem', not a technical one, the advice Handbook provides has held up well over the years, even though many of the technical details are out-of-date.
Software Inspection
by Tom Gilb and Dorothy Graham
Addison-Wesley, 1993
ISBN 0-201-63181-4
Gilb and Graham are convinced that the most rigorous formal inspections will pay off in higher quality, shorter schedules, and fewer bugs reaching the released product. Unfortunately, they downplay the value of less formal reviews, fearing that their smaller successes will encourage management to be satisfied with less than they could achieve. Even so, this is a good book, one that everyone who is interested in reviews should read.
Successful Software Process Improvement
by Robert Grady,
PTR Prentice Hall, May 1997
ISBN 0-13-626623-1
I haven't seen this book. My sources tell me it contains data from studies done at Hewlett Packard on their software process improvement efforts. It should contain hard data that you can show to upper management, to help justify reviews in your organization.
What Did You Say? The Art of Giving and Receiving Feedback
by Charles N. Seashore, Edith Whitfield Seashore, and Gerald M. Weinberg
available from:
Bingham House Books (410) 997-2829
10001 Windstream Drive, Suite 902
Columbia, MD 21044
This little paperback is the best tutorial on feedback and communication skills that I've ever read. And peer reviews will live and die on the skill with which team members give and receive feedback.
The Psychology of Computer Programming
by Gerald M. Weinberg
published 1971, by Van Nostrand Reinhold
(Note-a Silver Anniversary edition is planned, by Dorset House.)
This classic book includes the first mention of 'ego-less' programming.
Quality Software Management, vol. 4, Anticipating Change
by Gerald M. Weinberg
Dorset House, 1997
ISBN 0-932633-32-3
This book, the latest in Jerry's QSM series, talks a lot about doing effective reviews, and about how to 'grow' a healthy organization. It's got many valuable things to say to any manager who wants to change things at work.
Software Inspection, An Industry Best Practice
edited by Wheeler, Brykczynski, and Meeson
published 1996, IEEE Computer Society Press
ISBN 0-8186-7340-0
This book, a large format paperback, includes a foreword by Michael Fagan and 22 articles that discuss various aspects of inspections, along with an extensive bibliography, an author list, and a glossary. Most of the articles are reprints from other publications, such as IEEE Software, AT&T Technical Journal, IBM Systems Journal, and various conference proceedings. Software Inspection reprints Fagan's 1976 article, along with many others from the late 1980s and early 1990s. It includes sections on 'Introduction to Software Inspection', 'Experience Reports', 'Process Measurement', 'Inspection of Products Other Than Code', and 'Other Inspection-Related Work.'
I was especially interested in 'Key Lessons in Achieving Widespread Inspection Use', by Robert B. Grady and Tom Van Slack. This article describes the history of inspection adoption at Hewlett Packard, and discusses the lessons they have learned about encouraging the spread of process improvements throughout an organization.
You can find more information about this book (including the Table of Contents, Preface, and Foreword) at: http://www.computer.org/cspress/catalog/bp07340/insp-tc.htm
Copyright © by Sue Petersen. All Rights Reserved.