Error Detection in Software Engineering

Last March, Claude Y. Laporte, a software engineering professor at École de technologie supérieure (ÉTS) in Montreal, was the guest of a science show during which he explained the general concepts of this new engineering discipline. This article is about Software quality issues.

____________________________________

Broadcast Sundays from 12:10 pm to 2:00 pm, Les années lumière, a radio show (ICI Radio-Canada) focusses on science and related topics including health, the environment, fundamental research, demography, city planning, etc. Mr. Claude Y. Laporte, a software professor at l’ÉTS, recently participated in “La règle de 3” (The Rule of 3), a segment during which researchers are invited to appear over a period of 3 weeks where they present a different major topic each week.

After a first part dedicated to software engineering definition and a second on software quality, Mr. Laporte approaches this week error Detection in Software Engineering in the manufacturing and production of software.

Are defects inevitable?

Originally used to describe the presence of insects in electromechanical computers, the word “bug” has, unfortunately become an integral part of the software engineering world. We no longer find real insects in our computers, but defects are still being injected involuntarily throughout the development process from the translation of client requirements into design specifications, the design of the software architecture, the coding of the software itself, software testing and even software documentation.

The complexity of the software is, itself, the main reason why so many software errors exist and persist

If software was linear, the task would be much easier, but software code is always composed of a large set of decision points (true/false questions such as data comparisons, etc.). Each of the answers to these questions leads to a unique software path (called a “branch”) which must not only be coded but also tested. In general, there is approximately 1 branch for every 10 lines of code. A very small, but critical software[1] consisting of 100 different branches (i.e. 1,000 lines of code) might require several thousands of tests in order to eliminate the defects.

When you consider the fact that the software in our automobiles, airplanes, cell phones, medical systems and subway systems are composed of millions of lines of code, you can imagine the variety and number of defects that might appear as well as the complexity and level of effort which might be required to detect and correct them!

The Importance of Establishing Good Software Engineering Practices

One of the founding fathers of modern software engineering, Watts Humphrey, performed a study involving approx. 800 experienced developers in order to examine the software development process in more detail. During this study, he discovered that, on average, every 10 lines of code included a defect. This result made it quite evident that how crucial it was to develop better software practices to detect and correct these defects and to reduce the number of defects in new programs.

Developed by a former IBM Product Development Manager, Michael Fagan, the Fagan Inspection Process, described in software engineering standards, consists of performing peer reviews and rigorously studying all phases of the development process including the creation of specifications, the architecture, the tests and, finally, the code itself. As one example, when a particular, large American company published its data, it was found that 50% of the software defects were being introduced in the specifications phase while another 20% were being injected when the architecture was being developed. This meant that 70% of all the defects were being introduced into the product before even a single line of code had been written! As a result of its study, this company implemented rigorous software engineering practices and now detects more than 90% of the defects injected at each stage of the development process. This has dramatically reduced its costs of rework.

It should be noted that software engineers are not solely responsible for introducing software defects! Software developers, architecture designers, and even the front-end, requirements engineering specialists are also injecting defects. It was critical to develop good diagnostics and defect-detection software (which, of course, can themselves contain defects!) in order to find and eliminate as many sub-system faults as possible. Today, automobile diagnostics software can include up to a third of a million lines of code dedicated to diagnostics. This code informs the user about the operating status of various sub-systems, such as the brakes, and can tell him when they are defective or require maintenance.

The cockpit of the Airbus A-380: Lots of software systems and millions of lines of code!

Zero-Defect Software?

Capers Jones performed a study in the United States which found that, on average, manufacturers were eliminating 85% of defects. This means, however, that 15% of the defects go undetected! On the bright side, the statistics are much better (5%) among companies who have implemented rigorous software processes with software detection and correction practices such as peer reviews or inspections. This is a relatively low number, but when we are talking about automobile software involving about 20 million lines of code, the absolute number of residual defects can be quite large!

Shouldn’t we adopt 95% defect-elimination as the minimum standard for professionals who develop and maintain critical software?

The Future of Software Engineering

In view of the numerous and growing industrial applications and their importance in today’s society and ever year, at l’ÉTS, there is a growing number of new enrollments. It is evident that software engineering is still in its infancy, and engineering, as a discipline, is continuously being pressed to grow and improve. Today, in order to be effective, software developers must not only learn how to develop good software, but must also learn the detailed intricacies and special requirements of the various software clients who work in a wide range of fields (medical, pharmaceuticals, transportation, communications, etc.).

To listen to this radio interview (in French), click here.