7OUTSOURCE

This essay presents a tutorial that discusses software quality in IBM in the context of total quality management (TQM). Beginning with a historical perspective of software engineering, the tutorial examines the definition of software quality and discusses TQM as a management philosophy along with its key elements: customer focus, process improvement, the human side of quality, and data, measurement, and analysis. It then focuses on the software-development specifics and the advancements made on many fronts that are related to each of the TQM elements. In conclusion, key directions for software quality improvements are summarized.

History

In the 1970s, when software development was characterized by massive schedule delays and cost overrun, the focus was on planning and control of software projects. In the 1980s, hardware costs continued to decline. Information technology permeated every fact of our institutions, and at the same time it became available to individuals. As competition in the computer industry became keen and low-cost applications became widely implemented, the importance of productivity in software development increased significantly. Various cost models in software engineering were developed and used. In the late 1980s, the importance of quality was also recognized.

The 1990s and beyond is certainly the quality era. As state-of-the-art technology is now able to provide abundant functionality, customers demand high quality. Demand for quality is further intensified by the ever-increasing dependence of our society on software. Billing errors, large-scale disruptions of telephone services, and even a missile failure during the recent Gulf War can all be traced to the issue of software quality. In this era, quality has been brought to the centre of the software development process. From the standpoint of software vendors, quality has become a necessary condition to compete in the marketplace.

Meaning and definition of software quality

Quality must be define and measured for improvement to be achieved. Yet, a major problem in quality engineering is that the work quality lacks a commonly recognized operational definition. Perhaps the confusion is because quality is not a single idea, but a multidimensional concept.

Your customers are in a perfect position to tell you about Quality, because that’s all they’re really buying. They’re not buying a product. They’re buying your assurances that their expectations for that product will be meet.

From a high-level definition of a concept, to a product being operationally defined, many steps are involved, each of which may be exposed to possible shortcomings. For example, to achieve the state of conformance to requirements, customers’ requirements must first be gathered and analyzed, then specifications from those requirements must be developed, and the product must be developed and manufactured appropriately. In each phase of the process, errors may have occurred that will negatively affect the quality of the finished product. The requirements may be erroneous (especially in the case of software development), the development and manufacturing process may be subject to variable that induce defects, and so forth. Form the customer’s perspective, satisfaction after the purchase of the product is the ultimate validation that the product conforms to requirements and is fit to use. From the producer’s perspective, once requirements are specified, developing and producing the product in accordance with the specifications is the basic step to achieving quality. Usually, for product quality, lack of functional defeats and good reliability are the most basic measures. In order to be “fit for use,” the product first has to be reliably functional.

Software quality is that of process quality versus end-product quality. From requirements to the delivery of software products, the development process is complex. It often involves a series of stage, each with feedback paths. In each stage, an intermediate deliverable is produced for an intermediate user–the next stage. Each stage also receives an intermediate deliverable from the preceding stage. Each intermediate deliverable has certain quality attributes the affect the quality of the end product. Intriguingly, if we extend the concept of customer in the definition of quality to include both external and internal customers, the definition also applies to process quality. If each stage of the development process meets the requirements of its intermediate user (next stage). The end product thus developed and produced will meet the specified requirements. This statement, of course, is an oversimplification of reality, because in each stage numerous factors exist that will affect the ability of the stage to fulfil its requirements completely. However, if each person performing an activity thought about the customers of the intermediate product being developed, and applied the concepts discussed above, we would come a long way toward improving the final product quality.

Another view of process quality is aimed at improving the processes themselves, i.e., defining a set of ideal processes and measuring the existing processes of organizations against these ideals. The concept has become very popular in the last decade and provides a mechanism for companies to be related with regard to process.

To improve quality during development, we need models of the development process, and within the process we need to select and deploy specific methods and approaches and employ proper tools and technologies. We need measures of the characteristics and quality parameters of the development process and its stages. We need metrics and quality models to help ensure that the development process is under control to meet the quality objective of the product.

Total quality management in IBM software

Total quality management (TQM) is a term that was originally coined in 1985 by the Naval Air Systems Command to describe its Japanese-style management approach to quality improvement. It has taken on a number of meanings, depending on who is interpreting the phrase and how they are applying it, but in general, it represents a style of management that is aimed at achieving long-term success by linking quality with customer satisfaction. Basic to the approach is the creation of a culture in which all members of the organization participate in the improvement of processes, products, and services.

Since the 1980s, many companies in the United States have begun adopting the TQM approach to quality. The Malcolm Baldrige National Quality Award (MBNQA), established by the U.S. government in 1988, highlighted the embracement of such a philosophy and management style. In the computer and electronic industry, examples of successful implementation include Hewlett-Packard Co.’s total quality control (TQC), Motorola Inc.’s six sigma strategy, IBM’s market-driven quality, and others. In fact, Motorola won the first MBNQA award in 1988, and IBM’s AS/400 Division in Rochester, Minnesota, was one of the winners in 1990.

Hewlett-Packard’s TQC focuses on key areas such as management commitment, leadership, customer focus, total participation, and systematic analysis. There are strategies and plans in each from causal analysis meetings. The team is also involved in providing feedback to the organization, reports to management on the status of its activities, publicizing success stories, and taking the lead in various aspects of the process.

Stage kickoff meetings
These meetings are conducted by technical members at the beginning of each development stage. They serve as the key preventive measure as well as primary feedback mechanism of progress made. The emphasis is on the technical aspect of the development process as well as on quality: “What is the right process? What are the tools and methods that can help? What are the common errors to avoid? What improvements and actions have been implemented?”

Action tracking and data collection
An action database is needed to track action status, to facilitate implementation, to prevent action items from being lost over, and to enhance communications among groups.

Different from post-mortem analysis, the DPP is a real-time process, integrated into every stage of the development process. Through the action teams and action tracking tools and methodology, DPP provides a systematic, objective, data-based mechanism for action implementation. It injects intelligence into the development process and enables the development process to refine itself. Developed and first used by IBM Research Triangle Park in the mid-1980s, it is now being used by many other IBM development organizations and by other companies in the software industry. DPP can be applied regardless to the type of development process.

Design reviews
Designs reviews, code inspection, and code walk-through or code reading are the classical techniques to ensure in-process quality. When used in the waterfall process, they are usually part of the exit criteri for development phases. For instance, when the high-level design is done, a formal review is held; design defects found by the review must be fixed before existing the phase (to the low-level design phase). Recent improvements in reviews and inspections include: the phased inspection method, in which specific tasks are assigned to specific inspectors and two or more phases of inspection (of different complexity levels) are conducted; various inline review tools (for example, REVUFILE, used at the IBM Santa Teresa laboratory); and the use of multimedia technology (for example, the Santa Teresa laboratory’s multimedia approach that includes large-screen projector, BARCO machine, Personal System/2* computers, intelligent source code editor, and other tools). Improvements in theses techniques will lead to better quality of the front-end development process.

It should be cautioned that software reviews and inspections are distinctly different from manufacturing inspections. The latter are at the back end of the production process and are known to be a poor method for quality assurance. TQM teachings often call for the abandonment of manufacturing inspections in favour of acceptance sampling (with the front-end focus on design quality). Software reviews and inspections, on the contrary, are the vital techniques at the front end of the software development process. To have reviews and inspections by peers on software design and implementation is beneficial. As new languages emerge that can make the mundane task of code implementation a lot simpler (for example, languages with strong typing), the burden of code inspection can be lessened. However, design reviews or inspections well become ever more important, regardless of the development process.

Formal methods. In computation theory, computer scientists have developed models based on mathematical formalisms. These formalisms include, for example, predicate calculus, functional verification, and state machines. Because of the difficulty in scaling up to reasonable-sized systems and the mathematical training required, these models have not been used effectively in practice.

The last several years have been seen the initial applications of formal methods in software development. Examples include the Vienna Development Method (VDM), Z notation, Input/Output Requirements Language (IORL), and the Clean room methodology. It appears that Z notation and VDM have primarily has been used by developers in Europe, whereas Clean room projects have taken place mostly in the United States.

The Clean room methodology involves box structure specification of user function and system object architecture, function-theoretic design and correctness verification, and statistical usage testing for quality certification. Clean room project development is based on incremental development and certification of the pipeline of user-function increments that accumulate into the final product. Since the early pilot projects in 1987 and 1988, more than a dozen projects have been completed with a total size of more than half a million lines of code. The average defect rate found in first-time execution was 209 defects per thousand lines of code (KLOC), which is significantly better than the industry average. To facilitate the adoption of Clean room, a phased implementation approach has recently been proposed.

Design paradigms
The object-oriented approach to design and programming, which was introduced in the 1980s, represented a major paradigm shift in software development. This approach will continue to have a major effect in producing software for many years to come. Like the paradigm of structural design and functional decomposition, the object-oriented approach will become a major cornerstone in software engineering. It influence on software reuse and productivity has already proven to be profound. To date, a number of object-oriented development projects have been successfully completed in the industry. Several of the finished products are integrated support environments for object-oriented development. After the initial learning curve, these projects showed significant increases in productivity and quality.

Programming languages. Good performing practices and programming languages that support the object-oriented paraigm continue to evolve. Object-oriented languages such as Ada**, Objective-C**, C++, and Smalltalk usually have strong typing and can enable object coherence checks. The object-based language Modula-2 has similar characteristics in terms of pre- and post-condition checking. These characteristics will lead to better development quality. Our limited experience so far (for example, projects at the IBM Rochester development laboratory) has lent support to this argument.

Automated software synthesis is a term now used to describe the translation of very-high-level languages (VHLL) into machine code. Current research in this area could bring a breakthrough in software development in terms of both productivity and quality. In software development, the higher the level of the programming language used, the more true productivity achieved (the more functionality implemented by the same amount of source code). When the automated software synthesis technology is mature and transferred, very-high-level languages can be used as the development languages.

Development environments

The last decade has also seen a significant improvement in development environments and tools. Integrated support systems have enabled configuration management, complex library systems, and change control for large-scale development. For large developers most of these support systems are developed internally, but support-system products are available in the industry for different software platforms.

With the advent of powerful workstations, software development is shifting to distributed environments. With ample processing power on the individual developer’s desk and the availability of CASE tools, the outcome is more efficiency, higher productivity, and better quality.

Software reuse

The software industry has been continuously “reinventing the wheel” in terms of code development. During the past several years, code reuse has been receiving the attention for which it was long overdue. Some developers have been systematically stepping up their efforts in reuse. By using proven design and code in new software projects, both productivity and quality can be increased. Reusable parts, however, must meet pre-established criteria and must be of excellent quality. Parts with latent defects, and which are reused widely, can have a disastrous effect on new products. Software reusable parts should be certified.

The object-oriented approach should make reuse easier. Low-level object-oriented reusable components are available in the industry in the form of class libraries.

In software development, reuse should not be limited to code. Project experience, defect models, process models, and so forth, should be reused to the extent possible for effective development. In this broader sense of reuse, the Experience Factory approach established by the NASA Software Engineering Laboratory (SEL) can provide a good situation. As discussed earlier, the Experience Factory is a logical or physical organization that undertakes systematic learning and packaging of reusable experiences. It supports project development by acting as a repository for experiencing, analyzing and synthesizing the experience, and supplying the experience to various projects on demand. It evaluates experience and builds models and measures of software processes, products, and other forms of knowledge. It uses people, documents, and automated support to do so. Other than by NASA SEL, the Experience Factory approach is being experimented with and used by some developers. For instance, the software development laboratory of IBM Toronto for the past three years has established an experience warehouse, an early form of the Experience Factory, to facilitate the reuse of software experiences.

Data, measurement, and model

What is measured is improved. Data and measurements are the most basic prerequisites for the improvement and maturity of any scientific or engineering discipline. Yet, in the discipline of software engineering, this area is perhaps one that has many critical problems and one that needs concerted effort for improvement.

The use of measurements, metrics, and models in software development assumes the availability of good data. In fact, the poor quality of data is a large obstacle in quality improvement. In general, data gathered during the formal machine testing phases are more accurate than those collected at the front end, such as requirements analysis and design reviews. Some companies do not even collect data at the front end of the development process, and for them in-process quality management means only monitoring data during the formal testing phases. To enhance data accuracy, a good tracking system for the entire development process must be in place, and the system must address the data validation issue. Such a system enables data-based decision-making for project and quality management. The amount of data to be collected and the number of metrics to be used need not be overwhelming. It is more important that the information extracted from the data be focused, accurate, and useful.

With regard to models in software engineering, cost and schedule models have moved from research and development into application. Models on software quality have also emerged. They can be broadly classified into three categories, each for a separate purpose: the reliability models, for reliability assessment and projection; the quality management models, for managing quality during the development process; and the complexity models and metrics, which can be used by software engineers for quality improvement in their work.

Software reliability modelling is more mature than the other two types, and numerous software reliability models exist. Simply put. Reliability models treat the software product as a black box, monitor its external behaviour, and use sophisticated statistical extrapolation methods to predict its reliability.

Human side of software quality

In the TQM philosophy, the human side of quality includes factors such as total participation, management commitment and leadership, employee buy-in and empowerment, and other social and cultural factors. Of these factors, perhaps commitment, both from bottom-up and top-down, is the most important. Without commitment from the entire organization, the chance for success is slim. Commitment can best be measured by individual behavioural changes. There are criteria for an organization to assess whether it is truly practicing TQM. The first question is on the behavioural changes of management. Specially, if the decisions made to improve quality expect behavioural changes of the employees but not of the executives and management, the organization is practicing quality by proclamation instead of TQM.

In software development at the operational level, TQM can be viewed as the integration of project, process, and quality management. During the past few decades, management teams of successful software developers have accumulated vast experience in project and schedule management. To avoid cost and schedule overruns, schedule progress is often managed at the microscopic level. In contrast, there is much less experience in process and quality management, especially during the development cycle. Product and development managers must manage in-process quality in the way in which they manage schedule for quality improvement to happen. Quality, process, and schedule management must be totally integrated for software development to be effective. Some developers in the industry have started doing so. It may take some time, however, for this integrated software project management style to be ingrained in practice.

Conclusion

We have discussed the definition of software quality, the total quality management (TQM) approach to quality improvement, and the advancements on many fronts of software engineering as they relate to the key TQM elements: customer focus, process, the human side of quality, and data, measurements, and analysis. In this modern quality era, TQM is widely embraced by numerous industries. However, the gap between TQM as a management style and the operational quality improvements in specific engineering disciplines is seldom understood and often ignored. We have tried to show the relevance of TQM in software development by discussing software-specific topic and their progress in the TQM framework.

Process, product, and quality models and other forms of structured experience have been defined, use, or accumulated, and will continue to aid in the practical engineering of software. New technologies (object-oriented paradigm, automated software synthesis, etc.) have emerged. Existing technologies are becoming better focused (customer feedback process, prototyping, DPP), more disciplined (refuse, design reviews, measurement approach), and more practical (formal development methods). The new advancements in social psychology and software sociology will enrich software engineering. However, significant challenges remain. Transferring software technologies into development organizations and bridging the gap between the state of the art and the state of practice needs concerted efforts by both researchers and practitioners. The use of quantitative approaches in software development, with feedback and learning through model-based measurements, needs to become ingrained in practice.

To bring about TQM in software development, we must take a systematic engineering approaching to address the improvements of the numerous elements of software engineering, many of which we briefly covered in this tutorial. In contrast, the holistic TQM framework will aid software engineering to mature. Software engineering must move in this direction to become a true engineering discipline and to meet the increasing demands from society for effective development with high-level quality.

Finally, like software development, software quality is a domain-specific expertise. Software management and software developers are responsible for the implementation of software quality engineers play a very important role in process improvement, measurement, analysis, evaluation, and recommendation. Upper management leads the quality improvement direction and ensures a constant focus on continuous improvement. Significant improvement in software quality can be realized and sustained only through the total participation of all who are involved.

Reference:

[1.] A. V. Feigenbaum, Total Quality Control: Engineering and Management, McGraw-Hill Book Co., New York (1961).

[2.] A. V. Feigenbaum, Total Quality Control, McGraw-Hill Book Co., New York (1991).

[3.] K. Ishikawa, What Is Total Quality Control? The Japanese Way, Prentice-Hall, Inc., Englewood Cliffs, NJ (1985).

[4.] D.N. Card and R. L. Glass, Measuring Software Design Quality, Prentice-Hall, Inc.,Englewood Cliffs, NJ (1990).

[5.] S. H. Kan, “Modeling and Software Development Quality,” IBM Systems Journal 30, No. 3, 351-362 (1991).