|
Cleanroom Summary |
The Cleanroom software engineering process was originally published by Dr. Harlan Mills in 1987. It is modeled from the field of electronics with the main focus on preventing software defects from the beginning of the process rather than fixing them at later stages.
The Cleanroom approach is team based and divides software development into four categories: Management Process, Specifications Process, Development Process, and Certification Process. Each of the processes is further divided into activities, each with predefined inputs and outputs.
The strengths of the Cleanroom approach include: the proactive approach to quality management and requirements validation, functional models are developed in a step by step approach, customer involvement, the formal validation and peer review of the software significantly improves software quality, the iterative approach to development allows the project to respond to changing requirements, progress is measured, the use of statistics to evaluate and certify software quality, and the involvement of multiple individuals in each step of the process allows for knowledge transfer and peer review.
The weaknesses of the Cleanroom approach include: individuals involved in development and management require a strong mathematical and statistical background, the formal validation and statistical analysis is time consuming, the approach does not specifically deal with project risks, requirements must be stable for each iteration though they may not be complete, statistical testing methods may miss certain functionality, and the team approach make increase the length of time in the software cycle.
The lecture presented the use of the Cleanroom process by Raytheon Systems as a case study. Raytheon Systems used portions of the cleanroom process with the goal of increasing productivity and improving quality. Due to the real-time nature of the software projects, Raytheon Systems modified the process of requirements analysis and testing. A number of other examples of use of the Cleanroom process in industry and academia were provided including: IBM's Cobol Structuring Facility, NASA's Goddard Space Flight Center, the US Department of Defense's Picatinny Arsenal, Ericsson, the University of Tennessee, John Shipman from New Mexico Tech, and Dr. Victor Basili from the University of Maryland. As well, a number of organizations which provide information or consultation regarding Cleanroom practices were listed.
The Cleanroom process was discussed in terms of it's compliance with CMM with the conclusion that while it meets a number of CMM's requirements, it does not address all the requirements.
The Cleanroom process was compared to other software engineering methodologies including: OO, SSM, JAD/PD, QFD, SASD, and Agile Technology.
The exercise was to develop a software system definition for a C-Train vending machine using the Cleanroom approach. Specifically, a functional view of the problem, a black box representation, a state box representation, a clear box representation, and usage specifications were to be defined. The system was to include the following requirements: the user should be able to purchase one-way or day pass tickets for youth and adult, each of the aforementioned tickets has a different price value, and the machine requires the exact amount using any combination of coins but it will not return change.
1. Functional view of the C-Train Vending Machine
Ticket=f(Ticket selection, Money)
| Cleanroom vs. QFD | While both the Cleanroom process and QFD focus on quality, the definition of quality differs. In Cleanroom, quality is primarily measured through formal mathematical and statistical validation against the requirements, whereas QFD focuses more on quality from an organizational or customer's viewpoint with the focus on value provided instead of the number of bugs per lines of code. |
| Cleanroom vs. SSM | Though both Cleanroom and SSM encourage user participation in the requirements phase, there is very little in common. Cleanroom mainly focuses on system quality from a "number of bugs" perspective, SSM addresses quality from the viewpoint of various stakeholders and the impact the system would have on them. |
| Cleanroom vs. SASD | Both the Cleanroom process and SASD provide a means of user involvement in the requirements elicitation, definition and verification stage. As well, both provide a means to model the system design from a high level view to a more detailed level. However, Cleanroom provides an overall model of the software life cycle whereas SASD focuses specifically on the requirements analysis and design stages. |
| Cleanroom vs. JAD/PD | Both the Cleanroom process and JAD have a high degree of user involvement and acceptance during the requirements elicitation and analysis phase. However, JAD involves end users during the design phase whereas Cleanroom does not. |
| Cleanroom vs. Agile Processes | Both the cleanroom approach and Agile Processes use an incremental approach to development and customer validation of the system's ability to address requirements. However, agile processes do not incorporate the formal mathematical validation component that Cleanroom does. |
The main strengths of the Cleanroom approach is the focus on validation of software requirements and project planning with the user, the formal mathematical and statistical methods used in code validation and testing, and the statistical monitoring of the project. These qualities help ensure that the system meets the customer's requirements and that the quality of the system is high in terms of the ability of the system to carry out the desired functionality accurately. As a result, this method is well suited to systems which have a high degree of risk in terms on the possibility of impacting life or the environment (ex. nuclear power plant) or systems which are expensive to produce and fix if there is a bug such (ex. computer chip circuit). As well, the quantitative measurements made during the development cycle allow analysis on techniques used to be performed with the goal of identifying and correcting weak elements.
As mentioned during the presentation and in the case study provided, the main draw back of this method is the strong knowledge needed in mathematical proofs and statistical analysis. In my experience, most users and developers do not have the necessary knowledge of mathematical and logical proofs needed to perform the formal validation. A second drawback is the time involved in formally validating algorithms an in involving the whole development team in code review. It is important for the sponsoring organization to assess the risks of the system being developed and determine whether the formal methods should be used or whether system failure can be handled effectively and therefore the formal methods may cost more than they benefit given the alternative.
When determining whether the Cleanroom approach will be used during system development, it is important to consider the environment in which the proposed system is to operate. For example, if the system being developed is a banking application and it is to run using a database which is unstable, the cleanroom approach may provide a high assurance of accuracy within the application, however, the system as a whole will be unstable and the benefits from the formal methods may not be realized.
Group 1 Cleanroom Presentation