Developing a configuration management tool compliance standard

Abstract

m Automation of software engineering activities through tool use has not been as successful as xexpected. In part, this is due to the difficulty of understanding a tool’s capabilities and performance characteristics, so that its usefulness for the organization can be assessed. The development of a standard for third-party certification of configuration management tools is about to begin. This article addresses the steps necessary to develop this standard, including identifying a broad base of participation, gaining the necessary international support to initiate the project, and ensuring that the effort can be completed in a reasonable amount of time. In addition, the result must maintain the balance necessary to be valuable to and welcomed by its three audiences: configuration management (CM) tool users, tool evaluators, and tool manufacturers. Moreover, insight into the standards development process is provided by addressing the boundary conditions which must be met: conformance with ISO/IEC guidelines and regulations, consistency with other standards, both existing and in the final stages of adoption, and liaison with various organizations. utomated support for software engineering processes (using, for example, CASE tools, environments, or workbenches) has not been as successful as many had expected. Of the various possible causes for this lack of success, such as the immaturity of tools and technology, the two most common are a lack of organizational readiness and commitment, and a misunderstanding of what a tool will and will not do. The newly adopted IEEE 1348, A Recommended Practice for the Adoption of CASE Tools, will help organizations establish realistic expectations for the use of automated tools in software engineering. An ISO/IEC technical report on the same topic is also being prepared. A major source of dissatisfaction among tool users is their perception that tools frequently don’t perform as advertised or as expected. Their disappointment is usually the result of inadequate tool evaluation. Evaluating complex technologies is an expensive, time consuming task that requires special expertise. Few organizations have the resources to evaluate a tool thoroughly enough to come to a full understanding of how it would perform in that organization’s environment. While it is not clear what constitutes a sufficient level of understanding, the difficulties experienced by organizations in adopting CASE clearly indicate that that level is not always reached. One way to alleviate this problem is to provide for thirdparty evaluation and certification to ensure that a tool meets certain specific requirements. In 1995 two new international standards were adopted which provide the framework for an ambitious undertaking: the development of a configuration management (CM) tool compliance standard.