Quality Assurance for Open Source Software Configuration Management

Abstract

Commonly used open source configuration management systems, such as Puppet, Chef and CFEngine, allow for system configurations to be expressed as scripts. A number of quality issues that may arise when executing these scripts are identified. An automated quality assurance service is proposed that identifies the presence of these issues by automatically executing scripts across a range of environments. Test results are automatically published to a format capable of being consumed by script catalogues and social coding sites. This would serve as an independent signal of script trustworthiness and quality to script consumers and would allow developers to be made quickly aware of quality issues. As a result, potential consumers of scripts can be assured that a script is likely to work when applied to their particular environment. Script developers can be notified of compatibility issues and take steps to address them.