A comprehensive evaluation methodology for domain specific software benchmarking

Abstract

With the wide using of information system, software users are demanding higher quality software and the software developers are pursue the “best-in-class” practice in a specific domain. But till now it is difficult to get an evaluation result due to lack of benchmark method. This paper proposes a systematic, practical and simplified evaluation methodology for domain specific software benchmarking. In this methodology, an evaluation model with software test characteristics (elementary set) and domain characteristics (extended set) is described; in order to weaken the uncertainty of subjective and objective, the rough set theory is applied to gain the attributes weights, and the gray analysis method is introduced to remedy the incomplete and inadequate information. The method is useful to assess characteristics, sub-characteristics and attribute that influence the domain specific software product quality qualitative and quantitative. The evaluation and benchmarking results prove that the model is practical and effective.