Methodological features of in-group evaluation of experts’ competence in determining the efficiency of space-rocket complexes

Expert examination methods greatly facilitate the solution of difficult-to-formalize problems. However, in this case the solution is affected by a subjective factor. The decision-making theory has a number of methodological techniques that diminish its effect on the decision made. This paper presents a method of quantitative evaluation of experts’ competence from the results of an expert examination of the efficiency determination of unique, technically complex systems of special and dual purpose, in particular space-rocket complexes. In an expert examination of projects of such systems, it is suggested that the experts’ competence be quantitatively evaluated in two stages: a preliminary evaluation of the experts’ competence from their factual data and a refined evaluation of the experts’ competence just before the calculation of the expected indices of target efficiency using the results of expert examinations made by the procedure developed. The proposed method of quantitative evaluation of experts’ competence is based on evaluating the qualification of the experts involved in the target efficiency determination of a complex engineering system. A rank matrix constructed on the basis of partial criteria of technical efficiency and additional factors of indirect control is proposed as a tool to eliminate cases where at a high level of expert evaluation consistency the most accurate expert evaluations may be considered anomalous in the expert evaluation of the technical and target efficiency of space-rocket systems. The presented mathematical model of quantitative evaluation of experts’ competence includes parameters that adjust the mathematical model to specific conditions of the expert evaluation (expert evaluation methods employed, measurement scales, specific limitations, etc.). The mathematical model is constructed around the axiom that the “true” estimates of the significance of the objects under evaluation lie within the expert evaluation domain. The paper also presents an enlarged algorithm for adjustment parameter calculation from the results of expert estimate preprocessing. The presented mathematical model and algorithm make it possible to develop a computer program for determining experts’ competence from expert evaluation results.