On Evaluating Multi-level Modeling

Abstract

Multi-Level Modeling is receiving increasing levels of interest and its active research community is continuing to make progress. However, to advance the discipline effectively it is necessary to increase industry adoption and achieve better community cohesion. We believe that the key to addressing both these challenges is to promote the creation of more comparisons in the multi-level modeling field based on meaningful objective evaluations. In this position paper, we provide our view on what constitutes meaningful evaluations and discuss some of the issues involved in obtaining them, while presenting a broad overview of existing multi-level modeling evaluations. In particular, we emphasize the importance of understanding and managing the difference between internal and external qualities.