Analysis of the Scale Types and Measurement Units in Enterprise Architecture (EA) Measurement

Aim/Purpose: This study identifies the scale types and measurement units used in the measurement of enterprise architecture (EA) and analyzes the admissibility of the mathematical operations used. Background: The majority of measurement solutions proposed in the EA literature are based on researchers’ opinions and many with limited empirical validation and weak metrological properties. This means that the results generated by these solutions may not be reliable, trustworthy, or comparable, and may even lead to wrong investment decisions. While the literature proposes a number of EA measurement solutions, the designs of the mathematical operations used to measure EA have not yet been independently analyzed. It is imperative that the EA community works towards developing robust, reliable, and widely accepted measurement solutions. Only then can senior management make informed decisions about the allocation of resources for EA initiatives and ensure that their investment yields optimal results. Methodology: In previous research, we identified, through a systematic literature review, the EA measurement solutions proposed in the literature and classified them by EA entity types. In a subsequent study, we evaluated their metrology coverage from both a theoretical and empirical perspective. The metrology coverage was designed using a combination of the evaluation theory, best practices from the software measurement literature including the measurement context model, and representational theory of measurement to evaluate whether EA measurement solutions satisfy the metrology criteria. The research study reported here presents a more in-depth analysis of the mathematical operations within the proposed EA measurement solutions, and for each EA entity type, each mathematical operation used to measure EA was examined in terms of the scale types and measurement units of the inputs, their transformations through mathematical operations, the impact in terms of scale types, and measurement units of the proposed outputs. Contribution: This study adds to the body of knowledge on EA measurement by offering a metrology-based approach to analyze and design better EA measurement solutions that satisfy the validity of scale type transformations in mathematical operations and the use of explicit measurement units to allow measurement consistency for their usage in decision-making models. Findings: The findings from this study reveal that some important metrology and quantification issues have been overlooked in the design of EA measurement solutions proposed in the literature: a number of proposed EA mathematical operations produce numbers with unknown units and scale types, often the result of an aggregation of undetermined assumptions rather than explicit quantitative knowledge. The significance of such aggregation is uncertain, leading to numbers that have suffered information loss and lack clear meaning. It is also unclear if it is appropriate to add or multiply these numbers together. Such EA numbers are deemed to have low metrological quality and could potentially lead to incorrect decisions with serious and costly consequences. Recommendations for Practitioners: The results of the study provide valuable insights for professionals in the field of EA. Identifying the metrology limitations and weaknesses of existing EA measurement solutions may indicate, for instance, that practitioners should wait before using them until their design has been strengthened. In addition, practitioners can make informed choices and select solutions with a more robust metrology design. This, in turn, will benefit enterprise architects, software engineers, and other EA professionals in decision making, by enabling them to take into consideration factors more adequately such as cost, quality, risk, and value when assessing EA features. The study’s findings thus contribute to the development of more reliable and effective EA measurement solutions. Recommendation for Researchers: Researchers can use with greater confidence the EA measurement solutions with admissible mathematical operations and measurement units to develop new decision-making models. Other researchers can carry on research to address the weaknesses identified in this study and propose improved ones. Impact on Society: Developers, architects, and managers may be making inappropriate decisions based on seriously flawed EA measurement solutions proposed in the literature and providing undue confidence and a waste of resources when based on bad measurement design. Better quantitative tools will ultimately lead to better decision making in the EA domain, as in domains with a long history of rigor in the design of the measurement tools. Such advancements will benefit enterprise architects, software engineers, and other practitioners, by providing them with more meaningful measurements for informed decision making. Future Research: While the analysis described in this study has been explicitly applied to evaluating EA measurement solutions, researchers and practitioners in other domains can also examine measurement solutions proposed in their respective domains and design new ones.