From Data Analysis to Human Input: Navigating the Complexity of Software Evaluation and Assessment

Abstract

It is the time of trust and transformation in software. We want explainable AI to assist us in dialogue, write our programs, test our software, and improve how we communicate. It is the time of digitalization, but we must ask ourselves - on what data in what format, when do we collect it, and what is the source? Does “data” make sense? Every action can be automated, should eventually be automated, and as such should be traceable and explainable. The transformation of software – and how we can now train, and feedback in a fast way, enable us to not only utilize existing technologies, but also aids us in faster embracing new technologies. This transformation is much to slow even if things change at a lightning speed. Change is the only thing we can be sure will happen. Evaluating and assessing quality of software sounds easy but is only as good as you design it to be. We, often simplify the problem so we can move forward, but it is the complications that is the real issue – our context, our combination of tools, languages, hardware, history, and way of working. We simply need the labeling, the meta-data, the context – and this data in a form with “many” perspectives to draw the more “accurate” scientific picture. Having a multi-facetted perspective is important when analyzing complex contexts. In software, listening skills and asking the right questions to the right people is often invaluable to complement blunt data. On the other side - much information is probably missing as you are too easily getting “only” what you asked for. So, we cannot judge what we cannot observe – and analyzing this data, is another issue all together. We need to know what is right – because if we cannot trust the source – or double check the outcome, how would we know it is not just a “fake” data? What does the outlier really mean? Is it a sign of a new trend is it the first time we capture this odd event? Therefore, it is easy to lose perspective in a fast-changing world. Despite drowning in tools, we still miss a lot of them. The threshold of using a tool is high, as we cannot trust them, and we cannot be sure that the data these tools collect does represent what we want to investigate. Therefore, the role of the scientist is more important than ever. Trusting the scientific process, utilizing multiple methods, and combining them is the receipt! Another goal is doing our best to select topics and collaborators – as building better software (quality) for humanity. It starts with you and me. I hope I will in this context be able to touch upon areas like security, testing, automation, AI/ML, ethics and “human in the loop”, analysis, tools, and technical debt, with a focus on evaluations and assessments.