On Retrofitting Provenance for Transparent and Fair Software - Drivers and Challenges

Abstract

There have been ongoing discussions about how to ensure transparency and fairness in software that utilise artificial intelligence (AI). However, transparency and fairness are not limited to AI. Modern (non-AI) software is often constructed in a black box fashion. This means, components and services provide some functionality, but details on how this is achieved are hidden. Common software development and design principles like encapsulation and information hiding promote this. Engineers often only look inside the black boxes when they need to fix problems, e.g., when tracing bugs or vulnerabilities. The demand for transparency has created a need to open those black boxes also to non-engineers. For instance, businesses need to demonstrate regulation compliance, and end users want to understand how systems make fair decisions that affect them. However, adding provenance (i.e., the ability to gather information about data and algorithms used in systems) to existing systems is invasive and costly, and current approaches to collect provenance data are not designed to expose data to end users. We argue that this requires “provenance retrofitting”, i.e., adding provenance capabilities to systems mechanically, and exposing provenance data through standard language and service application programming interfaces (APIs). This could facilitate an infrastructure that supports transparency, which then can in turn be used to create feedback mechanisms for users that in the long term can improve the fairness of software. In this paper we discuss drivers, objectives, key challenges and some possible approaches to provenance retrofitting.