Vijay Arya, R. Bellamy, Pin-Yu Chen, Amit Dhurandhar, M. Hind, Samuel C. Hoffman, Stephanie Houde, Q. Liao, Ronny Luss, A. Mojsilovic, Sami Mourad, Pablo Pedemonte, R. Raghavendra, John T. Richards, P. Sattigeri, Karthikeyan Shanmugam, Moninder Singh, Kush R. Varshney, Dennis Wei, Yunfeng Zhang
This tutorial will teach participants to use and contribute to a new open-source Python package named AI Explainability 360 (AIX360) (https://aix360.mybluemix.net), a comprehensive and extensible toolkit that supports interpretability and explainability of data and machine learning models. Motivation for the toolkit. The AIX360 toolkit illustrates that there is no single approach to explainability that works best for all situations. There are many ways to explain: data vs. model, direct vs. post-hoc explanation, local vs. global, etc. The toolkit includes ten state of the art algorithms that cover different dimensions of explanations along with proxy explainability metrics. Moreover, one of our prime objectives is for AIX360 to serve as an educational tool even for non-machine learning experts (viz. social scientists, healthcare experts). To this end, the toolkit has an interactive demonstration, highly descriptive Jupyter notebooks covering diverse real-world use cases, and guidance materials, all helping one navigate the complex explainability space. Compared to existing open-source efforts on AI explainability, AIX360 takes a step forward in focusing on a greater diversity of ways of explaining, usability in industry, and software engineering. By integrating these three aspects, we hope that AIX360 will attract researchers in AI explainability and help translate our collective research results for practicing data scientists and developers deploying solutions in a variety of industries. Regarding the first aspect of diversity, Table 1 in [1] compares AIX360 to existing toolkits in terms of the types of explainability methods offered. The table shows that AIX360 not only covers more types of methods but also has metrics which can act as proxies for judging the quality of explanations. Regarding the second aspect of industry usage, AIX360 illustrates how these explainability algorithms can be applied in specific contexts (please see Audience, goals, and outcomes below). In just a few months since its initial release, the AIX360 toolkit already has a vibrant slack community with over 120 members and has been forked almost 80 times accumulating over 400 stars. This response leads us to believe that there is significant interest in the community in learning more about the toolkit and explainability in general. Audience, goals, and outcomes. The presentations in the tutorial will be aimed at an audience with different backgrounds and computer science expertise levels. For all audience members and especially those unfamiliar with Python programming, AIX360 provides an interactive experience (http://aix360.mybluemix.net/data) centered around a credit approval scenario as a gentle and grounded introduction to the concepts and capabilities of the toolkit. We will also teach all participants which type of explainability algorithm is most appropriate for a given use case, not only for those in the toolkit but also from the broader explainability literature
{"title":"AI explainability 360: hands-on tutorial","authors":"Vijay Arya, R. Bellamy, Pin-Yu Chen, Amit Dhurandhar, M. Hind, Samuel C. Hoffman, Stephanie Houde, Q. Liao, Ronny Luss, A. Mojsilovic, Sami Mourad, Pablo Pedemonte, R. Raghavendra, John T. Richards, P. Sattigeri, Karthikeyan Shanmugam, Moninder Singh, Kush R. Varshney, Dennis Wei, Yunfeng Zhang","doi":"10.1145/3351095.3375667","DOIUrl":"https://doi.org/10.1145/3351095.3375667","url":null,"abstract":"This tutorial will teach participants to use and contribute to a new open-source Python package named AI Explainability 360 (AIX360) (https://aix360.mybluemix.net), a comprehensive and extensible toolkit that supports interpretability and explainability of data and machine learning models. Motivation for the toolkit. The AIX360 toolkit illustrates that there is no single approach to explainability that works best for all situations. There are many ways to explain: data vs. model, direct vs. post-hoc explanation, local vs. global, etc. The toolkit includes ten state of the art algorithms that cover different dimensions of explanations along with proxy explainability metrics. Moreover, one of our prime objectives is for AIX360 to serve as an educational tool even for non-machine learning experts (viz. social scientists, healthcare experts). To this end, the toolkit has an interactive demonstration, highly descriptive Jupyter notebooks covering diverse real-world use cases, and guidance materials, all helping one navigate the complex explainability space. Compared to existing open-source efforts on AI explainability, AIX360 takes a step forward in focusing on a greater diversity of ways of explaining, usability in industry, and software engineering. By integrating these three aspects, we hope that AIX360 will attract researchers in AI explainability and help translate our collective research results for practicing data scientists and developers deploying solutions in a variety of industries. Regarding the first aspect of diversity, Table 1 in [1] compares AIX360 to existing toolkits in terms of the types of explainability methods offered. The table shows that AIX360 not only covers more types of methods but also has metrics which can act as proxies for judging the quality of explanations. Regarding the second aspect of industry usage, AIX360 illustrates how these explainability algorithms can be applied in specific contexts (please see Audience, goals, and outcomes below). In just a few months since its initial release, the AIX360 toolkit already has a vibrant slack community with over 120 members and has been forked almost 80 times accumulating over 400 stars. This response leads us to believe that there is significant interest in the community in learning more about the toolkit and explainability in general. Audience, goals, and outcomes. The presentations in the tutorial will be aimed at an audience with different backgrounds and computer science expertise levels. For all audience members and especially those unfamiliar with Python programming, AIX360 provides an interactive experience (http://aix360.mybluemix.net/data) centered around a credit approval scenario as a gentle and grounded introduction to the concepts and capabilities of the toolkit. We will also teach all participants which type of explainability algorithm is most appropriate for a given use case, not only for those in the toolkit but also from the broader explainability literature","PeriodicalId":377829,"journal":{"name":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125069309","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Critiques of 'algorithmic fairness' have counseled against a purely technical approach. Recent work from the FAT* conference has warned specifically about abstracting away the social context that these automated systems are operating within and has suggested that "[fairness work] require[s] technical researchers to learn new skills or partner with social scientists" [Fairness and abstraction in sociotechnical systems, Selbst et al. 2019, FAT* '19]. That "social context" includes groups outside the academy organizing for data and/or tech justice (e.g., Allied Media Projects, Stop LAPD Spying Coalition, data4blacklives, etc). These struggles have deep historical roots but have become prominent in the past several years alongside broader citizen-science efforts. In this CRAFT session we as STEM researchers hope to initiate conversation about methods used by community organizers to analyze power relations present in that social context. We will take this time to learn together and discuss if/how these and other methods, collaborations and efforts can be used to actualize oft-mentioned critiques of algorithmic fairness and move toward a data justice-oriented approach. Many scholars and activists have spoken on how to approach social context when discussing algorithmic fairness interventions. Community organizing and attendant methods for power analysis present one such approach: documenting all stakeholders and entities relevant to an issue and the nature of the power differentials between them. The facilitators for this session are not experts in community organizing theory or practice. Instead, we will share what we have learned from our readings of decades of rich work and writings from community organizers. This session is a collective, interdisciplinary learning experience, open to all who see their interests as relevant to the conversation. We will open with a discussion of community organizing practice: What is community organizing, what are its goals, methods, past and ongoing examples? What disciplines and intellectual lineages does it draw from? We will incorporate key sources we have found helpful for synthesizing this knowledge so that participants can continue exposing themselves to the field after the conference. We will also consider the concept of social power, including power that the algorithmic fairness community holds. Noting that there are many ways to theorize and understand power, we will share the framings that have been most useful to us. We plan to present different tools, models and procedures for doing power analysis in various organizing settings. We will propose to our group that we conduct a power analysis of our own. We have prepared a hypothetical but realistic scenario involving risk assessment in a hospital setting as an example. However, we encourage participants to bring their own experiences to the table, especially if they pertain in any way to data injustice. We also invite participants to bring examples of ongo
对“算法公平性”的批评反对采用纯粹的技术方法。FAT*会议最近的工作特别警告了抽象这些自动化系统运行的社会背景，并建议“[公平工作]需要技术研究人员学习新技能或与社会科学家合作”[社会技术系统中的公平和抽象，Selbst等人，2019,FAT* '19]。这种“社会背景”包括学院外组织数据和/或技术正义的团体(例如，Allied Media Projects, Stop LAPD间谍联盟，data4blacklives等)。这些斗争有着深刻的历史根源，但在过去几年里，随着更广泛的公民科学努力，这些斗争变得更加突出。在这次CRAFT会议上，我们作为STEM研究人员希望发起关于社区组织者用来分析社会背景下权力关系的方法的讨论。我们将利用这段时间一起学习和讨论是否/如何使用这些方法和其他方法、合作和努力来实现经常提到的对算法公平性的批评，并朝着以数据公正为导向的方法发展。许多学者和活动家在讨论算法公平干预时谈到了如何处理社会背景。社区组织和随之而来的权力分析方法提供了这样一种方法:记录与问题相关的所有利益相关者和实体，以及它们之间权力差异的本质。本次会议的主持人并非社区组织理论或实践方面的专家。相反，我们将分享我们从阅读社区组织者几十年来丰富的工作和著作中学到的东西。这个会议是一个集体的，跨学科的学习经验，开放给所有谁看到他们的兴趣相关的谈话。我们将以社区组织实践的讨论开始:什么是社区组织，它的目标、方法、过去和正在进行的例子是什么?它借鉴了哪些学科和知识谱系?我们将结合我们发现的有助于综合这些知识的关键来源，以便与会者在会议结束后继续接触该领域。我们还将考虑社会权力的概念，包括算法公平社区所拥有的权力。注意到有许多方法可以理论化和理解权力，我们将分享对我们最有用的框架。我们计划介绍不同的工具、模型和程序，在不同的组织环境中进行功率分析。我们将向我们的团队提议，我们自己也进行一次权力分析。我们准备了一个假设但现实的场景，以医院环境中的风险评估为例。然而，我们鼓励参与者将他们自己的经历带到桌面上，特别是如果他们以任何方式与数据不公正有关。我们还邀请参与者提供正在进行的组织努力的例子，让算法公平研究人员能够团结一致。参与者将带着以下内容离开本课程:1)了解进一步了解这些主题所需的关键术语和资源;2)在现实的、有基础的场景中分析电力的初步经验。
{"title":"Manifesting the sociotechnical: experimenting with methods for social context and social justice","authors":"E. Goss, Lily Hu, Manuel Sabin, Stephanie Teeple","doi":"10.1145/3351095.3375682","DOIUrl":"https://doi.org/10.1145/3351095.3375682","url":null,"abstract":"Critiques of 'algorithmic fairness' have counseled against a purely technical approach. Recent work from the FAT* conference has warned specifically about abstracting away the social context that these automated systems are operating within and has suggested that \"[fairness work] require[s] technical researchers to learn new skills or partner with social scientists\" [Fairness and abstraction in sociotechnical systems, Selbst et al. 2019, FAT* '19]. That \"social context\" includes groups outside the academy organizing for data and/or tech justice (e.g., Allied Media Projects, Stop LAPD Spying Coalition, data4blacklives, etc). These struggles have deep historical roots but have become prominent in the past several years alongside broader citizen-science efforts. In this CRAFT session we as STEM researchers hope to initiate conversation about methods used by community organizers to analyze power relations present in that social context. We will take this time to learn together and discuss if/how these and other methods, collaborations and efforts can be used to actualize oft-mentioned critiques of algorithmic fairness and move toward a data justice-oriented approach. Many scholars and activists have spoken on how to approach social context when discussing algorithmic fairness interventions. Community organizing and attendant methods for power analysis present one such approach: documenting all stakeholders and entities relevant to an issue and the nature of the power differentials between them. The facilitators for this session are not experts in community organizing theory or practice. Instead, we will share what we have learned from our readings of decades of rich work and writings from community organizers. This session is a collective, interdisciplinary learning experience, open to all who see their interests as relevant to the conversation. We will open with a discussion of community organizing practice: What is community organizing, what are its goals, methods, past and ongoing examples? What disciplines and intellectual lineages does it draw from? We will incorporate key sources we have found helpful for synthesizing this knowledge so that participants can continue exposing themselves to the field after the conference. We will also consider the concept of social power, including power that the algorithmic fairness community holds. Noting that there are many ways to theorize and understand power, we will share the framings that have been most useful to us. We plan to present different tools, models and procedures for doing power analysis in various organizing settings. We will propose to our group that we conduct a power analysis of our own. We have prepared a hypothetical but realistic scenario involving risk assessment in a hospital setting as an example. However, we encourage participants to bring their own experiences to the table, especially if they pertain in any way to data injustice. We also invite participants to bring examples of ongo","PeriodicalId":377829,"journal":{"name":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126491256","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
James Wexler, Mahima Pushkarna, Sara Robinson, Tolga Bolukbasi, Andrew Zaldivar
As more and more industries use machine learning, it's important to understand how these models make predictions, and where bias can be introduced in the process. In this tutorial we'll walk through two open source frameworks for analyzing your models from a fairness perspective. We'll start with the What-If Tool, a visualization tool that you can run inside a Python notebook to analyze an ML model. With the What-If Tool, you can identify dataset imbalances, see how individual features impact your model's prediction through partial dependence plots, and analyze human-centered ML models from a fairness perspective using various optimization strategies. Then we'll look at SHAP, a tool for interpreting the output of any machine learning model, and seeing how a model arrived at predictions for individual datapoints. We will then show how to use SHAP and the What-If Tool together. After the tutorial you'll have the skills to get started with both of these tools on your own datasets, and be better equipped to analyze your models from a fairness perspective.
{"title":"Probing ML models for fairness with the what-if tool and SHAP: hands-on tutorial","authors":"James Wexler, Mahima Pushkarna, Sara Robinson, Tolga Bolukbasi, Andrew Zaldivar","doi":"10.1145/3351095.3375662","DOIUrl":"https://doi.org/10.1145/3351095.3375662","url":null,"abstract":"As more and more industries use machine learning, it's important to understand how these models make predictions, and where bias can be introduced in the process. In this tutorial we'll walk through two open source frameworks for analyzing your models from a fairness perspective. We'll start with the What-If Tool, a visualization tool that you can run inside a Python notebook to analyze an ML model. With the What-If Tool, you can identify dataset imbalances, see how individual features impact your model's prediction through partial dependence plots, and analyze human-centered ML models from a fairness perspective using various optimization strategies. Then we'll look at SHAP, a tool for interpreting the output of any machine learning model, and seeing how a model arrived at predictions for individual datapoints. We will then show how to use SHAP and the What-If Tool together. After the tutorial you'll have the skills to get started with both of these tools on your own datasets, and be better equipped to analyze your models from a fairness perspective.","PeriodicalId":377829,"journal":{"name":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128869894","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Recent work has documented instances of unfairness in deployed machine learning models, and significant researcher effort has been dedicated to creating algorithms that intrinsically consider fairness. In this work, we highlight another source of unfairness: market forces that drive differential investment in the data pipeline for differing groups. We develop a high-level model to study this question. First, we show that our model predicts unfairness in a monopoly setting. Then, we show that under all but the most extreme models, competition does not eliminate this tendency, and may even exacerbate it. Finally, we consider two avenues for regulating a machine-learning driven monopolist - relative error inequality and absolute error-bounds - and quantify the price of fairness (and who pays it). These models imply that mitigating fairness concerns may require policy-driven solutions, not only technological ones.
{"title":"The effects of competition and regulation on error inequality in data-driven markets","authors":"Hadi Elzayn, Benjamin Fish","doi":"10.1145/3351095.3372842","DOIUrl":"https://doi.org/10.1145/3351095.3372842","url":null,"abstract":"Recent work has documented instances of unfairness in deployed machine learning models, and significant researcher effort has been dedicated to creating algorithms that intrinsically consider fairness. In this work, we highlight another source of unfairness: market forces that drive differential investment in the data pipeline for differing groups. We develop a high-level model to study this question. First, we show that our model predicts unfairness in a monopoly setting. Then, we show that under all but the most extreme models, competition does not eliminate this tendency, and may even exacerbate it. Finally, we consider two avenues for regulating a machine-learning driven monopolist - relative error inequality and absolute error-bounds - and quantify the price of fairness (and who pays it). These models imply that mitigating fairness concerns may require policy-driven solutions, not only technological ones.","PeriodicalId":377829,"journal":{"name":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134490810","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
K. Baxter, Yoav Schlesinger, Sarah E. Aerni, Lewis Baker, Julie Dawson, K. Kenthapadi, Isabel M. Kloumann, Hanna M. Wallach
The study of fairness in machine learning applications has seen significant academic inquiry, research and publication in recent years. Concurrently, technology companies have begun to instantiate nascent program in AI ethics and product ethics more broadly. As a result of these efforts, AI ethics practitioners have piloted new processes to evaluate and ensure fairness in their machine learning applications. In this session, six industry practitioners, hailing from LinkedIn, Yoti, Microsoft, Pymetrics, Facebook, and Salesforce share insights from the work they have undertaken in the area of fairness, what has worked and what has not, lessons learned and best practices instituted as a result. • Krishnaram Kenthapadi presents LinkedIn's fairness-aware reranking for talent search. • Julie Dawson shares how Yoti applies ML fairness research to age estimation in their digital identity platform. • Hanna Wallach contributes how Microsoft is applying fairness principles in practice. • Lewis Baker presents Pymetric's fairness mechanisms in their hiring algorithm. • Isabel Kloumann presents Facebook's fairness assessment framework through a case study of fairness in a content moderation system. • Sarah Aerni contributes how Salesforce is building fairness features into the Einstein AI platform. Building on those insights, we discuss insights and brainstorm modalities through which to build upon the practitioners' work. Opportunities for further research or collaboration are identified, with the goal of developing a shared understanding of experiences and needs of AI ethics practitioners. Ultimately, the aim is to develop a playbook for more ethical and fair AI product development and deployment.
{"title":"Bridging the gap from AI ethics research to practice","authors":"K. Baxter, Yoav Schlesinger, Sarah E. Aerni, Lewis Baker, Julie Dawson, K. Kenthapadi, Isabel M. Kloumann, Hanna M. Wallach","doi":"10.1145/3351095.3375680","DOIUrl":"https://doi.org/10.1145/3351095.3375680","url":null,"abstract":"The study of fairness in machine learning applications has seen significant academic inquiry, research and publication in recent years. Concurrently, technology companies have begun to instantiate nascent program in AI ethics and product ethics more broadly. As a result of these efforts, AI ethics practitioners have piloted new processes to evaluate and ensure fairness in their machine learning applications. In this session, six industry practitioners, hailing from LinkedIn, Yoti, Microsoft, Pymetrics, Facebook, and Salesforce share insights from the work they have undertaken in the area of fairness, what has worked and what has not, lessons learned and best practices instituted as a result. • Krishnaram Kenthapadi presents LinkedIn's fairness-aware reranking for talent search. • Julie Dawson shares how Yoti applies ML fairness research to age estimation in their digital identity platform. • Hanna Wallach contributes how Microsoft is applying fairness principles in practice. • Lewis Baker presents Pymetric's fairness mechanisms in their hiring algorithm. • Isabel Kloumann presents Facebook's fairness assessment framework through a case study of fairness in a content moderation system. • Sarah Aerni contributes how Salesforce is building fairness features into the Einstein AI platform. Building on those insights, we discuss insights and brainstorm modalities through which to build upon the practitioners' work. Opportunities for further research or collaboration are identified, with the goal of developing a shared understanding of experiences and needs of AI ethics practitioners. Ultimately, the aim is to develop a playbook for more ethical and fair AI product development and deployment.","PeriodicalId":377829,"journal":{"name":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129589611","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Krishna Gade, S. Geyik, K. Kenthapadi, Varun Mithal, Ankur Taly
Artificial Intelligence is increasingly playing an integral role in determining our day-to-day experiences. Moreover, with the proliferation of AI based solutions in areas such as hiring, lending, criminal justice, healthcare, and education, the resulting personal and professional implications of AI have become far-reaching. The dominant role played by AI models in these domains has led to a growing concern regarding potential bias in these models, and a demand for model transparency and interpretability [2, 4]. Model explainability is considered a prerequisite for building trust and adoption of AI systems in high stakes domains such as lending and healthcare [1] requiring reliability, safety, and fairness. It is also critical to automated transportation, and other industrial applications with significant socio-economic implications such as predictive maintenance, exploration of natural resources, and climate change modeling. As a consequence, AI researchers and practitioners have focused their attention on explainable AI to help them better trust and understand models at scale [5, 6, 8]. In fact, the field of explainability in AI/ML is at an inflexion point. There is a tremendous need from the societal, regulatory, commercial, end-user, and model developer perspectives. Consequently, practical and scalable explainability approaches are rapidly becoming available. The challenges for the research community include: (i) achieving consensus on the right notion of model explainability, (ii) identifying and formalizing explainability tasks from the perspectives of various stakeholders, and (iii) designing measures for evaluating explainability techniques. In this tutorial, we will first motivate the need for model interpretability and explainability in AI [3] from various perspectives. We will then provide a brief overview of several explainability techniques and tools. The rest of the tutorial will focus on the real-world application of explainability techniques in industry. We will present case studies spanning several domains such as: • Search and Recommendation systems: Understanding of search and recommendations systems, as well as how retrieval and ranking decisions happen in real-time [7]. Example applications include explanation of decisions made by an AI system towards job recommendations, ranking of potential candidates for job posters, and content recommendations. • Sales: Understanding of sales predictions in terms of customer up-sell/churn. • Fraud Detection: Examining and explaining AI systems that determine whether a content or event is fraudulent. • Lending: How to understand/interpret lending decisions made by an AI system. We will focus on the sociotechnical dimensions, practical challenges, and lessons learned during development and deployment of these systems, which would be beneficial for researchers and practitioners interested in explainable AI. Finally, we will discuss open challenges and research directions for the community.
{"title":"Explainable AI in industry: practical challenges and lessons learned: implications tutorial","authors":"Krishna Gade, S. Geyik, K. Kenthapadi, Varun Mithal, Ankur Taly","doi":"10.1145/3351095.3375664","DOIUrl":"https://doi.org/10.1145/3351095.3375664","url":null,"abstract":"Artificial Intelligence is increasingly playing an integral role in determining our day-to-day experiences. Moreover, with the proliferation of AI based solutions in areas such as hiring, lending, criminal justice, healthcare, and education, the resulting personal and professional implications of AI have become far-reaching. The dominant role played by AI models in these domains has led to a growing concern regarding potential bias in these models, and a demand for model transparency and interpretability [2, 4]. Model explainability is considered a prerequisite for building trust and adoption of AI systems in high stakes domains such as lending and healthcare [1] requiring reliability, safety, and fairness. It is also critical to automated transportation, and other industrial applications with significant socio-economic implications such as predictive maintenance, exploration of natural resources, and climate change modeling. As a consequence, AI researchers and practitioners have focused their attention on explainable AI to help them better trust and understand models at scale [5, 6, 8]. In fact, the field of explainability in AI/ML is at an inflexion point. There is a tremendous need from the societal, regulatory, commercial, end-user, and model developer perspectives. Consequently, practical and scalable explainability approaches are rapidly becoming available. The challenges for the research community include: (i) achieving consensus on the right notion of model explainability, (ii) identifying and formalizing explainability tasks from the perspectives of various stakeholders, and (iii) designing measures for evaluating explainability techniques. In this tutorial, we will first motivate the need for model interpretability and explainability in AI [3] from various perspectives. We will then provide a brief overview of several explainability techniques and tools. The rest of the tutorial will focus on the real-world application of explainability techniques in industry. We will present case studies spanning several domains such as: • Search and Recommendation systems: Understanding of search and recommendations systems, as well as how retrieval and ranking decisions happen in real-time [7]. Example applications include explanation of decisions made by an AI system towards job recommendations, ranking of potential candidates for job posters, and content recommendations. • Sales: Understanding of sales predictions in terms of customer up-sell/churn. • Fraud Detection: Examining and explaining AI systems that determine whether a content or event is fraudulent. • Lending: How to understand/interpret lending decisions made by an AI system. We will focus on the sociotechnical dimensions, practical challenges, and lessons learned during development and deployment of these systems, which would be beneficial for researchers and practitioners interested in explainable AI. Finally, we will discuss open challenges and research directions for the community.","PeriodicalId":377829,"journal":{"name":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129219869","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Corinne Cath, Mark Latonero, Vidushi Marda, Roya Pakzad
The premise of this translation tutorial is that human rights serves as a complementary framework - in addition to Fairness, Accountability, Transparency, and Ethics - for guiding and governing artificial intelligence (AI) and machine learning research and development. Attendees will participate in a case study, which will demonstrate show how a human rights framework, grounded in international law, fundamental values, and global systems of accountability, can offer the technical community a practical approach to addressing global AI risks and harms. This tutorial discusses how human rights frameworks can inform, guide and govern AI policy and practice in a manner that is complementary to Fairness, Accountability, Transparency, and Ethics (FATE) frameworks. Using the case study of researchers developing a facial recognition API at a tech company and its use by a law enforcement client, we will engage the audience to think through the benefits and challenges of applying human rights frameworks to AI system design and deployment. We will do so by providing a brief overview of the international human rights law, and various non-binding human rights frameworks in relation to our current discussions around FATE and then apply them to contemporary debates and case studies
{"title":"Leap of FATE: human rights as a complementary framework for AI policy and practice","authors":"Corinne Cath, Mark Latonero, Vidushi Marda, Roya Pakzad","doi":"10.1145/3351095.3375665","DOIUrl":"https://doi.org/10.1145/3351095.3375665","url":null,"abstract":"The premise of this translation tutorial is that human rights serves as a complementary framework - in addition to Fairness, Accountability, Transparency, and Ethics - for guiding and governing artificial intelligence (AI) and machine learning research and development. Attendees will participate in a case study, which will demonstrate show how a human rights framework, grounded in international law, fundamental values, and global systems of accountability, can offer the technical community a practical approach to addressing global AI risks and harms. This tutorial discusses how human rights frameworks can inform, guide and govern AI policy and practice in a manner that is complementary to Fairness, Accountability, Transparency, and Ethics (FATE) frameworks. Using the case study of researchers developing a facial recognition API at a tech company and its use by a law enforcement client, we will engage the audience to think through the benefits and challenges of applying human rights frameworks to AI system design and deployment. We will do so by providing a brief overview of the international human rights law, and various non-binding human rights frameworks in relation to our current discussions around FATE and then apply them to contemporary debates and case studies","PeriodicalId":377829,"journal":{"name":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128549710","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The work within the Fairness, Accountability, and Transparency of ML (fair-ML) community will positively benefit from appreciating the role of organizational culture and structure in the effective practice of fair-ML efforts of individuals, teams, and initiatives within industry. In this tutorial session we will explore various organizational structures and possible leverage points to effectively intervene in the process of design, development, and deployment of AI systems, towards contributing to positive fair-ML outcomes. We will begin by presenting the results of interviews conducted during an ethnographic study among practitioners working in industry, including themes related to: origination and evolution, common challenges, ethical tensions, and effective enablers. The study was designed through the lens of Industrial Organizational Psychology and aims to create a mapping of the current state of the fair-ML organizational structures inside major AI companies. We also look at the most-desired future state to enable effective work to increase algorithmic accountability, as well as the key elements in the transition from the current to that future state. We investigate drivers for change as well as the tensions between creating an 'ethical' system vs one that is 'ethical' enough. After presenting our preliminary findings, the rest of the tutorial will be highly interactive. Starting with a facilitated activity in break out groups, we will discuss the already identified challenges, best practices, and mitigation strategies. Finally, we hope to create space for productive discussion among AI practitioners in industry, academic researchers within various fields working directly on algorithmic accountability and transparency, advocates for various communities most impacted by technology, and others. Based on the interactive component of the tutorial, facilitators and interested participants will collaborate on further developing the discussed challenges into scenarios and guidelines that will be published as a follow up report.
{"title":"Assessing the intersection of organizational structure and FAT* efforts within industry: implications tutorial","authors":"B. Rakova, Rumman Chowdhury, Jingying Yang","doi":"10.1145/3351095.3375672","DOIUrl":"https://doi.org/10.1145/3351095.3375672","url":null,"abstract":"The work within the Fairness, Accountability, and Transparency of ML (fair-ML) community will positively benefit from appreciating the role of organizational culture and structure in the effective practice of fair-ML efforts of individuals, teams, and initiatives within industry. In this tutorial session we will explore various organizational structures and possible leverage points to effectively intervene in the process of design, development, and deployment of AI systems, towards contributing to positive fair-ML outcomes. We will begin by presenting the results of interviews conducted during an ethnographic study among practitioners working in industry, including themes related to: origination and evolution, common challenges, ethical tensions, and effective enablers. The study was designed through the lens of Industrial Organizational Psychology and aims to create a mapping of the current state of the fair-ML organizational structures inside major AI companies. We also look at the most-desired future state to enable effective work to increase algorithmic accountability, as well as the key elements in the transition from the current to that future state. We investigate drivers for change as well as the tensions between creating an 'ethical' system vs one that is 'ethical' enough. After presenting our preliminary findings, the rest of the tutorial will be highly interactive. Starting with a facilitated activity in break out groups, we will discuss the already identified challenges, best practices, and mitigation strategies. Finally, we hope to create space for productive discussion among AI practitioners in industry, academic researchers within various fields working directly on algorithmic accountability and transparency, advocates for various communities most impacted by technology, and others. Based on the interactive component of the tutorial, facilitators and interested participants will collaborate on further developing the discussed challenges into scenarios and guidelines that will be published as a follow up report.","PeriodicalId":377829,"journal":{"name":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency","volume":"53 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127440724","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evelyn Wan, A. D. Groot, Shazade Jameson, M. Paun, Phillip Lücking, Goda Klumbytė, Danny Lämmerhirt
There are gaps in understanding in and between those who design systems of AI/ ML, those who critique them, and those positioned between these discourses. This gap can be defined in multiple ways - e.g. methodological, epistemological, linguistic, or cultural. To bridge this gap requires a set of translations: the generation of a collaborative space and a new set of shared sensibilities that traverse disciplinary boundaries. This workshop aims to explore translations across multiple fields, and translations between theory and practice, as well as how interdisciplinary work could generate new operationalizable approaches. We define 'knowledge' as a social product (L. Code) which requires fair and broad epistemic cooperation in its generation, development, and dissemination. As a "marker for truth" (B. Williams) and therefore a basis for action, knowledge circulation sustains the systems of power which produce it in the first place (M. Foucault). Enabled by epistemic credence, authority or knowledge, epistemic power can be an important driver of, but also result from, other (e.g. economic, political) powers. To produce reliable output, our standards and methods should serve us all and exclude no-one. Critical theorists have long revealed failings of epistemic practices, resulting in the marginalization and exclusion of some types of knowledge. How can we cultivate more reflexive epistemic practices in the interdisciplinary research setting of FAT*? We frame this ideal as 'epistemic justice' (M. Geuskens), the positive of 'epistemic injustice', defined by M. Fricker as injustice that exists when people are wronged as a knower or as an epistemic subject. Epistemic justice is the proper use and allocation of epistemic power; the inclusion and balancing of all epistemic sources. As S. Jasanoff reminds us, any authoritative way of seeing must be legitimized in discourse and practice, showing that practices can be developed to value and engage with other viewpoints and possibly reshape our ways of knowing. Our workshop aims to address the following questions: how could critical theory or higher level critiques be translated into and anchored in ML/AI design practices - and vice versa? What kind of cartographies and methodologies are needed in order to identify issues that can act as the basis of collaborative research and design? How can we (un)learn our established ways of thinking for such collaborative work to take place? During the workshop, participants will create, share and explode prototypical workflows of designing, researching and critiquing algorithmic systems. We will identify moments in which translations and interdisciplinary interventions could or should happen in order to build actionable steps and methodological frameworks that advance epistemic justice and are conducive to future interdisciplinary collaboration.
{"title":"Lost in translation: an interactive workshop mapping interdisciplinary translations for epistemic justice","authors":"Evelyn Wan, A. D. Groot, Shazade Jameson, M. Paun, Phillip Lücking, Goda Klumbytė, Danny Lämmerhirt","doi":"10.1145/3351095.3375685","DOIUrl":"https://doi.org/10.1145/3351095.3375685","url":null,"abstract":"There are gaps in understanding in and between those who design systems of AI/ ML, those who critique them, and those positioned between these discourses. This gap can be defined in multiple ways - e.g. methodological, epistemological, linguistic, or cultural. To bridge this gap requires a set of translations: the generation of a collaborative space and a new set of shared sensibilities that traverse disciplinary boundaries. This workshop aims to explore translations across multiple fields, and translations between theory and practice, as well as how interdisciplinary work could generate new operationalizable approaches. We define 'knowledge' as a social product (L. Code) which requires fair and broad epistemic cooperation in its generation, development, and dissemination. As a \"marker for truth\" (B. Williams) and therefore a basis for action, knowledge circulation sustains the systems of power which produce it in the first place (M. Foucault). Enabled by epistemic credence, authority or knowledge, epistemic power can be an important driver of, but also result from, other (e.g. economic, political) powers. To produce reliable output, our standards and methods should serve us all and exclude no-one. Critical theorists have long revealed failings of epistemic practices, resulting in the marginalization and exclusion of some types of knowledge. How can we cultivate more reflexive epistemic practices in the interdisciplinary research setting of FAT*? We frame this ideal as 'epistemic justice' (M. Geuskens), the positive of 'epistemic injustice', defined by M. Fricker as injustice that exists when people are wronged as a knower or as an epistemic subject. Epistemic justice is the proper use and allocation of epistemic power; the inclusion and balancing of all epistemic sources. As S. Jasanoff reminds us, any authoritative way of seeing must be legitimized in discourse and practice, showing that practices can be developed to value and engage with other viewpoints and possibly reshape our ways of knowing. Our workshop aims to address the following questions: how could critical theory or higher level critiques be translated into and anchored in ML/AI design practices - and vice versa? What kind of cartographies and methodologies are needed in order to identify issues that can act as the basis of collaborative research and design? How can we (un)learn our established ways of thinking for such collaborative work to take place? During the workshop, participants will create, share and explode prototypical workflows of designing, researching and critiquing algorithmic systems. We will identify moments in which translations and interdisciplinary interventions could or should happen in order to build actionable steps and methodological frameworks that advance epistemic justice and are conducive to future interdisciplinary collaboration.","PeriodicalId":377829,"journal":{"name":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128457816","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
B. Wagner, Krisztina Rozgonyi, Marie-Theres Sekwenz, Jennifer Cobbe, Jatinder Singh
Regulatory regimes designed to ensure transparency often struggle to ensure that transparency is meaningful in practice. This challenge is particularly great when coupled with the widespread usage of dark patterns --- design techniques used to manipulate users. The following article analyses the implementation of the transparency provisions of the German Network Enforcement Act (NetzDG) by Facebook and Twitter, as well as the consequences of these implementations for the effective regulation of online platforms. This question of effective regulation is particularly salient, due to an enforcement action in 2019 by Germany's Federal Office of Justice (BfJ) against Facebook for what the BfJ claim were insufficient compliance with transparency requirements, under NetzDG. This article provides an overview of the transparency requirements of NetzDG and contrasts these with the transparency requirements of other relevant regulations. It will then discuss how transparency concerns not only providing data, but also how the visibility of the data that is made transparent is managed, by deciding how the data is provided and is framed. We will then provide an empirical analysis of the design choices made by Facebook and Twitter, to assess the ways in which their implementations differ. The consequences of these two divergent implementations on interface design and user behaviour are then discussed, through a comparison of the transparency reports and reporting mechanisms used by Facebook and Twitter. As a next step, we will discuss the BfJ's consideration of the design of Facebook's content reporting mechanisms, and what this reveals about their respective interpretations of NetzDG's scope. Finally, in recognising that this situation is one in which a regulator is considering design as part of their action - we develop a wider argument on the potential for regulatory enforcement around dark patterns, and design practices more generally, for which this case is an early, indicative example.
{"title":"Regulating transparency?: Facebook, Twitter and the German Network Enforcement Act","authors":"B. Wagner, Krisztina Rozgonyi, Marie-Theres Sekwenz, Jennifer Cobbe, Jatinder Singh","doi":"10.1145/3351095.3372856","DOIUrl":"https://doi.org/10.1145/3351095.3372856","url":null,"abstract":"Regulatory regimes designed to ensure transparency often struggle to ensure that transparency is meaningful in practice. This challenge is particularly great when coupled with the widespread usage of dark patterns --- design techniques used to manipulate users. The following article analyses the implementation of the transparency provisions of the German Network Enforcement Act (NetzDG) by Facebook and Twitter, as well as the consequences of these implementations for the effective regulation of online platforms. This question of effective regulation is particularly salient, due to an enforcement action in 2019 by Germany's Federal Office of Justice (BfJ) against Facebook for what the BfJ claim were insufficient compliance with transparency requirements, under NetzDG. This article provides an overview of the transparency requirements of NetzDG and contrasts these with the transparency requirements of other relevant regulations. It will then discuss how transparency concerns not only providing data, but also how the visibility of the data that is made transparent is managed, by deciding how the data is provided and is framed. We will then provide an empirical analysis of the design choices made by Facebook and Twitter, to assess the ways in which their implementations differ. The consequences of these two divergent implementations on interface design and user behaviour are then discussed, through a comparison of the transparency reports and reporting mechanisms used by Facebook and Twitter. As a next step, we will discuss the BfJ's consideration of the design of Facebook's content reporting mechanisms, and what this reveals about their respective interpretations of NetzDG's scope. Finally, in recognising that this situation is one in which a regulator is considering design as part of their action - we develop a wider argument on the potential for regulatory enforcement around dark patterns, and design practices more generally, for which this case is an early, indicative example.","PeriodicalId":377829,"journal":{"name":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116681927","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：