Lisa Anne Hendricks, Anna Rohrbach, Bernt Schiele, Trevor Darrell, Zeynep Akata
{"title":"Generating visual explanations with natural language","authors":"Lisa Anne Hendricks, Anna Rohrbach, Bernt Schiele, Trevor Darrell, Zeynep Akata","doi":"10.1002/ail2.55","DOIUrl":null,"url":null,"abstract":"<p>We generate natural language explanations for a fine-grained visual recognition task. Our explanations fulfill two criteria. First, explanations are <i>class discriminative</i>, meaning they mention attributes in an image which are important to identify a class. Second, explanations are <i>image relevant</i>, meaning they reflect the actual content of an image. Our system, composed of an explanation sampler and phrase-critic model, generates class discriminative and image relevant explanations. In addition, we demonstrate that our explanations can help humans decide whether to accept or reject an AI decision.</p>","PeriodicalId":72253,"journal":{"name":"Applied AI letters","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2021-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/ail2.55","citationCount":"5","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Applied AI letters","FirstCategoryId":"1085","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/ail2.55","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 5
Abstract
We generate natural language explanations for a fine-grained visual recognition task. Our explanations fulfill two criteria. First, explanations are class discriminative, meaning they mention attributes in an image which are important to identify a class. Second, explanations are image relevant, meaning they reflect the actual content of an image. Our system, composed of an explanation sampler and phrase-critic model, generates class discriminative and image relevant explanations. In addition, we demonstrate that our explanations can help humans decide whether to accept or reject an AI decision.