Multilevel Interpretability Of Artificial Neural Networks: Leveraging Framework And Methods From Neuroscience

Zhonghao He, Jascha Achterberg, Katie Collins, Kevin Nejad, Danyal Akarca, Yinzhu Yang, Wes Gurnee, Ilia Sucholutsky, Yuhan Tang, Rebeca Ianov, George Ogden, Chole Li, Kai Sandbrink, Stephen Casper, Anna Ivanova, Grace W. Lindsay
{"title":"Multilevel Interpretability Of Artificial Neural Networks: Leveraging Framework And Methods From Neuroscience","authors":"Zhonghao He, Jascha Achterberg, Katie Collins, Kevin Nejad, Danyal Akarca, Yinzhu Yang, Wes Gurnee, Ilia Sucholutsky, Yuhan Tang, Rebeca Ianov, George Ogden, Chole Li, Kai Sandbrink, Stephen Casper, Anna Ivanova, Grace W. Lindsay","doi":"arxiv-2408.12664","DOIUrl":null,"url":null,"abstract":"As deep learning systems are scaled up to many billions of parameters,\nrelating their internal structure to external behaviors becomes very\nchallenging. Although daunting, this problem is not new: Neuroscientists and\ncognitive scientists have accumulated decades of experience analyzing a\nparticularly complex system - the brain. In this work, we argue that\ninterpreting both biological and artificial neural systems requires analyzing\nthose systems at multiple levels of analysis, with different analytic tools for\neach level. We first lay out a joint grand challenge among scientists who study\nthe brain and who study artificial neural networks: understanding how\ndistributed neural mechanisms give rise to complex cognition and behavior. We\nthen present a series of analytical tools that can be used to analyze\nbiological and artificial neural systems, organizing those tools according to\nMarr's three levels of analysis: computation/behavior,\nalgorithm/representation, and implementation. Overall, the multilevel\ninterpretability framework provides a principled way to tackle neural system\ncomplexity; links structure, computation, and behavior; clarifies assumptions\nand research priorities at each level; and paves the way toward a unified\neffort for understanding intelligent systems, may they be biological or\nartificial.","PeriodicalId":501517,"journal":{"name":"arXiv - QuanBio - Neurons and Cognition","volume":"89 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - QuanBio - Neurons and Cognition","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2408.12664","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

Abstract

As deep learning systems are scaled up to many billions of parameters, relating their internal structure to external behaviors becomes very challenging. Although daunting, this problem is not new: Neuroscientists and cognitive scientists have accumulated decades of experience analyzing a particularly complex system - the brain. In this work, we argue that interpreting both biological and artificial neural systems requires analyzing those systems at multiple levels of analysis, with different analytic tools for each level. We first lay out a joint grand challenge among scientists who study the brain and who study artificial neural networks: understanding how distributed neural mechanisms give rise to complex cognition and behavior. We then present a series of analytical tools that can be used to analyze biological and artificial neural systems, organizing those tools according to Marr's three levels of analysis: computation/behavior, algorithm/representation, and implementation. Overall, the multilevel interpretability framework provides a principled way to tackle neural system complexity; links structure, computation, and behavior; clarifies assumptions and research priorities at each level; and paves the way toward a unified effort for understanding intelligent systems, may they be biological or artificial.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
人工神经网络的多层次可解释性:利用神经科学的框架和方法
随着深度学习系统扩展到数十亿个参数,将其内部结构与外部行为联系起来变得非常具有挑战性。这个问题虽然令人生畏,但并不新鲜:神经科学家和认知科学家已经积累了数十年分析大脑这一特别复杂系统的经验。在这项工作中,我们认为,要解释生物和人工神经系统,就必须在多个分析层次上对这些系统进行分析,并在每个层次上使用不同的分析工具。我们首先提出了研究大脑和人工神经网络的科学家共同面临的巨大挑战:理解分布式神经机制如何产生复杂的认知和行为。然后,我们介绍了一系列可用于分析生物和人工神经系统的分析工具,并根据马尔的三个分析层次对这些工具进行了组织:计算/行为、算法/表示和实现。总之,多层次可解释性框架为解决神经系统的复杂性提供了一种原则性方法;将结构、计算和行为联系起来;明确了每个层次的假设和研究重点;并为统一理解智能系统(无论是生物还是人工智能系统)的努力铺平了道路。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
自引率
0.00%
发文量
0
期刊最新文献
Early reduced dopaminergic tone mediated by D3 receptor and dopamine transporter in absence epileptogenesis Contrasformer: A Brain Network Contrastive Transformer for Neurodegenerative Condition Identification Identifying Influential nodes in Brain Networks via Self-Supervised Graph-Transformer Contrastive Learning in Memristor-based Neuromorphic Systems Self-Attention Limits Working Memory Capacity of Transformer-Based Models
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1