Zhonghao He, Jascha Achterberg, Katie Collins, Kevin Nejad, Danyal Akarca, Yinzhu Yang, Wes Gurnee, Ilia Sucholutsky, Yuhan Tang, Rebeca Ianov, George Ogden, Chole Li, Kai Sandbrink, Stephen Casper, Anna Ivanova, Grace W. Lindsay
{"title":"Multilevel Interpretability Of Artificial Neural Networks: Leveraging Framework And Methods From Neuroscience","authors":"Zhonghao He, Jascha Achterberg, Katie Collins, Kevin Nejad, Danyal Akarca, Yinzhu Yang, Wes Gurnee, Ilia Sucholutsky, Yuhan Tang, Rebeca Ianov, George Ogden, Chole Li, Kai Sandbrink, Stephen Casper, Anna Ivanova, Grace W. Lindsay","doi":"arxiv-2408.12664","DOIUrl":null,"url":null,"abstract":"As deep learning systems are scaled up to many billions of parameters,\nrelating their internal structure to external behaviors becomes very\nchallenging. Although daunting, this problem is not new: Neuroscientists and\ncognitive scientists have accumulated decades of experience analyzing a\nparticularly complex system - the brain. In this work, we argue that\ninterpreting both biological and artificial neural systems requires analyzing\nthose systems at multiple levels of analysis, with different analytic tools for\neach level. We first lay out a joint grand challenge among scientists who study\nthe brain and who study artificial neural networks: understanding how\ndistributed neural mechanisms give rise to complex cognition and behavior. We\nthen present a series of analytical tools that can be used to analyze\nbiological and artificial neural systems, organizing those tools according to\nMarr's three levels of analysis: computation/behavior,\nalgorithm/representation, and implementation. Overall, the multilevel\ninterpretability framework provides a principled way to tackle neural system\ncomplexity; links structure, computation, and behavior; clarifies assumptions\nand research priorities at each level; and paves the way toward a unified\neffort for understanding intelligent systems, may they be biological or\nartificial.","PeriodicalId":501517,"journal":{"name":"arXiv - QuanBio - Neurons and Cognition","volume":"89 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - QuanBio - Neurons and Cognition","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2408.12664","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
Abstract
As deep learning systems are scaled up to many billions of parameters,
relating their internal structure to external behaviors becomes very
challenging. Although daunting, this problem is not new: Neuroscientists and
cognitive scientists have accumulated decades of experience analyzing a
particularly complex system - the brain. In this work, we argue that
interpreting both biological and artificial neural systems requires analyzing
those systems at multiple levels of analysis, with different analytic tools for
each level. We first lay out a joint grand challenge among scientists who study
the brain and who study artificial neural networks: understanding how
distributed neural mechanisms give rise to complex cognition and behavior. We
then present a series of analytical tools that can be used to analyze
biological and artificial neural systems, organizing those tools according to
Marr's three levels of analysis: computation/behavior,
algorithm/representation, and implementation. Overall, the multilevel
interpretability framework provides a principled way to tackle neural system
complexity; links structure, computation, and behavior; clarifies assumptions
and research priorities at each level; and paves the way toward a unified
effort for understanding intelligent systems, may they be biological or
artificial.