Nethmi Jayasinghe;Maeesha Binte Hashem;Dinithi Jayasuriya;Leila Rahimifard;Min-A Kang;Vinod K. Sangwan;Mark C. Hersam;Amit Ranjan Trivedi
{"title":"Single-Step Extraction of Transformer Attention With Dual-Gated Memtransistor Crossbars","authors":"Nethmi Jayasinghe;Maeesha Binte Hashem;Dinithi Jayasuriya;Leila Rahimifard;Min-A Kang;Vinod K. Sangwan;Mark C. Hersam;Amit Ranjan Trivedi","doi":"10.1109/LED.2024.3435540","DOIUrl":null,"url":null,"abstract":"We discuss how a dual-gated \n<italic>memtransistor</i>\n crossbar can accelerate the extraction of the Transformer’s attention scores. A memtransistor is a novel two-dimensional material-based device that offers non-volatile programmability and gate tunability. Leveraging these attributes, we demonstrate the extraction of quadratic-order products on a single memtransistor and the single-step extraction of attention scores without inferring intermediate query/key vectors. The query/key-free processing of memtransistor-based attention scoring results in \n<inline-formula> <tex-math>$2.37\\times $ </tex-math></inline-formula>\n lower energy with less than half crossbar cells.","PeriodicalId":13198,"journal":{"name":"IEEE Electron Device Letters","volume":null,"pages":null},"PeriodicalIF":4.1000,"publicationDate":"2024-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Electron Device Letters","FirstCategoryId":"5","ListUrlMain":"https://ieeexplore.ieee.org/document/10614197/","RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0
Abstract
We discuss how a dual-gated
memtransistor
crossbar can accelerate the extraction of the Transformer’s attention scores. A memtransistor is a novel two-dimensional material-based device that offers non-volatile programmability and gate tunability. Leveraging these attributes, we demonstrate the extraction of quadratic-order products on a single memtransistor and the single-step extraction of attention scores without inferring intermediate query/key vectors. The query/key-free processing of memtransistor-based attention scoring results in
$2.37\times $
lower energy with less than half crossbar cells.
期刊介绍：
IEEE Electron Device Letters publishes original and significant contributions relating to the theory, modeling, design, performance and reliability of electron and ion integrated circuit devices and interconnects, involving insulators, metals, organic materials, micro-plasmas, semiconductors, quantum-effect structures, vacuum devices, and emerging materials with applications in bioelectronics, biomedical electronics, computation, communications, displays, microelectromechanics, imaging, micro-actuators, nanoelectronics, optoelectronics, photovoltaics, power ICs and micro-sensors.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
