Jack White, Jaime Ruiz-Serra, Stephen Petrie, Tatiana Kameneva, Chris McCarthy
{"title":"基于自我注意力的义眼视觉处理技术","authors":"Jack White, Jaime Ruiz-Serra, Stephen Petrie, Tatiana Kameneva, Chris McCarthy","doi":"10.1109/EMBC40787.2023.10341053","DOIUrl":null,"url":null,"abstract":"<p><p>We investigate Self-Attention (SA) networks for directly learning visual representations for prosthetic vision. Specifically, we explore how the SA mechanism can be leveraged to produce task-specific scene representations for prosthetic vision, overcoming the need for explicit hand-selection of learnt features and post-processing. Further, we demonstrate how the mapping of importance to image regions can serve as an explainability tool to analyse the learnt vision processing behaviour, providing enhanced validation and interpretation capability than current learning-based methods for prosthetic vision. We investigate our approach in the context of an orientation and mobility (OM) task, and demonstrate its feasibility for learning vision processing pipelines for prosthetic vision.</p>","PeriodicalId":72237,"journal":{"name":"Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2023-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Self-Attention Based Vision Processing for Prosthetic Vision.\",\"authors\":\"Jack White, Jaime Ruiz-Serra, Stephen Petrie, Tatiana Kameneva, Chris McCarthy\",\"doi\":\"10.1109/EMBC40787.2023.10341053\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>We investigate Self-Attention (SA) networks for directly learning visual representations for prosthetic vision. Specifically, we explore how the SA mechanism can be leveraged to produce task-specific scene representations for prosthetic vision, overcoming the need for explicit hand-selection of learnt features and post-processing. Further, we demonstrate how the mapping of importance to image regions can serve as an explainability tool to analyse the learnt vision processing behaviour, providing enhanced validation and interpretation capability than current learning-based methods for prosthetic vision. We investigate our approach in the context of an orientation and mobility (OM) task, and demonstrate its feasibility for learning vision processing pipelines for prosthetic vision.</p>\",\"PeriodicalId\":72237,\"journal\":{\"name\":\"Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2023-07-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/EMBC40787.2023.10341053\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/EMBC40787.2023.10341053","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
Self-Attention Based Vision Processing for Prosthetic Vision.
We investigate Self-Attention (SA) networks for directly learning visual representations for prosthetic vision. Specifically, we explore how the SA mechanism can be leveraged to produce task-specific scene representations for prosthetic vision, overcoming the need for explicit hand-selection of learnt features and post-processing. Further, we demonstrate how the mapping of importance to image regions can serve as an explainability tool to analyse the learnt vision processing behaviour, providing enhanced validation and interpretation capability than current learning-based methods for prosthetic vision. We investigate our approach in the context of an orientation and mobility (OM) task, and demonstrate its feasibility for learning vision processing pipelines for prosthetic vision.