{"title":"基于虚拟世界探索和视听感官替代的阅读障碍读者数学内容浏览","authors":"Rynhardt Kruger, Febe de Wet, T. Niesler","doi":"10.1145/3584365","DOIUrl":null,"url":null,"abstract":"Documents containing mathematical content remain largely inaccessible to blind and visually impaired readers because they are predominantly published as untagged PDFs, which do not include the semantic data necessary for effective accessibility. Equations in such documents consist of text interlaced with lines and other graphical elements and cannot be interpreted using a screen reader. We present a browsing approach for print-disabled readers specifically aimed at such mathematical content. This approach draws on the navigational mechanisms often used to explore the virtual worlds of text adventure games with audio-visual sensory substitution for graphical content. The relative spatial placement of the elements of an equation are represented as a virtual world so the reader can navigate between elements. Text elements are announced conventionally using synthesised speech, while graphical elements, such as roots and fraction lines, are rendered using a modification of the vOICe algorithm. The virtual world allows the reader to interactively discover the spatial structure of the equation, while the rendition of graphical elements as sound allows the shape and identity of elements that cannot be synthesised as speech to be discovered and recognised. The browsing approach was evaluated by 11 blind and 14 sighted participants in a user trial that included identifying twelve equations extracted from PDF documents. Overall, equations were identified completely correctly in 78% of cases (74% and 83%, respectively, for blind and sighted subjects). If partial correctness is considered, then the performance is substantially higher. Feedback from the blind subjects indicated that the technique allows spatial information and graphical detail to be discovered. We conclude that the integration of a spatial model represented as a virtual world in conjunction with audio-visual sensory substitution for non-textual elements can be an effective way for blind and visually impaired readers to read currently inaccessible mathematical content in PDF documents.","PeriodicalId":54128,"journal":{"name":"ACM Transactions on Accessible Computing","volume":null,"pages":null},"PeriodicalIF":2.5000,"publicationDate":"2022-02-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":"{\"title\":\"Mathematical Content Browsing for Print-disabled Readers Based on Virtual-world Exploration and Audio-visual Sensory Substitution\",\"authors\":\"Rynhardt Kruger, Febe de Wet, T. Niesler\",\"doi\":\"10.1145/3584365\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Documents containing mathematical content remain largely inaccessible to blind and visually impaired readers because they are predominantly published as untagged PDFs, which do not include the semantic data necessary for effective accessibility. Equations in such documents consist of text interlaced with lines and other graphical elements and cannot be interpreted using a screen reader. We present a browsing approach for print-disabled readers specifically aimed at such mathematical content. This approach draws on the navigational mechanisms often used to explore the virtual worlds of text adventure games with audio-visual sensory substitution for graphical content. The relative spatial placement of the elements of an equation are represented as a virtual world so the reader can navigate between elements. Text elements are announced conventionally using synthesised speech, while graphical elements, such as roots and fraction lines, are rendered using a modification of the vOICe algorithm. The virtual world allows the reader to interactively discover the spatial structure of the equation, while the rendition of graphical elements as sound allows the shape and identity of elements that cannot be synthesised as speech to be discovered and recognised. The browsing approach was evaluated by 11 blind and 14 sighted participants in a user trial that included identifying twelve equations extracted from PDF documents. Overall, equations were identified completely correctly in 78% of cases (74% and 83%, respectively, for blind and sighted subjects). If partial correctness is considered, then the performance is substantially higher. Feedback from the blind subjects indicated that the technique allows spatial information and graphical detail to be discovered. We conclude that the integration of a spatial model represented as a virtual world in conjunction with audio-visual sensory substitution for non-textual elements can be an effective way for blind and visually impaired readers to read currently inaccessible mathematical content in PDF documents.\",\"PeriodicalId\":54128,\"journal\":{\"name\":\"ACM Transactions on Accessible Computing\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":2.5000,\"publicationDate\":\"2022-02-03\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"1\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"ACM Transactions on Accessible Computing\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1145/3584365\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"ACM Transactions on Accessible Computing","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3584365","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}
Mathematical Content Browsing for Print-disabled Readers Based on Virtual-world Exploration and Audio-visual Sensory Substitution
Documents containing mathematical content remain largely inaccessible to blind and visually impaired readers because they are predominantly published as untagged PDFs, which do not include the semantic data necessary for effective accessibility. Equations in such documents consist of text interlaced with lines and other graphical elements and cannot be interpreted using a screen reader. We present a browsing approach for print-disabled readers specifically aimed at such mathematical content. This approach draws on the navigational mechanisms often used to explore the virtual worlds of text adventure games with audio-visual sensory substitution for graphical content. The relative spatial placement of the elements of an equation are represented as a virtual world so the reader can navigate between elements. Text elements are announced conventionally using synthesised speech, while graphical elements, such as roots and fraction lines, are rendered using a modification of the vOICe algorithm. The virtual world allows the reader to interactively discover the spatial structure of the equation, while the rendition of graphical elements as sound allows the shape and identity of elements that cannot be synthesised as speech to be discovered and recognised. The browsing approach was evaluated by 11 blind and 14 sighted participants in a user trial that included identifying twelve equations extracted from PDF documents. Overall, equations were identified completely correctly in 78% of cases (74% and 83%, respectively, for blind and sighted subjects). If partial correctness is considered, then the performance is substantially higher. Feedback from the blind subjects indicated that the technique allows spatial information and graphical detail to be discovered. We conclude that the integration of a spatial model represented as a virtual world in conjunction with audio-visual sensory substitution for non-textual elements can be an effective way for blind and visually impaired readers to read currently inaccessible mathematical content in PDF documents.
期刊介绍:
Computer and information technologies have re-designed the way modern society operates. Their widespread use poses both opportunities and challenges for people who experience various disabilities including age-related disabilities. That is, while there are new avenues to assist individuals with disabilities and provide tools and resources to alleviate the traditional barriers encountered by these individuals, in many cases the technology itself presents barriers to use. ACM Transactions on Accessible Computing (TACCESS) is a quarterly peer-reviewed journal that publishes refereed articles addressing issues of computing that seek to address barriers to access, either creating new solutions or providing for the more inclusive design of technology to provide access for individuals with diverse abilities. The journal provides a technical forum for disseminating innovative research that covers either applications of computing and information technologies to provide assistive systems or inclusive technologies for individuals with disabilities. Some examples are web accessibility for those with visual impairments and blindness as well as web search explorations for those with limited cognitive abilities, technologies to address stroke rehabilitation or dementia care, language support systems deaf signers or those with limited language abilities, and input systems for individuals with limited ability to control traditional mouse and keyboard systems. The journal is of particular interest to SIGACCESS members and delegates to its affiliated conference (i.e., ASSETS) as well as other international accessibility conferences. It serves as a forum for discussions and information exchange between researchers, clinicians, and educators; including rehabilitation personnel who administer assistive technologies; and policy makers concerned with equitable access to information technologies.