Music and movies are powerful art forms that elicit deep feelings and emotions and help us reflect on our and other people's lives on subjects such as: dreams, mental states, routines, society and culture. The evolution of technology has been easing access to these forms of entertainment and education for everyone, everywhere in the world. Given the easy and frequent interaction with a huge amount of movies and music daily, and the impact of these in our emotions, it becomes more and more relevant to address and think of ways to augment people's emotional perception and awareness of multimedia content in and through movies and music. In this paper, we present the motivation and describe the background for these challenges, and propose an approach for the design, development and expansion of interactive features that allow users to visualize and access emotions felt while engaging with movies and music. A special focus is put on the content in these forms of entertainment that have in some way meant something or can be associated with a significant memory, providing insights and helping to manage and regulate emotions, allowing to revisit content with an increased awareness or even recommend new content, taking into account users’ personality.
{"title":"Enhancing Emotional Awareness and Regulation in Movies and Music Based on Personality","authors":"D. Pereira, T. Chambel","doi":"10.1145/3573381.3596462","DOIUrl":"https://doi.org/10.1145/3573381.3596462","url":null,"abstract":"Music and movies are powerful art forms that elicit deep feelings and emotions and help us reflect on our and other people's lives on subjects such as: dreams, mental states, routines, society and culture. The evolution of technology has been easing access to these forms of entertainment and education for everyone, everywhere in the world. Given the easy and frequent interaction with a huge amount of movies and music daily, and the impact of these in our emotions, it becomes more and more relevant to address and think of ways to augment people's emotional perception and awareness of multimedia content in and through movies and music. In this paper, we present the motivation and describe the background for these challenges, and propose an approach for the design, development and expansion of interactive features that allow users to visualize and access emotions felt while engaging with movies and music. A special focus is put on the content in these forms of entertainment that have in some way meant something or can be associated with a significant memory, providing insights and helping to manage and regulate emotions, allowing to revisit content with an increased awareness or even recommend new content, taking into account users’ personality.","PeriodicalId":120872,"journal":{"name":"Proceedings of the 2023 ACM International Conference on Interactive Media Experiences","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129466751","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
As the use of Virtual Reality (VR) increases, museums have been using it to create simulations of their artefact collections. However, the level of accessibility, inclusiveness, and engagement of these simulations with senior citizens has been understudied. To address the problem, this case study presents the design of the “Pop-up VR Museum”, a VR experience based on cultural heritage artefacts from the Design Museum in Helsinki that attempts to engage with audiences of wide age ranges. Users can interact with virtual artefacts and listen to stories contributed by different communities. The Pop-up VR Museum has been tested with 254 users at the museum and taken to several elderly care homes. Evaluation is based on users’ gameplay data and their responses to post-experience questionnaires. Results indicate some variation in types of engagement based on users’ age groups. Despite potential limitations, this study provides valuable insights for other museums to create inclusive VR experiences.
{"title":"Enhancing Engagement through Digital Cultural Heritage: A Case Study about Senior Citizens using a Virtual Reality Museum","authors":"G. Vishwanath","doi":"10.1145/3573381.3596154","DOIUrl":"https://doi.org/10.1145/3573381.3596154","url":null,"abstract":"As the use of Virtual Reality (VR) increases, museums have been using it to create simulations of their artefact collections. However, the level of accessibility, inclusiveness, and engagement of these simulations with senior citizens has been understudied. To address the problem, this case study presents the design of the “Pop-up VR Museum”, a VR experience based on cultural heritage artefacts from the Design Museum in Helsinki that attempts to engage with audiences of wide age ranges. Users can interact with virtual artefacts and listen to stories contributed by different communities. The Pop-up VR Museum has been tested with 254 users at the museum and taken to several elderly care homes. Evaluation is based on users’ gameplay data and their responses to post-experience questionnaires. Results indicate some variation in types of engagement based on users’ age groups. Despite potential limitations, this study provides valuable insights for other museums to create inclusive VR experiences.","PeriodicalId":120872,"journal":{"name":"Proceedings of the 2023 ACM International Conference on Interactive Media Experiences","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132080440","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Virtual reality (VR) offers opportunities in human-computer interaction research, to embody users in immersive environments and observe how they interact with 3D scenarios under well-controlled environments. VR content has stronger influences on users physical and emotional states as compared to traditional 2D media, however, a fuller understanding of this kind of embodied interaction is currently limited by the extent to which attention and behaviour can be observed in a VR environment, and the accuracy at which these observations can be interpreted as, and mapped to, real-world interactions and intentions. This thesis aims at the creation of a system to help designers in the understanding of the embodied user experience in VR environment: how they feel, what is their intentions when interacting with a certain object, provide them guidance based on their needs and attention. Controlled guided scenarios will help to reduce the gap of perception between the designer building an experience, and the user living it, leading to more efficient behaviour analysis in VR systems.
{"title":"Analysing and Understanding Embodied Interactions in Virtual Reality Systems","authors":"Florent Robert","doi":"10.1145/3573381.3597234","DOIUrl":"https://doi.org/10.1145/3573381.3597234","url":null,"abstract":"Virtual reality (VR) offers opportunities in human-computer interaction research, to embody users in immersive environments and observe how they interact with 3D scenarios under well-controlled environments. VR content has stronger influences on users physical and emotional states as compared to traditional 2D media, however, a fuller understanding of this kind of embodied interaction is currently limited by the extent to which attention and behaviour can be observed in a VR environment, and the accuracy at which these observations can be interpreted as, and mapped to, real-world interactions and intentions. This thesis aims at the creation of a system to help designers in the understanding of the embodied user experience in VR environment: how they feel, what is their intentions when interacting with a certain object, provide them guidance based on their needs and attention. Controlled guided scenarios will help to reduce the gap of perception between the designer building an experience, and the user living it, leading to more efficient behaviour analysis in VR systems.","PeriodicalId":120872,"journal":{"name":"Proceedings of the 2023 ACM International Conference on Interactive Media Experiences","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129345610","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Automatically generating character motion is one of the technologies required for virtual reality, graphics, and robotics. Motion synthesis with deep learning is an emerging research topic. A key component of the development of such an algorithm involves the design of a proper objective metric to evaluate the quality and diversity of the synthesized motion dataset, two key factors of the performance of generative models. The Fréchet distance is nowadays a common method to assess this performance. In the motion generation field, the validation of such evaluation methods relies on the computation of the Fréchet distance between embeddings of the ground truth dataset and motion samples polluted by synthetic noise to mimic the artifacts produced by generative algorithms. However, the synthetic noise degradation does not fully represent motion perturbations that are commonly perceived. One of these artifacts is foot skating: the unnatural foot slides on the ground during locomotion. In this work-in-progress paper, we tested how well the Fréchet Motion Distance (FMD), which was proposed in previous works, is able to measure foot skating artifacts, and we found that FMD is not able to measure efficiently the intensity of the skating degradation.
{"title":"Validating Objective Evaluation Metric: Is Fréchet Motion Distance able to Capture Foot Skating Artifacts ?","authors":"Antoine Maiorca, Youngwoo Yoon, T. Dutoit","doi":"10.1145/3573381.3596460","DOIUrl":"https://doi.org/10.1145/3573381.3596460","url":null,"abstract":"Automatically generating character motion is one of the technologies required for virtual reality, graphics, and robotics. Motion synthesis with deep learning is an emerging research topic. A key component of the development of such an algorithm involves the design of a proper objective metric to evaluate the quality and diversity of the synthesized motion dataset, two key factors of the performance of generative models. The Fréchet distance is nowadays a common method to assess this performance. In the motion generation field, the validation of such evaluation methods relies on the computation of the Fréchet distance between embeddings of the ground truth dataset and motion samples polluted by synthetic noise to mimic the artifacts produced by generative algorithms. However, the synthetic noise degradation does not fully represent motion perturbations that are commonly perceived. One of these artifacts is foot skating: the unnatural foot slides on the ground during locomotion. In this work-in-progress paper, we tested how well the Fréchet Motion Distance (FMD), which was proposed in previous works, is able to measure foot skating artifacts, and we found that FMD is not able to measure efficiently the intensity of the skating degradation.","PeriodicalId":120872,"journal":{"name":"Proceedings of the 2023 ACM International Conference on Interactive Media Experiences","volume":"53 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131368053","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AI is becoming increasingly popular in artistic work. Yet tools for calculating environmental impact of AI are more adapted for other contexts than creative practices, making them sometimes hard to comprehend for the non-expert. In this study, based on interviews with AI artists, a design artifact called The Green Notebook was developed: a physical notebook where the AI artist could discuss ideas and receive feedback of their expected environmental impact. The conversational experience between the artist and the interface was informed by online content analysis of artistic work processes. The Notebook was explored and assessed with five artists in Wizard-of-Oz and focus group studies. Generally, the participants found a co-creation process with the enhanced ability to reflect on sustainability an accessible way to engage with sustainability considerations of their AI artistic practices. We provide insights of the Notebook’s perceived role and the conversational strategies used by the artists. Furthermore, we discuss trade-offs between politeness vs. efficiency and focus vs. integration to inform future research.
{"title":"The Green Notebook - A Co-Creativity Partner for Facilitating Sustainability Reflection","authors":"Yawen Deng, Petra Jääskeläinen, V. Popova","doi":"10.1145/3573381.3596465","DOIUrl":"https://doi.org/10.1145/3573381.3596465","url":null,"abstract":"AI is becoming increasingly popular in artistic work. Yet tools for calculating environmental impact of AI are more adapted for other contexts than creative practices, making them sometimes hard to comprehend for the non-expert. In this study, based on interviews with AI artists, a design artifact called The Green Notebook was developed: a physical notebook where the AI artist could discuss ideas and receive feedback of their expected environmental impact. The conversational experience between the artist and the interface was informed by online content analysis of artistic work processes. The Notebook was explored and assessed with five artists in Wizard-of-Oz and focus group studies. Generally, the participants found a co-creation process with the enhanced ability to reflect on sustainability an accessible way to engage with sustainability considerations of their AI artistic practices. We provide insights of the Notebook’s perceived role and the conversational strategies used by the artists. Furthermore, we discuss trade-offs between politeness vs. efficiency and focus vs. integration to inform future research.","PeriodicalId":120872,"journal":{"name":"Proceedings of the 2023 ACM International Conference on Interactive Media Experiences","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114935995","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Victor Roger, Yves Duvivier, Matthieu Perreira da Silva, Yannick Prié
Experiments in immersive environments allow the collection of large amounts of data that are closely related to individual behaviour. The recording of such experiments allows for the complex study of under-constrained tasks. That is, tasks that allow for a high degree of contingency in their resolution. This contingency allows for better discrimination of individual behaviour. However, the high complexity of the tasks makes them difficult to analyse. My thesis aims to discuss the advantages of Immersive Analytics for analysing hybrid sequential data (trajectory and events) generated in immersive environments. The analysis needs to be performed at a very high level of abstraction due to the high contingency of behaviours extracted from immersive environments. The massive amount of data generated highlights the need to build a model that allows feature extraction at a high level of abstraction. Since the exploration scheme is unknown in advance, the visualisations provided to the analyst should be highly interactive and adaptable to follow the analyst’s queries as he or she searches for new insights in the data.
{"title":"AI-Human Collaboration for in Situ Interactive Exploration of Behaviours From Immersive Environment","authors":"Victor Roger, Yves Duvivier, Matthieu Perreira da Silva, Yannick Prié","doi":"10.1145/3573381.3596506","DOIUrl":"https://doi.org/10.1145/3573381.3596506","url":null,"abstract":"Experiments in immersive environments allow the collection of large amounts of data that are closely related to individual behaviour. The recording of such experiments allows for the complex study of under-constrained tasks. That is, tasks that allow for a high degree of contingency in their resolution. This contingency allows for better discrimination of individual behaviour. However, the high complexity of the tasks makes them difficult to analyse. My thesis aims to discuss the advantages of Immersive Analytics for analysing hybrid sequential data (trajectory and events) generated in immersive environments. The analysis needs to be performed at a very high level of abstraction due to the high contingency of behaviours extracted from immersive environments. The massive amount of data generated highlights the need to build a model that allows feature extraction at a high level of abstraction. Since the exploration scheme is unknown in advance, the visualisations provided to the analyst should be highly interactive and adaptable to follow the analyst’s queries as he or she searches for new insights in the data.","PeriodicalId":120872,"journal":{"name":"Proceedings of the 2023 ACM International Conference on Interactive Media Experiences","volume":"63 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125473270","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In traditional subjective video quality experiments, the presented sequences are short and quality ratings are based on a single interaction with a service (i.e., one session). However, in real-life scenarios, users interact with a video service for a longer period of time. If decisions are made, such as to abandon a service, they are formulated based on longitudinal multi-episodic interaction. Therefore, it is important to better understand how quality is perceived in a longer interaction and how quality perception is linked to behavioral implications. My PhD work encompasses a longitudinal study of users’ interactions with a video service using a mobile device. In our study, which consists of six phases, we use different study designs to investigate how users perceive quality in a more ecologically valid setting. The study is carried out using a previously validated setup, which consists of compression software and a mobile application.
{"title":"Quality Assessment of Video Services in the Long Term","authors":"Natalia Cieplińska, L. Janowski, K. De Moor","doi":"10.1145/3573381.3596509","DOIUrl":"https://doi.org/10.1145/3573381.3596509","url":null,"abstract":"In traditional subjective video quality experiments, the presented sequences are short and quality ratings are based on a single interaction with a service (i.e., one session). However, in real-life scenarios, users interact with a video service for a longer period of time. If decisions are made, such as to abandon a service, they are formulated based on longitudinal multi-episodic interaction. Therefore, it is important to better understand how quality is perceived in a longer interaction and how quality perception is linked to behavioral implications. My PhD work encompasses a longitudinal study of users’ interactions with a video service using a mobile device. In our study, which consists of six phases, we use different study designs to investigate how users perceive quality in a more ecologically valid setting. The study is carried out using a previously validated setup, which consists of compression software and a mobile application.","PeriodicalId":120872,"journal":{"name":"Proceedings of the 2023 ACM International Conference on Interactive Media Experiences","volume":"84 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134183338","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ali Ak, Anne-Flore Perrin, Denise Noyes, I. Katsavounidis, P. Le Callet
User expectations are one of the main factors on providing satisfactory QoE for streaming service providers. Measuring acceptability and annoyance of video content, therefore, provide a valuable insight when measured under a given context. In this ongoing work, we measure video QoE in terms of acceptability and annoyance for the remaining data in a mobile data plan context.. We show that simple logos can be used during the experiment to prompt the context to subjects and the different context levels may impact the user expectations and consequently their satisfactions. Finally, we show that objective metrics can be used to determine the acceptability and annoyance thresholds for a given context.
{"title":"Video Consumption in Context: Influence of Data Plan Consumption on QoE","authors":"Ali Ak, Anne-Flore Perrin, Denise Noyes, I. Katsavounidis, P. Le Callet","doi":"10.1145/3573381.3596474","DOIUrl":"https://doi.org/10.1145/3573381.3596474","url":null,"abstract":"User expectations are one of the main factors on providing satisfactory QoE for streaming service providers. Measuring acceptability and annoyance of video content, therefore, provide a valuable insight when measured under a given context. In this ongoing work, we measure video QoE in terms of acceptability and annoyance for the remaining data in a mobile data plan context.. We show that simple logos can be used during the experiment to prompt the context to subjects and the different context levels may impact the user expectations and consequently their satisfactions. Finally, we show that objective metrics can be used to determine the acceptability and annoyance thresholds for a given context.","PeriodicalId":120872,"journal":{"name":"Proceedings of the 2023 ACM International Conference on Interactive Media Experiences","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115771305","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Miguel Fernández-Dasí, Mario Montagud Climent, Isaac Fraile, Josep Paradells, S. Fernández
This paper reports on the research being done towards enabling and understanding interactive social VR360 video viewing scenarios, by exclusively relying on web-based technologies, and using different types of consumption devices. After motivating the relevance of the research topic and associated impact, the paper elaborates on key requirements, features, and system components to effectively enable such scenarios, such as: adaptive and low-latency streaming, media synchronization, social presence, interaction channels, and assistive methods. For each of these features and components, different alternatives are assessed and proof of concept implementations are being provided. With an effective combination and integration of all these contributions, an end-to-end platform can be built and used as a research framework to explore the applicability and potential benefits of social VR360viewing in a variety of use cases, like education, culture or surveillance, by tailoring the technological components based on lessons learned from experimental studies. These use case studies can also provide relevant insights into activity patterns, behaviors, and preferences in Social Viewing scenarios.
{"title":"Enabling and Understanding Interactive Social VR360 Video Viewing","authors":"Miguel Fernández-Dasí, Mario Montagud Climent, Isaac Fraile, Josep Paradells, S. Fernández","doi":"10.1145/3573381.3597216","DOIUrl":"https://doi.org/10.1145/3573381.3597216","url":null,"abstract":"This paper reports on the research being done towards enabling and understanding interactive social VR360 video viewing scenarios, by exclusively relying on web-based technologies, and using different types of consumption devices. After motivating the relevance of the research topic and associated impact, the paper elaborates on key requirements, features, and system components to effectively enable such scenarios, such as: adaptive and low-latency streaming, media synchronization, social presence, interaction channels, and assistive methods. For each of these features and components, different alternatives are assessed and proof of concept implementations are being provided. With an effective combination and integration of all these contributions, an end-to-end platform can be built and used as a research framework to explore the applicability and potential benefits of social VR360viewing in a variety of use cases, like education, culture or surveillance, by tailoring the technological components based on lessons learned from experimental studies. These use case studies can also provide relevant insights into activity patterns, behaviors, and preferences in Social Viewing scenarios.","PeriodicalId":120872,"journal":{"name":"Proceedings of the 2023 ACM International Conference on Interactive Media Experiences","volume":"68 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124876530","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The aim of this project is to create an interactive assistant that incorporates different assistive features for blind and visually impaired people. The assistant might incorporate screen readers, magnifiers, voice synthesis, OCR, GPS, face recognition, and object recognition among other tools. Recently, the work done by OpenAI and Be My Eyes with the implementation of GPT-4 is comparable to the aim of this project. It shows the development of an interactive assistant has become simpler due to recent developments in large language models. However, older methods like named entity recognition and intent classification are still valuable to build lightweight assistants. A hybrid solution combining both methods seems possible, would help to reduce the computational cost of the assistant, and would facilitate the data collection process. Despite being more complex to implement in a multilingual and multimodal context, a hybrid solution has the potential to be used offline and to consume less resources.
这个项目的目的是为盲人和视障人士创造一个包含不同辅助功能的互动助手。该助手可能包含屏幕阅读器、放大镜、语音合成、OCR、GPS、人脸识别和物体识别等工具。最近，OpenAI和Be My Eyes在实施GPT-4方面所做的工作与这个项目的目标相当。它表明，由于最近大型语言模型的发展，交互式助手的开发变得更加简单。然而，像命名实体识别和意图分类这样的老方法对于构建轻量级助手仍然很有价值。结合这两种方法的混合解决方案似乎是可能的，这将有助于减少助手的计算成本，并将促进数据收集过程。尽管在多语言和多模式上下文中实现起来更加复杂，但混合解决方案具有离线使用和消耗更少资源的潜力。
{"title":"Developing an Interactive Agent for Blind and Visually Impaired People","authors":"V. Stragier, Omar Seddati, T. Dutoit","doi":"10.1145/3573381.3596471","DOIUrl":"https://doi.org/10.1145/3573381.3596471","url":null,"abstract":"The aim of this project is to create an interactive assistant that incorporates different assistive features for blind and visually impaired people. The assistant might incorporate screen readers, magnifiers, voice synthesis, OCR, GPS, face recognition, and object recognition among other tools. Recently, the work done by OpenAI and Be My Eyes with the implementation of GPT-4 is comparable to the aim of this project. It shows the development of an interactive assistant has become simpler due to recent developments in large language models. However, older methods like named entity recognition and intent classification are still valuable to build lightweight assistants. A hybrid solution combining both methods seems possible, would help to reduce the computational cost of the assistant, and would facilitate the data collection process. Despite being more complex to implement in a multilingual and multimodal context, a hybrid solution has the potential to be used offline and to consume less resources.","PeriodicalId":120872,"journal":{"name":"Proceedings of the 2023 ACM International Conference on Interactive Media Experiences","volume":"99 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128499600","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：