Pub Date : 2023-09-12DOI: 10.1007/s12193-023-00411-y
Martin Dobiasch, Stefan Oppl, Michael Stöckl, Arnold Baca
Abstract Feedback is essential for athletes in order to improve their sport performance. Feedback systems try to provide athletes and coaches not only with visualisations of acquired data, but moreover, with insights into—possibly—invisible aspects of their performance. With the widespread adoption of smartphones and the increase in their capabilities, their use as a device for applications of feedback systems is becoming increasingly popular. However, developing mobile feedback systems requires a high level of expertise from researchers and practitioners. The Direct Mobile Coaching model is a design-paradigm for mobile feedback systems. In order to reduce programming efforts, PEGASOS, a framework for creating feedback systems implementing the so-called Direct Mobile Coaching model, is introduced. The paper compares this framework with state-of-the-art research with regard to their ability of providing different variants feedback and offering multimodality to users.
{"title":"Pegasos: a framework for the creation of direct mobile coaching feedback systems","authors":"Martin Dobiasch, Stefan Oppl, Michael Stöckl, Arnold Baca","doi":"10.1007/s12193-023-00411-y","DOIUrl":"https://doi.org/10.1007/s12193-023-00411-y","url":null,"abstract":"Abstract Feedback is essential for athletes in order to improve their sport performance. Feedback systems try to provide athletes and coaches not only with visualisations of acquired data, but moreover, with insights into—possibly—invisible aspects of their performance. With the widespread adoption of smartphones and the increase in their capabilities, their use as a device for applications of feedback systems is becoming increasingly popular. However, developing mobile feedback systems requires a high level of expertise from researchers and practitioners. The Direct Mobile Coaching model is a design-paradigm for mobile feedback systems. In order to reduce programming efforts, PEGASOS, a framework for creating feedback systems implementing the so-called Direct Mobile Coaching model, is introduced. The paper compares this framework with state-of-the-art research with regard to their ability of providing different variants feedback and offering multimodality to users.","PeriodicalId":17529,"journal":{"name":"Journal on Multimodal User Interfaces","volume":"40 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135824683","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-09-09DOI: 10.1007/s12193-023-00414-9
Adrian B. Latupeirissa, Roberto Bresin
Abstract This paper presents the design and development of PepperOSC, an interface that connects Pepper and NAO robots with sound production tools to enable the development of interactive sonification in human-robot interaction (HRI). The interface uses Open Sound Control (OSC) messages to stream kinematic data from robots to various sound design and music production tools. The goals of PepperOSC are twofold: (i) to provide a tool for HRI researchers in developing multimodal user interfaces through sonification, and (ii) to lower the barrier for sound designers to contribute to HRI. To demonstrate the potential use of PepperOSC, this paper also presents two applications we have conducted: (i) a course project by two master’s students who created a robot sound model in Pure Data, and (ii) a museum installation of Pepper robot, employing sound models developed by a sound designer and a composer/researcher in music technology using MaxMSP and SuperCollider respectively. Furthermore, we discuss the potential use cases of PepperOSC in social robotics and artistic contexts. These applications demonstrate the versatility of PepperOSC and its ability to explore diverse aesthetic strategies for robot movement sonification, offering a promising approach to enhance the effectiveness and appeal of human-robot interactions.
{"title":"PepperOSC: enabling interactive sonification of a robot’s expressive movement","authors":"Adrian B. Latupeirissa, Roberto Bresin","doi":"10.1007/s12193-023-00414-9","DOIUrl":"https://doi.org/10.1007/s12193-023-00414-9","url":null,"abstract":"Abstract This paper presents the design and development of PepperOSC, an interface that connects Pepper and NAO robots with sound production tools to enable the development of interactive sonification in human-robot interaction (HRI). The interface uses Open Sound Control (OSC) messages to stream kinematic data from robots to various sound design and music production tools. The goals of PepperOSC are twofold: (i) to provide a tool for HRI researchers in developing multimodal user interfaces through sonification, and (ii) to lower the barrier for sound designers to contribute to HRI. To demonstrate the potential use of PepperOSC, this paper also presents two applications we have conducted: (i) a course project by two master’s students who created a robot sound model in Pure Data, and (ii) a museum installation of Pepper robot, employing sound models developed by a sound designer and a composer/researcher in music technology using MaxMSP and SuperCollider respectively. Furthermore, we discuss the potential use cases of PepperOSC in social robotics and artistic contexts. These applications demonstrate the versatility of PepperOSC and its ability to explore diverse aesthetic strategies for robot movement sonification, offering a promising approach to enhance the effectiveness and appeal of human-robot interactions.","PeriodicalId":17529,"journal":{"name":"Journal on Multimodal User Interfaces","volume":"149 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136108559","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-08-30DOI: 10.1007/s12193-023-00413-w
J. Fitzpatrick, Flaithrí Neff
{"title":"Perceptually congruent sonification of auditory line charts","authors":"J. Fitzpatrick, Flaithrí Neff","doi":"10.1007/s12193-023-00413-w","DOIUrl":"https://doi.org/10.1007/s12193-023-00413-w","url":null,"abstract":"","PeriodicalId":17529,"journal":{"name":"Journal on Multimodal User Interfaces","volume":"1 1","pages":""},"PeriodicalIF":2.9,"publicationDate":"2023-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48937999","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-08-28DOI: 10.1007/s12193-023-00412-x
Guoxuan Ning, Brianna Grant, B. Kapralos, A. Quevedo, Kc Collins, K. Kanev, A. Dubrowski
{"title":"Correction to: Understanding virtual drilling perception using sound, and kinesthetic cues obtained with a mouse and keyboard","authors":"Guoxuan Ning, Brianna Grant, B. Kapralos, A. Quevedo, Kc Collins, K. Kanev, A. Dubrowski","doi":"10.1007/s12193-023-00412-x","DOIUrl":"https://doi.org/10.1007/s12193-023-00412-x","url":null,"abstract":"","PeriodicalId":17529,"journal":{"name":"Journal on Multimodal User Interfaces","volume":"17 1","pages":"165 - 165"},"PeriodicalIF":2.9,"publicationDate":"2023-08-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46417378","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-08-18DOI: 10.1007/s12193-023-00409-6
Haram Choi, Joungheum Kwon, Sanghun Nam
{"title":"Research on the application of gaze visualization interface on virtual reality training systems","authors":"Haram Choi, Joungheum Kwon, Sanghun Nam","doi":"10.1007/s12193-023-00409-6","DOIUrl":"https://doi.org/10.1007/s12193-023-00409-6","url":null,"abstract":"","PeriodicalId":17529,"journal":{"name":"Journal on Multimodal User Interfaces","volume":"17 1","pages":"203 - 211"},"PeriodicalIF":2.9,"publicationDate":"2023-08-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44814008","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-08-14DOI: 10.1007/s12193-023-00410-z
Paula Castro Sánchez, Casey C. Bennett
{"title":"Facial expression recognition via transfer learning in cooperative game paradigms for enhanced social AI","authors":"Paula Castro Sánchez, Casey C. Bennett","doi":"10.1007/s12193-023-00410-z","DOIUrl":"https://doi.org/10.1007/s12193-023-00410-z","url":null,"abstract":"","PeriodicalId":17529,"journal":{"name":"Journal on Multimodal User Interfaces","volume":"17 1","pages":"187 - 201"},"PeriodicalIF":2.9,"publicationDate":"2023-08-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45061503","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-08-10DOI: 10.1007/s12193-023-00408-7
Santiago Villarreal-Narvaez, J. Pérez-Medina, J. Vanderdonckt
{"title":"Exploring user-defined gestures for lingual and palatal interaction","authors":"Santiago Villarreal-Narvaez, J. Pérez-Medina, J. Vanderdonckt","doi":"10.1007/s12193-023-00408-7","DOIUrl":"https://doi.org/10.1007/s12193-023-00408-7","url":null,"abstract":"","PeriodicalId":17529,"journal":{"name":"Journal on Multimodal User Interfaces","volume":"17 1","pages":"167 - 185"},"PeriodicalIF":2.9,"publicationDate":"2023-08-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41925603","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-08-04DOI: 10.1007/s12193-023-00407-8
Guoxuan Ning, Brianna Grant, B. Kapralos, A. Quevedo, Kc Collins, K. Kanev, A. Dubrowski
{"title":"Understanding virtual drilling perception using sound, and kinesthetic cues obtained with a mouse and keyboard","authors":"Guoxuan Ning, Brianna Grant, B. Kapralos, A. Quevedo, Kc Collins, K. Kanev, A. Dubrowski","doi":"10.1007/s12193-023-00407-8","DOIUrl":"https://doi.org/10.1007/s12193-023-00407-8","url":null,"abstract":"","PeriodicalId":17529,"journal":{"name":"Journal on Multimodal User Interfaces","volume":"17 1","pages":"151 - 163"},"PeriodicalIF":2.9,"publicationDate":"2023-08-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43977025","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-07-27DOI: 10.1007/s12193-023-00406-9
Sophie Dewil, Shterna Kuptchik, Mingxiao Liu, Sean Sanford, Troy Bradbury, Elena Davis, Amanda Clemente, R. Nataraj
{"title":"The cognitive basis for virtual reality rehabilitation of upper-extremity motor function after neurotraumas","authors":"Sophie Dewil, Shterna Kuptchik, Mingxiao Liu, Sean Sanford, Troy Bradbury, Elena Davis, Amanda Clemente, R. Nataraj","doi":"10.1007/s12193-023-00406-9","DOIUrl":"https://doi.org/10.1007/s12193-023-00406-9","url":null,"abstract":"","PeriodicalId":17529,"journal":{"name":"Journal on Multimodal User Interfaces","volume":"17 1","pages":"105 - 120"},"PeriodicalIF":2.9,"publicationDate":"2023-07-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44628442","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-07-08DOI: 10.1007/s12193-023-00404-x
Elias Elmquist, A. Bock, Jonas Lundberg, A. Ynnerman, N. Rönnberg
{"title":"SonAir: the design of a sonification of radar data for air traffic control","authors":"Elias Elmquist, A. Bock, Jonas Lundberg, A. Ynnerman, N. Rönnberg","doi":"10.1007/s12193-023-00404-x","DOIUrl":"https://doi.org/10.1007/s12193-023-00404-x","url":null,"abstract":"","PeriodicalId":17529,"journal":{"name":"Journal on Multimodal User Interfaces","volume":"17 1","pages":"137 - 149"},"PeriodicalIF":2.9,"publicationDate":"2023-07-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41396354","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：