Dylan Wootton, Amy Rae Fox, Evan Peck, Arvind Satyanarayan
Interactive visualizations are powerful tools for Exploratory Data Analysis (EDA), but how do they affect the observations analysts make about their data? We conducted a qualitative experiment with 13 professional data scientists analyzing two datasets with Jupyter notebooks, collecting a rich dataset of interaction traces and think-aloud utterances. By qualitatively coding participant utterances, we introduce a formalism that describes EDA as a sequence of analysis states, where each state is comprised of either a representation an analyst constructs (e.g., the output of a data frame, an interactive visualization, etc.) or an observation the analyst makes (e.g., about missing data, the relationship between variables, etc.). By applying our formalism to our dataset, we identify that interactive visualizations, on average, lead to earlier and more complex insights about relationships between dataset attributes compared to static visualizations. Moreover, by calculating metrics such as revisit count and representational diversity, we uncover that some representations serve more as "planning aids" during EDA rather than tools strictly for hypothesis-answering. We show how these measures help identify other patterns of analysis behavior, such as the "80-20 rule", where a small subset of representations drove the majority of observations. Based on these findings, we offer design guidelines for interactive exploratory analysis tooling and reflect on future directions for studying the role that visualizations play in EDA.
{"title":"Charting EDA: Characterizing Interactive Visualization Use in Computational Notebooks with a Mixed-Methods Formalism","authors":"Dylan Wootton, Amy Rae Fox, Evan Peck, Arvind Satyanarayan","doi":"arxiv-2409.10450","DOIUrl":"https://doi.org/arxiv-2409.10450","url":null,"abstract":"Interactive visualizations are powerful tools for Exploratory Data Analysis\u0000(EDA), but how do they affect the observations analysts make about their data?\u0000We conducted a qualitative experiment with 13 professional data scientists\u0000analyzing two datasets with Jupyter notebooks, collecting a rich dataset of\u0000interaction traces and think-aloud utterances. By qualitatively coding\u0000participant utterances, we introduce a formalism that describes EDA as a\u0000sequence of analysis states, where each state is comprised of either a\u0000representation an analyst constructs (e.g., the output of a data frame, an\u0000interactive visualization, etc.) or an observation the analyst makes (e.g.,\u0000about missing data, the relationship between variables, etc.). By applying our\u0000formalism to our dataset, we identify that interactive visualizations, on\u0000average, lead to earlier and more complex insights about relationships between\u0000dataset attributes compared to static visualizations. Moreover, by calculating\u0000metrics such as revisit count and representational diversity, we uncover that\u0000some representations serve more as \"planning aids\" during EDA rather than tools\u0000strictly for hypothesis-answering. We show how these measures help identify\u0000other patterns of analysis behavior, such as the \"80-20 rule\", where a small\u0000subset of representations drove the majority of observations. Based on these\u0000findings, we offer design guidelines for interactive exploratory analysis\u0000tooling and reflect on future directions for studying the role that\u0000visualizations play in EDA.","PeriodicalId":501541,"journal":{"name":"arXiv - CS - Human-Computer Interaction","volume":"210 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142269799","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This article focuses on the articulatory characteristics of epistemic gestures (i.e., gestures used to express certainty or uncertainty) in co-speech gestures (CSG) in French and in French Sign Language (LSF). It presents a new methodology for analysis, which relies on the complementary use of manual annotation (using Typannot) and semi-automatic annotation (using AlphaPose) to highlight the kinesiological characteristics of these epistemic gestures. The presented methodology allows to analyze the flexion/extension movements of the head in epistemic contexts. The results of this analysis show that in CSG and LSF: (1) head nods passing through the neutral position (i.e., head straight with no flexion/extension) and high movement speed are markers of certainty; and (2) holding the head position away from the neutral position and low movement speed indicate uncertainty. This study is conducted within the framework of the ANR LexiKHuM project, which develops kinesthetic communication solutions for human-machine interaction.
{"title":"Protocol for identifying shared articulatory features of gestures and LSF: application to epistemic gesture","authors":"Fanny CatteauSFL, Claudia S BianchiniFoReLLIS","doi":"arxiv-2409.10079","DOIUrl":"https://doi.org/arxiv-2409.10079","url":null,"abstract":"This article focuses on the articulatory characteristics of epistemic\u0000gestures (i.e., gestures used to express certainty or uncertainty) in co-speech\u0000gestures (CSG) in French and in French Sign Language (LSF). It presents a new\u0000methodology for analysis, which relies on the complementary use of manual\u0000annotation (using Typannot) and semi-automatic annotation (using AlphaPose) to\u0000highlight the kinesiological characteristics of these epistemic gestures. The\u0000presented methodology allows to analyze the flexion/extension movements of the\u0000head in epistemic contexts. The results of this analysis show that in CSG and\u0000LSF: (1) head nods passing through the neutral position (i.e., head straight\u0000with no flexion/extension) and high movement speed are markers of certainty;\u0000and (2) holding the head position away from the neutral position and low\u0000movement speed indicate uncertainty. This study is conducted within the\u0000framework of the ANR LexiKHuM project, which develops kinesthetic communication\u0000solutions for human-machine interaction.","PeriodicalId":501541,"journal":{"name":"arXiv - CS - Human-Computer Interaction","volume":"16 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142252475","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mine Dastan, Michele Fiorentino, Elias D. Walter, Christian Diegritz, Antonio E. Uva, Ulrich Eck, Nassir Navab
Mixed Reality (MR) is proven in the literature to support precise spatial dental drill positioning by superimposing 3D widgets. Despite this, the related knowledge about widget's visual design and interactive user feedback is still limited. Therefore, this study is contributed to by co-designed MR drill tool positioning widgets with two expert dentists and three MR experts. The results of co-design are two static widgets (SWs): a simple entry point, a target axis, and two dynamic widgets (DWs), variants of dynamic error visualization with and without a target axis (DWTA and DWEP). We evaluated the co-designed widgets in a virtual reality simulation supported by a realistic setup with a tracked phantom patient, a virtual magnifying loupe, and a dentist's foot pedal. The user study involved 35 dentists with various backgrounds and years of experience. The findings demonstrated significant results; DWs outperform SWs in positional and rotational precision, especially with younger generations and subjects with gaming experiences. The user preference remains for DWs (19) instead of SWs (16). However, findings indicated that the precision positively correlates with the time trade-off. The post-experience questionnaire (NASA-TLX) showed that DWs increase mental and physical demand, effort, and frustration more than SWs. Comparisons between DWEP and DWTA show that the DW's complexity level influences time, physical and mental demands. The DWs are extensible to diverse medical and industrial scenarios that demand precision.
{"title":"Co-Designing Dynamic Mixed Reality Drill Positioning Widgets: A Collaborative Approach with Dentists in a Realistic Setup","authors":"Mine Dastan, Michele Fiorentino, Elias D. Walter, Christian Diegritz, Antonio E. Uva, Ulrich Eck, Nassir Navab","doi":"arxiv-2409.10258","DOIUrl":"https://doi.org/arxiv-2409.10258","url":null,"abstract":"Mixed Reality (MR) is proven in the literature to support precise spatial\u0000dental drill positioning by superimposing 3D widgets. Despite this, the related\u0000knowledge about widget's visual design and interactive user feedback is still\u0000limited. Therefore, this study is contributed to by co-designed MR drill tool\u0000positioning widgets with two expert dentists and three MR experts. The results\u0000of co-design are two static widgets (SWs): a simple entry point, a target axis,\u0000and two dynamic widgets (DWs), variants of dynamic error visualization with and\u0000without a target axis (DWTA and DWEP). We evaluated the co-designed widgets in\u0000a virtual reality simulation supported by a realistic setup with a tracked\u0000phantom patient, a virtual magnifying loupe, and a dentist's foot pedal. The\u0000user study involved 35 dentists with various backgrounds and years of\u0000experience. The findings demonstrated significant results; DWs outperform SWs\u0000in positional and rotational precision, especially with younger generations and\u0000subjects with gaming experiences. The user preference remains for DWs (19)\u0000instead of SWs (16). However, findings indicated that the precision positively\u0000correlates with the time trade-off. The post-experience questionnaire\u0000(NASA-TLX) showed that DWs increase mental and physical demand, effort, and\u0000frustration more than SWs. Comparisons between DWEP and DWTA show that the DW's\u0000complexity level influences time, physical and mental demands. The DWs are\u0000extensible to diverse medical and industrial scenarios that demand precision.","PeriodicalId":501541,"journal":{"name":"arXiv - CS - Human-Computer Interaction","volume":"192 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142252472","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yifan Wang, David Stevens, Pranay Shah, Wenwen Jiang, Miao Liu, Xu Chen, Robert Kuo, Na Li, Boying Gong, Daniel Lee, Jiabo Hu, Ning Zhang, Bob Kamma
The growing demand for AI training data has transformed data annotation into a global industry, but traditional approaches relying on human annotators are often time-consuming, labor-intensive, and prone to inconsistent quality. We propose the Model-in-the-Loop (MILO) framework, which integrates AI/ML models into the annotation process. Our research introduces a collaborative paradigm that leverages the strengths of both professional human annotators and large language models (LLMs). By employing LLMs as pre-annotation and real-time assistants, and judges on annotator responses, MILO enables effective interaction patterns between human annotators and LLMs. Three empirical studies on multimodal data annotation demonstrate MILO's efficacy in reducing handling time, improving data quality, and enhancing annotator experiences. We also introduce quality rubrics for flexible evaluation and fine-grained feedback on open-ended annotations. The MILO framework has implications for accelerating AI/ML development, reducing reliance on human annotation alone, and promoting better alignment between human and machine values.
{"title":"Model-in-the-Loop (MILO): Accelerating Multimodal AI Data Annotation with LLMs","authors":"Yifan Wang, David Stevens, Pranay Shah, Wenwen Jiang, Miao Liu, Xu Chen, Robert Kuo, Na Li, Boying Gong, Daniel Lee, Jiabo Hu, Ning Zhang, Bob Kamma","doi":"arxiv-2409.10702","DOIUrl":"https://doi.org/arxiv-2409.10702","url":null,"abstract":"The growing demand for AI training data has transformed data annotation into\u0000a global industry, but traditional approaches relying on human annotators are\u0000often time-consuming, labor-intensive, and prone to inconsistent quality. We\u0000propose the Model-in-the-Loop (MILO) framework, which integrates AI/ML models\u0000into the annotation process. Our research introduces a collaborative paradigm\u0000that leverages the strengths of both professional human annotators and large\u0000language models (LLMs). By employing LLMs as pre-annotation and real-time\u0000assistants, and judges on annotator responses, MILO enables effective\u0000interaction patterns between human annotators and LLMs. Three empirical studies\u0000on multimodal data annotation demonstrate MILO's efficacy in reducing handling\u0000time, improving data quality, and enhancing annotator experiences. We also\u0000introduce quality rubrics for flexible evaluation and fine-grained feedback on\u0000open-ended annotations. The MILO framework has implications for accelerating\u0000AI/ML development, reducing reliance on human annotation alone, and promoting\u0000better alignment between human and machine values.","PeriodicalId":501541,"journal":{"name":"arXiv - CS - Human-Computer Interaction","volume":"54 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142252431","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Despite being an integral tool for finding health-related information online, YouTube has faced criticism for disseminating COVID-19 misinformation globally to its users. Yet, prior audit studies have predominantly investigated YouTube within the Global North contexts, often overlooking the Global South. To address this gap, we conducted a comprehensive 10-day geolocation-based audit on YouTube to compare the prevalence of COVID-19 misinformation in search results between the United States (US) and South Africa (SA), the countries heavily affected by the pandemic in the Global North and the Global South, respectively. For each country, we selected 3 geolocations and placed sock-puppets, or bots emulating "real" users, that collected search results for 48 search queries sorted by 4 search filters for 10 days, yielding a dataset of 915K results. We found that 31.55% of the top-10 search results contained COVID-19 misinformation. Among the top-10 search results, bots in SA faced significantly more misinformative search results than their US counterparts. Overall, our study highlights the contrasting algorithmic behaviors of YouTube search between two countries, underscoring the need for the platform to regulate algorithmic behavior consistently across different regions of the Globe.
{"title":"Algorithmic Behaviors Across Regions: A Geolocation Audit of YouTube Search for COVID-19 Misinformation between the United States and South Africa","authors":"Hayoung Jung, Prerna Juneja, Tanushree Mitra","doi":"arxiv-2409.10168","DOIUrl":"https://doi.org/arxiv-2409.10168","url":null,"abstract":"Despite being an integral tool for finding health-related information online,\u0000YouTube has faced criticism for disseminating COVID-19 misinformation globally\u0000to its users. Yet, prior audit studies have predominantly investigated YouTube\u0000within the Global North contexts, often overlooking the Global South. To\u0000address this gap, we conducted a comprehensive 10-day geolocation-based audit\u0000on YouTube to compare the prevalence of COVID-19 misinformation in search\u0000results between the United States (US) and South Africa (SA), the countries\u0000heavily affected by the pandemic in the Global North and the Global South,\u0000respectively. For each country, we selected 3 geolocations and placed\u0000sock-puppets, or bots emulating \"real\" users, that collected search results for\u000048 search queries sorted by 4 search filters for 10 days, yielding a dataset of\u0000915K results. We found that 31.55% of the top-10 search results contained\u0000COVID-19 misinformation. Among the top-10 search results, bots in SA faced\u0000significantly more misinformative search results than their US counterparts.\u0000Overall, our study highlights the contrasting algorithmic behaviors of YouTube\u0000search between two countries, underscoring the need for the platform to\u0000regulate algorithmic behavior consistently across different regions of the\u0000Globe.","PeriodicalId":501541,"journal":{"name":"arXiv - CS - Human-Computer Interaction","volume":"42 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142252527","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jindan Huang, Isaac Sheidlower, Reuben M. Aronson, Elaine Schaertl Short
Human-in-the-loop learning is gaining popularity, particularly in the field of robotics, because it leverages human knowledge about real-world tasks to facilitate agent learning. When people instruct robots, they naturally adapt their teaching behavior in response to changes in robot performance. While current research predominantly focuses on integrating human teaching dynamics from an algorithmic perspective, understanding these dynamics from a human-centered standpoint is an under-explored, yet fundamental problem. Addressing this issue will enhance both robot learning and user experience. Therefore, this paper explores one potential factor contributing to the dynamic nature of human teaching: robot errors. We conducted a user study to investigate how the presence and severity of robot errors affect three dimensions of human teaching dynamics: feedback granularity, feedback richness, and teaching time, in both forced-choice and open-ended teaching contexts. The results show that people tend to spend more time teaching robots with errors, provide more detailed feedback over specific segments of a robot's trajectory, and that robot error can influence a teacher's choice of feedback modality. Our findings offer valuable insights for designing effective interfaces for interactive learning and optimizing algorithms to better understand human intentions.
{"title":"On the Effect of Robot Errors on Human Teaching Dynamics","authors":"Jindan Huang, Isaac Sheidlower, Reuben M. Aronson, Elaine Schaertl Short","doi":"arxiv-2409.09827","DOIUrl":"https://doi.org/arxiv-2409.09827","url":null,"abstract":"Human-in-the-loop learning is gaining popularity, particularly in the field\u0000of robotics, because it leverages human knowledge about real-world tasks to\u0000facilitate agent learning. When people instruct robots, they naturally adapt\u0000their teaching behavior in response to changes in robot performance. While\u0000current research predominantly focuses on integrating human teaching dynamics\u0000from an algorithmic perspective, understanding these dynamics from a\u0000human-centered standpoint is an under-explored, yet fundamental problem.\u0000Addressing this issue will enhance both robot learning and user experience.\u0000Therefore, this paper explores one potential factor contributing to the dynamic\u0000nature of human teaching: robot errors. We conducted a user study to\u0000investigate how the presence and severity of robot errors affect three\u0000dimensions of human teaching dynamics: feedback granularity, feedback richness,\u0000and teaching time, in both forced-choice and open-ended teaching contexts. The\u0000results show that people tend to spend more time teaching robots with errors,\u0000provide more detailed feedback over specific segments of a robot's trajectory,\u0000and that robot error can influence a teacher's choice of feedback modality. Our\u0000findings offer valuable insights for designing effective interfaces for\u0000interactive learning and optimizing algorithms to better understand human\u0000intentions.","PeriodicalId":501541,"journal":{"name":"arXiv - CS - Human-Computer Interaction","volume":"110 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142268607","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Inhwa Song, SoHyun Park, Sachin R. Pendse, Jessica Lee Schleider, Munmun De Choudhury, Young-Ho Kim
Expressing stressful experiences in words is proven to improve mental and physical health, but individuals often disengage with writing interventions as they struggle to organize their thoughts and emotions. Reflective prompts have been used to provide direction, and large language models (LLMs) have demonstrated the potential to provide tailored guidance. Current systems often limit users' flexibility to direct their reflections. We thus present ExploreSelf, an LLM-driven application designed to empower users to control their reflective journey. ExploreSelf allows users to receive adaptive support through dynamically generated questions. Through an exploratory study with 19 participants, we examine how participants explore and reflect on personal challenges using ExploreSelf. Our findings demonstrate that participants valued the balance between guided support and freedom to control their reflective journey, leading to deeper engagement and insight. Building on our findings, we discuss implications for designing LLM-driven tools that promote user empowerment through effective reflective practices.
{"title":"ExploreSelf: Fostering User-driven Exploration and Reflection on Personal Challenges with Adaptive Guidance by Large Language Models","authors":"Inhwa Song, SoHyun Park, Sachin R. Pendse, Jessica Lee Schleider, Munmun De Choudhury, Young-Ho Kim","doi":"arxiv-2409.09662","DOIUrl":"https://doi.org/arxiv-2409.09662","url":null,"abstract":"Expressing stressful experiences in words is proven to improve mental and\u0000physical health, but individuals often disengage with writing interventions as\u0000they struggle to organize their thoughts and emotions. Reflective prompts have\u0000been used to provide direction, and large language models (LLMs) have\u0000demonstrated the potential to provide tailored guidance. Current systems often\u0000limit users' flexibility to direct their reflections. We thus present\u0000ExploreSelf, an LLM-driven application designed to empower users to control\u0000their reflective journey. ExploreSelf allows users to receive adaptive support\u0000through dynamically generated questions. Through an exploratory study with 19\u0000participants, we examine how participants explore and reflect on personal\u0000challenges using ExploreSelf. Our findings demonstrate that participants valued\u0000the balance between guided support and freedom to control their reflective\u0000journey, leading to deeper engagement and insight. Building on our findings, we\u0000discuss implications for designing LLM-driven tools that promote user\u0000empowerment through effective reflective practices.","PeriodicalId":501541,"journal":{"name":"arXiv - CS - Human-Computer Interaction","volume":"46 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142268567","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Subigya Nepal, Arvind Pillai, William Campbell, Talie Massachi, Michael V. Heinz, Ashmita Kunwar, Eunsol Soul Choi, Orson Xu, Joanna Kuc, Jeremy Huckins, Jason Holden, Sarah M. Preum, Colin Depp, Nicholas Jacobson, Mary Czerwinski, Eric Granholm, Andrew T. Campbell
Mental health concerns are prevalent among college students, highlighting the need for effective interventions that promote self-awareness and holistic well-being. MindScape pioneers a novel approach to AI-powered journaling by integrating passively collected behavioral patterns such as conversational engagement, sleep, and location with Large Language Models (LLMs). This integration creates a highly personalized and context-aware journaling experience, enhancing self-awareness and well-being by embedding behavioral intelligence into AI. We present an 8-week exploratory study with 20 college students, demonstrating the MindScape app's efficacy in enhancing positive affect (7%), reducing negative affect (11%), loneliness (6%), and anxiety and depression, with a significant week-over-week decrease in PHQ-4 scores (-0.25 coefficient), alongside improvements in mindfulness (7%) and self-reflection (6%). The study highlights the advantages of contextual AI journaling, with participants particularly appreciating the tailored prompts and insights provided by the MindScape app. Our analysis also includes a comparison of responses to AI-driven contextual versus generic prompts, participant feedback insights, and proposed strategies for leveraging contextual AI journaling to improve well-being on college campuses. By showcasing the potential of contextual AI journaling to support mental health, we provide a foundation for further investigation into the effects of contextual AI journaling on mental health and well-being.
{"title":"MindScape Study: Integrating LLM and Behavioral Sensing for Personalized AI-Driven Journaling Experiences","authors":"Subigya Nepal, Arvind Pillai, William Campbell, Talie Massachi, Michael V. Heinz, Ashmita Kunwar, Eunsol Soul Choi, Orson Xu, Joanna Kuc, Jeremy Huckins, Jason Holden, Sarah M. Preum, Colin Depp, Nicholas Jacobson, Mary Czerwinski, Eric Granholm, Andrew T. Campbell","doi":"arxiv-2409.09570","DOIUrl":"https://doi.org/arxiv-2409.09570","url":null,"abstract":"Mental health concerns are prevalent among college students, highlighting the\u0000need for effective interventions that promote self-awareness and holistic\u0000well-being. MindScape pioneers a novel approach to AI-powered journaling by\u0000integrating passively collected behavioral patterns such as conversational\u0000engagement, sleep, and location with Large Language Models (LLMs). This\u0000integration creates a highly personalized and context-aware journaling\u0000experience, enhancing self-awareness and well-being by embedding behavioral\u0000intelligence into AI. We present an 8-week exploratory study with 20 college\u0000students, demonstrating the MindScape app's efficacy in enhancing positive\u0000affect (7%), reducing negative affect (11%), loneliness (6%), and anxiety and\u0000depression, with a significant week-over-week decrease in PHQ-4 scores (-0.25\u0000coefficient), alongside improvements in mindfulness (7%) and self-reflection\u0000(6%). The study highlights the advantages of contextual AI journaling, with\u0000participants particularly appreciating the tailored prompts and insights\u0000provided by the MindScape app. Our analysis also includes a comparison of\u0000responses to AI-driven contextual versus generic prompts, participant feedback\u0000insights, and proposed strategies for leveraging contextual AI journaling to\u0000improve well-being on college campuses. By showcasing the potential of\u0000contextual AI journaling to support mental health, we provide a foundation for\u0000further investigation into the effects of contextual AI journaling on mental\u0000health and well-being.","PeriodicalId":501541,"journal":{"name":"arXiv - CS - Human-Computer Interaction","volume":"15 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142252481","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dasom Choi, SoHyun Park, Kyungah Lee, Hwajung Hong, Young-Ho Kim
As minimally verbal autistic (MVA) children communicate with parents through few words and nonverbal cues, parents often struggle to encourage their children to express subtle emotions and needs and to grasp their nuanced signals. We present AACessTalk, a tablet-based, AI-mediated communication system that facilitates meaningful exchanges between an MVA child and a parent. AACessTalk provides real-time guides to the parent to engage the child in conversation and, in turn, recommends contextual vocabulary cards to the child. Through a two-week deployment study with 11 MVA child-parent dyads, we examine how AACessTalk fosters everyday conversation practice and mutual engagement. Our findings show high engagement from all dyads, leading to increased frequency of conversation and turn-taking. AACessTalk also encouraged parents to explore their own interaction strategies and empowered the children to have more agency in communication. We discuss the implications of designing technologies for balanced communication dynamics in parent-MVA child interaction.
{"title":"AACessTalk: Fostering Communication between Minimally Verbal Autistic Children and Parents with Contextual Guidance and Card Recommendation","authors":"Dasom Choi, SoHyun Park, Kyungah Lee, Hwajung Hong, Young-Ho Kim","doi":"arxiv-2409.09641","DOIUrl":"https://doi.org/arxiv-2409.09641","url":null,"abstract":"As minimally verbal autistic (MVA) children communicate with parents through\u0000few words and nonverbal cues, parents often struggle to encourage their\u0000children to express subtle emotions and needs and to grasp their nuanced\u0000signals. We present AACessTalk, a tablet-based, AI-mediated communication\u0000system that facilitates meaningful exchanges between an MVA child and a parent.\u0000AACessTalk provides real-time guides to the parent to engage the child in\u0000conversation and, in turn, recommends contextual vocabulary cards to the child.\u0000Through a two-week deployment study with 11 MVA child-parent dyads, we examine\u0000how AACessTalk fosters everyday conversation practice and mutual engagement.\u0000Our findings show high engagement from all dyads, leading to increased\u0000frequency of conversation and turn-taking. AACessTalk also encouraged parents\u0000to explore their own interaction strategies and empowered the children to have\u0000more agency in communication. We discuss the implications of designing\u0000technologies for balanced communication dynamics in parent-MVA child\u0000interaction.","PeriodicalId":501541,"journal":{"name":"arXiv - CS - Human-Computer Interaction","volume":"20 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142252479","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Motor imagery (MI) classification is key for brain-computer interfaces (BCIs). Until recent years, numerous models had been proposed, ranging from classical algorithms like Common Spatial Pattern (CSP) to deep learning models such as convolutional neural networks (CNNs) and transformers. However, these models have shown limitations in areas such as generalizability, contextuality and scalability when it comes to effectively extracting the complex spatial-temporal information inherent in electroencephalography (EEG) signals. To address these limitations, we introduce Spatial-Temporal Mamba Network (STMambaNet), an innovative model leveraging the Mamba state space architecture, which excels in processing extended sequences with linear scalability. By incorporating spatial and temporal Mamba encoders, STMambaNet effectively captures the intricate dynamics in both space and time, significantly enhancing the decoding performance of EEG signals for MI classification. Experimental results on BCI Competition IV 2a and 2b datasets demonstrate STMambaNet's superiority over existing models, establishing it as a powerful tool for advancing MI-based BCIs and improving real-world BCI systems.
运动图像（MI）分类是脑机接口（BCI）的关键。近年来，从通用空间模式（CSP）等经典算法到卷积神经网络（CNN）和变换器等深度学习模型，人们已经提出了许多模型。为了解决这些局限性，我们引入了空间-时间曼巴网络（STMambaNet），这是一种利用曼巴状态空间架构的创新模型，在处理扩展序列时具有出色的线性可扩展性。通过结合空间和时间曼巴编码器，STMambaNet 有效地捕捉了空间和时间的复杂动态，大大提高了用于 MI 分类的脑电信号的解码性能。在 BCI Competition IV 2a 和 2b 数据集上的实验结果证明了 STMambaNet 优于现有模型，使其成为推进基于 MI 的 BCI 和改进实际 BCI 系统的有力工具。
{"title":"Spatial-Temporal Mamba Network for EEG-based Motor Imagery Classification","authors":"Xiaoxiao Yang, Ziyu Jia","doi":"arxiv-2409.09627","DOIUrl":"https://doi.org/arxiv-2409.09627","url":null,"abstract":"Motor imagery (MI) classification is key for brain-computer interfaces\u0000(BCIs). Until recent years, numerous models had been proposed, ranging from\u0000classical algorithms like Common Spatial Pattern (CSP) to deep learning models\u0000such as convolutional neural networks (CNNs) and transformers. However, these\u0000models have shown limitations in areas such as generalizability, contextuality\u0000and scalability when it comes to effectively extracting the complex\u0000spatial-temporal information inherent in electroencephalography (EEG) signals.\u0000To address these limitations, we introduce Spatial-Temporal Mamba Network\u0000(STMambaNet), an innovative model leveraging the Mamba state space\u0000architecture, which excels in processing extended sequences with linear\u0000scalability. By incorporating spatial and temporal Mamba encoders, STMambaNet\u0000effectively captures the intricate dynamics in both space and time,\u0000significantly enhancing the decoding performance of EEG signals for MI\u0000classification. Experimental results on BCI Competition IV 2a and 2b datasets\u0000demonstrate STMambaNet's superiority over existing models, establishing it as a\u0000powerful tool for advancing MI-based BCIs and improving real-world BCI systems.","PeriodicalId":501541,"journal":{"name":"arXiv - CS - Human-Computer Interaction","volume":"9 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142252480","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：