Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies最新文献

Measuring activities and postures is an important area of research in ubiquitous computing, human-computer interaction, and personal health informatics. One approach that researchers use to collect large amounts of labeled data to develop models for activity recognition and measurement is asking participants to self-report their daily activities. Although participants can typically recall their sequence of daily activities, remembering the precise start and end times of each activity is significantly more challenging. ACAI is a novel, context-assisted ACtivity Annotation Interface that enables participants to efficiently label their activities by accepting or adjusting system-generated activity suggestions while explicitly expressing uncertainty about temporal boundaries. We evaluated ACAI using two complementary studies: a usability study with 11 participants and a two-week, free-living study with 14 participants. We compared our activity annotation system with the current gold-standard methods for activity recall in health sciences research: 24PAR and its computerized version, ACT24. Our system reduced annotation time and perceived effort while significantly improving data validity and fidelity compared to both standard human-supervised and unsupervised activity recall approaches. We discuss the limitations of our design and implications for developing adaptive, human-in-the-loop activity recognition systems used to collect self-report data on activity.

Many breast cancer survivors are prescribed daily oral medications called endocrine therapy that prevent cancer recurrence. Despite its clinical importance, maintaining consistent daily adherence remains challenging due to the dynamic and interrelated influences of behavioral, physiological, and psychological factors. While prior studies have explored adherence prediction using mobile sensing, they often rely on single-modality data, limited temporal granularity, or aggregate-level modeling-limiting their ability to capture short and long-term behavioral variability and to facilitate deeper understanding of non-adherence and tailored interventions. To address these gaps, we propose a multimodal sensing framework that explicitly models daily adherence dynamics using temporally adaptive inputs. We recruited a sample of breast cancer survivors (N = 20) and collected longitudinal data streams including wearable-derived physiological features (Fitbit), medication event monitoring system (MEMS) data, and ecological momentary assessments (EMAs). Using multimodal data across varying time windows, we examined whether recent patterns in behavioral, physiological, psychological, and environmental factors improve the prediction of next-day endocrine therapy adherence. Our results demonstrate the feasibility of using multimodal sensing data to predict daily adherence with moderate accuracy. Moreover, models integrating multimodal data consistently outperformed those relying on a single modality. Importantly, we observed that the predictive value of each modality varied depending on the temporal proximity of the input signals, underscoring the importance of modeling immediate and longer-term behavioral patterns. The findings offer valuable insights for advancing adherence monitoring systems, suggesting that incorporating personalized and temporally adaptive data fusion strategies may significantly enhance the effectiveness of intervention design and delivery.

Microinteraction ecological momentary assessment (μEMA) is a type of EMA that uses single-question prompts on a smartwatch to collect real-world self-reports. Smaller-scale studies show that μEMA yields higher response rates than EMA for up to 4 weeks. In this paper, we evaluated μEMA's longitudinal engagement in a 12-month study. Each participant completed EMA surveys (one smartphone prompt/hour for 96 days in 4-day bursts) and μEMA surveys (four smartwatch prompts/hour for the 270 days). Using data from 177 participants ( 1.37 million μEMA and 14.9K EMA surveys), we compared engagement across three groups: those who completed 12 months of EMA data collection(Completed), those who voluntarily withdrew after six months of EMA data collection (Withdrew), and those unenrolled by staff after six months of poor EMA response rates (Unenrolled). Compared to EMA, unenrolled participants were 2.25 times, those who withdrew were 1.65 times, and completed participants were 1.53 times more likely to answer μEMA prompts (p < 0.001). Regardless of response rates, μEMA was perceived as less burdensome than EMA (p < 0.001). These results suggest μEMA is a viable method for intensive longitudinal data collection, particularly for participants who find EMA unsustainable.

Independent navigation remains a significant challenge for blind and low vision individuals, especially in unfamiliar environments. In this paper, we introduce the Parsimonious Instructions design principle, which aims to enhance navigation safety while minimizing the number of instructions delivered to the user. We demonstrate the application of this principle through NavGraph, a navigation application adopting a modular architecture comprising four components: localization, routing, guidance, and user interface. NavGraph is designed to provide effective, non-intrusive navigation assistance by optimizing route computation and instruction delivery. We evaluated NavGraph in a user study with 10 blind participants, comparing it to a baseline solution. Results show that NavGraph significantly reduces the number of instructions and improves clarity and safety, without compromising navigation time. These findings support the potential of the Parsimonious Instructions design principle in assistive navigation technologies.

Ecological momentary assessment (EMA) is an approach to collect self-reported data repeatedly on mobile devices in natural settings. EMAs allow for temporally dense, ecologically valid data collection, but frequent interruptions with lengthy surveys on mobile devices can burden users, impacting compliance and data quality. We propose a method that reduces the length of each EMA question set measuring interrelated constructs, with only modest information loss. By estimating the potential information gain of each EMA question using question-answer prediction models, this method can prioritize the presentation of the most informative question in a question-by-question sequence and skip uninformative questions. We evaluated the proposed method by simulating question omission using four real-world datasets from three different EMA studies. When compared against the random question omission approach that skips 50% of the questions, our method reduces imputation errors by 15%-52%. In surveys with five answer options for each question, our method can reduce the mean survey length by 34%-56% with a real-time prediction accuracy of 72%-95% for the skipped questions. The proposed method may either allow more constructs to be surveyed without adding user burden or reduce response burden for more sustainable longitudinal EMA data collection.

Researchers in ubiquitous computing have long promised that passive sensing will revolutionize mental health measurement by detecting individuals in a population experiencing a mental health disorder or specific symptoms. Recent work suggests that detection tools do not generalize well when trained and tested in more heterogeneous samples. In this work, we contribute a narrative review and findings from two studies with 41 mental health clinicians to understand these generalization challenges. Our findings motivate research on actionable sensing, as an alternative to detection research, studying how passive sensing can augment traditional mental health measures to support actions in clinical care. Specifically, we identify how passive sensing can support clinical actions by revealing patients' presenting problems for treatment and identifying targets for behavior change and symptom reduction, but passive data requires additional contextual information to be appropriately interpreted and used in care. We conclude by suggesting research at the intersection of actionable sensing and mental healthcare, to align technical research in ubiquitous computing with clinical actions and needs.

Mental health concerns are prevalent among college students, highlighting the need for effective interventions that promote self-awareness and holistic well-being. MindScape explores a novel approach to AI-powered journaling by integrating passively collected behavioral patterns such as conversational engagement, sleep, and location with Large Language Models (LLMs). This integration creates a highly personalized and context-aware journaling experience, enhancing self-awareness and well-being by embedding behavioral intelligence into AI. We present an 8-week exploratory study with 20 college students, demonstrating the MindScape app's efficacy in enhancing positive affect (7%), reducing negative affect (11%), loneliness (6%), and anxiety and depression, with a significant week-over-week decrease in PHQ-4 scores (-0.25 coefficient). The study highlights the advantages of contextual AI journaling, with participants particularly appreciating the tailored prompts and insights provided by the MindScape app. Our analysis also includes a comparison of responses to AI-driven contextual versus generic prompts, participant feedback insights, and proposed strategies for leveraging contextual AI journaling to improve well-being on college campuses. By showcasing the potential of contextual AI journaling to support mental health, we provide a foundation for further investigation into the effects of contextual AI journaling on mental health and well-being.

In automated sleep monitoring systems, bed occupancy detection is the foundation or the first step before other downstream tasks, such as inferring sleep activities and vital signs. The existing methods do not generalize well to real-world environments due to single environment settings and rely on threshold-based approaches. Manually selecting thresholds requires observing a large amount of data and may not yield optimal results. In contrast, acquiring extensive labeled sensory data poses significant challenges regarding cost and time. Hence, developing models capable of generalizing across diverse environments with limited data is imperative. This paper introduces SeismoDot, which consists of a self-supervised learning module and a spectral-temporal feature fusion module for bed occupancy detection. Unlike conventional methods that require separate pre-training and fine-tuning, our self-supervised learning module is co-optimized with the primary target task, which directs learned representations toward a task-relevant embedding space while expanding the feature space. The proposed feature fusion module enables the simultaneous exploitation of temporal and spectral features, enhancing the diversity of information from both domains. By combining these techniques, SeismoDot expands the diversity of embedding space for both the temporal and spectral domains to enhance its generalizability across different environments. SeismoDot not only achieves high accuracy (98.49%) and F1 scores (98.08%) across 13 diverse environments, but it also maintains high performance (97.01% accuracy and 96.54% F1 score) even when trained with just 20% (4 days) of the total data. This demonstrates its exceptional ability to generalize across various environmental settings, even with limited data availability.

Wearable cameras provide an objective method to visually confirm and automate the detection of health-risk behaviors such as smoking and overeating, which is critical for developing and testing adaptive treatment interventions. Despite the potential of wearable camera systems, adoption is hindered by inadequate clinician input in the design, user privacy concerns, and user burden. To address these barriers, we introduced HabitSense, an open-source, multi-modal neck-worn platform developed with input from focus groups with clinicians (N=36) and user feedback from in-wild studies involving 105 participants over 35 days. Optimized for monitoring health-risk behaviors, the platform utilizes RGB, thermal, and inertial measurement unit sensors to detect eating and smoking events in real time. In a 7-day study involving 15 participants, HabitSense recorded 768 hours of footage, capturing 420.91 minutes of hand-to-mouth gestures associated with eating and smoking data crucial for training machine learning models, achieving a 92% F1-score in gesture recognition. To address privacy concerns, the platform records only during likely health-risk behavior events using SECURE, a smart activation algorithm. Additionally, HabitSense employs on-device obfuscation algorithms that selectively obfuscate the background during recording, maintaining individual privacy while leaving gestures related to health-risk behaviors unobfuscated. Our implementation of SECURE has resulted in a 48% reduction in storage needs and a 30% increase in battery life. This paper highlights the critical roles of clinician feedback, extensive field testing, and privacy-enhancing algorithms in developing an unobtrusive, lightweight, and reproducible wearable system that is both feasible and acceptable for monitoring health-risk behaviors in real-world settings.

μEMA is a data collection method that prompts research participants with quick, answer-at-a-glance, single-multiple-choice self-report behavioral questions, thus enabling high-temporal-density self-report of up to four times per hour when implemented on a smartwatch. However, due to the small watch screen, μEMA is better used to select among 2 to 5 multiple-choice answers versus allowing the collection of open-ended responses. We introduce an alternative and novel form of micro-interaction self-report using speech input - audio-μEMA- where a short beep or vibration cues participants to verbally report their behavioral states, allowing for open-ended, temporally dense self-reports. We conducted a one-hour usability study followed by a within-subject, 6-day to 21-day free-living feasibility study in which participants self-reported their physical activities and postures once every 2 to 5 minutes. We qualitatively explored the usability of the system and identified factors impacting the response rates of this data collection method. Despite being interrupted 12 to 20 times per hour, participants in the free-living study were highly engaged with the system, with an average response rate of 67.7% for audio-μEMA for up to 14 days. We discuss the factors that impacted feasibility; some implementation, methodological, and participant challenges we observed; and important considerations relevant to deploying audio-μEMA in real-time activity recognition systems.