Fei Wang, Yizhe Lv, Mengdie Zhu, Han Ding, Jinsong Han
Radio frequency (RF) devices such as Wi-Fi transceivers, radio frequency identification tags, and millimeter-wave radars have appeared in large numbers in daily lives. The presence and movement of humans can affect the propagation of RF signals, further, this phenomenon is exploited for human action recognition. Compared to camera solutions, RF approaches exhibit greater resilience to occlusions and lighting conditions, while also raising fewer privacy concerns in indoor scenarios. However, current works have many limitations, including the unavailability of datasets, insufficient training samples, and simple or limited action categories for specific applications, which seriously hinder the growth of RF solutions, presenting a significant obstacle in transitioning RF sensing research from the laboratory to a wide range of everyday life applications. To facilitate the transitioning, in this paper, we introduce and release a large-scale multiple radio frequency dataset, named XRF55, for indoor human action analysis. XRF55 encompasses 42.9K RF samples and 55 action classes of human-object interactions, human-human interactions, fitness, body motions, and human-computer interactions, collected from 39 subjects within 100 days. These actions were meticulously selected from 19 RF sensing papers and 16 video action recognition datasets. Each action is chosen to support various applications with high practical value, such as elderly fall detection, fatigue monitoring, domestic violence detection, etc. Moreover, XRF55 contains 23 RFID tags at 922.38MHz, 9 Wi-Fi links at 5.64GHz, one mmWave radar at 60-64GHz, and one Azure Kinect with RGB+D+IR sensors, covering frequency across decimeter wave, centimeter wave, and millimeter wave. In addition, we apply a mutual learning strategy over XRF55 for the task of action recognition. Unlike simple modality fusion, under mutual learning, three RF modalities are trained collaboratively and then work solely. We find these three RF modalities will promote each other. It is worth mentioning that, with synchronized Kinect, XRF55 also supports the exploration of action detection, action segmentation, pose estimation, human parsing, mesh reconstruction, etc., with RF-only or RF-Vision approaches.
{"title":"XRF55","authors":"Fei Wang, Yizhe Lv, Mengdie Zhu, Han Ding, Jinsong Han","doi":"10.1145/3643543","DOIUrl":"https://doi.org/10.1145/3643543","url":null,"abstract":"Radio frequency (RF) devices such as Wi-Fi transceivers, radio frequency identification tags, and millimeter-wave radars have appeared in large numbers in daily lives. The presence and movement of humans can affect the propagation of RF signals, further, this phenomenon is exploited for human action recognition. Compared to camera solutions, RF approaches exhibit greater resilience to occlusions and lighting conditions, while also raising fewer privacy concerns in indoor scenarios. However, current works have many limitations, including the unavailability of datasets, insufficient training samples, and simple or limited action categories for specific applications, which seriously hinder the growth of RF solutions, presenting a significant obstacle in transitioning RF sensing research from the laboratory to a wide range of everyday life applications. To facilitate the transitioning, in this paper, we introduce and release a large-scale multiple radio frequency dataset, named XRF55, for indoor human action analysis. XRF55 encompasses 42.9K RF samples and 55 action classes of human-object interactions, human-human interactions, fitness, body motions, and human-computer interactions, collected from 39 subjects within 100 days. These actions were meticulously selected from 19 RF sensing papers and 16 video action recognition datasets. Each action is chosen to support various applications with high practical value, such as elderly fall detection, fatigue monitoring, domestic violence detection, etc. Moreover, XRF55 contains 23 RFID tags at 922.38MHz, 9 Wi-Fi links at 5.64GHz, one mmWave radar at 60-64GHz, and one Azure Kinect with RGB+D+IR sensors, covering frequency across decimeter wave, centimeter wave, and millimeter wave. In addition, we apply a mutual learning strategy over XRF55 for the task of action recognition. Unlike simple modality fusion, under mutual learning, three RF modalities are trained collaboratively and then work solely. We find these three RF modalities will promote each other. It is worth mentioning that, with synchronized Kinect, XRF55 also supports the exploration of action detection, action segmentation, pose estimation, human parsing, mesh reconstruction, etc., with RF-only or RF-Vision approaches.","PeriodicalId":20553,"journal":{"name":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140077860","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Maryam Khalid, E. Klerman, A. McHill, A. Phillips, Akane Sano
Sleep behavior significantly impacts health and acts as an indicator of physical and mental well-being. Monitoring and predicting sleep behavior with ubiquitous sensors may therefore assist in both sleep management and tracking of related health conditions. While sleep behavior depends on, and is reflected in the physiology of a person, it is also impacted by external factors such as digital media usage, social network contagion, and the surrounding weather. In this work, we propose SleepNet, a system that exploits social contagion in sleep behavior through graph networks and integrates it with physiological and phone data extracted from ubiquitous mobile and wearable devices for predicting next-day sleep labels about sleep duration. Our architecture overcomes the limitations of large-scale graphs containing connections irrelevant to sleep behavior by devising an attention mechanism. The extensive experimental evaluation highlights the improvement provided by incorporating social networks in the model. Additionally, we conduct robustness analysis to demonstrate the system's performance in real-life conditions. The outcomes affirm the stability of SleepNet against perturbations in input data. Further analyses emphasize the significance of network topology in prediction performance revealing that users with higher eigenvalue centrality are more vulnerable to data perturbations.
{"title":"SleepNet","authors":"Maryam Khalid, E. Klerman, A. McHill, A. Phillips, Akane Sano","doi":"10.1145/3643508","DOIUrl":"https://doi.org/10.1145/3643508","url":null,"abstract":"Sleep behavior significantly impacts health and acts as an indicator of physical and mental well-being. Monitoring and predicting sleep behavior with ubiquitous sensors may therefore assist in both sleep management and tracking of related health conditions. While sleep behavior depends on, and is reflected in the physiology of a person, it is also impacted by external factors such as digital media usage, social network contagion, and the surrounding weather. In this work, we propose SleepNet, a system that exploits social contagion in sleep behavior through graph networks and integrates it with physiological and phone data extracted from ubiquitous mobile and wearable devices for predicting next-day sleep labels about sleep duration. Our architecture overcomes the limitations of large-scale graphs containing connections irrelevant to sleep behavior by devising an attention mechanism. The extensive experimental evaluation highlights the improvement provided by incorporating social networks in the model. Additionally, we conduct robustness analysis to demonstrate the system's performance in real-life conditions. The outcomes affirm the stability of SleepNet against perturbations in input data. Further analyses emphasize the significance of network topology in prediction performance revealing that users with higher eigenvalue centrality are more vulnerable to data perturbations.","PeriodicalId":20553,"journal":{"name":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140077951","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Audio-based human activity recognition (HAR) is very popular because many human activities have unique sound signatures that can be detected using machine learning (ML) approaches. These audio-based ML HAR pipelines often use common featurization techniques, such as extracting various statistical and spectral features by converting time domain signals to the frequency domain (using an FFT) and using them to train ML models. Some of these approaches also claim privacy benefits by preventing the identification of human speech. However, recent deep learning-based automatic speech recognition (ASR) models pose new privacy challenges to these featurization techniques. In this paper, we systematically evaluate various featurization approaches for audio data, assessing their privacy risks through metrics like speech intelligibility (PER and WER) while considering the utility tradeoff in terms of ML-based activity recognition accuracy. Our findings reveal the susceptibility of these approaches to speech content recovery when exposed to recent ASR models, especially under re-tuning or retraining conditions. Notably, fine-tuned ASR models achieved an average Phoneme Error Rate (PER) of 39.99% and Word Error Rate (WER) of 44.43% in speech recognition for these approaches. To overcome these privacy concerns, we propose Kirigami, a lightweight machine learning-based audio speech filter that removes human speech segments reducing the efficacy of ASR models (70.48% PER and 101.40% WER) while also maintaining HAR accuracy (76.0% accuracy). We show that Kirigami can be implemented on common edge microcontrollers with limited computational capabilities and memory, providing a path to deployment on a variety of IoT devices. Finally, we conducted a real-world user study and showed the robustness of Kirigami on a laptop and an ARM Cortex-M4F microcontroller under three different background noises.
基于音频的人类活动识别（HAR）非常流行，因为许多人类活动都有独特的声音特征，可以使用机器学习（ML）方法进行检测。这些基于音频的 ML HAR 管道通常使用常见的特征化技术，例如通过将时域信号转换到频域（使用 FFT）来提取各种统计和频谱特征，并将其用于训练 ML 模型。其中一些方法还声称可以通过防止识别人类语音来保护隐私。然而，最近基于深度学习的自动语音识别（ASR）模型给这些特征化技术带来了新的隐私挑战。在本文中，我们系统地评估了音频数据的各种特征化方法，通过语音清晰度（PER 和 WER）等指标评估了它们的隐私风险，同时考虑了基于 ML 的活动识别准确率方面的效用权衡。我们的研究结果表明，当这些方法暴露在最新的 ASR 模型中时，特别是在重新调整或重新训练的条件下，很容易出现语音内容恢复问题。值得注意的是，经过微调的 ASR 模型在这些方法的语音识别中平均达到了 39.99% 的音素错误率（PER）和 44.43% 的单词错误率（WER）。为了克服这些隐私问题，我们提出了基于机器学习的轻量级音频语音过滤器 Kirigami，它可以去除人类语音片段，降低 ASR 模型的效率（PER 为 70.48%，WER 为 101.40%），同时还能保持 HAR 准确率（76.0%）。我们的研究表明，Kirigami 可以在计算能力和内存有限的普通边缘微控制器上实现，为在各种物联网设备上部署提供了途径。最后，我们进行了一项实际用户研究，在笔记本电脑和 ARM Cortex-M4F 微控制器上展示了 Kirigami 在三种不同背景噪声下的鲁棒性。
{"title":"Kirigami","authors":"Sudershan Boovaraghavan, Haozhe Zhou, Mayank Goel, Yuvraj Agarwal","doi":"10.1145/3643502","DOIUrl":"https://doi.org/10.1145/3643502","url":null,"abstract":"Audio-based human activity recognition (HAR) is very popular because many human activities have unique sound signatures that can be detected using machine learning (ML) approaches. These audio-based ML HAR pipelines often use common featurization techniques, such as extracting various statistical and spectral features by converting time domain signals to the frequency domain (using an FFT) and using them to train ML models. Some of these approaches also claim privacy benefits by preventing the identification of human speech. However, recent deep learning-based automatic speech recognition (ASR) models pose new privacy challenges to these featurization techniques. In this paper, we systematically evaluate various featurization approaches for audio data, assessing their privacy risks through metrics like speech intelligibility (PER and WER) while considering the utility tradeoff in terms of ML-based activity recognition accuracy. Our findings reveal the susceptibility of these approaches to speech content recovery when exposed to recent ASR models, especially under re-tuning or retraining conditions. Notably, fine-tuned ASR models achieved an average Phoneme Error Rate (PER) of 39.99% and Word Error Rate (WER) of 44.43% in speech recognition for these approaches. To overcome these privacy concerns, we propose Kirigami, a lightweight machine learning-based audio speech filter that removes human speech segments reducing the efficacy of ASR models (70.48% PER and 101.40% WER) while also maintaining HAR accuracy (76.0% accuracy). We show that Kirigami can be implemented on common edge microcontrollers with limited computational capabilities and memory, providing a path to deployment on a variety of IoT devices. Finally, we conducted a real-world user study and showed the robustness of Kirigami on a laptop and an ARM Cortex-M4F microcontroller under three different background noises.","PeriodicalId":20553,"journal":{"name":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140078110","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
User authentication on smartphones needs to balance both security and convenience. Many image-based face authentication methods are vulnerable to spoofing and are plagued by privacy breaches, so models based on acoustic sensing have emerged to achieve reliable user authentication. However, they can only achieve reasonable performance under specific conditions (i.e., a fixed range), and they can not resist 3D printing attacks. To address these limitations, we present a novel user authentication system, referred to as AFace. The system mainly consists of two parts: an iso-depth model and a range-adaptive (RA) algorithm. The iso-depth model establishes a connection between acoustic echoes and facial structures, while taking into account the influence of biological materials on echo energy, making it resistant to 3D printing attacks (as it's difficult to replicate material information in 3D printing). RA algorithm can adaptively compensate for the distance between the user and the smartphone, enabling flexible authentication modes. Results from experiments with 40 volunteers demonstrate that AFace achieves an average accuracy of 96.9% and an F1 score of 96.9%, and no image/video-based attack is observed to succeed in spoofing.
智能手机上的用户身份验证需要兼顾安全性和便利性。许多基于图像的人脸身份验证方法容易被欺骗，而且存在隐私泄露问题，因此出现了基于声学传感的模型，以实现可靠的用户身份验证。然而，它们只能在特定条件下（如固定范围）实现合理的性能，而且无法抵御 3D 打印攻击。针对这些局限性，我们提出了一种新型用户身份验证系统，称为 AFace。该系统主要由两部分组成：等深模型和范围自适应（RA）算法。等深线模型建立了声学回声和面部结构之间的联系，同时考虑了生物材料对回声能量的影响，使其能够抵御 3D 打印攻击（因为在 3D 打印中很难复制材料信息）。RA 算法可以自适应地补偿用户与智能手机之间的距离，从而实现灵活的认证模式。40 名志愿者的实验结果表明，AFace 的平均准确率达到 96.9%，F1 分数达到 96.9%，没有观察到基于图像/视频的攻击能成功欺骗。
{"title":"AFace","authors":"Zhaopeng Xu, Tong Liu, Ruobing Jiang, Pengfei Hu, Zhongwen Guo, Chao Liu","doi":"10.1145/3643510","DOIUrl":"https://doi.org/10.1145/3643510","url":null,"abstract":"User authentication on smartphones needs to balance both security and convenience. Many image-based face authentication methods are vulnerable to spoofing and are plagued by privacy breaches, so models based on acoustic sensing have emerged to achieve reliable user authentication. However, they can only achieve reasonable performance under specific conditions (i.e., a fixed range), and they can not resist 3D printing attacks. To address these limitations, we present a novel user authentication system, referred to as AFace. The system mainly consists of two parts: an iso-depth model and a range-adaptive (RA) algorithm. The iso-depth model establishes a connection between acoustic echoes and facial structures, while taking into account the influence of biological materials on echo energy, making it resistant to 3D printing attacks (as it's difficult to replicate material information in 3D printing). RA algorithm can adaptively compensate for the distance between the user and the smartphone, enabling flexible authentication modes. Results from experiments with 40 volunteers demonstrate that AFace achieves an average accuracy of 96.9% and an F1 score of 96.9%, and no image/video-based attack is observed to succeed in spoofing.","PeriodicalId":20553,"journal":{"name":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140078433","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sungjin Hwang, Jiwoong Heo, Youngwug Cho, Jucheol Moon, Yushin Lee, Hansung Kim, Jaehyuk Cha, K. Kim
Transportation mode detection (TMD) has been proposed as a computational technology to obtain mobility information. However, previous TMD studies mainly focused on improving detection performance and have not investigated the social implications of mobility information. This is the first study to use TMD to predict the life satisfaction of wheelchair users. Our goal is to develop TMD for wheelchair users (wTMD) utilizing smartphone location data and apply it to determine how transportation behaviors affect the life satisfaction of wheelchair users. First, we propose a wTMD technology by collecting a new dataset from wheelchair and non-wheelchair users. Second, we conduct regression analyses on existing in-the-wild dataset of wheelchair users. The result shows that the portion of subways in an individual's travel time is directly connected to wheelchair users' life satisfaction in Seoul, South Korea. We hope our findings are a good example for future social science studies and ultimately help to design wheelchair-friendly urban planning and accessibility.
{"title":"Transportation Mode Detection Technology to Predict Wheelchair Users' Life Satisfaction in Seoul, South Korea","authors":"Sungjin Hwang, Jiwoong Heo, Youngwug Cho, Jucheol Moon, Yushin Lee, Hansung Kim, Jaehyuk Cha, K. Kim","doi":"10.1145/3643506","DOIUrl":"https://doi.org/10.1145/3643506","url":null,"abstract":"Transportation mode detection (TMD) has been proposed as a computational technology to obtain mobility information. However, previous TMD studies mainly focused on improving detection performance and have not investigated the social implications of mobility information. This is the first study to use TMD to predict the life satisfaction of wheelchair users. Our goal is to develop TMD for wheelchair users (wTMD) utilizing smartphone location data and apply it to determine how transportation behaviors affect the life satisfaction of wheelchair users. First, we propose a wTMD technology by collecting a new dataset from wheelchair and non-wheelchair users. Second, we conduct regression analyses on existing in-the-wild dataset of wheelchair users. The result shows that the portion of subways in an individual's travel time is directly connected to wheelchair users' life satisfaction in Seoul, South Korea. We hope our findings are a good example for future social science studies and ultimately help to design wheelchair-friendly urban planning and accessibility.","PeriodicalId":20553,"journal":{"name":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140078163","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sleep posture plays a crucial role in maintaining good morpheus quality and overall health. As a result, long-term monitoring of 3D sleep postures is significant for sleep analysis and chronic disease prevention. To recognize sleep postures, traditional methods either use cameras to record image data or require the user to wear wearable devices or sleep on pressure mattresses. However, these methods could raise privacy concerns and cause discomfort during sleep. Accordingly, the RF (Radio Frequency) based method has emerged as a promising alternative. Despite most of these methods achieving high precision in classifying sleep postures, they struggle to retrieve 3D sleep postures due to difficulties in capturing 3D positions of static body joints. In this work, we propose TagSleep3D to resolve all the above issues. Specifically, inspired by the concept of RFID tag sheets, we explore the possibility of recognizing 3D sleep posture by deploying an RFID tag array under the bedsheet. When a user sleeps in bed, the signals of some tags could be blocked or reflected by the sleep posture, which can produce a body imprint. We then propose a novel deep learning model composed of the attention mechanism, convolutional neural network, and together with two data augmentation methods to retrieve the 3D sleep postures by analyzing these body imprints. We evaluate TagSleep3D with 43 users and we totally collect 27,300 sleep posture samples. Our extensive experiments demonstrate that TagSleep3D can recognize each joint on the human skeleton with a median MPJPE (Mean Per Joint Position Error) of 4.76 cm for seen users and 7.58 cm for unseen users.
睡眠姿势对保持良好的睡眠质量和整体健康起着至关重要的作用。因此，长期监测三维睡眠姿势对于睡眠分析和慢性疾病预防具有重要意义。要识别睡眠姿势，传统方法要么使用摄像头记录图像数据，要么要求用户佩戴可穿戴设备或睡在压力床垫上。然而，这些方法可能会引起隐私方面的担忧，并在睡眠过程中造成不适。因此，基于射频（RF）的方法成为一种有前途的替代方法。尽管这些方法大多能高精度地对睡眠姿势进行分类，但由于难以捕捉静态身体关节的三维位置，它们在检索三维睡眠姿势方面却举步维艰。在这项工作中，我们提出 TagSleep3D 来解决上述所有问题。具体来说，受 RFID 标签床单概念的启发，我们探索了通过在床单下部署 RFID 标签阵列来识别三维睡眠姿势的可能性。当用户在床上睡觉时，一些标签的信号可能会被睡姿阻挡或反射，从而产生身体印记。我们随后提出了一种由注意力机制和卷积神经网络组成的新型深度学习模型，并结合两种数据增强方法，通过分析这些身体印记来检索三维睡眠姿势。我们通过 43 位用户对 TagSleep3D 进行了评估，共收集了 27,300 个睡眠姿势样本。广泛的实验证明，TagSleep3D 可以识别人体骨骼上的每个关节，对可见用户的中位数 MPJPE（平均每个关节位置误差）为 4.76 厘米，对未可见用户的中位数 MPJPE（平均每个关节位置误差）为 7.58 厘米。
{"title":"TagSleep3D","authors":"Chen Liu, Zixuan Dong, Li Huang, Wenlong Yan, Xin Wang, Dingyi Fang, Xiaojiang Chen","doi":"10.1145/3643512","DOIUrl":"https://doi.org/10.1145/3643512","url":null,"abstract":"Sleep posture plays a crucial role in maintaining good morpheus quality and overall health. As a result, long-term monitoring of 3D sleep postures is significant for sleep analysis and chronic disease prevention. To recognize sleep postures, traditional methods either use cameras to record image data or require the user to wear wearable devices or sleep on pressure mattresses. However, these methods could raise privacy concerns and cause discomfort during sleep. Accordingly, the RF (Radio Frequency) based method has emerged as a promising alternative. Despite most of these methods achieving high precision in classifying sleep postures, they struggle to retrieve 3D sleep postures due to difficulties in capturing 3D positions of static body joints. In this work, we propose TagSleep3D to resolve all the above issues. Specifically, inspired by the concept of RFID tag sheets, we explore the possibility of recognizing 3D sleep posture by deploying an RFID tag array under the bedsheet. When a user sleeps in bed, the signals of some tags could be blocked or reflected by the sleep posture, which can produce a body imprint. We then propose a novel deep learning model composed of the attention mechanism, convolutional neural network, and together with two data augmentation methods to retrieve the 3D sleep postures by analyzing these body imprints. We evaluate TagSleep3D with 43 users and we totally collect 27,300 sleep posture samples. Our extensive experiments demonstrate that TagSleep3D can recognize each joint on the human skeleton with a median MPJPE (Mean Per Joint Position Error) of 4.76 cm for seen users and 7.58 cm for unseen users.","PeriodicalId":20553,"journal":{"name":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140078186","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Familiarity significantly impacts public willingness to try new foods, making novel foods unlikely to be consumed. However, exposing eaters to various sensory modalities---encompassing vision, smell, sound, and touch---prior to eating may enhance their willingness to try these foods. Although multisensory methodologies have already been developed to transmit the sensory cues of food digitally, existing technology is limited to replication, and novel technologies remain relatively scarce in traditional food practices. Through a sensory design activity, multisensory interactions were incorporated into a self-ordering kiosk used for daily food selection. Subsequently, we conducted an experiment to observe how participants perceived a novel food on a multisensory interface, and found them to exhibit a significantly increased willingness to try the food regardless of level of fear. We also observed the multisensory interface to yield statistically significant results in food familiarity and overall satisfaction. These findings suggest that technology that integrates sensory exposure into daily life can effectively educate and familiarize people with novel food products.
{"title":"A Multi-sensory Kiosk Interface to Familiarize Users with New Foods","authors":"Eunsol Sol Choi, Younah Kang","doi":"10.1145/3643545","DOIUrl":"https://doi.org/10.1145/3643545","url":null,"abstract":"Familiarity significantly impacts public willingness to try new foods, making novel foods unlikely to be consumed. However, exposing eaters to various sensory modalities---encompassing vision, smell, sound, and touch---prior to eating may enhance their willingness to try these foods. Although multisensory methodologies have already been developed to transmit the sensory cues of food digitally, existing technology is limited to replication, and novel technologies remain relatively scarce in traditional food practices. Through a sensory design activity, multisensory interactions were incorporated into a self-ordering kiosk used for daily food selection. Subsequently, we conducted an experiment to observe how participants perceived a novel food on a multisensory interface, and found them to exhibit a significantly increased willingness to try the food regardless of level of fear. We also observed the multisensory interface to yield statistically significant results in food familiarity and overall satisfaction. These findings suggest that technology that integrates sensory exposure into daily life can effectively educate and familiarize people with novel food products.","PeriodicalId":20553,"journal":{"name":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140078438","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A long-term perspective toward the future enables a more comprehensive approach to decision-making, considering a variety of potential scenarios. The forecasting of mental health was anticipated to promote proactive planning, however, it faces challenges such as a short forecasting period and a lack of intuitive understanding of the relationship between actions and the forecast. This study presents a novel mental health indicator that incorporates a long-term perspective by considering past actions. A four-week experiment was conducted with 105 participants to evaluate the effects of a one-week forecast. Qualitative analysis reveals the effects of the one-week forecast on behavioral planning, emotional states, and reasons for disregarding the forecasts. Findings indicate that conventional mood indicators prompt participants to prioritize pre-existing schedules and perceive the forecast as infeasible, whereas the proposed indicator enhances the ability to plan work schedules in advance. Our results offer valuable insights into the presentation of forecasts for effectively managing mental health, considering the time constraints of everyday life.
{"title":"Planning the Future in a Longer Perspective","authors":"Naoki Tateyama, Ryota Yokomura, Yuki Ban, Shin'ichi Warisawa, Rui Fukui","doi":"10.1145/3643538","DOIUrl":"https://doi.org/10.1145/3643538","url":null,"abstract":"A long-term perspective toward the future enables a more comprehensive approach to decision-making, considering a variety of potential scenarios. The forecasting of mental health was anticipated to promote proactive planning, however, it faces challenges such as a short forecasting period and a lack of intuitive understanding of the relationship between actions and the forecast. This study presents a novel mental health indicator that incorporates a long-term perspective by considering past actions. A four-week experiment was conducted with 105 participants to evaluate the effects of a one-week forecast. Qualitative analysis reveals the effects of the one-week forecast on behavioral planning, emotional states, and reasons for disregarding the forecasts. Findings indicate that conventional mood indicators prompt participants to prioritize pre-existing schedules and perceive the forecast as infeasible, whereas the proposed indicator enhances the ability to plan work schedules in advance. Our results offer valuable insights into the presentation of forecasts for effectively managing mental health, considering the time constraints of everyday life.","PeriodicalId":20553,"journal":{"name":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140078475","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Biyun Sheng, Rui Han, Fu Xiao, Zhengxin Guo, Linqing Gui
WiFi based action recognition has attracted increasing attentions due to its convenience and universality in real-world applications, whereas the domain dependency leads to poor generalization ability towards new sensing environments or subjects. The majority of existing solutions fail to sufficiently extract action-related features from WiFi signals. Moreover, they are unable to make full use of the target data with only the labelled samples taken into consideration. To cope with these issues, we propose a WiFi-based sensing system, MetaFormer, which can effectively recognize actions from unseen domains with only one labelled target sample per category. Specifically, MetaFormer achieves this by firstly constructing a novel spatial-temporal transformer feature extraction structure with dense-sparse input named DS-STT to capture action primary and affiliated movements. It then designs Meta-teacher framework which meta-pre-trains source tasks and updates model parameters by dynamic pseudo label enhancement to bridge the relationship among the labelled and unlabelled target samples. In order to validate the performance of MetaFormer, we conduct comprehensive evaluations on SignFi, Widar and Wiar datasets and achieve superior performances under the one-shot case.
{"title":"MetaFormer","authors":"Biyun Sheng, Rui Han, Fu Xiao, Zhengxin Guo, Linqing Gui","doi":"10.1145/3643550","DOIUrl":"https://doi.org/10.1145/3643550","url":null,"abstract":"WiFi based action recognition has attracted increasing attentions due to its convenience and universality in real-world applications, whereas the domain dependency leads to poor generalization ability towards new sensing environments or subjects. The majority of existing solutions fail to sufficiently extract action-related features from WiFi signals. Moreover, they are unable to make full use of the target data with only the labelled samples taken into consideration. To cope with these issues, we propose a WiFi-based sensing system, MetaFormer, which can effectively recognize actions from unseen domains with only one labelled target sample per category. Specifically, MetaFormer achieves this by firstly constructing a novel spatial-temporal transformer feature extraction structure with dense-sparse input named DS-STT to capture action primary and affiliated movements. It then designs Meta-teacher framework which meta-pre-trains source tasks and updates model parameters by dynamic pseudo label enhancement to bridge the relationship among the labelled and unlabelled target samples. In order to validate the performance of MetaFormer, we conduct comprehensive evaluations on SignFi, Widar and Wiar datasets and achieve superior performances under the one-shot case.","PeriodicalId":20553,"journal":{"name":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140078565","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Rongrong Wang, Rui Tan, Zhenyu Yan, Chris Xiaoxuan Lu
Identifying new sensing modalities for indoor localization is an interest of research. This paper studies powerline-induced alternating magnetic field (AMF) that fills the indoor space for the orientation-aware three-dimensional (3D) simultaneous localization and mapping (SLAM). While an existing study has adopted a uniaxial AMF sensor for SLAM in a plane surface, the design falls short of addressing the vector field nature of AMF and is therefore susceptible to sensor orientation variations. Moreover, although the higher spatial variability of AMF in comparison with indoor geomagnetism promotes location sensing resolution, extra SLAM algorithm designs are needed to achieve robustness to trajectory deviations from the constructed map. To address the above issues, we design a new triaxial AMF sensor and a new SLAM algorithm that constructs a 3D AMF intensity map regularized and augmented by a Gaussian process. The triaxial sensor's orientation estimation is free of the error accumulation problem faced by inertial sensing. From extensive evaluation in eight indoor environments, our AMF-based 3D SLAM achieves sub-1m to 3m median localization errors in spaces of up to 500 m2, sub-2° mean error in orientation sensing, and outperforms the SLAM systems based on Wi-Fi, geomagnetism, and uniaxial AMF by more than 30%.
为室内定位确定新的传感模式是一项令人感兴趣的研究。本文研究了电力线引起的交变磁场（AMF），该磁场可填充室内空间，用于方位感知三维（3D）同步定位和绘图（SLAM）。虽然现有研究采用单轴交变磁场传感器在平面上进行 SLAM，但该设计未能解决交变磁场的矢量场特性，因此容易受到传感器方向变化的影响。此外，虽然 AMF 与室内地磁相比具有更高的空间可变性，可提高位置传感分辨率，但仍需要额外的 SLAM 算法设计，以实现轨迹偏离构建地图的鲁棒性。为解决上述问题，我们设计了一种新的三轴 AMF 传感器和一种新的 SLAM 算法，该算法可构建由高斯过程正则化和增强的三维 AMF 强度图。三轴传感器的方位估计不存在惯性传感所面临的误差累积问题。通过在八个室内环境中进行广泛评估，我们基于 AMF 的三维 SLAM 在面积达 500 平方米的空间中实现了低于 1 米至 3 米的中值定位误差，在方位感应中实现了低于 2° 的平均误差，比基于 Wi-Fi、地磁和单轴 AMF 的 SLAM 系统优胜 30% 以上。
{"title":"Orientation-Aware 3D SLAM in Alternating Magnetic Field from Powerlines","authors":"Rongrong Wang, Rui Tan, Zhenyu Yan, Chris Xiaoxuan Lu","doi":"10.1145/3631446","DOIUrl":"https://doi.org/10.1145/3631446","url":null,"abstract":"Identifying new sensing modalities for indoor localization is an interest of research. This paper studies powerline-induced alternating magnetic field (AMF) that fills the indoor space for the orientation-aware three-dimensional (3D) simultaneous localization and mapping (SLAM). While an existing study has adopted a uniaxial AMF sensor for SLAM in a plane surface, the design falls short of addressing the vector field nature of AMF and is therefore susceptible to sensor orientation variations. Moreover, although the higher spatial variability of AMF in comparison with indoor geomagnetism promotes location sensing resolution, extra SLAM algorithm designs are needed to achieve robustness to trajectory deviations from the constructed map. To address the above issues, we design a new triaxial AMF sensor and a new SLAM algorithm that constructs a 3D AMF intensity map regularized and augmented by a Gaussian process. The triaxial sensor's orientation estimation is free of the error accumulation problem faced by inertial sensing. From extensive evaluation in eight indoor environments, our AMF-based 3D SLAM achieves sub-1m to 3m median localization errors in spaces of up to 500 m2, sub-2° mean error in orientation sensing, and outperforms the SLAM systems based on Wi-Fi, geomagnetism, and uniaxial AMF by more than 30%.","PeriodicalId":20553,"journal":{"name":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-01-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139437305","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：