Pressure distribution based 2D in-bed keypoint prediction under interfered scenes

In-bed pose estimation holds significant potential in various domains, including healthcare, sleep studies, and smart homes. Pressure-sensitive bed sheets have emerged as a promising solution for addressing this task considering the advantages of convenience, comfort, and privacy protection. However, existing studies primarily rely on ideal datasets that do not consider the presence of common daily objects such as pillows and quilts referred to as interference, which can significantly impact the pressure distribution. As a result, there is still a gap between the models trained with ideal data and the real-life application. Besides the end-to-end training approach, one potential solution is to recognize the interference and fuse the interference information to the model during training. In this study, we created a well-annotated dataset, consisting of eight in-bed scenes and four common types of interference: pillows, quilts, a laptop, and a package. To facilitate the analysis, the pixels in the pressure image were categorized into five classes based on the relative position between the interference and the human. We then evaluated the performance of five neural network models for pixel-level interference recognition. The best-performing model achieved an accuracy of 80.0% in recognizing the five categories. Subsequently, we validated the utility of interference recognition in improving pose estimation accuracy. The ideal model initially shows an average joint position error of up to 30.59 cm and a Percentage of Correct Keypoints (PCK) of 0.332 on data from scenes with interferences. After retraining on data including interference, the error is reduced to 13.54 cm and the PCK increases to 0.747. By integrating interference recognition information, either by excluding the parts of the interference or using the recognition results as input, the error can be further minimized to 12.44 cm and the PCK can be maximized up to 0.777. Our findings represent an initial step towards the practical deployment of pressure-sensitive bed sheets in everyday life.