In order to address the inconsistency problem caused by parasitic backend wiring among multiple ramp generators and among multiple columns in large-array CMOS image sensors (CIS), this paper proposes a high-precision compensation technology combining average voltage technology, adaptive negative feedback dynamic adjustment technology, and digital correlation double sampling technology to complete the design of an adaptive ramp signals inconsistency calibration scheme. The method proposed in this article has been successfully applied to a CIS with a pixel array of 8192(H) × 8192(V), based on the 55 nm 1P4M CMOS process, with a pixel size of 10×10μm2. The chip area is 88(H) × 89(V) mm2, and the frame rate is 10 fps. The column-level analog-to-digital converter is a 12-bit single-slope analog-to-digital converter (SS ADC). The experimental results show that the ramp generation circuit proposed in this paper can reduce the inconsistency among the ramp signals to 0.4% LSB, decreases the column fixed pattern noise (CFPN) caused by inconsistent ramps of each column to 0.000037% (0.15 e-), and increases the overall chip area and power consumption by only 0.6% and 0.5%, respectively. This method provides an effective solution to the influence of non-ideal factors on the consistency of ramp signals in large area array CIS.
{"title":"High Consistency Ramp Design Method for Low Noise Column Level Readout Chain.","authors":"Zhongjie Guo, Lin Li, Ruiming Xu, Suiyang Liu, Ningmei Yu, Yuan Yang, Longsheng Wu","doi":"10.3390/s24217057","DOIUrl":"10.3390/s24217057","url":null,"abstract":"<p><p>In order to address the inconsistency problem caused by parasitic backend wiring among multiple ramp generators and among multiple columns in large-array CMOS image sensors (CIS), this paper proposes a high-precision compensation technology combining average voltage technology, adaptive negative feedback dynamic adjustment technology, and digital correlation double sampling technology to complete the design of an adaptive ramp signals inconsistency calibration scheme. The method proposed in this article has been successfully applied to a CIS with a pixel array of 8192(H) × 8192(V), based on the 55 nm 1P4M CMOS process, with a pixel size of 10×10μm2. The chip area is 88(H) × 89(V) mm2, and the frame rate is 10 fps. The column-level analog-to-digital converter is a 12-bit single-slope analog-to-digital converter (SS ADC). The experimental results show that the ramp generation circuit proposed in this paper can reduce the inconsistency among the ramp signals to 0.4% LSB, decreases the column fixed pattern noise (CFPN) caused by inconsistent ramps of each column to 0.000037% (0.15 e-), and increases the overall chip area and power consumption by only 0.6% and 0.5%, respectively. This method provides an effective solution to the influence of non-ideal factors on the consistency of ramp signals in large area array CIS.</p>","PeriodicalId":21698,"journal":{"name":"Sensors","volume":"24 21","pages":""},"PeriodicalIF":3.4,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11548106/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142627276","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"综合性期刊","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nonlinearity in sensor measurements reduces the sensor's accuracy. Therefore, accurate calibration is necessary for reliable sensor operation. This study proposes a segmented calibration method that divides the input range into multiple sections and calculates the optimized calibration functions for each one. This approach reduces the overall error rate and improves the calibration accuracy by isolating distinctive regions. The modified progressive polynomial calibration technique is used to calculate the calibration function. This algorithm addresses the computational complexity, allowing for reduced polynomial degrees and improving the accuracy. The segmented calibration method achieves a significantly lower error rate of 0.000006% compared to the original single calibration method, which has an error rate of 0.0823%, when using the same six calibration points and a fifth-degree polynomial function. This method maintains improved accuracy with fewer calibration points, and its ability to reduce the computational complexity and calculation time while using lower polynomial degrees is confirmed. Additionally, it can be extended to two dimensions to reduce the errors caused by cross-sensitivity. The results from a two-dimensional simulation show a reduction in the error rate ranging from 15.84% to 2.07% in an 8-bit signed fixed-point system. These results indicate that the segmented calibration method is an effective and scalable solution for various typical sensors.
{"title":"Segmented Two-Dimensional Progressive Polynomial Calibration Method for Nonlinear Sensors.","authors":"Jae-Lim Lee, Dong-Sun Kim","doi":"10.3390/s24217058","DOIUrl":"10.3390/s24217058","url":null,"abstract":"<p><p>Nonlinearity in sensor measurements reduces the sensor's accuracy. Therefore, accurate calibration is necessary for reliable sensor operation. This study proposes a segmented calibration method that divides the input range into multiple sections and calculates the optimized calibration functions for each one. This approach reduces the overall error rate and improves the calibration accuracy by isolating distinctive regions. The modified progressive polynomial calibration technique is used to calculate the calibration function. This algorithm addresses the computational complexity, allowing for reduced polynomial degrees and improving the accuracy. The segmented calibration method achieves a significantly lower error rate of 0.000006% compared to the original single calibration method, which has an error rate of 0.0823%, when using the same six calibration points and a fifth-degree polynomial function. This method maintains improved accuracy with fewer calibration points, and its ability to reduce the computational complexity and calculation time while using lower polynomial degrees is confirmed. Additionally, it can be extended to two dimensions to reduce the errors caused by cross-sensitivity. The results from a two-dimensional simulation show a reduction in the error rate ranging from 15.84% to 2.07% in an 8-bit signed fixed-point system. These results indicate that the segmented calibration method is an effective and scalable solution for various typical sensors.</p>","PeriodicalId":21698,"journal":{"name":"Sensors","volume":"24 21","pages":""},"PeriodicalIF":3.4,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11548594/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142627362","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"综合性期刊","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Augmented reality technologies provide transformative solutions in various surgical fields. Our research focuses on the use of an advanced augmented reality system that projects 3D holographic images directly into surgical footage, potentially improving the surgeon's orientation to the surgical field and lowering the cognitive load. We created a novel system that combines exoscopic surgical footage from the "ORBEYE" and displays both the surgical field and 3D holograms on a single screen. This setup enables surgeons to use the system without using head-mounted displays, instead viewing the integrated images on a 3D monitor. Thirteen surgeons and surgical assistants completed tasks with 2D and 3D graphical surgical guides. The NASA Task Load Index was used to assess mental, physical, and temporal demands. The use of 3D graphical surgical guides significantly improved performance metrics in cochlear implant surgeries by lowering mental, physical, temporal, and frustration levels. However, for Bonebridge implantation, the 2D graphical surgical guide performed better overall (p = 0.045). Participants found the augmented reality system's video latency to be imperceptible, measuring 0.13 ± 0.01 s. This advanced augmented reality system significantly improves the efficiency and precision of cochlear implant surgeries by lowering cognitive load and improving spatial orientation.
{"title":"Integration of Augmented Reality in Temporal Bone and Skull Base Surgeries.","authors":"Taku Ito, Taro Fujikawa, Takamori Takeda, Yoshimaru Mizoguchi, Kouta Okubo, Shinya Onogi, Yoshikazu Nakajima, Takeshi Tsutsumi","doi":"10.3390/s24217063","DOIUrl":"10.3390/s24217063","url":null,"abstract":"<p><p>Augmented reality technologies provide transformative solutions in various surgical fields. Our research focuses on the use of an advanced augmented reality system that projects 3D holographic images directly into surgical footage, potentially improving the surgeon's orientation to the surgical field and lowering the cognitive load. We created a novel system that combines exoscopic surgical footage from the \"ORBEYE\" and displays both the surgical field and 3D holograms on a single screen. This setup enables surgeons to use the system without using head-mounted displays, instead viewing the integrated images on a 3D monitor. Thirteen surgeons and surgical assistants completed tasks with 2D and 3D graphical surgical guides. The NASA Task Load Index was used to assess mental, physical, and temporal demands. The use of 3D graphical surgical guides significantly improved performance metrics in cochlear implant surgeries by lowering mental, physical, temporal, and frustration levels. However, for Bonebridge implantation, the 2D graphical surgical guide performed better overall (<i>p</i> = 0.045). Participants found the augmented reality system's video latency to be imperceptible, measuring 0.13 ± 0.01 s. This advanced augmented reality system significantly improves the efficiency and precision of cochlear implant surgeries by lowering cognitive load and improving spatial orientation.</p>","PeriodicalId":21698,"journal":{"name":"Sensors","volume":"24 21","pages":""},"PeriodicalIF":3.4,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11548182/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142627442","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"综合性期刊","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lorenzo Brognara, Antonio Mazzotti, Simone Ottavio Zielli, Alberto Arceri, Elena Artioli, Francesco Traina, Cesare Faldini
Foot and ankle disorders are a very common diseases, represent a risk factor for falls in older people, and are associated with difficulty performing activities of daily living. With an increasing demand for cost-effective and high-quality clinical services, wearable technology can be strategic in extending our reach to patients with foot and ankle disorders. In recent years, wearable sensors have been increasingly utilized to assess the clinical outcomes of surgery, rehabilitation, and orthotic treatments. This article highlights recent achievements and developments in wearable sensor-based foot and ankle clinical assessment. An increasing number of studies have established the feasibility and effectiveness of wearable technology tools for foot and ankle disorders. Different methods and outcomes for feasibility studies have been introduced, such as satisfaction and efficacy in rehabilitation, surgical, and orthotic treatments. Currently, the widespread application of wearable sensors in clinical fields is hindered by a lack of robust evidence; in fact, only a few tests and analysis protocols are validated with cut-off values reported in the literature. However, nowadays, these tools are useful in quantifying clinical results before and after clinical treatments, providing useful data, also collected in real-life conditions, on the results of therapies.
{"title":"Wearable Technology Applications and Methods to Assess Clinical Outcomes in Foot and Ankle Disorders: Achievements and Perspectives.","authors":"Lorenzo Brognara, Antonio Mazzotti, Simone Ottavio Zielli, Alberto Arceri, Elena Artioli, Francesco Traina, Cesare Faldini","doi":"10.3390/s24217059","DOIUrl":"10.3390/s24217059","url":null,"abstract":"<p><p>Foot and ankle disorders are a very common diseases, represent a risk factor for falls in older people, and are associated with difficulty performing activities of daily living. With an increasing demand for cost-effective and high-quality clinical services, wearable technology can be strategic in extending our reach to patients with foot and ankle disorders. In recent years, wearable sensors have been increasingly utilized to assess the clinical outcomes of surgery, rehabilitation, and orthotic treatments. This article highlights recent achievements and developments in wearable sensor-based foot and ankle clinical assessment. An increasing number of studies have established the feasibility and effectiveness of wearable technology tools for foot and ankle disorders. Different methods and outcomes for feasibility studies have been introduced, such as satisfaction and efficacy in rehabilitation, surgical, and orthotic treatments. Currently, the widespread application of wearable sensors in clinical fields is hindered by a lack of robust evidence; in fact, only a few tests and analysis protocols are validated with cut-off values reported in the literature. However, nowadays, these tools are useful in quantifying clinical results before and after clinical treatments, providing useful data, also collected in real-life conditions, on the results of therapies.</p>","PeriodicalId":21698,"journal":{"name":"Sensors","volume":"24 21","pages":""},"PeriodicalIF":3.4,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11548473/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142627702","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"综合性期刊","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Flexible thermoelectric generators (TEGs) with pn-junction single-walled carbon nanotube (SWCNT) films on a polyimide substrate have attracted considerable attention for energy harvesting. This is because they generate electricity through the photo-thermoelectric effect by self-generated temperature gradient under uniform sunlight irradiation. To increase the performance and durability of the pn-junction TEGs, n-type films need to be improved as a priority. In this study, bismuth telluride (Bi2Te3) nanoplates synthesized by the solvothermal method were added to the n-type SWCNT films, including a cationic surfactant to form the nanocomposite films because Bi2Te3 has high n-type thermoelectric properties and high durability. The performances of the pn-junction TEGs were investigated by varying the heat treatment times. When the artificial sunlight was uniformly irradiated to the pn-junction TEGs, a stable output voltage of 0.47 mV was observed in the TEG with nanocomposite films heat-treated at 1 h. The output voltage decreased with increasing heat treatment time due to the decrease in the p-type region. The output voltage of TEG at 1 h is higher than that of the TEGs without Bi2Te3 nanoplates under the same conditions. Therefore, the addition of Bi2Te3 nanoplates was found to improve the performance of the pn-junction TEGs. These findings may aid in the development of facile and flexible optical devices, including photodetectors and hybrid devices integrating solar cells.
{"title":"N-Type Nanocomposite Films Combining SWCNTs, Bi<sub>2</sub>Te<sub>3</sub> Nanoplates, and Cationic Surfactant for Pn-Junction Thermoelectric Generators with Self-Generated Temperature Gradient Under Uniform Sunlight Irradiation.","authors":"Koki Hoshino, Hisatoshi Yamamoto, Ryota Tamai, Takumi Nakajima, Shugo Miyake, Masayuki Takashiri","doi":"10.3390/s24217060","DOIUrl":"10.3390/s24217060","url":null,"abstract":"<p><p>Flexible thermoelectric generators (TEGs) with pn-junction single-walled carbon nanotube (SWCNT) films on a polyimide substrate have attracted considerable attention for energy harvesting. This is because they generate electricity through the photo-thermoelectric effect by self-generated temperature gradient under uniform sunlight irradiation. To increase the performance and durability of the pn-junction TEGs, n-type films need to be improved as a priority. In this study, bismuth telluride (Bi<sub>2</sub>Te<sub>3</sub>) nanoplates synthesized by the solvothermal method were added to the n-type SWCNT films, including a cationic surfactant to form the nanocomposite films because Bi<sub>2</sub>Te<sub>3</sub> has high n-type thermoelectric properties and high durability. The performances of the pn-junction TEGs were investigated by varying the heat treatment times. When the artificial sunlight was uniformly irradiated to the pn-junction TEGs, a stable output voltage of 0.47 mV was observed in the TEG with nanocomposite films heat-treated at 1 h. The output voltage decreased with increasing heat treatment time due to the decrease in the p-type region. The output voltage of TEG at 1 h is higher than that of the TEGs without Bi<sub>2</sub>Te<sub>3</sub> nanoplates under the same conditions. Therefore, the addition of Bi<sub>2</sub>Te<sub>3</sub> nanoplates was found to improve the performance of the pn-junction TEGs. These findings may aid in the development of facile and flexible optical devices, including photodetectors and hybrid devices integrating solar cells.</p>","PeriodicalId":21698,"journal":{"name":"Sensors","volume":"24 21","pages":""},"PeriodicalIF":3.4,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11548406/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636087","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"综合性期刊","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hongzhe Zhang, Jihui L Diaz, Soohyun Kim, Zilong Yu, Yiyuan Wu, Emily Carter, Samprit Banerjee
Recent advancements in mobile health (mHealth) technology and the ubiquity of wearable devices and smartphones have expanded a market for digital health and have emerged as innovative tools for data collection on individualized behavior. Heterogeneous levels of device usage across users and across days within a single user may result in different degrees of underestimation in passive sensing data, subsequently introducing biases if analyzed without addressing this issue. In this work, we propose an unsupervised 2-Stage Pre-processing Algorithm for Passively Sensed mHealth Data (2SpamH) algorithm that uses device usage variables to infer the quality of passive sensing data from mobile devices. This article provides a series of simulation studies to show the utility of the proposed algorithm compared to existing methods. Application to a real clinical dataset is also illustrated.
{"title":"2SpamH: A Two-Stage Pre-Processing Algorithm for Passively Sensed mHealth Data.","authors":"Hongzhe Zhang, Jihui L Diaz, Soohyun Kim, Zilong Yu, Yiyuan Wu, Emily Carter, Samprit Banerjee","doi":"10.3390/s24217053","DOIUrl":"10.3390/s24217053","url":null,"abstract":"<p><p>Recent advancements in mobile health (mHealth) technology and the ubiquity of wearable devices and smartphones have expanded a market for digital health and have emerged as innovative tools for data collection on individualized behavior. Heterogeneous levels of device usage across users and across days within a single user may result in different degrees of underestimation in passive sensing data, subsequently introducing biases if analyzed without addressing this issue. In this work, we propose an unsupervised 2-Stage Pre-processing Algorithm for Passively Sensed mHealth Data (2SpamH) algorithm that uses device usage variables to infer the quality of passive sensing data from mobile devices. This article provides a series of simulation studies to show the utility of the proposed algorithm compared to existing methods. Application to a real clinical dataset is also illustrated.</p>","PeriodicalId":21698,"journal":{"name":"Sensors","volume":"24 21","pages":""},"PeriodicalIF":3.4,"publicationDate":"2024-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11548539/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142627579","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"综合性期刊","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A crucial role is played by steering-angle prediction in the control of autonomous vehicles (AVs). It mainly includes the prediction and control of the steering angle. However, the prediction accuracy and calculation efficiency of traditional YOLOv5 are limited. For the control of the steering angle, angular velocity is difficult to measure, and the angle control effect is affected by external disturbances and unknown friction. This paper proposes a lightweight steering angle prediction network model called YOLOv5Ms, based on YOLOv5, aiming to achieve accurate prediction while enhancing computational efficiency. Additionally, an adaptive output feedback control scheme with output constraints based on neural networks is proposed to regulate the predicted steering angle using the YOLOv5Ms algorithm effectively. Firstly, given that most lane-line data sets consist of simulated images and lack diversity, a novel lane data set derived from real roads is manually created to train the proposed network model. To improve real-time accuracy in steering-angle prediction and enhance effectiveness in steering control, we update the bounding box regression loss function with the generalized intersection over union (GIoU) to Shape-IoU_Loss as a better-converging regression loss function for bounding-box improvement. The YOLOv5Ms model achieves a 30.34% reduction in weight storage space while simultaneously improving accuracy by 7.38% compared to the YOLOv5s model. Furthermore, an adaptive output feedback control scheme with output constraints based on neural networks is introduced to regulate the predicted steering angle via YOLOv5Ms effectively. Moreover, utilizing the backstepping control method and introducing the Lyapunov barrier function enables us to design an adaptive neural network output feedback controller with output constraints. Finally, a strict stability analysis based on Lyapunov stability theory ensures the boundedness of all signals within the closed-loop system. Numerical simulations and experiments have shown that the proposed method provides a 39.16% better root mean squared error (RMSE) score than traditional backstepping control, and it achieves good estimation performance for angles, angular velocity, and unknown disturbances.
{"title":"Steering-Angle Prediction and Controller Design Based on Improved YOLOv5 for Steering-by-Wire System.","authors":"Cunliang Ye, Yunlong Wang, Yongfu Wang, Yan Liu","doi":"10.3390/s24217035","DOIUrl":"10.3390/s24217035","url":null,"abstract":"<p><p>A crucial role is played by steering-angle prediction in the control of autonomous vehicles (AVs). It mainly includes the prediction and control of the steering angle. However, the prediction accuracy and calculation efficiency of traditional YOLOv5 are limited. For the control of the steering angle, angular velocity is difficult to measure, and the angle control effect is affected by external disturbances and unknown friction. This paper proposes a lightweight steering angle prediction network model called YOLOv5Ms, based on YOLOv5, aiming to achieve accurate prediction while enhancing computational efficiency. Additionally, an adaptive output feedback control scheme with output constraints based on neural networks is proposed to regulate the predicted steering angle using the YOLOv5Ms algorithm effectively. Firstly, given that most lane-line data sets consist of simulated images and lack diversity, a novel lane data set derived from real roads is manually created to train the proposed network model. To improve real-time accuracy in steering-angle prediction and enhance effectiveness in steering control, we update the bounding box regression loss function with the generalized intersection over union (GIoU) to Shape-IoU_Loss as a better-converging regression loss function for bounding-box improvement. The YOLOv5Ms model achieves a 30.34% reduction in weight storage space while simultaneously improving accuracy by 7.38% compared to the YOLOv5s model. Furthermore, an adaptive output feedback control scheme with output constraints based on neural networks is introduced to regulate the predicted steering angle via YOLOv5Ms effectively. Moreover, utilizing the backstepping control method and introducing the Lyapunov barrier function enables us to design an adaptive neural network output feedback controller with output constraints. Finally, a strict stability analysis based on Lyapunov stability theory ensures the boundedness of all signals within the closed-loop system. Numerical simulations and experiments have shown that the proposed method provides a 39.16% better root mean squared error (RMSE) score than traditional backstepping control, and it achieves good estimation performance for angles, angular velocity, and unknown disturbances.</p>","PeriodicalId":21698,"journal":{"name":"Sensors","volume":"24 21","pages":""},"PeriodicalIF":3.4,"publicationDate":"2024-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11548791/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142627590","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"综合性期刊","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xiangan Wan, Jianping Ju, Jianying Tang, Mingyu Lin, Ning Rao, Deng Chen, Tingting Liu, Jing Li, Fan Bian, Nicholas Xiong
The objective of 3D hand pose estimation (HPE) based on depth images is to accurately locate and predict keypoints of the hand. However, this task remains challenging because of the variations in hand appearance from different viewpoints and severe occlusions. To effectively address these challenges, this study introduces a novel approach, called the multi-perspective cue-aware joint relationship representation for 3D HPE via the Swin Transformer (MPCTrans, for short). This approach is designed to learn multi-perspective cues and essential information from hand depth images. To achieve this goal, three novel modules are proposed to utilize features from multiple virtual views of the hand, namely, the adaptive virtual multi-viewpoint (AVM), hierarchy feature estimation (HFE), and virtual viewpoint evaluation (VVE) modules. The AVM module adaptively adjusts the angles of the virtual viewpoint and learns the ideal virtual viewpoint to generate informative multiple virtual views. The HFE module estimates hand keypoints through hierarchical feature extraction. The VVE module evaluates virtual viewpoints by using chained high-level functions from the HFE module. Transformer is used as a backbone to extract the long-range semantic joint relationships in hand depth images. Extensive experiments demonstrate that the MPCTrans model achieves state-of-the-art performance on four challenging benchmark datasets.
{"title":"MPCTrans: Multi-Perspective Cue-Aware Joint Relationship Representation for 3D Hand Pose Estimation via Swin Transformer.","authors":"Xiangan Wan, Jianping Ju, Jianying Tang, Mingyu Lin, Ning Rao, Deng Chen, Tingting Liu, Jing Li, Fan Bian, Nicholas Xiong","doi":"10.3390/s24217029","DOIUrl":"10.3390/s24217029","url":null,"abstract":"<p><p>The objective of 3D hand pose estimation (HPE) based on depth images is to accurately locate and predict keypoints of the hand. However, this task remains challenging because of the variations in hand appearance from different viewpoints and severe occlusions. To effectively address these challenges, this study introduces a novel approach, called the multi-perspective cue-aware joint relationship representation for 3D HPE via the Swin Transformer (MPCTrans, for short). This approach is designed to learn multi-perspective cues and essential information from hand depth images. To achieve this goal, three novel modules are proposed to utilize features from multiple virtual views of the hand, namely, the adaptive virtual multi-viewpoint (AVM), hierarchy feature estimation (HFE), and virtual viewpoint evaluation (VVE) modules. The AVM module adaptively adjusts the angles of the virtual viewpoint and learns the ideal virtual viewpoint to generate informative multiple virtual views. The HFE module estimates hand keypoints through hierarchical feature extraction. The VVE module evaluates virtual viewpoints by using chained high-level functions from the HFE module. Transformer is used as a backbone to extract the long-range semantic joint relationships in hand depth images. Extensive experiments demonstrate that the MPCTrans model achieves state-of-the-art performance on four challenging benchmark datasets.</p>","PeriodicalId":21698,"journal":{"name":"Sensors","volume":"24 21","pages":""},"PeriodicalIF":3.4,"publicationDate":"2024-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11548048/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142627598","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"综合性期刊","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The research on how to effectively extract the instantaneous characteristic components of non-stationary signals continues to be both a research hotspot and a very challenging topic. In this paper, a new method of multi-component decomposition is proposed to decompose a signal into finite mono-component signals and extract their Instantaneous Amplitude (IA), Instantaneous Phase (IP), and Instantaneous Frequency (IF), which is called Sinusoidal Fitting Decomposition (SFD). The proposed method can ensure that the IA extracted from the given signal must be positive, the IP is monotonically increasing, and the signal synthesized by both IA and IP must be mono-componential and smooth. It transforms the decomposition process into a synthesis iterative process and does not rely on any dictionary or basis function space or carry out the sifting operation. In addition, the proposed method can describe the instantaneous-frequency-amplitude characteristics of the signal very well on the time-frequency plane. The results of numerical simulation and the qualitative analysis of the amount of calculation show that the proposed method is effective.
如何有效提取非稳态信号的瞬时特征分量一直是研究热点,也是极具挑战性的课题。本文提出了一种新的多分量分解方法,将信号分解成有限的单分量信号,并提取其瞬时振幅(IA)、瞬时相位(IP)和瞬时频率(IF),即正弦拟合分解(SFD)。所提出的方法能确保从给定信号中提取的 IA 必须是正值,IP 是单调递增的,并且 IA 和 IP 合成的信号必须是单调且平滑的。它将分解过程转化为合成迭代过程,不依赖任何字典或基函数空间,也不进行筛选操作。此外,所提出的方法能在时频平面上很好地描述信号的瞬时频率-振幅特性。数值模拟和计算量定性分析的结果表明,所提出的方法是有效的。
{"title":"Sinusoidal Fitting Decomposition for Instantaneous Characteristic Representation of Multi-Componential Signal.","authors":"Donghu Nie, Xin Su, Gang Qiao","doi":"10.3390/s24217032","DOIUrl":"10.3390/s24217032","url":null,"abstract":"<p><p>The research on how to effectively extract the instantaneous characteristic components of non-stationary signals continues to be both a research hotspot and a very challenging topic. In this paper, a new method of multi-component decomposition is proposed to decompose a signal into finite mono-component signals and extract their Instantaneous Amplitude (IA), Instantaneous Phase (IP), and Instantaneous Frequency (IF), which is called Sinusoidal Fitting Decomposition (SFD). The proposed method can ensure that the IA extracted from the given signal must be positive, the IP is monotonically increasing, and the signal synthesized by both IA and IP must be mono-componential and smooth. It transforms the decomposition process into a synthesis iterative process and does not rely on any dictionary or basis function space or carry out the sifting operation. In addition, the proposed method can describe the instantaneous-frequency-amplitude characteristics of the signal very well on the time-frequency plane. The results of numerical simulation and the qualitative analysis of the amount of calculation show that the proposed method is effective.</p>","PeriodicalId":21698,"journal":{"name":"Sensors","volume":"24 21","pages":""},"PeriodicalIF":3.4,"publicationDate":"2024-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11548266/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142627476","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"综合性期刊","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Songjun Han, Zhipeng Feng, Ying Zhang, Minggang Du, Yang Yang
Sensory data are the basis for the intelligent health state awareness of planetary gearboxes, which are the critical components of electromechanical systems. Despite the advantages of intelligent diagnostic techniques for detecting intricate fault patterns and improving diagnostic speed, challenges still persist, which include the limited availability of fault data, the lack of labeling information and the discrepancies in features across different signals. Targeting this issue, a subdomain distribution adversarial adaptation diagnosis method (SDAA) is proposed for faults diagnosis of planetary gearboxes across different conditions. Firstly, nonstationary vibration signals are converted into a two-dimensional time-frequency representation to extract intrinsic information and avoid frequency overlapping. Secondly, an adversarial training mechanism is designed to evaluate subclass feature distribution differences between the source and target domain. A conditional distribution adaptation is employed to account for correlations among data from different subclasses. Finally, the proposed method is validated through experiments on planetary gearboxes, and the results demonstrate that SDAA can effectively diagnose faults under crossing conditions with an accuracy of 96.7% in diagnosing gear faults and 95.2% in diagnosing planet bearing faults. It outperforms other methods in both accuracy and model robustness. This confirms that this approach can refine domain-invariant information for transfer learning with less information loss from the sub-class level of fault data instead of the overall class level.
{"title":"Intelligent Fault Diagnosis of Planetary Gearbox Across Conditions Based on Subdomain Distribution Adversarial Adaptation.","authors":"Songjun Han, Zhipeng Feng, Ying Zhang, Minggang Du, Yang Yang","doi":"10.3390/s24217017","DOIUrl":"10.3390/s24217017","url":null,"abstract":"<p><p>Sensory data are the basis for the intelligent health state awareness of planetary gearboxes, which are the critical components of electromechanical systems. Despite the advantages of intelligent diagnostic techniques for detecting intricate fault patterns and improving diagnostic speed, challenges still persist, which include the limited availability of fault data, the lack of labeling information and the discrepancies in features across different signals. Targeting this issue, a subdomain distribution adversarial adaptation diagnosis method (SDAA) is proposed for faults diagnosis of planetary gearboxes across different conditions. Firstly, nonstationary vibration signals are converted into a two-dimensional time-frequency representation to extract intrinsic information and avoid frequency overlapping. Secondly, an adversarial training mechanism is designed to evaluate subclass feature distribution differences between the source and target domain. A conditional distribution adaptation is employed to account for correlations among data from different subclasses. Finally, the proposed method is validated through experiments on planetary gearboxes, and the results demonstrate that SDAA can effectively diagnose faults under crossing conditions with an accuracy of 96.7% in diagnosing gear faults and 95.2% in diagnosing planet bearing faults. It outperforms other methods in both accuracy and model robustness. This confirms that this approach can refine domain-invariant information for transfer learning with less information loss from the sub-class level of fault data instead of the overall class level.</p>","PeriodicalId":21698,"journal":{"name":"Sensors","volume":"24 21","pages":""},"PeriodicalIF":3.4,"publicationDate":"2024-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11548617/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142627443","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"综合性期刊","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}