Nariman Niknejad, Jessica L. Caro, Rafael Bidese-Puhl, Y. Bao, E. Staiger
{"title":"使用立体录像和深度学习的马运动学步态分析:步幅和站立时间估计","authors":"Nariman Niknejad, Jessica L. Caro, Rafael Bidese-Puhl, Y. Bao, E. Staiger","doi":"10.13031/ja.15386","DOIUrl":null,"url":null,"abstract":"Highlights Stereo machine vision and deep learning techniques were investigated for infield equine kinematic gait analysis. The proposed pipeline tracks equine body landmarks in 3D space and estimates stride length and stance duration. The system can serve as a cost-effective, rapid, and easy-to-use tool for equine locomotion research. Abstract. Equine kinematic gait analysis (EKGA) currently requires a complicated, expensive, and labor-intensive procedure for equine locomotion research. An automated stereo video processing pipeline was developed and evaluated for measuring equine biomechanical parameters. Using stereo videos of 40 different walking horses, a DeepLabCut (DLC) model was trained to detect body landmarks in individual frames. With an autoregressive integrated moving average filter, the landmark detection had a root mean square error of 5.14 pixels and a mean absolute error of 4.87 pixels. As a case study, methods were developed to extract stride length (SL) and stance duration (SD). Individual hoof gait phase detection was achieved using a fine-tuned Faster R-CNN model and a mode filter, yielding precision and recall values of 0.83 and 0.95, respectively. The semi-global block matching (SGBM) algorithm was used to estimate depth maps, and the accuracy was assessed by comparing head length estimation with infield measurements. A Bland-Altman analysis for DLC-detected head length in combination with SGBM-based 3D reconstruction yielded a bias of -0.014 m with upper and lower limits of agreement (LoAs) of 0.03 m and -0.061 m, respectively. Furthermore, Bland-Altman analyses on SD and SL when compared to image-level manual measurements showed biases of -0.02 sec and -0.042 m, respectively. The corresponding LoAs were (0.01907 sec, -0.24 sec) for SD and (0.04 m, -0.12 m) for SL. The proposed method showed promising potential in performing EKGA in an automated, cost-effective, and rapid manner under field conditions. Keywords: 3D Reconstruction, Animal Pose Estimation, Deep Learning, Equine Kinematic Gait Analysis, Stereo Matching.","PeriodicalId":29714,"journal":{"name":"Journal of the ASABE","volume":"44 1","pages":""},"PeriodicalIF":1.2000,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Equine Kinematic Gait Analysis Using Stereo Videography and Deep Learning: Stride Length and Stance Duration Estimation\",\"authors\":\"Nariman Niknejad, Jessica L. Caro, Rafael Bidese-Puhl, Y. Bao, E. Staiger\",\"doi\":\"10.13031/ja.15386\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Highlights Stereo machine vision and deep learning techniques were investigated for infield equine kinematic gait analysis. The proposed pipeline tracks equine body landmarks in 3D space and estimates stride length and stance duration. The system can serve as a cost-effective, rapid, and easy-to-use tool for equine locomotion research. Abstract. Equine kinematic gait analysis (EKGA) currently requires a complicated, expensive, and labor-intensive procedure for equine locomotion research. An automated stereo video processing pipeline was developed and evaluated for measuring equine biomechanical parameters. Using stereo videos of 40 different walking horses, a DeepLabCut (DLC) model was trained to detect body landmarks in individual frames. With an autoregressive integrated moving average filter, the landmark detection had a root mean square error of 5.14 pixels and a mean absolute error of 4.87 pixels. As a case study, methods were developed to extract stride length (SL) and stance duration (SD). Individual hoof gait phase detection was achieved using a fine-tuned Faster R-CNN model and a mode filter, yielding precision and recall values of 0.83 and 0.95, respectively. The semi-global block matching (SGBM) algorithm was used to estimate depth maps, and the accuracy was assessed by comparing head length estimation with infield measurements. A Bland-Altman analysis for DLC-detected head length in combination with SGBM-based 3D reconstruction yielded a bias of -0.014 m with upper and lower limits of agreement (LoAs) of 0.03 m and -0.061 m, respectively. Furthermore, Bland-Altman analyses on SD and SL when compared to image-level manual measurements showed biases of -0.02 sec and -0.042 m, respectively. The corresponding LoAs were (0.01907 sec, -0.24 sec) for SD and (0.04 m, -0.12 m) for SL. The proposed method showed promising potential in performing EKGA in an automated, cost-effective, and rapid manner under field conditions. Keywords: 3D Reconstruction, Animal Pose Estimation, Deep Learning, Equine Kinematic Gait Analysis, Stereo Matching.\",\"PeriodicalId\":29714,\"journal\":{\"name\":\"Journal of the ASABE\",\"volume\":\"44 1\",\"pages\":\"\"},\"PeriodicalIF\":1.2000,\"publicationDate\":\"2023-01-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of the ASABE\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.13031/ja.15386\",\"RegionNum\":4,\"RegionCategory\":\"农林科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"AGRICULTURAL ENGINEERING\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of the ASABE","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.13031/ja.15386","RegionNum":4,"RegionCategory":"农林科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"AGRICULTURAL ENGINEERING","Score":null,"Total":0}
Equine Kinematic Gait Analysis Using Stereo Videography and Deep Learning: Stride Length and Stance Duration Estimation
Highlights Stereo machine vision and deep learning techniques were investigated for infield equine kinematic gait analysis. The proposed pipeline tracks equine body landmarks in 3D space and estimates stride length and stance duration. The system can serve as a cost-effective, rapid, and easy-to-use tool for equine locomotion research. Abstract. Equine kinematic gait analysis (EKGA) currently requires a complicated, expensive, and labor-intensive procedure for equine locomotion research. An automated stereo video processing pipeline was developed and evaluated for measuring equine biomechanical parameters. Using stereo videos of 40 different walking horses, a DeepLabCut (DLC) model was trained to detect body landmarks in individual frames. With an autoregressive integrated moving average filter, the landmark detection had a root mean square error of 5.14 pixels and a mean absolute error of 4.87 pixels. As a case study, methods were developed to extract stride length (SL) and stance duration (SD). Individual hoof gait phase detection was achieved using a fine-tuned Faster R-CNN model and a mode filter, yielding precision and recall values of 0.83 and 0.95, respectively. The semi-global block matching (SGBM) algorithm was used to estimate depth maps, and the accuracy was assessed by comparing head length estimation with infield measurements. A Bland-Altman analysis for DLC-detected head length in combination with SGBM-based 3D reconstruction yielded a bias of -0.014 m with upper and lower limits of agreement (LoAs) of 0.03 m and -0.061 m, respectively. Furthermore, Bland-Altman analyses on SD and SL when compared to image-level manual measurements showed biases of -0.02 sec and -0.042 m, respectively. The corresponding LoAs were (0.01907 sec, -0.24 sec) for SD and (0.04 m, -0.12 m) for SL. The proposed method showed promising potential in performing EKGA in an automated, cost-effective, and rapid manner under field conditions. Keywords: 3D Reconstruction, Animal Pose Estimation, Deep Learning, Equine Kinematic Gait Analysis, Stereo Matching.