Pub Date : 2022-08-29eCollection Date: 2022-09-01DOI: 10.1159/000525897
Bohdana Ratitch, Isaac R Rodriguez-Chavez, Abhishek Dabral, Adriano Fontanari, Julio Vega, Francesco Onorati, Benjamin Vandendriessche, Stuart Morton, Yasaman Damestani
Background: The proliferation and increasing maturity of biometric monitoring technologies allow clinical investigators to measure the health status of trial participants in a more holistic manner, especially outside of traditional clinical settings. This includes capturing meaningful aspects of health in daily living and a more granular and objective manner compared to traditional tools in clinical settings.
Summary: Within multidisciplinary teams, statisticians and data scientists are increasingly involved in clinical trials that incorporate digital clinical measures. They are called upon to provide input into trial planning, generation of evidence on the clinical validity of novel clinical measures, and evaluation of the adequacy of existing evidence. Analysis objectives related to demonstrating clinical validity of novel clinical measures differ from typical objectives related to demonstrating safety and efficacy of therapeutic interventions using established measures which statisticians are most familiar with.
Key messages: This paper discusses key considerations for generating evidence for clinical validity through the lens of the type and intended use of a clinical measure. This paper also briefly discusses the regulatory pathways through which clinical validity evidence may be reviewed and highlights challenges that investigators may encounter while dealing with data from biometric monitoring technologies.
{"title":"Considerations for Analyzing and Interpreting Data from Biometric Monitoring Technologies in Clinical Trials.","authors":"Bohdana Ratitch, Isaac R Rodriguez-Chavez, Abhishek Dabral, Adriano Fontanari, Julio Vega, Francesco Onorati, Benjamin Vandendriessche, Stuart Morton, Yasaman Damestani","doi":"10.1159/000525897","DOIUrl":"https://doi.org/10.1159/000525897","url":null,"abstract":"<p><strong>Background: </strong>The proliferation and increasing maturity of biometric monitoring technologies allow clinical investigators to measure the health status of trial participants in a more holistic manner, especially outside of traditional clinical settings. This includes capturing meaningful aspects of health in daily living and a more granular and objective manner compared to traditional tools in clinical settings.</p><p><strong>Summary: </strong>Within multidisciplinary teams, statisticians and data scientists are increasingly involved in clinical trials that incorporate digital clinical measures. They are called upon to provide input into trial planning, generation of evidence on the clinical validity of novel clinical measures, and evaluation of the adequacy of existing evidence. Analysis objectives related to demonstrating clinical validity of novel clinical measures differ from typical objectives related to demonstrating safety and efficacy of therapeutic interventions using established measures which statisticians are most familiar with.</p><p><strong>Key messages: </strong>This paper discusses key considerations for generating evidence for clinical validity through the lens of the type and intended use of a clinical measure. This paper also briefly discusses the regulatory pathways through which clinical validity evidence may be reviewed and highlights challenges that investigators may encounter while dealing with data from biometric monitoring technologies.</p>","PeriodicalId":11242,"journal":{"name":"Digital Biomarkers","volume":"6 3","pages":"83-97"},"PeriodicalIF":0.0,"publicationDate":"2022-08-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/62/9e/dib-0006-0083.PMC9716191.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"35345247","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-21eCollection Date: 2022-01-01DOI: 10.1159/000525698
Leif Simmatis, Carolina Barnett, Reeman Marzouqah, Babak Taati, Mark Boulos, Yana Yunusova
Introduction: Telehealth/remote assessment using readily available 2D mobile cameras and deep learning-based analyses is rapidly becoming a viable option for detecting orofacial and speech impairments associated with neurological and neurodegenerative disease during telehealth practice. However, the psychometric properties (e.g., internal consistency and reliability) of kinematics obtained from these systems have not been established, which is a crucial next step before their clinical usability is established.
Methods: Participants were assessed in lab using a 3 dimensional (3D)-capable camera and at home using a readily-available 2D camera in a tablet. Orofacial kinematics was estimated from videos using a deep facial landmark tracking model. Kinematic features quantified the clinically relevant constructs of velocity, range of motion, and lateralization. In lab, all participants performed the same oromotor task. At home, participants were split into two groups that each performed a variant of the in-lab task. We quantified within-assessment consistency (Cronbach's α), reliability (intraclass correlation coefficient [ICC]), and fitted linear mixed-effects models to at-home data to capture individual-/task-dependent longitudinal trajectories.
Results: Both in lab and at home, Cronbach's α was typically high (>0.80) and ICCs were often good (>0.70). The linear mixed-effect models that best fit the longitudinal data were those that accounted for individual- or task-dependent effects.
Discussion: Remotely gathered orofacial kinematics were as internally consistent and reliable as those gathered in a controlled laboratory setting using a high-performance 3D-capable camera and could additionally capture individual- or task-dependent changes over time. These results highlight the potential of remote assessment tools as digital biomarkers of disease status and progression and demonstrate their suitability for novel telehealth applications.
{"title":"Reliability of Automatic Computer Vision-Based Assessment of Orofacial Kinematics for Telehealth Applications.","authors":"Leif Simmatis, Carolina Barnett, Reeman Marzouqah, Babak Taati, Mark Boulos, Yana Yunusova","doi":"10.1159/000525698","DOIUrl":"https://doi.org/10.1159/000525698","url":null,"abstract":"<p><strong>Introduction: </strong>Telehealth/remote assessment using readily available 2D mobile cameras and deep learning-based analyses is rapidly becoming a viable option for detecting orofacial and speech impairments associated with neurological and neurodegenerative disease during telehealth practice. However, the psychometric properties (e.g., internal consistency and reliability) of kinematics obtained from these systems have not been established, which is a crucial next step before their clinical usability is established.</p><p><strong>Methods: </strong>Participants were assessed in lab using a 3 dimensional (3D)-capable camera and at home using a readily-available 2D camera in a tablet. Orofacial kinematics was estimated from videos using a deep facial landmark tracking model. Kinematic features quantified the clinically relevant constructs of velocity, range of motion, and lateralization. In lab, all participants performed the same oromotor task. At home, participants were split into two groups that each performed a variant of the in-lab task. We quantified within-assessment consistency (Cronbach's α), reliability (intraclass correlation coefficient [ICC]), and fitted linear mixed-effects models to at-home data to capture individual-/task-dependent longitudinal trajectories.</p><p><strong>Results: </strong>Both in lab and at home, Cronbach's α was typically high (>0.80) and ICCs were often good (>0.70). The linear mixed-effect models that best fit the longitudinal data were those that accounted for individual- or task-dependent effects.</p><p><strong>Discussion: </strong>Remotely gathered orofacial kinematics were as internally consistent and reliable as those gathered in a controlled laboratory setting using a high-performance 3D-capable camera and could additionally capture individual- or task-dependent changes over time. These results highlight the potential of remote assessment tools as digital biomarkers of disease status and progression and demonstrate their suitability for novel telehealth applications.</p>","PeriodicalId":11242,"journal":{"name":"Digital Biomarkers","volume":"6 2","pages":"71-82"},"PeriodicalIF":0.0,"publicationDate":"2022-07-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/c1/96/dib-0006-0071.PMC9574208.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"40644965","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-04eCollection Date: 2022-05-01DOI: 10.1159/000525080
Charmaine Demanuele, Cynthia Lokker, Krishna Jhaveri, Pirinka Georgiev, Emre Sezgin, Cindy Geoghegan, Kelly H Zou, Elena Izmailova, Marie McCarthy
Background: Digital health technologies are attracting attention as novel tools for data collection in clinical research. They present alternative methods compared to in-clinic data collection, which often yields snapshots of the participants' physiology, behavior, and function that may be prone to biases and artifacts, e.g., white coat hypertension, and not representative of the data in free-living conditions. Modern digital health technologies equipped with multi-modal sensors combine different data streams to derive comprehensive endpoints that are important to study participants and are clinically meaningful. Used for data collection in clinical trials, they can be deployed as provisioned products where technology is given at study start or in a bring your own "device" (BYOD) manner where participants use their technologies to generate study data.
Summary: The BYOD option has the potential to be more user-friendly, allowing participants to use technologies that they are familiar with, ensuring better participant compliance, and potentially reducing the bias that comes with introducing new technologies. However, this approach presents different technical, operational, regulatory, and ethical challenges to study teams. For example, BYOD data can be more heterogeneous, and recruiting historically underrepresented populations with limited access to technology and the internet can be challenging. Despite the rapid increase in digital health technologies for clinical and healthcare research, BYOD use in clinical trials is limited, and regulatory guidance is still evolving.
Key messages: We offer considerations for academic researchers, drug developers, and patient advocacy organizations on the design and deployment of BYOD models in clinical research. These considerations address: (1) early identification and engagement with internal and external stakeholders; (2) study design including informed consent and recruitment strategies; (3) outcome, endpoint, and technology selection; (4) data management including compliance and data monitoring; (5) statistical considerations to meet regulatory requirements. We believe that this article acts as a primer, providing insights into study design and operational requirements to ensure the successful implementation of BYOD clinical studies.
{"title":"Considerations for Conducting Bring Your Own \"Device\" (BYOD) Clinical Studies.","authors":"Charmaine Demanuele, Cynthia Lokker, Krishna Jhaveri, Pirinka Georgiev, Emre Sezgin, Cindy Geoghegan, Kelly H Zou, Elena Izmailova, Marie McCarthy","doi":"10.1159/000525080","DOIUrl":"https://doi.org/10.1159/000525080","url":null,"abstract":"<p><strong>Background: </strong>Digital health technologies are attracting attention as novel tools for data collection in clinical research. They present alternative methods compared to in-clinic data collection, which often yields snapshots of the participants' physiology, behavior, and function that may be prone to biases and artifacts, e.g., white coat hypertension, and not representative of the data in free-living conditions. Modern digital health technologies equipped with multi-modal sensors combine different data streams to derive comprehensive endpoints that are important to study participants and are clinically meaningful. Used for data collection in clinical trials, they can be deployed as provisioned products where technology is given at study start or in a bring your own \"device\" (BYOD) manner where participants use their technologies to generate study data.</p><p><strong>Summary: </strong>The BYOD option has the potential to be more user-friendly, allowing participants to use technologies that they are familiar with, ensuring better participant compliance, and potentially reducing the bias that comes with introducing new technologies. However, this approach presents different technical, operational, regulatory, and ethical challenges to study teams. For example, BYOD data can be more heterogeneous, and recruiting historically underrepresented populations with limited access to technology and the internet can be challenging. Despite the rapid increase in digital health technologies for clinical and healthcare research, BYOD use in clinical trials is limited, and regulatory guidance is still evolving.</p><p><strong>Key messages: </strong>We offer considerations for academic researchers, drug developers, and patient advocacy organizations on the design and deployment of BYOD models in clinical research. These considerations address: (1) early identification and engagement with internal and external stakeholders; (2) study design including informed consent and recruitment strategies; (3) outcome, endpoint, and technology selection; (4) data management including compliance and data monitoring; (5) statistical considerations to meet regulatory requirements. We believe that this article acts as a primer, providing insights into study design and operational requirements to ensure the successful implementation of BYOD clinical studies.</p>","PeriodicalId":11242,"journal":{"name":"Digital Biomarkers","volume":" ","pages":"47-60"},"PeriodicalIF":0.0,"publicationDate":"2022-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/c6/02/dib-0006-0047.PMC9294934.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"40616532","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-06-29eCollection Date: 2022-05-01DOI: 10.1159/000525255
Hiromasa Mori, Stig Johan Wiklund, Jason Yuren Zhang
Introduction: Digital biomarkers have significant potential to transform drug development, but only a few have contributed meaningfully to bring new treatments to market. There are uncertainties in how they will generate quantifiable benefits in clinical trial performance and ultimately to the chances of phase 3 success. Here we have proposed a statistical framework and ran a proof-of-concept model with hypothetical digital biomarkers and visualized them in a familiar manner to study power calculation.
Methods: A Monte Carlo simulation for Parkinson's disease (PD) was performed using the Captario SUM® platform and illustrative study technology impact calculations were generated. We took inspiration from the EMA-qualified wearable-derived digital endpoint stride velocity 95th centile (SV95C) for Duchenne muscular dystrophy, and we imagined a similar measurement for PD would be available in the future. DaTscan enrichment and "SV95C-like" endpoint biomarkers were assumed on a hypothetical disease-modifying drug pivotal trial aiming for an 80% probability of achieving a study p value of less than 0.05.
Results: Four scenarios with different combinations of technologies were illustrated. The model illustrated a way to quantify the magnitude of the contributions that enrichment and endpoint technologies could make to drug development studies.
Discussion/conclusion: Quantitative models could be valuable not only for the study sponsors but also as an interactive and collaborative engagement tool for technology players and multi-stakeholder consortia. Establishing values of digital biomarkers could also facilitate business cases and financial investments.
{"title":"Quantifying the Benefits of Digital Biomarkers and Technology-Based Study Endpoints in Clinical Trials: Project Moneyball.","authors":"Hiromasa Mori, Stig Johan Wiklund, Jason Yuren Zhang","doi":"10.1159/000525255","DOIUrl":"https://doi.org/10.1159/000525255","url":null,"abstract":"<p><strong>Introduction: </strong>Digital biomarkers have significant potential to transform drug development, but only a few have contributed meaningfully to bring new treatments to market. There are uncertainties in how they will generate quantifiable benefits in clinical trial performance and ultimately to the chances of phase 3 success. Here we have proposed a statistical framework and ran a proof-of-concept model with hypothetical digital biomarkers and visualized them in a familiar manner to study power calculation.</p><p><strong>Methods: </strong>A Monte Carlo simulation for Parkinson's disease (PD) was performed using the Captario SUM® platform and illustrative study technology impact calculations were generated. We took inspiration from the EMA-qualified wearable-derived digital endpoint stride velocity 95<sup>th</sup> centile (SV95C) for Duchenne muscular dystrophy, and we imagined a similar measurement for PD would be available in the future. DaTscan enrichment and \"SV95C-like\" endpoint biomarkers were assumed on a hypothetical disease-modifying drug pivotal trial aiming for an 80% probability of achieving a study <i>p</i> value of less than 0.05.</p><p><strong>Results: </strong>Four scenarios with different combinations of technologies were illustrated. The model illustrated a way to quantify the magnitude of the contributions that enrichment and endpoint technologies could make to drug development studies.</p><p><strong>Discussion/conclusion: </strong>Quantitative models could be valuable not only for the study sponsors but also as an interactive and collaborative engagement tool for technology players and multi-stakeholder consortia. Establishing values of digital biomarkers could also facilitate business cases and financial investments.</p>","PeriodicalId":11242,"journal":{"name":"Digital Biomarkers","volume":" ","pages":"36-46"},"PeriodicalIF":0.0,"publicationDate":"2022-06-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/6a/e0/dib-0006-0036.PMC9297703.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"40616533","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-06-08eCollection Date: 2022-05-01DOI: 10.1159/000525061
Thomas Hügle, Leo Caratsch, Matteo Caorsi, Jules Maglione, Diana Dan, Alexandre Dumusc, Marc Blanchard, Gabriel Kalweit, Maria Kalweit
Digital biomarkers such as wearables are of increasing interest in monitoring rheumatic diseases, but they usually lack disease specificity. In this study, we apply convolutional neural networks (CNN) to real-world hand photographs in order to automatically detect, extract, and analyse dorsal finger fold lines as a correlate of proximal interphalangeal (PIP) joint swelling in patients with rheumatoid arthritis (RA). Hand photographs of RA patients were taken by a smartphone camera in a standardized manner. Overall, 190 PIP joints were categorized as either swollen or not swollen based on clinical judgement and ultrasound. Images were automatically preprocessed by cropping PIP joints and extracting dorsal finger folds. Subsequently, metrical analysis of dorsal finger folds was performed, and a CNN was trained to classify the dorsal finger lines into swollen versus non-swollen joints. Representative horizontal finger folds were also quantified in a subset of patients before and after resolution of PIP swelling and in patients with disease flares. In swollen joints, the number of automatically extracted deep skinfold imprints was significantly reduced compared to non-swollen joints (1.3, SD 0.8 vs. 3.3, SD 0.49, p < 0.01). The joint diameter/deep skinfold length ratio was significantly higher in swollen (4.1, SD 1.4) versus non-swollen joints (2.1, SD 0.6, p < 0.01). The CNN model successfully differentiated swollen from non-swollen joints based on finger fold patterns with a validation accuracy of 0.84, a sensitivity of 88%, and a specificity of 75%. A heatmap of the original images obtained by an extraction algorithm confirmed finger folds as the region of interest for correct classification. After significant response to disease-modifying antirheumatic drug ± corticosteroid therapy, longitudinal metrical analysis of eight representative deep finger folds showed a decrease in the mean diameter/finger fold length (finger fold index, FFI) from 3.03 (SD 0.68) to 2.08 (SD 0.57). Conversely, the FFI increased in patients with disease flares. In conclusion, automated preprocessing and the application of CNN algorithms in combination with longitudinal metrical analysis of dorsal finger fold patterns extracted from real-world hand photos might serve as a digital biomarker in RA.
可穿戴设备等数字生物标志物在监测风湿病方面越来越受关注,但它们通常缺乏疾病特异性。在这项研究中,我们将卷积神经网络(CNN)应用于现实世界的手部照片,以便自动检测、提取和分析类风湿关节炎(RA)患者近端指间关节肿胀与手指背襞线的相关性。采用智能手机相机对RA患者进行标准化的手拍。总体而言,根据临床判断和超声检查,190个PIP关节分为肿胀或不肿胀。通过裁剪PIP关节和提取手指背襞对图像进行自动预处理。随后,对手指背襞进行测量分析,并训练CNN将手指背线分为肿胀关节和非肿胀关节。在PIP肿胀消退前后和疾病发作患者的一个亚组中,代表性的水平指襞也被量化。在肿胀关节中,与非肿胀关节相比,自动提取的深度皮褶印迹数量显著减少(1.3,SD 0.8 vs. 3.3, SD 0.49, p < 0.01)。肿胀的关节直径/深皮褶长度比(4.1,SD 1.4)明显高于非肿胀的关节(2.1,SD 0.6, p < 0.01)。CNN模型基于手指褶皱模式成功区分了肿胀和非肿胀关节,验证准确率为0.84,灵敏度为88%,特异性为75%。通过提取算法获得的原始图像热图确认手指褶皱是正确分类的兴趣区域。在接受抗风湿药物治疗和皮质类固醇治疗后,8个具有代表性的深指沟纵向测量分析显示,平均直径/指沟长度(指沟指数,FFI)从3.03 (SD 0.68)下降到2.08 (SD 0.57)。相反,疾病发作患者的FFI增加。综上所述,自动预处理和应用CNN算法结合对真实手照中提取的指背褶皱进行纵向测量分析,可能作为RA的数字生物标志物。
{"title":"Dorsal Finger Fold Recognition by Convolutional Neural Networks for the Detection and Monitoring of Joint Swelling in Patients with Rheumatoid Arthritis.","authors":"Thomas Hügle, Leo Caratsch, Matteo Caorsi, Jules Maglione, Diana Dan, Alexandre Dumusc, Marc Blanchard, Gabriel Kalweit, Maria Kalweit","doi":"10.1159/000525061","DOIUrl":"https://doi.org/10.1159/000525061","url":null,"abstract":"<p><p>Digital biomarkers such as wearables are of increasing interest in monitoring rheumatic diseases, but they usually lack disease specificity. In this study, we apply convolutional neural networks (CNN) to real-world hand photographs in order to automatically detect, extract, and analyse dorsal finger fold lines as a correlate of proximal interphalangeal (PIP) joint swelling in patients with rheumatoid arthritis (RA). Hand photographs of RA patients were taken by a smartphone camera in a standardized manner. Overall, 190 PIP joints were categorized as either swollen or not swollen based on clinical judgement and ultrasound. Images were automatically preprocessed by cropping PIP joints and extracting dorsal finger folds. Subsequently, metrical analysis of dorsal finger folds was performed, and a CNN was trained to classify the dorsal finger lines into swollen versus non-swollen joints. Representative horizontal finger folds were also quantified in a subset of patients before and after resolution of PIP swelling and in patients with disease flares. In swollen joints, the number of automatically extracted deep skinfold imprints was significantly reduced compared to non-swollen joints (1.3, SD 0.8 vs. 3.3, SD 0.49, <i>p</i> < 0.01). The joint diameter/deep skinfold length ratio was significantly higher in swollen (4.1, SD 1.4) versus non-swollen joints (2.1, SD 0.6, <i>p</i> < 0.01). The CNN model successfully differentiated swollen from non-swollen joints based on finger fold patterns with a validation accuracy of 0.84, a sensitivity of 88%, and a specificity of 75%. A heatmap of the original images obtained by an extraction algorithm confirmed finger folds as the region of interest for correct classification. After significant response to disease-modifying antirheumatic drug ± corticosteroid therapy, longitudinal metrical analysis of eight representative deep finger folds showed a decrease in the mean diameter/finger fold length (finger fold index, FFI) from 3.03 (SD 0.68) to 2.08 (SD 0.57). Conversely, the FFI increased in patients with disease flares. In conclusion, automated preprocessing and the application of CNN algorithms in combination with longitudinal metrical analysis of dorsal finger fold patterns extracted from real-world hand photos might serve as a digital biomarker in RA.</p>","PeriodicalId":11242,"journal":{"name":"Digital Biomarkers","volume":" ","pages":"31-35"},"PeriodicalIF":0.0,"publicationDate":"2022-06-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9247561/pdf/dib-0006-0031.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"40616536","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Daniel S Rubin, Sylvia L Ranjeva, Jacek K Urbanek, Marta Karas, Maria Lucia L Madariaga, Megan Huisingh-Scheetz
Background: Functional capacity assessment is a critical step in the preoperative evaluation to identify patients at increased risk of cardiac complications and disability after major noncardiac surgery. Smartphones offer the potential to objectively measure functional capacity but are limited by inaccuracy in patients with poor functional capacity. Open-source methods exist to analyze accelerometer data to estimate gait cadence (steps/min), which is directly associated with activity intensity. Here, we used an updated Step Test smartphone application with an open-source method to analyze accelerometer data to estimate gait cadence and functional capacity in older adults.
Methods: We performed a prospective observational cohort study within the Frailty, Activity, Body Composition and Energy Expenditure in Aging study at the University of Chicago. Participants completed the Duke Activity Status Index (DASI) and performed an in-clinic 6-min walk test (6MWT) while using the Step Test application on a study smartphone. Gait cadence was measured from the raw accelerometer data using an adaptive empirical pattern transformation method, which has been previously validated. A 6MWT distance of 370 m was used as an objective threshold to identify patients at high risk. We performed multivariable logistic regression to predict walking distance using a priori explanatory variables.
Results: Sixty patients were enrolled in the study. Thirty-seven patients completed the protocol and were included in the final data analysis. The median (IQR) age of the overall cohort was 71 (69-74) years, with a body mass index of 31 (27-32). There were no differences in any clinical characteristics or functional measures between participants that were able to walk 370 m during the 6MWT and those that could not walk that distance. Median (IQR) gait cadence for the entire cohort was 110 (102-114) steps/min during the 6MWT. Median (IQR) gait cadence was higher in participants that walked more than 370 m during the 6MWT 112 (108-118) versus 106 (96-114) steps/min; p = 0.0157). The final multivariable model to identify participants that could not walk 370 m included only median gait cadence. The Youden's index cut-point was 107 steps/min with a sensitivity of 0.81 (95% CI: 0.77, 0.85) and a specificity of 0.57 (95% CI: 0.55, 0.59) and an AUCROC of 0.69 (95% CI: 0.51, 0.87).
Conclusions: Our pilot study demonstrates the feasibility of using gait cadence as a measure to estimate functional capacity. Our study was limited by a smaller than expected sample size due to COVID-19, and thus, a prospective study with preoperative patients that measures outcomes is necessary to validate our findings.
{"title":"Smartphone-Based Gait Cadence to Identify Older Adults with Decreased Functional Capacity.","authors":"Daniel S Rubin, Sylvia L Ranjeva, Jacek K Urbanek, Marta Karas, Maria Lucia L Madariaga, Megan Huisingh-Scheetz","doi":"10.1159/000525344","DOIUrl":"https://doi.org/10.1159/000525344","url":null,"abstract":"<p><strong>Background: </strong>Functional capacity assessment is a critical step in the preoperative evaluation to identify patients at increased risk of cardiac complications and disability after major noncardiac surgery. Smartphones offer the potential to objectively measure functional capacity but are limited by inaccuracy in patients with poor functional capacity. Open-source methods exist to analyze accelerometer data to estimate gait cadence (steps/min), which is directly associated with activity intensity. Here, we used an updated Step Test smartphone application with an open-source method to analyze accelerometer data to estimate gait cadence and functional capacity in older adults.</p><p><strong>Methods: </strong>We performed a prospective observational cohort study within the Frailty, Activity, Body Composition and Energy Expenditure in Aging study at the University of Chicago. Participants completed the Duke Activity Status Index (DASI) and performed an in-clinic 6-min walk test (6MWT) while using the Step Test application on a study smartphone. Gait cadence was measured from the raw accelerometer data using an adaptive empirical pattern transformation method, which has been previously validated. A 6MWT distance of 370 m was used as an objective threshold to identify patients at high risk. We performed multivariable logistic regression to predict walking distance using a priori explanatory variables.</p><p><strong>Results: </strong>Sixty patients were enrolled in the study. Thirty-seven patients completed the protocol and were included in the final data analysis. The median (IQR) age of the overall cohort was 71 (69-74) years, with a body mass index of 31 (27-32). There were no differences in any clinical characteristics or functional measures between participants that were able to walk 370 m during the 6MWT and those that could not walk that distance. Median (IQR) gait cadence for the entire cohort was 110 (102-114) steps/min during the 6MWT. Median (IQR) gait cadence was higher in participants that walked more than 370 m during the 6MWT 112 (108-118) versus 106 (96-114) steps/min; <i>p</i> = 0.0157). The final multivariable model to identify participants that could not walk 370 m included only median gait cadence. The Youden's index cut-point was 107 steps/min with a sensitivity of 0.81 (95% CI: 0.77, 0.85) and a specificity of 0.57 (95% CI: 0.55, 0.59) and an AUCROC of 0.69 (95% CI: 0.51, 0.87).</p><p><strong>Conclusions: </strong>Our pilot study demonstrates the feasibility of using gait cadence as a measure to estimate functional capacity. Our study was limited by a smaller than expected sample size due to COVID-19, and thus, a prospective study with preoperative patients that measures outcomes is necessary to validate our findings.</p>","PeriodicalId":11242,"journal":{"name":"Digital Biomarkers","volume":"6 2","pages":"61-70"},"PeriodicalIF":0.0,"publicationDate":"2022-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/3a/3b/dib-0006-0061.PMC9386413.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10401207","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-03-31eCollection Date: 2022-01-01DOI: 10.1159/000522185
Janine M Knijff, Euphemia C A M Houdijk, Daniëlle C M van der Kaay, Youri van Berkel, Luc Filippini, Frederik E Stuurman, Adam F Cohen, Gertjan J A Driessen, Matthijs D Kruizinga
Introduction: Clinical research and treatment of childhood obesity is challenging, and objective biomarkers obtained in a home-setting are needed. The aim of this study was to determine the potential of novel digital endpoints gathered by a home-monitoring platform in pediatric obesity.
Methods: In this prospective observational study, 28 children with obesity aged 6-16 years were included and monitored for 28 days. Patients wore a smartwatch, which measured physical activity (PA), heart rate (HR), and sleep. Furthermore, daily blood pressure (BP) measurements were performed. Data from 128 healthy children were utilized for comparison. Differences between patients and controls were assessed via linear mixed effect models.
Results: Data from 28 patients (average age 11.6 years, 46% male, average body mass index 30.9) and 128 controls (average age 11.1 years, 46% male, average body mass index 18.0) were analyzed. Patients were recruited between November 2018 and February 2020. For patients, the median compliance for the measurements ranged from 55% to 100% and the highest median compliance was observed for the smartwatch-related measurements (81-100%). Patients had a lower daily PA level (4,597 steps vs. 6,081 steps, 95% confidence interval [CI] 862-2,108) and peak PA level (1,115 steps vs. 1,392 steps, 95% CI 136-417), a higher nighttime HR (81 bpm vs. 71 bpm, 95% CI 6.3-12.3) and daytime HR (98 bpm vs. 88 bpm, 95% CI 7.6-12.6), a higher systolic BP (115 mm Hg vs. 104 mm Hg, 95% CI 8.1-14.5) and diastolic BP (76 mm Hg vs. 65 mm Hg, 95% CI 8.7-12.7), and a shorter sleep duration (difference 0.5 h, 95% CI 0.2-0.7) compared to controls.
Conclusion: Remote monitoring via wearables in pediatric obesity has the potential to objectively measure the disease burden in the home-setting. The novel endpoints demonstrate significant differences in PA level, HR, BP, and sleep duration between patients and controls. Future studies are needed to determine the capacity of the novel digital endpoints to detect effect of interventions.
{"title":"Objective Home-Monitoring of Physical Activity, Cardiovascular Parameters, and Sleep in Pediatric Obesity.","authors":"Janine M Knijff, Euphemia C A M Houdijk, Daniëlle C M van der Kaay, Youri van Berkel, Luc Filippini, Frederik E Stuurman, Adam F Cohen, Gertjan J A Driessen, Matthijs D Kruizinga","doi":"10.1159/000522185","DOIUrl":"https://doi.org/10.1159/000522185","url":null,"abstract":"<p><strong>Introduction: </strong>Clinical research and treatment of childhood obesity is challenging, and objective biomarkers obtained in a home-setting are needed. The aim of this study was to determine the potential of novel digital endpoints gathered by a home-monitoring platform in pediatric obesity.</p><p><strong>Methods: </strong>In this prospective observational study, 28 children with obesity aged 6-16 years were included and monitored for 28 days. Patients wore a smartwatch, which measured physical activity (PA), heart rate (HR), and sleep. Furthermore, daily blood pressure (BP) measurements were performed. Data from 128 healthy children were utilized for comparison. Differences between patients and controls were assessed via linear mixed effect models.</p><p><strong>Results: </strong>Data from 28 patients (average age 11.6 years, 46% male, average body mass index 30.9) and 128 controls (average age 11.1 years, 46% male, average body mass index 18.0) were analyzed. Patients were recruited between November 2018 and February 2020. For patients, the median compliance for the measurements ranged from 55% to 100% and the highest median compliance was observed for the smartwatch-related measurements (81-100%). Patients had a lower daily PA level (4,597 steps vs. 6,081 steps, 95% confidence interval [CI] 862-2,108) and peak PA level (1,115 steps vs. 1,392 steps, 95% CI 136-417), a higher nighttime HR (81 bpm vs. 71 bpm, 95% CI 6.3-12.3) and daytime HR (98 bpm vs. 88 bpm, 95% CI 7.6-12.6), a higher systolic BP (115 mm Hg vs. 104 mm Hg, 95% CI 8.1-14.5) and diastolic BP (76 mm Hg vs. 65 mm Hg, 95% CI 8.7-12.7), and a shorter sleep duration (difference 0.5 h, 95% CI 0.2-0.7) compared to controls.</p><p><strong>Conclusion: </strong>Remote monitoring via wearables in pediatric obesity has the potential to objectively measure the disease burden in the home-setting. The novel endpoints demonstrate significant differences in PA level, HR, BP, and sleep duration between patients and controls. Future studies are needed to determine the capacity of the novel digital endpoints to detect effect of interventions.</p>","PeriodicalId":11242,"journal":{"name":"Digital Biomarkers","volume":"6 1","pages":"19-29"},"PeriodicalIF":0.0,"publicationDate":"2022-03-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9301882/pdf/dib-0006-0019.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"33478861","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
L. Lonini, Y. Moon, Kyle R. Embry, R. Cotton, K. McKenzie, Sophia Jenz, A. Jayaraman
Recent advancements in deep learning have produced significant progress in markerless human pose estimation, making it possible to estimate human kinematics from single camera videos without the need for reflective markers and specialized labs equipped with motion capture systems. Such algorithms have the potential to enable the quantification of clinical metrics from videos recorded with a handheld camera. Here we used DeepLabCut, an open-source framework for markerless pose estimation, to fine-tune a deep network to track 5 body keypoints (hip, knee, ankle, heel, and toe) in 82 below-waist videos of 8 patients with stroke performing overground walking during clinical assessments. We trained the pose estimation model by labeling the keypoints in 2 frames per video and then trained a convolutional neural network to estimate 5 clinically relevant gait parameters (cadence, double support time, swing time, stance time, and walking speed) from the trajectory of these keypoints. These results were then compared to those obtained from a clinical system for gait analysis (GAITRite®, CIR Systems). Absolute accuracy (mean error) and precision (standard deviation of error) for swing, stance, and double support time were within 0.04 ± 0.11 s; Pearson’s correlation with the reference system was moderate for swing times (r = 0.4–0.66), but stronger for stance and double support time (r = 0.93–0.95). Cadence mean error was −0.25 steps/min ± 3.9 steps/min (r = 0.97), while walking speed mean error was −0.02 ± 0.11 m/s (r = 0.92). These preliminary results suggest that single camera videos and pose estimation models based on deep networks could be used to quantify clinically relevant gait metrics in individuals poststroke, even while using assistive devices in uncontrolled environments. Such development opens the door to applications for gait analysis both inside and outside of clinical settings, without the need of sophisticated equipment.
{"title":"Video-Based Pose Estimation for Gait Analysis in Stroke Survivors during Clinical Assessments: A Proof-of-Concept Study","authors":"L. Lonini, Y. Moon, Kyle R. Embry, R. Cotton, K. McKenzie, Sophia Jenz, A. Jayaraman","doi":"10.1159/000520732","DOIUrl":"https://doi.org/10.1159/000520732","url":null,"abstract":"Recent advancements in deep learning have produced significant progress in markerless human pose estimation, making it possible to estimate human kinematics from single camera videos without the need for reflective markers and specialized labs equipped with motion capture systems. Such algorithms have the potential to enable the quantification of clinical metrics from videos recorded with a handheld camera. Here we used DeepLabCut, an open-source framework for markerless pose estimation, to fine-tune a deep network to track 5 body keypoints (hip, knee, ankle, heel, and toe) in 82 below-waist videos of 8 patients with stroke performing overground walking during clinical assessments. We trained the pose estimation model by labeling the keypoints in 2 frames per video and then trained a convolutional neural network to estimate 5 clinically relevant gait parameters (cadence, double support time, swing time, stance time, and walking speed) from the trajectory of these keypoints. These results were then compared to those obtained from a clinical system for gait analysis (GAITRite®, CIR Systems). Absolute accuracy (mean error) and precision (standard deviation of error) for swing, stance, and double support time were within 0.04 ± 0.11 s; Pearson’s correlation with the reference system was moderate for swing times (r = 0.4–0.66), but stronger for stance and double support time (r = 0.93–0.95). Cadence mean error was −0.25 steps/min ± 3.9 steps/min (r = 0.97), while walking speed mean error was −0.02 ± 0.11 m/s (r = 0.92). These preliminary results suggest that single camera videos and pose estimation models based on deep networks could be used to quantify clinically relevant gait metrics in individuals poststroke, even while using assistive devices in uncontrolled environments. Such development opens the door to applications for gait analysis both inside and outside of clinical settings, without the need of sophisticated equipment.","PeriodicalId":11242,"journal":{"name":"Digital Biomarkers","volume":"6 1","pages":"9 - 18"},"PeriodicalIF":0.0,"publicationDate":"2022-01-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48700761","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A. Page, Norman C W Yung, P. Auinger, C. Venuto, Alistair Glidden, E. Macklin, L. Omberg, M. Schwarzschild, E. Dorsey
Background: Smartphones can generate objective measures of Parkinson’s disease (PD) and supplement traditional in-person rating scales. However, smartphone use in clinical trials has been limited. Objective: This study aimed to determine the feasibility of introducing a smartphone research application into a PD clinical trial and to evaluate the resulting measures. Methods: A smartphone application was introduced part-way into a phase 3 randomized clinical trial of inosine. The application included finger tapping, gait, and cognition tests, and participants were asked to complete an assessment battery at home and in clinic alongside the Movement Disorder Society-Unified Parkinson’s Disease Rating Scale (MDS-UPDRS). Results: Of 236 eligible participants in the parent study, 88 (37%) consented to participate, and 59 (27 randomized to inosine and 32 to placebo) completed a baseline smartphone assessment. These 59 participants collectively completed 1,292 batteries of assessments. The proportion of participants who completed at least one smartphone assessment was 61% at 3, 54% at 6, and 35% at 12 months. Finger tapping speed correlated weakly with the part III motor portion (r = −0.16, left hand; r = −0.04, right hand) and total (r = −0.14) MDS-UPDRS. Gait speed correlated better with the same measures (r = −0.25, part III motor; r = −0.34, total). Over 6 months, finger tapping speed, gait speed, and memory scores did not differ between those randomized to active drug or placebo. Conclusions: Introducing a smartphone application midway into a phase 3 clinical trial was challenging. Measures of bradykinesia and gait speed correlated modestly with traditional outcomes and were consistent with the study’s overall findings, which found no benefit of the active drug.
{"title":"A Smartphone Application as an Exploratory Endpoint in a Phase 3 Parkinson’s Disease Clinical Trial: A Pilot Study","authors":"A. Page, Norman C W Yung, P. Auinger, C. Venuto, Alistair Glidden, E. Macklin, L. Omberg, M. Schwarzschild, E. Dorsey","doi":"10.1159/000521232","DOIUrl":"https://doi.org/10.1159/000521232","url":null,"abstract":"Background: Smartphones can generate objective measures of Parkinson’s disease (PD) and supplement traditional in-person rating scales. However, smartphone use in clinical trials has been limited. Objective: This study aimed to determine the feasibility of introducing a smartphone research application into a PD clinical trial and to evaluate the resulting measures. Methods: A smartphone application was introduced part-way into a phase 3 randomized clinical trial of inosine. The application included finger tapping, gait, and cognition tests, and participants were asked to complete an assessment battery at home and in clinic alongside the Movement Disorder Society-Unified Parkinson’s Disease Rating Scale (MDS-UPDRS). Results: Of 236 eligible participants in the parent study, 88 (37%) consented to participate, and 59 (27 randomized to inosine and 32 to placebo) completed a baseline smartphone assessment. These 59 participants collectively completed 1,292 batteries of assessments. The proportion of participants who completed at least one smartphone assessment was 61% at 3, 54% at 6, and 35% at 12 months. Finger tapping speed correlated weakly with the part III motor portion (r = −0.16, left hand; r = −0.04, right hand) and total (r = −0.14) MDS-UPDRS. Gait speed correlated better with the same measures (r = −0.25, part III motor; r = −0.34, total). Over 6 months, finger tapping speed, gait speed, and memory scores did not differ between those randomized to active drug or placebo. Conclusions: Introducing a smartphone application midway into a phase 3 clinical trial was challenging. Measures of bradykinesia and gait speed correlated modestly with traditional outcomes and were consistent with the study’s overall findings, which found no benefit of the active drug.","PeriodicalId":11242,"journal":{"name":"Digital Biomarkers","volume":"6 1","pages":"1 - 8"},"PeriodicalIF":0.0,"publicationDate":"2022-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44824532","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-13eCollection Date: 2021-09-01DOI: 10.1159/000517885
Matthew Landers, Ray Dorsey, Suchi Saria
The assessment of health and disease requires a set of criteria to define health status and progression. These health measures are referred to as "endpoints." A "digital endpoint" is defined by its use of sensor-generated data often collected outside of a clinical setting such as in a patient's free-living environment. Applicable sensors exist in an array of devices and can be applied in a diverse set of contexts. For example, a smartphone's microphone might be used to diagnose or predict mild cognitive impairment due to Alzheimer's disease or a wrist-worn activity monitor (such as those found in smartwatches) may be used to measure a drug's effect on the nocturnal activity of patients with sickle cell disease. Digital endpoints are generating considerable excitement because they permit a more authentic assessment of the patient's experience, reveal formerly untold realities of disease burden, and can cut drug discovery costs in half. However, before these benefits can be realized, effort must be applied not only to the technical creation of digital endpoints but also to the environment that allows for their development and application. The future of digital endpoints rests on meaningful interdisciplinary collaboration, sufficient evidence that digital endpoints can realize their promise, and the development of an ecosystem in which the vast quantities of data that digital endpoints generate can be analyzed. The fundamental nature of health care is changing. With coronavirus disease 2019 serving as a catalyst, there has been a rapid expansion of home care models, telehealth, and remote patient monitoring. The increasing adoption of these health-care innovations will expedite the requirement for a digital characterization of clinical status as current assessment tools often rely upon direct interaction with patients and thus are not fit for purpose to be administered remotely. With the ubiquity of relatively inexpensive sensors, digital endpoints are positioned to drive this consequential change. It is therefore not surprising that regulators, physicians, researchers, and consultants have each offered their assessment of these novel tools. However, as we further describe later, the broad adoption of digital endpoints will require a cooperative effort. In this article, we present an analysis of the current state of digital endpoints. We also attempt to unify the perspectives of the parties involved in the development and deployment of these tools. We conclude with an interdependent list of challenges that must be collaboratively addressed before these endpoints are widely adopted.
{"title":"Digital Endpoints: Definition, Benefits, and Current Barriers in Accelerating Development and Adoption.","authors":"Matthew Landers, Ray Dorsey, Suchi Saria","doi":"10.1159/000517885","DOIUrl":"https://doi.org/10.1159/000517885","url":null,"abstract":"<p><p>The assessment of health and disease requires a set of criteria to define health status and progression. These health measures are referred to as \"endpoints.\" A \"digital endpoint\" is defined by its use of sensor-generated data often collected outside of a clinical setting such as in a patient's free-living environment. Applicable sensors exist in an array of devices and can be applied in a diverse set of contexts. For example, a smartphone's microphone might be used to diagnose or predict mild cognitive impairment due to Alzheimer's disease or a wrist-worn activity monitor (such as those found in smartwatches) may be used to measure a drug's effect on the nocturnal activity of patients with sickle cell disease. Digital endpoints are generating considerable excitement because they permit a more authentic assessment of the patient's experience, reveal formerly untold realities of disease burden, and can cut drug discovery costs in half. However, before these benefits can be realized, effort must be applied not only to the technical creation of digital endpoints but also to the environment that allows for their development and application. The future of digital endpoints rests on meaningful interdisciplinary collaboration, sufficient evidence that digital endpoints can realize their promise, and the development of an ecosystem in which the vast quantities of data that digital endpoints generate can be analyzed. The fundamental nature of health care is changing. With coronavirus disease 2019 serving as a catalyst, there has been a rapid expansion of home care models, telehealth, and remote patient monitoring. The increasing adoption of these health-care innovations will expedite the requirement for a digital characterization of clinical status as current assessment tools often rely upon direct interaction with patients and thus are not fit for purpose to be administered remotely. With the ubiquity of relatively inexpensive sensors, digital endpoints are positioned to drive this consequential change. It is therefore not surprising that regulators, physicians, researchers, and consultants have each offered their assessment of these novel tools. However, as we further describe later, the broad adoption of digital endpoints will require a cooperative effort. In this article, we present an analysis of the current state of digital endpoints. We also attempt to unify the perspectives of the parties involved in the development and deployment of these tools. We conclude with an interdependent list of challenges that must be collaboratively addressed before these endpoints are widely adopted.</p>","PeriodicalId":11242,"journal":{"name":"Digital Biomarkers","volume":"5 3","pages":"216-223"},"PeriodicalIF":0.0,"publicationDate":"2021-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8490914/pdf/dib-0005-0216.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"39570045","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}