Ayyuce Begum Bektas, Lynn Hakki, Asama Khan, Maria Widmar, Iris H Wei, Emmanouil Pappou, J Joshua Smith, Garrett M Nash, Philip B Paty, Julio Garcia-Aguilar, Andrea Cercek, Zsofia Stadler, Neil H Segal, Jinru Shia, Mithat Gonen, Martin R Weiser
Purpose: Outcome for patients with nonmetastatic, microsatellite instability (MSI) colon cancer is favorable: however, high-risk cohorts exist. This study was aimed at developing and validating a nomogram model to predict freedom from recurrence (FFR) for patients with resected MSI colon cancer.
Patients and methods: Data from patients who underwent curative resection of stage I, II, or III MSI colon cancer in 2014-2021 (model training cohort, 384 patients, 33 events; median follow-up, 38.8 months) were retrospectively collected from institutional databases. Variables associated with recurrence in multivariable analysis were selected for inclusion in the clinical calculator. The calculator's predictive accuracy was measured with the concordance index and validated using data from patients who underwent treatment for MSI colon cancer in 2007-2013 (validation cohort, 164 patients, eight events; median follow-up, 84.8 months).
Results: T category and number of positive lymph nodes were significantly associated with recurrence in multivariable analysis and were selected for inclusion in the clinical calculator. The calculator's concordance index for FFR in the model training cohort was 0.812 (95% CI, 0.742 to 0.873), compared with 0.759 (95% CI, 0.683 to 0.840) for the staging schema of the eighth edition of the American Joint Committee on Cancer Staging Manual. The concordance index for the validation cohort was 0.744 (95% CI, 0.666 to 0.822), confirming robust predictive accuracy.
Conclusion: Although in general patients with nonmetastatic MSI colon cancer had favorable outcome, patients with advanced T category and multiple metastatic lymph nodes had higher risk of recurrence. The clinical calculator identified patients with MSI colon cancer at high risk for recurrence, and this could inform surveillance strategies. In addition, the model could be used in trial design to identify patients suitable for novel adjuvant therapy.
{"title":"Clinical Calculator for Predicting Freedom From Recurrence After Resection of Stage I-III Colon Cancer in Patients With Microsatellite Instability.","authors":"Ayyuce Begum Bektas, Lynn Hakki, Asama Khan, Maria Widmar, Iris H Wei, Emmanouil Pappou, J Joshua Smith, Garrett M Nash, Philip B Paty, Julio Garcia-Aguilar, Andrea Cercek, Zsofia Stadler, Neil H Segal, Jinru Shia, Mithat Gonen, Martin R Weiser","doi":"10.1200/CCI.23.00233","DOIUrl":"10.1200/CCI.23.00233","url":null,"abstract":"<p><strong>Purpose: </strong>Outcome for patients with nonmetastatic, microsatellite instability (MSI) colon cancer is favorable: however, high-risk cohorts exist. This study was aimed at developing and validating a nomogram model to predict freedom from recurrence (FFR) for patients with resected MSI colon cancer.</p><p><strong>Patients and methods: </strong>Data from patients who underwent curative resection of stage I, II, or III MSI colon cancer in 2014-2021 (model training cohort, 384 patients, 33 events; median follow-up, 38.8 months) were retrospectively collected from institutional databases. Variables associated with recurrence in multivariable analysis were selected for inclusion in the clinical calculator. The calculator's predictive accuracy was measured with the concordance index and validated using data from patients who underwent treatment for MSI colon cancer in 2007-2013 (validation cohort, 164 patients, eight events; median follow-up, 84.8 months).</p><p><strong>Results: </strong>T category and number of positive lymph nodes were significantly associated with recurrence in multivariable analysis and were selected for inclusion in the clinical calculator. The calculator's concordance index for FFR in the model training cohort was 0.812 (95% CI, 0.742 to 0.873), compared with 0.759 (95% CI, 0.683 to 0.840) for the staging schema of the eighth edition of the American Joint Committee on Cancer Staging Manual. The concordance index for the validation cohort was 0.744 (95% CI, 0.666 to 0.822), confirming robust predictive accuracy.</p><p><strong>Conclusion: </strong>Although in general patients with nonmetastatic MSI colon cancer had favorable outcome, patients with advanced T category and multiple metastatic lymph nodes had higher risk of recurrence. The clinical calculator identified patients with MSI colon cancer at high risk for recurrence, and this could inform surveillance strategies. In addition, the model could be used in trial design to identify patients suitable for novel adjuvant therapy.</p>","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"8 ","pages":"e2300233"},"PeriodicalIF":3.3,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11323037/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141910173","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Timothy J Brown, Phyllis A Gimotty, Ronac Mamtani, Thomas B Karasic, Yu-Xiao Yang
Purpose: Systemic therapy with atezolizumab and bevacizumab can extend life for patients with advanced hepatocellular carcinoma (HCC). However, there is substantial variability in response to therapy and overall survival. Although current prognostic models have been validated in HCC, they primarily consider covariates that may be reflective of the severity of the underlying liver disease of patients with HCC. We developed and internally validated a classification and regression tree (CART) to identify patient characteristics associated with risks of early mortality, at or before 6 months from treatment initiation.
Methods: This retrospective cohort study used the nationwide Flatiron Health electronic health record-derived deidentified database and included patients with a diagnosis of HCC after January 1, 2020, who received initial systemic therapy with atezolizumab and bevacizumab. CART was developed from available baseline clinical and demographic information to predict mortality within 6 months from treatment initiation. Model characteristics were compared to the albumin-bilirubin (ALBI) model and was further validated against a contemporary validation cohort of patients after a data update.
Results: A total of 293 patients were analyzed. The CART identified seven cohorts of patients from baseline demographic and laboratory characteristics. The model had an area under the receiver operating curve (AUROC) of 0.739 (95% CI, 0.683 to 0.794) for predicting 6-month mortality. This model was internally valid and performed more favorably than the ALBI model, which had an AUROC of 0.608 (95% CI, 0.557 to 0.660). The model applied to the contemporary validation cohort (n = 111) had an AUROC of 0.666 (95% CI, 0.506 to 0.826).
Conclusion: Using CART, we identified unique cohorts of patients with HCC treated with atezolizumab and bevacizumab with distinct risks of early mortality. This approach outperformed the ALBI model and used clinical and laboratory characteristics that are readily available to oncologists caring for these patients.
{"title":"Classification and Regression Trees to Predict for Survival for Patients With Hepatocellular Carcinoma Treated With Atezolizumab and Bevacizumab.","authors":"Timothy J Brown, Phyllis A Gimotty, Ronac Mamtani, Thomas B Karasic, Yu-Xiao Yang","doi":"10.1200/CCI.23.00220","DOIUrl":"10.1200/CCI.23.00220","url":null,"abstract":"<p><strong>Purpose: </strong>Systemic therapy with atezolizumab and bevacizumab can extend life for patients with advanced hepatocellular carcinoma (HCC). However, there is substantial variability in response to therapy and overall survival. Although current prognostic models have been validated in HCC, they primarily consider covariates that may be reflective of the severity of the underlying liver disease of patients with HCC. We developed and internally validated a classification and regression tree (CART) to identify patient characteristics associated with risks of early mortality, at or before 6 months from treatment initiation.</p><p><strong>Methods: </strong>This retrospective cohort study used the nationwide Flatiron Health electronic health record-derived deidentified database and included patients with a diagnosis of HCC after January 1, 2020, who received initial systemic therapy with atezolizumab and bevacizumab. CART was developed from available baseline clinical and demographic information to predict mortality within 6 months from treatment initiation. Model characteristics were compared to the albumin-bilirubin (ALBI) model and was further validated against a contemporary validation cohort of patients after a data update.</p><p><strong>Results: </strong>A total of 293 patients were analyzed. The CART identified seven cohorts of patients from baseline demographic and laboratory characteristics. The model had an area under the receiver operating curve (AUROC) of 0.739 (95% CI, 0.683 to 0.794) for predicting 6-month mortality. This model was internally valid and performed more favorably than the ALBI model, which had an AUROC of 0.608 (95% CI, 0.557 to 0.660). The model applied to the contemporary validation cohort (n = 111) had an AUROC of 0.666 (95% CI, 0.506 to 0.826).</p><p><strong>Conclusion: </strong>Using CART, we identified unique cohorts of patients with HCC treated with atezolizumab and bevacizumab with distinct risks of early mortality. This approach outperformed the ALBI model and used clinical and laboratory characteristics that are readily available to oncologists caring for these patients.</p>","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"8 ","pages":"e2300220"},"PeriodicalIF":3.3,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11296500/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141876704","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Emergence of Digital Toxicity and the Need for an Integrated, Patient-Centric Approach to the Development, Evaluation, and Use of Digital Health Tools for Oncology.","authors":"Chris Gibbons, Carole Baas, Caroline Chung","doi":"10.1200/CCI.23.00105","DOIUrl":"10.1200/CCI.23.00105","url":null,"abstract":"","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"8 ","pages":"e2300105"},"PeriodicalIF":3.3,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141898874","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ricardo Ahumada, Jocelyn Dunstan, Inti Paredes, Pablo Báez
{"title":"Response to Kempf et al on Methodological and Practical Aspects of a Distant Metastasis Detection Model.","authors":"Ricardo Ahumada, Jocelyn Dunstan, Inti Paredes, Pablo Báez","doi":"10.1200/CCI-24-00154","DOIUrl":"https://doi.org/10.1200/CCI-24-00154","url":null,"abstract":"","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"8 ","pages":"e2400154"},"PeriodicalIF":3.3,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142074532","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Marinde J G Bond, Maarten van Smeden, Koen Degeling, Chiara Cremolini, Hans-Joachim Schmoll, Carlotta Antoniotti, Sara Lonardi, Sabina Murgioni, Daniele Rossini, Stefan Ibach, Miriam Koopman, Rutger-Jan Swijnenburg, Cornelis J A Punt, Anne M May, Johannes J M Kwakman
Purpose: Patient outcomes may differ from randomized trial averages. We aimed to predict benefit from FOLFOXIRI versus infusional fluorouracil, leucovorin, and oxaliplatin/fluorouracil, leucovorin, and irinotecan (FOLFOX/FOLFIRI), both plus bevacizumab, in patients with metastatic colorectal cancer (mCRC).
Methods: A Cox model with prespecified clinical, molecular, and laboratory variables was developed in 639 patients from the TRIBE2 trial for predicting 2-year mortality. Data from the CHARTA (n = 232), TRIBE1 (n = 504), and CAIRO5 (liver-only mCRC, n = 287) trials were used for external validation and heterogeneity of treatment effects (HTE) analysis. This involves categorizing patients into risk groups and assessing treatment effects across these groups. Performance was assessed by the C-index and calibration plots. The C-for-benefit was calculated to assess evidence for HTE. The c-for-benefit is specifically designed for HTE analysis. Like the commonly known c-statistic, it summarizes the discrimination of a model. Values over 0.5 indicate evidence for HTE.
Results: In TRIBE2, the overoptimism-corrected C-index was 0.66 (95% CI, 0.63 to 0.69). At external validation, the C-index was 0.69 (95% CI, 0.64 to 0.75), 0.68 (95% CI, 0.64 to 0.72), and 0.65 (95% CI, 0.65 to 0.66), in CHARTA, TRIBE1, and CAIRO5, respectively. Calibration plots indicated slight underestimation of mortality. The c-for-benefit indicated evidence for HTE in CHARTA (0.56, 95% CI, 0.48 to 0.65), but not in TRIBE1 (0.49, 95% CI, 0.44 to 0.55) and CAIRO5 (0.40, 95% CI, 0.32 to 0.48).
Conclusion: Although 2-year mortality could be reasonably estimated, the HTE analysis showed that clinically available variables did not reliably identify which patients with mCRC benefit from FOLFOXIRI versus FOLFOX/FOLFIRI, both plus bevacizumab, across the three studies.
{"title":"Predicting Benefit From FOLFOXIRI Plus Bevacizumab in Patients With Metastatic Colorectal Cancer.","authors":"Marinde J G Bond, Maarten van Smeden, Koen Degeling, Chiara Cremolini, Hans-Joachim Schmoll, Carlotta Antoniotti, Sara Lonardi, Sabina Murgioni, Daniele Rossini, Stefan Ibach, Miriam Koopman, Rutger-Jan Swijnenburg, Cornelis J A Punt, Anne M May, Johannes J M Kwakman","doi":"10.1200/CCI.24.00037","DOIUrl":"https://doi.org/10.1200/CCI.24.00037","url":null,"abstract":"<p><strong>Purpose: </strong>Patient outcomes may differ from randomized trial averages. We aimed to predict benefit from FOLFOXIRI versus infusional fluorouracil, leucovorin, and oxaliplatin/fluorouracil, leucovorin, and irinotecan (FOLFOX/FOLFIRI), both plus bevacizumab, in patients with metastatic colorectal cancer (mCRC).</p><p><strong>Methods: </strong>A Cox model with prespecified clinical, molecular, and laboratory variables was developed in 639 patients from the TRIBE2 trial for predicting 2-year mortality. Data from the CHARTA (n = 232), TRIBE1 (n = 504), and CAIRO5 (liver-only mCRC, n = 287) trials were used for external validation and heterogeneity of treatment effects (HTE) analysis. This involves categorizing patients into risk groups and assessing treatment effects across these groups. Performance was assessed by the C-index and calibration plots. The C-for-benefit was calculated to assess evidence for HTE. The c-for-benefit is specifically designed for HTE analysis. Like the commonly known c-statistic, it summarizes the discrimination of a model. Values over 0.5 indicate evidence for HTE.</p><p><strong>Results: </strong>In TRIBE2, the overoptimism-corrected C-index was 0.66 (95% CI, 0.63 to 0.69). At external validation, the C-index was 0.69 (95% CI, 0.64 to 0.75), 0.68 (95% CI, 0.64 to 0.72), and 0.65 (95% CI, 0.65 to 0.66), in CHARTA, TRIBE1, and CAIRO5, respectively. Calibration plots indicated slight underestimation of mortality. The c-for-benefit indicated evidence for HTE in CHARTA (0.56, 95% CI, 0.48 to 0.65), but not in TRIBE1 (0.49, 95% CI, 0.44 to 0.55) and CAIRO5 (0.40, 95% CI, 0.32 to 0.48).</p><p><strong>Conclusion: </strong>Although 2-year mortality could be reasonably estimated, the HTE analysis showed that clinically available variables did not reliably identify which patients with mCRC benefit from FOLFOXIRI versus FOLFOX/FOLFIRI, both plus bevacizumab, across the three studies.</p>","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"8 ","pages":"e2400037"},"PeriodicalIF":3.3,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141635710","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Amr Muhammed, Rafaat A Bakheet, Karam Kenawy, Ahmed M A Ahmed, Muhammed Abdelhamid, Walaa Gamal Soliman
Purpose: Contrast enhancement is necessary for visualizing, diagnosing, and treating brain tumors. Through this study, we aimed to examine the potential role of general adversarial neural networks in generating artificial intelligence-based enhancement of tumors using a lightweight model.
Patients and methods: A retrospective study was conducted on magnetic resonance imaging scans of patients diagnosed with brain tumors between 2020 and 2023. A generative adversarial neural network was built to generate images that would mimic the real contrast enhancement of these tumors. The performance of the neural network was evaluated quantitatively by VGG-16, ResNet, binary cross-entropy loss, mean absolute error, mean squared error, and structural similarity index measures. Regarding the qualitative evaluation, nine cases were randomly selected from the test set and were used to build a short satisfaction survey for experienced medical professionals.
Results: One hundred twenty-nine patients with 156 scans were identified from the hospital database. The data were randomly split into a training set and validation set (90%) and a test set (10%). The VGG loss function for training, validation, and test sets were 2,049.8, 2,632.6, and 4,276.9, respectively. Additionally, the structural similarity index measured 0.366, 0.356, and 0.3192, respectively. At the time of submitting the article, 23 medical professionals responded to the survey. The median overall satisfaction score was 7 of 10.
Conclusion: Our network would open the door for using lightweight models in performing artificial contrast enhancement. Further research is necessary in this field to reach the point of clinical practicality.
{"title":"Potential Role of Generative Adversarial Networks in Enhancing Brain Tumors.","authors":"Amr Muhammed, Rafaat A Bakheet, Karam Kenawy, Ahmed M A Ahmed, Muhammed Abdelhamid, Walaa Gamal Soliman","doi":"10.1200/CCI.23.00266","DOIUrl":"https://doi.org/10.1200/CCI.23.00266","url":null,"abstract":"<p><strong>Purpose: </strong>Contrast enhancement is necessary for visualizing, diagnosing, and treating brain tumors. Through this study, we aimed to examine the potential role of general adversarial neural networks in generating artificial intelligence-based enhancement of tumors using a lightweight model.</p><p><strong>Patients and methods: </strong>A retrospective study was conducted on magnetic resonance imaging scans of patients diagnosed with brain tumors between 2020 and 2023. A generative adversarial neural network was built to generate images that would mimic the real contrast enhancement of these tumors. The performance of the neural network was evaluated quantitatively by VGG-16, ResNet, binary cross-entropy loss, mean absolute error, mean squared error, and structural similarity index measures. Regarding the qualitative evaluation, nine cases were randomly selected from the test set and were used to build a short satisfaction survey for experienced medical professionals.</p><p><strong>Results: </strong>One hundred twenty-nine patients with 156 scans were identified from the hospital database. The data were randomly split into a training set and validation set (90%) and a test set (10%). The VGG loss function for training, validation, and test sets were 2,049.8, 2,632.6, and 4,276.9, respectively. Additionally, the structural similarity index measured 0.366, 0.356, and 0.3192, respectively. At the time of submitting the article, 23 medical professionals responded to the survey. The median overall satisfaction score was 7 of 10.</p><p><strong>Conclusion: </strong>Our network would open the door for using lightweight models in performing artificial contrast enhancement. Further research is necessary in this field to reach the point of clinical practicality.</p>","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"8 ","pages":"e2300266"},"PeriodicalIF":3.3,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141728278","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Purpose: Denosumab is used to treat patients with bone metastasis from solid tumors, but sometimes causes severe hypocalcemia, so careful clinical management is important. This study aims to externally validate our previously developed risk prediction model for denosumab-induced hypocalcemia by using data from two facilities with different characteristics in Japan and to develop an updated model with improved performance and generalizability.
Methods: In the external validation, retrospective data of Kameda General Hospital (KGH) and Miyagi Cancer Center (MCC) between June 2013 and June 2022 were used and receiver operating characteristic (ROC)-AUC was mainly evaluated. A scoring-based updated model was developed using the same data set from a hospital-based administrative database as previously employed. Selection of variables related to prediction of hypocalcemia was based on the results of external validation.
Results: For the external validation, data from 235 KGH patients and 224 MCC patients were collected. ROC-AUC values in the original model were 0.879 and 0.774, respectively. The updated model consisting of clinical laboratory tests (calcium, albumin, and alkaline phosphatase) afforded similar ROC-AUC values in the two facilities (KGH, 0.837; MCC, 0.856).
Conclusion: We developed an updated risk prediction model for denosumab-induced hypocalcemia with small interfacility differences. Our results indicate the importance of using data from plural facilities with different characteristics in the external validation of generalized prediction models and may be generally relevant to the clinical application of risk prediction models. Our findings are expected to contribute to improved management of bone metastasis treatment.
{"title":"External Validation and Update of the Risk Prediction Model for Denosumab-Induced Hypocalcemia Developed From a Hospital-Based Administrative Database.","authors":"Keisuke Ikegami, Shungo Imai, Osamu Yasumuro, Masami Tsuchiya, Naomi Henmi, Mariko Suzuki, Katsuhisa Hayashi, Chisato Miura, Haruna Abe, Hayato Kizaki, Ryohkan Funakoshi, Yasunori Sato, Satoko Hori","doi":"10.1200/CCI.24.00078","DOIUrl":"10.1200/CCI.24.00078","url":null,"abstract":"<p><strong>Purpose: </strong>Denosumab is used to treat patients with bone metastasis from solid tumors, but sometimes causes severe hypocalcemia, so careful clinical management is important. This study aims to externally validate our previously developed risk prediction model for denosumab-induced hypocalcemia by using data from two facilities with different characteristics in Japan and to develop an updated model with improved performance and generalizability.</p><p><strong>Methods: </strong>In the external validation, retrospective data of Kameda General Hospital (KGH) and Miyagi Cancer Center (MCC) between June 2013 and June 2022 were used and receiver operating characteristic (ROC)-AUC was mainly evaluated. A scoring-based updated model was developed using the same data set from a hospital-based administrative database as previously employed. Selection of variables related to prediction of hypocalcemia was based on the results of external validation.</p><p><strong>Results: </strong>For the external validation, data from 235 KGH patients and 224 MCC patients were collected. ROC-AUC values in the original model were 0.879 and 0.774, respectively. The updated model consisting of clinical laboratory tests (calcium, albumin, and alkaline phosphatase) afforded similar ROC-AUC values in the two facilities (KGH, 0.837; MCC, 0.856).</p><p><strong>Conclusion: </strong>We developed an updated risk prediction model for denosumab-induced hypocalcemia with small interfacility differences. Our results indicate the importance of using data from plural facilities with different characteristics in the external validation of generalized prediction models and may be generally relevant to the clinical application of risk prediction models. Our findings are expected to contribute to improved management of bone metastasis treatment.</p>","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"8 ","pages":"e2400078"},"PeriodicalIF":3.3,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11371100/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141621791","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sergey D Goryachev, Cenk Yildirim, Clark DuMontier, Jennifer La, Mayuri Dharne, J Michael Gaziano, Mary T Brophy, Nikhil C Munshi, Jane A Driver, Nhan V Do, Nathanael R Fillmore
Purpose: Stage in multiple myeloma (MM) is an essential measure of disease risk, but its measurement in large databases is often lacking. We aimed to develop and validate a natural language processing (NLP) algorithm to extract oncologists' documentation of stage in the national Veterans Affairs (VA) Healthcare System.
Methods: Using nationwide electronic health record (EHR) and cancer registry data from the VA Corporate Data Warehouse, we developed and validated a rule-based NLP algorithm to extract oncologist-determined MM stage. To that end, a clinician annotated MM stage within over 5,000 short snippets of clinical notes, and annotated MM stage at MM treatment initiation for 200 patients. These were allocated into snippet- and patient-level development and validation sets. We developed MM stage extraction and roll-up algorithms within the development sets. After the algorithms were finalized, we validated them using standard measures in held-out validation sets.
Results: We developed algorithms for three different MM staging systems that have been in widespread use (Revised International Staging System [R-ISS], International Staging System [ISS], and Durie-Salmon [DS]) and for stage reported without a clearly defined system. Precision and recall were uniformly high for MM stage at the snippet level, ranging from 0.92 to 0.99 for the different MM staging systems. Performance in identifying for MM stage at treatment initiation at the patient level was also excellent, with precision of 0.92, 0.96, 0.90, and 0.86 and recall of 0.99, 0.98, 0.94, and 0.92 for R-ISS, ISS, DS, and unclear stage, respectively.
Conclusion: Our MM stage extraction algorithm uses rule-based NLP and data aggregation to accurately measure MM stage documented in oncology notes and pathology reports in VA's national EHR system. It may be adapted to other systems where MM stage is recorded in clinical notes.
目的:多发性骨髓瘤(MM)的分期是衡量疾病风险的一个重要指标,但在大型数据库中往往缺乏对分期的测量。我们旨在开发并验证一种自然语言处理(NLP)算法,以提取全国退伍军人事务(VA)医疗保健系统中肿瘤学家对分期的记录:利用退伍军人事务部企业数据仓库(VA Corporate Data Warehouse)中的全国电子健康记录(EHR)和癌症登记数据,我们开发并验证了一种基于规则的 NLP 算法,用于提取肿瘤学家确定的 MM 分期。为此,一名临床医生在 5000 多份简短的临床笔记片段中注释了 MM 分期,并在 200 名患者开始 MM 治疗时注释了 MM 分期。这些数据被分配到片段级和患者级的开发集和验证集。我们在开发集内开发了 MM 阶段提取和卷积算法。算法确定后,我们在保留的验证集中使用标准测量方法对其进行了验证:我们为三种广泛使用的不同 MM 分期系统(修订版国际分期系统 [R-ISS]、国际分期系统 [ISS] 和 Durie-Salmon [DS])以及没有明确定义系统的分期报告开发了算法。在片段水平上,MM 分期的精确度和召回率都很高,不同 MM 分期系统的精确度和召回率从 0.92 到 0.99 不等。在患者层面识别开始治疗时的 MM 分期也非常出色,R-ISS、ISS、DS 和不明确分期的精确度分别为 0.92、0.96、0.90 和 0.86,召回率分别为 0.99、0.98、0.94 和 0.92:我们的MM分期提取算法使用基于规则的NLP和数据聚合来准确测量退伍军人事务部国家电子病历系统中肿瘤笔记和病理报告中记录的MM分期。该算法可适用于在临床笔记中记录 MM 分期的其他系统。
{"title":"Natural Language Processing Algorithm to Extract Multiple Myeloma Stage From Oncology Notes in the Veterans Affairs Healthcare System.","authors":"Sergey D Goryachev, Cenk Yildirim, Clark DuMontier, Jennifer La, Mayuri Dharne, J Michael Gaziano, Mary T Brophy, Nikhil C Munshi, Jane A Driver, Nhan V Do, Nathanael R Fillmore","doi":"10.1200/CCI.23.00197","DOIUrl":"10.1200/CCI.23.00197","url":null,"abstract":"<p><strong>Purpose: </strong>Stage in multiple myeloma (MM) is an essential measure of disease risk, but its measurement in large databases is often lacking. We aimed to develop and validate a natural language processing (NLP) algorithm to extract oncologists' documentation of stage in the national Veterans Affairs (VA) Healthcare System.</p><p><strong>Methods: </strong>Using nationwide electronic health record (EHR) and cancer registry data from the VA Corporate Data Warehouse, we developed and validated a rule-based NLP algorithm to extract oncologist-determined MM stage. To that end, a clinician annotated MM stage within over 5,000 short snippets of clinical notes, and annotated MM stage at MM treatment initiation for 200 patients. These were allocated into snippet- and patient-level development and validation sets. We developed MM stage extraction and roll-up algorithms within the development sets. After the algorithms were finalized, we validated them using standard measures in held-out validation sets.</p><p><strong>Results: </strong>We developed algorithms for three different MM staging systems that have been in widespread use (Revised International Staging System [R-ISS], International Staging System [ISS], and Durie-Salmon [DS]) and for stage reported without a clearly defined system. Precision and recall were uniformly high for MM stage at the snippet level, ranging from 0.92 to 0.99 for the different MM staging systems. Performance in identifying for MM stage at treatment initiation at the patient level was also excellent, with precision of 0.92, 0.96, 0.90, and 0.86 and recall of 0.99, 0.98, 0.94, and 0.92 for R-ISS, ISS, DS, and unclear stage, respectively.</p><p><strong>Conclusion: </strong>Our MM stage extraction algorithm uses rule-based NLP and data aggregation to accurately measure MM stage documented in oncology notes and pathology reports in VA's national EHR system. It may be adapted to other systems where MM stage is recorded in clinical notes.</p>","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"8 ","pages":"e2300197"},"PeriodicalIF":3.3,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11371094/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141749645","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kathi Mooney, Susan L Beck, Christina Wilson, Lorinda Coombs, Meagan Whisenant, Ann Marie Moraitis, Elizabeth A Sloss, Natalya Alekhina, Jennifer Lloyd, Mary Steinbach, Bridget Nicholson, Eli Iacob, Gary Donaldson
Purpose: People with cancer experience poorly controlled symptoms that persist between treatment visits. Automated digital technology can remotely monitor and facilitate symptom management at home. Essential to digital interventions is patient engagement, user satisfaction, and intervention benefits that are distributed across patient populations so as not to perpetuate inequities. We evaluated Symptom Care at Home (SCH), an automated digital platform, to determine patient engagement, satisfaction, and whether intervention subgroups gained similar symptom reduction benefits.
Methods: 358 patients with cancer receiving a course of chemotherapy were randomly assigned to SCH or usual care (UC). Both groups reported daily on 11 symptoms and completed the SF36 (Short Form Health Survey) monthly. SCH participants received immediate automated self-care coaching on reported symptoms. As needed, nurse practitioners followed up for poorly controlled symptoms.
Results: The average participant was White (83%), female (75%), and urban-dwelling (78.6%). Daily call adherence was 90% of expected days. Participants reported high user satisfaction. SCH participants had lower symptom burden than UC in all subgroups: age, sex, race, income, residence type, diagnosis, and stage (all P < .001 effect size 0.33-0.65), except for stages I and II cancers. Non-White and lower-income SCH participants gained a higher magnitude of symptom reduction than White participants and higher-income participants. Additionally, SCH men gained higher SF36 mental health (MH) benefit. There were no differences on other SF36 indices.
Conclusion: Participants were highly satisfied and consistently engaged the SCH platform. SCH men gained large MH improvements, perhaps from increased comfort in sharing concerns through automated interactions. Although all intervention subgroups benefited, non-White participants and those with lower income gained higher symptom reduction benefit, suggesting that systematic care through digital tools can overcome existing disparities in symptom care outcomes.
目的:癌症患者的症状控制不佳,在两次治疗之间持续存在。自动化数字技术可以远程监控并促进在家进行症状管理。数字干预的关键在于患者的参与度、用户满意度以及在不同患者群体中的干预效果,从而避免不公平现象的长期存在。我们对自动数字平台 "居家症状护理"(SCH)进行了评估,以确定患者的参与度、满意度以及干预亚组是否获得了类似的症状缓解效果。方法:358 名接受化疗的癌症患者被随机分配到 SCH 或常规护理(UC)组。两组患者每天报告 11 种症状,每月填写 SF36(简表健康调查)。SCH组的参与者会立即接受有关所报告症状的自动自我护理指导。必要时,执业护士会对控制不佳的症状进行跟踪:参与者平均为白人(83%)、女性(75%)和城市居民(78.6%)。每天坚持呼叫的比例为预期天数的 90%。参与者表示用户满意度很高。在年龄、性别、种族、收入、居住地类型、诊断和分期等所有分组中,SCH 参与者的症状负担均低于 UC 参与者(所有 P < .001 的效应大小为 0.33-0.65),但 I 期和 II 期癌症除外。与白人和高收入人群相比,非白人和低收入人群的症状减轻程度更高。此外,SCH 男性获得的 SF36 心理健康(MH)益处更高。其他 SF36 指数没有差异:结论:参与者对 SCH 平台非常满意,并持续参与其中。SCH男性在心理健康方面获得了很大改善,这可能是由于他们通过自动互动分享了更多的担忧。尽管所有干预亚组都从中受益,但非白人参与者和收入较低者在症状减轻方面获益更大,这表明通过数字工具进行系统护理可以克服症状护理结果方面的现有差异。
{"title":"Assessing Patient Perspectives and the Health Equity of a Digital Cancer Symptom Remote Monitoring and Management System.","authors":"Kathi Mooney, Susan L Beck, Christina Wilson, Lorinda Coombs, Meagan Whisenant, Ann Marie Moraitis, Elizabeth A Sloss, Natalya Alekhina, Jennifer Lloyd, Mary Steinbach, Bridget Nicholson, Eli Iacob, Gary Donaldson","doi":"10.1200/CCI.23.00243","DOIUrl":"10.1200/CCI.23.00243","url":null,"abstract":"<p><strong>Purpose: </strong>People with cancer experience poorly controlled symptoms that persist between treatment visits. Automated digital technology can remotely monitor and facilitate symptom management at home. Essential to digital interventions is patient engagement, user satisfaction, and intervention benefits that are distributed across patient populations so as not to perpetuate inequities. We evaluated Symptom Care at Home (SCH), an automated digital platform, to determine patient engagement, satisfaction, and whether intervention subgroups gained similar symptom reduction benefits.</p><p><strong>Methods: </strong>358 patients with cancer receiving a course of chemotherapy were randomly assigned to SCH or usual care (UC). Both groups reported daily on 11 symptoms and completed the SF36 (Short Form Health Survey) monthly. SCH participants received immediate automated self-care coaching on reported symptoms. As needed, nurse practitioners followed up for poorly controlled symptoms.</p><p><strong>Results: </strong>The average participant was White (83%), female (75%), and urban-dwelling (78.6%). Daily call adherence was 90% of expected days. Participants reported high user satisfaction. SCH participants had lower symptom burden than UC in all subgroups: age, sex, race, income, residence type, diagnosis, and stage (all <i>P</i> < .001 effect size 0.33-0.65), except for stages I and II cancers. Non-White and lower-income SCH participants gained a higher magnitude of symptom reduction than White participants and higher-income participants. Additionally, SCH men gained higher SF36 mental health (MH) benefit. There were no differences on other SF36 indices.</p><p><strong>Conclusion: </strong>Participants were highly satisfied and consistently engaged the SCH platform. SCH men gained large MH improvements, perhaps from increased comfort in sharing concerns through automated interactions. Although all intervention subgroups benefited, non-White participants and those with lower income gained higher symptom reduction benefit, suggesting that systematic care through digital tools can overcome existing disparities in symptom care outcomes.</p>","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"8 ","pages":"e2300243"},"PeriodicalIF":3.3,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141753365","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Purpose: Early prediction of response to immunotherapy may help guide patient management by identifying resistance to treatment and allowing adaptation of therapies. This analysis evaluated a mathematical model of response to immunotherapy that provides patient-specific prediction of outcome using the initial change in tumor size/burden from baseline to the first follow-up visit on standard imaging scans.
Methods: We applied the model to 600 patients with advanced solid tumors who received durvalumab in Study 1108, a phase I/II trial, and compared outcome prediction performance versus size-based criteria with RECIST version 1.1 best overall response (BOR), baseline circulating tumor (ct)DNA level, and other clinical/pathologic predictors of immunotherapy response.
Results: In multiple solid tumors, the mathematical parameter representing net tumor growth rate at the first on-treatment computed tomography (CT) scan assessed around 6 weeks after starting durvalumab (α1) had a concordance index to predict overall survival (OS) of 0.66-0.77 on multivariate analyses. This measurement of early tumor dynamics significantly improved multivariate OS models that included standard RECIST v1.1 criteria, baseline ctDNA levels, and other clinical/pathologic factors in predicting OS. Furthermore, α1 was assessed consistently at the first on-treatment CT scan, whereas all traditional RECIST BOR groups were confirmed only after this time.
Conclusion: These results support further exploring α1 as an integral biomarker of response to immunotherapy. This biomarker may be predictive of further benefit and can be assessed before RECIST response groups can be assigned, potentially providing an opportunity to personalize oncologic management.
目的:对免疫疗法反应的早期预测有助于识别治疗耐药性并调整疗法,从而为患者管理提供指导。这项分析评估了一个免疫疗法反应数学模型,该模型利用标准成像扫描显示的肿瘤大小/负担从基线到首次随访的初始变化,提供针对患者的疗效预测:我们将该模型应用于600名在I/II期试验1108研究中接受了durvalumab治疗的晚期实体瘤患者,并比较了该模型与基于肿瘤大小的标准、RECIST 1.1版最佳总体反应(BOR)、基线循环肿瘤(ct)DNA水平以及其他免疫疗法反应的临床/病理预测指标之间的结果预测性能:在多种实体瘤中,在开始使用度伐卢单抗后6周左右进行首次治疗计算机断层扫描(CT)评估时,代表肿瘤净生长率的数学参数(α1)在多变量分析中预测总生存期(OS)的一致性指数为0.66-0.77。这种早期肿瘤动态测量方法显著改善了预测 OS 的多变量 OS 模型,这些模型包括标准 RECIST v1.1 标准、基线 ctDNA 水平和其他临床/病理因素。此外,α1在首次治疗CT扫描时就得到了一致的评估,而所有传统的RECIST BOR组别都是在这一时间之后才得到确认:这些结果支持进一步探索将α1作为免疫疗法反应的综合生物标志物。结论:这些结果支持进一步探索将α1作为免疫疗法反应的综合生物标志物,该生物标志物可预测进一步的获益,并可在分配RECIST反应组别之前进行评估,从而有可能为个性化肿瘤治疗提供机会。
{"title":"Clinical Validation of Mathematically Derived Early Tumor Dynamics for Solid Tumors in Response to Durvalumab.","authors":"Qin Li, Vittorio Cristini, Ashok Gupta, Ikbel Achour, J Carl Barrett, Eugene J Koay","doi":"10.1200/CCI.23.00254","DOIUrl":"10.1200/CCI.23.00254","url":null,"abstract":"<p><strong>Purpose: </strong>Early prediction of response to immunotherapy may help guide patient management by identifying resistance to treatment and allowing adaptation of therapies. This analysis evaluated a mathematical model of response to immunotherapy that provides patient-specific prediction of outcome using the initial change in tumor size/burden from baseline to the first follow-up visit on standard imaging scans.</p><p><strong>Methods: </strong>We applied the model to 600 patients with advanced solid tumors who received durvalumab in Study 1108, a phase I/II trial, and compared outcome prediction performance versus size-based criteria with RECIST version 1.1 best overall response (BOR), baseline circulating tumor (ct)DNA level, and other clinical/pathologic predictors of immunotherapy response.</p><p><strong>Results: </strong>In multiple solid tumors, the mathematical parameter representing net tumor growth rate at the first on-treatment computed tomography (CT) scan assessed around 6 weeks after starting durvalumab (<i>α</i><sub>1</sub>) had a concordance index to predict overall survival (OS) of 0.66-0.77 on multivariate analyses. This measurement of early tumor dynamics significantly improved multivariate OS models that included standard RECIST v1.1 criteria, baseline ctDNA levels, and other clinical/pathologic factors in predicting OS. Furthermore, <i>α</i><sub>1</sub> was assessed consistently at the first on-treatment CT scan, whereas all traditional RECIST BOR groups were confirmed only after this time.</p><p><strong>Conclusion: </strong>These results support further exploring <i>α</i><sub>1</sub> as an integral biomarker of response to immunotherapy. This biomarker may be predictive of further benefit and can be assessed before RECIST response groups can be assigned, potentially providing an opportunity to personalize oncologic management.</p>","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"8 ","pages":"e2300254"},"PeriodicalIF":3.3,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141602106","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}