JCO Clinical Cancer Informatics最新文献

Purpose: Limited studies have used natural language processing (NLP) in the context of non-small cell lung cancer (NSCLC). This study aimed to validate the application of an NLP model to an NSCLC cohort by extracting NSCLC concepts from free-text medical notes and converting them to structured, interpretable data.

Methods: Patients with a lung neoplasm, NSCLC histology, and treatment information in their notes were selected from a repository of over 27 million patients. From these, 200 were randomly selected for this study with the longest and the most recent note included for each patient. An NLP model developed and validated on a large solid and blood cancer oncology cohort was applied to this NSCLC cohort. Two certified tumor registrars and a curator abstracted concepts from the notes: neoplasm, histology, stage, TNM values, and metastasis sites. This manually abstracted gold standard was compared with the NLP model output. Precision and recall scores were calculated.

Results: The NLP model extracted the NSCLC concepts with excellent precision and recall with the following scores, respectively: Lung neoplasm 100% and 100%, NSCLC histology 99% and 88%, histology correctly linked to neoplasm 98% and 79%, stage value 98.8% and 92%, stage TNM value 93% and 98%, and metastasis site 97% and 89%. High precision is related to a low number of false positives, and therefore, extracted concepts are likely accurate. High recall indicates that the model captured most of the desired concepts.

Conclusion: This study validates that Optum's oncology NLP model has high precision and recall with clinical real-world data and is a reliable model to support research studies and clinical trials. This validation study shows that our nonspecific solid tumor and blood cancer oncology model is generalizable to successfully extract clinical information from specific cancer cohorts.

Purpose: A previous study demonstrated that power against the (unobserved) true effect for the primary end point (PEP) of most phase III oncology trials is low, suggesting an increased risk of false-negative findings in the field of late-phase oncology. Fitting models with prognostic covariates is a potential solution to improve power; however, the extent to which trials leverage this approach, and its impact on trial interpretation at scale, is unknown. To that end, we hypothesized that phase III trials using multivariable PEP analyses are more likely to demonstrate superiority versus trials with univariable analyses.

Methods: PEP analyses were reviewed from trials registered on ClinicalTrials.gov. Adjusted odds ratios (aORs) were calculated by logistic regressions.

Results: Of the 535 trials enrolling 454,824 patients, 69% (n = 368) used a multivariable PEP analysis. Trials with multivariable PEP analyses were more likely to demonstrate PEP superiority (57% [209 of 368] v 42% [70 of 167]; aOR, 1.78 [95% CI, 1.18 to 2.72]; P = .007). Among trials with a multivariable PEP model, 16 conditioned on covariates and 352 stratified by covariates. However, 108 (35%) of 312 trials with stratified analyses lost power by categorizing a continuous variable, which was especially common among immunotherapy trials (aOR, 2.45 [95% CI, 1.23 to 4.92]; P = .01).

Conclusion: Trials increasing power by fitting multivariable models were more likely to demonstrate PEP superiority than trials with unadjusted analysis. Underutilization of conditioning models and empirical power loss associated with covariate categorization required by stratification were identified as barriers to power gains. These findings underscore the opportunity to increase power in phase III trials with conventional methodology and improve patient access to effective novel therapies.

Purpose: Electronic health records (EHRs) are valuable information repositories that offer insights for enhancing clinical research on breast cancer (BC) using real-world data. The objective of this study was to develop a natural language processing (NLP) model specifically designed to extract structured data from BC pathology reports written in natural language.

Methods: During the initial phase, the algorithm's development cohort comprised 193 pathology reports from 116 patients with BC from 2012 to 2016. A rule-based NLP algorithm was applied to extract 26 variables for analysis and was compared with the manual extraction of data performed by both a data entry specialist and an oncologist. Following the first approach, the data set was expanded to include 513 reports, and a Named Entity Recognition (NER)-NLP model was trained and evaluated using K-fold cross-validation.

Results: The first approach led to a concordance analysis, which revealed an 82.9% agreement between the algorithm and the oncologist, whereas the concordance between the data entry specialist and the oncologist was 90.8%. The second training approach introduced the definition of an NER-NLP model, in which the accuracy showed remarkable potential (97.8%). Notably, the model demonstrated remarkable performance, especially for parameters such as estrogen receptor, progesterone receptor, human epidermal growth factor receptor 2, and Ki-67 (F1-score 1.0).

Conclusion: The present study aligns with the rapidly evolving field of artificial intelligence (AI) applications in oncology, seeking to expedite the development of complex cancer databases and registries. The results of the model are currently undergoing postprocessing procedures to organize the data into tabular structures, facilitating their utilization in real-world clinical and research endeavors.

Purpose: Identifying cancer symptoms in electronic health record (EHR) narratives is feasible with natural language processing (NLP). However, more efficient NLP systems are needed to detect various symptoms and distinguish observed symptoms from negated symptoms and medication-related side effects. We evaluated the accuracy of NLP in (1) detecting 14 symptom groups (ie, pain, fatigue, swelling, depressed mood, anxiety, nausea/vomiting, pruritus, headache, shortness of breath, constipation, numbness/tingling, decreased appetite, impaired memory, disturbed sleep) and (2) distinguishing observed symptoms in EHR narratives among patients with cancer.

Methods: We extracted 902,508 notes for 11,784 unique patients diagnosed with cancer and developed a gold standard corpus of 1,112 notes labeled for presence or absence of 14 symptom groups. We trained an embeddings-augmented NLP system integrating human and machine intelligence and conventional machine learning algorithms. NLP metrics were calculated on a gold standard corpus subset for testing.

Results: The interannotator agreement for labeling the gold standard corpus was excellent at 92%. The embeddings-augmented NLP model achieved the best performance (F1 score = 0.877). The highest NLP accuracy was observed in pruritus (F1 score = 0.937) while the lowest accuracy was in swelling (F1 score = 0.787). After classifying the entire data set with embeddings-augmented NLP, we found that 41% of the notes included symptom documentation. Pain was the most documented symptom (29% of all notes) while impaired memory was the least documented (0.7% of all notes).

Conclusion: We illustrated the feasibility of detecting 14 symptom groups in EHR narratives and showed that an embeddings-augmented NLP system outperforms conventional machine learning algorithms in detecting symptom information and differentiating observed symptoms from negated symptoms and medication-related side effects.

Purpose: Understanding the real-world experience of patients with early breast cancer (eBC) is imperative for optimizing outcomes and evolving patient care. However, there is a lack of patient-level data, hindering clinical development. This social listening study was performed to understand patient insights into symptoms and impacts of hormone therapy (HT) for eBC using posts from patient forums on breastcancer.org to inform future clinical research.

Methods: Natural language processing (NLP) and machine learning techniques were used to identify themes related to eBC from a sample of 500,000 posts. After relevant data selection, 362,074 eBC posts were retained for further analysis of symptoms and impacts related to HT, as well as insights into symptom severity, pain locations, and symptom management using exercise and yoga.

Results: Overall, 32 symptoms and nine impacts had significant associations with ≥one HT. Hot flush (relative risk [RR], 6.70 [95% CI, 3.36 to 13.36]), arthralgia (RR, 6.67 [95% CI, 3.53 to 12.59]), weight increased (RR, 4.83 [95% CI, 3.20 to 7.28]), mood swings (RR, 7.36 [95% CI, 5.75 to 9.42]), insomnia (RR, 4.76 [95% CI, 3.14 to 7.22]), and depression (RR, 3.05 [95% CI, 1.71 to 5.44]) demonstrated the strongest associations. Severe headache, dizziness, back pain, and muscle spasms showed significant associations with ≥one HT despite their low overall prevalence in eBC posts.

Conclusion: The social listening approach allowed the identification of real-world insights from posts specific to eBC HT from a large-scale online breast cancer forum that captured experiences from a uniquely diverse group of patients. Using NLP has a potential to scale analysis of patient feedback and reveal actionable insights into patient experiences of treatment that can inform the development of future therapies and improve the care of patients with eBC.

Purpose: Palliative care is recommended for patients with cancer with a life expectancy of <12 months. Machine learning (ML) techniques can help in predicting survival outcomes among patients with cancer and may help distinguish who benefits the most from palliative care support. We aim to explore the importance of several objective and subjective self-reported variables. Subjective variables were collected through electronic psycho-oncologic and palliative care self-assessment screenings. We used these variables to predict 1-year mortality.

Materials and methods: Between April 1, 2020, and March 31, 2021, a total of 265 patients with advanced cancer completed a patient-reported outcome tool. We documented objective and subjective variables collected from electronic health records, self-reported subjective variables, and all clinical variables combined. We used logistic regression (LR), 20-fold cross-validation, decision trees, and random forests to predict 1-year mortality. We analyzed the receiver operating characteristic (ROC) curve-AUC, the precision-recall curve-AUC (PR-AUC)-and the feature importance of the ML models.

Results: The performance of clinical nonpatient variables in predictions (LR reaches 0.81 [ROC-AUC] and 0.72 [F1 score]) are much more predictive than that of subjective patient-reported variables (LR reaches 0.55 [ROC-AUC] and 0.52 [F1 score]).

Conclusion: The results show that objective variables used in this study are much more predictive than subjective patient-reported variables, which measure subjective burden. These findings indicate that subjective burden cannot be reliably used to predict survival. Further research is needed to clarify the role of self-reported patient burden and mortality prediction using ML.

Purpose: Patient-centered outcomes (PCOs) are pivotal in cancer treatment, as they directly reflect patients' quality of life. Although multiple studies suggest that factors affecting breast cancer-related morbidity and survival are influenced by treatment side effects and adherence to long-term treatment, such data are generally only available on a smaller scale or from a single center. The primary challenge with collecting these data is that the outcomes are captured as free text in clinical narratives written by clinicians.

Materials and methods: Given the complexity of PCO documentation in these narratives, computerized methods are necessary to unlock the wealth of information buried in unstructured text notes that often document PCOs. Inspired by the success of large language models (LLMs), we examined the adaptability of three LLMs, GPT-2, BioGPT, and PMC-LLaMA, on PCO tasks across three institutions, Mayo Clinic, Emory University Hospital, and Stanford University. We developed an open-source framework for fine-tuning LLM that can directly extract the five different categories of PCO from the clinic notes.

Results: We found that these LLMs without fine-tuning (zero-shot) struggle with challenging PCO extraction tasks, displaying almost random performance, even with some task-specific examples (few-shot learning). The performance of our fine-tuned, task-specific models is notably superior compared with their non-fine-tuned LLM models. Moreover, the fine-tuned GPT-2 model has demonstrated a significantly better performance than the other two larger LLMs.

Conclusion: Our discovery indicates that although LLMs serve as effective general-purpose models for tasks across various domains, they require fine-tuning when applied to the clinician domain. Our proposed approach has the potential to lead more efficient, adaptable models for PCO information extraction, reducing reliance on extensive computational resources while still delivering superior performance for specific tasks.

Purpose: Cure models are a useful alternative to Cox proportional hazards models in oncology studies when there is a subpopulation of patients who will not experience the event of interest. Although software is available to fit cure models, there are limited tools to evaluate, report, and visualize model results. This article introduces the cureit R package, an end-to-end pipeline for building mixture cure models, and demonstrates its use in a data set of patients with primary extremity and truncal liposarcoma.

Methods: To assess associations between liposarcoma histologic subtypes and disease-specific death (DSD) in patients treated at Memorial Sloan Kettering Cancer Center between July 1982 and September 2017, mixture cure models were fit and evaluated using the cureit package. Liposarcoma histologic subtypes were defined as well-differentiated, dedifferentiated, myxoid, round cell, and pleomorphic.

Results: All other analyzed liposarcoma histologic subtypes were significantly associated with higher DSD in cure models compared with well-differentiated. In multivariable models, myxoid (odds ratio [OR], 6.25 [95% CI, 1.32 to 29.6]) and round cell (OR, 16.2 [95% CI, 2.80 to 93.2]) liposarcoma had higher incidences of DSD compared with well-differentiated patients. By contrast, dedifferentiated liposarcoma was associated with the latency of DSD (hazard ratio, 10.6 [95% CI, 1.48 to 75.9]). Pleomorphic liposarcomas had significantly higher risk in both incidence and the latency of DSD (P < .0001). Brier scores indicated comparable predictive accuracy between cure and Cox models.

Conclusion: We developed the cureit pipeline to fit and evaluate mixture cure models and demonstrated its clinical utility in the liposarcoma disease setting, shedding insights on the subtype-specific associations with incidence and/or latency.

Purpose: Real-world data (RWD) holds promise for ascribing a real-world (rw) outcome to a drug intervention; however, ascertaining rw-response to treatment from RWD can be challenging. Friends of Cancer Research formed a collaboration to assess available data attributes related to rw-response across RWD sources to inform methods for capturing, defining, and evaluating rw-response.

Materials and methods: This retrospective noninterventional (observational) study included seven electronic health record data companies (data providers) providing summary-level deidentified data from 200 patients diagnosed with metastatic non-small cell lung cancer (mNSCLC) and treated with first-line platinum doublet chemotherapy following a common protocol. Data providers reviewed the availability and frequency of data components to assess rw-response (ie, images, radiology imaging reports, and clinician response assessments). A common protocol was used to assess and report rw-response end points, including rw-response rate (rwRR), rw-duration of response (rwDOR), and the association of rw-response with rw-overall survival (rwOS), rw-time to treatment discontinuation (rwTTD), and rw-time to next treatment (rwTTNT).

Results: The availability and timing of clinician assessments was relatively consistent across data sets in contrast to images and image reports. Real-world response was analyzed using clinician response assessments (median proportion of patients evaluable, 77.5%), which had the highest consistency in the timing of assessments. Relative consistency was observed across data sets for rwRR (median 46.5%), as well as the median and directionality of rwOS, rwTTD, and rwTTNT. There was variability in rwDOR across data sets.

Conclusion: This collaborative effort demonstrated the feasibility of aligning disparate data sources to evaluate rw-response end points using clinician-documented responses in patients with mNSCLC. Heterogeneity exists in the availability of data components to assess response and related rw-end points, and further work is needed to inform drug effectiveness evaluation within RWD sources.