Patient2Trial: From patient to participant in clinical trials using large language models

Abstract

Purpose

Large language models (LLMs) exhibit promising language understanding and generation capabilities and have been adopted for various clinical use cases. Investigating the feasibility of leveraging LLMs in building a clinical trial retrieval system for patients is crucial as it can greatly enhance the patient enrollment process by prioritizing the most suitable trials pertaining to a patient. In this work, we develop an LLM-assisted system focused on a patient-initiated approach, allowing patients with specific conditions to directly find eligible trials by completing disorder-specific questionnaires.

Methods

We obtained clinical trial eligibility criteria (from ClinicalTrials.gov) and simulated patient questionnaires (or topics) from the Text REtrieval Conference (TREC) 2023 Clinical Trials Track conducted by the National Institute of Standards and Technology (NIST), in which we also participated. These topics cover eight disorders across diverse domains, namely glaucoma, anxiety, chronic obstructive pulmonary disease, breast cancer, Covid-19, rheumatoid arthritis, sickle cell anemia, and type 2 diabetes. A Generative Pre-trained Transformer model (GPT-4) was employed for system development. We conducted both quantitative and qualitative evaluation using 37 patient topics.

Results

The system achieved an overall Precision@10 (proportion of relevant trials) of 0.7351 and NDCG@10 (considers ranking order of relevant trials) of 0.8109, indicating its effectiveness in retrieving ranked lists of suitable trials for patients. Notably, for eight out of 37 patient topics, all the top 10 retrieved trials were relevant. The system scored the highest on breast cancer (NDCG@10 = 0.9347, Precision@10 = 0.84) and the lowest on type 2 diabetes (NDCG@10 = 0.61, Precision@10 = 0.475). One probable reason could be that the information in breast cancer topics is relatively straightforward to match. Qualitative error analysis classified errors into four categories (e.g., difficulty in correctly matching inclusion criteria) and further highlighted strengths (e.g., ability to make clinical inference).

Conclusion

We demonstrated the feasibility of integrating LLMs in identifying and ranking suitable trials for patients across multiple disorders. Further work is required to assess the system's generalizability on other disorders and patient information sources. This system has the potential to expedite the patient-trial matching process by suggesting a ranked list of applicable trials to patients and clinicians.