Chat GPT vs. Clinical Decision Support Systems in the Analysis of Drug–Drug Interactions

Abstract

The current standard method for the analysis of potential drug–drug interactions (pDDIs) is time-consuming and includes the use of multiple clinical decision support systems (CDSSs) and the interpretation by healthcare professionals. With the emergence of large language models developed with artificial intelligence, an interesting alternative arose. This retrospective study included 30 patients with polypharmacy, who underwent a pDDI analysis between October 2022 and August 2023, and compared the performance of Chat GPT and established CDSSs (MediQ®, Lexicomp®, Micromedex®) in the analysis of pDDIs. A multidisciplinary team interpreted the obtained results and decided upon clinical relevance and assigned severity grades using three categories: (i) contraindicated, (ii) severe, (iii) moderate. The expert review identified a total of 280 clinically relevant pDDIs (3 contraindications, 13 severe, 264 moderate) using established CDSSs, compared with 80 pDDIs (2 contraindications, 5 severe, 73 moderate) using Chat GPT. Chat GPT almost entirely neglected pDDIs with the risk to QTc prolongation (85 vs. 8), which could also not be sufficiently improved by using a specific prompt. To assess the consistency of the results provided by Chat GPT, we repeated each query and found inconsistent results in 90% of the cases. In contrast, Chat GPT provided acceptable and comprehensible recommendations for specific questions on side effects. The use of Chat GPT for the identification of pDDIs cannot be recommended currently, because clinically relevant pDDIs were not detected, there were obvious errors and results were inconsistent. However, if these limitations are addressed accordingly, it is a promising platform for the future.