Assessing the Informational Value of Large Language Models Responses in Aesthetic Surgery: A Comparative Analysis with Expert Opinions

Abstract

Background

The increasing popularity of Large Language Models (LLMs) in various healthcare settings has raised questions about their ability to provide accurate and reliable information. This study aimed to evaluate the informational value of Large Language Models responses in aesthetic plastic surgery by comparing them with the opinions of experienced surgeons.

Methods

Thirty patients undergoing three common aesthetic procedures—dermal fillers, botulinum toxin injections, and aesthetic blepharoplasty—were selected. The most frequently asked questions by these patients were recorded and submitted to ChatGpt 3.5 and Google Bard v.1.53. The answers provided by the Large Language Models were then evaluated by 13 experienced aesthetic plastic surgeons on a Likert scale for accessibility, accuracy, and overall usefulness.

Results

The overall ratings of the chatbot responses were moderate, with surgeons generally finding them to be accurate and clear. However, the lack of transparency regarding the sources of the information provided by the LLMs made it impossible to fully evaluate their credibility.

Conclusions

While chatbots have the potential to provide patients with convenient access to information about aesthetic plastic surgery, their current limitations in terms of transparency and comprehensiveness warrant caution in their use as a primary source of information. Further research is needed to develop more robust and reliable LLMs for healthcare applications.

Level of Evidence I

This journal requires that authors assign a level of evidence to each article. For a full description of these Evidence-Based Medicine ratings, please refer to the Table of Contents or the online Instructions to Authors www.springer.com/00266.