Battle of the authors: Comparing neurosurgery articles written by humans and AI

Abstract

Background

The advancement of artificial intelligence (AI) has led to its application in various fields, including medical literature. This study compares the quality of neurosurgery articles written by human authors and those generated by ChatGPT, an advanced AI model. The objective was to determine if AI-generated articles meet the standards of human-written academic papers.

Methods

A total of 10 neurosurgery articles, 5 written by humans and 5 by ChatGPT, were evaluated by a panel of blinded experts. The assessment parameters included overall impression, readability, criteria satisfaction, and degree of detail. Additionally, readability scores were calculated using the Lix score and the Flesch-Kincaid grade level. Preference and identification tests were also conducted to determine if experts could distinguish between the two types of articles.

Results

The study found no significant differences in the overall quality parameters between human-written and ChatGPT −generated articles. Readability scores were higher for ChatGPT articles (Lix score: 35 vs. 26, Flesch-Kincaid grade level: 10 vs. 8). Experts correctly identified the authorship of the articles 61% of the time, with preferences almost evenly split (47% preferred CHATGPT, 44% preferred human, and 9% had no preference). The most statistically significant result was the higher readability scores of CHATGPT-generated articles, indicating that AI can produce more readable content than human authors.

Conclusion

ChatGPT is capable of generating neurosurgery articles that are comparable in quality to those written by humans. The higher readability scores of AI-generated articles suggest that ChatGPT can enhance the accessibility of scientific literature. This study supports the potential integration of AI in academic writing, offering a valuable tool for researchers and medical professionals.