Generative Artificial Intelligence Platform for Automating Social Media Posts From Urology Journal Articles: A Cross-Sectional Study and Randomized Assessment.

Abstract

Purpose: This cross-sectional study assessed a generative artificial intelligence platform to automate the creation of accurate, appropriate, and compelling social media (SoMe) posts from urological journal articles.

Materials and methods: One hundred SoMe posts from the top 3 journals in urology X (formerly Twitter) profiles were collected from August 2022 to October 2023. A freeware generative pre-trained transformer (GPT) tool was developed to autogenerate SoMe posts, which included title summarization, key findings, pertinent emojis, hashtags, and digital object identifier links to the article. Three physicians independently evaluated GPT-generated posts for achieving tetrafecta of accuracy and appropriateness criteria. Fifteen scenarios were created from 5 randomly selected posts from each journal. Each scenario contained both the original and the GPT-generated post for the same article. Five questions were formulated to investigate the posts' likability, shareability, engagement, understandability, and comprehensiveness. The paired posts were then randomized and presented to blinded academic authors and general public through Amazon Mechanical Turk (AMT) responders for preference evaluation.

Results: Median time for post autogeneration was 10.2 seconds (interquartile range 8.5-12.5). Of the 150 rated GPT-generated posts, 115 (76.6%) met the correctness tetrafecta: 144 (96%) accurately summarized the title, 147 (98%) accurately presented the articles' main findings, 131 (87.3%) appropriately used emojis, and 138 (92%) appropriately used hashtags. A total of 258 academic urologists and 493 AMT responders answered the surveys, wherein the GPT-generated posts consistently outperformed the original journals' posts for both academicians and AMT responders (P < .05).

Conclusions: Generative artificial intelligence can automate the creation of SoMe posts from urology journal abstracts that are both accurate and preferable by the academic community and general public.