JMIR Formative Research最新文献

Background: The popularity of large language models (LLMs) has grown exponentially across health care. Despite the wealth of literature on proposed applications in medical education, there remains a critical gap regarding their real-world use, benefits, and challenges as experienced by medical students themselves.

Objective: We aimed to explore qualitatively and characterize the perceived benefits, facilitators, and barriers associated with the use of LLMs among a cohort of London-based medical students.

Methods: Semistructured interviews were conducted with 15 medical students from preclinical and clinical stages at London-based medical schools. Guided by the technology acceptance model, interview transcripts underwent an inductive thematic analysis to identify themes on actual system use, perceived usefulness, ease of use, and attitudes toward LLMs.

Results: All participants reported frequent use of ChatGPT for concise topic summarization, clarification of complex concepts, generation of examination-style questions, and summarization of research. Students described LLMs as a complementary tool to traditional materials, valuing their immediacy ("Instead of getting a textbook, I can ask ChatGPT to summarise something in X words and read it in under a minute") and ease of use. Peer demonstration and device-agnostic accessibility emerged as key facilitators. Of note, wider applications such as simulating clinical interviews were discovered through peers rather than through formal teaching. Significant barriers were reported. Hallucinations, fabricated references, and outdated information led to loss of trust, with more junior students finding inaccurate outputs difficult to detect ("I stopped using it because I found it to be inaccurate, and I don't want to be learning the wrong things"). Half of the participants interviewed reported a sense of overreliance, defaulting to its use for answers with a perceived loss of critical thinking ability. Students noted inequalities in access to advanced features and voiced concerns about privacy when using LLMs in clinical scenarios.

Conclusions: LLMs have been widely adopted by medical students. While students perceived the efficiency, flexibility, and conversational interface of LLMs as beneficial, substantial reservations remain regarding their reliability, potential de-skilling, and the loss of academic integrity. These findings underpin the urgent need for curricula to both support safe LLM use and also adapt assessment and teaching strategies for artificial intelligence-augmented student practice. Future research should broaden geographical representation, investigate applications in low-resource settings, and integrate educators' perspectives to establish future curricular guidance in an artificial intelligence era.

Background: COVID-19 accelerated the adoption of health services, with a growing number of psychosomatic patients turning to the internet for health-related decisions. This study explored changes in communication behavior, information-seeking habits, and post-pandemic consultation preferences among psychosomatic patients during and after COVID-19.

Objective: This study explored changes in communication behavior, information-seeking habits, and post-pandemic consultation preferences among psychosomatic patients during and after COVID-19.

Methods: In a cross-sectional study, 150 adult patients (>18 y) from the psychosomatic outpatient department in Tübingen, Germany, were invited to complete an ad hoc questionnaire to identify e-patients' preferences related to communication, information-seeking behavior, subjective explanations, and postpandemic preferences. Group comparisons and multiple linear regression analyses were conducted.

Results: The study revealed a slight increase in online-based communication between patients and caregivers (eg, caregivers' system use +10.8%; live video consultations +30.8%), as well as in patient-patient interactions (eg, online correspondence +16.9%). Significant group differences were observed for social media correspondence by patient age (χ2¬=17.44, P<.001) and for live video consultations by gender (χ2¬=70.17, P<.001). For both age and gender, significant group differences were found in the use of medical videos (age χ2¬=6.36, P=.04; gender χ2¬=76.70, P<.001). Age, gender, and preferences for live video consultations were identified as significant predictors (F1,11=14.195, P<.001, R²=0.299) of patients' future preferences for on-site versus online consultations.

Conclusions: Although a slight increase in online-based communication was observed, resilience in patient-caregiver and patient-patient communication indicated relative stability under challenging circumstances. Older adults and female patients expressed a preference for on-site consultations, emphasizing the significance of in-person care. Male patients demonstrated greater openness towards online consultations, indicating potential for expansion of eHealth services. Identifying preferences is a core essential for providing future eHealth implementations that account for diverse patient needs and for designing care offerings. Incorporating these findings may enhance patient engagement and satisfaction in the evolving landscape of eHealth services for better health care outcomes.

Background: People living with HIV and substance use disorders (SUDs) have complex health care needs requiring adaptive and effective support systems. While mobile health apps can foster virtual communities grounded in shared lived experiences, little is known about the dynamics within these digital spaces.

Objective: We examined the formation of a virtual community on the Addiction Comprehensive Health Enhancement Support System (A-CHESS; The Center for Health Enhancement Systems Studies, University of Wisconsin-Madison College of Engineering) message board, a mobile app designed to support HIV care engagement among individuals living with HIV and SUD.

Methods: We conducted a computer-mediated discourse analysis of A-CHESS message board posts to examine communication patterns, interaction structures, and engagement dynamics. Quantitative comparisons were used to assess differences between posters and nonposters using t tests and chi-square tests. We then applied qualitative coding to categorize messages by type, speaker, and function to understand how staff and participants coconstructed a supportive virtual environment.

Results: Among 208 participants, 87 (42%) posted at least once on the A-CHESS message board, contributing 1834 messages between April 2019 and May 2021. Posters and nonposters did not differ significantly in age (t206=-0.64; P=.52), gender (χ²1=0.14; P=.71), or race (χ²1=0.52; P=.47). We identified 3 message types: premeditated, adlib, and participant-driven. Staff initially led with premeditated messages (eg, recovery stories, HIV risk information, and "Thought of the Day" inspiration), which participants often interpreted and adapted to their own SUD recovery. Over time, staff incorporated adlib messaging styles using personalized narratives and polls to sustain engagement. Participants then developed their own posts using similar formats, incorporating Alcoholics Anonymous literature, sharing legal and personal challenges, and suggesting new app features (eg, medication check-ins to support adherence).

Conclusions: A-CHESS staff adapted communication styles to increase engagement, while participants appropriated the app's message board to reflect personal goals and lived experiences. Mobile health interventions may benefit from design elements that support participant-led discourse and customization, fostering ownership, support, and relevance within virtual care communities.

Background: Mobile health tools that use the Experience Sampling Method (ESM) appear to be a promising tool to streamline and improve the treatment of substance use disorders. However, patient involvement in the development of these tools is uncommon, and research on the preferences of people being treated for substance use disorders has been scarce. In the scope of the European Union IMMERSE (Implementing Mobile Mental health Recording Strategy for Europe) consortium, an ESM-based tool for Digital Mobile Mental Health (DMMH) was first codeveloped and later tested in 4 European countries.

Objective: This study aimed to achieve an understanding of preferences for features of DMMH among mental health service users with problem substance use.

Methods: In 4 European countries, service users were recruited for a semistructured qualitative interview, which started with the presentation of a prototype of the DMMH. Data analysis was performed through directed qualitative content analysis.

Results: The analytical sample consisted of 12 (5 female, 6 male, and 1 nonbinary person) participants with problem substance use aged 18-50 years. There were 4 participants from Slovakia, 3 from Belgium, 4 from Germany, and 1 from Scotland. Patient preferences were classified into 7 categories: notifications, questions, user interface, functionality changes, visualizations, sharing data with clinicians, and sharing data with others. The proposed intensive notification schedule was deemed acceptable by service users as long as the questionnaire is short. Participants expressed a preference for open-text response options, Ecological Momentary Interventions, and options for individual customization of several elements of the tool. Data visualization was considered an important aid for communication with clinicians, with whom all participants wanted to share data obtained with DMMH. The possibility of sharing data with other people depended on the quality of the relationship with them.

Conclusions: In the development of ESM-based mobile health tools for people with problem substance use, their preferences for content, functionality, and appearance of the tools should be considered so they match patients' treatment needs.

Background: Online harm affects many people and has been associated with self-harm and suicidal ideation. Although there is an emerging body of evidence that addressing adverse online experiences should be part of the support offered to people who are at risk of self-harm and suicide, there has been little guidance to date on how this support might be provided and how safe conversations can be had on the subject. A UK charity dedicated to offering emotional support to anyone experiencing mental discomfort, having difficulty coping, or being at risk of suicide developed a digital intervention, the Online Safety Hub (the Hub), to address this shortfall.

Objective: The study aimed to evaluate the impact of the Hub on practitioners (people who provide support) and people with lived experiences of suicide and self-harm and to determine what learning environment is best suited to increase and maintain learning in the context of the Hub.

Methods: A sequential explanatory mixed methods evaluation comprised a rapid literature review, data collected from people with lived experience (n=6) and practitioners through an analysis of the Hub's activity data, 2 surveys (survey 1: n=45; survey 2: n=368), interviews (n=9), and focus groups (n=7). Surveys were analyzed for descriptive purposes only, and the interview and focus group analyses comprised coding of data and thematic analysis. The study design was informed by a panel of people with lived experience of online harm resulting in either self-harm and/or suicidal ideation.

Results: Initially, the evaluation found limited uptake of the Hub. Engagement with the Hub was impeded by a lack of clarity on the part of practitioners as to whether they were the intended audience. The evaluation process prompted the charity to design and deliver webinars to facilitate uptake of the Hub. Practitioners who engaged with the Hub via webinars found the content useful and were able to consider incorporating their learning into practice. The webinars offered a more social learning experience than individual engagement with the Hub, providing a community of practice for people with common interests across diverse organizational settings. Opportunities for shared learning and the supportive nature of the community of practice were valued when learning about the sensitive and difficult topic of online harm in relation to self-harm and suicide. The Hub contributed to awareness-raising and shared learning.

Conclusions: Online resources alone may not be sufficient for an intervention to effectively raise awareness and change practice. Social learning facilitated through communities of practice can enhance engagement, uptake, and learning.

Background: The evolution of language models, particularly large language models, has introduced transformative potential for psychological assessment, challenging traditional rating scale methods that have dominated clinical practice for over a century.

Objective: This study aimed to develop and validate an automated assessment paradigm that integrates natural language processing with conventional measurement tools to assess depressive symptoms, exploring its feasibility as a novel approach in psychological evaluation.

Methods: A cohort of 115 participants, including 28 (24.3%) individuals diagnosed with depression, completed the Beck Depression Inventory Fast Screen via a custom ChatGPT interface (BDI-FS-GPT) and the Chinese version of the Patient Health Questionnaire-9 (PHQ-9). Statistical analyses included the Spearman correlation (PHQ-9 vs BDI-FS-GPT scores), Cohen κ (diagnostic agreement), and area under the curve (AUC) evaluation.

Results: Spearman analysis revealed a moderate correlation between PHQ-9 and BDI-FS-GPT scores. The Cohen κ indicated moderate diagnostic agreement between the PHQ-9 and the BDI-FS-GPT (κ=0.43; 76.5% agreement), substantial agreement between the BDI-FS-GPT and the clinical diagnosis (κ=0.72; 88.7% agreement), and moderate agreement between the PHQ-9 and the clinical diagnosis (κ=0.55; 71.4% agreement). The BDI-FS-GPT demonstrated excellent diagnostic accuracy (AUC=0.953) at a cutoff of 3, detecting 89.3% of participants with depression with an 11.5% false-positive rate compared to the PHQ-9 (AUC=0.859) at a cutoff of 5 (sensitivity=71.4%; false-positive rate=13.8%). Participants also reported significantly higher satisfaction with the automated assessment compared to the traditional scale (P=.02).

Conclusions: The automated assessment paradigm framework combines the interactivity and personalization of natural language processing-powered tools with the psychometric rigor of traditional scales, suggesting a preliminary feasibility paradigm for future psychological assessment. Its ability to enhance engagement while maintaining reliability and validity provides encouraging evidence, warranting validation in larger and more diverse studies as large language model technology advances.

International registered report identifier (irrid): RR2-10.1101/2024.07.19.24310543.