首页 > 最新文献

JMIR Formative Research最新文献

英文 中文
Mapping the Availability of Rehabilitation Providers Using Public Licensure and Population Data for a Geographic Information System-Based Approach to Workforce Planning: Cross-Sectional Feasibility Study. 利用公共执照和人口数据映射康复提供者的可用性,以地理信息系统为基础的劳动力规划方法:横断面可行性研究。
IF 2 Q3 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-12-23 DOI: 10.2196/85025
Madeline Ratoza, Rupal M Patel, Julia Chevan, Wayne Brewer, Katy Mitchell

Background: Access to rehabilitation services is a critical yet under-studied dimension of health equity. Among the 6 domains of access, health care provider availability, defined as the presence of sufficient health care providers to meet population needs, is particularly underexplored in rehabilitation professions such as physical and occupational therapy. Current data reporting often lacks the geographic granularity required for effective workforce planning.

Objective: The purpose of this study was to demonstrate the feasibility of mapping rehabilitation provider availability at the census tract level using geographic information systems and integrating public licensure and population data to inform equitable workforce planning.

Methods: A descriptive, cross-sectional study was conducted using publicly available state licensure data for physical and occupational therapists and demographic data from the American Community Survey. Residential addresses of rehabilitation providers were geocoded and matched to 2020 census tracts. Population-to-provider ratios were calculated and mapped using choropleth and bivariate mapping techniques. Population-to-provider ratios were calculated per tract and summarized overall and by rurality using 2020 Rural-Urban Commuting Area (RUCA) codes (urban: RUCA of 1-3; rural: RUCA of ≥4). The spatial dependence of ratios was tested using a spatial autocorrelation statistic, the global Moran I, in ArcGIS Pro using edge contiguity neighbors and row standardization.

Results: Across 6896 tracts, ratios ranged from 4.5 to 11,147 persons per provider (median 1131, IQR 537-2501). By rurality, urban tracts (n=5734, 83.1%) had a median ratio of 1141 (IQR 2054), and rural tracts (n=1162, 16.9%) had a median ratio of 1093 (IQR 1690), indicating a broadly similar central tendency with somewhat greater variability in urban areas. The population-to-provider ratio exhibited significant positive spatial autocorrelation (global Moran I=0.305; Z=40.28; P<.001), consistent with clustered pockets of high and low availability rather than random dispersion.

Conclusions: A replicable geographic information system protocol can integrate licensure and demographic data to produce interpretable population-to-provider metrics and spatial diagnostics at the census-tract level. In Texas, rehabilitation workforce availability is spatially clustered and not explained solely by an urban-rural divide, underscoring the value of small-area mapping for equitable workforce planning and policy decisions.

背景:获得康复服务是卫生公平的一个关键但尚未得到充分研究的方面。在6个可获得领域中,医疗保健提供者的可获得性(定义为有足够的医疗保健提供者来满足人口需求)在康复专业(如物理和职业治疗)中尤其未得到充分探索。当前的数据报告通常缺乏有效的劳动力规划所需的地理粒度。目的:本研究的目的是证明利用地理信息系统和整合公共执照和人口数据,在人口普查区水平绘制康复提供者可用性地图的可行性,从而为公平的劳动力规划提供信息。方法:一项描述性的横断面研究使用公开的国家物理和职业治疗师执照数据和美国社区调查的人口统计数据进行。对康复服务提供者的居住地址进行地理编码,并与2020年人口普查区相匹配。计算了人口与提供者的比率,并使用了choropleth和二元绘图技术进行了绘图。使用2020年城乡通勤区(RUCA)代码(城市:RUCA为1-3;农村:RUCA≥4)计算每个区域的人口与提供者比率,并按总体和乡村性进行汇总。使用ArcGIS Pro中的空间自相关统计量,即全球Moran I,使用边缘邻近和行标准化来测试比率的空间依赖性。结果:在6896个区域中,每个提供者的比率为4.5至11,147人(中位数为1131,IQR为537-2501)。从乡村性来看,城市地区(n=5734, 83.1%)的中位数比率为1141 (IQR 2054),农村地区(n=1162, 16.9%)的中位数比率为1093 (IQR 1690),表明两者的集中趋势大致相似,但城市地区的变动性略大。结论:一个可复制的地理信息系统协议可以整合许可证和人口统计数据,产生可解释的人口与提供者指标,并在普查区域水平上进行空间诊断。在德克萨斯州,康复劳动力的可用性在空间上是聚集的,而不仅仅是由城乡划分来解释,这强调了小区域测绘对公平的劳动力规划和政策决策的价值。
{"title":"Mapping the Availability of Rehabilitation Providers Using Public Licensure and Population Data for a Geographic Information System-Based Approach to Workforce Planning: Cross-Sectional Feasibility Study.","authors":"Madeline Ratoza, Rupal M Patel, Julia Chevan, Wayne Brewer, Katy Mitchell","doi":"10.2196/85025","DOIUrl":"10.2196/85025","url":null,"abstract":"<p><strong>Background: </strong>Access to rehabilitation services is a critical yet under-studied dimension of health equity. Among the 6 domains of access, health care provider availability, defined as the presence of sufficient health care providers to meet population needs, is particularly underexplored in rehabilitation professions such as physical and occupational therapy. Current data reporting often lacks the geographic granularity required for effective workforce planning.</p><p><strong>Objective: </strong>The purpose of this study was to demonstrate the feasibility of mapping rehabilitation provider availability at the census tract level using geographic information systems and integrating public licensure and population data to inform equitable workforce planning.</p><p><strong>Methods: </strong>A descriptive, cross-sectional study was conducted using publicly available state licensure data for physical and occupational therapists and demographic data from the American Community Survey. Residential addresses of rehabilitation providers were geocoded and matched to 2020 census tracts. Population-to-provider ratios were calculated and mapped using choropleth and bivariate mapping techniques. Population-to-provider ratios were calculated per tract and summarized overall and by rurality using 2020 Rural-Urban Commuting Area (RUCA) codes (urban: RUCA of 1-3; rural: RUCA of ≥4). The spatial dependence of ratios was tested using a spatial autocorrelation statistic, the global Moran I, in ArcGIS Pro using edge contiguity neighbors and row standardization.</p><p><strong>Results: </strong>Across 6896 tracts, ratios ranged from 4.5 to 11,147 persons per provider (median 1131, IQR 537-2501). By rurality, urban tracts (n=5734, 83.1%) had a median ratio of 1141 (IQR 2054), and rural tracts (n=1162, 16.9%) had a median ratio of 1093 (IQR 1690), indicating a broadly similar central tendency with somewhat greater variability in urban areas. The population-to-provider ratio exhibited significant positive spatial autocorrelation (global Moran I=0.305; Z=40.28; P<.001), consistent with clustered pockets of high and low availability rather than random dispersion.</p><p><strong>Conclusions: </strong>A replicable geographic information system protocol can integrate licensure and demographic data to produce interpretable population-to-provider metrics and spatial diagnostics at the census-tract level. In Texas, rehabilitation workforce availability is spatially clustered and not explained solely by an urban-rural divide, underscoring the value of small-area mapping for equitable workforce planning and policy decisions.</p>","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":"9 ","pages":"e85025"},"PeriodicalIF":2.0,"publicationDate":"2025-12-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12775756/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145810292","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Correction: Effectiveness, Usability, and Satisfaction of a Self-Administered Digital Intervention for Reducing Depression, Anxiety, and Stress in a University Community in the Andean Region of Peru: Randomized Controlled Trial. 更正:秘鲁安第斯地区大学社区中自我管理的数字干预减少抑郁、焦虑和压力的有效性、可用性和满意度:随机对照试验。
IF 2 Q3 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-12-22 DOI: 10.2196/87717
Rosario Yslado-Méndez, Stefan Escobar-Agreda, David Villarreal-Zegarra, Wilfredo Manuel Trejo Flores, Junior Duberli Sánchez-Broncano, Ana Lucia Vilela-Estrada, Jovanna Hasel Olivares Córdova, C Mahony Reategui-Rivera, Claudia Alvarez-Yslado, Leonardo Rojas-Mezarina
{"title":"Correction: Effectiveness, Usability, and Satisfaction of a Self-Administered Digital Intervention for Reducing Depression, Anxiety, and Stress in a University Community in the Andean Region of Peru: Randomized Controlled Trial.","authors":"Rosario Yslado-Méndez, Stefan Escobar-Agreda, David Villarreal-Zegarra, Wilfredo Manuel Trejo Flores, Junior Duberli Sánchez-Broncano, Ana Lucia Vilela-Estrada, Jovanna Hasel Olivares Córdova, C Mahony Reategui-Rivera, Claudia Alvarez-Yslado, Leonardo Rojas-Mezarina","doi":"10.2196/87717","DOIUrl":"10.2196/87717","url":null,"abstract":"","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":"9 ","pages":"e87717"},"PeriodicalIF":2.0,"publicationDate":"2025-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12721221/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145810287","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Usability of a Tablet-Based Cognitive Assessment Administered by Medical Assistants in General Practice: Implementation Study. 由全科医生助理管理的基于平板电脑的认知评估的可用性:实施研究。
IF 2 Q3 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-12-22 DOI: 10.2196/76010
Philipp Schaper, Alexander Hanke, Stephan Jonas, Leon Nissen, Lara Marie Reimer, Florian Schweizer, Michael Wagner, Kristin Rolke, Carolin Rosendahl, Judith Tillmann, Klaus Weckbecker, Jochen René Thyrian

Background: Digital short cognitive tests administered by medical assistants (MAs) in general practitioners' (GPs) practices have great potential for the timely identification of patients with dementia, because they can lead to targeted specialist referrals or to immediate reassurance of patients regarding their perceived concerns. However, integration of this testing approach into clinical practice requires good usability for the test itself, especially for cognitively impaired older adults.

Objective: In this implementation study, the digital version of the Montreal Cognitive Assessment (MoCA) Duo was conducted by MAs in general practice. We tested if the interaction with the test is associated with usability problems for the patients and aimed to find additional relevant constructs that should be considered for the potential implementation of such digital tests into clinical practice. We focused the study on subjective success, usability, and workload as well as their association with the result of the cognitive test to assess whether the MoCA Duo can be implemented into general practice.

Methods: In total, 10 GPs took part in the study. Within their practices, 299 GP patients (aged 51-97 years) with cognitive concerns completed the MoCA Duo administered by MAs. Subsequently, patients and MAs completed digital questionnaires regarding the interaction with the app. Usability was measured using the adapted System Usability Scale, and perceived workload using the National Aeronautics and Space Administration Task Load Index. For the perceived workload, we included an assessment of the patient by the MA. Results of the MoCA Duo were supplied to the GPs for their consultation with the patient.

Results: The results indicated good usability for the MoCA Duo. Self-assessment by the patients indicated that 64% (191/299) could perform in the test to the best of their ability, affected by their MoCA score. We found significant higher usability ratings by patients with better MoCA scores as well as by younger patients. Furthermore, the perceived workload showed overall medium workload. We found significant correlations between the subjective perceived workload of the patients and the assessment by MAs. Self-assessments as well as assessments by the MAs were significantly influenced by the MoCA scores and the age of the participants.

Conclusions: The results indicate good usability of the digital MoCA within the sample, supporting the idea that the resulting scores are adequate to assess cognitive status without dependence on technological affinity. Furthermore, the results highlight the relevance of heterogenous samples for comparable evaluation studies, based on the significant effect of cognitive status and age on usability and workload.

背景:在全科医生(gp)的实践中,由医疗助理(MAs)管理的数字简短认知测试在及时识别痴呆症患者方面具有巨大的潜力,因为它们可以导致有针对性的专家转诊或立即向患者保证其感知到的担忧。然而,将这种测试方法整合到临床实践中需要测试本身具有良好的可用性,特别是对于认知受损的老年人。目的:在本实施研究中,由MAs在一般实践中进行蒙特利尔认知评估(MoCA) Duo的数字版本。我们测试了与测试的交互是否与患者的可用性问题相关,并旨在找到其他相关结构,这些结构应该考虑到在临床实践中实施此类数字测试的可能性。我们将研究重点放在主观成功、可用性和工作量以及它们与认知测试结果的关联上,以评估MoCA Duo是否可以在一般实践中实施。方法:共10名全科医生参与研究。在他们的实践中,299名有认知问题的GP患者(51-97岁)完成了由MAs管理的MoCA Duo。随后,患者和MAs完成了关于与应用程序交互的数字问卷。使用适应性系统可用性量表测量可用性,并使用美国国家航空航天局任务负载指数测量感知工作量。对于感知的工作量,我们纳入了MA对患者的评估。MoCA Duo的结果提供给全科医生,供他们与患者咨询。结果:MoCA Duo具有较好的可用性。患者自我评估显示,64%(191/299)的患者能在MoCA评分的影响下尽其所能完成测试。我们发现MoCA评分较高的患者和年轻患者的可用性评分明显较高。此外,感知到的工作量总体上表现为中等工作量。我们发现患者主观感知的工作量与MAs的评估之间存在显著的相关性。自我评价和MAs评价均受到MoCA分数和参与者年龄的显著影响。结论:结果表明样本内的数字MoCA具有良好的可用性,支持结果分数足以评估认知状态而不依赖于技术亲和力的观点。此外,基于认知状态和年龄对可用性和工作量的显著影响,结果突出了异质样本与可比评估研究的相关性。
{"title":"Usability of a Tablet-Based Cognitive Assessment Administered by Medical Assistants in General Practice: Implementation Study.","authors":"Philipp Schaper, Alexander Hanke, Stephan Jonas, Leon Nissen, Lara Marie Reimer, Florian Schweizer, Michael Wagner, Kristin Rolke, Carolin Rosendahl, Judith Tillmann, Klaus Weckbecker, Jochen René Thyrian","doi":"10.2196/76010","DOIUrl":"10.2196/76010","url":null,"abstract":"<p><strong>Background: </strong>Digital short cognitive tests administered by medical assistants (MAs) in general practitioners' (GPs) practices have great potential for the timely identification of patients with dementia, because they can lead to targeted specialist referrals or to immediate reassurance of patients regarding their perceived concerns. However, integration of this testing approach into clinical practice requires good usability for the test itself, especially for cognitively impaired older adults.</p><p><strong>Objective: </strong>In this implementation study, the digital version of the Montreal Cognitive Assessment (MoCA) Duo was conducted by MAs in general practice. We tested if the interaction with the test is associated with usability problems for the patients and aimed to find additional relevant constructs that should be considered for the potential implementation of such digital tests into clinical practice. We focused the study on subjective success, usability, and workload as well as their association with the result of the cognitive test to assess whether the MoCA Duo can be implemented into general practice.</p><p><strong>Methods: </strong>In total, 10 GPs took part in the study. Within their practices, 299 GP patients (aged 51-97 years) with cognitive concerns completed the MoCA Duo administered by MAs. Subsequently, patients and MAs completed digital questionnaires regarding the interaction with the app. Usability was measured using the adapted System Usability Scale, and perceived workload using the National Aeronautics and Space Administration Task Load Index. For the perceived workload, we included an assessment of the patient by the MA. Results of the MoCA Duo were supplied to the GPs for their consultation with the patient.</p><p><strong>Results: </strong>The results indicated good usability for the MoCA Duo. Self-assessment by the patients indicated that 64% (191/299) could perform in the test to the best of their ability, affected by their MoCA score. We found significant higher usability ratings by patients with better MoCA scores as well as by younger patients. Furthermore, the perceived workload showed overall medium workload. We found significant correlations between the subjective perceived workload of the patients and the assessment by MAs. Self-assessments as well as assessments by the MAs were significantly influenced by the MoCA scores and the age of the participants.</p><p><strong>Conclusions: </strong>The results indicate good usability of the digital MoCA within the sample, supporting the idea that the resulting scores are adequate to assess cognitive status without dependence on technological affinity. Furthermore, the results highlight the relevance of heterogenous samples for comparable evaluation studies, based on the significant effect of cognitive status and age on usability and workload.</p>","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":"9 ","pages":"e76010"},"PeriodicalIF":2.0,"publicationDate":"2025-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12770926/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145810285","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Development of an Evaluation Index System for Health Recommender Systems Based on the Health Technology Assessment Framework: Cross-Sectional Delphi Study. 基于卫生技术评价框架的健康推荐系统评价指标体系构建:横断面德尔菲研究。
IF 2 Q3 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-12-22 DOI: 10.2196/79997
Yue Sun, Shijie Hou, Siye Chen, Minmin Leng, Zhiwen Wang
<p><strong>Background: </strong>Health recommender systems (HRSs) are digital platforms designed to deliver personalized health information, resources, and interventions tailored to users' specific needs. However, existing evaluations of HRSs largely focus on algorithmic performance, with limited scientific evidence supporting user-centered assessment approaches and insufficiently defined evaluation metrics. Moreover, no unified or scientifically validated framework currently exists for evaluating these systems, resulting in limited cross-study comparability and constraining regulatory and implementation decision-making.</p><p><strong>Objective: </strong>This study aimed to develop a comprehensive, consensus-based evaluation index system for HRSs grounded in the health technology assessment (HTA) framework.</p><p><strong>Methods: </strong>This cross-sectional study used a 2-round Delphi process conducted with 18 experts comprising clinicians, digital health researchers, and policymakers who possessed relevant professional experience and domain knowledge in HRSs. The age range of the experts was between 30 and 58 years, with 67% (n=12) of them possessing over 10 years of professional experience. On the basis of literature analysis and HTA principles, a preliminary indicator set comprising 5 primary and 16 secondary indicators was constructed. Experts rated the importance of each indicator using a 5-point Likert scale and provided qualitative suggestions for refinement. After the Delphi process, the analytic hierarchy process was applied to determine indicator weights and assess consistency.</p><p><strong>Results: </strong>The Delphi survey reached full participation in the first round (18/18, 100%) and maintained an 88.9% (16/18) response rate in the second round. The final evaluation index system of HRSs contained 5 first-level indicators (performance, effectiveness, safety, economy, and social appropriateness) and 18 second-level indicators. The mean importance scores of the second-level indicators ranged from 4.25 (SD 0.45) to 5.00 (SD 0.00), with coefficients of variation between 0.000 and 0.220. Among the first-level indicators, safety received the highest weight (0.289), followed by social appropriateness (0.251), effectiveness (0.193), performance (0.136), and economy (0.132).</p><p><strong>Conclusions: </strong>This study presents an evaluation index system for HRSs grounded in the HTA framework and validated through expert consensus. The resulting framework not only provides actionable guidance for the design, optimization, and implementation of HRSs but also fills a methodological gap in the field by offering quantifiable, hierarchical evaluation indicators with validated weighting. Future research will involve iterative refinement and empirical validation of the system in real-world deployment settings, thereby enabling continuous improvement and facilitating the establishment of unified evaluation standards for HRS research and practic
背景:健康推荐系统(HRSs)是一种数字平台,旨在根据用户的具体需求提供个性化的健康信息、资源和干预措施。然而,现有的人力资源系统评估主要集中在算法性能上,支持以用户为中心的评估方法的科学证据有限,评估指标定义不足。此外,目前还没有统一的或经过科学验证的框架来评估这些系统,导致交叉研究的可比性有限,并制约了监管和实施决策。目的:在卫生技术评价(HTA)框架下,建立一套全面的、基于共识的卫生服务质量评价指标体系。方法:采用2轮德尔菲法进行横断面研究,共有18名专家参与,包括临床医生、数字卫生研究人员和政策制定者,他们具有相关的人力资源服务专业经验和领域知识。专家的年龄在30 - 58岁之间,其中有67% (n=12)的专家拥有10年以上的专业经验。在文献分析和HTA原理的基础上,构建了包含5个一级指标和16个二级指标的初步指标集。专家们使用5分李克特量表对每个指标的重要性进行评级,并提供改进的定性建议。在德尔菲法之后,运用层次分析法确定指标权重和一致性评价。结果:德尔菲调查在第一轮达到了全员参与(18/18,100%),在第二轮保持了88.9%(16/18)的回复率。最终的HRSs评价指标体系包括5个一级指标(绩效、有效性、安全性、经济性和社会适宜性)和18个二级指标。二级指标的平均重要性评分范围为4.25 (SD 0.45) ~ 5.00 (SD 0.00),变异系数为0.000 ~ 0.220。在一级指标中,安全性权重最高(0.289),其次是社会适宜性(0.251)、有效性(0.193)、绩效(0.136)和经济性(0.132)。结论:本研究提出了基于HTA框架并经专家共识验证的HRSs评价指标体系。由此产生的框架不仅为HRSs的设计、优化和实施提供了可操作的指导,而且通过提供具有有效权重的可量化、分层评估指标,填补了该领域的方法论空白。未来的研究将涉及系统在实际部署环境中的迭代细化和经验验证,从而实现持续改进,促进HRS研究和实践统一评价标准的建立。
{"title":"Development of an Evaluation Index System for Health Recommender Systems Based on the Health Technology Assessment Framework: Cross-Sectional Delphi Study.","authors":"Yue Sun, Shijie Hou, Siye Chen, Minmin Leng, Zhiwen Wang","doi":"10.2196/79997","DOIUrl":"10.2196/79997","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Background: &lt;/strong&gt;Health recommender systems (HRSs) are digital platforms designed to deliver personalized health information, resources, and interventions tailored to users' specific needs. However, existing evaluations of HRSs largely focus on algorithmic performance, with limited scientific evidence supporting user-centered assessment approaches and insufficiently defined evaluation metrics. Moreover, no unified or scientifically validated framework currently exists for evaluating these systems, resulting in limited cross-study comparability and constraining regulatory and implementation decision-making.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;This study aimed to develop a comprehensive, consensus-based evaluation index system for HRSs grounded in the health technology assessment (HTA) framework.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;This cross-sectional study used a 2-round Delphi process conducted with 18 experts comprising clinicians, digital health researchers, and policymakers who possessed relevant professional experience and domain knowledge in HRSs. The age range of the experts was between 30 and 58 years, with 67% (n=12) of them possessing over 10 years of professional experience. On the basis of literature analysis and HTA principles, a preliminary indicator set comprising 5 primary and 16 secondary indicators was constructed. Experts rated the importance of each indicator using a 5-point Likert scale and provided qualitative suggestions for refinement. After the Delphi process, the analytic hierarchy process was applied to determine indicator weights and assess consistency.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;The Delphi survey reached full participation in the first round (18/18, 100%) and maintained an 88.9% (16/18) response rate in the second round. The final evaluation index system of HRSs contained 5 first-level indicators (performance, effectiveness, safety, economy, and social appropriateness) and 18 second-level indicators. The mean importance scores of the second-level indicators ranged from 4.25 (SD 0.45) to 5.00 (SD 0.00), with coefficients of variation between 0.000 and 0.220. Among the first-level indicators, safety received the highest weight (0.289), followed by social appropriateness (0.251), effectiveness (0.193), performance (0.136), and economy (0.132).&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Conclusions: &lt;/strong&gt;This study presents an evaluation index system for HRSs grounded in the HTA framework and validated through expert consensus. The resulting framework not only provides actionable guidance for the design, optimization, and implementation of HRSs but also fills a methodological gap in the field by offering quantifiable, hierarchical evaluation indicators with validated weighting. Future research will involve iterative refinement and empirical validation of the system in real-world deployment settings, thereby enabling continuous improvement and facilitating the establishment of unified evaluation standards for HRS research and practic","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":"9 ","pages":"e79997"},"PeriodicalIF":2.0,"publicationDate":"2025-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12721223/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145804500","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Radiology Staff Experiences With Integrating Artificial Intelligence Into Radiology Practice in a Swedish Hospital: Qualitative Case Study. 瑞典一家医院放射科工作人员将人工智能融入放射科实践的经验:定性案例研究。
IF 2 Q3 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-12-22 DOI: 10.2196/77843
Per Nilsen, Petra Svedberg, Ingrid Larsson, Lena Petersson, Jens Nygren, Emilie Steerling, Margit Neher

Background: The integration of artificial intelligence (AI) in radiology has advanced significantly, but research on how it affects the daily work of radiology staff is limited.

Objective: This study aimed to explore the experiences of radiology staff on the integration of an AI application in a radiology department in Sweden. This understanding is essential for developing strategies to address potential challenges in AI integration and optimize the use of AI applications in radiology practice.

Methods: This qualitative case study was conducted in a single radiology department with 40 employees in 1 hospital in southwestern Sweden. The study concerned the integration of an AI-powered medical imaging software designed to assist radiologists in analyzing and interpreting medical images. Using a qualitative design, interviews were conducted with 7 radiologists (physicians), 4 radiologic technologists, and 1 physician assistant. Their experience within radiology varied between 13 months and 38 years. The data were analyzed using qualitative content analysis.

Results: Participants cited numerous strengths and advantages of significant value in integrating AI into radiology practice. Numerous challenges were also revealed in terms of difficulties associated with choosing, acquiring, and deploying the AI application and operational issues in radiology practice. They discussed experiences with diverse strategies to facilitate the integration of AI in radiology and to address various challenges and problems.

Conclusions: The findings illustrate how AI integration was experienced in 1 hospital. While not generalizable, the study provides insights that may be useful for similar settings. Radiology staff believed AI integration enhanced decision-making and quality of care, but they encountered challenges from preadoption to routine use of AI in radiology practice. Strategies such as internal training and workflow adaptation facilitated the successful integration of AI in radiology.

背景:人工智能(AI)与放射学的融合取得了显著进展,但对其如何影响放射科工作人员日常工作的研究有限。目的:本研究旨在探讨瑞典某放射科放射科工作人员在人工智能应用集成方面的经验。这种理解对于制定战略以应对人工智能集成中的潜在挑战和优化人工智能应用在放射学实践中的使用至关重要。方法:本定性案例研究是在瑞典西南部一家医院的一个有40名员工的放射科进行的。该研究涉及人工智能医学成像软件的集成,旨在帮助放射科医生分析和解释医学图像。采用定性设计,对7名放射科医师、4名放射技师和1名医师助理进行访谈。他们在放射学领域的经验从13个月到38年不等。采用定性内容分析法对数据进行分析。结果:与会者列举了将人工智能整合到放射学实践中的许多重要价值的优势和优势。在选择、获取和部署人工智能应用程序以及放射学实践中的操作问题方面,还揭示了许多挑战。他们讨论了各种策略的经验,以促进人工智能在放射学中的整合,并解决各种挑战和问题。结论:研究结果说明了1家医院如何体验人工智能集成。虽然不能一概而论,但这项研究提供了可能对类似情况有用的见解。放射科工作人员认为,人工智能集成提高了决策和护理质量,但他们在放射科实践中遇到了从预采用到常规使用人工智能的挑战。内部培训和工作流程适应等策略促进了人工智能在放射学中的成功整合。
{"title":"Radiology Staff Experiences With Integrating Artificial Intelligence Into Radiology Practice in a Swedish Hospital: Qualitative Case Study.","authors":"Per Nilsen, Petra Svedberg, Ingrid Larsson, Lena Petersson, Jens Nygren, Emilie Steerling, Margit Neher","doi":"10.2196/77843","DOIUrl":"10.2196/77843","url":null,"abstract":"<p><strong>Background: </strong>The integration of artificial intelligence (AI) in radiology has advanced significantly, but research on how it affects the daily work of radiology staff is limited.</p><p><strong>Objective: </strong>This study aimed to explore the experiences of radiology staff on the integration of an AI application in a radiology department in Sweden. This understanding is essential for developing strategies to address potential challenges in AI integration and optimize the use of AI applications in radiology practice.</p><p><strong>Methods: </strong>This qualitative case study was conducted in a single radiology department with 40 employees in 1 hospital in southwestern Sweden. The study concerned the integration of an AI-powered medical imaging software designed to assist radiologists in analyzing and interpreting medical images. Using a qualitative design, interviews were conducted with 7 radiologists (physicians), 4 radiologic technologists, and 1 physician assistant. Their experience within radiology varied between 13 months and 38 years. The data were analyzed using qualitative content analysis.</p><p><strong>Results: </strong>Participants cited numerous strengths and advantages of significant value in integrating AI into radiology practice. Numerous challenges were also revealed in terms of difficulties associated with choosing, acquiring, and deploying the AI application and operational issues in radiology practice. They discussed experiences with diverse strategies to facilitate the integration of AI in radiology and to address various challenges and problems.</p><p><strong>Conclusions: </strong>The findings illustrate how AI integration was experienced in 1 hospital. While not generalizable, the study provides insights that may be useful for similar settings. Radiology staff believed AI integration enhanced decision-making and quality of care, but they encountered challenges from preadoption to routine use of AI in radiology practice. Strategies such as internal training and workflow adaptation facilitated the successful integration of AI in radiology.</p>","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":"9 ","pages":"e77843"},"PeriodicalIF":2.0,"publicationDate":"2025-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12721490/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145810290","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Facilitators, Barriers, and Cultural Appropriateness of Mindfulness-Based Interventions Among Saudi Female University Students: Qualitative Study. 沙特女大学生正念干预的促进因素、障碍和文化适宜性:定性研究。
IF 2 Q3 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-12-19 DOI: 10.2196/78532
Duaa H Alrashdi, Carly Meyer, Rebecca L Gould
<p><strong>Background: </strong>Mindfulness-based interventions (MBIs) have been shown to improve university students' well-being. However, previous studies have not systematically explored factors that can facilitate or hinder engagement in MBIs among Saudi university students, nor how MBIs can be culturally adapted to meet their needs.</p><p><strong>Objective: </strong>This study aimed to (1) explore the perspectives of Saudi female university students about factors influencing engagement with MBIs, (2) explore the cultural appropriateness of MBIs, and (3) systematically identify recommendations for developing a culturally appropriate MBI.</p><p><strong>Methods: </strong>A qualitative research approach was used to collect data using semistructured individual interviews and focus groups. Two established frameworks for behavioral interventions were applied to guide the interview topics and data analysis. The COM-B (Capability, Opportunity, and Motivation Domains of Behavior Change) model was applied to identify potential enablers and barriers influencing students' engagement with MBIs. The cultural adaptation framework by Bernal et al was used to explore the cultural appropriateness of MBIs. Subsequently, recommendations for developing MBIs, with a specific focus on an online version, were systematically formulated using the Theory and Techniques Tool. Data were analyzed using mixed inductive-deductive thematic analysis.</p><p><strong>Results: </strong>Fourteen Saudi female university students (mean age 24, SD 4.9 years) participated in semistructured interviews and focus groups. Numerous potential enablers and barriers to MBI engagement were identified. Factors that may influence engagement pertained to capability (variation in knowledge of mindfulness), opportunity (anticipated difficulty finding time), and motivation (variation in anticipated and experienced benefits of mindfulness). Participants also highlighted several considerations that may enhance the cultural relevance of MBIs, drawing on the cultural adaptation domains by Bernal et al. These included the importance of aligning MBIs with the local cultural context, incorporating metaphors and examples rooted in Saudi and Arab culture, and accommodating students' preferences for the duration of MBIs. Key recommendations for developing culturally appropriate MBIs for Saudi university students included providing clear information to improve understanding of mindfulness, providing practical strategies and skills to overcome barriers such as time constraints, delivering MBIs in both Arabic and English, and ensuring that MBIs' content aligns with local cultural values and contexts.</p><p><strong>Conclusions: </strong>Findings and recommendations aim to enhance the feasibility, acceptability, engagement, and effectiveness of MBIs among Saudi university students, particularly female students. However, whether they do in fact achieve these aims is unknown. Future research should endeavor to evalu
背景:正念干预(mbi)已被证明可以改善大学生的幸福感。然而,以前的研究并没有系统地探讨沙特大学生参与MBIs的因素,也没有系统地探讨MBIs如何在文化上适应以满足他们的需求。目的:本研究旨在(1)探讨沙特女大学生对mbbi参与影响因素的看法,(2)探讨mbbi的文化适宜性,(3)系统地确定发展文化适宜的MBI的建议。方法:采用定性研究方法,采用半结构化的个人访谈和焦点小组的方式收集数据。采用两个已建立的行为干预框架来指导访谈主题和数据分析。COM-B(行为改变的能力、机会和动机领域)模型被用于识别影响学生参与MBIs的潜在因素和障碍。本研究采用Bernal等人的文化适应框架来探讨MBIs的文化适宜性。随后,使用理论和技术工具系统地制定了开发mbi的建议,特别是在线版本。数据分析采用混合归纳-演绎主题分析。结果:14名沙特女大学生(平均24岁,SD 4.9岁)参加了半结构化访谈和焦点小组。确定了MBI参与的许多潜在促成因素和障碍。可能影响参与的因素与能力(正念知识的变化)、机会(预期的找时间困难)和动机(正念预期和体验的好处的变化)有关。与会者还借鉴Bernal等人的文化适应领域,强调了可能增强mbi文化相关性的几个考虑因素。其中包括将mbi与当地文化背景结合起来的重要性,纳入根植于沙特和阿拉伯文化的隐喻和例子,以及在mbi期间适应学生的偏好。为沙特大学生开发适合文化的mbi的主要建议包括提供清晰的信息以提高对正念的理解,提供实用的策略和技能以克服时间限制等障碍,以阿拉伯语和英语提供mbi,并确保mbi的内容符合当地的文化价值观和背景。结论:研究结果和建议旨在提高沙特大学生,特别是女大学生MBIs的可行性、可接受性、参与度和有效性。然而,他们是否真的实现了这些目标是未知的。未来的研究应努力评估建议的有效性,并探索在更广泛的沙特学生群体中参与MBI的推动因素和障碍。
{"title":"Facilitators, Barriers, and Cultural Appropriateness of Mindfulness-Based Interventions Among Saudi Female University Students: Qualitative Study.","authors":"Duaa H Alrashdi, Carly Meyer, Rebecca L Gould","doi":"10.2196/78532","DOIUrl":"10.2196/78532","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Background: &lt;/strong&gt;Mindfulness-based interventions (MBIs) have been shown to improve university students' well-being. However, previous studies have not systematically explored factors that can facilitate or hinder engagement in MBIs among Saudi university students, nor how MBIs can be culturally adapted to meet their needs.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;This study aimed to (1) explore the perspectives of Saudi female university students about factors influencing engagement with MBIs, (2) explore the cultural appropriateness of MBIs, and (3) systematically identify recommendations for developing a culturally appropriate MBI.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;A qualitative research approach was used to collect data using semistructured individual interviews and focus groups. Two established frameworks for behavioral interventions were applied to guide the interview topics and data analysis. The COM-B (Capability, Opportunity, and Motivation Domains of Behavior Change) model was applied to identify potential enablers and barriers influencing students' engagement with MBIs. The cultural adaptation framework by Bernal et al was used to explore the cultural appropriateness of MBIs. Subsequently, recommendations for developing MBIs, with a specific focus on an online version, were systematically formulated using the Theory and Techniques Tool. Data were analyzed using mixed inductive-deductive thematic analysis.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;Fourteen Saudi female university students (mean age 24, SD 4.9 years) participated in semistructured interviews and focus groups. Numerous potential enablers and barriers to MBI engagement were identified. Factors that may influence engagement pertained to capability (variation in knowledge of mindfulness), opportunity (anticipated difficulty finding time), and motivation (variation in anticipated and experienced benefits of mindfulness). Participants also highlighted several considerations that may enhance the cultural relevance of MBIs, drawing on the cultural adaptation domains by Bernal et al. These included the importance of aligning MBIs with the local cultural context, incorporating metaphors and examples rooted in Saudi and Arab culture, and accommodating students' preferences for the duration of MBIs. Key recommendations for developing culturally appropriate MBIs for Saudi university students included providing clear information to improve understanding of mindfulness, providing practical strategies and skills to overcome barriers such as time constraints, delivering MBIs in both Arabic and English, and ensuring that MBIs' content aligns with local cultural values and contexts.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Conclusions: &lt;/strong&gt;Findings and recommendations aim to enhance the feasibility, acceptability, engagement, and effectiveness of MBIs among Saudi university students, particularly female students. However, whether they do in fact achieve these aims is unknown. Future research should endeavor to evalu","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":"9 ","pages":"e78532"},"PeriodicalIF":2.0,"publicationDate":"2025-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12716633/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145793968","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Association Between eHealth Literacy and Mental Health Literacy: Cross-Sectional Study. 电子健康素养与心理健康素养的关系:横断面研究
IF 2 Q3 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-12-19 DOI: 10.2196/76812
Efrat Neter, Refael Youngmann, Naama Gruper

Unlabelled: Associations between eHealth literacy and mental health literacy were examined; no significant association was identified between overall eHealth and mental health literacy and only weak associations between specific skills were recorded. Results are interpreted in lieu of a difference between perceived ability and actual performance.

未标记:检查了电子健康素养与心理健康素养之间的关系;总体电子健康和心理健康素养之间没有明显的联系,特定技能之间只有微弱的联系。结果被解释为代替感知能力和实际表现之间的差异。
{"title":"Association Between eHealth Literacy and Mental Health Literacy: Cross-Sectional Study.","authors":"Efrat Neter, Refael Youngmann, Naama Gruper","doi":"10.2196/76812","DOIUrl":"10.2196/76812","url":null,"abstract":"<p><strong>Unlabelled: </strong>Associations between eHealth literacy and mental health literacy were examined; no significant association was identified between overall eHealth and mental health literacy and only weak associations between specific skills were recorded. Results are interpreted in lieu of a difference between perceived ability and actual performance.</p>","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":"9 ","pages":"e76812"},"PeriodicalIF":2.0,"publicationDate":"2025-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12716232/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145793871","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
mHealth as a Key Component of a New Model of Primary Care for Older Adults. 移动医疗是老年人初级保健新模式的关键组成部分。
IF 2 Q3 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-12-19 DOI: 10.2196/82262
Jean Woo, Ruby Yu, Maggie Wong, Ken Cheung, Nicole Fung

Unlabelled: With population aging, an increase in total life expectancy at birth (TLE) should ideally be accompanied by an equal increase in health span (HS), or by a trend in increasing HS/TLE ratio. Hong Kong has one of the longest life expectancies in the world; however, there is a trend of declining HS/TLE ratio, such that the absolute number of people with dependencies is increasing. To address this challenge, the World Health Organization proposed the model of integrated care for older people (ICOPE) that combines both health and social elements in community care, using the measurement of intrinsic capacity (IC) as a metric for monitoring the performance in different countries. The use of technology is essential in achieving a wide coverage of the population in assessing IC, followed by an individually tailored plan of action. This model can be adapted to different health and social care systems in different countries. Hong Kong has an extensive network of community centers, where the basic assessment may be based, followed by further assessments and personalized activities, and referral to medical professionals may only be needed in the presence of disease. Conversely, the medical sector may refer patients to the community for activities designed to optimize the various domains of IC. Such a model of care has the potential to address manpower shortage and mitigate inequalities in healthy aging, as well as enable the monitoring of physiological systems in community-dwelling adults using digital biomarkers as a metric of IC.

未标明:随着人口老龄化,出生时总预期寿命(TLE)的增加,理想情况下应伴随着健康寿命(HS)的同等增长,或HS/TLE比率呈上升趋势。香港是世界上预期寿命最长的地区之一;然而,HS/TLE比率呈下降趋势,因此依赖者的绝对人数正在增加。为了应对这一挑战,世界卫生组织提出了老年人综合护理模式(ICOPE),将社区护理中的健康和社会因素结合起来,使用内在能力(IC)的衡量作为监测不同国家绩效的指标。技术的使用对于实现广泛覆盖人口来评估综合评估,然后制定适合个人的行动计划是至关重要的。这一模式可适用于不同国家的不同卫生和社会保健系统。香港有一个广泛的社区中心网络,在那里可以进行基本评估,然后进行进一步评估和个性化活动,只有在出现疾病时才需要转介给医疗专业人员。相反,医疗部门可能会将患者转介到社区进行旨在优化IC各个领域的活动。这种护理模式有可能解决人力短缺问题,减轻健康老龄化方面的不平等,并能够使用数字生物标志物作为IC的度量来监测社区居住成年人的生理系统。
{"title":"mHealth as a Key Component of a New Model of Primary Care for Older Adults.","authors":"Jean Woo, Ruby Yu, Maggie Wong, Ken Cheung, Nicole Fung","doi":"10.2196/82262","DOIUrl":"10.2196/82262","url":null,"abstract":"<p><strong>Unlabelled: </strong>With population aging, an increase in total life expectancy at birth (TLE) should ideally be accompanied by an equal increase in health span (HS), or by a trend in increasing HS/TLE ratio. Hong Kong has one of the longest life expectancies in the world; however, there is a trend of declining HS/TLE ratio, such that the absolute number of people with dependencies is increasing. To address this challenge, the World Health Organization proposed the model of integrated care for older people (ICOPE) that combines both health and social elements in community care, using the measurement of intrinsic capacity (IC) as a metric for monitoring the performance in different countries. The use of technology is essential in achieving a wide coverage of the population in assessing IC, followed by an individually tailored plan of action. This model can be adapted to different health and social care systems in different countries. Hong Kong has an extensive network of community centers, where the basic assessment may be based, followed by further assessments and personalized activities, and referral to medical professionals may only be needed in the presence of disease. Conversely, the medical sector may refer patients to the community for activities designed to optimize the various domains of IC. Such a model of care has the potential to address manpower shortage and mitigate inequalities in healthy aging, as well as enable the monitoring of physiological systems in community-dwelling adults using digital biomarkers as a metric of IC.</p>","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":"9 ","pages":"e82262"},"PeriodicalIF":2.0,"publicationDate":"2025-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12716121/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145793995","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Comparing ChatGPT and DeepSeek for Assessment of Multiple-Choice Questions in Orthopedic Medical Education: Cross-Sectional Study. 比较ChatGPT和DeepSeek在骨科医学教育中多项选择题的评估:横断面研究。
IF 2 Q3 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-12-19 DOI: 10.2196/75607
Chirathit Anusitviwat, Sitthiphong Suwannaphisit, Jongdee Bvonpanttarananon, Boonsin Tangtrakulwanich

Background: Multiple-choice questions (MCQs) are essential in medical education for assessing knowledge and clinical reasoning. Traditional MCQ development involves expert reviews and revisions, which can be time-consuming and subject to bias. Large language models (LLMs) have emerged as potential tools for evaluating MCQ accuracy and efficiency. However, direct comparisons of these models in orthopedic MCQ assessments are limited.

Objective: This study compared the performance of ChatGPT and DeepSeek in terms of correctness, response time, and reliability when answering MCQs from an orthopedic examination for medical students.

Methods: This cross-sectional study included 209 orthopedic MCQs from summative assessments during the 2023-2024 academic year. ChatGPT (including the "Reason" function) and DeepSeek (including the "DeepThink" function) were used to identify the correct answers. Correctness and response times were recorded and compared using a χ2 test and Mann-Whitney U test where appropriate. The two LLMs' reliability was assessed using the Cohen κ coefficient. The MCQs incorrectly answered by both models were reviewed by orthopedic faculty to identify ambiguities or content issues.

Results: ChatGPT achieved a correctness rate of 80.38% (168/209), while DeepSeek achieved 74.2% (155/209; P=.04). ChatGPT's Reason function also outperformed DeepSeek's DeepThink function (177/209, 84.7% vs 168/209, 80.4%; P=.12). The average response time for ChatGPT was 10.40 (SD 13.29) seconds, significantly shorter than DeepSeek's 34.42 (SD 25.48) seconds (P<.001). Regarding reliability, ChatGPT demonstrated an almost perfect agreement (κ=0.81), whereas DeepSeek showed substantial agreement (κ=0.78). A completely false response was recorded in 7.7% (16/209) of responses for both models.

Conclusions: ChatGPT outperformed DeepSeek in correctness and response time, demonstrating its efficiency in evaluating orthopedic MCQs. This high reliability suggests its potential for integration into medical assessments. However, our results indicate that some MCQs will require revisions by instructors to improve their clarity. Further studies are needed to evaluate the role of artificial intelligence in other disciplines and to validate other LLMs.

背景:在医学教育中,多项选择题(mcq)是评估知识和临床推理的必要条件。传统的MCQ开发涉及专家评审和修订,这既耗时又容易产生偏见。大型语言模型(llm)已经成为评估MCQ准确性和效率的潜在工具。然而,这些模型在骨科MCQ评估中的直接比较是有限的。目的:本研究比较了ChatGPT和DeepSeek在回答医学生骨科检查mcq时的正确性、响应时间和可靠性。方法:本横断面研究纳入2023-2024学年总结性评估的209例骨科mcq。ChatGPT(包括“Reason”功能)和DeepSeek(包括“DeepThink”功能)被用来识别正确答案。记录正确性和响应时间,并使用χ2检验和Mann-Whitney U检验进行比较。采用Cohen κ系数评估两种llm的信度。骨科教师对两种模型错误回答的mcq进行了审查,以确定歧义或内容问题。结果:ChatGPT的正确率为80.38% (168/209),DeepSeek的正确率为74.2% (155/209;P=.04)。ChatGPT的Reason函数也优于DeepSeek的DeepThink函数(177/209,84.7% vs 168/209, 80.4%; P= 0.12)。ChatGPT的平均响应时间为10.40 (SD 13.29)秒,显著短于DeepSeek的34.42 (SD 25.48)秒。结论:ChatGPT在正确率和响应时间上均优于DeepSeek,证明了其在骨科mcq评估中的有效性。这种高可靠性表明它有可能整合到医学评估中。然而,我们的研究结果表明,一些mcq需要教师进行修订,以提高其清晰度。需要进一步的研究来评估人工智能在其他学科中的作用,并验证其他法学硕士。
{"title":"Comparing ChatGPT and DeepSeek for Assessment of Multiple-Choice Questions in Orthopedic Medical Education: Cross-Sectional Study.","authors":"Chirathit Anusitviwat, Sitthiphong Suwannaphisit, Jongdee Bvonpanttarananon, Boonsin Tangtrakulwanich","doi":"10.2196/75607","DOIUrl":"10.2196/75607","url":null,"abstract":"<p><strong>Background: </strong>Multiple-choice questions (MCQs) are essential in medical education for assessing knowledge and clinical reasoning. Traditional MCQ development involves expert reviews and revisions, which can be time-consuming and subject to bias. Large language models (LLMs) have emerged as potential tools for evaluating MCQ accuracy and efficiency. However, direct comparisons of these models in orthopedic MCQ assessments are limited.</p><p><strong>Objective: </strong>This study compared the performance of ChatGPT and DeepSeek in terms of correctness, response time, and reliability when answering MCQs from an orthopedic examination for medical students.</p><p><strong>Methods: </strong>This cross-sectional study included 209 orthopedic MCQs from summative assessments during the 2023-2024 academic year. ChatGPT (including the \"Reason\" function) and DeepSeek (including the \"DeepThink\" function) were used to identify the correct answers. Correctness and response times were recorded and compared using a χ2 test and Mann-Whitney U test where appropriate. The two LLMs' reliability was assessed using the Cohen κ coefficient. The MCQs incorrectly answered by both models were reviewed by orthopedic faculty to identify ambiguities or content issues.</p><p><strong>Results: </strong>ChatGPT achieved a correctness rate of 80.38% (168/209), while DeepSeek achieved 74.2% (155/209; P=.04). ChatGPT's Reason function also outperformed DeepSeek's DeepThink function (177/209, 84.7% vs 168/209, 80.4%; P=.12). The average response time for ChatGPT was 10.40 (SD 13.29) seconds, significantly shorter than DeepSeek's 34.42 (SD 25.48) seconds (P<.001). Regarding reliability, ChatGPT demonstrated an almost perfect agreement (κ=0.81), whereas DeepSeek showed substantial agreement (κ=0.78). A completely false response was recorded in 7.7% (16/209) of responses for both models.</p><p><strong>Conclusions: </strong>ChatGPT outperformed DeepSeek in correctness and response time, demonstrating its efficiency in evaluating orthopedic MCQs. This high reliability suggests its potential for integration into medical assessments. However, our results indicate that some MCQs will require revisions by instructors to improve their clarity. Further studies are needed to evaluate the role of artificial intelligence in other disciplines and to validate other LLMs.</p>","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":"9 ","pages":"e75607"},"PeriodicalIF":2.0,"publicationDate":"2025-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12716854/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145793915","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Identification of Design Requirements for a Software Application for Use by Clinicians That Collects Acute Stroke Treatment Data During Clinical Workflow: Pilot Study. 在临床工作流程中收集急性中风治疗数据的临床医生使用的软件应用程序的设计需求识别:试点研究。
IF 2 Q3 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-12-19 DOI: 10.2196/64800
Adam Forward, Gizem Koca, Aymane Sahli, Noreen Kamal
<p><strong>Background: </strong>Clinical registries are critical for monitoring processes of care in diseases and driving quality improvements. However, many smaller hospitals lack the required resources to collect the necessary data to contribute to registries.</p><p><strong>Objective: </strong>This study aims to design and evaluate a data collection tool for acute stroke treatment that streamlines the collection of process data and provides tools to aid clinician users while not interfering with clinical workflow. The evaluation will identify key design requirements that facilitate prospective data collection and add value for clinicians.</p><p><strong>Methods: </strong>We developed a prototype tool for testing using Figma Pro for use on an iPad. Clinicians were recruited through convenience sampling to test the prototype's use in a small-scale simulated clinical field experiment, during which participant were asked to think aloud and then complete a series of tasks to mimic a mock stroke treatment while inputting the required data into the prototype. Follow-up semistructured interviews were conducted to gain feedback on how the prototype integrated into the workflow and on the aspects of the prototype they felt helped and hindered their use of it. Qualitative data analysis combined review of the experiment recordings to identify the most frequent errors made during the scenario and deductive thematic analysis from the follow-up interviews to determine user needs for the following prototype iteration. The insights from the feedback identified design requirements that were implemented in the iterated design and documented to provide a reference for future product designers.</p><p><strong>Results: </strong>Three participants were recruited from 2 hospitals between April 18 and June 6, 2024, for the simulated field experiment. The scenario took 10-12 minutes, with 1.2-3.7 minutes spent using the prototype, depending on whether optional features such as the NIHSS (National Institute of Health Stroke Scale) calculator were used. The simple and condensed layout and features such as NIHSS calculators, benchmark metric timers, and the final pop-up summary received the most positive feedback from each participant. Issues identified included small target sizes causing higher error rates, lack of color in important features reducing their visibility, and grouping of mandatory and optional information field layouts leading to a disjointed flow. The key design requirements include prioritizing simple dynamic layouts, sufficient target sizes to prevent errors, useful features with clear visual cues, and prompt data feedback to facilitate seamless integration.</p><p><strong>Conclusions: </strong>A prospective data collection tool for clinicians to use during stroke treatment can add value for clinicians and, with further testing, can be integrated into workflow. The design requirements identified through this study can provide a basis for streamlining the col
背景:临床登记对于监测疾病护理过程和推动质量改进至关重要。然而,许多较小的医院缺乏必要的资源来收集必要的数据,以促进登记。目的:本研究旨在设计和评估一种用于急性脑卒中治疗的数据收集工具,该工具简化了过程数据的收集,并为临床医生用户提供了工具,同时不干扰临床工作流程。评估将确定促进前瞻性数据收集和为临床医生增加价值的关键设计要求。方法:我们开发了一个原型工具来测试Figma Pro在iPad上的使用。通过方便抽样的方式招募临床医生,在小规模的模拟临床现场实验中测试原型的使用情况,在此过程中,参与者被要求大声思考,然后完成一系列模仿模拟中风治疗的任务,同时将所需的数据输入到原型中。随后进行了半结构化访谈,以获得关于原型如何集成到工作流程中的反馈,以及他们认为原型的哪些方面有助于和阻碍了他们对原型的使用。定性数据分析结合了对实验记录的回顾,以确定场景中最常见的错误,并从后续访谈中进行演绎主题分析,以确定后续原型迭代的用户需求。来自反馈的见解确定了在迭代设计中实现的设计需求,并记录下来,为未来的产品设计师提供参考。结果:于2024年4月18日至6月6日从2家医院招募3名受试者进行模拟野外实验。根据是否使用NIHSS(美国国立卫生研究院卒中量表)计算器等可选功能,该场景需要10-12分钟,其中1.2-3.7分钟用于使用原型。简洁的布局和NIHSS计算器、基准度量计时器和最终弹出式摘要等功能从每个参与者那里获得了最积极的反馈。确定的问题包括小的目标尺寸导致更高的错误率,重要功能缺少颜色降低了其可见性,以及强制性和可选信息字段布局的分组导致不连贯的流程。关键的设计要求包括优先考虑简单的动态布局,足够的目标尺寸以防止错误,具有清晰视觉提示的有用功能,以及及时的数据反馈以促进无缝集成。结论:临床医生在卒中治疗期间使用的前瞻性数据收集工具可以为临床医生增加价值,并且通过进一步的测试,可以整合到工作流程中。通过本研究确定的设计需求可以为简化准确数据的收集提供基础,同时增加工具对用户的价值,未来的产品设计师应该考虑为他们的软件增加价值并改善用户体验。
{"title":"Identification of Design Requirements for a Software Application for Use by Clinicians That Collects Acute Stroke Treatment Data During Clinical Workflow: Pilot Study.","authors":"Adam Forward, Gizem Koca, Aymane Sahli, Noreen Kamal","doi":"10.2196/64800","DOIUrl":"10.2196/64800","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Background: &lt;/strong&gt;Clinical registries are critical for monitoring processes of care in diseases and driving quality improvements. However, many smaller hospitals lack the required resources to collect the necessary data to contribute to registries.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;This study aims to design and evaluate a data collection tool for acute stroke treatment that streamlines the collection of process data and provides tools to aid clinician users while not interfering with clinical workflow. The evaluation will identify key design requirements that facilitate prospective data collection and add value for clinicians.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;We developed a prototype tool for testing using Figma Pro for use on an iPad. Clinicians were recruited through convenience sampling to test the prototype's use in a small-scale simulated clinical field experiment, during which participant were asked to think aloud and then complete a series of tasks to mimic a mock stroke treatment while inputting the required data into the prototype. Follow-up semistructured interviews were conducted to gain feedback on how the prototype integrated into the workflow and on the aspects of the prototype they felt helped and hindered their use of it. Qualitative data analysis combined review of the experiment recordings to identify the most frequent errors made during the scenario and deductive thematic analysis from the follow-up interviews to determine user needs for the following prototype iteration. The insights from the feedback identified design requirements that were implemented in the iterated design and documented to provide a reference for future product designers.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;Three participants were recruited from 2 hospitals between April 18 and June 6, 2024, for the simulated field experiment. The scenario took 10-12 minutes, with 1.2-3.7 minutes spent using the prototype, depending on whether optional features such as the NIHSS (National Institute of Health Stroke Scale) calculator were used. The simple and condensed layout and features such as NIHSS calculators, benchmark metric timers, and the final pop-up summary received the most positive feedback from each participant. Issues identified included small target sizes causing higher error rates, lack of color in important features reducing their visibility, and grouping of mandatory and optional information field layouts leading to a disjointed flow. The key design requirements include prioritizing simple dynamic layouts, sufficient target sizes to prevent errors, useful features with clear visual cues, and prompt data feedback to facilitate seamless integration.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Conclusions: &lt;/strong&gt;A prospective data collection tool for clinicians to use during stroke treatment can add value for clinicians and, with further testing, can be integrated into workflow. The design requirements identified through this study can provide a basis for streamlining the col","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":"9 ","pages":"e64800"},"PeriodicalIF":2.0,"publicationDate":"2025-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12759296/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145793920","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
JMIR Formative Research
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1