Pub Date : 2025-08-17DOI: 10.1177/01632787251368457
Deepti Bhasin, Karl Kilian Konrad Wiener
Objective: Notification of misconduct is a requirement by the Australian Health Practitioner Regulatory Agency. The study focuses on examining the factors that influence the intention to report misconduct by applying the Theory of planned behavior model. Method: The quantitative online survey study using vignettes and questionnaires examined one hundred and seventy-two regulated health professionals on factors that may impact the willingness to report on notifiable conduct. Results: The findings indicate that clinicians were more inclined to report on sexual misconduct and alcohol misuse conduct, however, they did not report on clinicians' competencies. Perceived behavior control, descriptive norms, and subjective norms predicted intention to report notifiable conduct, while attitude was not a predictor. Clinicians with a higher reporting intention were more likely to engage in actual reporting behavior. The behavioral pattern of reporting notifiable conduct did not differ among the three health professional groups. Conclusion: The findings identify important factors that assist clinicians in their decision-making when reporting observed misconduct. Awareness of these factors reduces health care related misconduct. That is, organizations are encouraged to develop specific programs that facilitate clinicians' decision making by educating and refreshing their knowledge of the factors impacting their intention to report misconducts.
{"title":"Factors Influencing the Reporting of Notifiable Conduct in Health Professionals in Australia.","authors":"Deepti Bhasin, Karl Kilian Konrad Wiener","doi":"10.1177/01632787251368457","DOIUrl":"https://doi.org/10.1177/01632787251368457","url":null,"abstract":"<p><p><b>Objective:</b> Notification of misconduct is a requirement by the Australian Health Practitioner Regulatory Agency. The study focuses on examining the factors that influence the intention to report misconduct by applying the Theory of planned behavior model. <b>Method:</b> The quantitative online survey study using vignettes and questionnaires examined one hundred and seventy-two regulated health professionals on factors that may impact the willingness to report on notifiable conduct. <b>Results:</b> The findings indicate that clinicians were more inclined to report on sexual misconduct and alcohol misuse conduct, however, they did not report on clinicians' competencies. Perceived behavior control, descriptive norms, and subjective norms predicted intention to report notifiable conduct, while attitude was not a predictor. Clinicians with a higher reporting intention were more likely to engage in actual reporting behavior. The behavioral pattern of reporting notifiable conduct did not differ among the three health professional groups. <b>Conclusion:</b> The findings identify important factors that assist clinicians in their decision-making when reporting observed misconduct. Awareness of these factors reduces health care related misconduct. That is, organizations are encouraged to develop specific programs that facilitate clinicians' decision making by educating and refreshing their knowledge of the factors impacting their intention to report misconducts.</p>","PeriodicalId":12315,"journal":{"name":"Evaluation & the Health Professions","volume":" ","pages":"1632787251368457"},"PeriodicalIF":1.6,"publicationDate":"2025-08-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144872200","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-08-14DOI: 10.1177/01632787251368438
Maud J J Ter Bogt, Kirsten E Bevelander, Iris E Scholte, Tessa Huttenhuis, Allia C de Rooij, Gerard R M Molleman, Maria E T C van den Muijsenbergh, Gerdine A J Fransen
Learning communities (LCs) stimulate the learning and cross-sectoral collaboration that are essential in multidisciplinary challenges, such as healthy weight approaches (HWAs). Previous research on two multidisciplinary LCs about HWAs devised a causal loop diagram (CLD) identifying dynamics (i.e., mechanisms) that describe the functioning of LCs during their starting phase (first 6 months after LC start). As LCs are likely to develop over project phases, this study aimed to ascertain whether and how the LC system dynamics were perceived to develop over time. Qualitative interviews conducted with the LC members at a second juncture were processed to be able to update the initial CLD. The updated diagram indeed illustrated how the multidisciplinary LCs were perceived to develop. The CLD became more extensive and consequently further explained three aspects: the complexity and interrelatedness of group dynamics, gaining insights through exchange, and conditions to execute actions. To ensure adequate group dynamics, learning, and action execution, LC stakeholders can regularly use the CLD as a blueprint to identify solutions for LC bottlenecks, such as members limitedly arranging the LC jointly. Future research is needed to investigate whether such developments are caused by different LC project phases and/or by the number of LC meetings and reflections.
{"title":"Mapping Longitudinal Dynamics of Learning Communities Dealing With Dutch Healthy Weight Approaches: An Updated Causal Loop Diagram.","authors":"Maud J J Ter Bogt, Kirsten E Bevelander, Iris E Scholte, Tessa Huttenhuis, Allia C de Rooij, Gerard R M Molleman, Maria E T C van den Muijsenbergh, Gerdine A J Fransen","doi":"10.1177/01632787251368438","DOIUrl":"https://doi.org/10.1177/01632787251368438","url":null,"abstract":"<p><p>Learning communities (LCs) stimulate the learning and cross-sectoral collaboration that are essential in multidisciplinary challenges, such as healthy weight approaches (HWAs). Previous research on two multidisciplinary LCs about HWAs devised a causal loop diagram (CLD) identifying dynamics (i.e., mechanisms) that describe the functioning of LCs during their starting phase (first 6 months after LC start). As LCs are likely to develop over project phases, this study aimed to ascertain whether and how the LC system dynamics were perceived to develop over time. Qualitative interviews conducted with the LC members at a second juncture were processed to be able to update the initial CLD. The updated diagram indeed illustrated how the multidisciplinary LCs were perceived to develop. The CLD became more extensive and consequently further explained three aspects: the complexity and interrelatedness of group dynamics, gaining insights through exchange, and conditions to execute actions. To ensure adequate group dynamics, learning, and action execution, LC stakeholders can regularly use the CLD as a blueprint to identify solutions for LC bottlenecks, such as members limitedly arranging the LC jointly. Future research is needed to investigate whether such developments are caused by different LC project phases and/or by the number of LC meetings and reflections.</p>","PeriodicalId":12315,"journal":{"name":"Evaluation & the Health Professions","volume":" ","pages":"1632787251368438"},"PeriodicalIF":1.6,"publicationDate":"2025-08-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144845073","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-08-11DOI: 10.1177/01632787251362506
Steve Sussman, Eric Shanazari, Leah Meza, Myriam Forster, Artur Galimov
Persons in substance use disorders (SUD) treatment report a high prevalence of tobacco use. Often neglected in SUD treatment, some recent state-level policies mandated screening and offering tobacco use treatment or referral. We interviewed 85 providers from 68 SUD treatment facilities across 20 US states, assessed providers' estimates of tobacco use prevalence in their facilities, their perceptions on types of nicotine-containing products used by their service users, tobacco policies that were implemented at their facilities, and whether treatment (counseling or pharmacologic treatment) was offered. Prevalence estimates replicate prior findings; however, the variability of the types of nicotine-containing products used was greater than what has been reported in prior studies (cigarettes, e-cigarettes, and modern oral nicotine products). According to providers, 44 (64.7%) of the facilities offered tobacco cessation treatment, and 49 (72.1%) offered FDA-approved pharmacologic adjuncts. Treatment centers in our sample in states mandating assessment/treatment of tobacco use were relatively less likely to offer tobacco cessation treatment (52.8% vs. 78.1%), whereas facilities that accepted insurance as a payment option were more likely to provide these services (77.6% vs. 31.6%). These data suggest that policy mandates may be less effective than financial incentives at promoting tobacco use cessation in SUD facilities.
{"title":"How is Tobacco Use Addressed at US Substance Use Disorder Treatment Facilities? The Importance of Financial Support for Tobacco Cessation Treatment.","authors":"Steve Sussman, Eric Shanazari, Leah Meza, Myriam Forster, Artur Galimov","doi":"10.1177/01632787251362506","DOIUrl":"https://doi.org/10.1177/01632787251362506","url":null,"abstract":"<p><p>Persons in substance use disorders (SUD) treatment report a high prevalence of tobacco use. Often neglected in SUD treatment, some recent state-level policies mandated screening and offering tobacco use treatment or referral. We interviewed 85 providers from 68 SUD treatment facilities across 20 US states, assessed providers' estimates of tobacco use prevalence in their facilities, their perceptions on types of nicotine-containing products used by their service users, tobacco policies that were implemented at their facilities, and whether treatment (counseling or pharmacologic treatment) was offered. Prevalence estimates replicate prior findings; however, the variability of the types of nicotine-containing products used was greater than what has been reported in prior studies (cigarettes, e-cigarettes, and modern oral nicotine products). According to providers, 44 (64.7%) of the facilities offered tobacco cessation treatment, and 49 (72.1%) offered FDA-approved pharmacologic adjuncts. Treatment centers in our sample in states mandating assessment/treatment of tobacco use were relatively less likely to offer tobacco cessation treatment (52.8% vs. 78.1%), whereas facilities that accepted insurance as a payment option were more likely to provide these services (77.6% vs. 31.6%). These data suggest that policy mandates may be less effective than financial incentives at promoting tobacco use cessation in SUD facilities.</p>","PeriodicalId":12315,"journal":{"name":"Evaluation & the Health Professions","volume":" ","pages":"1632787251362506"},"PeriodicalIF":1.6,"publicationDate":"2025-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144816084","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-08-07DOI: 10.1177/01632787251367407
Jon Agley
Increasing numbers of research studies are completed entirely online, often using "crowdsourcing" platforms to recruit participants, such as Amazon's Mechanical Turk (MTurk) or Prolific. Over time, crowdsourced data collection has faced multiple threats to validity including bots, inattention, and misrepresentation. Fortunately, research and development of quality control techniques has resulted in approaches to conducting credible research using crowdsourcing platforms. However, advances in large language models (LLMs), particularly computer-using agents (CUAs), such as OpenAI's "Operator," may pose novel threats to the validity of crowdsourced research data in the future. This brief report explores how and why that might be the case and encourages further discussion of this issue.
越来越多的研究完全在线完成,通常使用“众包”平台招募参与者,例如亚马逊的Mechanical Turk (MTurk)或多产(高产)。随着时间的推移,众包数据收集面临着多种有效性威胁,包括机器人、注意力不集中和虚假陈述。幸运的是,质量控制技术的研究和发展已经产生了使用众包平台进行可信研究的方法。然而,大型语言模型(llm)的进步,特别是计算机使用代理(CUAs),如OpenAI的“操作员”,可能会对未来众包研究数据的有效性构成新的威胁。这份简短的报告探讨了如何以及为什么会出现这种情况,并鼓励进一步讨论这个问题。
{"title":"Planning for New Threats to Online Research Data Validity: The Issue of Computer-Using Agents.","authors":"Jon Agley","doi":"10.1177/01632787251367407","DOIUrl":"https://doi.org/10.1177/01632787251367407","url":null,"abstract":"<p><p>Increasing numbers of research studies are completed entirely online, often using \"crowdsourcing\" platforms to recruit participants, such as Amazon's Mechanical Turk (MTurk) or Prolific. Over time, crowdsourced data collection has faced multiple threats to validity including bots, inattention, and misrepresentation. Fortunately, research and development of quality control techniques has resulted in approaches to conducting credible research using crowdsourcing platforms. However, advances in large language models (LLMs), particularly computer-using agents (CUAs), such as OpenAI's \"Operator,\" may pose novel threats to the validity of crowdsourced research data in the future. This brief report explores how and why that might be the case and encourages further discussion of this issue.</p>","PeriodicalId":12315,"journal":{"name":"Evaluation & the Health Professions","volume":" ","pages":"1632787251367407"},"PeriodicalIF":1.6,"publicationDate":"2025-08-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144793893","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-07-28DOI: 10.1177/01632787251361481
Byeolbee Um, Junhee Park
This study aimed to investigate the construct of burnout in counseling trainees using item response theory. The psychometric properties of the Maslach Burnout Inventory-General Survey for Students (MBI-GS(S)) were examined with a sample of counseling trainees (N = 246). Based on the graded response model, the results supported only the three-factor solution of counseling trainees' burnout and provided little evidence for general burnout as a potential construct. Specific item-level information from multidimensional item response theory analysis is also presented, including item discrimination parameters, item difficulty location parameters, test information, and reliability. Implications for assessing and addressing counseling trainees' burnout in terms of educating future mental health professionals are discussed.
{"title":"Unidimensional or Multidimensional? Understanding Burnout of Counseling Trainees Using Item Response Theory.","authors":"Byeolbee Um, Junhee Park","doi":"10.1177/01632787251361481","DOIUrl":"https://doi.org/10.1177/01632787251361481","url":null,"abstract":"<p><p>This study aimed to investigate the construct of burnout in counseling trainees using item response theory. The psychometric properties of the Maslach Burnout Inventory-General Survey for Students (MBI-GS(S)) were examined with a sample of counseling trainees (<i>N</i> = 246). Based on the graded response model, the results supported only the three-factor solution of counseling trainees' burnout and provided little evidence for general burnout as a potential construct. Specific item-level information from multidimensional item response theory analysis is also presented, including item discrimination parameters, item difficulty location parameters, test information, and reliability. Implications for assessing and addressing counseling trainees' burnout in terms of educating future mental health professionals are discussed.</p>","PeriodicalId":12315,"journal":{"name":"Evaluation & the Health Professions","volume":" ","pages":"1632787251361481"},"PeriodicalIF":1.6,"publicationDate":"2025-07-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144729070","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-07-18DOI: 10.1177/01632787251361540
Emine Nur Celik, Dilek Avci
This methodological study aimed to assess the validity and reliability of the Received Social Support Scale for Persons with Mental Illness (rSSS-MI), developed by Chronister et al. (2022), by adapting it into Turkish. The study consisted of 274 people who applied to the psychiatry outpatient clinic of a state hospital between April 2023 and April 2024. Data were collected using a Personal Information Form, the Received Social Support Scale for Persons with Mental Illness, and the Multidimensional Scale of Perceived Social Support. The Turkish version of the rSSS-MI demonstrated a 3-factor structure consisting of 21-items, and the items' factor loadings ranged from 0.82 to 0.96. The goodness of fit indices were χ2/df = 3.335, GFI = 0.823, NFI = 0.933, TLI = 0.944, IFI = 0.952, CFI = 0.952, RMSEA = 0.092, and SRMR = 0.024. Additionally, Cronbach's alpha coefficient of the rSSS-MI was calculated as 0.98, and item-total score correlation coefficients were between 0.80 and 0.92. These findings indicate that the Turkish version of the rSSS-MI is a valid and reliable measurement tool.
{"title":"Received Social Support Scale for Persons With Mental Illness: Psychometric Properties, Validity and Reliability of the Turkish Version.","authors":"Emine Nur Celik, Dilek Avci","doi":"10.1177/01632787251361540","DOIUrl":"https://doi.org/10.1177/01632787251361540","url":null,"abstract":"<p><p>This methodological study aimed to assess the validity and reliability of the Received Social Support Scale for Persons with Mental Illness (rSSS-MI), developed by Chronister et al. (2022), by adapting it into Turkish. The study consisted of 274 people who applied to the psychiatry outpatient clinic of a state hospital between April 2023 and April 2024. Data were collected using a Personal Information Form, the Received Social Support Scale for Persons with Mental Illness, and the Multidimensional Scale of Perceived Social Support. The Turkish version of the rSSS-MI demonstrated a 3-factor structure consisting of 21-items, and the items' factor loadings ranged from 0.82 to 0.96. The goodness of fit indices were χ2/df = 3.335, GFI = 0.823, NFI = 0.933, TLI = 0.944, IFI = 0.952, CFI = 0.952, RMSEA = 0.092, and SRMR = 0.024. Additionally, Cronbach's alpha coefficient of the rSSS-MI was calculated as 0.98, and item-total score correlation coefficients were between 0.80 and 0.92. These findings indicate that the Turkish version of the rSSS-MI is a valid and reliable measurement tool.</p>","PeriodicalId":12315,"journal":{"name":"Evaluation & the Health Professions","volume":" ","pages":"1632787251361540"},"PeriodicalIF":2.2,"publicationDate":"2025-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144658842","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-07-18DOI: 10.1177/01632787251362213
Eda Altinoz, Dilek Avci
Although there is growing interest in assessing the quality of life (QoL) in mental health services, there is currently no specific measurement tool for mental disorders in Turkey. This study aimed to evaluate the psychometric features of the Turkish version of the Mental Health Quality of Life Questionnaire (MHQoL). This methodological study was conducted with 310 patients diagnosed with mental disorders between October 2022 and June 2023. The MHQoL includes a descriptive system (MHQoL-7D) of seven items representing distinct dimensions- self-image, independence, mood, relationships, daily activities, physical health, and future- alongside a visual analog scale measuring general psychological well-being (MHQoL-VAS). Turkish MHQoL yielded a one-factor structure with 7 items and factor loadings ranging from 0.426 to 0.727. Goodness-of-fit indexes of the MHQoL were found to be χ2/df = 1.212, SRMR = 0.026, RMSEA = 0.026, GFI = 0.985, AGFI = 0.969, CFI = 0.995, IFI = 0.995, and TLI = 0.991. Cronbach's alpha coefficient was measured at 0.792, with item-total correlation coefficients from 0.378 to 0.609. Turkish MHQoL is a reliable and valid tool for evaluating QoL in individuals with mental health challenges.
{"title":"Psychometric Evaluation of the Turkish Version of the Mental Health Quality of Life Questionnaire.","authors":"Eda Altinoz, Dilek Avci","doi":"10.1177/01632787251362213","DOIUrl":"https://doi.org/10.1177/01632787251362213","url":null,"abstract":"<p><p>Although there is growing interest in assessing the quality of life (QoL) in mental health services, there is currently no specific measurement tool for mental disorders in Turkey. This study aimed to evaluate the psychometric features of the Turkish version of the Mental Health Quality of Life Questionnaire (MHQoL). This methodological study was conducted with 310 patients diagnosed with mental disorders between October 2022 and June 2023. The MHQoL includes a descriptive system (MHQoL-7D) of seven items representing distinct dimensions- self-image, independence, mood, relationships, daily activities, physical health, and future- alongside a visual analog scale measuring general psychological well-being (MHQoL-VAS). Turkish MHQoL yielded a one-factor structure with 7 items and factor loadings ranging from 0.426 to 0.727. Goodness-of-fit indexes of the MHQoL were found to be χ2/df = 1.212, SRMR = 0.026, RMSEA = 0.026, GFI = 0.985, AGFI = 0.969, CFI = 0.995, IFI = 0.995, and TLI = 0.991. Cronbach's alpha coefficient was measured at 0.792, with item-total correlation coefficients from 0.378 to 0.609. Turkish MHQoL is a reliable and valid tool for evaluating QoL in individuals with mental health challenges.</p>","PeriodicalId":12315,"journal":{"name":"Evaluation & the Health Professions","volume":" ","pages":"1632787251362213"},"PeriodicalIF":2.2,"publicationDate":"2025-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144658841","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-07-15DOI: 10.1177/01632787251361514
Ariel Linden
Single-group interrupted time-series analysis (ITSA) is a popular non-experimental study design in healthcare research. However, little guidance is available to inform the power requirements of ITSA studies under most common usages. We performed simulations to estimate the number of time periods (ranging from 10 to 100) required for percentage increases in level and trend (from baseline), to achieve statistical significance (p < 0.05, p < 0.01) with >80% and >90% power, when autocorrelation ranges from -0.90 to 0.90, and the intervention is introduced at 33%, 50% and 67% of the time series. Larger effect sizes were required for shorter studies, as well as with increasing autocorrelation, and when the intervention was introduced earlier or later than halfway in the time series. The required effect sizes were generally lower for estimating a change in the level of the time series as compared with the change in the trend, but the opposite was true when the number of time periods was larger. Simulations of studies with 10 time periods consistently produced unreliable estimates. The tables created from these analyses as well as a new community-contributed Stata package called POWER_ITSA will guide healthcare researchers in determining the most efficient way to achieve anticipated treatment effects in single-group ITSA studies.
单组中断时间序列分析(ITSA)是医疗保健研究中常用的一种非实验研究设计。但是,在大多数常见的用法下,几乎没有指导信息技术安全管理局研究的功率要求。我们进行了模拟,以估计水平和趋势(从基线)百分比增加所需的时间段数(范围从10到100),当自相关范围为-0.90至0.90,并且在33%,50%和67%的时间序列中引入干预时,>的功率为80%,>的功率为90%,达到统计显著性(p < 0.05, p < 0.01)。对于较短的研究,以及随着自相关性的增加,以及当干预措施在时间序列的半程之前或之后引入时,需要更大的效应量。与趋势变化相比,估计时间序列水平变化所需的效应量通常较低,但当时间段数量较大时,情况正好相反。对10个时间段的模拟研究始终得出不可靠的估计。从这些分析中创建的表格以及一个新的社区贡献的Stata包POWER_ITSA将指导医疗保健研究人员确定在单组ITSA研究中实现预期治疗效果的最有效方法。
{"title":"A Comprehensive Simulation Study to Evaluate the Effect Size and Study Length Relationship in Single-Group Interrupted Time Series Analysis.","authors":"Ariel Linden","doi":"10.1177/01632787251361514","DOIUrl":"https://doi.org/10.1177/01632787251361514","url":null,"abstract":"<p><p>Single-group interrupted time-series analysis (ITSA) is a popular non-experimental study design in healthcare research. However, little guidance is available to inform the power requirements of ITSA studies under most common usages. We performed simulations to estimate the number of time periods (ranging from 10 to 100) required for percentage increases in level and trend (from baseline), to achieve statistical significance (<i>p</i> < 0.05, <i>p</i> < 0.01) with >80% and >90% power, when autocorrelation ranges from -0.90 to 0.90, and the intervention is introduced at 33%, 50% and 67% of the time series. Larger effect sizes were required for shorter studies, as well as with increasing autocorrelation, and when the intervention was introduced earlier or later than halfway in the time series. The required effect sizes were generally lower for estimating a change in the level of the time series as compared with the change in the trend, but the opposite was true when the number of time periods was larger. Simulations of studies with 10 time periods consistently produced unreliable estimates. The tables created from these analyses as well as a new community-contributed Stata package called <i>POWER_ITSA</i> will guide healthcare researchers in determining the most efficient way to achieve anticipated treatment effects in single-group ITSA studies.</p>","PeriodicalId":12315,"journal":{"name":"Evaluation & the Health Professions","volume":" ","pages":"1632787251361514"},"PeriodicalIF":2.2,"publicationDate":"2025-07-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144636645","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-07-10DOI: 10.1177/01632787251357596
Mimi M Kim, Roland J Thorpe, Keith E Whitfield
Qualitative research methodologies offer critical contextual insights into community-engaged program evaluations, addressing limitations in solely quantitative approaches. While randomized controlled trials (RCTs) are inherently rigorous, their design does not consider nuanced contextual factors and lived experiences that are valuable to understanding a program's impact on community and health outcomes within diverse populations. In this theoretical discussion, we posit that qualitative methods, particularly when grounded in a Community-Based Participatory Research (CBPR) framework, foster trust and collaboration, yielding richer, more community-specific data to enhance program evaluation. We also provide evidence-based, practical consideration for conducting qualitative research for community-engaged program evaluations. We explore various qualitative approaches and their application in program evaluations. We highlight the iterative nature of qualitative data analysis and emphasize the importance of qualitative methodological rigor - including coding, triangulation, and member checking - to ensure the program evaluation's community value, credibility, and deeper contextual evaluation. Ultimately, we underscore the value of integrating qualitative methods throughout a program's lifecycle for comprehensive assessments, improved program effectiveness, and enhanced equity.
{"title":"Commentary - A Synthesis of Qualitative Research in Program Evaluation: Considerations for a Community-Engaged Approach.","authors":"Mimi M Kim, Roland J Thorpe, Keith E Whitfield","doi":"10.1177/01632787251357596","DOIUrl":"https://doi.org/10.1177/01632787251357596","url":null,"abstract":"<p><p>Qualitative research methodologies offer critical contextual insights into community-engaged program evaluations, addressing limitations in solely quantitative approaches. While randomized controlled trials (RCTs) are inherently rigorous, their design does not consider nuanced contextual factors and lived experiences that are valuable to understanding a program's impact on community and health outcomes within diverse populations. In this theoretical discussion, we posit that qualitative methods, particularly when grounded in a Community-Based Participatory Research (CBPR) framework, foster trust and collaboration, yielding richer, more community-specific data to enhance program evaluation. We also provide evidence-based, practical consideration for conducting qualitative research for community-engaged program evaluations. We explore various qualitative approaches and their application in program evaluations. We highlight the iterative nature of qualitative data analysis and emphasize the importance of qualitative methodological rigor - including coding, triangulation, and member checking - to ensure the program evaluation's community value, credibility, and deeper contextual evaluation. Ultimately, we underscore the value of integrating qualitative methods throughout a program's lifecycle for comprehensive assessments, improved program effectiveness, and enhanced equity.</p>","PeriodicalId":12315,"journal":{"name":"Evaluation & the Health Professions","volume":" ","pages":"1632787251357596"},"PeriodicalIF":2.2,"publicationDate":"2025-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144599874","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-06-26DOI: 10.1177/01632787251352837
Kristina Arnahoutova, Sabina De Geest, Juliane Mielke, Annette Boaz, Helene Schoemans, Sabine Valenta
Stakeholder involvement (SI) is essential for effective and sustainable intervention implementation, yet practical guidance is lacking. This study mapped SI use in implementation science studies, identified gaps, and proposed a practical framework for improved SI planning. Using an evidence gap map approach, this study built on Mielke et al.'s (2022) methodology, which identified implementation studies from 2015-2020. The search was updated to include studies from 2021-2023 from PubMed, using the same search strategy and inclusion criteria. Data extraction followed the Guidance for Reporting on Involvement of Patients and the Public reporting checklist. From 10,184 studies, a random sample of 2,005 was screened, adding 162 implementation science studies to Mielke et al.'s 110, totaling 272 studies for SI analysis. SI was reported in 89% of studies, but often lacked depth and strategic planning. Stakeholders were mainly engaged during the preparatory phase. Most studies involved micro- and meso-level stakeholders, rarely including macro-level stakeholders. Few described stakeholder selection or preparation. SI was mostly consultative, via interviews, surveys, and focus groups, with limited active collaboration. SI processes and costs were rarely evaluated. Our findings underscore the need for structured, comprehensive SI planning and offer practical recommendations to strengthen SI efforts in implementation research.
{"title":"Exploring Stakeholder Involvement in Intervention Implementation Studies: Systematic Evidence Synthesis With an Evidence Gap Map Approach.","authors":"Kristina Arnahoutova, Sabina De Geest, Juliane Mielke, Annette Boaz, Helene Schoemans, Sabine Valenta","doi":"10.1177/01632787251352837","DOIUrl":"https://doi.org/10.1177/01632787251352837","url":null,"abstract":"<p><p>Stakeholder involvement (SI) is essential for effective and sustainable intervention implementation, yet practical guidance is lacking. This study mapped SI use in implementation science studies, identified gaps, and proposed a practical framework for improved SI planning. Using an evidence gap map approach, this study built on Mielke et al.'s (2022) methodology, which identified implementation studies from 2015-2020. The search was updated to include studies from 2021-2023 from PubMed, using the same search strategy and inclusion criteria. Data extraction followed the Guidance for Reporting on Involvement of Patients and the Public reporting checklist. From 10,184 studies, a random sample of 2,005 was screened, adding 162 implementation science studies to Mielke et al.'s 110, totaling 272 studies for SI analysis. SI was reported in 89% of studies, but often lacked depth and strategic planning. Stakeholders were mainly engaged during the preparatory phase. Most studies involved micro- and meso-level stakeholders, rarely including macro-level stakeholders. Few described stakeholder selection or preparation. SI was mostly consultative, via interviews, surveys, and focus groups, with limited active collaboration. SI processes and costs were rarely evaluated. Our findings underscore the need for structured, comprehensive SI planning and offer practical recommendations to strengthen SI efforts in implementation research.</p>","PeriodicalId":12315,"journal":{"name":"Evaluation & the Health Professions","volume":" ","pages":"1632787251352837"},"PeriodicalIF":2.2,"publicationDate":"2025-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144505308","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}