Pub Date : 2025-08-23DOI: 10.1016/j.evalprogplan.2025.102692
Wanessa Debôrtoli de Miranda , Anelise Andrade de Souza , Luísa da Matta Machado Fernades , Daniela Corrêa , Wanderson Costa Bomfim , Helvécio Miranda Magalhães Júnior , Alice de Rezende Brandão , Rômulo Paes-Sousa
This is an Evaluability Assessment (EA) of the Specialized Care for Vulnerable Populations (SEAS) and Specialized Reference Centers for People Experiencing Homelessness (Centro POP) in Belo Horizonte, Brazil. These are social protection services for people in vulnerable situations who use the streets for housing and/or survival. The Thurston and Ramaliu model was employed with the construction of theoretical and logical models and SWOT matrix, using the analysis of legal frameworks, technical documents, on-site visits, and discussion groups with local managers. Questions were developed to compose questionnaires validated using the Delphi technique. This validation enhanced the evaluative questions, increasing the analytical power of the tools. Significant lessons were identified regarding the object of investigation and the use of the EA. Regarding the object, the findings highlight how challenging it is to offer services tailored to this population, necessitating intensive intra- and intersectoral efforts. Conversely, significant potentials for the operation of these services were elucidated. Regarding the use of the EA, three key lessons stand out: i) EA supports the implementation of participatory evaluation, ii) employing various tools and data collection methods paves the way for applying multiple evaluation approaches, and iii) there are diverse stakeholders interested in the results of an EA. The feasibility of conducting a more extensive systematic evaluation was confirmed, making this EA a pioneering effort for services aimed at the people experiencing homelessness within the field of social assistance.
{"title":"Evaluability assessment of social protection services for people experiencing homelessness in Brazil","authors":"Wanessa Debôrtoli de Miranda , Anelise Andrade de Souza , Luísa da Matta Machado Fernades , Daniela Corrêa , Wanderson Costa Bomfim , Helvécio Miranda Magalhães Júnior , Alice de Rezende Brandão , Rômulo Paes-Sousa","doi":"10.1016/j.evalprogplan.2025.102692","DOIUrl":"10.1016/j.evalprogplan.2025.102692","url":null,"abstract":"<div><div>This is an Evaluability Assessment (EA) of the Specialized Care for Vulnerable Populations (SEAS) and Specialized Reference Centers for People Experiencing Homelessness (Centro POP) in Belo Horizonte, Brazil. These are social protection services for people in vulnerable situations who use the streets for housing and/or survival. The Thurston and Ramaliu model was employed with the construction of theoretical and logical models and SWOT matrix, using the analysis of legal frameworks, technical documents, on-site visits, and discussion groups with local managers. Questions were developed to compose questionnaires validated using the Delphi technique. This validation enhanced the evaluative questions, increasing the analytical power of the tools. Significant lessons were identified regarding the object of investigation and the use of the EA. Regarding the object, the findings highlight how challenging it is to offer services tailored to this population, necessitating intensive intra- and intersectoral efforts. Conversely, significant potentials for the operation of these services were elucidated. Regarding the use of the EA, three key lessons stand out: i) EA supports the implementation of participatory evaluation, ii) employing various tools and data collection methods paves the way for applying multiple evaluation approaches, and iii) there are diverse stakeholders interested in the results of an EA. The feasibility of conducting a more extensive systematic evaluation was confirmed, making this EA a pioneering effort for services aimed at the people experiencing homelessness within the field of social assistance.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"113 ","pages":"Article 102692"},"PeriodicalIF":2.0,"publicationDate":"2025-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144916868","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-08-22DOI: 10.1016/j.evalprogplan.2025.102691
A. Nifo , S. Ruberto , G. Vecchione
In the dynamic center-periphery, the universities located in the peripheral areas, characterized by income, employment and infrastructural gaps, as well as increasing rates of skilled migration, have experienced increasing difficulties in recent years in terms of enrolment, recruitment and the ability to intercept sources of funding. This dynamic is particularly present in the Italian system, characterized by structural weaknesses that persist for at least a decade. In Italy, this marginalization phenomenon is impoverishing peripherical Universities jeopardising their role as civic actors for the most peripheral territories. This paper addresses the case of Italian universities in which we are witnessing the gradual decline of universities in the more peripheral areas and a constant growth of universities located in the most dynamic center of the country, as emerged within a working group operating between 2020 and 2021 on behalf of the Ministry of University. In this scenario by 2040, in line with demographic estimates, some universities in peripheral areas may need to close, resulting in a social and economic impoverishment of their home territories. This paper aims to evaluate the introduction of tax exemptions measures introduced in 2017 and extended by the government in 2021 in the aim of stemming this decline. The results of our analysis indicate that tax exemptions and more generous bursaries positively affect the retention rate of central universities and the attraction rate of the peripheral ones. Hence, it represents a significant policy factor capable of increasing the presence of universities in peripheral regions and, consequently, their capacity to generate societal impact within them.
{"title":"The social role of peripheral universities and tax exemptions, the Italian case","authors":"A. Nifo , S. Ruberto , G. Vecchione","doi":"10.1016/j.evalprogplan.2025.102691","DOIUrl":"10.1016/j.evalprogplan.2025.102691","url":null,"abstract":"<div><div>In the dynamic center-periphery, the universities located in the peripheral areas, characterized by income, employment and infrastructural gaps, as well as increasing rates of skilled migration, have experienced increasing difficulties in recent years in terms of enrolment, recruitment and the ability to intercept sources of funding. This dynamic is particularly present in the Italian system, characterized by structural weaknesses that persist for at least a decade. In Italy, this marginalization phenomenon is impoverishing peripherical Universities jeopardising their role as civic actors for the most peripheral territories. This paper addresses the case of Italian universities in which we are witnessing the gradual decline of universities in the more peripheral areas and a constant growth of universities located in the most dynamic center of the country, as emerged within a working group operating between 2020 and 2021 on behalf of the Ministry of University. In this scenario by 2040, in line with demographic estimates, some universities in peripheral areas may need to close, resulting in a social and economic impoverishment of their home territories. This paper aims to evaluate the introduction of tax exemptions measures introduced in 2017 and extended by the government in 2021 in the aim of stemming this decline. The results of our analysis indicate that tax exemptions and more generous bursaries positively affect the retention rate of central universities and the attraction rate of the peripheral ones. Hence, it represents a significant policy factor capable of increasing the presence of universities in peripheral regions and, consequently, their capacity to generate societal impact within them.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"113 ","pages":"Article 102691"},"PeriodicalIF":2.0,"publicationDate":"2025-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144911764","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-08-16DOI: 10.1016/j.evalprogplan.2025.102690
Rachel Bush , Petra K. Staiger , Ilona M. McNeill , Rhonda Brown , Dan Lubman , Ruth McNair
Background
Studies indicate that same-sex attracted women (SSAW) who engage in problem drinking often do not seek support. This is at least partially due to a lack of SSAW-tailored alcohol programs. This paper describes the co-development of a short message service (SMS) alcohol program culturally tailored for SSAW.
Method
We applied the systematic stages of the Intervention Mapping framework to guide development, and used a combination of literature reviews and focus group discussions.
Results
The initial needs assessment identified three change objectives: 1) reduce alcohol consumption, 2) improved wellbeing, and 3) increased help-seeking, and three change determinants: a) motivation, b) self-regulatory ability, and c) social support. Based on a selection of relevant BCTs we then developed 70 initial SMS statements, 37 of which were selected through codesign for the final program. The resulting program (referred to as The Step One Program) consists of one to two daily SMS messages over a four-week period.
Conclusions
The Step One Program is the first published SMS-based alcohol intervention for SSAW. During development, SSAW highlighted increased anonymity and better accessibility as compared to face-to-face services as well as the inclusion of SSAW and LGBT specific resources and culturally inclusive messaging as key benefits.
{"title":"Development of an SMS-based alcohol intervention program for same sex attracted women using an intervention mapping framework","authors":"Rachel Bush , Petra K. Staiger , Ilona M. McNeill , Rhonda Brown , Dan Lubman , Ruth McNair","doi":"10.1016/j.evalprogplan.2025.102690","DOIUrl":"10.1016/j.evalprogplan.2025.102690","url":null,"abstract":"<div><h3>Background</h3><div>Studies indicate that same-sex attracted women (SSAW) who engage in problem drinking often do not seek support. This is at least partially due to a lack of SSAW-tailored alcohol programs. This paper describes the co-development of a short message service (SMS) alcohol program culturally tailored for SSAW.</div></div><div><h3>Method</h3><div>We applied the systematic stages of the Intervention Mapping framework to guide development, and used a combination of literature reviews and focus group discussions.</div></div><div><h3>Results</h3><div>The initial needs assessment identified three change objectives: 1) reduce alcohol consumption, 2) improved wellbeing, and 3) increased help-seeking, and three change determinants: a) motivation, b) self-regulatory ability, and c) social support. Based on a selection of relevant BCTs we then developed 70 initial SMS statements, 37 of which were selected through codesign for the final program. The resulting program (referred to as The Step One Program) consists of one to two daily SMS messages over a four-week period.</div></div><div><h3>Conclusions</h3><div>The Step One Program is the first published SMS-based alcohol intervention for SSAW. During development, SSAW highlighted increased anonymity and better accessibility as compared to face-to-face services as well as the inclusion of SSAW and LGBT specific resources and culturally inclusive messaging as key benefits.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"113 ","pages":"Article 102690"},"PeriodicalIF":2.0,"publicationDate":"2025-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144886524","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-08-13DOI: 10.1016/j.evalprogplan.2025.102676
Aitana González-Ortiz-de-Zárate, Miguel Aurelio Alonso-García, María de los Ángeles Gómez-Flechoso, Irene Aliagas
Peer mentoring programs in higher education aim to facilitate student integration and enhance both retention and academic performance. This study investigated the effectiveness of a peer mentoring program at a Spanish university over five academic years (2018–19–2022–23), encompassing the COVID-19 pandemic. Employing a quasi-experimental posttest-only control group design, we analyzed data from 4962 students (mentees: n = 2481; controls: n = 2481). Statistical analyses included independent t-tests, Cohen's d, phi coefficient, chi-square tests, and two-way ANOVA. Results demonstrated significantly lower dropout rates and higher academic performance among mentees compared to controls across all study years. These positive effects were observed inconsistently across most of the studied student characteristics, including gender, study choice order, region of origin, and field of study. Furthermore, our findings supported the mediating role of academic performance in the relationship between peer mentoring and dropout. This research provides valuable empirical evidence that supports the theoretical frameworks on peer mentoring, student development, and successful integration within the context of higher education, particularly during times of crisis.
{"title":"Peer mentoring, university dropout and academic performance before, during, and after the pandemic in Spain","authors":"Aitana González-Ortiz-de-Zárate, Miguel Aurelio Alonso-García, María de los Ángeles Gómez-Flechoso, Irene Aliagas","doi":"10.1016/j.evalprogplan.2025.102676","DOIUrl":"10.1016/j.evalprogplan.2025.102676","url":null,"abstract":"<div><div>Peer mentoring programs in higher education aim to facilitate student integration and enhance both retention and academic performance. This study investigated the effectiveness of a peer mentoring program at a Spanish university over five academic years (2018–19–2022–23), encompassing the COVID-19 pandemic. Employing a quasi-experimental posttest-only control group design, we analyzed data from 4962 students (mentees: n = 2481; controls: n = 2481). Statistical analyses included independent t-tests, Cohen's d, phi coefficient, chi-square tests, and two-way ANOVA. Results demonstrated significantly lower dropout rates and higher academic performance among mentees compared to controls across all study years. These positive effects were observed inconsistently across most of the studied student characteristics, including gender, study choice order, region of origin, and field of study. Furthermore, our findings supported the mediating role of academic performance in the relationship between peer mentoring and dropout. This research provides valuable empirical evidence that supports the theoretical frameworks on peer mentoring, student development, and successful integration within the context of higher education, particularly during times of crisis.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"113 ","pages":"Article 102676"},"PeriodicalIF":2.0,"publicationDate":"2025-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144858074","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-08-12DOI: 10.1016/j.evalprogplan.2025.102675
Lary Nel B. Abao , Ruth M. Sonaco , Miguel V. Guioguio , Alvin Paul J. Dirain , Roland Y. Fajardo , Joseph Frank Mgaya
The 3-year Integrated National Swine Production Initiatives for Recovery and Expansion (INSPIRE) program is the national repopulation and recovery program of the Philippine Department of Agriculture-National Livestock Program, offering a mix of interventions aimed at reviving the swine industry, which was significantly impacted by African Swine Fever (ASF). The study focused on the interventions provided to the three regions with the highest allocation from fiscal years (FYs) 2022 and 2023: Central Luzon, CALABARZON, and the Davao region. A combination of SARIMA modeling and financial analyses was employed to measure the impacts of the repopulation program.
Two forecasts were created: one assumes a situation without African swine fever outbreaks, using datasets from 2000Q1 to 2019Q2, whereas the other reflects the situation prior to the implementation of the INSPIRE program, which utilizes datasets from 2000Q1 to 2021Q4.
The study estimated the impact of the INSPIRE program in these three regions, revealing that Central Luzon and CALABARZON significantly benefited from the program. Without the INSPIRE program, pig production in Central Luzon could have been wiped out by the second quarter of 2022 (2022Q2), whereas CALABARZON would have faced a similar scenario by the first quarter of 2023 (2023Q1). In contrast, the INSPIRE program had a lesser impact in the Davao Region, as it was affected by ASF later.
The cost benefit analysis and net present value of the program investments yielded favorable ratios that indicate their financial viability and success. Based on the impact analysis, it is recommended that the swine repopulation program be continued beyond 2025 with increased funding.
This is the first study in the Philippines which utilizes time series SARIMA forecasting to evaluate the impact of a pig repopulation program.
{"title":"Ex-post impact analysis of the Philippine pig repopulation program","authors":"Lary Nel B. Abao , Ruth M. Sonaco , Miguel V. Guioguio , Alvin Paul J. Dirain , Roland Y. Fajardo , Joseph Frank Mgaya","doi":"10.1016/j.evalprogplan.2025.102675","DOIUrl":"10.1016/j.evalprogplan.2025.102675","url":null,"abstract":"<div><div>The 3-year Integrated National Swine Production Initiatives for Recovery and Expansion (INSPIRE) program is the national repopulation and recovery program of the Philippine Department of Agriculture-National Livestock Program, offering a mix of interventions aimed at reviving the swine industry, which was significantly impacted by African Swine Fever (ASF). The study focused on the interventions provided to the three regions with the highest allocation from fiscal years (FYs) 2022 and 2023: Central Luzon, CALABARZON, and the Davao region. A combination of SARIMA modeling and financial analyses was employed to measure the impacts of the repopulation program.</div><div>Two forecasts were created: one assumes a situation without African swine fever outbreaks, using datasets from 2000Q1 to 2019Q2, whereas the other reflects the situation prior to the implementation of the INSPIRE program, which utilizes datasets from 2000Q1 to 2021Q4.</div><div>The study estimated the impact of the INSPIRE program in these three regions, revealing that Central Luzon and CALABARZON significantly benefited from the program. Without the INSPIRE program, pig production in Central Luzon could have been wiped out by the second quarter of 2022 (2022Q2), whereas CALABARZON would have faced a similar scenario by the first quarter of 2023 (2023Q1). In contrast, the INSPIRE program had a lesser impact in the Davao Region, as it was affected by ASF later.</div><div>The cost benefit analysis and net present value of the program investments yielded favorable ratios that indicate their financial viability and success. Based on the impact analysis, it is recommended that the swine repopulation program be continued beyond 2025 with increased funding.</div><div>This is the first study in the Philippines which utilizes time series SARIMA forecasting to evaluate the impact of a pig repopulation program.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"113 ","pages":"Article 102675"},"PeriodicalIF":2.0,"publicationDate":"2025-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144852920","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-08-12DOI: 10.1016/j.evalprogplan.2025.102678
Cornelia Klein
Stakeholder involvement is an essential part of conducting social return on investment evaluations (SROI). This systematic literature review takes stock of the scholarly discussion of how stakeholders are defined, identified, and involved during SROI evaluations, including the involvement of vulnerable groups as key stakeholders of evaluated projects or organisations in the social sector. The author searched three databases Google Scholar, SCOPUS, and Web of Science for peer-reviewed articles published in English between 1996 and 2022, where the term ‘Social Return on Investment’ or the abbreviation ‘SROI’ was found in titles, abstracts or author keywords and also in the main text. All articles that used the phrase or abbreviation in the sense of the SROI framework and discussed any of its aspects were included in the review. Articles solely focusing on the results of a conducted SROI evaluation were excluded, while case studies discussing the application of the SROI framework were included. As a result, 101 peer-reviewed articles published between 2004 and 2022 were found to meet the inclusion criteria for the review. Qualitative data analysis with MAXQDA revealed strong agreement on the definition of stakeholders and underline prior findings of inconsistencies in choosing and involving stakeholders. The review further uncovers strategies and methods applicable for future evaluations, which also enable the successful involvement of vulnerable groups. These findings address critical issues of the SROI framework that contribute to furthering the scholarly debate and may even enhance the approaches to involving stakeholders in social impact evaluations in general.
利益相关者的参与是进行社会投资回报评估（SROI）的重要组成部分。本系统的文献综述盘点了在SROI评估过程中如何定义、识别和参与利益相关者的学术讨论，包括弱势群体作为被评估项目或社会部门组织的关键利益相关者的参与。作者在谷歌Scholar、SCOPUS和Web of Science三个数据库中检索了1996年至2022年间发表的同行评议的英文文章，在标题、摘要或作者关键词以及正文中都发现了“社会投资回报”或缩写“SROI”。在SROI框架的意义上使用短语或缩写并讨论其任何方面的所有文章都包括在审查中。仅关注进行的SROI评估结果的文章被排除在外，而讨论SROI框架应用的案例研究被包括在内。结果发现，2004年至2022年间发表的101篇同行评议文章符合该审查的纳入标准。使用MAXQDA进行的定性数据分析揭示了对利益相关者定义的强烈一致，并强调了在选择和参与利益相关者方面的不一致的先前发现。审查进一步揭示了适用于今后评价的战略和方法，这也使脆弱群体能够成功地参与。这些发现解决了SROI框架的关键问题，有助于推动学术辩论，甚至可能在总体上加强利益相关者参与社会影响评估的方法。
{"title":"The scientific discussion of stakeholder involvement in social return on investment analysis? A systematic literature review","authors":"Cornelia Klein","doi":"10.1016/j.evalprogplan.2025.102678","DOIUrl":"10.1016/j.evalprogplan.2025.102678","url":null,"abstract":"<div><div>Stakeholder involvement is an essential part of conducting social return on investment evaluations (SROI). This systematic literature review takes stock of the scholarly discussion of how stakeholders are defined, identified, and involved during SROI evaluations, including the involvement of vulnerable groups as key stakeholders of evaluated projects or organisations in the social sector. The author searched three databases <em>Google Scholar</em>, <em>SCOPUS</em>, and <em>Web of Science</em> for peer-reviewed articles published in English between 1996 and 2022, where the term ‘Social Return on Investment’ or the abbreviation ‘SROI’ was found in titles, abstracts or author keywords and also in the main text. All articles that used the phrase or abbreviation in the sense of the SROI framework and discussed any of its aspects were included in the review. Articles solely focusing on the results of a conducted SROI evaluation were excluded, while case studies discussing the application of the SROI framework were included. As a result, 101 peer-reviewed articles published between 2004 and 2022 were found to meet the inclusion criteria for the review. Qualitative data analysis with MAXQDA revealed strong agreement on the definition of stakeholders and underline prior findings of inconsistencies in choosing and involving stakeholders. The review further uncovers strategies and methods applicable for future evaluations, which also enable the successful involvement of vulnerable groups. These findings address critical issues of the SROI framework that contribute to furthering the scholarly debate and may even enhance the approaches to involving stakeholders in social impact evaluations in general.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"113 ","pages":"Article 102678"},"PeriodicalIF":2.0,"publicationDate":"2025-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144841486","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This study, initiated by the Turkish Higher Education Quality Council (THEQC), develops and validates a rubric-based approach for evaluating institutional quality assurance (QA) processes within the Turkish higher education ecosystem. Effective quality assurance (QA) in higher education requires evaluation tools that are methodologically robust, responsive to institutional needs, and aligned with international benchmarks. A rubric-based approach offers a structured and transparent framework for consistent assessments. The rubric was created through an iterative, evidence-based process incorporating expert feedback, international benchmarks, site visits, and prior research. It consists of 22 main criteria across four key domains—Leadership, Governance and Quality; Learning and Teaching; Research and Development; and Service to Society—subdivided into 46 sub-criteria, each assessed on a five-point maturity scale. The validity and reliability of the rubric were rigorously examined. Content validity was established through evaluations by 10 QA professionals, addressing four key dimensions: relevance, alignment with the intended domain, appropriateness for the target audience, and clarity of language. Construct and criterion validity were assessed through consistency checks and usability analyses, with data collected from 252 external evaluators across 57 higher education institutions. Additionally, 360-degree feedback was gathered from both external evaluation teams and the institutions involved in the evaluation process.
Results
demonstrate strong content validity (Lawshe coefficient = 1.00, p < .05), high interrater consistency, and strong usability across various evaluator groups. Further evaluation confirmed the rubric’s applicability and robustness across diverse institutional contexts. This study concludes that the developed rubric is a reliable and valid tool for enhancing and evaluating quality assurance practices in higher education.
{"title":"Developing and validating a rubric-based approach to quality assurance in Turkish higher education","authors":"Buket Akkoyunlu , Salih Bardakcı , Sibel Aksu Yıldırım , M. Dilek Avşaroğlu , Gonca Uludağ , Ayhan Koçer , Muzaffer Elmas","doi":"10.1016/j.evalprogplan.2025.102673","DOIUrl":"10.1016/j.evalprogplan.2025.102673","url":null,"abstract":"<div><div>This study, initiated by the Turkish Higher Education Quality Council (THEQC), develops and validates a rubric-based approach for evaluating institutional quality assurance (QA) processes within the Turkish higher education ecosystem. Effective quality assurance (QA) in higher education requires evaluation tools that are methodologically robust, responsive to institutional needs, and aligned with international benchmarks. A rubric-based approach offers a structured and transparent framework for consistent assessments. The rubric was created through an iterative, evidence-based process incorporating expert feedback, international benchmarks, site visits, and prior research. It consists of 22 main criteria across four key domains—Leadership, Governance and Quality; Learning and Teaching; Research and Development; and Service to Society—subdivided into 46 sub-criteria, each assessed on a five-point maturity scale. The validity and reliability of the rubric were rigorously examined. Content validity was established through evaluations by 10 QA professionals, addressing four key dimensions: relevance, alignment with the intended domain, appropriateness for the target audience, and clarity of language. Construct and criterion validity were assessed through consistency checks and usability analyses, with data collected from 252 external evaluators across 57 higher education institutions. Additionally, 360-degree feedback was gathered from both external evaluation teams and the institutions involved in the evaluation process.</div></div><div><h3>Results</h3><div>demonstrate strong content validity (Lawshe coefficient = 1.00, p < .05), high interrater consistency, and strong usability across various evaluator groups. Further evaluation confirmed the rubric’s applicability and robustness across diverse institutional contexts. This study concludes that the developed rubric is a reliable and valid tool for enhancing and evaluating quality assurance practices in higher education.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"113 ","pages":"Article 102673"},"PeriodicalIF":2.0,"publicationDate":"2025-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144827904","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-08-08DOI: 10.1016/j.evalprogplan.2025.102677
Nathan Kumasenu Mensah , Godwin Adzakpah , Jonathan Kissi , Christabell Opoku , Emmanuel Osei , Jessica Abakah , Hawawu Ahmed , Hawa Mustapha , Precious Mensah , Richard Okyere Boadu , Daniel Kwame Kwansah Quansah , Hannah Benedicta Taylor-Abdulai
Background
Internship programs are essential for bridging the gap between theoretical education and practical application. They equip students with the practical skills, relevant knowledge, and professional networks necessary for career success. This study explores the experiences of Health Information Management (HIM) stakeholders, including faculty, interns, and hospital supervisors in Ghana.
Methods
The study was conducted across ten hospitals in six of Ghana’s sixteen administrative regions between 18th July and 30th August 2024. A concurrent mixed methods design incorporating qualitative (In-depth interviews) and quantitative (survey) was used. Kolb’s Experiential Learning Theory provided the conceptual framework to interpret how interns learn through real-world engagement.
Results
Seventy-nine HIM students participated in the quantitative survey. Of the interns surveyed, 96.2 % (n = 76) reported that site supervisors could readily address their questions and provide guidance. A similar percentage reported gaining new knowledge, with most interns noting significant improvement in teamwork skills. Unadjusted regression analysis identified key factors influencing overall satisfaction, highlighting opportunities to demonstrate initiatives beyond basic requirements, perceptions of enhanced employment prospects, and contributing to meaningful outcomes. Qualitative data from in-depth interviews revealed challenges like non-adherence to logbook modules and inadequate supervision. Despite these issues, the presence of interns was perceived as a benefit to the facilities, as they were seen to enhance staff capacity.
Conclusion
This study highlights the critical role internships play in professional development, while also drawing attention to key challenges such as non-adherence to logbook requirements, limited financial support, and program durations. Recommendations for improving the internship experience include ensuring consistent and effective supervision, extending the program durations, adhering to logbook modules, and providing financial assistance to support interns.
{"title":"Evaluation of Health Information Management students’ internship program: A mixed methods study","authors":"Nathan Kumasenu Mensah , Godwin Adzakpah , Jonathan Kissi , Christabell Opoku , Emmanuel Osei , Jessica Abakah , Hawawu Ahmed , Hawa Mustapha , Precious Mensah , Richard Okyere Boadu , Daniel Kwame Kwansah Quansah , Hannah Benedicta Taylor-Abdulai","doi":"10.1016/j.evalprogplan.2025.102677","DOIUrl":"10.1016/j.evalprogplan.2025.102677","url":null,"abstract":"<div><h3>Background</h3><div>Internship programs are essential for bridging the gap between theoretical education and practical application. They equip students with the practical skills, relevant knowledge, and professional networks necessary for career success. This study explores the experiences of Health Information Management (HIM) stakeholders, including faculty, interns, and hospital supervisors in Ghana.</div></div><div><h3>Methods</h3><div>The study was conducted across ten hospitals in six of Ghana’s sixteen administrative regions between 18th July and 30th August 2024. A concurrent mixed methods design incorporating qualitative (In-depth interviews) and quantitative (survey) was used. Kolb’s Experiential Learning Theory provided the conceptual framework to interpret how interns learn through real-world engagement.</div></div><div><h3>Results</h3><div>Seventy-nine HIM students participated in the quantitative survey. Of the interns surveyed, 96.2 % (n = 76) reported that site supervisors could readily address their questions and provide guidance. A similar percentage reported gaining new knowledge, with most interns noting significant improvement in teamwork skills. Unadjusted regression analysis identified key factors influencing overall satisfaction, highlighting opportunities to demonstrate initiatives beyond basic requirements, perceptions of enhanced employment prospects, and contributing to meaningful outcomes. Qualitative data from in-depth interviews revealed challenges like non-adherence to logbook modules and inadequate supervision. Despite these issues, the presence of interns was perceived as a benefit to the facilities, as they were seen to enhance staff capacity.</div></div><div><h3>Conclusion</h3><div>This study highlights the critical role internships play in professional development, while also drawing attention to key challenges such as non-adherence to logbook requirements, limited financial support, and program durations. Recommendations for improving the internship experience include ensuring consistent and effective supervision, extending the program durations, adhering to logbook modules, and providing financial assistance to support interns.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"113 ","pages":"Article 102677"},"PeriodicalIF":2.0,"publicationDate":"2025-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144826413","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-08-07DOI: 10.1016/j.evalprogplan.2025.102672
Kelly A. Laurila , Katharine Sanderson , Taylor S. Lane , Diane Stearns , Derrick C. Tabor , Julie A. Baldwin
This case study examines key components of research readiness for Doctoral Universities with High Research Activity (R2) through the lens of the National Institutes of Health (NIH) funded Research Centers in Minority Institutions (RCMI) program. While there are long-standing systems in place for ranking research institutions based on grant expenditures, there is a need to establish methods to evaluate institutions’ readiness for externally funded competitive research. Drawing on literature across multiple disciplines and framed in alignment with the RCMI program goals, the authors developed a multidimensional instrument to examine research readiness centered on two primary domains (investigator and institutional readiness). The largest barriers to proposal development and conducting research/scholarship projects were managing time for competing expectations including scholarship, teaching, mentoring students, and service and recruiting skilled students who are prepared to assist the research process (respectively). Building research capacity requires research infrastructure, equitable distribution of institutional and grant resources, protected time for research, and focused areas research. Findings inform evaluation methods to assess research readiness, identify indicators and set benchmarks to measure changes in capacity and offer strategies for universities to enhance research capacity-building efforts and optimize institutions’ ability to conduct NIH mission relevant research.
{"title":"Building capacity for health research in higher education: Evaluating readiness for research and scholarship","authors":"Kelly A. Laurila , Katharine Sanderson , Taylor S. Lane , Diane Stearns , Derrick C. Tabor , Julie A. Baldwin","doi":"10.1016/j.evalprogplan.2025.102672","DOIUrl":"10.1016/j.evalprogplan.2025.102672","url":null,"abstract":"<div><div>This case study examines key components of research readiness for Doctoral Universities with High Research Activity (R2) through the lens of the National Institutes of Health (NIH) funded Research Centers in Minority Institutions (RCMI) program. While there are long-standing systems in place for ranking research institutions based on grant expenditures, there is a need to establish methods to evaluate institutions’ readiness for externally funded competitive research. Drawing on literature across multiple disciplines and framed in alignment with the RCMI program goals, the authors developed a multidimensional instrument to examine research readiness centered on two primary domains (investigator and institutional readiness). The largest barriers to proposal development and conducting research/scholarship projects were managing time for competing expectations including scholarship, teaching, mentoring students, and service and recruiting skilled students who are prepared to assist the research process (respectively). Building research capacity requires research infrastructure, equitable distribution of institutional and grant resources, protected time for research, and focused areas research. Findings inform evaluation methods to assess research readiness, identify indicators and set benchmarks to measure changes in capacity and offer strategies for universities to enhance research capacity-building efforts and optimize institutions’ ability to conduct NIH mission relevant research.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"113 ","pages":"Article 102672"},"PeriodicalIF":2.0,"publicationDate":"2025-08-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144896087","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Given high rates of unintended pregnancies and sexually transmitted infections among Black teens in Florida, the need exists for evidence-based interventions to prevent negative and promote positive sexual health outcomes. Building on positive youth development principles, Love Notes is an evidence-based sexuality education program designed to help youth develop and maintain healthy relationships while preparing for the future. Two versions of Love Notes exist – comprehensive sexuality education (CSE) and sexual risk avoidance (SRA). CSE programs are preferred over SRA programs, which explains why most Love Notes research has used the CSE version. However, some communities will not support CSE programming. Addressing the need to evaluate Love Notes-SRA, this study examined the feasibility, acceptability, and potential utility of Love Notes-SRA programming for Black middle and high school students in Tampa Bay, Florida. Existing program evaluation data collected from 452 youth aged 11–19 indicated lower self-reported intentions to engage in risky behaviors and increased self-reported ability and intentions to resist peer pressure, regulate emotions, weigh the consequences of a decision, and engage in future planning after completing the program. Although CSE programming for all youth is preferred, findings suggest Love Notes-SRA is feasible, acceptable, and potentially useful for Black youth.
{"title":"Feasibility, acceptability, and potential utility of an evidence-based pregnancy prevention program among Black/African American middle and high school students in Florida","authors":"Tiffany Chenneville , Serena Wasilewski , Morgan Haskett , Kemesha Gabbidon","doi":"10.1016/j.evalprogplan.2025.102671","DOIUrl":"10.1016/j.evalprogplan.2025.102671","url":null,"abstract":"<div><div>Given high rates of unintended pregnancies and sexually transmitted infections among Black teens in Florida, the need exists for evidence-based interventions to prevent negative and promote positive sexual health outcomes. Building on positive youth development principles, Love Notes is an evidence-based sexuality education program designed to help youth develop and maintain healthy relationships while preparing for the future. Two versions of Love Notes exist – comprehensive sexuality education (CSE) and sexual risk avoidance (SRA). CSE programs are preferred over SRA programs, which explains why most Love Notes research has used the CSE version. However, some communities will not support CSE programming. Addressing the need to evaluate Love Notes-SRA, this study examined the feasibility, acceptability, and potential utility of Love Notes-SRA programming for Black middle and high school students in Tampa Bay, Florida. Existing program evaluation data collected from 452 youth aged 11–19 indicated lower self-reported intentions to engage in risky behaviors and increased self-reported ability and intentions to resist peer pressure, regulate emotions, weigh the consequences of a decision, and engage in future planning after completing the program. Although CSE programming for all youth is preferred, findings suggest Love Notes-SRA is feasible, acceptable, and potentially useful for Black youth.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"112 ","pages":"Article 102671"},"PeriodicalIF":2.0,"publicationDate":"2025-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144781022","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号