Evaluation and Program Planning最新文献

Overview of rural credit environment in China: Measurement logic, evaluation system, and case analysis 中国农村信用环境概述：衡量逻辑、评价体系和案例分析。

IF 1.5 4区社会学 Q2 SOCIAL SCIENCES, INTERDISCIPLINARY

Evaluation and Program Planning

Pub Date : 2024-11-17 DOI: 10.1016/j.evalprogplan.2024.102519

Xihui Chen , Zhouyi Gu , Luca Esposito , Jiayan Lv

A rational evaluation of the rural credit environment's current state and identification of its critical issues are crucial for enhancing the construction of rural social credit systems. Drawing on the "Outline for the Construction of the Social Credit System (2014–2020)" and related literature, this paper clarifies the concepts and measurement principles of the rural credit environment. This study innovatively constructs an evaluation framework for the rural credit environment and conducts quantitative measurements and statistical analyses using a combined weighting method to delineate the environment's current state. Using Banqiao Town as a case study, this paper employs the cloud model to evaluate the rural credit environment's quality levels and to identify key factors influencing its quality effectively. From a macro perspective, statistical measurements and index analyses establish evaluation standards and quantify the state of the rural credit environment. In contrast, from a micro perspective, case analysis focuses on evaluating the quality levels and identifying crucial issues in specific locales. Integrating macro and micro perspectives offers a novel approach to evaluating the rural credit environment, offering theoretical approaches and practical strategies for promoting a favorable credit environment and advancing the construction of rural social credit systems in China.

理性评估农村信用环境现状，找准关键问题，是加强农村社会信用体系建设的关键。本文借鉴《社会信用体系建设纲要（2014-2020 年）》及相关文献，明确了农村信用环境的概念和测评原则。本研究创新性地构建了农村信用环境评价框架，并采用组合加权法进行定量测算和统计分析，以勾勒出农村信用环境的现状。本文以板桥镇为例，运用云模型对农村信用环境质量水平进行评价，并有效识别影响农村信用环境质量的关键因素。从宏观角度看，通过统计测算和指标分析，建立评价标准，量化农村信用环境状况。而从微观角度来看，案例分析则侧重于评价具体地方的质量水平并找出关键问题。宏观与微观的结合为农村信用环境的评价提供了一种新的方法，为促进良好的信用环境、推进中国农村社会信用体系建设提供了理论方法和实践策略。

{"title":"Overview of rural credit environment in China: Measurement logic, evaluation system, and case analysis","authors":"Xihui Chen , Zhouyi Gu , Luca Esposito , Jiayan Lv","doi":"10.1016/j.evalprogplan.2024.102519","DOIUrl":"10.1016/j.evalprogplan.2024.102519","url":null,"abstract":"<div><div>A rational evaluation of the rural credit environment's current state and identification of its critical issues are crucial for enhancing the construction of rural social credit systems. Drawing on the \"Outline for the Construction of the Social Credit System (2014–2020)\" and related literature, this paper clarifies the concepts and measurement principles of the rural credit environment. This study innovatively constructs an evaluation framework for the rural credit environment and conducts quantitative measurements and statistical analyses using a combined weighting method to delineate the environment's current state. Using Banqiao Town as a case study, this paper employs the cloud model to evaluate the rural credit environment's quality levels and to identify key factors influencing its quality effectively. From a macro perspective, statistical measurements and index analyses establish evaluation standards and quantify the state of the rural credit environment. In contrast, from a micro perspective, case analysis focuses on evaluating the quality levels and identifying crucial issues in specific locales. Integrating macro and micro perspectives offers a novel approach to evaluating the rural credit environment, offering theoretical approaches and practical strategies for promoting a favorable credit environment and advancing the construction of rural social credit systems in China.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"108 ","pages":"Article 102519"},"PeriodicalIF":1.5,"publicationDate":"2024-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142682225","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Using social research and development methods in artist-mediated workshops with Myanmar refugee women to design a culturally responsive community resettlement center 在以艺术家为媒介、与缅甸难民妇女共同举办的研讨会中采用社会研究和发展方法，设计具有文化敏感性的社区安置中心

IF 1.5 4区社会学 Q2 SOCIAL SCIENCES, INTERDISCIPLINARY

Evaluation and Program Planning

Pub Date : 2024-11-14 DOI: 10.1016/j.evalprogplan.2024.102518

Chie Noyori-Corbett , David P. Moxley

The authors describe an arts-based social research and development approach in helping Myanmar refugee women share their ideas about the elements of a culturally responsive community support system for their resettlement in Dallas, Texas. To illuminate the relevance of the arts in social research and development using group work methods in the form of workshops, the authors involved 60 Myanmar refugee women in the co-design of a community resource in partnership with social workers. The authors recommend that evaluators and program planners gain skills in artist-mediated participatory group work methods in resettlement using the process of social research and development.

作者描述了一种以艺术为基础的社会研究和发展方法，帮助缅甸难民妇女分享她们关于在得克萨斯州达拉斯市重新安置的文化响应型社区支持系统要素的想法。为了阐明艺术在社会研究和发展中的相关性，作者采用小组工作方法，以工作坊的形式，让 60 名缅甸难民妇女与社会工作者合作，共同设计社区资源。作者建议评估人员和项目规划人员在社会研究与发展过程中，掌握以艺术家为媒介的参与式小组工作方法。

引用次数: 0

Uses of the viable validity concept: A systematic scoping review 可行有效性概念的用途：系统性范围审查。

IF 1.5 4区社会学 Q2 SOCIAL SCIENCES, INTERDISCIPLINARY

Evaluation and Program Planning

Pub Date : 2024-11-13 DOI: 10.1016/j.evalprogplan.2024.102516

Césarine Sambou , Charlotte Decroix , Judith Martin-Fernandez , Linda Cambon , François Alla

Objective

The concept of viable validity was first defined in 2010 within the framework of the integrative validity model. The concept has continued to evolve in the intervening years, and the purpose of this systematic scoping review is to describe and analyze the ways in which it has been deployed and appropriated by various research traditions.

Methods

We began by including all articles which cite Chen’s original article “The bottom-up approach to integrative validity: a new perspective for program evaluation (Eval Program Plann. 2010;33(3):205–14) and/or contain the terms “viable validity” or “viable cogency,” sourced from 5 databases (PubMed, Web of Science, Scopus, Psycinfo and ResearchGate).

Results

we selected and included 31 articles published between 2011 and 2022. These studies fall into three major research traditions (evaluation science, population health intervention research and humanities and social sciences), providing a broad overview of the conceptual mobilization of viable validity. Paradoxically, our literature reveals the concept of viable validity to be poorly operationalized and only partially mature, owing to a lack of consensus among the research traditions with regard to its definition, as well as the porous boundaries between this concept and adjacent concepts such as feasibility and acceptability.

Conclusion

Viable validity is a complex concept, and its operational application constitutes a major challenge for research into and evaluation of population health interventions.

目的：可行有效性的概念于2010年在综合有效性模型的框架内首次被定义。在这几年中，这一概念不断演变，本系统性综述的目的是描述和分析不同研究传统对这一概念的部署和运用方式：我们首先收录了所有引用 Chen 原文 "自下而上的综合有效性方法：项目评估的新视角（Eval Program Plann.结果：我们从 5 个数据库（PubMed、Web of Science、Scopus、Psycinfo 和 ResearchGate）中选择并收录了 2011 年至 2022 年间发表的 31 篇文章。这些研究分属三大研究传统（评估科学、人口健康干预研究以及人文和社会科学），为可行有效性的概念调动提供了一个广泛的概览。矛盾的是，我们的文献显示，由于各研究传统对可行有效性的定义缺乏共识，以及这一概念与可行性和可接受性等相邻概念之间的界限不清，可行有效性这一概念的可操作性很差，而且只是部分成熟：可行有效性是一个复杂的概念，其实际应用对人口健康干预措施的研究和评估构成了重大挑战。

{"title":"Uses of the viable validity concept: A systematic scoping review","authors":"Césarine Sambou , Charlotte Decroix , Judith Martin-Fernandez , Linda Cambon , François Alla","doi":"10.1016/j.evalprogplan.2024.102516","DOIUrl":"10.1016/j.evalprogplan.2024.102516","url":null,"abstract":"<div><h3>Objective</h3><div>The concept of viable validity was first defined in 2010 within the framework of the integrative validity model. The concept has continued to evolve in the intervening years, and the purpose of this systematic scoping review is to describe and analyze the ways in which it has been deployed and appropriated by various research traditions.</div></div><div><h3>Methods</h3><div>We began by including all articles which cite Chen’s original article “<em>The bottom-up approach to integrative validity: a new perspective for program evaluation</em> (Eval Program Plann. 2010;33(3):205–14) and/or contain the terms “viable validity” or “viable cogency,” sourced from 5 databases (PubMed, Web of Science, Scopus, Psycinfo and ResearchGate).</div></div><div><h3>Results</h3><div>we selected and included 31 articles published between 2011 and 2022. These studies fall into three major research traditions (evaluation science, population health intervention research and humanities and social sciences), providing a broad overview of the conceptual mobilization of viable validity. Paradoxically, our literature reveals the concept of viable validity to be poorly operationalized and only partially mature, owing to a lack of consensus among the research traditions with regard to its definition, as well as the porous boundaries between this concept and adjacent concepts such as feasibility and acceptability.</div></div><div><h3>Conclusion</h3><div>Viable validity is a complex concept, and its operational application constitutes a major challenge for research into and evaluation of population health interventions.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"108 ","pages":"Article 102516"},"PeriodicalIF":1.5,"publicationDate":"2024-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142644972","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Analyzing the impact of the new educational policy 2020: A comprehensive review of India's educational reforms 分析 2020 年新教育政策的影响：全面回顾印度的教育改革。

IF 1.5 4区社会学 Q2 SOCIAL SCIENCES, INTERDISCIPLINARY

Evaluation and Program Planning

Pub Date : 2024-11-13 DOI: 10.1016/j.evalprogplan.2024.102515

T. Ashokkumar , T. Russel Raj , A. Rajadurai , A.H. Abishini , A.H. Anchani

The National Education Policy 2020 (NEP 2020) marks a significant milestone in India's education system, aiming to transform the country's educational landscape. This review provides a comprehensive analysis of NEP 2020, exploring its alignment with global educational trends, emphasis on competency-based progression, and potential impact on student learning outcomes, teacher training, and educational infrastructure. The policy's scientific significance lies in its potential to promote equity, quality, and innovation in Indian education, addressing longstanding challenges and preparing students for the demands of the 21st century

2020 年国家教育政策》（NEP 2020）是印度教育系统的一个重要里程碑，旨在改变印度的教育面貌。本评论对《2020 年国家教育政策》进行了全面分析，探讨了其与全球教育趋势的一致性、对基于能力的进步的重视，以及对学生学习成果、教师培训和教育基础设施的潜在影响。该政策的科学意义在于它有可能促进印度教育的公平、质量和创新，解决长期存在的挑战，培养学生适应 21 世纪的需求。

引用次数: 0

A deep dive into the factors affecting household solar photovoltaic adoption: An extended perspective of the value-based adoption model 深入探究影响家用太阳能光伏发电应用的因素：基于价值的采用模式的扩展视角。

IF 1.5 4区社会学 Q2 SOCIAL SCIENCES, INTERDISCIPLINARY

Evaluation and Program Planning

Pub Date : 2024-11-09 DOI: 10.1016/j.evalprogplan.2024.102514

Chien-Wei Ho , Cheng-Chien Kuo , Hong-Chan Chang

Solar photovoltaic (PV) products have emerged as a promising solution to address environmental and green energy issues in light of increasing concerns about carbon emissions, environmental awareness, and renewable energy. This research uses the value-based adoption model (VAM) to examine how perceived benefits and sacrifices impact the adoption intention of PV technology. It additionally explores the role of familiarity with the service as a moderator and its impact on perceived harm risk, perceived value, and adoption intention. The results reveal that economic and environmental benefits significantly influence perceived value, while perceived financial and harm risks also play substantial roles in shaping perceived value. Moreover, the study shows a positive relationship between perceived value and the intention to adopt solar PV and also suggests that familiarity with solar services moderates the relationship between perceived harm risk and perceived value. These insights are valuable for policymakers and energy companies seeking to understand consumers’ perspectives better and increase their intentions to adopt PV technology.

随着人们对碳排放、环保意识和可再生能源的日益关注，太阳能光伏（PV）产品已成为解决环境和绿色能源问题的一种前景广阔的解决方案。本研究采用基于价值的采用模型（VAM）来考察感知到的利益和牺牲如何影响光伏技术的采用意向。此外，研究还探讨了对服务的熟悉程度作为调节因素的作用及其对感知危害风险、感知价值和采用意愿的影响。研究结果表明，经济和环境效益对感知价值有显著影响，而感知的经济风险和危害风险也对感知价值的形成起着重要作用。此外，研究还表明，感知价值与采用太阳能光伏发电的意愿之间存在正相关关系，而且对太阳能服务的熟悉程度也会缓和感知危害风险与感知价值之间的关系。这些见解对于决策者和能源公司更好地了解消费者的观点并提高他们采用光伏技术的意愿非常有价值。

{"title":"A deep dive into the factors affecting household solar photovoltaic adoption: An extended perspective of the value-based adoption model","authors":"Chien-Wei Ho , Cheng-Chien Kuo , Hong-Chan Chang","doi":"10.1016/j.evalprogplan.2024.102514","DOIUrl":"10.1016/j.evalprogplan.2024.102514","url":null,"abstract":"<div><div>Solar photovoltaic (PV) products have emerged as a promising solution to address environmental and green energy issues in light of increasing concerns about carbon emissions, environmental awareness, and renewable energy. This research uses the value-based adoption model (VAM) to examine how perceived benefits and sacrifices impact the adoption intention of PV technology. It additionally explores the role of familiarity with the service as a moderator and its impact on perceived harm risk, perceived value, and adoption intention. The results reveal that economic and environmental benefits significantly influence perceived value, while perceived financial and harm risks also play substantial roles in shaping perceived value. Moreover, the study shows a positive relationship between perceived value and the intention to adopt solar PV and also suggests that familiarity with solar services moderates the relationship between perceived harm risk and perceived value. These insights are valuable for policymakers and energy companies seeking to understand consumers’ perspectives better and increase their intentions to adopt PV technology.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"108 ","pages":"Article 102514"},"PeriodicalIF":1.5,"publicationDate":"2024-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142644970","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Meta-analysis of chemistry-based interdisciplinary informal research experience program for high school students 以化学为基础的高中生跨学科非正式研究体验计划的元分析。

IF 1.5 4区社会学 Q2 SOCIAL SCIENCES, INTERDISCIPLINARY

Evaluation and Program Planning

Pub Date : 2024-11-06 DOI: 10.1016/j.evalprogplan.2024.102517

Noora J. Al-thani , Nitha Siby , Azza Saad , Jolly Bhadra , Noora Qahtani , Abdellatif Sellami , Zubair Ahmad

Research-based learning models, particularly interdisciplinary programs, are crucial in promoting STEM skills and fostering positive attitudes among students. This study presents a meta-analysis evaluating the impact of a chemistry-based interdisciplinary research experience program on high school students in Qatar. Over ten cohorts from 2010 to 2020, 230 students participated in multidisciplinary, hands-on research experiences. The study used pre-post questionnaires and t-test analyses, with further meta-analysis conducted using Comprehensive Meta-Analysis (CMA) software to evaluate the program’s effectiveness in shaping STEM attitudes. Significant findings emerged, with a notable large effect size (Cohen’s d = 0.885), underscoring the program’s efficacy in bolstering positive STEM attitudes among students. The study observed that research complexity and gender do influence the outcome of the program, aligning with the tenets of Social Cognitive Theory, which posits that both individual and contextual elements are influential in attitude formation and learning processes. Conclusively, the research substantiates that interdisciplinary, experiential learning significantly heightens student interest in STEM fields. It also points to the potential benefits of further explorations into the nuances of gender disparities and the complexity of research in enhancing the scope and impact of such educational programs.

以研究为基础的学习模式，尤其是跨学科项目，对于提高学生的 STEM 技能和培养他们的积极态度至关重要。本研究通过荟萃分析，评估了基于化学的跨学科研究体验项目对卡塔尔高中生的影响。在 2010 年至 2020 年的十批学生中，有 230 名学生参加了多学科实践研究体验活动。研究采用了前后问卷调查和 t 检验分析，并使用综合元分析（CMA）软件进行了进一步的元分析，以评估该计划在塑造 STEM 态度方面的有效性。研究发现，该计划在增强学生对科学、技术和工程学的积极态度方面具有显著的效果（Cohen's d = 0.885）。研究发现，研究的复杂性和性别确实会影响计划的结果，这与社会认知理论的信条一致，即个人和环境因素都会影响态度的形成和学习过程。总之，研究证实，跨学科的体验式学习大大提高了学生对 STEM 领域的兴趣。研究还指出，进一步探索性别差异的细微差别和研究的复杂性，对扩大此类教育计划的范围和影响具有潜在的益处。

{"title":"Meta-analysis of chemistry-based interdisciplinary informal research experience program for high school students","authors":"Noora J. Al-thani , Nitha Siby , Azza Saad , Jolly Bhadra , Noora Qahtani , Abdellatif Sellami , Zubair Ahmad","doi":"10.1016/j.evalprogplan.2024.102517","DOIUrl":"10.1016/j.evalprogplan.2024.102517","url":null,"abstract":"<div><div>Research-based learning models, particularly interdisciplinary programs, are crucial in promoting STEM skills and fostering positive attitudes among students. This study presents a meta-analysis evaluating the impact of a chemistry-based interdisciplinary research experience program on high school students in Qatar. Over ten cohorts from 2010 to 2020, 230 students participated in multidisciplinary, hands-on research experiences. The study used pre-post questionnaires and t-test analyses, with further meta-analysis conducted using Comprehensive Meta-Analysis (CMA) software to evaluate the program’s effectiveness in shaping STEM attitudes. Significant findings emerged, with a notable large effect size (Cohen’s d = 0.885), underscoring the program’s efficacy in bolstering positive STEM attitudes among students. The study observed that research complexity and gender do influence the outcome of the program, aligning with the tenets of Social Cognitive Theory, which posits that both individual and contextual elements are influential in attitude formation and learning processes. Conclusively, the research substantiates that interdisciplinary, experiential learning significantly heightens student interest in STEM fields. It also points to the potential benefits of further explorations into the nuances of gender disparities and the complexity of research in enhancing the scope and impact of such educational programs.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"108 ","pages":"Article 102517"},"PeriodicalIF":1.5,"publicationDate":"2024-11-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142606949","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Using Intervention Mapping to co-design a psychosocial service with youth experiencing mental illness 利用 "干预图谱 "为患有精神疾病的青少年共同设计社会心理服务

IF 1.5 4区社会学 Q2 SOCIAL SCIENCES, INTERDISCIPLINARY

Evaluation and Program Planning

Pub Date : 2024-10-18 DOI: 10.1016/j.evalprogplan.2024.102513

Melissa Savaglio , Marie B.H. Yap , Grace Mitchell , Mandy O’Connor , Ash Vincent , Helen Skouteris

Background

Descriptions of service development processes in the youth mental health sector are lacking. Further, youth with lived experience of mental illness are rarely involved in service design. Intervention Mapping (IM) is a well-established framework for program development, implementation and evaluation, yet its applicability in the youth mental health sector is unknown. This paper describes the use of IM methodology to co-design and develop a psychosocial service to support youth aged 10–25 years experiencing mental illness in Tasmania, Australia.

Methods

The six steps of IM were followed: 1) needs assessment; 2) define program outcomes and objectives; 3) program design; 4) program production; 5) implementation planning; and 6) evaluation planning.

Results

Key outputs of each IM step are described. The service was successfully co-designed with young people at the centre of each step. The service includes wrap-around psychosocial support from lived-experience peer-support workers; outreach; and flexible frequency/intensity/duration to achieve young people’s psychosocial goals.

Conclusions

This is the first study to document the use of IM in co-designing a psychosocial service with youth experiencing mental illness. IM may provide a valuable roadmap for the youth mental health sector in supporting collaborative service design, implementation and evaluation planning, and systematic documentation of service development.

背景缺乏对青少年心理健康领域服务开发过程的描述。此外，有精神疾病生活经历的青少年很少参与服务设计。干预图法（IM）是一种成熟的项目开发、实施和评估框架，但其在青少年心理健康领域的适用性尚不清楚。本文介绍了在澳大利亚塔斯马尼亚州使用 IM 方法共同设计和开发一项社会心理服务，以支持 10-25 岁患有精神疾病的青少年：1) 需求评估；2) 确定项目成果和目标；3) 项目设计；4) 项目制作；5) 实施规划；6) 评估规划。在每个步骤中，都成功地以年轻人为中心共同设计了服务。该服务包括由有生活经验的同伴支持工作者提供的全方位社会心理支持；外联；以及灵活的频率/强度/持续时间，以实现青少年的社会心理目标。在支持合作服务设计、实施和评估规划，以及系统地记录服务发展的过程中，IM 可以为青少年心理健康领域提供有价值的路线图。

{"title":"Using Intervention Mapping to co-design a psychosocial service with youth experiencing mental illness","authors":"Melissa Savaglio , Marie B.H. Yap , Grace Mitchell , Mandy O’Connor , Ash Vincent , Helen Skouteris","doi":"10.1016/j.evalprogplan.2024.102513","DOIUrl":"10.1016/j.evalprogplan.2024.102513","url":null,"abstract":"<div><h3>Background</h3><div>Descriptions of service development processes in the youth mental health sector are lacking. Further, youth with lived experience of mental illness are rarely involved in service design. Intervention Mapping (IM) is a well-established framework for program development, implementation and evaluation, yet its applicability in the youth mental health sector is unknown. This paper describes the use of IM methodology to co-design and develop a psychosocial service to support youth aged 10–25 years experiencing mental illness in Tasmania, Australia.</div></div><div><h3>Methods</h3><div>The six steps of IM were followed: 1) needs assessment; 2) define program outcomes and objectives; 3) program design; 4) program production; 5) implementation planning; and 6) evaluation planning.</div></div><div><h3>Results</h3><div>Key outputs of each IM step are described. The service was successfully co-designed with young people at the centre of each step. The service includes wrap-around psychosocial support from lived-experience peer-support workers; outreach; and flexible frequency/intensity/duration to achieve young people’s psychosocial goals.</div></div><div><h3>Conclusions</h3><div>This is the first study to document the use of IM in co-designing a psychosocial service with youth experiencing mental illness. IM may provide a valuable roadmap for the youth mental health sector in supporting collaborative service design, implementation and evaluation planning, and systematic documentation of service development.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"108 ","pages":"Article 102513"},"PeriodicalIF":1.5,"publicationDate":"2024-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142539501","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Anticipatory evaluation. How to incorporate an anticipatory technique into a theory-driven evaluation process. Results of application in a case study. 预期评估。如何将预测技术纳入理论驱动的评价过程。案例研究中的应用结果。

IF 1.5 4区社会学 Q2 SOCIAL SCIENCES, INTERDISCIPLINARY

Evaluation and Program Planning

Pub Date : 2024-10-16 DOI: 10.1016/j.evalprogplan.2024.102509

Mariangel Pacheco – Troisi , Mónica García-Melón , Fernando Jiménez-Sáez

In recent years, there has been increased focus on strategic learning from impact analysis, including in the field of science, technology, and innovation. In this paper, we propose combining techniques from two fields of study. Firstly, we adopt the approach of addressing impact through productive interactions between science and society, and secondly, we incorporate an anticipatory dimension by integrating game analysis involving key actors. Through a theory-driven evaluation design, we consider expected impacts as promises of the future. Within an anticipatory perspective, the future can be shaped by the interactions among different actors in the present. In this article, we apply this approach step-by-step to a research institute program in Uruguay. We demonstrate how the achieved results offer strategic insights to the program manager for anticipating and attaining the desired impacts. Additionally, we provide summative inputs for accountability using a flexible technique applicable at any stage of the program life cycle. The article concludes with a discussion of the advantages and disadvantages compared to other techniques, along with lessons learned that may benefit other evaluators seeking to replicate this approach. Furthermore, we explore potential extensions and opportunities for further improvement in this research.

近年来，人们越来越关注从影响分析中进行战略学习，包括在科学、技术和创新领域。在本文中，我们建议结合两个研究领域的技术。首先，我们采用了通过科学与社会之间富有成效的互动来处理影响的方法；其次，我们通过整合涉及关键参与者的博弈分析，纳入了预测维度。通过理论驱动的评估设计，我们将预期影响视为对未来的承诺。从预期的角度来看，未来可以通过当前不同参与者之间的互动来塑造。在本文中，我们将这种方法逐步应用于乌拉圭的一项研究所计划。我们展示了所取得的成果是如何为计划管理者提供战略洞察力，以预测和实现预期影响的。此外，我们还利用一种适用于计划生命周期任何阶段的灵活技术，为问责制提供了总结性投入。文章最后讨论了与其他技术相比的优缺点，并总结了经验教训，这些经验教训可能会使其他试图复制这种方法的评估人员受益。此外，我们还探讨了这一研究的潜在延伸和进一步改进的机会。

{"title":"Anticipatory evaluation. How to incorporate an anticipatory technique into a theory-driven evaluation process. Results of application in a case study.","authors":"Mariangel Pacheco – Troisi , Mónica García-Melón , Fernando Jiménez-Sáez","doi":"10.1016/j.evalprogplan.2024.102509","DOIUrl":"10.1016/j.evalprogplan.2024.102509","url":null,"abstract":"<div><div>In recent years, there has been increased focus on strategic learning from impact analysis, including in the field of science, technology, and innovation. In this paper, we propose combining techniques from two fields of study. Firstly, we adopt the approach of addressing impact through productive interactions between science and society, and secondly, we incorporate an anticipatory dimension by integrating game analysis involving key actors. Through a theory-driven evaluation design, we consider expected impacts as promises of the future. Within an anticipatory perspective, the future can be shaped by the interactions among different actors in the present. In this article, we apply this approach step-by-step to a research institute program in Uruguay. We demonstrate how the achieved results offer strategic insights to the program manager for anticipating and attaining the desired impacts. Additionally, we provide summative inputs for accountability using a flexible technique applicable at any stage of the program life cycle. The article concludes with a discussion of the advantages and disadvantages compared to other techniques, along with lessons learned that may benefit other evaluators seeking to replicate this approach. Furthermore, we explore potential extensions and opportunities for further improvement in this research.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"108 ","pages":"Article 102509"},"PeriodicalIF":1.5,"publicationDate":"2024-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142477833","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

What evaluation criteria are used in policy evaluation research: A cross-field literature review 政策评估研究中使用哪些评估标准：跨领域文献综述

IF 1.5 4区社会学 Q2 SOCIAL SCIENCES, INTERDISCIPLINARY

Evaluation and Program Planning

Pub Date : 2024-10-10 DOI: 10.1016/j.evalprogplan.2024.102512

Céline Mavrot , Oto Potluka , Lars Balzer , Véronique Eicher , Sigrid Haunberger , Christine Heuer , François-Xavier Viallon

This literature review offers a comprehensive overview of the use of evaluation criteria across five policy fields: social services, land-use planning, teaching in higher education, vocational education, and the environment. Though it is a key part of the evaluation process, the question of how criteria are defined, chosen, and applied generates surprisingly little debate among the evaluation community. In evaluation practice, criteria are often taken for granted – and occasionally even used in ways that are neither explicit nor transparent. This cross-field literature review shows a strong presence of routinized evaluation criteria (relating to the specifics of each policy field), while some new sets of higher-degree criteria also emerge in the face of social challenges relating to sustainability, public acceptance, or social justice. Criteria development draws on both inductive bottom-up processes (which can include policy stakeholders) and top-down deductive processes (which derive criteria from the literature, as well as from national and international standards). A more profound reflection on evaluation criteria (that is, the dimensions used by societies to assess the success of policy interventions) might be required in the future of evaluation research and planning; a deeper cross-field dialogue could support this endeavor.

本文献综述全面概述了社会服务、土地使用规划、高等教育教学、职业教育和环境这五个政策领域中评价标准的使用情况。尽管这是评估过程中的一个关键部分，但如何定义、选择和应用标准的问题在评估界引起的争论却少得令人吃惊。在评估实践中，标准往往被认为是理所当然的，有时甚至以既不明确也不透明的方式使用。这篇跨领域的文献综述显示，常规化的评估标准（与每个政策领域的具体情况有关）非常普遍，同时，面对与可持续性、公众接受度或社会公正有关的社会挑战，也出现了一些新的更高级别的标准。标准的制定既有自下而上的归纳过程（可包括政策利益相关者），也有自上而下的演绎过程（从文献以及国家和国际标准中得出标准）。在未来的评估研究和规划中，可能需要对评估标准（即社会用来评估政策干预措施成功与否的维度）进行更深入的思考；更深入的跨领域对话可以为这一努力提供支持。

{"title":"What evaluation criteria are used in policy evaluation research: A cross-field literature review","authors":"Céline Mavrot , Oto Potluka , Lars Balzer , Véronique Eicher , Sigrid Haunberger , Christine Heuer , François-Xavier Viallon","doi":"10.1016/j.evalprogplan.2024.102512","DOIUrl":"10.1016/j.evalprogplan.2024.102512","url":null,"abstract":"<div><div>This literature review offers a comprehensive overview of the use of evaluation criteria across five policy fields: social services, land-use planning, teaching in higher education, vocational education, and the environment. Though it is a key part of the evaluation process, the question of how criteria are defined, chosen, and applied generates surprisingly little debate among the evaluation community. In evaluation practice, criteria are often taken for granted – and occasionally even used in ways that are neither explicit nor transparent. This cross-field literature review shows a strong presence of routinized evaluation criteria (relating to the specifics of each policy field), while some new sets of higher-degree criteria also emerge in the face of social challenges relating to sustainability, public acceptance, or social justice. Criteria development draws on both inductive bottom-up processes (which can include policy stakeholders) and top-down deductive processes (which derive criteria from the literature, as well as from national and international standards). A more profound reflection on evaluation criteria (that is, the dimensions used by societies to assess the success of policy interventions) might be required in the future of evaluation research and planning; a deeper cross-field dialogue could support this endeavor.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"108 ","pages":"Article 102512"},"PeriodicalIF":1.5,"publicationDate":"2024-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142444898","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A program evaluation of the new choices workforce development program: An appreciative inquiry approach 新选择劳动力发展计划的项目评估：欣赏式探究方法

IF 1.5 4区社会学 Q2 SOCIAL SCIENCES, INTERDISCIPLINARY

Evaluation and Program Planning

Pub Date : 2024-10-09 DOI: 10.1016/j.evalprogplan.2024.102507

Denise Whitacre

A significant amount of money ($1.76B annually in the United States) is spent on workforce development programs, while there is limited research on the effectiveness of workforce development programs in meeting their program objectives and assisting program participants in attaining employment. This study evaluated the New Choices Program, a workforce development program offered by PA Women Work, to help its clients obtain employment and overcome personal and professional barriers. The program has historically been offered in a 30-hour in person format but was forced to be modified to a 10-hour virtual program when the COVID-19 pandemic restrictions began. This program evaluation included a comparative analysis of the perceptions of participants in the 30-hour in person program and the 10-hour virtual program. It was found that participants in both the 30-hour in person program and 10-hour virtual program perceived the program positively, experienced an increase in self-confidence and belonging, which led to either obtaining employment or being better prepared for the job search process. The data will help inform the New Choices program stakeholders on programmatic improvements and how best to structure the program in the post-pandemic employment world.

劳动力发展计划花费了大量资金（美国每年花费 17.6 亿美元），但有关劳动力发展计划在实现其计划目标和帮助计划参与者实现就业方面的有效性的研究却十分有限。本研究评估了 "新选择计划"（New Choices Program），这是宾夕法尼亚州妇女工作组织提供的一项劳动力发展计划，旨在帮助其客户获得就业并克服个人和职业障碍。该计划历来以 30 小时的面授形式提供，但在 COVID-19 大流行限制开始时被迫改为 10 小时的虚拟计划。该项目评估包括对 30 小时面授项目和 10 小时虚拟项目参与者的看法进行比较分析。结果发现，30 小时面授课程和 10 小时虚拟课程的参与者都对该课程持积极态度，自信心和归属感都有所增强，从而获得了工作或为求职过程做好了更充分的准备。这些数据将帮助 "新选择 "计划的利益相关者了解计划的改进情况，以及在大流行后的就业形势下如何以最佳方式组织该计划。

{"title":"A program evaluation of the new choices workforce development program: An appreciative inquiry approach","authors":"Denise Whitacre","doi":"10.1016/j.evalprogplan.2024.102507","DOIUrl":"10.1016/j.evalprogplan.2024.102507","url":null,"abstract":"<div><div>A significant amount of money ($1.76B annually in the United States) is spent on workforce development programs, while there is limited research on the effectiveness of workforce development programs in meeting their program objectives and assisting program participants in attaining employment. This study evaluated the New Choices Program, a workforce development program offered by PA Women Work, to help its clients obtain employment and overcome personal and professional barriers. The program has historically been offered in a 30-hour in person format but was forced to be modified to a 10-hour virtual program when the COVID-19 pandemic restrictions began. This program evaluation included a comparative analysis of the perceptions of participants in the 30-hour in person program and the 10-hour virtual program. It was found that participants in both the 30-hour in person program and 10-hour virtual program perceived the program positively, experienced an increase in self-confidence and belonging, which led to either obtaining employment or being better prepared for the job search process. The data will help inform the New Choices program stakeholders on programmatic improvements and how best to structure the program in the post-pandemic employment world.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"108 ","pages":"Article 102507"},"PeriodicalIF":1.5,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142438413","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0