Lena Dedyukina, Celina Wolki, Denise Wolki, S. Wesche, T. Kenny, Kelly Skinner
This study investigates the implementation of the Nutrition North Canada (NNC)-funded cooking circle program in the Inuvialuit (Inuit) hamlet of Paulatuk, Northwest Territories. The objectives of this study are to co-develop a culturally relevant logic model and to conduct a process evaluation of program implementation to identify and assess key enablers and barriers. The co-developed Mukluk Logic Model played an instrumental role in the conceptualization of the process evaluation. The process evaluation results indicated that the long-standing sustainability of the program is related to the consistency of program funding, engaging facilitation practices, and creative utilization of the multi-purpose space for program activities. However, significant barriers limit program sustainability. These include funding amounts and distribution, space and equipment limitations, and human resources challenges. This study illustrates the utility of qualitative process evaluation research in a Canadian Arctic community context and generates important place-based knowledge and insights to better support northern community-based food preparation programs.
{"title":"Process Evaluation of a Cooking Circle Program in the Arctic: Developing the Mukluk Logic Model and Identifying Key Enablers and Barriers for Program Implementation","authors":"Lena Dedyukina, Celina Wolki, Denise Wolki, S. Wesche, T. Kenny, Kelly Skinner","doi":"10.3138/cjpe-2023-0031","DOIUrl":"https://doi.org/10.3138/cjpe-2023-0031","url":null,"abstract":"This study investigates the implementation of the Nutrition North Canada (NNC)-funded cooking circle program in the Inuvialuit (Inuit) hamlet of Paulatuk, Northwest Territories. The objectives of this study are to co-develop a culturally relevant logic model and to conduct a process evaluation of program implementation to identify and assess key enablers and barriers. The co-developed Mukluk Logic Model played an instrumental role in the conceptualization of the process evaluation. The process evaluation results indicated that the long-standing sustainability of the program is related to the consistency of program funding, engaging facilitation practices, and creative utilization of the multi-purpose space for program activities. However, significant barriers limit program sustainability. These include funding amounts and distribution, space and equipment limitations, and human resources challenges. This study illustrates the utility of qualitative process evaluation research in a Canadian Arctic community context and generates important place-based knowledge and insights to better support northern community-based food preparation programs.","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":"475 ","pages":""},"PeriodicalIF":0.8,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138992695","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Rich Janzen, Sarah Switzer, Jean de Dieu Basabose, Madeline Docherty, Paul Heidebrecht
The purpose of this practice note is to demonstrate how a community-based approach to evaluation can be used to support the evaluations of the United Nation’s Sustainable Development Goals in local community settings. The authors argue that using a community-based evaluation approach that is stakeholder-driven, participatory, and action-oriented is one mechanism to address this challenge. They illustrate the utility of such an evaluation approach by identifying their key learnings through a national capacity-building project.
{"title":"Community-Based Evaluation When Localizing Sustainable Development Goals","authors":"Rich Janzen, Sarah Switzer, Jean de Dieu Basabose, Madeline Docherty, Paul Heidebrecht","doi":"10.3138/cjpe.75810","DOIUrl":"https://doi.org/10.3138/cjpe.75810","url":null,"abstract":"The purpose of this practice note is to demonstrate how a community-based approach to evaluation can be used to support the evaluations of the United Nation’s Sustainable Development Goals in local community settings. The authors argue that using a community-based evaluation approach that is stakeholder-driven, participatory, and action-oriented is one mechanism to address this challenge. They illustrate the utility of such an evaluation approach by identifying their key learnings through a national capacity-building project.","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":"203 1","pages":""},"PeriodicalIF":0.8,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139013599","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Shelbi Nahwilet Meissner, Jeremy Braithwaite, Karan Thorne, Art Martinez, Elizabeth Lycett
This paper offers some key characteristics of Indigenous feminist approaches to evaluation and spotlights a unique and promising example of Indigenous feminist evaluation methods in the My Two Aunties (M2A) program. Though Indigenous feminist evaluation methods are diverse, complex, and community-specific, some general characteristics we point to in this analysis are commitments to anti-colonial conceptions of family, gender, and belonging, an assertion of the epistemic and evaluative importance of felt knowledge, the explicit confrontation of settler colonialism’s impact on Indigenous life, and the commitment to the transformative potential of community-led caretaking. We then turn to what we see as an exemplar of Indigenous feminist evaluation methods—the evaluation component of the My Two Aunties (M2A) program. Our paper will provide theoretical scaffolding for Indigenous feminist evaluation and add to the growing body of Indigenous scholarship that challenges what “counts” as evidence in settler scholarship arenas.
{"title":"Indigenous Feminist Evaluation Methods: A Case Study in “My Two Aunties”","authors":"Shelbi Nahwilet Meissner, Jeremy Braithwaite, Karan Thorne, Art Martinez, Elizabeth Lycett","doi":"10.3138/cjpe-2023-0042","DOIUrl":"https://doi.org/10.3138/cjpe-2023-0042","url":null,"abstract":"This paper offers some key characteristics of Indigenous feminist approaches to evaluation and spotlights a unique and promising example of Indigenous feminist evaluation methods in the My Two Aunties (M2A) program. Though Indigenous feminist evaluation methods are diverse, complex, and community-specific, some general characteristics we point to in this analysis are commitments to anti-colonial conceptions of family, gender, and belonging, an assertion of the epistemic and evaluative importance of felt knowledge, the explicit confrontation of settler colonialism’s impact on Indigenous life, and the commitment to the transformative potential of community-led caretaking. We then turn to what we see as an exemplar of Indigenous feminist evaluation methods—the evaluation component of the My Two Aunties (M2A) program. Our paper will provide theoretical scaffolding for Indigenous feminist evaluation and add to the growing body of Indigenous scholarship that challenges what “counts” as evidence in settler scholarship arenas.","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":"369 1","pages":""},"PeriodicalIF":0.8,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139022180","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Arts-based evaluation is an effective way to engage people and uncover meaningful, valid results. In the project that was used to inform this article, an arts-based approach was used to gain an understanding of the effects the Communities ChooseWell Program has had according to the Champions’ experiences. Communities ChooseWell is an Alberta Recreation and Parks Association Program that promotes healthy eating and active living in communities across Alberta. The evaluation was completed using an arts-based approach which allowed us to explore varied long-term effects in different contexts. The arts used within this project include painting, writing narratives, and quilting. This article will present some reflections and recommendations that could inform evaluators that are interested in using arts-based approaches. This article starts with information from relevant literature about arts-based approaches to evaluation followed by what was learned throughout the ChooseWell arts-based evaluation and recommendations for evaluators when planning an arts-based evaluation.
{"title":"Arts-Based Evaluation","authors":"Karyn Hurlbut, Astrid Brousselle","doi":"10.3138/cjpe-2023-0024","DOIUrl":"https://doi.org/10.3138/cjpe-2023-0024","url":null,"abstract":"Arts-based evaluation is an effective way to engage people and uncover meaningful, valid results. In the project that was used to inform this article, an arts-based approach was used to gain an understanding of the effects the Communities ChooseWell Program has had according to the Champions’ experiences. Communities ChooseWell is an Alberta Recreation and Parks Association Program that promotes healthy eating and active living in communities across Alberta. The evaluation was completed using an arts-based approach which allowed us to explore varied long-term effects in different contexts. The arts used within this project include painting, writing narratives, and quilting. This article will present some reflections and recommendations that could inform evaluators that are interested in using arts-based approaches. This article starts with information from relevant literature about arts-based approaches to evaluation followed by what was learned throughout the ChooseWell arts-based evaluation and recommendations for evaluators when planning an arts-based evaluation.","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":"181 ","pages":""},"PeriodicalIF":0.8,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138988310","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mixed-methods citation analysis can be used to conduct research on evaluation (RoE) studies with a cross-country focus. In particular, this RoE study employs mixed-methods citation analysis to investigate the influence of an evaluation theorist’s concepts transnationally. We use Jennifer Greene’s program evaluation theories as a case example for observing the imprint of a theorists’ concepts on evaluators theorizing and practicing outside of the United States. Conducting this case analysis highlights the methodological affordances and shortcomings of citation analysis. The article addresses equity and diversity considerations in using citation analysis, particularly for investigating multidirectional transnational influences in evaluation theory.
{"title":"Employing Mixed-Methods Citation Analysis to Investigate Transnational Influence in Evaluation Theory","authors":"M. Goodnight, Cherie M. Avent, Ramya Kumaran","doi":"10.3138/cjpe-2023-0009","DOIUrl":"https://doi.org/10.3138/cjpe-2023-0009","url":null,"abstract":"Mixed-methods citation analysis can be used to conduct research on evaluation (RoE) studies with a cross-country focus. In particular, this RoE study employs mixed-methods citation analysis to investigate the influence of an evaluation theorist’s concepts transnationally. We use Jennifer Greene’s program evaluation theories as a case example for observing the imprint of a theorists’ concepts on evaluators theorizing and practicing outside of the United States. Conducting this case analysis highlights the methodological affordances and shortcomings of citation analysis. The article addresses equity and diversity considerations in using citation analysis, particularly for investigating multidirectional transnational influences in evaluation theory.","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":"51 ","pages":""},"PeriodicalIF":0.8,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139016849","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Betty Onyura, Emilia Main, Claudia Barned, Alexandra Wong, Tin D. Vo, Nivetha Chandran, Nazi Torabi, Deena M. Hamza
There is growing recognition of the complex moral and ethical tensions associated with evaluation practice. However, there are scant evidence-informed frameworks for cultivating ethical awareness or informing ethical deliberation across the evaluation landscape. Thus, we aimed to synthesize research evidence on evaluation ethics, and draw on these findings to develop an evidence-informed evaluation ethics framework. Our methodological approach involved, first, conducting a meta-narrative review of empirical studies on evaluation ethics. Specifically, we conducted a systematic peer-reviewed and grey literature search, then identified, extracted, and thematically organize data from 20 studies that meet inclusion criteria. Second, in consultation with an ethicist, we curated findings on ethical concerns within an integrated evaluation ethics framework. Our results illustrate six thematic patterns of research inquiry on evaluation ethics and highlight trends, and gaps. The ethics framework (ACAP) we develop includes four multi-faceted categories. It outlines six Accountabilities (where ethical consideration is owed), illustrates how ethical Concerns can manifest in practice, and outlines diverse stakeholder groups’ Agency over the management of ethical concerns. Critically, it outlines five meta-categories of ethical principles (P) including systematic and transparent inquiry, accordant self-determination, fairness, beneficence and non-maleficence, and reflexive stewardship. Implications for priming ethical awareness, navigating ethical conflicts, and advancing evaluation ethics education and research are discussed.
{"title":"The “What” and “Why” of (Un)Ethical Evaluation Practice: A Meta-Narrative Review and Ethical Awareness Framework","authors":"Betty Onyura, Emilia Main, Claudia Barned, Alexandra Wong, Tin D. Vo, Nivetha Chandran, Nazi Torabi, Deena M. Hamza","doi":"10.3138/cjpe-2023-0023","DOIUrl":"https://doi.org/10.3138/cjpe-2023-0023","url":null,"abstract":"There is growing recognition of the complex moral and ethical tensions associated with evaluation practice. However, there are scant evidence-informed frameworks for cultivating ethical awareness or informing ethical deliberation across the evaluation landscape. Thus, we aimed to synthesize research evidence on evaluation ethics, and draw on these findings to develop an evidence-informed evaluation ethics framework. Our methodological approach involved, first, conducting a meta-narrative review of empirical studies on evaluation ethics. Specifically, we conducted a systematic peer-reviewed and grey literature search, then identified, extracted, and thematically organize data from 20 studies that meet inclusion criteria. Second, in consultation with an ethicist, we curated findings on ethical concerns within an integrated evaluation ethics framework. Our results illustrate six thematic patterns of research inquiry on evaluation ethics and highlight trends, and gaps. The ethics framework (ACAP) we develop includes four multi-faceted categories. It outlines six Accountabilities (where ethical consideration is owed), illustrates how ethical Concerns can manifest in practice, and outlines diverse stakeholder groups’ Agency over the management of ethical concerns. Critically, it outlines five meta-categories of ethical principles (P) including systematic and transparent inquiry, accordant self-determination, fairness, beneficence and non-maleficence, and reflexive stewardship. Implications for priming ethical awareness, navigating ethical conflicts, and advancing evaluation ethics education and research are discussed.","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":"141 ","pages":""},"PeriodicalIF":0.8,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139025739","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The U.S. National Institute of Occupational Safety and Health (NIOSH) supports 11 regional centers to conduct research and outreach in agriculture, fishing, and forestry. The Southwest Center for Agricultural Health, Injury Prevention, and Education serves Arkansas, Louisiana, New Mexico, Oklahoma, and Texas. The 2016–2022 evaluation used contribution analysis since it is effective in evaluating complex systems with diverse stakeholders and multiple projects, with partners working toward the same outcome over time. The evaluation responded to the disruptions in program implementation caused by the COVID-19 pandemic and to NIOSH modifications to key elements of contribution analysis procedures.
{"title":"Contribution Analysis of a Complex System During Disruptions","authors":"Sharon L. Newbill","doi":"10.3138/cjpe.75803","DOIUrl":"https://doi.org/10.3138/cjpe.75803","url":null,"abstract":"The U.S. National Institute of Occupational Safety and Health (NIOSH) supports 11 regional centers to conduct research and outreach in agriculture, fishing, and forestry. The Southwest Center for Agricultural Health, Injury Prevention, and Education serves Arkansas, Louisiana, New Mexico, Oklahoma, and Texas. The 2016–2022 evaluation used contribution analysis since it is effective in evaluating complex systems with diverse stakeholders and multiple projects, with partners working toward the same outcome over time. The evaluation responded to the disruptions in program implementation caused by the COVID-19 pandemic and to NIOSH modifications to key elements of contribution analysis procedures.","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":"168 ","pages":""},"PeriodicalIF":0.8,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139025731","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A metaevaluation is a quality cross-check to examine the conduct of an evaluation and validate the results. Of the few metaevaluation studies, almost none have reported on the metaevaluation procedure through a practical example evaluation. This study reports on the strengths and weaknesses of a program evaluation study in terms of the four main standards: utility, feasibility, propriety, and accuracy. It includes a metaevaluation process that involves both quantitative and qualitative analysis of data from eight meta-evaluators. It was found that while the evaluation study had very good utility and accuracy standards, the feasibility and propriety standards were only fair.
{"title":"How to Conduct a Metaevaluation?: A Metaevaluation Practice","authors":"Esra Kerimoğlu, Muazzez Nihal Öykü Ülker, Ş. Berk","doi":"10.3138/cjpe.71619","DOIUrl":"https://doi.org/10.3138/cjpe.71619","url":null,"abstract":"A metaevaluation is a quality cross-check to examine the conduct of an evaluation and validate the results. Of the few metaevaluation studies, almost none have reported on the metaevaluation procedure through a practical example evaluation. This study reports on the strengths and weaknesses of a program evaluation study in terms of the four main standards: utility, feasibility, propriety, and accuracy. It includes a metaevaluation process that involves both quantitative and qualitative analysis of data from eight meta-evaluators. It was found that while the evaluation study had very good utility and accuracy standards, the feasibility and propriety standards were only fair.","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":" ","pages":""},"PeriodicalIF":0.8,"publicationDate":"2023-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46432301","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Ethics for Evaluation. Beyond “Doing No Harm” to “Tackling Bad” and “Doing Good”, par Rob van den Berg, Penny Hawkins et Nicoletta Stame (dir.)","authors":"Steve Jacob","doi":"10.3138/cjpe.76031","DOIUrl":"https://doi.org/10.3138/cjpe.76031","url":null,"abstract":"","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":" ","pages":""},"PeriodicalIF":0.8,"publicationDate":"2023-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48441877","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Thematic Analysis: A Practical Guide, by Virginia Braun and Victoria Clarke","authors":"C. Roseveare","doi":"10.3138/cjpe.76737","DOIUrl":"https://doi.org/10.3138/cjpe.76737","url":null,"abstract":"","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":" ","pages":""},"PeriodicalIF":0.8,"publicationDate":"2023-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44066825","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：