We share experiences from a mixed methods evaluation in rural India that combines a randomized controlled trial (RCT) of 400 villages with embedded case studies in four villages. Specifically, we present two lessons from the multi-stage sampling approach adopted to select the four case-study villages, which first prioritizedkey-informant observations regarding intervention status in order to shortlist locations and subsequently used data from the RCT’s baseline survey to select the final sample. In doing so, we highlight how large-scale mixed methods program evaluations in education can go beyond questions of “what works” to answering those of “how,” “why,” and “why not.”
{"title":"Multi-Stage Approach to Qualitative Sampling within a Mixed Methods Evaluation: Some Reflections on Purpose and Process","authors":"P. Ramanujan, S. Bhattacharjea, B. Alcott","doi":"10.3138/cjpe.71237","DOIUrl":"https://doi.org/10.3138/cjpe.71237","url":null,"abstract":"We share experiences from a mixed methods evaluation in rural India that combines a randomized controlled trial (RCT) of 400 villages with embedded case studies in four villages. Specifically, we present two lessons from the multi-stage sampling approach adopted to select the four case-study villages, which first prioritizedkey-informant observations regarding intervention status in order to shortlist locations and subsequently used data from the RCT’s baseline survey to select the final sample. In doing so, we highlight how large-scale mixed methods program evaluations in education can go beyond questions of “what works” to answering those of “how,” “why,” and “why not.”","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":" ","pages":""},"PeriodicalIF":0.8,"publicationDate":"2022-03-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48005671","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Paisley Worthington, Jennifer Hughes, Michelle Searle
{"title":"Mixed Methods Design in Evaluation by Donna Mertens","authors":"Paisley Worthington, Jennifer Hughes, Michelle Searle","doi":"10.3138/cjpe.73684","DOIUrl":"https://doi.org/10.3138/cjpe.73684","url":null,"abstract":"","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":" ","pages":""},"PeriodicalIF":0.8,"publicationDate":"2022-03-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45529263","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kristina Kopp, Ralph Bodor, Leona Makokis, Selby Quinn, Kaila Kornberger, Stephanie Tyler, Carol Turner, Pauline Smale
Recently a conscious effort has been made by human service providers across Canada to Indigenize programming as reconciliation. However, while the delivery of programs shifted, how they are evaluated remains rooted in Western ideologies and methodologies. In response to the tension created in using Western evaluation methods for assessing Indigenous-designed programs, we developed an Indigenous program evaluation framework. The framework is based in nêhiyaw (Cree) teachings and was co-created by Elders and Knowledge Keepers. We use a case study to demonstrate that an appropriately developed Indigenous program evaluation framework leads to more comprehensive, accurate and meaningful data collection and evaluation.
{"title":"kawiyahîtamik kesi wîcehtâsôk","authors":"Kristina Kopp, Ralph Bodor, Leona Makokis, Selby Quinn, Kaila Kornberger, Stephanie Tyler, Carol Turner, Pauline Smale","doi":"10.3138/cjpe.71495","DOIUrl":"https://doi.org/10.3138/cjpe.71495","url":null,"abstract":"Recently a conscious effort has been made by human service providers across Canada to Indigenize programming as reconciliation. However, while the delivery of programs shifted, how they are evaluated remains rooted in Western ideologies and methodologies. In response to the tension created in using Western evaluation methods for assessing Indigenous-designed programs, we developed an Indigenous program evaluation framework. The framework is based in nêhiyaw (Cree) teachings and was co-created by Elders and Knowledge Keepers. We use a case study to demonstrate that an appropriately developed Indigenous program evaluation framework leads to more comprehensive, accurate and meaningful data collection and evaluation. ","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":" ","pages":""},"PeriodicalIF":0.8,"publicationDate":"2021-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44096346","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Conclusion: key messages for enhancing evaluation practice
结论：加强评价工作的关键信息
{"title":"Conclusion","authors":"Marlène Laeubli Loud","doi":"10.3138/cjpe.72730","DOIUrl":"https://doi.org/10.3138/cjpe.72730","url":null,"abstract":"Conclusion: key messages for enhancing evaluation practice","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":" ","pages":""},"PeriodicalIF":0.8,"publicationDate":"2021-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42910263","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Over the last two decades, evaluation in the Global South has grown by leaps and bounds. Yet a vast majority of the theories and practices in evaluation emanate from the Global North, often shaped by North-South cooperation (NSC) ‘aid’ financing. It is only recently that South-South cooperation (SSC) has become a force to be reckoned with as a result of shifts in economic power from West to East. A collaboration shaped by shared histories, socioeconomic conditions and a set of principles very different from those underlying NSC, the group of around 140 countries in the Global South known as the ‘G77+China’ holds significant potential for changing the dominant, often destructive narratives and practices about development that have shaped much of the world today. Yet this potential is largely underreported and underexplored, and therefore undervalued. The field of development evaluation has been largely absent from, and also at times caught up in the often highly charged political and technical forums where SSC and its evaluation practices are being shaped. Instead, development evaluation continues to be shaped by North-South interactions, diminishing the potential value of innovative engagements with efforts to advance South-South cooperation as well as South-South cooperation in evaluation for the benefit of both the Global South and the global challenges that humanity faces in the era of the Anthropocene. This article introduces some of the key dynamics in South-South cooperation, and highlights some of the opportunities that can inspire more dynamic engagement and collaboration between the global evaluation community – especially those from, and working in the Global South - and the researchers, economists, financiers and decisionmakers involved in South-South cooperation. This can help accelerate efforts to transform the field of evaluation to be more suited to this era.
{"title":"The role of South-South Cooperation in shifting dominant narratives and models in development and in evaluation","authors":"Z. Ofir","doi":"10.3138/cjpe.71630","DOIUrl":"https://doi.org/10.3138/cjpe.71630","url":null,"abstract":"Over the last two decades, evaluation in the Global South has grown by leaps and bounds. Yet a vast majority of the theories and practices in evaluation emanate from the Global North, often shaped by North-South cooperation (NSC) ‘aid’ financing. It is only recently that South-South cooperation (SSC) has become a force to be reckoned with as a result of shifts in economic power from West to East. A collaboration shaped by shared histories, socioeconomic conditions and a set of principles very different from those underlying NSC, the group of around 140 countries in the Global South known as the ‘G77+China’ holds significant potential for changing the dominant, often destructive narratives and practices about development that have shaped much of the world today. Yet this potential is largely underreported and underexplored, and therefore undervalued. The field of development evaluation has been largely absent from, and also at times caught up in the often highly charged political and technical forums where SSC and its evaluation practices are being shaped. Instead, development evaluation continues to be shaped by North-South interactions, diminishing the potential value of innovative engagements with efforts to advance South-South cooperation as well as South-South cooperation in evaluation for the benefit of both the Global South and the global challenges that humanity faces in the era of the Anthropocene. This article introduces some of the key dynamics in South-South cooperation, and highlights some of the opportunities that can inspire more dynamic engagement and collaboration between the global evaluation community – especially those from, and working in the Global South - and the researchers, economists, financiers and decisionmakers involved in South-South cooperation. This can help accelerate efforts to transform the field of evaluation to be more suited to this era.","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":" ","pages":""},"PeriodicalIF":0.8,"publicationDate":"2021-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47956157","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper we set ourselves three challenges, to examine: i) the potential of the data revolution to aid the transformational change required to achieve the sustainable development goals; ii) the ability of evaluation to contribute to (i) through greater engagement with data science technologies; and iii) the capacity of data science to further evaluation as an innovative and progressive field of inquiry. We also discuss the political, economic, cultural, and ethical challenges data science presents to sustainable development and evaluation. We conclude that data science and evaluation can enhance each other to address the key development challenges of our time.
{"title":"Data science frontiers and evaluation’s transformational potential for sustainable development","authors":"A. Hejnowicz, S. Chaplowe","doi":"10.3138/cjpe.71527","DOIUrl":"https://doi.org/10.3138/cjpe.71527","url":null,"abstract":"In this paper we set ourselves three challenges, to examine: i) the potential of the data revolution to aid the transformational change required to achieve the sustainable development goals; ii) the ability of evaluation to contribute to (i) through greater engagement with data science technologies; and iii) the capacity of data science to further evaluation as an innovative and progressive field of inquiry. We also discuss the political, economic, cultural, and ethical challenges data science presents to sustainable development and evaluation. We conclude that data science and evaluation can enhance each other to address the key development challenges of our time.","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":" ","pages":""},"PeriodicalIF":0.8,"publicationDate":"2021-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44231387","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The United Nations is calling on governments, private sector and civil society to “build back better” post-COVID-19. To transform our societies for healthier and more resilient futures for people and nature. It is at this nexus of theory and practice on urgent global environmental changes that appropriate evaluation insights, approaches and methods are needed to nurture new pathways forward. We need new ideas that challenge ‘ business as usual’ to do this. This article argues the Rights of Nature is one such emerging concept that evaluators interested in transformation have the power and means to engage with already in hand.
{"title":"The Rights of Nature","authors":"L. Gallagher, Z. Ofir","doi":"10.3138/cjpe.71575","DOIUrl":"https://doi.org/10.3138/cjpe.71575","url":null,"abstract":"The United Nations is calling on governments, private sector and civil society to “build back better” post-COVID-19. To transform our societies for healthier and more resilient futures for people and nature. It is at this nexus of theory and practice on urgent global environmental changes that appropriate evaluation insights, approaches and methods are needed to nurture new pathways forward. We need new ideas that challenge ‘ business as usual’ to do this. This article argues the Rights of Nature is one such emerging concept that evaluators interested in transformation have the power and means to engage with already in hand. \u0000 ","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":" ","pages":""},"PeriodicalIF":0.8,"publicationDate":"2021-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47653315","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Visionary Evaluation for a Sustainable, Equitable Future","authors":"B. Case","doi":"10.3138/cjpe.72833","DOIUrl":"https://doi.org/10.3138/cjpe.72833","url":null,"abstract":"","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":" ","pages":""},"PeriodicalIF":0.8,"publicationDate":"2021-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43659364","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
An evaluation advisory group can be a vital resource, particularly for evaluations of collaboration-driven initiatives, but only if we apply skills and knowledge to use them effectively. In this paper, we will discuss the basics of evaluation advisory groups on four themes: purposes, structures, processes, and pitfalls. We will discuss insights from a review and synthesis of the evaluation advisory group literature and illustrate with our own real-world experiences of developing and implementing evaluation advisory groups on a national and state-wide scale, in a university setting, and within a localized health setting.
{"title":"Evaluation Advisory Groups: Considerations for Design and Management","authors":"Christopher Voegeli","doi":"10.3138/cjpe.69949","DOIUrl":"https://doi.org/10.3138/cjpe.69949","url":null,"abstract":"An evaluation advisory group can be a vital resource, particularly for evaluations of collaboration-driven initiatives, but only if we apply skills and knowledge to use them effectively. In this paper, we will discuss the basics of evaluation advisory groups on four themes: purposes, structures, processes, and pitfalls. We will discuss insights from a review and synthesis of the evaluation advisory group literature and illustrate with our own real-world experiences of developing and implementing evaluation advisory groups on a national and state-wide scale, in a university setting, and within a localized health setting.","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":" ","pages":""},"PeriodicalIF":0.8,"publicationDate":"2021-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47584922","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Trena M. Paulus and Jessica N. Lester (2021), Doing Qualitative Research in a Digital World, Thousand Oaks, CA: Sage. VitalSource. 376 pages. (ISBN: 9781544321585) Reviewed by Kenza Bennani
Trena M. Paulus和Jessica N. Lester(2021)，在数字世界中进行定性研究，加州千橡市:Sage。VitalSource。376页。(ISBN: 9781544321585)由Kenza Bennani审阅
{"title":"Doing Qualitative Research in a Digital World","authors":"Kenza Bennani","doi":"10.3138/cjpe.72911","DOIUrl":"https://doi.org/10.3138/cjpe.72911","url":null,"abstract":"Trena M. Paulus and Jessica N. Lester (2021), Doing Qualitative Research in a Digital World, Thousand Oaks, CA: Sage. VitalSource. 376 pages. (ISBN: 9781544321585) \u0000Reviewed by Kenza Bennani","PeriodicalId":43924,"journal":{"name":"Canadian Journal of Program Evaluation","volume":" ","pages":""},"PeriodicalIF":0.8,"publicationDate":"2021-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43661492","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：