Pub Date : 2022-03-24DOI: 10.1177/1035719X221080575
Giulia Puinean, R. Gokiert, Mischa Taylor, S. Jun, Pieter de Vos
Children’s early experiences and environments profoundly impact their development; therefore, ensuring the well-being of children through effective supports and services is critical. Evaluation is a tool that can be used to understand the effectiveness of early childhood development (ECD) practices, programs, and policies. A deeper understanding of the evaluation landscape in the ECD field is needed at this time. The purpose of this scoping review was to explore the state of evaluation in the ECD field across four constructs: community-driven evaluation, culturally responsive evaluation, evaluation capacity building, and evaluation use and influence. A comprehensive search of 7 electronic databases, including Canadian and international literature published in English from 2000 to 2020, was conducted. A total of 30 articles met the inclusion criteria. Findings demonstrate that some studies include aspects of a community-engaged approach to evaluation; however, comprehensive approaches to community-driven evaluation, culturally responsive evaluation, evaluation capacity building, and evaluation use in the field of ECD are not commonly achieved. This review will inform strategies for bridging evaluation gaps in the ECD sector, ultimately equipping organisations with the evaluative tools to improve practices, programs, and policies that impact the children, families, and communities they serve.
{"title":"Evaluation in the field of early childhood development: A scoping review","authors":"Giulia Puinean, R. Gokiert, Mischa Taylor, S. Jun, Pieter de Vos","doi":"10.1177/1035719X221080575","DOIUrl":"https://doi.org/10.1177/1035719X221080575","url":null,"abstract":"Children’s early experiences and environments profoundly impact their development; therefore, ensuring the well-being of children through effective supports and services is critical. Evaluation is a tool that can be used to understand the effectiveness of early childhood development (ECD) practices, programs, and policies. A deeper understanding of the evaluation landscape in the ECD field is needed at this time. The purpose of this scoping review was to explore the state of evaluation in the ECD field across four constructs: community-driven evaluation, culturally responsive evaluation, evaluation capacity building, and evaluation use and influence. A comprehensive search of 7 electronic databases, including Canadian and international literature published in English from 2000 to 2020, was conducted. A total of 30 articles met the inclusion criteria. Findings demonstrate that some studies include aspects of a community-engaged approach to evaluation; however, comprehensive approaches to community-driven evaluation, culturally responsive evaluation, evaluation capacity building, and evaluation use in the field of ECD are not commonly achieved. This review will inform strategies for bridging evaluation gaps in the ECD sector, ultimately equipping organisations with the evaluative tools to improve practices, programs, and policies that impact the children, families, and communities they serve.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"22 1","pages":"63 - 89"},"PeriodicalIF":0.0,"publicationDate":"2022-03-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49131773","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-01-12DOI: 10.1177/1035719X211057630
Milbert Gawaya, Desiree Terrill, Eleanor B. Williams
The COVID-19 pandemic required large-scale service delivery changes for government, and provided the opportunity for evaluators to step up and support decision makers to understand the impact of these changes. Rapid evaluation methods (REM) provide a pragmatic approach for generating timely information for evidence-based policy and decision-making. Grounded in developmental and utilisation-focused evaluation theory, REM incorporates a team-based, mixed methods design, executed over a 6–8-week period. Customised rubrics were used to rigorously assess effectiveness and scalability of practice changes to inform COVID-19 response planning. REM is an alternative approach to full-scale evaluation models frequently implemented to assess policies and programs. Adapted use of REM suggests that meaningful insights can be gained through use of smaller scale evaluations. This article shares lessons learned from a novel rapid evaluation method applied in the context of the COVID-19 pandemic. The rapid evaluation approach was implemented to provide real-time insights and evaluative conclusions for 15 program and practice adaptations across Victorian health and human service settings. The article shares insights about the practical applicability of balancing rigour and timeliness when implementing a rapid evaluation, and strengths and limitations of working within a fast-paced evaluation framework. Findings can inform evaluative practice in resource and time-limited settings.
{"title":"Using rapid evaluation methods to assess service delivery changes: Lessons learned for evaluation practice during the COVID-19 pandemic","authors":"Milbert Gawaya, Desiree Terrill, Eleanor B. Williams","doi":"10.1177/1035719X211057630","DOIUrl":"https://doi.org/10.1177/1035719X211057630","url":null,"abstract":"The COVID-19 pandemic required large-scale service delivery changes for government, and provided the opportunity for evaluators to step up and support decision makers to understand the impact of these changes. Rapid evaluation methods (REM) provide a pragmatic approach for generating timely information for evidence-based policy and decision-making. Grounded in developmental and utilisation-focused evaluation theory, REM incorporates a team-based, mixed methods design, executed over a 6–8-week period. Customised rubrics were used to rigorously assess effectiveness and scalability of practice changes to inform COVID-19 response planning. REM is an alternative approach to full-scale evaluation models frequently implemented to assess policies and programs. Adapted use of REM suggests that meaningful insights can be gained through use of smaller scale evaluations. This article shares lessons learned from a novel rapid evaluation method applied in the context of the COVID-19 pandemic. The rapid evaluation approach was implemented to provide real-time insights and evaluative conclusions for 15 program and practice adaptations across Victorian health and human service settings. The article shares insights about the practical applicability of balancing rigour and timeliness when implementing a rapid evaluation, and strengths and limitations of working within a fast-paced evaluation framework. Findings can inform evaluative practice in resource and time-limited settings.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"22 1","pages":"30 - 48"},"PeriodicalIF":0.0,"publicationDate":"2022-01-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42267449","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-12-09DOI: 10.1177/1035719x211054892
A. Rutter
Nicole tells us her story of coming to a decision to work more closely in the Indigenous space. She embarked on a journey of self-discovery which resulted in not only understanding herself but also her own people. One of her important discoveries was finding the Australian Evaluation Society (AES) and meeting people of like minds. She questioned projects where measurement is determined by the funder, not by the Indigenous people who are being impacted by the evaluation, and on the programmes which affect their lives. She has also realised that it is not enough to do evaluation in a culturally safe way, but that evaluation should be culturally safe and trauma informed. You need to understand the trauma histories of people and communities and that knowledge needs to be embedded in your methodology. Without that knowledge, you cannot expect to understand the community and its issues or produce outcomes that are healing.
{"title":"Evaluator Perspective: An interview with Nicole Tujague","authors":"A. Rutter","doi":"10.1177/1035719x211054892","DOIUrl":"https://doi.org/10.1177/1035719x211054892","url":null,"abstract":"Nicole tells us her story of coming to a decision to work more closely in the Indigenous space. She embarked on a journey of self-discovery which resulted in not only understanding herself but also her own people. One of her important discoveries was finding the Australian Evaluation Society (AES) and meeting people of like minds. She questioned projects where measurement is determined by the funder, not by the Indigenous people who are being impacted by the evaluation, and on the programmes which affect their lives. She has also realised that it is not enough to do evaluation in a culturally safe way, but that evaluation should be culturally safe and trauma informed. You need to understand the trauma histories of people and communities and that knowledge needs to be embedded in your methodology. Without that knowledge, you cannot expect to understand the community and its issues or produce outcomes that are healing.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"22 1","pages":"49 - 54"},"PeriodicalIF":0.0,"publicationDate":"2021-12-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42649905","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-12-05DOI: 10.1177/1035719X211055229
K. O'Rourke, N. Abdulghani, J. Yelland, M. Newton, T. Shafiei
Realist interviews are a data collection method used in realist evaluations. There is little available guidance for realist interviewing in cross-cultural contexts. Few published realist evaluations have included cross-cultural interviews, providing limited analyses of the cross-cultural application of realist methodology. This study integrated realist and cross-cultural qualitative methods in a realist evaluation of an Australian doula support program. The interviews were conducted with Arabic speaking clients of the program. The process included collaboration with a bicultural researcher, philosophically situating the study for methodologically coherent integration, bicultural review of the appropriateness of realist ‘how’ and ‘why’ questions, decisions about language translation and interpretation, pilot interviews, and co-facilitation of the interviews. Integration of the methods was feasible and valuable. This study may support other realist evaluators to give voice to people from culturally diverse groups, in a manner that is culturally safe, methodologically coherent and rigorous, and that produces trustworthy results.
{"title":"Cross-cultural realist interviews: An integration of the realist interview and cross-cultural qualitative research methods","authors":"K. O'Rourke, N. Abdulghani, J. Yelland, M. Newton, T. Shafiei","doi":"10.1177/1035719X211055229","DOIUrl":"https://doi.org/10.1177/1035719X211055229","url":null,"abstract":"Realist interviews are a data collection method used in realist evaluations. There is little available guidance for realist interviewing in cross-cultural contexts. Few published realist evaluations have included cross-cultural interviews, providing limited analyses of the cross-cultural application of realist methodology. This study integrated realist and cross-cultural qualitative methods in a realist evaluation of an Australian doula support program. The interviews were conducted with Arabic speaking clients of the program. The process included collaboration with a bicultural researcher, philosophically situating the study for methodologically coherent integration, bicultural review of the appropriateness of realist ‘how’ and ‘why’ questions, decisions about language translation and interpretation, pilot interviews, and co-facilitation of the interviews. Integration of the methods was feasible and valuable. This study may support other realist evaluators to give voice to people from culturally diverse groups, in a manner that is culturally safe, methodologically coherent and rigorous, and that produces trustworthy results.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"22 1","pages":"5 - 17"},"PeriodicalIF":0.0,"publicationDate":"2021-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49160355","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-12-05DOI: 10.1177/1035719X211053072
Kathryn Erskine, M. Healey
This paper details disruption and innovation in digital evaluation practice at Movember, as a result of the COVID-19 pandemic. The paper examines a men’s digital health intervention (DHI) – Movember Conversations – and the product pivot that was necessary to ensure it could respond to the pandemic. The paper focuses on the implications of the pivot for the evaluation and how the evaluation was adapted to the COVID-19 exigencies. It details the redesign of the evaluation to ensure methods wrapped around the modified product and could deliver real-time, practical insights. The paper seeks to fill knowledge gaps in the DHI evaluation space and outlines four key principles that support evaluation re-design in an agile setting. These include a user-centred approach to evaluation design, proportionate data collection, mixed (and flexible) methodologies, and agile evaluation reporting. The paper concludes with key lessons and reflections from the evaluators about what worked at Movember, to support other evaluators planning digital evaluations.
{"title":"Conversations that count: Lessons from evaluating a men’s digital mental health response during COVID-19","authors":"Kathryn Erskine, M. Healey","doi":"10.1177/1035719X211053072","DOIUrl":"https://doi.org/10.1177/1035719X211053072","url":null,"abstract":"This paper details disruption and innovation in digital evaluation practice at Movember, as a result of the COVID-19 pandemic. The paper examines a men’s digital health intervention (DHI) – Movember Conversations – and the product pivot that was necessary to ensure it could respond to the pandemic. The paper focuses on the implications of the pivot for the evaluation and how the evaluation was adapted to the COVID-19 exigencies. It details the redesign of the evaluation to ensure methods wrapped around the modified product and could deliver real-time, practical insights. The paper seeks to fill knowledge gaps in the DHI evaluation space and outlines four key principles that support evaluation re-design in an agile setting. These include a user-centred approach to evaluation design, proportionate data collection, mixed (and flexible) methodologies, and agile evaluation reporting. The paper concludes with key lessons and reflections from the evaluators about what worked at Movember, to support other evaluators planning digital evaluations.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"22 1","pages":"18 - 29"},"PeriodicalIF":0.0,"publicationDate":"2021-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49107508","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：