Pub Date : 2021-09-20DOI: 10.1177/1035719X211043674
Jeffery Adams, S. Neville
Evaluators are committed to practice that is high quality and conducted ethically. Despite this, gender and sexually diverse populations are not always adequately considered in evaluation practice. Ensuring fuller inclusion of gender and sexually diverse people is required to give effect to human rights obligations and to enable comparatively poorer wellbeing outcomes for these groups to be addressed. Based on our experience conducting evaluation (and research) with both general populations and gender and sexually diverse populations, we suggest a need to build inclusive practice among evaluators. To guide inclusive evaluation practice, we outline three domains for consideration – terminology and language, processes of research inclusion and implications of inclusion. These are not offered as a checklist but as a way to encourage reflexive practice among evaluators. Given evaluators are often concerned with promoting equity and social justice, we are hopeful that actions taken by evaluators can enhance the inclusion of gender and sexually diverse people in evaluation activities and contribute to better wellbeing outcomes for them.
{"title":"Enhancing the inclusion of gender and sexually diverse populations in evaluation: Reflections grounded in practice","authors":"Jeffery Adams, S. Neville","doi":"10.1177/1035719X211043674","DOIUrl":"https://doi.org/10.1177/1035719X211043674","url":null,"abstract":"Evaluators are committed to practice that is high quality and conducted ethically. Despite this, gender and sexually diverse populations are not always adequately considered in evaluation practice. Ensuring fuller inclusion of gender and sexually diverse people is required to give effect to human rights obligations and to enable comparatively poorer wellbeing outcomes for these groups to be addressed. Based on our experience conducting evaluation (and research) with both general populations and gender and sexually diverse populations, we suggest a need to build inclusive practice among evaluators. To guide inclusive evaluation practice, we outline three domains for consideration – terminology and language, processes of research inclusion and implications of inclusion. These are not offered as a checklist but as a way to encourage reflexive practice among evaluators. Given evaluators are often concerned with promoting equity and social justice, we are hopeful that actions taken by evaluators can enhance the inclusion of gender and sexually diverse people in evaluation activities and contribute to better wellbeing outcomes for them.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"21 1","pages":"189 - 205"},"PeriodicalIF":0.0,"publicationDate":"2021-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45635173","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-17DOI: 10.1177/1035719X211040857
J. Olson, Enrique L. P. Mergelsberg, J. Jensen, D. Schofield, N. Watson
People with diabetes face increased risk of serious COVID-19 complications, making self-care for optimal metabolic management crucial. However, the pandemic has reduced access to routine care among people with diabetes. The pandemic can also elicit distress, which can impact diabetes self-management and health. To understand the impact of COVID-19 on Western Australians with diabetes, we conducted an evaluation involving an online survey of consumers of diabetes health services and an analysis of routine program data (i.e. service utilisation/program attendance). Survey respondents were concerned about contracting COVID-19, many intended to change the way they utilised health services and many indicated they would continue to socially isolate. Utilisation of digital/telephone services peaked between April and June 2020. Despite the concerns indicated, a participation resurgence was observed upon resumption of face-to-face programs. Continued access to diabetes programs via multiple modes of delivery is critical to support optimal self-care and mitigate COVID-19 risks, distress and social isolation. This timely and pragmatic assessment of consumer beliefs synthesised with routinely collected evaluation data represents an agile approach to evaluation through an emerging public health crisis. The findings helped to ensure optimal service delivery to meet the needs of this priority population throughout the pandemic.
{"title":"COVID-19 concerns, health services utilisation and social support among Western Australians with diabetes during the pandemic","authors":"J. Olson, Enrique L. P. Mergelsberg, J. Jensen, D. Schofield, N. Watson","doi":"10.1177/1035719X211040857","DOIUrl":"https://doi.org/10.1177/1035719X211040857","url":null,"abstract":"People with diabetes face increased risk of serious COVID-19 complications, making self-care for optimal metabolic management crucial. However, the pandemic has reduced access to routine care among people with diabetes. The pandemic can also elicit distress, which can impact diabetes self-management and health. To understand the impact of COVID-19 on Western Australians with diabetes, we conducted an evaluation involving an online survey of consumers of diabetes health services and an analysis of routine program data (i.e. service utilisation/program attendance). Survey respondents were concerned about contracting COVID-19, many intended to change the way they utilised health services and many indicated they would continue to socially isolate. Utilisation of digital/telephone services peaked between April and June 2020. Despite the concerns indicated, a participation resurgence was observed upon resumption of face-to-face programs. Continued access to diabetes programs via multiple modes of delivery is critical to support optimal self-care and mitigate COVID-19 risks, distress and social isolation. This timely and pragmatic assessment of consumer beliefs synthesised with routinely collected evaluation data represents an agile approach to evaluation through an emerging public health crisis. The findings helped to ensure optimal service delivery to meet the needs of this priority population throughout the pandemic.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"21 1","pages":"206 - 225"},"PeriodicalIF":0.0,"publicationDate":"2021-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41784338","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-01DOI: 10.1177/1035719X211030713
E. Moretti
Evaluation practice takes place in a particularly awkward and challenging social context due to the fear, resistance and anxiety that is often associated with evaluation. Navigating this social context is taxing for evaluators and has the potential to negatively impact their well-being. This article begins with an exploration of the positioning of qualitative and relational approaches within the evaluation field over time, showing that they have been increasingly acknowledged and now widely accepted as crucial to the practice of evaluation. More recent literature is then used to identify six social competencies that are essential to on-the-ground evaluation practice. These competencies are in allaying fear and anxiety, establishing rapport, building and maintaining professional credibility, recognising tacit social dynamics, preventing and managing coercion attempts, and preventing and managing hostility. The article then explores the implications of working in this social context for evaluator well-being. Difficulties around self-assessing competency levels, contending with a poor reputation, emotional labour and self-care, and limited research specific to these matters are discussed. This article posits that practising evaluators should routinely reflect and take active steps to not only improve their social competence but also maintain their own well-being.
{"title":"Navigating the awkward, challenging social context of evaluation","authors":"E. Moretti","doi":"10.1177/1035719X211030713","DOIUrl":"https://doi.org/10.1177/1035719X211030713","url":null,"abstract":"Evaluation practice takes place in a particularly awkward and challenging social context due to the fear, resistance and anxiety that is often associated with evaluation. Navigating this social context is taxing for evaluators and has the potential to negatively impact their well-being. This article begins with an exploration of the positioning of qualitative and relational approaches within the evaluation field over time, showing that they have been increasingly acknowledged and now widely accepted as crucial to the practice of evaluation. More recent literature is then used to identify six social competencies that are essential to on-the-ground evaluation practice. These competencies are in allaying fear and anxiety, establishing rapport, building and maintaining professional credibility, recognising tacit social dynamics, preventing and managing coercion attempts, and preventing and managing hostility. The article then explores the implications of working in this social context for evaluator well-being. Difficulties around self-assessing competency levels, contending with a poor reputation, emotional labour and self-care, and limited research specific to these matters are discussed. This article posits that practising evaluators should routinely reflect and take active steps to not only improve their social competence but also maintain their own well-being.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"21 1","pages":"163 - 176"},"PeriodicalIF":0.0,"publicationDate":"2021-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42394929","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-01DOI: 10.1177/1035719X211026412
Linda Kelly, M. Whiteside, Hayley Barich, K. Tsey
This article reports the process of identifying a well-being monitoring and evaluation approach for a community development programme with Aboriginal Native Title Holders in Northern Australia. The process involved the use of an empowerment-based Aboriginal Family Well-Being framework to enable Native Title Holders to articulate domains of value to their local community. These domains aligned with an existing culturally sensitive Aboriginal well-being survey tool which the Native Title Holders saw as relevant for their use. The attempts to provide Aboriginal people with a broader and more long-term perspective from which to judge the value of short-term projects is a different approach to traditional programme assessment (monitoring and evaluation). It aims to provide Aboriginal people with a more relevant frame from which they can make judgements about the worth of any programme or project in their location, supporting local control and decision-making. Potentially it provides Aboriginal people with the information from which to advocate for other supports and to assess the value of Government and other projects.
{"title":"Checking up to keep on track: An Aboriginal-led approach to monitoring well-being","authors":"Linda Kelly, M. Whiteside, Hayley Barich, K. Tsey","doi":"10.1177/1035719X211026412","DOIUrl":"https://doi.org/10.1177/1035719X211026412","url":null,"abstract":"This article reports the process of identifying a well-being monitoring and evaluation approach for a community development programme with Aboriginal Native Title Holders in Northern Australia. The process involved the use of an empowerment-based Aboriginal Family Well-Being framework to enable Native Title Holders to articulate domains of value to their local community. These domains aligned with an existing culturally sensitive Aboriginal well-being survey tool which the Native Title Holders saw as relevant for their use. The attempts to provide Aboriginal people with a broader and more long-term perspective from which to judge the value of short-term projects is a different approach to traditional programme assessment (monitoring and evaluation). It aims to provide Aboriginal people with a more relevant frame from which they can make judgements about the worth of any programme or project in their location, supporting local control and decision-making. Potentially it provides Aboriginal people with the information from which to advocate for other supports and to assess the value of Government and other projects.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"21 1","pages":"132 - 145"},"PeriodicalIF":0.0,"publicationDate":"2021-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46991745","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-08-16DOI: 10.1177/1035719X211038767
Melissa Forbes
{"title":"Learning to make a difference: Value creation in social learning spaces","authors":"Melissa Forbes","doi":"10.1177/1035719X211038767","DOIUrl":"https://doi.org/10.1177/1035719X211038767","url":null,"abstract":"","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"22 1","pages":"55 - 57"},"PeriodicalIF":0.0,"publicationDate":"2021-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48495903","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-07-22DOI: 10.1177/1035719X211033717
Scott Bayley
All international development policies and programs are fuelled by a complex network of implicit ideas. Stakeholders may hold assumptions about program purposes, theories of change, outcomes, and the value of program evaluation—which may or may not be shared by the evaluators. A major barrier to viable evaluations is that development programs are often based on assumptions that are not well articulated. This lack of clarity masks critical risks to program success and also makes it challenging to evaluate such programs. Most of the evaluation methods that have attempted to address this dilemma have been popularised as ‘theory driven’ approaches. These approaches elaborate the sequence of changes/mini steps that lead to the long-term goal of interest and the connections between program activities and outcomes that occur at each step of the way. Unfortunately, they do not do enough to clarify how program managers or evaluators should deal with tacit assumptions. This book seeks to address this gap by discussing the crucial role that assumptions play in conceptualising, implementing, and evaluating development programs. It aims to offers practical ways for stakeholders and evaluators to 1) examine their assumptions about program theory and environmental conditions and 2) develop and carry out effective program monitoring and evaluation given those assumptions. Unfortunately, in the opinion of this reviewer, this book fails to deliver on its intended purpose.
{"title":"Working with assumptions in international development program evaluation","authors":"Scott Bayley","doi":"10.1177/1035719X211033717","DOIUrl":"https://doi.org/10.1177/1035719X211033717","url":null,"abstract":"All international development policies and programs are fuelled by a complex network of implicit ideas. Stakeholders may hold assumptions about program purposes, theories of change, outcomes, and the value of program evaluation—which may or may not be shared by the evaluators. A major barrier to viable evaluations is that development programs are often based on assumptions that are not well articulated. This lack of clarity masks critical risks to program success and also makes it challenging to evaluate such programs. Most of the evaluation methods that have attempted to address this dilemma have been popularised as ‘theory driven’ approaches. These approaches elaborate the sequence of changes/mini steps that lead to the long-term goal of interest and the connections between program activities and outcomes that occur at each step of the way. Unfortunately, they do not do enough to clarify how program managers or evaluators should deal with tacit assumptions. This book seeks to address this gap by discussing the crucial role that assumptions play in conceptualising, implementing, and evaluating development programs. It aims to offers practical ways for stakeholders and evaluators to 1) examine their assumptions about program theory and environmental conditions and 2) develop and carry out effective program monitoring and evaluation given those assumptions. Unfortunately, in the opinion of this reviewer, this book fails to deliver on its intended purpose.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"21 1","pages":"231 - 233"},"PeriodicalIF":0.0,"publicationDate":"2021-07-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719X211033717","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45827741","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-06-01DOI: 10.1177/1035719X211014398
Bronwyn Rossingh
Welcome to the June 2021 issue of the Evaluation Journal of Australasia (EJA) that brings a strong advocacy feel from the social justice and capacity-building perspectives. This dominant feature in the issue brings to light that social justice and capacity building are quite often the taken-for-granted aspects that do not receive the focus they warrant. Readers of this issue will find that in the papers presented, empowerment evaluation approaches remedy this situation when accountability and internal evaluation unite. Accountability is mapped through an organisation’s mission and vision and enacted by its leadership that then drives and dictates the performance of its functions in the best interests of those it serves. As the papers in this issue reveal, internal evaluation has inherent powers and potential for emancipatory and enabling outcomes that serve as an underlying source of empowerment to those that are impacted by the programmes being delivered. Fetterman (2005) provides relevant and useful support to further introduce the significant evaluative notions and sentiments of the papers in this issue. He refers to 10 guiding principles of empowerment evaluation that encourage and promote internal accountability and contends that these principles are sustainable and prevail even after external evaluators and funders have been and gone. These guiding principles of empowerment evaluation are as follows (Fetterman, 2005, pp. 43–50):
欢迎阅读2021年6月号的《澳大拉西亚评估杂志》(EJA)，该杂志从社会正义和能力建设的角度带来了强烈的倡导感。这一问题的主要特点表明，社会正义和能力建设往往被认为是理所当然的方面，而没有得到应有的关注。本期的读者会发现，在提出的论文中，授权评估方法在问责制和内部评估相结合时纠正了这种情况。问责制是通过组织的使命和愿景来体现的，并由其领导层制定，然后由领导层推动和指示其职能的履行，以实现其服务对象的最大利益。正如本期的论文所揭示的那样，内部评价具有固有的力量和潜力，可以产生解放和有利的结果，作为对受所提供方案影响的人赋权的潜在来源。Fetterman(2005)为进一步介绍本期论文的重要评价理念和观点提供了相关且有益的支持。他提到了授权评估的10项指导原则，这些原则鼓励和促进内部问责制，并认为这些原则是可持续的，即使在外部评估人员和资助者离开之后也会占上风。这些授权评估的指导原则如下(Fetterman, 2005, pp. 43-50):
{"title":"Editorial","authors":"Bronwyn Rossingh","doi":"10.1177/1035719X211014398","DOIUrl":"https://doi.org/10.1177/1035719X211014398","url":null,"abstract":"Welcome to the June 2021 issue of the Evaluation Journal of Australasia (EJA) that brings a strong advocacy feel from the social justice and capacity-building perspectives. This dominant feature in the issue brings to light that social justice and capacity building are quite often the taken-for-granted aspects that do not receive the focus they warrant. Readers of this issue will find that in the papers presented, empowerment evaluation approaches remedy this situation when accountability and internal evaluation unite. Accountability is mapped through an organisation’s mission and vision and enacted by its leadership that then drives and dictates the performance of its functions in the best interests of those it serves. As the papers in this issue reveal, internal evaluation has inherent powers and potential for emancipatory and enabling outcomes that serve as an underlying source of empowerment to those that are impacted by the programmes being delivered. Fetterman (2005) provides relevant and useful support to further introduce the significant evaluative notions and sentiments of the papers in this issue. He refers to 10 guiding principles of empowerment evaluation that encourage and promote internal accountability and contends that these principles are sustainable and prevail even after external evaluators and funders have been and gone. These guiding principles of empowerment evaluation are as follows (Fetterman, 2005, pp. 43–50):","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"21 1","pages":"65 - 68"},"PeriodicalIF":0.0,"publicationDate":"2021-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719X211014398","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43892854","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-06-01DOI: 10.1177/1035719X21998479
T. Mcdiarmid, A. Pineda, Amanda Scothern
Gender equality requires incremental and transformative change which occurs over generations. Strengthening women’s movements through collective action and learning is a key strategy in achieving such change. Capturing those changes in the voices of diverse women is critical to ethical, feminist, participatory evaluation. The purpose of this article is to explore the strengths, benefits and challenges in using Feminist Participatory Action Research (FPAR) to evaluate a women’s leadership programme. This article demonstrates the rationale and context for FPAR providing a brief overview of feminist approaches to evaluation and an outline of the key elements of FPAR that have influenced recent evaluative activities by the International Women’s Development Agency (IWDA). The case study of the Women’s Action for Voice and Empowerment (WAVE) programme is then presented to demonstrate the application of FPAR. Findings indicate the potential of this framework to influence evaluation design to strengthen the capacity of diverse women as co-evaluators; build on and generate knowledge as a resource of and for the women who create, own and share it; and, design evaluative spaces that promote authentic and inclusive forms of evidence. This article discusses insights from the findings, reflections and the lessons learned on using a FPAR framework.
{"title":"We are women! We are ready! Amplifying women’s voices through feminist participatory action research","authors":"T. Mcdiarmid, A. Pineda, Amanda Scothern","doi":"10.1177/1035719X21998479","DOIUrl":"https://doi.org/10.1177/1035719X21998479","url":null,"abstract":"Gender equality requires incremental and transformative change which occurs over generations. Strengthening women’s movements through collective action and learning is a key strategy in achieving such change. Capturing those changes in the voices of diverse women is critical to ethical, feminist, participatory evaluation. The purpose of this article is to explore the strengths, benefits and challenges in using Feminist Participatory Action Research (FPAR) to evaluate a women’s leadership programme. This article demonstrates the rationale and context for FPAR providing a brief overview of feminist approaches to evaluation and an outline of the key elements of FPAR that have influenced recent evaluative activities by the International Women’s Development Agency (IWDA). The case study of the Women’s Action for Voice and Empowerment (WAVE) programme is then presented to demonstrate the application of FPAR. Findings indicate the potential of this framework to influence evaluation design to strengthen the capacity of diverse women as co-evaluators; build on and generate knowledge as a resource of and for the women who create, own and share it; and, design evaluative spaces that promote authentic and inclusive forms of evidence. This article discusses insights from the findings, reflections and the lessons learned on using a FPAR framework.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"21 1","pages":"85 - 100"},"PeriodicalIF":0.0,"publicationDate":"2021-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719X21998479","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46084204","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：