Pub Date : 2020-08-04DOI: 10.1177/1035719X20944020
Kylie Evans-Locke, Ching-I Hsu
The child’s voice is often excluded in child and family service programme evaluations, with research traditionally favouring the views of parents, carers and other adults. Evaluations that do not provide adequate opportunities for children and young people (CYP) to be involved potentially risk misunderstanding wider programme impacts and minimising CYP-specific outcomes. CareSouth has piloted an evaluation utilising body-mapping to collect programme feedback from CYP to provide valuable insights into perceived outcomes in areas of social connection, cognitive skills and development of positive personal identity. This article will share insights and lessons CareSouth encountered during the evaluation, the methods used to ensure ethical evaluation and the lessons for other child and family service practitioners looking to extend meaningful consultation with CYP to ensure greater understanding of holistic programme impacts.
{"title":"Using participatory methods to evaluate the impacts of an early intervention programme on children and young people (CYP)","authors":"Kylie Evans-Locke, Ching-I Hsu","doi":"10.1177/1035719X20944020","DOIUrl":"https://doi.org/10.1177/1035719X20944020","url":null,"abstract":"The child’s voice is often excluded in child and family service programme evaluations, with research traditionally favouring the views of parents, carers and other adults. Evaluations that do not provide adequate opportunities for children and young people (CYP) to be involved potentially risk misunderstanding wider programme impacts and minimising CYP-specific outcomes. CareSouth has piloted an evaluation utilising body-mapping to collect programme feedback from CYP to provide valuable insights into perceived outcomes in areas of social connection, cognitive skills and development of positive personal identity. This article will share insights and lessons CareSouth encountered during the evaluation, the methods used to ensure ethical evaluation and the lessons for other child and family service practitioners looking to extend meaningful consultation with CYP to ensure greater understanding of holistic programme impacts.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"20 1","pages":"176 - 188"},"PeriodicalIF":0.0,"publicationDate":"2020-08-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719X20944020","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46001999","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-07-07DOI: 10.1177/1035719X20921562
R. Pitts
In this reflective analysis, I describe the conditions that led the leaders of a multisite initiative to adapt their program model towards a framing that centralized responsiveness as an organizing...
{"title":"Finding common ground: Centralizing responsiveness within a multisite initiative:","authors":"R. Pitts","doi":"10.1177/1035719X20921562","DOIUrl":"https://doi.org/10.1177/1035719X20921562","url":null,"abstract":"In this reflective analysis, I describe the conditions that led the leaders of a multisite initiative to adapt their program model towards a framing that centralized responsiveness as an organizing...","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"20 1","pages":"95-102"},"PeriodicalIF":0.0,"publicationDate":"2020-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719X20921562","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"65871551","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-06-29DOI: 10.1177/1035719X20927146
Andrew Hawkins, Jasper Odgers, A. Reeves, Alicia McCoy
Mental health counselling and support over the telephone or internet is increasingly common. Evaluating effectiveness requires outcome measures and understanding factors driving behaviour. This article describes a theory-driven evaluation of the one-month outcomes of a short-term solution-focused support session for anxiety or depression with a counsellor. The primary aim of the evaluation was to measure the outcomes of this session on service users’ help-seeking behaviour. It also sought to understand reasons for behaviour based on behaviour change theory. A secondary aim was to measure changes in feelings of stress and coping before and after the session, and collect evidence of the value of the service in terms of ‘consumer-defined recovery’. The evaluation found the service was effective, with the overwhelming majority taking some action, being more engaged with a health professional, having reduced feelings of distress, increased confidence to cope and less hopelessness. Improvements for service users included ‘reality testing’ the advice given and building commitment or intent to follow the advice, and ‘rehearsing’ so service users can demonstrate to themselves they have the skills required and can overcome any obstacles to following the advice.
{"title":"Evaluating telephone and online psychological support and referral","authors":"Andrew Hawkins, Jasper Odgers, A. Reeves, Alicia McCoy","doi":"10.1177/1035719X20927146","DOIUrl":"https://doi.org/10.1177/1035719X20927146","url":null,"abstract":"Mental health counselling and support over the telephone or internet is increasingly common. Evaluating effectiveness requires outcome measures and understanding factors driving behaviour. This article describes a theory-driven evaluation of the one-month outcomes of a short-term solution-focused support session for anxiety or depression with a counsellor. The primary aim of the evaluation was to measure the outcomes of this session on service users’ help-seeking behaviour. It also sought to understand reasons for behaviour based on behaviour change theory. A secondary aim was to measure changes in feelings of stress and coping before and after the session, and collect evidence of the value of the service in terms of ‘consumer-defined recovery’. The evaluation found the service was effective, with the overwhelming majority taking some action, being more engaged with a health professional, having reduced feelings of distress, increased confidence to cope and less hopelessness. Improvements for service users included ‘reality testing’ the advice given and building commitment or intent to follow the advice, and ‘rehearsing’ so service users can demonstrate to themselves they have the skills required and can overcome any obstacles to following the advice.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"20 1","pages":"157 - 175"},"PeriodicalIF":0.0,"publicationDate":"2020-06-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719X20927146","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43969820","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-06-19DOI: 10.1177/1035719x20927151
Joanne Cummings
{"title":"Book Review: Little Quick Fix: Write a questionnaire; Little Quick Fix: Do your interviews","authors":"Joanne Cummings","doi":"10.1177/1035719x20927151","DOIUrl":"https://doi.org/10.1177/1035719x20927151","url":null,"abstract":"","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"20 1","pages":"189 - 190"},"PeriodicalIF":0.0,"publicationDate":"2020-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719x20927151","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45632074","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-06-12DOI: 10.1177/1035719X20928164
G. VanLandingham
Evaluators have long sought a world in which our work makes a tangible difference to society, but that goal has often seemed out of reach. However, in recent years, advocates have proclaimed an era of evidence-based policymaking in which the What Works data generated by evaluations will be increasingly used to inform programme and policy choices. Four primary factors have been critical to the rise of this approach – attaining a critical mass of curated What Works’ evidence, growing interest among political leaders in considering this information when making choices, new budgetary mechanisms for using these data and new tools that facilitate rigorous outcome studies. However, the movement also faces critical challenges including the growing distrust of empirical data among some political factions, leaks in the evaluation pipeline that generates data to identify What Works and the replication failure of many evidence-based interventions. The evaluation field should support this movement through efforts to plug leaks in the evidence pipeline, stronger efforts to assess implementation challenges, training students in evidence-based approaches and assisting in outreach to policymakers.
{"title":"Evaluation in the age of evidence-based policymaking: Promises, pitfalls and paths forward","authors":"G. VanLandingham","doi":"10.1177/1035719X20928164","DOIUrl":"https://doi.org/10.1177/1035719X20928164","url":null,"abstract":"Evaluators have long sought a world in which our work makes a tangible difference to society, but that goal has often seemed out of reach. However, in recent years, advocates have proclaimed an era of evidence-based policymaking in which the What Works data generated by evaluations will be increasingly used to inform programme and policy choices. Four primary factors have been critical to the rise of this approach – attaining a critical mass of curated What Works’ evidence, growing interest among political leaders in considering this information when making choices, new budgetary mechanisms for using these data and new tools that facilitate rigorous outcome studies. However, the movement also faces critical challenges including the growing distrust of empirical data among some political factions, leaks in the evaluation pipeline that generates data to identify What Works and the replication failure of many evidence-based interventions. The evaluation field should support this movement through efforts to plug leaks in the evidence pipeline, stronger efforts to assess implementation challenges, training students in evidence-based approaches and assisting in outreach to policymakers.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"20 1","pages":"129 - 139"},"PeriodicalIF":0.0,"publicationDate":"2020-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719X20928164","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46928758","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-06-01DOI: 10.1177/1035719x20932697
Liz Gould
While writing the Editorial foreword for our December special issue on ‘values’, I discovered a feature article by Ernest House in 1996, published in this journal, on this very topic. While my December foreword referenced a later work by House on values published in the American Journal of Evaluation (House, 2001), I could not resist digging up this earlier piece from the EJA archives, written five years prior, to round out this second and final special issue on ‘values’ in evaluation. In the 1996 feature entitled ‘The Problem of Values in Evaluation’, House grapples with the practitioner’s task of making evaluative judgements to determine the oft-cited ‘merit, worth, or value of something’ (see Scriven, 1980, 1991). “How does one arrive at evaluative judgements legitimately, noting that the evaluator’s task is not an easy one . . . ? ” he asks, acknowledging that “many evaluative judgements are contestable by their nature” (p6). While there are professional techniques which help with collecting, interpreting, and weighing evidence, there are other relative judgements. House suggests that distinguishing the major audiences and stakeholders for an evaluation also helps with deciding evaluative criteria to employ (House, 1996, p8). In doing so, the evaluator must also consider conflicting interests. He acknowledges that evaluation does not eliminate conflict, rather, the evaluator produces the best judgement to be arrived at in the situation, given conflicts (House, 1996, p12). Other theorists – intentionally or otherwise – have adopted, explored and challenged aspects of this thinking. Nonetheless, issues that House raises remain crucial considerations in evaluation practice: determining the criteria for making evaluative judgements, distinguishing the values and preferences of audiences and stakeholders, and managing conflicting values. More than two decades on, what might newer ‘values’-thinking add to what is already known? How might contemporary debates and movements in the evaluation field (too numerous to capture here) benefit or redouble? On these matters, there is more work to be done. If – as a number of authors in this June issue of the EJA suggest – we are to be more explicit about values in evaluation, what does this look like? To cherry-pick a few examples within this issue: Blaser Mapitsa et al’s study, drawing on examples from parliaments in Southern and Eastern Africa, suggests that values both influence the 932697 EVJ0010.1177/1035719X20932697Evaluation Journal of AustralasiaEditorial editorial2020
{"title":"Editorial Foreword - revisiting ‘values’ in evaluation in times of crisis","authors":"Liz Gould","doi":"10.1177/1035719x20932697","DOIUrl":"https://doi.org/10.1177/1035719x20932697","url":null,"abstract":"While writing the Editorial foreword for our December special issue on ‘values’, I discovered a feature article by Ernest House in 1996, published in this journal, on this very topic. While my December foreword referenced a later work by House on values published in the American Journal of Evaluation (House, 2001), I could not resist digging up this earlier piece from the EJA archives, written five years prior, to round out this second and final special issue on ‘values’ in evaluation. In the 1996 feature entitled ‘The Problem of Values in Evaluation’, House grapples with the practitioner’s task of making evaluative judgements to determine the oft-cited ‘merit, worth, or value of something’ (see Scriven, 1980, 1991). “How does one arrive at evaluative judgements legitimately, noting that the evaluator’s task is not an easy one . . . ? ” he asks, acknowledging that “many evaluative judgements are contestable by their nature” (p6). While there are professional techniques which help with collecting, interpreting, and weighing evidence, there are other relative judgements. House suggests that distinguishing the major audiences and stakeholders for an evaluation also helps with deciding evaluative criteria to employ (House, 1996, p8). In doing so, the evaluator must also consider conflicting interests. He acknowledges that evaluation does not eliminate conflict, rather, the evaluator produces the best judgement to be arrived at in the situation, given conflicts (House, 1996, p12). Other theorists – intentionally or otherwise – have adopted, explored and challenged aspects of this thinking. Nonetheless, issues that House raises remain crucial considerations in evaluation practice: determining the criteria for making evaluative judgements, distinguishing the values and preferences of audiences and stakeholders, and managing conflicting values. More than two decades on, what might newer ‘values’-thinking add to what is already known? How might contemporary debates and movements in the evaluation field (too numerous to capture here) benefit or redouble? On these matters, there is more work to be done. If – as a number of authors in this June issue of the EJA suggest – we are to be more explicit about values in evaluation, what does this look like? To cherry-pick a few examples within this issue: Blaser Mapitsa et al’s study, drawing on examples from parliaments in Southern and Eastern Africa, suggests that values both influence the 932697 EVJ0010.1177/1035719X20932697Evaluation Journal of AustralasiaEditorial editorial2020","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"20 1","pages":"61 - 62"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719x20932697","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41632315","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-06-01DOI: 10.1177/1035719X20927145
D. Week
In an international development assistance context, in which representatives of higher income countries and global institutions meet with people of very different cultures, values can conflict. My experience working on a Community Based Building Program in the East Sepik Province of Papua New Guinea in the 1980s introduced me to the thinking styles, beliefs and values of another culture with traditions very different from my own. Evaluators experience this conflict when they work alongside members of another culture, while at the same time adhering to established methods and values from their own culture or professional practice. To help with this, evaluators can identify the benefits in local knowledge, and assist all parties to build an awareness of both explicit and tacit values employed – including those of the evaluators themselves – while in the process of undertaking an evaluation.
{"title":"Values in conflict in cross-cultural architectural encounters","authors":"D. Week","doi":"10.1177/1035719X20927145","DOIUrl":"https://doi.org/10.1177/1035719X20927145","url":null,"abstract":"In an international development assistance context, in which representatives of higher income countries and global institutions meet with people of very different cultures, values can conflict. My experience working on a Community Based Building Program in the East Sepik Province of Papua New Guinea in the 1980s introduced me to the thinking styles, beliefs and values of another culture with traditions very different from my own. Evaluators experience this conflict when they work alongside members of another culture, while at the same time adhering to established methods and values from their own culture or professional practice. To help with this, evaluators can identify the benefits in local knowledge, and assist all parties to build an awareness of both explicit and tacit values employed – including those of the evaluators themselves – while in the process of undertaking an evaluation.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"20 1","pages":"89 - 94"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719X20927145","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46592753","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-06-01DOI: 10.1177/1035719X20931250
K. Hassall, A. Gullickson, A. Boyce, Kelly M Hannum
“So what harm would be done if evaluators ignored values? In one sense, no harm would be done, because evaluations would still have values implicit in them. Values inevitably permeate the selection of independent and dependent variables, the choice of questions and stakeholders, and the social and political context from which many evaluations arise. Evaluators cannot avoid values even if they try. But in another sense, real harm is done if evaluators deal with values naively or poorly through their implicit choices.” Shadish, 1994, p. 35
{"title":"Editorial","authors":"K. Hassall, A. Gullickson, A. Boyce, Kelly M Hannum","doi":"10.1177/1035719X20931250","DOIUrl":"https://doi.org/10.1177/1035719X20931250","url":null,"abstract":"“So what harm would be done if evaluators ignored values? In one sense, no harm would be done, because evaluations would still have values implicit in them. Values inevitably permeate the selection of independent and dependent variables, the choice of questions and stakeholders, and the social and political context from which many evaluations arise. Evaluators cannot avoid values even if they try. But in another sense, real harm is done if evaluators deal with values naively or poorly through their implicit choices.” Shadish, 1994, p. 35","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"20 1","pages":"63 - 67"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719X20931250","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46627450","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-06-01DOI: 10.1177/1035719X20929014
A. Boyce
The Evaluation Journal of Australasia special issue on Values in Evaluation published in December 2019 gave an overview of current discourses associated with valuing in evaluation with the debut of ‘Praxis’ papers. This introduction summarizes the three praxis papers in this second Values in Evaluation special issue. The praxis papers contribute to knowledge about the ways in which we can and should engage with values across a variety of evaluation contexts with varying stakeholders.
2019年12月出版的《澳大拉西亚评估杂志》（Evaluation Journal of Australasia）关于评估价值观的特刊概述了当前与评估价值观相关的话语，并首次发表了“Praxis”论文。引言部分对第二期《评估价值》特刊中的三篇实践性论文进行了总结。实践论文有助于了解我们可以和应该在不同利益相关者的各种评估环境中参与价值观的方式。
{"title":"Introduction to values in evaluation praxis section","authors":"A. Boyce","doi":"10.1177/1035719X20929014","DOIUrl":"https://doi.org/10.1177/1035719X20929014","url":null,"abstract":"The Evaluation Journal of Australasia special issue on Values in Evaluation published in December 2019 gave an overview of current discourses associated with valuing in evaluation with the debut of ‘Praxis’ papers. This introduction summarizes the three praxis papers in this second Values in Evaluation special issue. The praxis papers contribute to knowledge about the ways in which we can and should engage with values across a variety of evaluation contexts with varying stakeholders.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"20 1","pages":"86 - 88"},"PeriodicalIF":0.0,"publicationDate":"2020-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719X20929014","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49118274","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：