Pub Date : 2020-05-14DOI: 10.1177/1035719X20918497
Aileen Reid
Stakeholder values regarding a programme’s worth and their own principles or standards can aid or impede an evaluation. The evaluator’s challenge and responsibility is to successfully engage multiple stakeholder value orientations in the evaluation process. Stakeholder engagement is essential within evaluations of programmes aimed at broadening participation of underrepresented individuals and institutions in science, technology, engineering and mathematics (STEM). This article describes an educative approach to engage stakeholder values within evaluations of STEM research and education programmes funded by the U.S. National Science Foundation (NSF). Reflections and implications for evaluation theory and practice applicable to any STEM evaluation context, and more broadly to the field of evaluation, are discussed.
{"title":"Applying an educative approach to engage stakeholder values in evaluations of STEM research and education programmes","authors":"Aileen Reid","doi":"10.1177/1035719X20918497","DOIUrl":"https://doi.org/10.1177/1035719X20918497","url":null,"abstract":"Stakeholder values regarding a programme’s worth and their own principles or standards can aid or impede an evaluation. The evaluator’s challenge and responsibility is to successfully engage multiple stakeholder value orientations in the evaluation process. Stakeholder engagement is essential within evaluations of programmes aimed at broadening participation of underrepresented individuals and institutions in science, technology, engineering and mathematics (STEM). This article describes an educative approach to engage stakeholder values within evaluations of STEM research and education programmes funded by the U.S. National Science Foundation (NSF). Reflections and implications for evaluation theory and practice applicable to any STEM evaluation context, and more broadly to the field of evaluation, are discussed.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"20 1","pages":"103 - 108"},"PeriodicalIF":0.0,"publicationDate":"2020-05-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719X20918497","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44158337","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-05-06DOI: 10.1177/1035719X20918370
Caitlin Blaser Mapitsa, Aisha J. Ali, Linda S. Khumalo
Monitoring and Evaluation discourse in Africa has evolved to focus on building systems at a national level. While this systemic approach has many advantages, its implementation often runs up against the uncomfortable reality that governments have complex incentives to use evidence, and this evidence can equally contribute to decision making that is neither development-focused nor democratic if values are not part of the conversation. Much of the literature on public-sector reform focuses on evidence-based policy making. While relevant, it does not reflect on values, and this article will argue that acknowledging the central role values play in interpreting evidence is critical to effective national evaluation system building. To make this argument, this article will present and discuss vignettes from the parliaments of Malawi, Tanzania, Uganda, Zambia and Zimbabwe that illustrate the pivotal role values have played in interpreting and acting on evidence in a parliamentary context. Finally, it makes a case for the discourse about evidence-based policy making to consider values-based policy making as an appropriate lens for parliaments to acknowledge and engage with the complex landscape of the politics of evidence.
{"title":"From evidence to values-based decision making in African parliaments","authors":"Caitlin Blaser Mapitsa, Aisha J. Ali, Linda S. Khumalo","doi":"10.1177/1035719X20918370","DOIUrl":"https://doi.org/10.1177/1035719X20918370","url":null,"abstract":"Monitoring and Evaluation discourse in Africa has evolved to focus on building systems at a national level. While this systemic approach has many advantages, its implementation often runs up against the uncomfortable reality that governments have complex incentives to use evidence, and this evidence can equally contribute to decision making that is neither development-focused nor democratic if values are not part of the conversation. Much of the literature on public-sector reform focuses on evidence-based policy making. While relevant, it does not reflect on values, and this article will argue that acknowledging the central role values play in interpreting evidence is critical to effective national evaluation system building. To make this argument, this article will present and discuss vignettes from the parliaments of Malawi, Tanzania, Uganda, Zambia and Zimbabwe that illustrate the pivotal role values have played in interpreting and acting on evidence in a parliamentary context. Finally, it makes a case for the discourse about evidence-based policy making to consider values-based policy making as an appropriate lens for parliaments to acknowledge and engage with the complex landscape of the politics of evidence.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"20 1","pages":"68 - 85"},"PeriodicalIF":0.0,"publicationDate":"2020-05-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719X20918370","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49651664","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-05-04DOI: 10.1177/1035719X20915805
K. Hassall
In this reading guide, I suggest a range of books for evaluators seeking to learn about values and incorporate values in evaluation. Some are practical, others are philosophical and theoretical, and some blend theory and practice. Learning about values in evaluation is not like learning a new technique or method. It requires an understanding of what values are, how they are enacted in society, and how to think critically about values at all points through an evaluation. Values, and their proper role in social science, have been debated for more than a century. Although many evaluators may reject the idea of value-free social science, ambivalence about values is embedded in social science practice. Learning about values, and working with values in evaluation, may first require unlearning older ideas about values and rethinking evaluation theories and practices informed by a contemporary understanding of values and their role in evaluative judgement. First in the list are books that explain the history of values in social science and provide new ways of thinking about values. Within the evaluation literature, there are books rethinking the role of values in evaluation, evaluation approaches guided by values, and practical guides to evaluation that include explicit discussion of values. Outside the evaluation field, there is practical and theoretical literature relevant to our challenges in evaluation. Evaluators can learn about methods for discussing and negotiating values from the literature on deliberative practice. The values-based practice movement in health and social care has resources that are relevant to evaluators, not just in health care contexts. In public administration and public policy, there is a growing literature on public values, seeking to understand the values that represent the broad public welfare in a democracy.
{"title":"Exploring values in evaluation: A guide to reading","authors":"K. Hassall","doi":"10.1177/1035719X20915805","DOIUrl":"https://doi.org/10.1177/1035719X20915805","url":null,"abstract":"In this reading guide, I suggest a range of books for evaluators seeking to learn about values and incorporate values in evaluation. Some are practical, others are philosophical and theoretical, and some blend theory and practice. Learning about values in evaluation is not like learning a new technique or method. It requires an understanding of what values are, how they are enacted in society, and how to think critically about values at all points through an evaluation. Values, and their proper role in social science, have been debated for more than a century. Although many evaluators may reject the idea of value-free social science, ambivalence about values is embedded in social science practice. Learning about values, and working with values in evaluation, may first require unlearning older ideas about values and rethinking evaluation theories and practices informed by a contemporary understanding of values and their role in evaluative judgement. First in the list are books that explain the history of values in social science and provide new ways of thinking about values. Within the evaluation literature, there are books rethinking the role of values in evaluation, evaluation approaches guided by values, and practical guides to evaluation that include explicit discussion of values. Outside the evaluation field, there is practical and theoretical literature relevant to our challenges in evaluation. Evaluators can learn about methods for discussing and negotiating values from the literature on deliberative practice. The values-based practice movement in health and social care has resources that are relevant to evaluators, not just in health care contexts. In public administration and public policy, there is a growing literature on public values, seeking to understand the values that represent the broad public welfare in a democracy.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"20 1","pages":"109 - 115"},"PeriodicalIF":0.0,"publicationDate":"2020-05-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719X20915805","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47107283","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-03-01DOI: 10.1177/1035719X20905056
J. Guenther, T. Benveniste, M. Redman-Maclaren, David J. Mander, Janya McCalman, Marnie O’Bryan, Sam Osborne, Richard M. Stewart
Many recent policy documents have outlined the challenges of delivering high-quality education in remote First Nations communities and proposed that boarding schools are one important solution. These documents have influenced the increasing uptake of boarding options and there has been considerable public investment in scholarships, residential facilities and transition support. Yet the outcomes of this investment and policy effort are not well understood. The authors of this article came together as a collaboration of researchers who have published about boarding school education for First Nations students to examine the evidence and develop a theory-driven understanding of how policies drive systems to produce both desirable and undesirable outcomes for First Nations boarding school students. We applied complexity theory and post-structural policy analysis techniques and produced a useful tool for the evaluation of boarding policy and its implementation.
{"title":"Thinking with theory as a policy evaluation tool: The case of boarding schools for remote First Nations students","authors":"J. Guenther, T. Benveniste, M. Redman-Maclaren, David J. Mander, Janya McCalman, Marnie O’Bryan, Sam Osborne, Richard M. Stewart","doi":"10.1177/1035719X20905056","DOIUrl":"https://doi.org/10.1177/1035719X20905056","url":null,"abstract":"Many recent policy documents have outlined the challenges of delivering high-quality education in remote First Nations communities and proposed that boarding schools are one important solution. These documents have influenced the increasing uptake of boarding options and there has been considerable public investment in scholarships, residential facilities and transition support. Yet the outcomes of this investment and policy effort are not well understood. The authors of this article came together as a collaboration of researchers who have published about boarding school education for First Nations students to examine the evidence and develop a theory-driven understanding of how policies drive systems to produce both desirable and undesirable outcomes for First Nations boarding school students. We applied complexity theory and post-structural policy analysis techniques and produced a useful tool for the evaluation of boarding policy and its implementation.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"20 1","pages":"34 - 52"},"PeriodicalIF":0.0,"publicationDate":"2020-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719X20905056","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47372701","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-03-01DOI: 10.1177/1035719X20911332
Amy Lawton, O. Hamilton, C. Jackson
The Aboriginal Family Planning Circle (AFPC) programme is an Aboriginal-led community programme, which works with Aboriginal families in Western Sydney to address their complex needs and reduce the risk of having their children put into out-of-home care (OOHC). This article explores two external evaluations undertaken by WESTIR Limited (WESTIR), a non-Aboriginal research service, on the AFPC programme. The purpose of the first evaluation was to provide an assessment of how effective the AFPC programme had been for participants and identify programme aspects that could be improved or developed. The second evaluation was undertaken to examine whether the AFPC programme had continued to meet its objectives and address some knowledge gaps, particularly the estimated savings and return on investment that the programme created for the OOHC system in New South Wales (NSW). The evaluations used qualitative and quantitative techniques, including interviews, focus groups and a return on investment analysis. This article outlines methods, results and recommendations from both evaluations, along with lessons learned to better inform evaluation practice. This case study shows that culturally responsive evaluations can provide an avenue for Aboriginal communities to advocate for the continued funding of their programmes. It also emphasises the need to adequately resource Aboriginal programme evaluations in the community services sector now and in the future.
{"title":"Aboriginal Family Planning Circle evaluation: Empowering Aboriginal communities in evaluating and future-proofing Aboriginal-led community programmes","authors":"Amy Lawton, O. Hamilton, C. Jackson","doi":"10.1177/1035719X20911332","DOIUrl":"https://doi.org/10.1177/1035719X20911332","url":null,"abstract":"The Aboriginal Family Planning Circle (AFPC) programme is an Aboriginal-led community programme, which works with Aboriginal families in Western Sydney to address their complex needs and reduce the risk of having their children put into out-of-home care (OOHC). This article explores two external evaluations undertaken by WESTIR Limited (WESTIR), a non-Aboriginal research service, on the AFPC programme. The purpose of the first evaluation was to provide an assessment of how effective the AFPC programme had been for participants and identify programme aspects that could be improved or developed. The second evaluation was undertaken to examine whether the AFPC programme had continued to meet its objectives and address some knowledge gaps, particularly the estimated savings and return on investment that the programme created for the OOHC system in New South Wales (NSW). The evaluations used qualitative and quantitative techniques, including interviews, focus groups and a return on investment analysis. This article outlines methods, results and recommendations from both evaluations, along with lessons learned to better inform evaluation practice. This case study shows that culturally responsive evaluations can provide an avenue for Aboriginal communities to advocate for the continued funding of their programmes. It also emphasises the need to adequately resource Aboriginal programme evaluations in the community services sector now and in the future.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"20 1","pages":"23 - 33"},"PeriodicalIF":0.0,"publicationDate":"2020-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719X20911332","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46443086","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-03-01Epub Date: 2020-04-02DOI: 10.1177/1035719x20909910
Ralph Renger, Marc D Basson, Gary Hart, Rick Van Eck, Eric Souvannasacd, Jessica Renger, Jirina Foltysova
This article shares lessons learned while evaluating the implementation of a Clinical and Translational Research Centre (CTR). To meet its overarching goals, the CTR consists of numerous research support units (e.g., biostatistics, community engagement, professional development) that are intended to work together collaboratively. It is then argued that an evaluation approach grounded in system thinking was the best fit to evaluate this key CTR design feature. The rationale for selecting systems evaluation theory (SET) as the evaluation framework best suited to evaluate the CTR infrastructure is then presented. The application of SET and the lessons learned are then shared. This article concludes that there are many similarly structured programmes worldwide to which the lessons learned can be applied and upfront investments in using a system approach are rewarded by providing meaningful and useful evaluation recommendations for system change.
{"title":"Lessons Learned in Evaluating the Infrastructure of a Centre for Translational Research (CTR).","authors":"Ralph Renger, Marc D Basson, Gary Hart, Rick Van Eck, Eric Souvannasacd, Jessica Renger, Jirina Foltysova","doi":"10.1177/1035719x20909910","DOIUrl":"https://doi.org/10.1177/1035719x20909910","url":null,"abstract":"This article shares lessons learned while evaluating the implementation of a Clinical and Translational Research Centre (CTR). To meet its overarching goals, the CTR consists of numerous research support units (e.g., biostatistics, community engagement, professional development) that are intended to work together collaboratively. It is then argued that an evaluation approach grounded in system thinking was the best fit to evaluate this key CTR design feature. The rationale for selecting systems evaluation theory (SET) as the evaluation framework best suited to evaluate the CTR infrastructure is then presented. The application of SET and the lessons learned are then shared. This article concludes that there are many similarly structured programmes worldwide to which the lessons learned can be applied and upfront investments in using a system approach are rewarded by providing meaningful and useful evaluation recommendations for system change.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"20 1","pages":"6-22"},"PeriodicalIF":0.0,"publicationDate":"2020-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719x20909910","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"39100819","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-03-01DOI: 10.1177/1035719x20911502
P. Hawkins
Penny Hawkins is an evaluation specialist with extensive experience in international development evaluation and public policy across a wide range of sectors and organisations. Penny is the former Head of Evaluation, UK Department for International Development (DFID). Over the past two decades, she has held evaluation leadership and management roles in the government and philanthropic sectors, including at The Rockefeller Foundation and as Head of Evaluation for the Ministry of Foreign Affairs and Trade (NZAID) and Social Development, New Zealand. Penny has also held international evaluation leadership roles, including as Chair of the Organisation for Economic Co-operation and Development–Development Assistance Committee (OECD-DAC) Network on Development Evaluation (2013–2016) and as a former President and current Fellow of the Australasian Evaluation Society (AES).
{"title":"Evaluator perspective","authors":"P. Hawkins","doi":"10.1177/1035719x20911502","DOIUrl":"https://doi.org/10.1177/1035719x20911502","url":null,"abstract":"Penny Hawkins is an evaluation specialist with extensive experience in international development evaluation and public policy across a wide range of sectors and organisations. Penny is the former Head of Evaluation, UK Department for International Development (DFID). Over the past two decades, she has held evaluation leadership and management roles in the government and philanthropic sectors, including at The Rockefeller Foundation and as Head of Evaluation for the Ministry of Foreign Affairs and Trade (NZAID) and Social Development, New Zealand. Penny has also held international evaluation leadership roles, including as Chair of the Organisation for Economic Co-operation and Development–Development Assistance Committee (OECD-DAC) Network on Development Evaluation (2013–2016) and as a former President and current Fellow of the Australasian Evaluation Society (AES).","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"20 1","pages":"53 - 56"},"PeriodicalIF":0.0,"publicationDate":"2020-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719x20911502","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44214556","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-12-01DOI: 10.1177/1035719X19895150
Liz Gould
Writing in 2001 in the American Journal of Evaluation, Ernest House suggested a key piece of ‘unfinished business’ in evaluation should be reconfigured – the fact/value dichotomy (House, 2001). This special issue of the Evaluation Journal of Australasia revisits aspects of this ‘unfinished business’, centring on this and other matters of ‘values’ further investigated by various theorists (Gates, 2018; Hall et al., 2012; Henry, 2002; House & Howe, 1999; Renger & Bourdeau, 2004). Authors in this special issue tackle difficult and intricate philosophical challenges relating to ‘valuing’ and ‘values’ in evaluation, as well as implications for practising evaluators. This issue does not lay out a singular definition of ‘values’ in evaluation. As others have noted, the concept of values has been explored in various disciplines – economics, philosophy, psychology, sociology and biology – ranging from concepts of ‘value’ as a product that is exchanged (e.g., for money); as functional or having utility; as culturally defined; deriving from scarcity or competition; an aesthetic which is relative or subjective; deriving from ethical choices; and so on. Indeed, there may be some utility in aspects of these approaches for evaluators, as posited by authors in this issue. So why does reflecting on ‘values’ matter for evaluators? How might we understand ‘values’ in appraising aspects of health, education or other social programmes, for example? And how might we understand ‘values’ in subjective concepts such as ‘wellbeing’? The significance of values to practitioners is a key area of investigation in this special issue. Authors were invited to present differing applications of ‘values’ in their own evaluative thinking and practice. Readers and evaluators are similarly encouraged to reflect on how values shape your thinking about evaluation and what values shape your evaluative practice. Judgements about value may assign ‘importance’: in what we focus on or exclude, through selection decisions (or biases), prioritisation of issues, privileging of some perspectives over others, and measurement choices. Values influence practice – from the evaluative questions we ask, the theories of change we consider, the objectives we aim for, the programme logics we develop, the methods we choose, the practices we employ, the stakeholders we identify, the views we gather, the findings we deem credible or significant and so on. The inspiration for this special issue came out of the Australian Evaluation Society’s 2018 conference and ensuing discussions, where the issue guest editors – Keryn, Mathea, Kelly and Amy – presented on topics relating to ‘values’. A special issue of 895150 EVJ0010.1177/1035719X19895150Evaluation Journal of Australasia X(X)Gould editorial2019
{"title":"Foreword: On ‘values’","authors":"Liz Gould","doi":"10.1177/1035719X19895150","DOIUrl":"https://doi.org/10.1177/1035719X19895150","url":null,"abstract":"Writing in 2001 in the American Journal of Evaluation, Ernest House suggested a key piece of ‘unfinished business’ in evaluation should be reconfigured – the fact/value dichotomy (House, 2001). This special issue of the Evaluation Journal of Australasia revisits aspects of this ‘unfinished business’, centring on this and other matters of ‘values’ further investigated by various theorists (Gates, 2018; Hall et al., 2012; Henry, 2002; House & Howe, 1999; Renger & Bourdeau, 2004). Authors in this special issue tackle difficult and intricate philosophical challenges relating to ‘valuing’ and ‘values’ in evaluation, as well as implications for practising evaluators. This issue does not lay out a singular definition of ‘values’ in evaluation. As others have noted, the concept of values has been explored in various disciplines – economics, philosophy, psychology, sociology and biology – ranging from concepts of ‘value’ as a product that is exchanged (e.g., for money); as functional or having utility; as culturally defined; deriving from scarcity or competition; an aesthetic which is relative or subjective; deriving from ethical choices; and so on. Indeed, there may be some utility in aspects of these approaches for evaluators, as posited by authors in this issue. So why does reflecting on ‘values’ matter for evaluators? How might we understand ‘values’ in appraising aspects of health, education or other social programmes, for example? And how might we understand ‘values’ in subjective concepts such as ‘wellbeing’? The significance of values to practitioners is a key area of investigation in this special issue. Authors were invited to present differing applications of ‘values’ in their own evaluative thinking and practice. Readers and evaluators are similarly encouraged to reflect on how values shape your thinking about evaluation and what values shape your evaluative practice. Judgements about value may assign ‘importance’: in what we focus on or exclude, through selection decisions (or biases), prioritisation of issues, privileging of some perspectives over others, and measurement choices. Values influence practice – from the evaluative questions we ask, the theories of change we consider, the objectives we aim for, the programme logics we develop, the methods we choose, the practices we employ, the stakeholders we identify, the views we gather, the findings we deem credible or significant and so on. The inspiration for this special issue came out of the Australian Evaluation Society’s 2018 conference and ensuing discussions, where the issue guest editors – Keryn, Mathea, Kelly and Amy – presented on topics relating to ‘values’. A special issue of 895150 EVJ0010.1177/1035719X19895150Evaluation Journal of Australasia X(X)Gould editorial2019","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"19 1","pages":"157 - 158"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1035719X19895150","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46602594","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：