D. Felt, Esrea Pérez-Bill, Megan M. Ruprecht, Michael Petillo, L. Beach, Erik Elías Glenn, Gregory Phillips
LGBTQ+ stories and histories have long been silenced as part of deliberate work by those in power to erase our identities and experiences. As evaluators, we contribute to the process of either silencing or uplifting LGBTQ+ stories. This aspect of our work begs a number of vital questions that each of us must reckon with when we approach an evaluation: What data are necessary to allow us to tell a story? What story will we tell with the data we have collected? And, most importantly, who does the telling of certain stories benefit, who might it harm, and what is our responsibility as evaluators to protect peoples’ stories? Proceeding from these questions, this chapter has three distinct parts. In Part One, we establish a common language. By integrating perspectives from the social sciences and LGBTQ+ community scholarship, we provide an overview of the complex and contextually specific nature of sex, sexual orientation, and gender, and discuss the implications of these complexities on how we approach collecting LGBTQ+ data. In Part Two, we consider the power of the stories we tell to impact the lives of LGBTQ+ people, and the frameworks, theories, and ethical imperatives which may help us to contribute to a narrative of LGBTQ+ liberation through our work. Finally, in Part Three, we offer an example tool for readers to use as they consider how they would approach this work in their own practices.
{"title":"Becoming an LGBTQ+ storyteller: Collecting and using data on gender, sex, and sexual orientation","authors":"D. Felt, Esrea Pérez-Bill, Megan M. Ruprecht, Michael Petillo, L. Beach, Erik Elías Glenn, Gregory Phillips","doi":"10.1002/ev.20518","DOIUrl":"https://doi.org/10.1002/ev.20518","url":null,"abstract":"LGBTQ+ stories and histories have long been silenced as part of deliberate work by those in power to erase our identities and experiences. As evaluators, we contribute to the process of either silencing or uplifting LGBTQ+ stories. This aspect of our work begs a number of vital questions that each of us must reckon with when we approach an evaluation: What data are necessary to allow us to tell a story? What story will we tell with the data we have collected? And, most importantly, who does the telling of certain stories benefit, who might it harm, and what is our responsibility as evaluators to protect peoples’ stories? Proceeding from these questions, this chapter has three distinct parts. In Part One, we establish a common language. By integrating perspectives from the social sciences and LGBTQ+ community scholarship, we provide an overview of the complex and contextually specific nature of sex, sexual orientation, and gender, and discuss the implications of these complexities on how we approach collecting LGBTQ+ data. In Part Two, we consider the power of the stories we tell to impact the lives of LGBTQ+ people, and the frameworks, theories, and ethical imperatives which may help us to contribute to a narrative of LGBTQ+ liberation through our work. Finally, in Part Three, we offer an example tool for readers to use as they consider how they would approach this work in their own practices.","PeriodicalId":35250,"journal":{"name":"New Directions for Evaluation","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41475715","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In the United States, human service, public health, and healthcare organizations are dedicated to improving health equity among our society's most vulnerable. A wealth of literature highlights the importance of targeting root causes of inequity, however, intervention‐based attempts to improve health outcomes and reduce disparities have varied in their success. Too frequently, public health interventions fail to center community priorities and challenge oppressive regimes. At the same time, calls grow to pilot and evaluate new systems of care and service to replace antiquated, patchwork systems that depend on power imbalances and resource hoarding. The authors of this article, as current and recent leaders of Black‐led, LGBTQ+ organizations, engage in a conversation, in which we reflect on the power dynamics and pitfalls associated with community‐academic partnerships. Through our dialogue, we invite readers to internalize our testimony and re‐envision the role of the evaluator as a champion of liberation. Only through disrupting the status quo can evaluation hope to stand in community with “priority populations” and join the fight to achieve health equity. As members of the communities we serve, we transgress traditional means of how power and stature are allocated by being present in this special issue. We speak bluntly to honor our truth and inform evaluators in the process of fostering partnerships.
{"title":"Perspectives from LGBTQ+ serving CBO leaders on equitable community‐academic partnerships in evaluation","authors":"LaSaia Wade, Stephanie Skora, Erik Elías Glenn","doi":"10.1002/ev.20516","DOIUrl":"https://doi.org/10.1002/ev.20516","url":null,"abstract":"In the United States, human service, public health, and healthcare organizations are dedicated to improving health equity among our society's most vulnerable. A wealth of literature highlights the importance of targeting root causes of inequity, however, intervention‐based attempts to improve health outcomes and reduce disparities have varied in their success. Too frequently, public health interventions fail to center community priorities and challenge oppressive regimes. At the same time, calls grow to pilot and evaluate new systems of care and service to replace antiquated, patchwork systems that depend on power imbalances and resource hoarding. The authors of this article, as current and recent leaders of Black‐led, LGBTQ+ organizations, engage in a conversation, in which we reflect on the power dynamics and pitfalls associated with community‐academic partnerships. Through our dialogue, we invite readers to internalize our testimony and re‐envision the role of the evaluator as a champion of liberation. Only through disrupting the status quo can evaluation hope to stand in community with “priority populations” and join the fight to achieve health equity. As members of the communities we serve, we transgress traditional means of how power and stature are allocated by being present in this special issue. We speak bluntly to honor our truth and inform evaluators in the process of fostering partnerships.","PeriodicalId":35250,"journal":{"name":"New Directions for Evaluation","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46265701","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Andrew Hartman, Brian Hoessler, Vincent Tom, C. Camman
Alternative approaches within evaluation increasingly allow space for evaluators to bring themselves to their work. As queers, we are gifted‐partially as a necessity for our survival‐with deeper understandings of and navigational capacities to work within complexity. Furthermore, existing as queer empowers us to think and operate outside what is the norm, known, familiar and comfortable, and thus enables us to challenge normative systems for purposes of social change. Our chapter offers situated insight into what queer evaluation practices look like and empowers us to practice bringing ourselves into different contexts, including uncharted spaces. We illustrate principles of queer evaluation through cases of our unique identities, contexts, landscapes, and evaluation experiences, within a process that is iterative, dialogic, and relational. We argue that the exploration of ourselves is critical as evaluators and invite readers to wander alongside us while actively searching their identities. Rather than hiding these biases and perspectives, we believe in the importance of knowing oneself and our connections to the histories of those who came before, which serve as our guides. Only from this point can we begin to unravel the unknown into the known and transform the inequitable into the equitable that has yet to exist. We argue that by embracing our identities we are better able to navigate the complexities that exist in our work and deepen our understanding of the contexts around us.
{"title":"Identity as a compass when navigating uncharted equitable spaces: Our queer evaluation practices","authors":"Andrew Hartman, Brian Hoessler, Vincent Tom, C. Camman","doi":"10.1002/ev.20514","DOIUrl":"https://doi.org/10.1002/ev.20514","url":null,"abstract":"Alternative approaches within evaluation increasingly allow space for evaluators to bring themselves to their work. As queers, we are gifted‐partially as a necessity for our survival‐with deeper understandings of and navigational capacities to work within complexity. Furthermore, existing as queer empowers us to think and operate outside what is the norm, known, familiar and comfortable, and thus enables us to challenge normative systems for purposes of social change. Our chapter offers situated insight into what queer evaluation practices look like and empowers us to practice bringing ourselves into different contexts, including uncharted spaces. We illustrate principles of queer evaluation through cases of our unique identities, contexts, landscapes, and evaluation experiences, within a process that is iterative, dialogic, and relational. We argue that the exploration of ourselves is critical as evaluators and invite readers to wander alongside us while actively searching their identities. Rather than hiding these biases and perspectives, we believe in the importance of knowing oneself and our connections to the histories of those who came before, which serve as our guides. Only from this point can we begin to unravel the unknown into the known and transform the inequitable into the equitable that has yet to exist. We argue that by embracing our identities we are better able to navigate the complexities that exist in our work and deepen our understanding of the contexts around us.","PeriodicalId":35250,"journal":{"name":"New Directions for Evaluation","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48411195","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
C. Pavão, K. McLeroy, Y. Lincoln, J. Burdine, E. Wright
This chapter reports on the evaluation of state and local level National Standards for Culturally and Linguistically Appropriate Services in Health Care (aka CLAS Standards), specifically those standards addressing the health needs of sexual minority individuals, with an emphasis on the inclusion of bisexual+ communities and the implications of bisexual+ (non)inclusion in CLAS standards. At the state and local levels, bisexual identity is rarely recognized as distinct from other sexual identities. This lack of representation raises an essential issue of how local communities, states, and the federal government struggle with sexual minority data classification and prioritizing health benchmarks for sexual minority populations and subpopulations. We also found that the CLAS cultural competency policy definition at the federal level lacks an appropriate degree of bi‐inclusivity. The findings from this study reveal that the five states in our sample implemented CLAS Standards in ways that demonstrated bi‐erasure. Specifically, states defined gender and sexual minorities through exclusionary categories that place emphasis on the “Other”. LGBTQ+ evaluators can rely on the Principles of LGBTQ+ Evaluation to create strategies that demonstrate how to effectively address the intersecting ramifications of bi‐erasure at the policy level.
{"title":"Assessment of the inclusivity of the national CLAS standards enhancement initiative of bisexual identities","authors":"C. Pavão, K. McLeroy, Y. Lincoln, J. Burdine, E. Wright","doi":"10.1002/ev.20515","DOIUrl":"https://doi.org/10.1002/ev.20515","url":null,"abstract":"This chapter reports on the evaluation of state and local level National Standards for Culturally and Linguistically Appropriate Services in Health Care (aka CLAS Standards), specifically those standards addressing the health needs of sexual minority individuals, with an emphasis on the inclusion of bisexual+ communities and the implications of bisexual+ (non)inclusion in CLAS standards. At the state and local levels, bisexual identity is rarely recognized as distinct from other sexual identities. This lack of representation raises an essential issue of how local communities, states, and the federal government struggle with sexual minority data classification and prioritizing health benchmarks for sexual minority populations and subpopulations. We also found that the CLAS cultural competency policy definition at the federal level lacks an appropriate degree of bi‐inclusivity. The findings from this study reveal that the five states in our sample implemented CLAS Standards in ways that demonstrated bi‐erasure. Specifically, states defined gender and sexual minorities through exclusionary categories that place emphasis on the “Other”. LGBTQ+ evaluators can rely on the Principles of LGBTQ+ Evaluation to create strategies that demonstrate how to effectively address the intersecting ramifications of bi‐erasure at the policy level.","PeriodicalId":35250,"journal":{"name":"New Directions for Evaluation","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46813640","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Gregory Phillips, D. Felt, Esrea Pérez-Bill, Megan M. Ruprecht, Erik Elías Glenn
The time is long overdue for the field of evaluation to critically reckon with how we have failed to appropriately consider the needs and experiences of lesbian, gay, bisexual, transgender, queer, and other sexual and gender minority (LGBTQ+) people. Perhaps even more importantly, there is a dire need for work that moves us forward in new directions which are more affirming and inclusive of LGBTQ+ people. To achieve this idealistic change in LGBTQ+ Evaluation will require a genuine, transformative paradigm shift within the evaluation field, encompassing everything from pedagogy to practice and all activities in between. As a first step toward a unified paradigm of LGBTQ+ Evaluation, this chapter proposes eight Principles of LGBTQ+ Evaluation to guide evaluators’ work in partnership with and in service of LGBTQ+ communities, organizations, and individuals. Here we are not seeking to provide a script or a rigid framework but rather to create guiding signposts that light the way for evaluators new to LGBTQ+ Evaluation.
{"title":"Principles of LGBTQ+ Evaluation","authors":"Gregory Phillips, D. Felt, Esrea Pérez-Bill, Megan M. Ruprecht, Erik Elías Glenn","doi":"10.1002/ev.20519","DOIUrl":"https://doi.org/10.1002/ev.20519","url":null,"abstract":"The time is long overdue for the field of evaluation to critically reckon with how we have failed to appropriately consider the needs and experiences of lesbian, gay, bisexual, transgender, queer, and other sexual and gender minority (LGBTQ+) people. Perhaps even more importantly, there is a dire need for work that moves us forward in new directions which are more affirming and inclusive of LGBTQ+ people. To achieve this idealistic change in LGBTQ+ Evaluation will require a genuine, transformative paradigm shift within the evaluation field, encompassing everything from pedagogy to practice and all activities in between. As a first step toward a unified paradigm of LGBTQ+ Evaluation, this chapter proposes eight Principles of LGBTQ+ Evaluation to guide evaluators’ work in partnership with and in service of LGBTQ+ communities, organizations, and individuals. Here we are not seeking to provide a script or a rigid framework but rather to create guiding signposts that light the way for evaluators new to LGBTQ+ Evaluation.","PeriodicalId":35250,"journal":{"name":"New Directions for Evaluation","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45664387","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Research and theory on evaluation capacity building (ECB) and organizational evaluation capacity have been developing at a good pace over the past decade. On the other hand, there is a paucity of research on the nature and consequences of organizational evaluation policy. Evaluation policies are developed and implemented ultimately to inform and shape evaluation practice and its consequences. It is therefore natural to consider the interface between evaluation policy and ECB. The present exploratory descriptive study examines 52 evaluation policies from bilateral and multilateral aid agencies to explore connections between evaluation policy content and ECB principles and considerations. The results shed light on some interesting relationships; they are discussed in terms of evaluation use, evaluation purposes, and organizational leadership. The study also resulted in a revision of Trochim's (2009) definition of evaluation policy and a refinement and expansion of his eight‐category taxonomy. Implications for ongoing inquiry are considered and practical implications are offered to organization members and evaluation policy developers.
{"title":"Evaluation policy and organizational evaluation capacity building: A study of international aid agency evaluation policies","authors":"Hind Al Hudib, J. Cousins","doi":"10.1002/ev.20494","DOIUrl":"https://doi.org/10.1002/ev.20494","url":null,"abstract":"Research and theory on evaluation capacity building (ECB) and organizational evaluation capacity have been developing at a good pace over the past decade. On the other hand, there is a paucity of research on the nature and consequences of organizational evaluation policy. Evaluation policies are developed and implemented ultimately to inform and shape evaluation practice and its consequences. It is therefore natural to consider the interface between evaluation policy and ECB. The present exploratory descriptive study examines 52 evaluation policies from bilateral and multilateral aid agencies to explore connections between evaluation policy content and ECB principles and considerations. The results shed light on some interesting relationships; they are discussed in terms of evaluation use, evaluation purposes, and organizational leadership. The study also resulted in a revision of Trochim's (2009) definition of evaluation policy and a refinement and expansion of his eight‐category taxonomy. Implications for ongoing inquiry are considered and practical implications are offered to organization members and evaluation policy developers.","PeriodicalId":35250,"journal":{"name":"New Directions for Evaluation","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48143530","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The BUILD initiative is part of the Diversity Program Consortium, which the National Institutes of Health funded to increase diversity in biomedical research. This chapter aims to identify implications for the field from the multisite evaluation of BUILD initiative programs by reviewing the work undertaken by the authors of the other chapters in this issue. Given the complexities involved in multisite evaluations, innovative approaches and methods were used to balance the needs of each site with the overall objectives of the broader initiative. These approaches included a flexible orientation to the evaluation, mixed-methods designs that prioritized understanding the context before measuring it, and innovative analytic techniques (e.g., meta-analysis) to recognize the uniqueness of each site while providing insights about their cumulative impact. The BUILD initiative evaluation also offered many other valuable lessons about engaging stakeholders, focusing on use, and responding to changing priorities over time.
{"title":"Large-scale evaluation efforts and their implications for the field.","authors":"Tarek Azzam","doi":"10.1002/ev.20503","DOIUrl":"https://doi.org/10.1002/ev.20503","url":null,"abstract":"<p><p>The BUILD initiative is part of the Diversity Program Consortium, which the National Institutes of Health funded to increase diversity in biomedical research. This chapter aims to identify implications for the field from the multisite evaluation of BUILD initiative programs by reviewing the work undertaken by the authors of the other chapters in this issue. Given the complexities involved in multisite evaluations, innovative approaches and methods were used to balance the needs of each site with the overall objectives of the broader initiative. These approaches included a flexible orientation to the evaluation, mixed-methods designs that prioritized understanding the context before measuring it, and innovative analytic techniques (e.g., meta-analysis) to recognize the uniqueness of each site while providing insights about their cumulative impact. The BUILD initiative evaluation also offered many other valuable lessons about engaging stakeholders, focusing on use, and responding to changing priorities over time.</p>","PeriodicalId":35250,"journal":{"name":"New Directions for Evaluation","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10275578/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10084941","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In 2017, the U.S. Commission on Evidence‐Based Policymaking recommended that federal agencies produce strategic plans focused on research and evaluation, referred to as learning agendas. This requirement was later incorporated into the Foundations for Evidence‐Based Policymaking Act of 2018 (Evidence Act) for the 24 largest federal agencies. Prior to the Evidence Act, only a few federal agencies had experimented with learning agendas, a relatively new concept in the evaluation literature. Learning agendas hold potential for supporting organizational strategic planning that focuses on the generation of relevant knowledge for decision‐makers, organizational leaders, and stakeholders. An inclusively‐ and strategically‐developed learning agenda provides a list of important questions as well as plans for addressing the questions, balancing the interests, informational needs, and time horizons for different organizational decision‐makers. We draw upon the policy design and the evaluation capacity building literature, our analysis of existing learning agendas, and interviews with federal evaluation leaders who guided their development to describe how the process of developing a learning agenda can support intentional learning and impactful evaluation practice within public agencies. Our work should contribute to the development of both theory and practice regarding the implementation of the new expectation to produce learning agendas in federal agencies that contribute to the increased use of evaluation and evidence in policymaking.
{"title":"Learning agendas: Motivation, engagement, and potential","authors":"K. Newcomer, K. Olejniczak, Nicholas R. Hart","doi":"10.1002/ev.20495","DOIUrl":"https://doi.org/10.1002/ev.20495","url":null,"abstract":"In 2017, the U.S. Commission on Evidence‐Based Policymaking recommended that federal agencies produce strategic plans focused on research and evaluation, referred to as learning agendas. This requirement was later incorporated into the Foundations for Evidence‐Based Policymaking Act of 2018 (Evidence Act) for the 24 largest federal agencies. Prior to the Evidence Act, only a few federal agencies had experimented with learning agendas, a relatively new concept in the evaluation literature. Learning agendas hold potential for supporting organizational strategic planning that focuses on the generation of relevant knowledge for decision‐makers, organizational leaders, and stakeholders. An inclusively‐ and strategically‐developed learning agenda provides a list of important questions as well as plans for addressing the questions, balancing the interests, informational needs, and time horizons for different organizational decision‐makers. We draw upon the policy design and the evaluation capacity building literature, our analysis of existing learning agendas, and interviews with federal evaluation leaders who guided their development to describe how the process of developing a learning agenda can support intentional learning and impactful evaluation practice within public agencies. Our work should contribute to the development of both theory and practice regarding the implementation of the new expectation to produce learning agendas in federal agencies that contribute to the increased use of evaluation and evidence in policymaking.","PeriodicalId":35250,"journal":{"name":"New Directions for Evaluation","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"51164527","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The purpose of this chapter is to examine engagement strategies used in a large, multisite evaluation study through the lens of Estrada, Woodcock, and Schultz's (2014) tailored panel management. The evaluation, called the Enhance Diversity Study (EDS), is part of an effort funded by the National Institutes of Health (NIH) to increase diversity in NIH‐funded research. The chapter discusses engagement with a large national cohort of student participants and outlines survey administration complexities, tailored engagement approaches, and annual survey response trends. It shows how the EDS expanded Estrada and colleagues’ concepts of credibility by integrating branding strategies that permeated all aspects of the study. The resulting practices, as modified over time, extend knowledge of how to increase survey response rates across a multisite, multiprogram, longitudinal evaluation. As data collection continues, subsequent analysis may provide more clarity on the impact of these strategies on retention. Future researchers should explore the impacts of incorporating fully developed branding strategies to enhance study commitment and cohort retention. While past research has guided surveys through phone, mail, and multimodal distribution, more research is needed to understand how to engage participants and retain them in an increasingly competitive and digital world.
{"title":"Describing engagement practices for the Enhance Diversity Study using principles of Tailored Panel Management.","authors":"Karina D Ramirez, Cynthia J Joseph, Hansook Oh","doi":"10.1002/ev.20500","DOIUrl":"https://doi.org/10.1002/ev.20500","url":null,"abstract":"The purpose of this chapter is to examine engagement strategies used in a large, multisite evaluation study through the lens of Estrada, Woodcock, and Schultz's (2014) tailored panel management. The evaluation, called the Enhance Diversity Study (EDS), is part of an effort funded by the National Institutes of Health (NIH) to increase diversity in NIH‐funded research. The chapter discusses engagement with a large national cohort of student participants and outlines survey administration complexities, tailored engagement approaches, and annual survey response trends. It shows how the EDS expanded Estrada and colleagues’ concepts of credibility by integrating branding strategies that permeated all aspects of the study. The resulting practices, as modified over time, extend knowledge of how to increase survey response rates across a multisite, multiprogram, longitudinal evaluation. As data collection continues, subsequent analysis may provide more clarity on the impact of these strategies on retention. Future researchers should explore the impacts of incorporating fully developed branding strategies to enhance study commitment and cohort retention. While past research has guided surveys through phone, mail, and multimodal distribution, more research is needed to understand how to engage participants and retain them in an increasingly competitive and digital world.","PeriodicalId":35250,"journal":{"name":"New Directions for Evaluation","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/b9/80/nihms-1903800.PMC10348780.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9831647","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This chapter describes how and why the U.S. Department of Labor (DOL) structured and implemented a comprehensive evidence‐building strategy in the years ahead of the federal legislation that now requires many of the same key components. In 2010, the Chief Evaluation Office was established in DOL at the departmental level to coordinate evaluation strategy and evidence building and to promote an organization‐wide culture of learning. This represented a new approach intended to elevate the priority on evidence, improve the scope and quality of evaluations and research, and expand the use of evidence. The DOL strategy included formalizing a departmental evaluation policy statement around key principles that govern high‐quality evaluations, developing a learning agenda process to strategically plan for evaluations and evidence‐building activities, and creating an evidence‐based clearinghouse to synthesize and share the results of rigorous evaluations. While each department is unique, DOL's experience highlights functions that were prioritized as well as challenges and limitations that had to be addressed in one department.
{"title":"Putting it all together: The case of the U.S. Department of Labor's evidence‐building strategy","authors":"M. Irwin, D. Nightingale","doi":"10.1002/ev.20489","DOIUrl":"https://doi.org/10.1002/ev.20489","url":null,"abstract":"This chapter describes how and why the U.S. Department of Labor (DOL) structured and implemented a comprehensive evidence‐building strategy in the years ahead of the federal legislation that now requires many of the same key components. In 2010, the Chief Evaluation Office was established in DOL at the departmental level to coordinate evaluation strategy and evidence building and to promote an organization‐wide culture of learning. This represented a new approach intended to elevate the priority on evidence, improve the scope and quality of evaluations and research, and expand the use of evidence. The DOL strategy included formalizing a departmental evaluation policy statement around key principles that govern high‐quality evaluations, developing a learning agenda process to strategically plan for evaluations and evidence‐building activities, and creating an evidence‐based clearinghouse to synthesize and share the results of rigorous evaluations. While each department is unique, DOL's experience highlights functions that were prioritized as well as challenges and limitations that had to be addressed in one department.","PeriodicalId":35250,"journal":{"name":"New Directions for Evaluation","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"51164379","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：