Evaluation Review最新文献

Given the scarcity of comprehensive review studies in the literature, this bibliometric analysis thrives on taking a scientific approach to delivering quantitative and qualitative information on the ever-evolving field of ESG. Utilizing the performance analysis and science mapping techniques which includes citation analysis, co-word analysis, the study reports a holistic overview of 693 (ESG) papers from 1991 to 2020. The paper aims to provide a unified perspective on the evolution and further identify future research directions in this field. The findings point to the most important countries, authors, studies, and top journals in this field. Results show that published articles on ESG gained momentum after 2006; however, an exponential rise is being observed in the past 5 years, with research focusing on sustainable finance, sustainable development, ESG, and CSR with themes extending to risk management, stakeholder engagement, and portfolio construction, among others. Furthermore, the research identifies how the practice of ESG reporting affects many variables such as financial performance, social performance, environmental performance, and sustainability score. The findings also indicate that the field of ESG is still evolving, with numerous unexplored themes. The work is novel and relevant on the pretext that it uses the Scopus and Web of Science databases for bibliometric mapping, whereas previous studies have used the Scopus database but have lacked a robust methodology, so the findings of this study provided strong support for identifying emerging paradigms in the ESG literature.

Uncertainty is an overarching aspect of life that is particularly pertinent to the present COVID-19 pandemic crisis; as seen by the pandemic's rapid worldwide spread, the nature and level of uncertainty have possibly increased due to the possible disconnects across national borders. The entire economy, especially the tourism industry, has been dramatically impacted by COVID-19. In the current study, we explore the impact of economic policy uncertainty (EPU) and pandemic uncertainty (PU) on inbound international tourism by using data gathered from Italy, Spain, and the United States for the years 1995-2021. Using the Quantile on Quantile (QQ) approach, the study confirms that EPU and PU negatively affected inbound tourism in all states. Wavelet-based Granger causality further reveals bi-directional causality running from EPU to inbound tourism and unidirectional causality from PU to inbound tourism in the long run. The overall findings show that COVID-19 has had a strong negative effect on tourism. So resilient skills are required to restore a sustainable tourism industry.

This paper describes how a multi-armed randomized experiment was used to test multiple variants of a behaviorally informed marketing strategy. In particular, we tested whether specific behavioral messages could be used to increase demand for a safety consultation service offered by the U.S. Occupational Safety and Health Administration. Our experiment used a partial factorial design with 19 study arms and a very large research sample-97,182 establishments-to test the impact of various message, formats, and delivery modes compared with an existing (not behaviorally informed) informational brochure and a no-marketing counterfactual. A secondary research goal was to predict the impact of the most successful marketing strategy (i.e., combination of message, format, and mode) so that OSHA would know what to anticipate if that strategy were implemented at scale. We used two related (but distinct) methods to address these two goals. Both begin with a common mixed (i.e., fixed and random effects) ANOVA model. We addressed the first research goal primarily from the fixed effects; we addressed the second research goal by calculating best linear unbiased predictions (BLUPs) from the full mixed model, where the BLUP involves "shrinkage" as in empirical Bayes (EB) approaches. Marketing via brochures was effective overall, nearly doubling the rate of requests for services. However, the behaviorally informed materials performed no better than OSHA's existing informational brochure. This study also highlights the conditions under which a factorial design can be used to efficiently address questions about which of several program variants are most effective.

Background: In the early 1970s, most researchers thought that randomized controlled trials (RCTs) could not be used to measure the effectiveness of large-scale operating welfare reform and employment programs. By the mid-1970s, the Supported Work Demonstration showed that, under certain conditions, this was both feasible and valuable. However, the experimental design was simple; a multi-arm test had been rejected as unrealistic. Within 10 years, a three-arm design was implemented in San Diego to assess both a welfare-to-work program's overall impact and the contribution of a specific component. Less than 10 years later, the Job Opportunities and Basic Skills Training (JOBS)/National Evaluation of Welfare-to-Work Strategies (NEWWS) study used a more complex design to determine the relative effectiveness of two strategies operated in the same locations: one emphasizing getting a job quickly and the other requiring basic education. In San Diego and JOBS/NEWWS, the tested reforms emerged from political processes and were funded through regular program budgets. In both cases, researchers inserted multi-arm RCTs into operating welfare offices, trading control over the treatment for scale (thousands of people) and real-world conditions. Both RCTs were successfully implemented.

Objectives and results: This article examines why multi-arm designs were attempted, how they were structured, why public administrators cooperated, what various actors sought to learn, and how the researchers determined what strategies the different experimental arms ended up to truly represent. The article concludes that these designs provide convincing evidence and can be inserted into operating programs if the studies address questions that are of keen and immediate interest to state or local program administrators and researchers.

In this article, we explore the reasons why multiarm trials have been conducted and the design and analysis issues they involve. We point to three fundamental reasons for such designs: (1) Multiarm designs allow the estimation of "response surfaces"-that is, the variation in response to an intervention across a range of one or more continuous policy parameters. (2) Multiarm designs are an efficient way to test multiple policy approaches to the same social problem simultaneously, either to compare the effects of the different approaches or to estimate the effect of each separately. (3) Multiarm designs may allow for the estimation of the separate and combined effects of discrete program components. We illustrate each of these objectives with examples from the history of public policy experimentation over the past 50 years and discuss some design and analysis issues raised by each, including sample allocation, statistical power, multiple comparisons, and alignment of analysis with goals of the evaluation.

Administered by the Office of Family Assistance in the U.S. Department of Health and Human Services' Administration for Children and Families (ACF), the Health Profession Opportunity Grants (HPOG) Program provided education and training to Temporary Assistance for Needy Families (TANF) recipients and other adults with low incomes for occupations in the healthcare field. The impact evaluation for the first cohort of grantees (HPOG 1.0) leveraged the program's implementation across many locations, using a three-armed evaluation design (including a second treatment arm) in some places, as a way to examine whether any of three selected program components, or enhancements, contributed to the program's overall impact. This article tells the story of the evaluation and draws lessons from that experience, discussing implications for future implementation of multi-armed experiments in a multi-site evaluation.

This article, prepared as part of a special issue on multiarmed experiments, describes the design of the RAND Health Insurance Experiment, paying particular attention to the choice of arms. It also describes how the results of the Experiment were used in a simulation model and, looking back, how the design might have differed, and how the results apply today, 4 decades after the Experiment was conducted.

Evaluation influence is a reconceptualization of evaluation use that reflects the broad and diffuse impacts an evaluation can have on social programs and policies. This way of thinking about impact provides an opportunity to investigate how and why evaluations influence social programs and policy. Twenty participants (practitioners and managers) from two child protection programs evaluated in the previous 24 months were interviewed about the influence of these evaluations, which was complemented with the collection of internal documents about changes to the programs. A qualitative case study analysis of evaluation influence was conducted using the interviews and documents to investigate the influence of two evaluations at different stages in the dissemination process. The participants identified that the evaluations appeared to have significant high-level policy level influence; however, limited examples of influence on practices in the programs were identified. There was some suggestion that the evaluations had increased practitioner interest in working with and participating in program evaluations. The findings suggest the importance of developmental evaluation approaches and practitioner engagement in evaluation to improve the influence and adoption of new knowledge from the evaluation of social programs.

Voluminous vaccine campaigns have been used globally, since the COVID-19 pandemic has brought devastating mortality and destructively unprecedented consequences to different aspects of economies. This study aimed to identify how the numbers of new deaths and new cases per million changed after half of the population had been vaccinated. This paper used actual pandemic consequence variables (death and infected rates) together with vaccination uptake rates from 127 countries to shed new light on the efficacy of COVID-19 vaccines. The 50% uptake rate was chosen as the threshold to estimate the real benefits of vaccination campaigns for reducing COVID-19 infection and death cases using the difference-in-differences (DiD) imputation estimator. In addition, a number of control variables, such as government interventions and people's mobility patterns during the pandemic, were also included in the study. The number of new deaths per million significantly decreased after half of the population was vaccinated, but the number of new cases did not change significantly. We found that the effects were more pronounced in Europe and North America than in other continents. Our results remain robust after using other proxies and testing the sensitivity of the vaccinated proportion. We show the causal evidence of significantly lower death rates in countries where half of the population is vaccinated globally. This paper expresses the importance of vaccine campaigns in saving human lives during the COVID-19 pandemic, and its results can be used to communicate the benefits of vaccines and to fight vaccine hesitancy.