Evaluation Review最新文献

Non-randomized studies of intervention effects (NRS), also called quasi-experiments, provide useful decision support about development impacts. However, the assumptions underpinning them are usually untestable, their verification resting on empirical replication. The internal replication study aims to do this by comparing results from a causal benchmark study, usually a randomized controlled trial (RCT), with those from an NRS conducted at the same time in the sampled population. We aimed to determine the credibility and generalizability of findings in internal replication studies in development economics, through a systematic review and meta-analysis. We systematically searched for internal replication studies of RCTs conducted on socioeconomic interventions in low- and middle-income countries. We critically appraised the benchmark randomized studies, using an adapted tool. We extracted and statistically synthesized empirical measures of bias. We included 600 estimates of correspondence between NRS and benchmark RCTs. All internal replication studies were found to have at least "some concerns" about bias and some had high risk of bias. We found that study designs with selection on unobservables, in particular regression discontinuity, on average produced absolute standardized bias estimates that were approximately zero, that is, equivalent to the estimates produced by RCTs. But study conduct also mattered. For example, matching using pre-tests and nearest neighbor algorithms corresponded more closely to the benchmarks. The findings from this systematic review confirm that NRS can produce unbiased estimates. Authors of internal replication studies should publish pre-analysis protocols to enhance their credibility.

Even though there has been increasing interest in the role of cognition in leadership and in identifying the personality traits of effective leaders, there is a paucity of studies that investigate the unique influence of managers' trait affectivity and cognitive ability on their different risk and time preferences. This paper investigates the role of managers' trait affectivity and cognitive ability in their loss aversion and present bias among 623 top managers at textile and garment firms in Vietnam. We combine data on preferences elicited through a lab-in-the-field experiment with survey data. We find that managers with high positive affectivity (PA) or cognitive ability are less subject to loss aversion and present bias. In contrast, a manager with high negative affectivity (NA) is more likely to be impatient and loss averse. Furthermore, heterogeneity of trait affectivity and cognitive ability determines different loss aversion and present bias levels of managers in SMEs vis-à-vis their counterparts in large firms. Remarkably, we observe striking evidence that trait affectivity and cognitive ability significantly affect loss aversion and present bias levels of managers who were born during the Vietnam War. Still, it is not the story of their counterparts born after the Vietnam War. The results of our study are expected to provide valuable information regarding the role played by trait affectivity and cognitive ability in determining managers' loss aversion and present bias in different pathways.

This study evaluates the sensitivity of government bond yields from the countries most affected by the COVID-19 pandemic to variations in some international risk factors during the period between January 2020 and April 2021. This sample period allows us to focus the study on the first, and the subsequent waves of the COVID-19 pandemic. Specifically, we propose an extended risk factor model estimated using the quantile regression approach. In addition, this study compares the COVID-19 pandemic period with a pre-pandemic and a post-vaccination period. Interesting differences among them are observed, remarking that gold is the key risk factor during the pandemic, whereas VIX and crude oil play that role in the pre-pandemic and the post-vaccination periods, respectively, mainly for bearish states. As expected, the explanatory power of the model is better at extreme quantiles, showing relevant differences between sensitivities, because the found effects are quantile-, country- and risk factor-dependent. The results during the pandemic are robust to the inclusion of a country-specific factor and a factor accounting for the mutual influence of the government bonds.

The COVID-19 pandemic poses a serious threat to investors in the crude oil market. Furthermore, investors have an increasing need to find a safe haven in their investment portfolios when facing unprecedented risks in crude oil markets during the COVID-19 pandemic. According to a review of the literature, there are contradictory findings on which investment is the safer haven for the oil market. Therefore, this paper aims to evaluate whether bitcoin is a safer haven for the crude oil market than the commonly used gold during the COVID-19 pandemic. Three spillover measurements based on the time, and frequency domains, and a network framework are employed to quantify the return spillover effects among bitcoin, gold and three major crude oil futures markets. We divide the sample into two periods, pre-COVID-19 and post-COVID-19. The results show that bitcoin has a weak safe-haven effect on the crude oil market only over a short period, while gold maintains a good safe-haven ability for crude oil futures across various time horizons (frequencies), both before and after the outbreak of the COVID-19 pandemic. The findings of this study have important implications for policy-makers, crude oil producers and global investors. In particularly, investors cannot ignore the importance of bitcoin and gold in selecting more profitable portfolio policies when searching for safe-haven assets.

The Indonesian government has evaluated the research performance of universities, whose measurement process is projected into resources, management, outputs, and revenues to determine the provision of incentives, grants, and program funding to universities. However, efficiency calculations have shown that the outputs and competition-based incentives that drive scientific productivity are more complex. The most competitive systems must also be the most productive when considering resources. This study aimed to analyze the research efficiency in the Indonesian higher education system. The efficiency was analyzed by maximizing the 13 product outputs from the research budget and university staff. The result was then compared with the existing performance measurement analysis. Data Envelopment Analysis (DEA) was used to evaluate the efficiency based on the data of 47 universities in the Mandiri cluster and 144 in the Utama cluster for the 2014-2018 period. These findings showed that about 68% of universities have an efficiency value of 1 for the Mandiri group, almost 40% in the Utama group, and 41% for the two groups combined. Additionally, this study compared the efficiency analysis and the impact of the performance evaluation. The comparison showed that adding efficiency or productivity factors in the performance evaluation assessment produced a more accurate result.

Evaluation of public programs has undergone many changes over the past four decades since Peter Rossi coined his "Iron Law" of program evaluation: "The expected value of any net impact assessment of any large-scale social program is zero." While that assessment may be somewhat overstated, the essence still holds. The failures far outnumber the successes, and the estimated favorable effects are rarely sizeable. Despite this grim assessment, much can be learned from "failed" experiments, and from ones that are successful in only some sites or subgroups. Advances in study design, statistical models, data, and how inferences are drawn from estimates have substantially improved our analyses and will continue to do so. However, the most actual learning about "what works" (and why, when, and where) is likely to come from gathering more detailed and comprehensive data on how the intervention was implemented and attempting to link that data to estimated impacts. Researchers need detailed data on the target population served, the content of the intervention, and the process by which it is delivered to participating service providers and individuals. Two examples presented here illustrate how researchers drew useful broader lessons from impact estimates for a set of related programs. Rossi posited three reasons most interventions fail-wrong question, wrong intervention, poor implementation. Speeding the accumulation of wisdom about how social programs can best help vulnerable populations will require that researchers work closely with program funders, developers, operators, and participants to gather and interpret these detailed data about program implementation.

Regression discontinuity is popular in finding treatment/policy effects when the treatment is determined by a continuous variable crossing a cutoff. Typically, a local linear regression (LLR) estimator is used to find the effects. For binary response, however, LLR is not suitable in extrapolating the treatment, as in doubling/tripling the treatment dose/intensity. The reason is that doubling/tripling the LLR estimate can give a number out of the bound $\left[-1,1\right]$, despite that the effect should be a change in probability. We propose local maximum likelihood estimators which overcome these shortcomings, while giving almost the same estimates as the LLR estimator does for the original treatment. A simulation study and an empirical analysis for effects of an income subsidy program on religion demonstrate these points.

The renewable energy consumption plays a significant role in achieving sustainable development, but a sectoral approach is necessary to design the better recommendations for each sector. In this context, the main objective of this paper is to assess the impact of the use of this type of energy on economic growth in 23 European Union (EU) member states in the period 1990-2020. Besides overall renewable energy consumption, different utilisations of this energy are considered: in industry, transport, in commercial and public services, and for residential purposes. The methodological background is built around panel data models that start from a Cobb-Douglas function. The renewable energy consumption is considered an important factor that should generate economic growth. The panel data approach based on causality analysis and Augmented Mean Group and Common Correlated Effects Mean Group estimators suggests that renewable energy use in industry does not determine economic growth, but economic development is a cause for more utilisation of this energy in industry. In addition, more renewable energy consumption in transport enhances economic growth. A high level of economic development can promote the consumption of renewable energies in industrial sectors. In this way, industrial companies can allocate more financial funds to research in the field of renewable energies and can afford to adopt renewable energy sources. Investment in biofuels can contribute to achieving sustainable transport in the EU.

Monetary policy changes have an irreplaceable impact on economic activity. Considering the close linkage among economic policies, we employ a bi-directional Granger causality test to investigate the potential linkages between monetary policy uncertainty (MPU) and other categorical economic policy uncertainty (CEPU) in the time and frequency domains. We consider all news-based U.S. categorical economic policy uncertainty indices (CEPU). All monthly CEPU indicators, covering January 1986 to January 2022, can be obtained from the website of Economic Policy Uncertainty. On an average, causality running from each CEPU to MPU is not apparent, while MPU can significantly affect six policy-related uncertainties: taxes, government spending, health care, national security, entitlement programs and regulation. A further frequency-domain study showed the dynamic changes in the relationship between them. For instance, we capture mid- and long-run causality running from tax uncertainty to MPU, while MPU has an impact on taxes in the medium run. Our findings provide policymakers with a better understanding of the nexus between MPU and other CEPU for formulating appropriate economic policies. Particularly, if a sectional government considers the long- and short-term effects of different policies when formulating strategies, risk transmission may be curbed to some extent.

Methods for estimating causal impact aim to either remove or reduce bias. This study estimates the degree of bias reduction obtained from regression adjustment and propensity score methods when only a weak set of predictors are available. The study uses an experimental test of providing emergency financial assistance to participants in a job training program to estimate an experimental benchmark and compares it to nonexperimental estimates of the impact of receiving assistance. When estimating the impact of receiving assistance, those who received it constitute the treatment group. The study explores two different comparison groups: those who could have (because they were assigned to the experimental treatment group) but did not receive emergency assistance; and those who could not receive emergency assistance because they were randomly assigned to the experimental control group. It uses these groups to estimate impacts by applying three estimation strategies: unadjusted mean comparison, regression adjustment, and inverse propensity weighting. It then compares these estimates to the experimental benchmark using statistical tests recommended by the within-study comparison literature. The nonexperimental approaches to addressing selection bias suggest large positive impacts. These are statistically different from the experimental benchmark, which shows that receipt of emergency assistance does not improve educational progress. Further, over 90% of the bias from a simple comparison of means remains. Unless a stronger set of predictors are available, future evaluations of such interventions should be wary of relying on these methods for either unbiased estimation of impacts or bias reduction.