Evaluation Review最新文献

This paper investigates the systemic risk spillovers from the oil and gold markets to China financial market. Specifically, this paper uses wavelet analysis methods combined with time-varying copula models to calculate the CoVaR to explore the risk tail correlation and the strength of systemic risk spillover effects among oil, gold and China financial market under time-frequency conditions. The empirical results are as follows: First, there is a systemic risk spillover from the oil and gold markets to China financial market with significant time-varying characteristics. Second, changes in oil prices exacerbate risk spillovers from China financial markets, while price volatility in the gold market has a moderating effect on its risk spillovers, with system-wide risk spillovers increasing significantly in times of major economic crisis events. Third, oil market has a greater intensity of systemic risk spillover to China financial market than the gold market. External economic events significantly affect the intensity of risk spillovers from oil to China financial market, and abnormal volatility shocks in internal stock markets significantly affect the intensity of risk spillovers from gold to China financial market more significantly than the impact of external events. Furthermore, in the frequency domain, the oil and gold markets have some lagging effect on the transmission of systemic risks to the China financial market.

This article reflects on how the Independent Evaluation Group (IEG) of the World Bank Group grapples with the issue of generalizability in conducting large-scale and complex development evaluations. It discusses a practical framework IEG uses to meet three needs: methodological, institutional, and didactic. IEG evaluations are aimed at informing significant organizational or strategic decisions encompassing broad scopes, assessing extensive portfolios of interventions by the organization across diverse contexts and spanning multiple years. These evaluations are inherently multimethod and need to bridge various logics of generalization. They seek to influence decision-makers, such as boards of directors or senior managerial teams, with the objective of guiding pivotal moments in the organization's trajectory; ensuring accountability for learning, results, or budget expenditures; and synthesizing substantial evidence to distill key success or failure factors for future strategic planning. The defensibility of the methodological scaffolding is paramount to the credibility of the evaluations. The article discusses the challenges inherent in such evaluations, including the need to generate findings that are valid at multiple levels of analysis and the reliance on multitiered mixed-methods approaches. It examines the use of a practical framework to bridge methodological principles and real-world challenges involved in evaluation to inform the design and implementation of large-scale evaluations. The framework is illustrated with examples from IEG's evaluations, and the article explores how practitioners and researchers can apply the framework in other settings to enhance the generalizability of their findings.

The relationship between climate risks and commodity markets remains insufficiently explored, especially when analyzed through the lens of high-frequency data. This study seeks to address this gap by investigating the spillover effects of global climate risks, both physical and transitional, on key commodity markets and employs a novel analytical framework. By utilizing newly developed climate risk indices alongside the innovative mixed-frequency spillover measure, this research combines high-frequency climate risk data with the responses of low-frequency commodity prices. Our results highlight notable spillover effects, demonstrating that climate risks serve as the primary drivers of spillovers to commodity markets in a mixed-frequency data context, whereas such effects are not observed within a common-frequency data environment. These findings have important implications for policy-makers and investors, indicating that current market analyses may not capture the influence of climate risk adequately.

I explore how "low-quality evidence" from program performance might still be useful in decision-making. Conceptually, a local government named "Here" is motivated to consider a program from "Elsewhere" that seems to show year-over-year exemplary performance. Here must manage five sources of uncertainty about whether and how to extrapolate from Elsewhere: chance in assessing Elsewhere's performance; illusion due to confounding variables; estimating the several powers of the program's components; substitutions in the design process made by Elsewhere and contemplated by Here; and estimating whether in the final analysis Here can meet its own breakeven criterion for going ahead. Here can begin with Elsewhere's experience, but it still must do much thinking and information-collecting on its own.

Gamified Flipped Classrooms (GFC) are increasingly implemented in teaching and have become a trending research subject. Gamification and Flipped Classroom (FC) literature expresses a clear need for a theoretical foundation that positively affects motivation, behavioral change, and learning. However, this requires an overview of the current theoretical foundations of GFC research. Therefore, this study conducted a systematic mixed studies review (SMSR) of the current theoretical foundations of GFC research and learning effects. Sixty-nine studies were identified after applying rigorously defined inclusion and exclusion criteria. The review revealed that a significant number of studies (25 of 69) lacked a theoretical foundation. In studies supported by theory, Self-Determination Theory (SDT) was most frequently employed. Moreover, only three studies used theories to design both gamification and FC components. While GFC research generally shows positive effects on learning, studies with a theoretical background reported a higher rate of positive GFC outcomes (56%), compared to only 29% for studies lacking a theoretical foundation. Future research should explore and attempt to link SDT and other theories to learning impacts.

This study introduces a novel aligned technical index, derived from multiple technical indicators, that encompasses a broader spectrum of technical measurement strategies than those obtained from previous 3PRF (Three-Pass Regression Filter) research. Our empirical results demonstrate that this index exhibits significant predictive power for new energy price returns in both in-sample and out-of-sample tests. This index is extracted using the 3PRF method and yields significantly better results than those obtained with traditional methods. Considering that the market typically operates in two states, we incorporate a regime-switching model with time-varying transition probabilities into our forecasting framework. The findings indicate that the technical index influences the probability of regime transitions between states and that the inclusion of a regime-switching model further enhances predictive performance. The incorporation of the regime-switching mechanism further improves the predictive performance of the model. Moreover, from an asset allocation perspective, both the technical index and regime-switching models deliver considerable economic value to mean-variance investors.

This brief report provides preliminary independent evidence of the efficacy of Football Beyond Borders (FBB), a targeted, school-based social and emotional learning (SEL) intervention for at-risk youth. FBB includes weekly SEL classroom sessions, activities on the football pitch, 1:1 therapy sessions, holiday support, and rewards trips. Propensity score matching and difference-in-differences estimation were used in a pre-test/post-test control group design to assess the impact of FBB on the mental wellbeing (assessed via the Short Warwick-Edinburgh Mental Wellbeing Scale, SWEMWBS) of participants designated at-risk (N = 46 aged 12-14, 78.3% male), passive learners (N = 72, aged 12-14, 84.7% male), and role models (N = 35, aged 12-14, 85.7% male), with matched control samples derived from a subset of the #BeeWell cohort study (N = 8015). A significant intervention effect was observed for at-risk youth, with FBB leading to an increase of approximately 2.4 SWEMWBS points (d = 0.44). No significant intervention effects were observed for passive learners or role models. These results indicate that FBB can improve the mental wellbeing of at-risk youth. Accordingly, an explanatory trial is warranted.

Official development agencies are increasingly supporting civil society lobby and advocacy (L&A) to address poverty and human rights. However, there are challenges in evaluating L&A. As programme objectives are often to change policies or practices in a single institution like a Government Ministry, L&A programmes are often not amenable to large-n impact evaluation methods. They often work in strategic partnerships to foster change; hence, contribution may be a more relevant evaluation question than attribution. Small-n qualitative approaches are available to measure the effectiveness of L&A which use the theory of change as their analytical framework. We conducted a meta-evaluation of 36 evaluations of multi-component international programmes to support civil society L&A across Asia, Africa and Latin America, comprising the majority of programmatic support from one international donor. We assessed the confidence in causal claims in the evaluations using a new tool that we developed. Assessments of the contribution of the programmes to the changes in outcomes were not provided in many of the evaluations, nor were predictable sources of bias addressed. Given that L&A programmes are likely to adopt an influencing approach where many different inside-track and outside-track engagement objectives, opportunities and strategies are attempted, many of which might be expected to fail, there appeared to be a clear bias in the evaluations towards reporting outcomes that were achieved, ignoring those that were not. We provide guidance on how to improve the design, conduct and reporting of small-n qualitative evaluations of aid effectiveness.

Unlike previous studies on fixed logistics nodes, this research explored how consumer distribution impacts store selection and inventory balance, integrating the ship-from-store strategy to increase fulfillment within multiperiod sales plans. Specifically, omnichannel retailers (O-tailer) must sequentially decide on inventory replenishment from suppliers to the distribution center (DC), allocation from the DC to stores, and which department will fulfill online orders. We introduce a multiperiod stochastic optimization model and solve it with a robust two-stage approach (RTA). In Stage 1, we use the K-means algorithm and silhouette coefficients to determine the optimal number of stores. In Stage 2, linear decision rule (LDR) are employed to decide on replenishment, allocation, and order fulfillment quantities. Numerical experiments show that RTA outperforms existing methods, achieving solutions with efficiency gaps of less than 10%, even when assumptions are not fully met. Additionally, the sensitivity analysis shows that variations in product prices, fulfillment costs, market share, and customer distribution consistently lead to greater profits with the ship-from-store strategy.

The recognition that researcher discretion coupled with unconscious biases and motivated reasoning sometimes leads to false findings ("p-hacking") led to the broad embrace of study preregistration and other open-science practices in experimental research. Paradoxically, the preregistration of quasi-experimental studies remains uncommon although such studies involve far more discretionary decisions and are the most prevalent approach to making causal claims in the social sciences. I discuss several forms of recent empirical evidence indicating that questionable research practices contribute to the comparative unreliability of quasi-experimental research and advocate for adopting the preregistration of such studies. The implementation of this recommendation would benefit from further consideration of key design details (e.g., how to balance data cleaning with credible preregistration) and a shift in research norms to allow for appropriately nuanced sensemaking across prespecified, confirmatory results and other exploratory findings.