Evaluation Review最新文献

This brief report provides preliminary independent evidence of the efficacy of Football Beyond Borders (FBB), a targeted, school-based social and emotional learning (SEL) intervention for at-risk youth. FBB includes weekly SEL classroom sessions, activities on the football pitch, 1:1 therapy sessions, holiday support, and rewards trips. Propensity score matching and difference-in-differences estimation were used in a pre-test/post-test control group design to assess the impact of FBB on the mental wellbeing (assessed via the Short Warwick-Edinburgh Mental Wellbeing Scale, SWEMWBS) of participants designated at-risk (N = 46 aged 12-14, 78.3% male), passive learners (N = 72, aged 12-14, 84.7% male), and role models (N = 35, aged 12-14, 85.7% male), with matched control samples derived from a subset of the #BeeWell cohort study (N = 8015). A significant intervention effect was observed for at-risk youth, with FBB leading to an increase of approximately 2.4 SWEMWBS points (d = 0.44). No significant intervention effects were observed for passive learners or role models. These results indicate that FBB can improve the mental wellbeing of at-risk youth. Accordingly, an explanatory trial is warranted.

Official development agencies are increasingly supporting civil society lobby and advocacy (L&A) to address poverty and human rights. However, there are challenges in evaluating L&A. As programme objectives are often to change policies or practices in a single institution like a Government Ministry, L&A programmes are often not amenable to large-n impact evaluation methods. They often work in strategic partnerships to foster change; hence, contribution may be a more relevant evaluation question than attribution. Small-n qualitative approaches are available to measure the effectiveness of L&A which use the theory of change as their analytical framework. We conducted a meta-evaluation of 36 evaluations of multi-component international programmes to support civil society L&A across Asia, Africa and Latin America, comprising the majority of programmatic support from one international donor. We assessed the confidence in causal claims in the evaluations using a new tool that we developed. Assessments of the contribution of the programmes to the changes in outcomes were not provided in many of the evaluations, nor were predictable sources of bias addressed. Given that L&A programmes are likely to adopt an influencing approach where many different inside-track and outside-track engagement objectives, opportunities and strategies are attempted, many of which might be expected to fail, there appeared to be a clear bias in the evaluations towards reporting outcomes that were achieved, ignoring those that were not. We provide guidance on how to improve the design, conduct and reporting of small-n qualitative evaluations of aid effectiveness.

Unlike previous studies on fixed logistics nodes, this research explored how consumer distribution impacts store selection and inventory balance, integrating the ship-from-store strategy to increase fulfillment within multiperiod sales plans. Specifically, omnichannel retailers (O-tailer) must sequentially decide on inventory replenishment from suppliers to the distribution center (DC), allocation from the DC to stores, and which department will fulfill online orders. We introduce a multiperiod stochastic optimization model and solve it with a robust two-stage approach (RTA). In Stage 1, we use the K-means algorithm and silhouette coefficients to determine the optimal number of stores. In Stage 2, linear decision rule (LDR) are employed to decide on replenishment, allocation, and order fulfillment quantities. Numerical experiments show that RTA outperforms existing methods, achieving solutions with efficiency gaps of less than 10%, even when assumptions are not fully met. Additionally, the sensitivity analysis shows that variations in product prices, fulfillment costs, market share, and customer distribution consistently lead to greater profits with the ship-from-store strategy.

The recognition that researcher discretion coupled with unconscious biases and motivated reasoning sometimes leads to false findings ("p-hacking") led to the broad embrace of study preregistration and other open-science practices in experimental research. Paradoxically, the preregistration of quasi-experimental studies remains uncommon although such studies involve far more discretionary decisions and are the most prevalent approach to making causal claims in the social sciences. I discuss several forms of recent empirical evidence indicating that questionable research practices contribute to the comparative unreliability of quasi-experimental research and advocate for adopting the preregistration of such studies. The implementation of this recommendation would benefit from further consideration of key design details (e.g., how to balance data cleaning with credible preregistration) and a shift in research norms to allow for appropriately nuanced sensemaking across prespecified, confirmatory results and other exploratory findings.

This introduction to the second special issue of Evaluation review on external validity and generalizability opens a dialogue between different ways to think about generalizability in program evaluation. It argues that generalizability in impact evaluation fundamentally is about inferring some form of causality at a level broader than the specific circumstances of the initial study or studies from which these inferences are drawn. The question, then, is about how one apprehends causality: in other words, what is being generalized and how? The first special issue mainly relied on a counterfactual conception of causality, embodied by experimental and quasi-experimental methods, that aimed at impact measurement. The articles in this volume, drawing on mixed methods, also mobilize generative and configurational causal inferences to provide further levers of generalizability, focusing on how the impact is produced. The introduction insists on the specific input of qualitative methods in this respect, as theorized by grounded theory.

In this letter to the editor, we compare six different event history models to estimate which eligible families participated in a subsidized rental housing program and when. Answering these questions can inform efforts to improve program marketing and outreach, staffing and budgeting, triage, bias identification, as well as benchmarking and evaluation. One of six specifications clearly outperforms the others and understanding how will inform similar research pursuits. Although decision-relevant participation patterns are available in state administrative records, deciphering them is difficult for several well-known reasons. Participants enter and exit the eligible risk pool at different times, for different reasons, and at different rates. To answer our questions of when and whom, we restructure the data from calendar to relative months and then employ event history models designed to accurately estimate a complete hypothetical risk trajectory from observed spells of varying lengths, many of which ended before families took up the rental subsidy, (i.e., censored observation spells). We find that eligible parents most likely to take up the subsidy live in high-rent counties, have relatively strong recent work history, short prior adult lifetime TANF receipt, and medium-size families. Program take-up fell substantially during the COVID-19 pandemic. Contrasting the application of six parallel specifications, we find that a Royston-Parmar proportional hazard model achieves an exceptional balance between the descriptive accuracy of discrete time approaches and the elegance of Cox regression.

The foremost index of caregiving quality is child attachment, as supported by attachment theory. Research supports the relevance of early parenting interventions in improving child outcomes in attachment quality to promote public health because of their long-term effects on mental health and functioning. This study aimed at evaluating the impact on both parenting and child outcomes of the Parent-Child Psychological Support Programme® (PCPS), a community-based program individually tailored to parents and their infants during periodic center-based visits to promote attachment security. The evaluation involved two cohorts from "vulnerable populations" and used the resources embedded in the program design without interfering with the normal functioning of the service. From an evaluative research approach, the effects on mothers and children were assessed using a quantitative approach. Pre- and post-test measures (parenting questionnaires) and child attachment quality assessments through the Strange Situation Procedure were examined. The equivalence of the cohorts was verified and used as a baseline for parenting outcomes. PCPS participants demonstrated increased parental competence and self-efficacy, as well as reduced levels of parenting stress. Analysis of the two cohorts showed a significant difference in the number of visits and proxies for intervention, which were associated with the expected pre-post changes in parenting dimensions. The proportion of securely attached children was significantly higher in the "medium-high intervention" group than in the "no/low intervention" group (72.7% vs 54.5%). Furthermore, compared with international baselines, this proportion showed no differences in the "no-/low-intervention" group but demonstrated expected significant differences in the "medium-high intervention" group.

Gold and stocks, which are conventionally regarded as a safe haven and risk assets, respectively, exhibit complex interrelationships, with significant implications for financial risk management. This paper builds on the sentiment categorization proposed by Liang et al. (2020) to distinguish between private and public sector sentiment. The construction of sentiment indices for both sectors aims to allow the exploration of the heterogeneous effects of these sector-specific sentiments on the gold-stock market linkages in China under different market conditions. The empirical results demonstrate a notable asymmetry in the impact of market sentiment between the public and private sectors, with distinct manifestations in stable versus highly volatile market environments. Specifically, positive sentiment in the public sector tends to diminish the safe-haven function of gold, whereas positive sentiment in the private sector tends to reinforce it. This disparity becomes particularly evident during periods of extreme market volatility. Our findings not only underscore the diverse impacts of market sentiment but also provide novel insights into the importance of incorporating sector-specific sentiment when devising hedging strategies for specific industries.

In modern firms, conflicts may arise between the chief executive officer and board of directors due to the separation of ownership and operations. Such conflicts may weaken listed companies' performance in the short term or affect their development in the long term. Using data on Shanghai and Shenzhen A-share listed companies from 2013 to 2022, this study empirically examines the impact of internal control quality on boardroom backscratching. We find that internal control quality significantly mitigates boardroom backscratching by reducing agency costs and increasing analyst attention. This effect is more significant in firms where the management holds less power and the product market is highly competitive. Furthermore, the mitigating effects of internal control quality on boardroom backscratching effectively reduce the risk of the company's share price collapsing. Overall, this study enriches the literature on boardroom backscratching and internal control quality, and provides valuable references for stakeholders, including listed companies, to improve firm governance efficiency and, thus, maintain the stability of capital markets.

The information technology revolution has fundamentally altered company operations around the world. The Internet has significantly enhanced employee connectedness in the workplace, eclipsing the antiquated brick-and-mortar model. Nonetheless, as information technology advances, cyberbullying has grown in popularity in the professional environment. Cyberbullying is not geographically, temporally, or milieu-specific. Arguments concerning this ubiquitous danger have plagued scholars and professionals for several decades as they debated its conception, prevalence, and implications. The current research digs into many facets of cyberbullying to facilitate the creation of constructive policies and effectively manage the labor environment. Additionally, a few ideas and remedies are offered, as well as the future course of action for effectively addressing the crucial issue of cyberbullying.