Evaluation Review最新文献

Most existing solutions to the current replication crisis in science address only the factors stemming from specific poor research practices. We introduce a novel mechanism that leverages the experts' predictive abilities to analyze the root causes of replication failures. It is backed by the principle that the most accurate predictor is the most qualified expert. This mechanism can be seamlessly integrated into the existing replication prediction market framework with minimal implementation costs. It relies on an objective rather than subjective process and unstructured expert opinions to effectively identify various influences contributing to the replication crisis.

The impact of sustainable mobility may be considered through the mode of non-motorization, which may vary by spatial environmental conditions, especially accessibility by active mobility in a necessarily safe, comfortable, and pleasant atmosphere. Sustainable mobility comprises willingness to allow commuting behaviors on non-motorized travel for safety and green ethics. These chiefly participate to create livable places, improve health and well-being, and promote quality of life and economic growth. By contrast, an impoverished street environment decreases the possibility of active mobility and relies on motorization. It is essential to explore the perception of road users through diverse environmental features of street networks and sidewalks directly affecting road user behavior and preferences about motorization and non-motorization. This study evaluated multi-indicators of sustainable urban mobility (SUM) for a group of road users, with data gathered by questionnaire-based survey. 1,998 Pathum Thani Province inhabitants in Thailand were asked to evaluate their neighborhood characteristics by SUM indicators. In addition to different social dimensions and socioeconomic backgrounds, transportation characteristics among diverse trip makers and different perceptions of mobility indicators were classified by factor analysis using principal components and spatial visualization. Results were that traffic management, including intersection controls and signage, as well as information, sidewalk, management, and personal mobility choices, is a key strategy to improve sustainable mobility. This is achieved by tailoring solutions to achieve a modal shift prioritized over other supportive facilities.

When individuals are released from prison, they typically enter a period of post confinement community supervision. While under community supervision, their behaviors are subject to special conditions requiring them to report to supervisors and prohibiting certain behaviors such as drug and alcohol use. Many supervisees are returned to prison because they violate those special conditions, or because they commit minor crimes that would not result in prison were they not being supervised. But others are returned to prison for serious new crimes. We distinguish the two as nuisance behaviors (the former) and pernicious behaviors (the latter). Our research applies competing events survival analysis to distinguish a structural model that accounts for nuisance behaviors from a structural model that accounts for pernicious behaviors. We demonstrate that returning offenders to prison for technical violations and minor crimes may reduce the incidence of major crimes because the occurrence of nuisance behaviors and pernicious behaviors are highly correlated. Our findings support the theory that nuisance behaviors signal the likelihood of pernicious behaviors.

Randomized experiments involving education interventions are typically implemented as cluster randomized trials, with schools serving as clusters. To design such a study, it is critical to understand the degree to which learning outcomes vary between versus within clusters (schools), specifically the intraclass correlation coefficient. It is also helpful to anticipate the benefits, in terms of statistical power, of collecting household data, testing students at baseline, or relying on administrative data on previous cohorts from the same school. We use data from multiple cluster-randomized trials in four Latin American countries to provide information on the intraclass correlations in early grade literacy outcomes. We also describe the proportion of variance explained by different types of covariates. These parameters will help future researchers conduct statistical power analysis, estimate the required sample size, and determine the necessity of collecting different types of baseline data such as child assessments, administrative data at the school level, or household surveys.

This article describes a conceptual and empirical approach for estimating a human capital production function of child development that incorporates mother- or child-fixed effects. The use of mother- or child-fixed effects is common in this applied economics literature, but its application is often inconsistent with human capital theory. We outline the problem and demonstrate its empirical importance with an analysis of the effect of Head Start and preschool on child and adult outcomes. The empirical specification we develop has broad implications for a variety of applied microeconomic analyses beyond our specific application. Results of our analysis indicate that attending Head Start or preschool had no economically or statistically significant effect on child or adult outcomes.

In this paper, based on the established Vitae Researcher Development Framework (RDF), we introduce a new framework, tailor-made specifically for Vietnamese educational researchers, namely, Vietnam's Framework for Educational Researchers (VFER). VFER is expected to serve as a tool for Vietnamese educational researchers to self-evaluate their skills and support them in developing their career qualities from junior to senior career ladders. The framework includes four domains with ten subdomains and 28 indicators. To date, VFER has been implemented in some Vietnamese universities of pedagogical education. We expect that other research fields will look to VFER as a reference to build their own research capacity framework.

Gold and stocks, which are conventionally regarded as a safe haven and risk assets, respectively, exhibit complex interrelationships, with significant implications for financial risk management. This paper builds on the sentiment categorization proposed by Liang et al. (2020) to distinguish between private and public sector sentiment. The construction of sentiment indices for both sectors aims to allow the exploration of the heterogeneous effects of these sector-specific sentiments on the gold-stock market linkages in China under different market conditions. The empirical results demonstrate a notable asymmetry in the impact of market sentiment between the public and private sectors, with distinct manifestations in stable versus highly volatile market environments. Specifically, positive sentiment in the public sector tends to diminish the safe-haven function of gold, whereas positive sentiment in the private sector tends to reinforce it. This disparity becomes particularly evident during periods of extreme market volatility. Our findings not only underscore the diverse impacts of market sentiment but also provide novel insights into the importance of incorporating sector-specific sentiment when devising hedging strategies for specific industries.

The foremost index of caregiving quality is child attachment, as supported by attachment theory. Research supports the relevance of early parenting interventions in improving child outcomes in attachment quality to promote public health because of their long-term effects on mental health and functioning. This study aimed at evaluating the impact on both parenting and child outcomes of the Parent-Child Psychological Support Programme® (PCPS), a community-based program individually tailored to parents and their infants during periodic center-based visits to promote attachment security. The evaluation involved two cohorts from "vulnerable populations" and used the resources embedded in the program design without interfering with the normal functioning of the service. From an evaluative research approach, the effects on mothers and children were assessed using a quantitative approach. Pre- and post-test measures (parenting questionnaires) and child attachment quality assessments through the Strange Situation Procedure were examined. The equivalence of the cohorts was verified and used as a baseline for parenting outcomes. PCPS participants demonstrated increased parental competence and self-efficacy, as well as reduced levels of parenting stress. Analysis of the two cohorts showed a significant difference in the number of visits and proxies for intervention, which were associated with the expected pre-post changes in parenting dimensions. The proportion of securely attached children was significantly higher in the "medium-high intervention" group than in the "no/low intervention" group (72.7% vs 54.5%). Furthermore, compared with international baselines, this proportion showed no differences in the "no-/low-intervention" group but demonstrated expected significant differences in the "medium-high intervention" group.

Official development agencies are increasingly supporting civil society lobby and advocacy (L&A) to address poverty and human rights. However, there are challenges in evaluating L&A. As programme objectives are often to change policies or practices in a single institution like a Government Ministry, L&A programmes are often not amenable to large-n impact evaluation methods. They often work in strategic partnerships to foster change; hence, contribution may be a more relevant evaluation question than attribution. Small-n qualitative approaches are available to measure the effectiveness of L&A which use the theory of change as their analytical framework. We conducted a meta-evaluation of 36 evaluations of multi-component international programmes to support civil society L&A across Asia, Africa and Latin America, comprising the majority of programmatic support from one international donor. We assessed the confidence in causal claims in the evaluations using a new tool that we developed. Assessments of the contribution of the programmes to the changes in outcomes were not provided in many of the evaluations, nor were predictable sources of bias addressed. Given that L&A programmes are likely to adopt an influencing approach where many different inside-track and outside-track engagement objectives, opportunities and strategies are attempted, many of which might be expected to fail, there appeared to be a clear bias in the evaluations towards reporting outcomes that were achieved, ignoring those that were not. We provide guidance on how to improve the design, conduct and reporting of small-n qualitative evaluations of aid effectiveness.

Performance management in university-based scientific research institutions is essential for driving reform, advancing education quality, and fostering innovation. However, current performance evaluation models often focus solely on research indicators, neglecting the critical interdependence between the education and research systems. This oversight leads to inefficiencies in resource allocation and an underestimation of overall institutional performance, particularly in universities with varying development levels. To address these deficiencies, this study introduces two innovative two-stage data envelopment analysis models: the group-frontier and meta-frontier evaluation models. The findings are validated using data from 144 universities across China. They demonstrate that the proposed models effectively mitigate the underestimation of efficiency in traditional models and accurately reflect the intertwined nature of university subsystems and the disparities in university development. These results offer valuable insights for improving the performance of scientific research institutions and informing the strategic decisions of university administrators and government education departments.