Evaluation Review最新文献

Sometimes, public policy outcomes disappoint when unintended consequences arise. In many such cases, the problems might be traced back to poor reasoning. For most of antiquity, logic was considered the core element for successful human endeavour. In this work, we argue that Aristotelian logic - specifically, the syllogism - remains highly relevant and could offer significant benefits for the development of sound public policy. To demonstrate the value of logic for contemporary public policymaking, we first provide an accessible explanation of the practical syllogism. Following this we set out our method for testing the value of syllogistic reasoning against an example of real-world public policymaking. Thereafter, we test both the validity and truth of the apparent syllogism. We conclude that the use of a practical syllogism would have prevented unintended harm from arising in the instance under consideration and also offer our thoughts around generalisability and future research directions.

Along with discussing bibliometric analyses' limitations and potential biases, this paper addresses the growing need for comprehensive guidelines in evaluating bibliometric research by providing systematic frameworks for both peer reviewers and readers. While numerous publications provide guidance on implementing bibliometric methods, there is a notable lack of frameworks for assessing such research, particularly regarding performance analysis and science mapping. Drawing from an extensive review of bibliometric practices and methodological literature, this paper develops structured evaluation frameworks that address the complexity of modern bibliometric analysis, introducing the VALOR framework (Verification, Alignment, Logging, Overview, Reproducibility) for assessing multi-source bibliometric studies. The paper's key contributions include comprehensive guidelines for evaluating data selection, cleaning, and analysis processes; specific criteria for assessing conceptual, intellectual, and social structure analyses; and practical guidance for integrating performance analysis with science mapping results. By providing structured frameworks for reviewers and practical guidelines for readers to interpret and apply bibliometric insights, this work enhances the rigor of bibliometric research evaluation while supporting more effective peer review processes and research planning. The paper also discusses potential areas for further development, including the integration of qualitative analysis with bibliometric data and the advancement of field-normalized metrics, ultimately aiming to support authors, reviewers, and readers in navigating the complexities of bibliometrics and enhancing the meaningfulness of bibliometric research.

We study the external validity of instrumental variable estimation. The key assumption we impose for external validity is conditional external unconfoundedness among compliers, which means that the treatment effect and target selection are independent among compliers conditional on covariates. We study this assumption with a case study about the impact of solid-fuel usage on women's average cooking time. Among the six countries examined, we find no statistical evidence that the assumptions required for external validity are violated for four countries (Ethiopia, Honduras, Kenya, and Zambia), but in Cambodia and Nepal, we find low external validity. These results provide suggestive evidence that the assumptions required for external validity are violated for these two countries.

Currency shocks and the stability of the global financial system play crucial roles in international trade and investment. However, there is a lack of literature that analyses these relationships, particularly in newly industrialized Asian countries. This study fills this gap by focusing on Thailand, India, and China. Employing the exchange rate (ER) and the OFR financial stress index (FSI), we conduct a comprehensive analysis. Our results show distinct relationships among these variables. In Thailand, the ER has a long-term impact on the FSI, accompanied by positive feedback between shocks. In India, the ER is highly sensitive to the FSI, with the FSI consistently triggering negative ER shocks. In China, the ER affects the FSI only in the short run, and this impact may be negatively correlated. Additionally, increased financial stress may also increase the Chinese ER on average. These findings not only reveal the diverse economic development patterns in newly industrialized countries but also offer valuable insights for policy-makers to formulate more targeted economic policies.

This study demonstrates how to evaluate a university-wide online course designed to support student transition into university by using Propensity Score Matching (PSM) and Doubly Robust Estimation (DRE). Using data from seven academic years, from 2016/17 to 2022/23, with more than 28,000 students, we examine whether enrolment in this optional pre-arrival course affects first-year pass rates. We also conducted additional analyses to compare outcomes from the year before and after the course's implementation, as well as to examine these patterns across recent cohorts to potentially account for contextual changes over time. Results indicate that enrolled students show a 6.2 percentage point increase in the likelihood of passing Year 1, controlling for factors including sex, domicile, age, ethnicity, disability and socioeconomic status. We demonstrate how utilising existing institutional data can potentially strengthen evidence of impact for centralised initiatives and conclude with reflections on the use of such institutional data and matching techniques and their viability for future evaluations.

Rapid approaches are essential when resources are limited and when findings are required in real-time to inform decisions. Limitations exist in their design and implementation, which can lead to a reduced level of trust in findings. This review sought to map the methods used across rapid evaluations and research to facilitate timeliness and support the rigour of studies. Four scientific databases and one search engine were searched between 11-16th August 2022. Screening led to the inclusion of 169 articles that provided a much-needed repository of methods that can be used during the design and implementation of rapid studies to improve their trustworthiness. No reporting guidelines specific to rapid research or evaluation were identified in the literature, we therefore suggest that this repository of methods informs the development of transparent reporting standards for future rapid research and evaluation.

By exploiting variations in Vietnamese districts affected by intense bombings, I establish a causal relationship: residents in heavily bombed areas are more willing to spend on worship practices. This relationship varies among regions, with the primary channel for this effect being the density of graveyards commemorating war martyrs.

Against the backdrop of China's new urbanization and regional coordination strategies, the social integration of rural-to-urban migrants has become an increasingly critical indicator of state governance modernization and institutional performance. Considering rural migrants in Jilin Province as the research subject, particularly in the context of its agricultural revitalization and urbanization policies, this study constructs an "embedding-activation" theoretical framework grounded in institutional theory and social capital theory. It proposes a mechanism in which formal institutions are embedded within informal institutions and generate effective governance outcomes through cognitive transformation facilitated by social capital. Empirically, the study applies structural equation modeling to identify the direct and indirect pathways through which policy cognition, social trust, identity, and economic integration influence social integration. Additionally, machine learning methods are employed to reveal the marginal contributions and nonlinear transition characteristics of individual variables, bridging the gap between causal explanation and predictive accuracy. The results demonstrate that social capital functions as a cognitive trigger for activating institutional efficacy and plays a key mediating role in promoting migrants' social integration. The success of institutional incentives depends on the accumulation of individual cognition and the threshold effects of trust and identity mechanisms, resulting in a behavioral shift from passive acceptance to active engagement. This dynamic promotes the expansion of scope and deepening of structure in social integration. By adopting an institution-cognition-behavior analytical lens, this study enriches the micro-foundations of institutional theory and provides support for enhancing the evaluation, responsiveness, and effectiveness of migration.

University students are increasingly turning to Generative Artificial Intelligence (GenAI) tools for help with their academic assessments, which has prompted major concerns relating to academic integrity. While Universities globally are building guidance on good practice in the use of GenAI, there is a lack of empirical understanding of student perceptions of what ethical and equitable use means to them. Developing insight into student understanding of GenAI is important in enabling institutions to offer more appropriate training and support to encourage good practice in assessment, not just from the perspective of avoiding malpractice, but in identifying ethical opportunities for integrating the technology as a useful tool. Using the GenAI literacy framework as the theoretical foundation, this paper analysed students' GenAI usage in academic assessments within the UK context. Data were collected from 80 participants through focus groups conducted by four UK institutions. Our findings show that students are exploring the potential of GenAI as a tool and are beginning to understand where the ethical boundaries might sit. They are keen to use the technology to support their learning but have significant concerns about locating that boundary between good practice and that which exposes them to accusations of cheating, or which limits their own learning. The paper reinforces the importance of providing GenAI literacy training to university students, so they may develop a better understanding of how GenAI can support learning processes in an ethical way.

The major global challenges of our time-such as climate change, environmental degradation, rising inequality, and the emergence of disruptive technologies-demand interdisciplinary research to generate effective solutions. A clear understanding of value is essential for guiding socio-cultural and economic transitions to address these issues. Despite numerous attempts to define value, existing approaches remain inconsistent across disciplines and lack a comprehensive framework. This paper introduces a novel perspective on value through the lens of granular interaction thinking theory, proposing an informational entropy-based notion of value. Grounded on quantum mechanics, Shannon's information theory, and the mindsponge theory, this framework integrates both subjective and objective considerations and is highly compatible with interdisciplinary research. The informational entropy-based notion of value effectively bridges diverse concepts of value, including use and exchange value in economics, personal values in psychology, and cultural, moral, ethical, and esthetic values in society. By offering a unifying perspective, granular interaction thinking theory provides a valuable framework for translating insights from quantum mechanics into socio-cultural, economic, and psychological contexts, enriching theoretical discourse and enhancing analytical effectiveness.