Evaluation Review最新文献

This study is the first to examine the factors influencing the likelihood of graduates in Vietnam experiencing vertical, horizontal, and full job mismatches. This study employs secondary data from the Labor Force Survey for 2018-2022. Using a multinomial logit model, we identify the key determinants of job mismatch, such as field of study, gender, job sector, and geographic location. Graduates in Education and Pedagogy exhibit a lower likelihood of mismatches than those in almost all other fields. We also observe gender differences, where men were more likely than women to experience all types of mismatches. Additionally, graduates employed in the private and foreign direct investment sectors face higher risks of vertical and full mismatches, whereas urban residents are less likely to experience mismatches than their rural counterparts. Finally, the findings from the doubly robust estimates reveal that a wage penalty is associated with vertical and full mismatches, particularly for women, whereas a horizontal mismatch has a minimal impact on wages.

Since 2021, Australia's international education and migration policies have undergone significant changes. However, a critical gap remains in understanding how these policy shifts are experienced, interpreted, and evaluated by international students themselves. Drawing on in-depth interviews with Chinese, Indian and Vietnamese international students, this article examines how these cohorts make sense of and navigate the evolving landscape of Australia's international education policies. It unpacks their experiences of administrative delays, visa insecurity and escalating visa fees, constrained employment opportunities, and emotional uncertainty, and shows how these vary across national backgrounds. By centring student voices, the analysis moves beyond official policy rhetoric to explore how international education and migration governance is lived, evaluated, and internalised in students' everyday life. In doing so, the article contributes to a deeper understanding of how seemingly technical policy instruments function as technologies of affect, shaping not only international education and migration outcomes but also students' sense of belonging, self-worth, and future possibilities. Particularly, this article offers an original conceptual framework by introducing three new concepts to the international education literature: 'temporal bordering', 'aspirational compromise', and 'affective governance in international education' to illuminate how international education-migration policies shape international students' experiences, aspirations, emotions, and sense of belonging. It reconceptualises international education governance as relational and affective, moving beyond macro-level policy analysis to highlight the affective dimensions of students' experiences with shifting geopolitical and policy contexts.

Various forms of housing price and rent control policies are implemented in many countries, and finding their impacts is an important issue. Over 2019-2023, the South Korean government announced a policy to put a ceiling on housing prices in some regions of Seoul, and then subsequently implemented, strengthened, weakened, and finally abolished the policy. This is a rather complicated scenario for a policy, and the goal of this paper is to assess the effects of the policy and its changes with difference in differences (DD). We establish a detailed DD-analysis protocol, employing diverse forms of DD. Applying the protocol where a systematic difference in the untreated outcome trajectories of the treated and control groups is allowed, we assess the policy impacts. We find that, despite the active involvement of the government in the housing market, the overall effect is about a 4-5% decline in housing prices in Seoul.

The implementation of transport infrastructure projects is a key priority of the Russian government. Large funds are allocated from the budget for road construction to ensure connectivity across the country's regions. Evaluating their economic feasibility is a key step in managing transportation projects. A social discount rate is required to estimate the present value of net social benefits. This study aims to provide a reasonable methodology for estimating the social discount rate for long-term transport projects in Russia. To determine this rate, we use the social opportunity cost of capital approach (SOC) and the social rate of time preference approach (SRTP). For 0-30 years, we recommend SRTP = 3.2% in the general case and SOC = 7.4% to attract private funds. The confidence interval for the SRTP ranges from 1.7% to 4%. For 31-60 years, decision-makers should use 3.2%, and a zero-discount rate for the later period. The proposed solutions will be useful for other post-Soviet countries preparing large-scale infrastructure projects.

Sometimes, public policy outcomes disappoint when unintended consequences arise. In many such cases, the problems might be traced back to poor reasoning. For most of antiquity, logic was considered the core element for successful human endeavour. In this work, we argue that Aristotelian logic - specifically, the syllogism - remains highly relevant and could offer significant benefits for the development of sound public policy. To demonstrate the value of logic for contemporary public policymaking, we first provide an accessible explanation of the practical syllogism. Following this we set out our method for testing the value of syllogistic reasoning against an example of real-world public policymaking. Thereafter, we test both the validity and truth of the apparent syllogism. We conclude that the use of a practical syllogism would have prevented unintended harm from arising in the instance under consideration and also offer our thoughts around generalisability and future research directions.

Along with discussing bibliometric analyses' limitations and potential biases, this paper addresses the growing need for comprehensive guidelines in evaluating bibliometric research by providing systematic frameworks for both peer reviewers and readers. While numerous publications provide guidance on implementing bibliometric methods, there is a notable lack of frameworks for assessing such research, particularly regarding performance analysis and science mapping. Drawing from an extensive review of bibliometric practices and methodological literature, this paper develops structured evaluation frameworks that address the complexity of modern bibliometric analysis, introducing the VALOR framework (Verification, Alignment, Logging, Overview, Reproducibility) for assessing multi-source bibliometric studies. The paper's key contributions include comprehensive guidelines for evaluating data selection, cleaning, and analysis processes; specific criteria for assessing conceptual, intellectual, and social structure analyses; and practical guidance for integrating performance analysis with science mapping results. By providing structured frameworks for reviewers and practical guidelines for readers to interpret and apply bibliometric insights, this work enhances the rigor of bibliometric research evaluation while supporting more effective peer review processes and research planning. The paper also discusses potential areas for further development, including the integration of qualitative analysis with bibliometric data and the advancement of field-normalized metrics, ultimately aiming to support authors, reviewers, and readers in navigating the complexities of bibliometrics and enhancing the meaningfulness of bibliometric research.

We study the external validity of instrumental variable estimation. The key assumption we impose for external validity is conditional external unconfoundedness among compliers, which means that the treatment effect and target selection are independent among compliers conditional on covariates. We study this assumption with a case study about the impact of solid-fuel usage on women's average cooking time. Among the six countries examined, we find no statistical evidence that the assumptions required for external validity are violated for four countries (Ethiopia, Honduras, Kenya, and Zambia), but in Cambodia and Nepal, we find low external validity. These results provide suggestive evidence that the assumptions required for external validity are violated for these two countries.

Currency shocks and the stability of the global financial system play crucial roles in international trade and investment. However, there is a lack of literature that analyses these relationships, particularly in newly industrialized Asian countries. This study fills this gap by focusing on Thailand, India, and China. Employing the exchange rate (ER) and the OFR financial stress index (FSI), we conduct a comprehensive analysis. Our results show distinct relationships among these variables. In Thailand, the ER has a long-term impact on the FSI, accompanied by positive feedback between shocks. In India, the ER is highly sensitive to the FSI, with the FSI consistently triggering negative ER shocks. In China, the ER affects the FSI only in the short run, and this impact may be negatively correlated. Additionally, increased financial stress may also increase the Chinese ER on average. These findings not only reveal the diverse economic development patterns in newly industrialized countries but also offer valuable insights for policy-makers to formulate more targeted economic policies.

This study demonstrates how to evaluate a university-wide online course designed to support student transition into university by using Propensity Score Matching (PSM) and Doubly Robust Estimation (DRE). Using data from seven academic years, from 2016/17 to 2022/23, with more than 28,000 students, we examine whether enrolment in this optional pre-arrival course affects first-year pass rates. We also conducted additional analyses to compare outcomes from the year before and after the course's implementation, as well as to examine these patterns across recent cohorts to potentially account for contextual changes over time. Results indicate that enrolled students show a 6.2 percentage point increase in the likelihood of passing Year 1, controlling for factors including sex, domicile, age, ethnicity, disability and socioeconomic status. We demonstrate how utilising existing institutional data can potentially strengthen evidence of impact for centralised initiatives and conclude with reflections on the use of such institutional data and matching techniques and their viability for future evaluations.

Rapid approaches are essential when resources are limited and when findings are required in real-time to inform decisions. Limitations exist in their design and implementation, which can lead to a reduced level of trust in findings. This review sought to map the methods used across rapid evaluations and research to facilitate timeliness and support the rigour of studies. Four scientific databases and one search engine were searched between 11-16th August 2022. Screening led to the inclusion of 169 articles that provided a much-needed repository of methods that can be used during the design and implementation of rapid studies to improve their trustworthiness. No reporting guidelines specific to rapid research or evaluation were identified in the literature, we therefore suggest that this repository of methods informs the development of transparent reporting standards for future rapid research and evaluation.