Evaluation Review最新文献

With the continuous advancement of digital technology, traditional manufacturing and service industries are facing dynamic changes with diversified demand and business models. Digital transformation (DT) and dynamic business modeling have become increasingly necessary for companies to evaluate and prioritize better opportunities to improve strategic planning and productivity. This study demonstrates that the system dynamics modeling process, based on the principles of sustainable system development, can fully reflect the comprehensiveness of systems thinking and avoid the drawbacks of linear thinking failing to integrate various departments. The proposed model enables the comprehensive integration of technology, production, marketing, management, and capital, thereby identifying and testing opportunities for successful DT with minimal costs and budgetary constraints. A real-world case study demonstrates the value of simulation-based dynamic business modeling with scientific data analysis for evaluating DT strategies.

Research on evaluation has mapped the landscape of quantitative evaluation methods. There are far fewer overviews for qualitative methods of evaluation. We present a review of scholarly articles from five widely read evaluation research journals, examining the types of methods used and the transparency of their quality criteria. We briefly look at a large sample of 1070 articles and then randomly select 50 for in-depth study. We document a remarkable variety of qualitative methods, but some stand out: Case studies and stakeholder analysis, often combined with interview techniques. Articles rarely define and conceptualize their methods explicitly. This is understandable from a practical point of view, but it can make it difficult to critically interrogate findings and build systematic knowledge. Finally, we find that the transparency of qualitative criteria required in the literature is not always sufficient, which can hinder the synthesis of results.

High-performance computing platforms (HPCP) serve as a critical component of institutional research infrastructure at universities, enhancing innovation efficiency. However, there is an absence of empirical research quantifying their impacts. Acknowledging this void, a difference-in-differences (DID) approach is employed in this study to systematically evaluate the influence of HPCP on university innovation efficiency. The findings reveal that: (1) universities with access to HPCP experience a significant improvement in innovation efficiency, with heterogeneities observed across regions, institutional rankings and university types; (2) HPCP drives innovation efficiency by attracting and cultivating top-tier researchers, expanding the scope of research activities and enabling the production of high-quality research outputs; and (3) the relationship between HPCP and innovation efficiency exhibits non-linearity, influenced by the quality and volume of research outputs as well as the presence of elite researchers. The contributions of this study are threefold viz: (1) providing empirical evidence quantifying HPCP's role in university innovation efficiency; (2) elucidating the mechanisms through which HPCP enhances innovation; and (3) identifying the non-linear relationship between HPCP and innovation efficiency, offering nuanced insights for policymakers to engender targeted strategies.

This paper analyzes the effects of caseworker screening of doctors' medical sick-leave certificates. The analysis uses data from people appealing caseworkers' denial of their benefit claims. Caseworkers at four units made decisions on the appeals. The distribution of the cases to the four units was based on the appealing person's birth date. One of the units was much stricter than the others (7.8% approved in contrast to 16.1% for the others). This allows us to use birth date as an instrument to estimate the effects of being denied sickness benefits. We find that the denial of sickness benefits, on average, has (i) positive effects on the labor-market outcomes and (ii) no negative effects on health outcomes. As the more liberal units deny sickness benefits for most screened medical certificates, we conclude that caseworker screening is very important in separating meritorious from non-meritorious claims. Doctors' conflicting roles make it difficult for them to act in the best of interest of both their patients and society.

The difference in differences design is widely used to assess treatment effects in natural experiments or other situations where random assignment cannot, or is not, used (see, e.g., Angrist & Pischke, 2009). The researcher must make important decisions about which comparisons to make, the measurements to make, and perhaps the number of individuals whose data is included in each timepoint. Also, interpretation of any statistical results, particularly null results, is improved by understanding the sensitivity of the design. This paper describes methods for computing the statistical power for tests of treatment effects in the difference in differences design. We describe alternative approaches to the analysis of the design, show which are equivalent, and provide expressions for computing statistical power and determining minimum detectable effect sizes. We then discuss how these methods could be generalized to unbalanced designs, designs with covariates, and designs more than two timepoints including difference in difference in differences designs.

Environmental, social, and governance (ESG) considerations are garnering increasing interest from various stakeholders. Nevertheless, the impact of ESG evaluation policies-whether voluntary or mandatory-are complex, particularly in competitive markets. Consequently, this study develops a game-theoretic model to investigate the propensity of competing manufacturers to invest in ESG initiatives. This study demonstrates that ESG evaluations reduce information asymmetry regarding ESG levels in competitive dynamics. However, this phenomenon may lower manufacturers' incentives to invest in ESG. Interestingly, this study finds that although mandatory ESG evaluations boost the overall profitability of competing manufacturers, they do not invariably lead to enhanced ESG outcomes.

This study investigates how students engage with GenAI during a business data analysis assessment, drawing on Social Constructivist Theory and the human-AI co-agency model. Within the assessment, students used GenAI tools to support their data analysis and reflected on their experiences by comparing AI-generated and manually derived results. Thematic analysis of 258 students' reflection, triangulated with academic performance data, revealed four key themes: epistemic beliefs, functional cognitive engagement, reflective metacognitive learning, and Human-GenAI co-agency for strategic foresight. Students demonstrated distinct patterns of engagement across performance groups. Higher-performing students approached GenAI as a collaborative partner, engaging in iterative prompt refinement, demonstrating critical evaluation of outputs, and exhibiting strong ethical awareness. In contrast, lower-performing students often showed polarised epistemic beliefs, limited critical reflection, and minimal iteration - accepting or rejecting GenAI outputs prematurely. These findings highlight the role of scaffolded reflection and prompt engineering in enabling students to develop deeper analytical and evaluative capacities. By reconceptualising GenAI as an active co-learner rather than a passive tool, this study extends Social Constructivism perspectives to accommodate emerging forms of human-GenAI interaction. Drawing on rich, in-situ qualitative evidence embedded within an authentic learning context, it offers new insight into how students' beliefs and strategies shape their engagement with GenAI. The study also emphasises the need for differentiated pedagogical designs that cultivate AI literacy, narrow digital divide, and support ethical, adaptive use of GenAI in higher education.

This study is the first to examine the factors influencing the likelihood of graduates in Vietnam experiencing vertical, horizontal, and full job mismatches. This study employs secondary data from the Labor Force Survey for 2018-2022. Using a multinomial logit model, we identify the key determinants of job mismatch, such as field of study, gender, job sector, and geographic location. Graduates in Education and Pedagogy exhibit a lower likelihood of mismatches than those in almost all other fields. We also observe gender differences, where men were more likely than women to experience all types of mismatches. Additionally, graduates employed in the private and foreign direct investment sectors face higher risks of vertical and full mismatches, whereas urban residents are less likely to experience mismatches than their rural counterparts. Finally, the findings from the doubly robust estimates reveal that a wage penalty is associated with vertical and full mismatches, particularly for women, whereas a horizontal mismatch has a minimal impact on wages.

Since 2021, Australia's international education and migration policies have undergone significant changes. However, a critical gap remains in understanding how these policy shifts are experienced, interpreted, and evaluated by international students themselves. Drawing on in-depth interviews with Chinese, Indian and Vietnamese international students, this article examines how these cohorts make sense of and navigate the evolving landscape of Australia's international education policies. It unpacks their experiences of administrative delays, visa insecurity and escalating visa fees, constrained employment opportunities, and emotional uncertainty, and shows how these vary across national backgrounds. By centring student voices, the analysis moves beyond official policy rhetoric to explore how international education and migration governance is lived, evaluated, and internalised in students' everyday life. In doing so, the article contributes to a deeper understanding of how seemingly technical policy instruments function as technologies of affect, shaping not only international education and migration outcomes but also students' sense of belonging, self-worth, and future possibilities. Particularly, this article offers an original conceptual framework by introducing three new concepts to the international education literature: 'temporal bordering', 'aspirational compromise', and 'affective governance in international education' to illuminate how international education-migration policies shape international students' experiences, aspirations, emotions, and sense of belonging. It reconceptualises international education governance as relational and affective, moving beyond macro-level policy analysis to highlight the affective dimensions of students' experiences with shifting geopolitical and policy contexts.

Various forms of housing price and rent control policies are implemented in many countries, and finding their impacts is an important issue. Over 2019-2023, the South Korean government announced a policy to put a ceiling on housing prices in some regions of Seoul, and then subsequently implemented, strengthened, weakened, and finally abolished the policy. This is a rather complicated scenario for a policy, and the goal of this paper is to assess the effects of the policy and its changes with difference in differences (DD). We establish a detailed DD-analysis protocol, employing diverse forms of DD. Applying the protocol where a systematic difference in the untreated outcome trajectories of the treated and control groups is allowed, we assess the policy impacts. We find that, despite the active involvement of the government in the housing market, the overall effect is about a 4-5% decline in housing prices in Seoul.