Evaluation and Program Planning最新文献

Evaluators are frequently asked to evaluate educational interventions that are one-time events. The Retrospective Pretest (RPT) methodology is well suited for these circumstances and the evaluation literature is replete with discussions about the pros and cons of this approach. In RPT, program participants rate their attitudes or knowledge now, and also rate how they were before participation in the intervention. The difference between now and before ratings constitutes the measure of change. Little published literature exists about whether the layout of RPT items within the evaluation questionnaire yields different results. This study compared six different layouts using a sample of 1941 caregivers who participated in one of 96 training workshops. The layouts were compared on inattentiveness, unexpected decline in perceived knowledge, and the degree of before-now change. Findings show that design matters. The best performing layout is where items are placed in the center with before response options on the left of the page and the now response options on the right. Results point to the need for evaluators to pay attention to RPT layout, and for the field to establish criteria for assessing survey layout quality. The unexpectedly high rate of inattentiveness calls for evaluators to pay more attention identifying and addressing this in their survey data.

In this study we use personal network analysis to examine the social relationships taking place throughout programme implementation. Previous literature on psychosocial intervention has used network analysis techniques to examine: (a) the interaction between participants, (b) the facilitators' link to the intervention target group, (c) the transfer of knowledge between experts and facilitators, and (d) the interaction of facilitators with each other. However, there has been little research on how facilitators connect with other figures in their organisational context, impacting both intervention fidelity and the fit of the programme to the local context. In this study we combine the analysis of personal networks with qualitative interviews with 102 teachers in 72 schools in Barranquilla (Colombia), with whom we describe the implementation of the psychoeducational programmes Pisotón and Metodologías Flexibles. The results show that programme implementation networks not only rely on facilitators but also on the contribution of the schools’ director of studies, the coupling with the regular classroom teachers, and occasional collaborations from the rest of the school staff. After conducting a cluster analysis, we detected the existence of two types of personal networks, some based on the functioning of highly cohesive teams and others with a greater level of centralisation around the head of studies. Implementation networks not only enable the implementation of programme activities, but also the integration of the programme into educational organisations. In the discussion we reflect on how the analysis of facilitators' personal networks can be used to improve the process of programme implementation.

Learning design has a multifaceted nature requiring a range of course- and institutional considerations. Analyzing the decision criteria’s influence on research-based learning design helps understand the causes of the success/failure of the approach in achieving the teaching goals to improve the study programs. This study explores the interrelationship between the curricular and pedagogical criteria for research-based learning design decisions at the undergraduate level. For this purpose, the DEcision-MAking Trial and Evaluation Laboratory (DEMATEL) method is used to systematically analyze the decisive criteria and their causal relationships. Feedback from education professionals and university professors from Scandinavian universities is used to validate the pedagogy decision framework and provides input into the DEMATEL method. The student's role in the course is identified as the central criterion, featuring the highest level of interactions in the network of curricular and pedagogical decision criteria. Results are supportive of the identified institutional and course-specific criteria as prerequisites for the study outcomes.

The concerns of students and young people have been on the rise in recent years. Recent events such as the Covid-19 pandemic or the invasion of Ukraine have contributed to this. Mental health, the high cost of living, the need to re-evaluate the way we work or the fight against climate change are among the main concerns of young people. The aim of this work is to find out the level of concern of university students about these problems that can condition their current life and to analyze whether these concerns can affect their academic performance, also distinguishing according to gender and age. To this end, a nationwide survey was carried out among this group in Spain. This work is the first of its kind to be carried out in Spain. The results obtained may be of great use to both educational institutions and government agents, with a view to offering policies that will improve the educational level and welfare of university students.

Implementing trauma-informed care in a special education environment serving youth from historically marginalized communities with high levels of exposure to potentially traumatic events (PTEs) requires a systematic tiered approach consistent with public health guidelines. Little is known about the implementation of this framework in special education settings where youth have significant emotional and behavioral difficulties. To address this need, a consultant-community partnership was forged between a hospital providing mental health services and a therapeutic day school that serves a special education cooperative. The current case study explores the design and implementation of a three-tiered model of trauma-informed care in a special education setting. This study will address the specific practices implemented at each tier, discuss successes and challenges, and summarize future directions for research, practice, and policy.

The COVID-19 pandemic has necessitated various unavoidable social restrictions, leading to questions about the effectiveness of public emergency interventions and their impact economic growth. Block et al. (2020) conducted a notably study using an agent-based model to evaluate policies for reducing contact and demonstrated how choices in contact behavior can influence the rate and spread of the virus. However, their approach did not consider the economic consequences of these social restrictions. In response, we propose a set of strategies for governments to plan and evaluate policies during emergencies, aiming to contain infections while minimizing negative economic consequences. Our results indicate that there is no trade-off between containment strategies and economic output loss, making containment measures necessary policy instruments. However, potential trade-offs do emerge when selecting the most effective strategy. In this context, we propose and evaluate various policy alternatives to extreme "social distancing" measures, which can partially restore essential social interactions while preventing economic disasters induced by productivity losses.

Understanding that good decision-making is paramount for the success of an organization and recognizing that strategic decision-making inside museums is a topic that has been over-passed by researchers, this paper intends to contribute to the relevance of the subject through the implementation of an innovative tool. Decision-making in museums is normally focused on the intuition, subjectivity, and experience of the curator, who brings all his knowledge to the exhibition programming. However, museums’ management and environment are constantly changing, thus the application of this tool would aim to make decisions in a more democratic, transparent, inclusive, and accurate manner. Besides, it will be easier to understand how distinct subjects can work together, demonstrating successful results to improve the decision-making process when programming and measuring temporary exhibition seasons in museums. This paper describes the design and implementation of the Multi-Attribute Utility Theory (MAUT) for the Museu Nacional d′Art de Catalunya (MNAC). It also analyzes and evaluates the decision-making process when scheduling exhibitions for a season.

Current understanding of what sustains public health programs is derived primarily from evaluations of programs delivered for only a few years. To improve program planning and knowledge on program sustainment, we examined real-world population programs that have been delivered for >2 years. Our focus was physical activity and nutrition programs for obesity prevention. We identified programs through published literature and searched through publicly available information on the theoretical factors determining sustainment. We reviewed 90 programs and explored intervention characteristics, provider, delivery and process level factors, and the outer environment influences.

Programs were sustained on average 15 years and more frequently characterised as behavioural change skills programs targeting children, funded by non-government organisations and delivered by community organisations. Most programs had undergone some modification (n = 55). Differences between programs were observed across key factors: Programs sustained 15+ years were mainly behaviour skills programs (n = 21); and characterised by interactive designs (i.e., face-to-face or digital) over static innovations (i.e., print material) compared with those sustained <15 years (p = 0.024). While government funding supported many of the programs (n = 45;), those sustained 15+ years were more likely to have been commercially funded (p = 0.044); and were less likely to have current community involvement (p = 0.013). Differences in ownership and funding were also observed across countries. While multiple factors may influence program sustainment, longer delivered programs where distinguished by their program characteristics and outer contextual factors, suggesting such factors may have an important role in sustaining programs in the longer-term.

The massive earthquakes experienced in August and November 1999 affected thousands of people in the Marmara region, the most densely populated and industrialized part of Turkey. The humanitarian and economic cost was so enormous, and these earthquakes have changed the Turkish disaster management system and the Turkish people’s approach to disasters. Marmara earthquakes are also considered as a milestone in the provision of psychosocial services for disaster victims. This paper aims to review the psychosocial interventions targeting children, adolescents, and their families after the 1999 earthquakes in Turkey. The progression from initial responses to more organized psychosocial interventions is outlined. Conducting the interventions at schools has ensured that thousands of children, teachers, and parents are reached in the most efficient and effective way possible. The significance of the school context in designing psychosocial interventions is highlighted and implications of the lessons learned for traumatic experiences of children and parents are also explored. It is evaluated that these inferences obtained from the Marmara earthquake in Turkey can be used in disasters around the world.

To improve immigrants’ integration, Korea Immigration & Integration Program (KIIP) has been delivered since 2009 in South Korea (hereafter Korea), yet little research has empirically estimated its net effect. Hence, the main aim of this study is to empirically and directly evaluate its net effect, using an Inverse-probability weighted analytical technique controlling selection bias that may distort its net effect. Utilizing the “Estimating social and economic effect of KIIP”, published by the Ministry of Justice, Korea in 2017, this study selects immigrants including participants (n = 800) and nonparticipants (n = 214). The findings indicate that the effect of KIIP is dubious. Specifically, KIIP plays a role in raising the level of naturalization of immigrants, accepting Korean culture, and satisfying their lives. However, there are statistically non-significant differences in employment, monthly income, keeping home country’s culture, social networks, Korean language skills, discrimination, and the sense of belonging to the Korean society between participants and nonparticipants. This shows both the effectiveness and ineffectiveness of KIIP. To find a way to improve integration, this study suggests strengthening advantages and offsetting disadvantages, in managing KIIP.