Evaluation and Program Planning最新文献

As an educational aid, the Korean government has developed and funded a graduate-level global information telecommunications technology program (ITTP) for ICT officers in developing countries. Questions have arisen about whether the program can achieve its goals and obtain mutual benefits between Korea and participating nations. Using a total quality management (TQM) framework as a unique case analysis, we evaluated ITTP’s resources and performance from 2006 to 2014. Our results show that ITTP has focused on the quantitative promotion of students and alums from various countries and their learning of Korea-centric ICT, policies, and culture. In particular, the Korean government’s lax evaluation of ITTP performance has led to weak responsibility, ambiguous task identity, missed opportunities for self-improvement, and low participation among students and stakeholders. These findings highlight the need for graduate programs and their stakeholders to reinforce internal and external evaluations of the program’s performance and provide feedback for reciprocal development.

TVET teachers need comprehensive competence in response to the societal change of digitalization. Comparing to well-developed preparation programs in general education, the training framework for TVET teacher still need to be examined, especially in school-based TVET system. This study aims to propose a teaching competence framework and investigates its fitness and effectiveness in on-service training. A training program was conducted corresponding to it in a case school in China. The program achieves significant improvement in competence by using pretest and posttest measure, and receives satisfactory feedback from a survey among participating teachers. Factors which may affect training operation and outcome are discussed.

Objectives

We explored views of care home managers when introducing PROCUIDA-Demencia a mixed-methods two-arm cluster randomised controlled pilot and clinical outcomes study aiming to optimise dementia care by introducing psychosocial interventions to reduce antipsychotic medication in care homes.

Method

We undertook secondary analysis of pre and post in-depth interviews conducted in summer 2018 with not-for-profit care home managers in Mexico who were allocated to the intervention group. Transcribed data were thematically analysed. Themes were mapped out with RE-AIM quality appraisal framework (Reach, Effectiveness, Adoption, Implementation and Maintenance) as preliminary evaluation to identify practice and future intervention development and evaluation.

Results

Two pre- and three post-intervention themes were constructed. Participants reported measurable positive impact; one home built a new specialist dementia care unit and others hired a psychologist and psychiatrist to sustain the changes. Antipsychotic medication was reduced for some participating residents which also minimised cost burden on family members.

Conclusion

Funding, systemic working across families, clinical and social teams and effective systems of governance are urgently required to sustain models like PROCUIDA-Demencia. The RE-AIM preliminary evaluation outlined care home managers’ long-term sustainable practice and positive impact on the dementia care system. These findings might inform staff retention strategies and care home systemic care practices. This evaluation is contributing to the Mexican Alzheimer’s and other dementias plan.

Background

Availability of evidence-based practices (EBPs) is critical for improving health care outcomes, but diffusion can be challenging. Implementation activities increase the adoption of EBPs and support sustainability. However, when implementation activities are a part of quality improvement processes, evaluation of the time and cost associated with these activities is challenged by the need for a correct classification of these activities to a known taxonomy of implementation strategies by implementation actors.

Design

Observational study of a four-stage, stakeholder-engaged process for identifying implementation activities and estimating the associated costs.

Results

A national initiative in the Veterans Health Administration (VHA) to improve Advance Care Planning (ACP) via Group Visits (ACP-GV) for rural veterans identified 49 potential implementation activities. Evaluators translated and reduced these to 14 strategies used across three groups with the aid of implementation actors. Data were collected to determine the total implementation effort and applied cost estimates to estimate the budget impact of implementation for VHA.

Limitations

Recall bias may influence the identification of potential implementation activities.

Conclusions

This process improved understanding of the implementation effort and allowed estimation of ACP-GV 's budget impact.

Implications

A four-stage, stakeholder-engaged methodology can be applied to other initiatives when a pragmatic evaluation of implementation efforts is needed.

This research is part of a project led by the Catholic University of Santiago de Guayaquil (Ecuador) and the National Distance Education University - UNED (Spain) aiming at designing a teaching/professional competences training model for teachers. The purpose of this study is to assess the meaning of professional competences and their significance within teachers’ teaching and training processes. Using the mixed methods approach, an ad hoc questionnaire was designed and given to discussion groups, achieving participation of more than a hundred teachers. Quantitative data analysis was carried out using the Logit model while Atlas ti 23 allowed analysis of the qualitative data. The results evidenced high scores in the competences evaluated, being leadership and research the lowest. In general, the teachers value, preferably, planning, communication, evaluation, methodology, digital, and tutoring competences, with a lower value among teachers ranging from 25 to 30 years old; those older than 55 stressed the importance of digital and innovation competences. Thus, it is recommended that training programs for teachers should focus on the set of related competences, especially on the digital competence, considering the challenges represented by investigation and pedagogical leadership.

This paper presents a case example of the Native-CHART Training Evaluation and describes the process of planning and administering a paper evaluation during the Native-CHART symposium in November 2019 led by the Center for Native American Health (CNAH) and an external evaluator. Training evaluation methodologies and the data collection instrument were grounded in the Health Belief Model (HBM) where health-related chronic disease and risk factor knowledge translates to perceived susceptibility, benefits, barriers, and self-efficacy. Kirkpatrick's Four-level Training Evaluation Model explored learning, reaction, behaviors, and results. The evaluation aims centered around the following questions: 1)Who attended the symposium, and why did they attend? 2)What knowledge did participants gain at the symposium? 3)Will attendees change their behaviors as a result of attending the symposium? 4) What parts of the symposium were most valuable? And 5) How can the symposium be improved? Data collected at the symposium answered these questions. After the Native-CHART symposium, CNAH staff and the external evaluator met to reflect on the steps necessary to plan and implement a participatory training evaluation. From these discussions, eight steps emerged. This paper presents these steps along with recommendations for future work. Participatory and collaborative approaches in training evaluation and the steps included in this case example may be useful to evaluators, communities, and programs working on designing and evaluating various trainings with Tribal populations.

The paper evaluates several second-order factors to explain the outcomes of the 2020 constitutional referendum held in Italy. This particular referendum serves as a relevant case study to assess the significance of second-order factors, given its unique characteristics, including the simplicity of the referendum question regarding the reduction in the number of parliamentarians (MPs), the minimal impact on public finances resulting from this reduction, and the subsequent decrease in territorial democratic representation. While it might have been expected that a significant majority would vote in favour of the “No” option, thus preserving the current levels of territorial coverage (and democratic representation) of MPs, the actual results saw a substantial majority in favour of the “Yes” vote (69%). Our argument posits that the overwhelming prevalence of the “Yes” vote (to reduce the number of MPs), especially in poorer areas of the country, can be attributed to specific factors that influence individual evaluations in a direct-democratic setting. In greater detail, by using cross-sectional data, the paper tests the role of socio-economic condition, trust in institutions and political participation in affecting the referendum outcome. The results of our empirical analysis confirm our hypotheses, demonstrating that second-order factors indeed influenced the referendum's outcome. Specifically, our analysis reveals that: (i) a higher socio-economic condition could generate a higher share of “No” votes; (ii) a higher trust in institutions could lead to an increase in the share of “No” votes; finally, that (iii) an increase in political participation could produce a decrease in the share of “No” votes. In the concluding section of the paper, we discuss how this analysis contributes new insights to the study of voting behaviour in direct-democratic contexts.

Theory of change (ToC) is an approach widely used to guide planning, implementing, and evaluating change initiatives. While there is substantial guidance, there has been little attention on equity within ToC research and practice. We propose and illustrate the metaphor of ‘knots’ to frame practical and ethical challenges that arise when centering equity within ToC processes. Drawing on our experiences using a ToC approach in two case examples, we identify and illustrate five equity-related knots: (a) clarify root causes, pathways, and success; (b) facilitate participation across power and perspective differences; (c) integrate research evidence and practitioner knowledge; (d) represent complex change visually; and (e) creatively navigate constraints. We show why framing these as knots can help practitioners make wise judgments within the circumstances and close with recommendations for including knots in ToC processes, reporting, and guidance.

Chronic Obstructive Pulmonary Disease (COPD) is a primary cause of adult hospitalizations and imposes substantial burdens on patients and healthcare systems. Initiatives that support providers and patients in addressing needs at each stage of this illness are needed. The INSPIRED COPD Outreach Program™ was introduced in 2010 in Halifax, Nova Scotia, Canada to improve care for those with advanced COPD. The huge success of the program led to its expansion to various contexts and geographic locations through a pan-Canadian quality improvement collaborative (QIC). Our study explored early elements of program planning and practice with an aim to understand factors that influence program sustainability. We evaluated the implementation process of 19 teams across Canada; post-collaborative team reports, focus groups, interviews and self-ratings of progress were collected from all teams. Analysis of data revealed three phases important to fostering sustainability (facilitating implementation, keeping the momentum, and securing sustainability) and each phase had unique supporting themes. Not surprisingly, teams that planned for sustainability early in the implementation process were more likely to successfully achieve program sustainability. However, teams also benefited more broadly from sustainability planning; this included being better able to plan for program spread as well as skill retention and knowledge transfer. This was also seen for quality improvement skills introduced early in the program that were more likely to be maintained and used through other contexts when sustainability planning was present. This study highlights that a QIC can be effective in not only influencing program sustainability but also beyond program implementation through improving knowledge and skill acquisition.