Instrument Development for Study Programs Excellence Evaluation Based on Kirkpatrick Evaluation Model

Abstract

Background: The study programs excellence stated in the institution's vision and mission acts as a guide for the institution in carrying out its activities. That excellence must be reflected in all aspects of academic activities. Evaluation of the implementation and achievement of study program’s excellence is important to fulfill the vision, and mission of the institution. This research aims to develop an evaluation instrument that is useful for monitoring the achievement of study program’s excellence. Methods: Two part of instrument development were carried out in this study, namely: instrument development, and instrument validation process. Instrument development part consist of two stage (item and scale development, and expert judgement). Meanwhile, instrument validation process part consist of 5 stages of comprehensive validation process by Sireci. This validation process used argument based validation concept. Results: This instrument uses the Kirkpatrick model which consists of 4 evaluation levels. Each level consists of several dimensions. A total of 55 instrument items were proven valid (r>0.217) and reliable (Cronbach’s Alpha>0.7). The instrument uses 2 types of data in the assessment process, namely primary data (measuring responses from the academic community using a questionnaire), and secondary data (assessment of document evidence). Conclusion: The instrument evaluation is prepared for the purpose of providing an overview of the implementation and achievement of institutional’s excellence. This is useful in providing valuable information to the institution to set strategies and policies to fulfill the institution's excellence.The instrument proved to be valid and reliable by an argument-based validation analysis.