Revitalizing university performance evaluation: the case of SETARA model in Malaysia

Abstract

Purpose

SETARA is a well-known university rating tool in Malaysia. The study aims to enhance the transparency, accuracy, and reliability of SETARA assessment instrument by improving its weighting scheme for the domains, sub-domains, criteria, and indicators.

Design/methodology/approach

The study utilized a quantitative research design and collected responses from 29 academic administrators in both public and private higher education institutions in Malaysia. The four steps of the Analytic Hierarchy Process (AHP) were employed. An AHP questionnaire was developed, adopting the domains, sub-domains, criteria, and indicators of the SETARA assessment instrument. The “Superdecision” software was used to synthesize the pairwise comparison judgments and compute the consistency ratio for all group pairwise comparison matrices.

Findings

The AHP-assigned weights differed significantly from those assigned by the SETARA rating system. For instance, the weight for “Input” increased from 20 to 41.05 out of 100, and the weight for “Output” decreased from 40 to 25.52. The new SETARA rating instrument is expected to be used by the Ministry of Higher Education (MOHE) due to its scientifically justified foundation to ensure transparency, accuracy, and reliability.

Research limitations/implications

The study is limited by small sample size for data collection. Future research could benefit from a larger and more diverse sample, including participants from a number of national-level universities and relevant agencies like Malaysia Qualification Agency (MQA) and MOHE. Additionally, the research focused solely on recalculating weights and did not consider modifying the criteria set. Further investigation is needed to determine if some criteria can be added or removed.

Practical implications

This research offers practical implications for refining the SETARA assessment mechanism and improving the quality of higher education in Malaysia. The modified weights provide guidance to Higher Education Institutions (HEIs) on where to focus their efforts to ensure quality. The transparent methodology introduced by this study can serve as a model for enhancing educational quality evaluations in Malaysia and potentially in other higher education systems worldwide.

Originality/value

The originality of this study lies in its innovative application of the AHP to the SETARA assessment tool, addressing a crucial gap in the assessment process. By providing a comprehensive and transparent methodology for assigning percentages and weights, this research offers a practical blueprint for enhancing the precision and impact of educational quality evaluations in Malaysia and beyond.