Introducing Standard Setting Into Malaysian Undergraduate Dental Programme

Abstract

This thesis investigates the implementation of standard setting methods in high-stakes assessments within the undergraduate dental programme Universiti Kebangsaan Malaysia (UKM). Recognising the need for evidence-based standards, this research introduces and evaluates standard setting to enhance decision validity. UKM’s competency-based dental education (CBDE) assesses students through composite scoring across multiple assessment components. The research aims to introduce standard setting, comparing modified Angoff and Ebel approaches, and provide guidelines for procedures, validity evidence, and psychometric analysis. An exploratory phase examined faculty perceptions of current assessment practices on fixed passing scores, identifying gaps in readiness. A workshop on the modified Angoff and Ebel methods trained 12 faculty judges, divided into two groups (Group A and B), applying the methods to final-year theory papers (short answer question and the one-best answer items) in the Comprehensive Dental Care course. Subsequent analyses evaluated internal, external, and procedural validity. Group A generally showed greater consistency and inter-rater agreement than Group B. Passing score reproducibility was mixed, with some variation between groups and formats. Correlations between ratings and empirical item difficulty were low, but moderate between modified Angoff and Ebel methods. Judges reported improved understanding and confidence, with focus group discussions highlighted the role of diverse expertise, alignment with course learning outcomes, barriers to effective standard setting, and the impact of training as a key catalyst. The thesis supports replacing arbitrary passing scores with standard setting at UKM, offering evidence to promote fairness and validity in dental assessments and directions for future research.

Date of Award	24 Jul 2025
Original language	English
Awarding Institution	The University of Manchester
Supervisor	Reza Roudsari (Supervisor) & Lucy O'Malley (Supervisor)