The Dilemma of Standard Setting for the OSCE

M Y  Sukkar

doi:10.52981/sjms.v10i4.979

M Y Sukkar Professor of physiology educ . Development Unit . Nile College , Sudan

DOI: https://doi.org/10.52981/sjms.v10i4.979

Abstract

Background: Recently disparities between the OSCE raw scores and global scores have resulted in the need treatment of the raw scores in different ways such as borderline regression and borderline group regression. The object of this paper is to present station scoring forms designed to satisfy predetermined criteria and minimum pass levels.

Methods: Available samples of marking sheets and checklists designed by various
examining bodies were scrutinized. Criteria were prioritized according to commonly used grading systems. The station rating scale (check list) was designed to allow the observer to concentrate on checking the performance of the candidate without marking at the same time. The station marks form enables entry of marks based on the criteria in the Station rating scale.

Results: Three forms were designed. Forms 1 & 2 should be prepared beforehand with real or standardized patients. Form 3 is a combination to be used when last minute stations are introduced. The mark allocated to each observable criterion is made within the limits of the specified criteria. The global score has been retained to check for inconsistencies and for longitudinal studies on validity and reliability.

Conclusions: Prototype forms are presented; using predetermined, categorized grading criteria. The forms enable examiners to separate the observation stage from the actual allocation of marks. As in all OSCE settings, objectivity, validity and reliability will depend on prioritizing the selection of stations, clarity of the selected criteria and the training of examiners.

References

1. Sara MortazHejri, Mohammed Jalili, Arno M. Muijtien and Cees P.M. Van Der Vieutem. Assessing the reliability of borderline regressions method as a standard setting procedure for objective structured clinical examination, J Res Med Sc 2013, 18(10): 887 -891.
2. Cohen, R, Rothman, A I, Poldre, P, Ross, J. Validity and gneralizability of global ratings in an objective structured clinical examination. Acad Med ,1991; 66:545-8
3. Wilkinson, T J Newble, D I, Frampton, C M. Standard setting in an objective structured clinical examination use of global ratings and borderline performance to determine the passing score. Med Educ.2001, 35:1043 – 1049.
4. Norcimi, J.J, Setting standards on educational tests. Med.Educ.2005;37: 464 -469
5. Pell, R, Fuller, R, Homer, M, Roberts T. International association for medical Education. How to measure quality of the OSCE: A review Adv Health Sc: Educ of metrics – AMEE guide no.49. Med Teach. 2010; 32: 802-11.
6. Newble ,D, Techniques for measuring clinical competence: Objective structured clinical examination. Med Educ 2004;38:199-203.
7. OCSE station forms Neel Burton, 2009/ Clinical skills for OSCEs,3rd edition [ISBN 9781904842590]Published by Scion Publishing Ltd.
8. Assessment Resource Center: Types of Assessment methods. OSCE rubrics. http://ar. cetl.hku/om_osce.htm
9. Newble D I , Hoare J , Sheldrake PF. The selection and training and examiners for clinical examinations. Med Educ 1980,19: 345-349.
10. Harden, R M, Stevenson, M,Downie W W, Wilson,G M. Assessment of clinical competence usingobjective structured examinations.BMJ 1975, 1: 447-51.
11. Reid, K J, Dodds, A. Comparing borderline group and borderline regression approaches to setting OSCE cut scores. J. Contemp. Med educ. 2014;2: 8-12.
12. Gormly, G, Summative OSCE in undergraduate medical education, Ulster med J 2011, 80: 127 -132
13. Pell, G, Homer, M S ,and Fuller, R, Investigating disparity between global grades and check list scores in OSCEs. Medical teacher, 2015 (in press), http://www.ncbi.nih.gov/pubmed/25683174.