INTER-RATER RELIABILITY OF OBJECTIVE STRUCTURED LONG EXAMINATION RECORD
AbstractBackground: Objective Structured Long Examination Record (OSLER) scale was introduced in 1997 by Gleeson to improve the long case examination. There is no psychometric evidence to support reliability of OSLER. This study was done to analyse inter-rater reliability of OSLER. Methods: Two groups of examiners assessed 105 students in long case examination of their final professional examination, using OSLER scale. Group 1 was composed of actual examiners while Group 2 was mock examiners. Kappa statistic and intraclass correlation coefficient (ICC) were used on SPSS 23 to calculate reliability. Results: Mean score awarded by actual examiners was 55.36 (SD=11.2) whereas mean score by mock examiners was 57.74 (SD=14.1). Cronbach’s alpha was 0.586, Kappa was 0.019 whereas inter-rater reliability on ICC was 0.413. Conclusion: Although OSLER is a practical modification of long case examination with good validity, the scale needs to be more structured to improve its reliability.Keywords: Long case; OSLER; Reliability
Dare AJ, Cardinal A, Kolbe J, Bagg W. What can history tell us? An argument for observed history-taking in the trainee intern long case assessment. N Z Med J 2008;121(1282):51–7.
Wilkinson TJ, Campbell PJ, Judd SJ. Reliability of the long case. Med Educ 2008;42(9):887–93.
Smee S. ABC of learning and teaching in medicine. Skill based assessment. Br Med J 2003;326(7391):703–6.
Troncon EA, Fernando ROD, Figueiredo C, Ferriolli E, Moriguti Lio C, Martinelli Ana LC, et al. A standardized, structured long-case examination of clinical competence of senior medical students. Med Teach 2000;22(4):380–5.
Wass V, Van der Vleuten C. The long case. Med Educ 2004;38(11):1176–80.
Teoh NC, Bowden FJ. The case for resurrecting the long case. BMJ 2008;336(7655):1250.
Thornton S. A literature review of the long case and its variants as a method of assessment. Educ Med J 2012;4(1):e5–14.
Ponnamperuma GG, Karunathilake IM, McAleer S, Davis MH. The long case and its modifications: a literature review. Med Educ. 2009;43(10):936–41.
Sood R. Long case examination - Can it be improved? J Indian Acad Clin Med 2001;2(4):252–5.
Norcini JJ, Lipner RS, Kimball HR. Certifying examination performance and patient outcomes following acute myocardial infarction. Med Educ 2002;36(9):853–9.
Newble DI. The observed long case in clinical assessment. Med Educ 1991;25(5):369–73.
Abouna GM. The integrated direct observation clinical encounter examination (IDOCEE) - an objective assessment of students’ clinical competence in a problem-based learning curriculum. Med Teach 1999;21(1):67–72.
Gleeson F. AMEEN medical education guide no 9: Assessment of clinical competence using the Objective Structured Long Examination Record (OSLER). Med Teach 1997;19(1):7–14.
Bannerji M, Capozzoli M, McSweeney L, Sinha D. Beyond Kappa: A review of inter-rater agreement measures. Can J Stat 1999;27(1):3–23.
Weir JP. Quantifying test-retest reliability using intraclass correlation coefficient and SEM. J Strength Cond Res 2005;19(1):231–40.
Viera JA, Garrett JM. Understanding inter-observer agreement: The Kappa Statistic. Fam Med 2005;37(5):360–3.
Barzansky B, Etzel SI. Medical schools in the United States, 2009-2010. JAMA 2010;304(11):1247–54.
Bentley BS, Hill RV. Objective and subjective assessment of reciprocal peer teaching in medical gross anatomy laboratory. Anat Sci Educ 2009;2(4):143–9.
Kamarudin MA, Mohamad N, Halizah MN, Yaman MN. The Relationship between Modified Long Case and Objective Structured Clinical Examination (OSCE) in final professional examination 2011 held in UKM Medical Centre. Procedia-Soc Behav Sci 2012;60:241–8.
Malik A, Bhugra D. Workplace based assessment methods: literature overview. In: Malik A, Bhugra D, Brittlebank A, editor. Workplace-based assessments in psychiatry. 2nd ed. London: RCPsych Publications, 2011; p.14–27.
Kroboth FJ, Hansusa BH, Parker S, coulehan JL, Kapoor WN, Brown FH, et al. The inter-rater reliability and internal consistency of a clinical evaluation exercise. J Gen Intern Med 1992;7(2):174–9.
Wass V, Jolly B. Does observation add to the validity of the long case? Med Educ 2001;35(8):729–34.
Nithyanandam S, Joseph M, Vasu U. Can conventional long case examination be improved? Indian J Ophthalmol 2012;60(4):333.