BARRIERS AND FACILITATORS TO WRITE GOOD QUALITY MCQS FOR DENTAL ASSESSMENTS: A QUALITATIVE CASE STUDY
DOI:
https://doi.org/10.55519/JAMC-01-9797Keywords:
Assessment, Item writing, Quality assurance, Undergraduate medical education, Faculty development, Academic training, Examination questionsAbstract
Background: To explore barriers and facilitators to write good quality items for undergraduate dental assessments. Methods: A qualitative case study was conducted from Feb–April 2021. Semi-structured interviews were conducted with a purposive sample of eighteen item writers from a public-sector dental institute of Rawalpindi, Pakistan. The data were transcribed verbatim and thematically analyzed to extract themes regarding barriers and facilitators to write good quality items. All quality assurance procedures of qualitative research were ensured during the research process. Results: Five themes related to barriers and three themes related to facilitators to write good quality items emerged from the data. The participants reported more barriers such as lack of frequent training and lack of peer review and feedback. Other barriers were demotivation due to lack of acknowledgement or monetary incentives, lack of content and construct expertise, clinical workload, and contextual barriers such as lack of internet facility, outdated library, and lack of place and time allocation for item construction. Facilitators were availability of peer review, feedback from post-hoc analysis, motivation due to the senior designation, clinical experience, and ample time for basic sciences faculty. Conclusion: Frequent item writing training, strong peer review process, pre-exam item vetting by the dental education department, and institutional improvements such as striving for content experts, time and place allocation for item construction, internet facility, updated library, and equal distribution of workload among faculty could enhance the quality of items. Moreover, ways to inculcate motivation among item writers such as appreciation or monetary incentives could be used to improve the quality of undergraduate assessments.References
Tariq S, Tariq S, Maqsood S, Jawed S, Baig M. Evaluation of cognitive levels and item writing flaws in medical pharmacology internal assessment examinations. Pak J Med Sci 2017;33(4):866–70.
Salam A, Yousuf R, Bakar SMA. Multiple Choice Questions in Medical Education: How to Construct High Quality Questions. Int J Hum Heal Sci 2020;4(2):79–88.
AlMahmoud T, Elzubeir MA, Shaban S, Branicki F. An enhancement-focused framework for developing high quality single best answer multiple choice questions. Educ Heal Chang Learn Pract 2015;28(3):194–200.
Baig M, Ali SK, Ali S, Huda N. Evaluation of multiple choice and short essay question items in basic medical sciences. Pakistan J Med Sci 2014;30(1):3–6.
Case SM, Swanson DB. Constructing written test questions for the basic and clinical sciences. 3rd ed. Philadelphia: National Board of Medical Examiners; 1998.
Puthiaparampil T, Rahman MM, Gudum HR, Brohi IB, Lim IF, Saimon R. How to grade items for a question bank and rank tests based on student performance. MedEdPublish 2020;9(1):1–9.
Gopalakrishnan S, Udayshankar PM. Question vetting: The process to ensure quality in assessment of medical students. J Clin Diagnostic Res 2014;8(9):XM01–3.
Hassan S, Simbak N, Yussof H. Structured vetting procedure of examination questions in medical education in Faculty of Medicine at Universiti Sultan Zainal Abidin Malaysia. J Public Heal Med 2016;16(3):20–37.
Zaidi NL, Grob KL, Monrad SM, Kurtz JB, Tai A, Ahmed AZ, et al. Pushing Critical Thinking Skills With Multiple-Choice Questions: Does Bloom’s Taxonomy Work? Acad Med 2018;93(6):856–9.
Karthikeyan S, O’Connor E, Hu W. Barriers and facilitators to writing quality items for medical school assessments - A scoping review. BMC Med Educ 2019;19(1):14–17.
Capan Melser M, Steiner-Hofbauer V, Lilaj B, Agis H, Knaus A, Holzinger A. Knowledge, application and how about competence? Qualitative assessment of multiple-choice questions for dental students. Med Educ Online 2020;25(1):1–8.
Cheek C, Hays R, Smith J, Allen P. Improving case study research in medical education: a system atised review. Med Educ 2018;52(5):480–7.
Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006;3(2):77–101.
Guba EG, Lincoln YS. Epistemological and Methodological Basis of Naturalistic Inquiry. ECTJ 1982;30(4):233–52.
Luailiayah A, Rahayu GR, Claramita M. Encouragement and Challenge for Lecturers at Faculty of Medicine in Writing MCQ Questions. Int J Hum Heal Sci 2020;04(02):120–7.
Dellinges MA, Curtis DA. Will a Short Training Session Improve Multiple-Choice Item-Writing Quality by Dental School Faculty? A Pilot Study. J Dent Educ 2017;81(8):948–55.
AlFaris E, Naeem N, Irfan F, Qureshi R, Saad H, Al Sadhan R, et al. A One-Day Dental Faculty Workshop in Writing Multiple-Choice Questions: An Impact Evaluation. J Dent Educ 2015;79(11):1305–13.
Abdulghani HM, Irshad M, Haque S, Ahmad T, Sattar K, Khalil MS. Effectiveness of longitudinal faculty development programs on MCQs items writing skills: A follow-up study. PLoS One 2017;12(10):1–14.
Steinert Y, Mann K, Anderson B, Barnett BM, Centeno A, Naismith L, et al. A systematic review of faculty development initiatives designed to enhance teaching effectiveness: A 10-year update: BEME Guide No. 40. Med Teach 2016;38(8):769–86.
Sajjad M, Iltaf S, Khan RA. Nonfunctional distractor analysis: An indicator for quality of multiple choice questions. Pakistan J Med Sci 2020;36(5):982–6.
Karthikeyan S, O’Connor E, Hu W. Motivations of assessment item writers in medical programs: a qualitative study. BMC Med Educ 2020;20(334):1–10
Published
Issue
Section
License
Journal of Ayub Medical College, Abbottabad is an OPEN ACCESS JOURNAL which means that all content is FREELY available without charge to all users whether registered with the journal or not. The work published by J Ayub Med Coll Abbottabad is licensed and distributed under the creative commons License CC BY ND Attribution-NoDerivs. Material printed in this journal is OPEN to access, and are FREE for use in academic and research work with proper citation. J Ayub Med Coll Abbottabad accepts only original material for publication with the understanding that except for abstracts, no part of the data has been published or will be submitted for publication elsewhere before appearing in J Ayub Med Coll Abbottabad. The Editorial Board of J Ayub Med Coll Abbottabad makes every effort to ensure the accuracy and authenticity of material printed in J Ayub Med Coll Abbottabad. However, conclusions and statements expressed are views of the authors and do not reflect the opinion/policy of J Ayub Med Coll Abbottabad or the Editorial Board.
USERS are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles, or use them for any other lawful purpose, without asking prior permission from the publisher or the author. This is in accordance with the BOAI definition of open access.
AUTHORS retain the rights of free downloading/unlimited e-print of full text and sharing/disseminating the article without any restriction, by any means including twitter, scholarly collaboration networks such as ResearchGate, Academia.eu, and social media sites such as Twitter, LinkedIn, Google Scholar and any other professional or academic networking site.
