RELATIONSHIP OF AWARDS IN MULTIPLE CHOICE QUESTIONS AND STRUCTURED ANSWER QUESTIONS IN THE UNDERGRADUATE YEARS AND THEIR EFFECTIVENESS IN EVALUATION

Junaid Sarfraz Khan, Osama Mukhtar, Saima Tabasum, Naveed Shaheen, M. Farooq, M. Abdul Irfan, Ajmal Sattar, M. Nabeel, M. Imran, Sadia Rafique, Maryam Iqbal, M. Sheraz Afzal, M. Shahbaz Hameed, Maryam Habib, Uzma Jabeen, Malik Hussain Mubbashar

Abstract


Background: A number of evaluation tools for assessing the cognitive and affective domains in
accordance with Bloom’s taxonomy are available for summative assessment. At the University of
Health Sciences, Lahore, Multiple Choice Questions (MCQs) and Structured Answer Questions
(SAQs) are used for the evaluation of the cognitive domain at all six hierarch levels of taxonomy
using the tables of specifications to ensure content validity. The rationale of having two evaluation
tools seemingly similar in their evaluative competency yet differing in feasibility of construction,
administration and marking is being challenged in this study. Methods: The MCQ and SAQ
awards of the ten percent sample population amounting to 985 students in fifteen Medical and
Dental Colleges across Punjab were entered into SPSS-15 and correlated according to the
cognitive and affective level of assessment in relation to the Bloom’s taxonomy and their grouping
in the Tables of Specifications, using parametric tests. 3494 anonymously administered
questionnaires were analyzed using ethnograph. Results: No statistically significant difference
was found in the mean marks obtained by the students when MCQs and SAQs were compared
according to their groupings in the Tables of Specifications at all levels of cognitive hierarchical
testing. End-of-year cognitive level testing targets set were not met and more questions were set at
the lower cognitive testing levels. Expenses incurred in setting MCQs and SAQs were comparable
but conduct and assessment costs for MCQs and SAQs were 6% and 94% of the total respectively.
In both MCQs and SAQs students performed better at higher cognitive testing levels whereas the
SAQs and MCQs were able to marginally test the lower levels of affective domain only. Student’s
feedback showed that attempting MCQs required critical thinking, experience and practice.
Conclusion: MCQs are more cost effective means at levels of cognitive domain assessment.
Keywords: Cognitive Domain, Affective Domain, Bloom’s Taxonomy, Tables of Specifications,
Evaluation

Full Text:

PDF

References


Schultheis NM. Writing cognitive educational objectives and

multiple-choice test questions. Am J Health Syst Pharm

;55:2397–401

Writing Multiple-Choice Questions that Demand Critical

Thinking. Available at: http://cit.necc.mass.edu/atlt/

TestCritThink.htm#anchor1046303 [Accessed January 2nd 2008].

Dressel, PL, and Associates. Evaluation in higher education.

Boston: Houghton Mifflin; 1961.

Airasian, PW. Classroom Assessment: Concepts and

Applications (4th Ed.). New York: McGraw-Hill; 2001.

McMillan JH. (Ed.). New directions in teaching and learning:

Assessing students’ learning. No. 34. San Francisco: JosseyBass; 1988.

Gronlund, NE. How to write and use instructional objectives (6th

Ed.) Upper Saddle River, NJ: Merril/ Prentice Hall; 2000.

Footlick JK. Truth and Consequences: How Colleges and

Universities Meet Public Crises. Phoenix Ariz: Ammerica

Council on Education; 1997.

Mayer D. Calamen DL. Gunderson A. Barach P. Telluride

Interdisciplinary Roundtable. Designing a Patient Safety

Undergraduate Medical Curriculum: The Telluride

Interdisciplinary Roundtable Experience. Teach Learn Med

;21(1):52–8.

Tarrant M, Knierim A, Hayes SK, Ware J. The frequency of item

writing flaws in multiple-choice questions used in high stakes

nursing assessments. Nurse Educ Today 2006;26:662–71.

Shepard, LA. Evaluating test validity. In L. Darling-Hammond

(Ed.),. Review of research in education (Vol. 19) Washington,

DC: American Educational Research Association; 1993.

p.405–50.

Palmer EJ, Devitt PG. Assessment of higher order cognitive

skills in undergraduate education: modified essay or multiple

choice questions?: research paper. BMC Med Edu 2007;7(1):49.

Louis C, Lawrence M, Keith M. Research Methods in Education

(6th Ed.) New York: Routledge; 2007.

Bloom Benjamin S, David R. Krathwohl. Taxonomy of

Educational Objectives: The Classification of Educational Goals,

by a committee of college and university examiners. Handbook I:

Cognitive Domain. New York: Longmans, Green; 1956.

Bloom B, Englehart M, Furst E, Hill W, Krathwohl D.

Taxonomy of educational objectives: The classification of

educational goals. Handbook I: Cognitive domain. New York,

Toronto: Longmans, Green; 1956.

Mukhopadhyay M, Bhowmick K, Chakraborty S, Roy D, Sen

PK, Chakraborty I. Evaluation of MCQs for Judgment of higher

levels of Cognitive learning. Gomal J Med Sci 2010; 8(2):112–6.

Moeen-Uz-zafar, Aljarallah B. Evaluation of MEQ and MCQ as

a tool for assessing the cognitive skills of undergraduate students

at department of medicine, Qassim University. J Fam

Community Med 2010;17(1):50–67.

Palmer EJ, Duggan P, Devitt PG, Russell R.. The modified essay

question: its exit from the exit examination?

Marshall J. Assessment of problem-solving ability. Med Educ

;11:329–34.

Rabinowitz HK, Hojat MD. A comparison of the modified essay

question and multiple choice question formats: Their

relationships to clinical performance. Fam Med 1989;21:364–7.

Palmer EJ, Devitt P. Constructing multiple choice questions as a

method for learning. Ann Acad Med Singapore 2006;35:604–8.

Collins J. Education techniques for lifelong learning: writing

multiple-choice questions for continuing medical education

activities and self-assessment modules. Radiographics

;26:543–51.

Rabinowitz HK. The modified essay question: an evaluation of

its use in a family medicine clerkship. Med Educ 1987;21:114–8.

Webber RH. Structured short answer questions: an alternative

examination method. Med Educ1992;26(1):58–62.

Pai MRSM, Sanji N, Pai PG, Kotian S. Comparative Assessment in

Pharmacology Multiple Choice Questions Versus Essay with focus

on Gender difference. J Clin Diagnostic Res 2010;4:2515–20.

Epstein RM. Assessment in medical education. N Engl J Med

;356:387–96.

DFES (Department for education and skills) Towards a Unified

e-Learning Strategy: consultation document, Nottingham: DFES

Publications; 2003.

Rust C. The impact of assessment on student learning: how can

the research literature practically help to inform the development

of departmental assessment strategies and learner-centered

assessment practices? Active Learning in Higher Education

;3(2):145–58.

Moosa MYH, Jeenah FY. The assessment of undergraduate

psychiatry training: a paradigm shift. A S Psychiatry Re Rev

;10:88–91.

Swanson DB. A measurement framework for performance based

tests. In: IR Hart and RM Harden, Editors, Further developments

in assessing clinical competence. Montreal: Can-Heal;

p.13–45.

Newble DI, Swanson DB. Psychometric characteristics of the

objective structured clinical examination. Med Educ

;22:325–34

Dauphinee, D. Determining the content of certification

examinations. In: D Newble, B Jolly and R Wakeford, Editors,

The certification and recertification of doctors: issues in the

assessment of clinical competence, Cambridge: Cambridge

University Press: 1994p. 92–104.

Hettiaratchi ES. A comparison of student performance in two

parallel physiology tests in multiple choice and short answer

forms. Med Educ 1978;12:290–6.

Leamnson R. Thinking about teaching and learning. Sterling,

VA: Stylus Publishing; 1999.


Refbacks

  • There are currently no refbacks.


Contact Number: +92-992-382571

email: [jamc] [@] [ayubmed.edu.pk]