A Comparison of Students, Scores in Multiple- Choice and Essay Questions in Mother and Child Health Examination

Document Type : Original Article

Authors

1 Department of Midwifery, School of Nursing and Midwifery, Shahrekord University of Medical sicenses, Shahrekord, IRAN

2 Shahrekord University of Medical Sciences, Shahrekord, IRAN

Abstract

 
Background: The method of evaluation in students’ learning affects their studying mode. The aim of this study was to compare nursing students scores in essay and multiple-choice questions in Mother and Child Health examination in Shahrekord University of Medical Sciences.
Methods: In this study, the  final scores of 121 of nursing students in a multiple-choice questionnaire consisting of 40 questions, during four semesters were compared with the scores obtained from an essay questionnaire composed of 40 questions. Difficulty and Discrimination Index of questions were similar in both kinds of questions. Data was analyzed employing paired t-test and Pearson correlation test and PResults:  Mean and standard deviation of students’ scores were reported 14.72±3.7 in essay questions and 16.53±2.4 in multiple-choice questions and showed a significant difference (P<0.001). A positive correlation was reported between the essay and multiple-choice questions (r=0.93). Students’ scores were higher in multiple-choice questionnaire compared to the essay one.
 Using multiple-choice questionnaire on its own may lead to the success of those students who do not  have the ability of perceive, resolve and apply.

Keywords


INTRODUCTION

Assessment is one of the most important components of each educational program, and if accomplished well, can improve students’ motivation for learning and provide educators with useful feedback [1]. Learning assessment is often one of the most difficult and time-consuming aspects of education, greatly affecting students’ studying mode [2]. Conventionally, different types of questions such as essay, short answer, structural, and objective have been employed for assessment of students’ knowledge. Essay questions, despite their greater credibility compared to other types of questions, were applied less due to being time-consuming in terms of scoring. This question format allows students to be flexible while writing answers and individually reflecting on their skills [3 ] .  Essay questions also provide specific feedback to direct future learning [2].  A study reported that essay questions had higher sensitivity in comparison with objective questions in predicting the students’ performance [4]. The objective questions, making students find correct answers, require comparably shorter time to score, and are employed in assessment of medical students in different courses. These questions are capable of covering most of the content and are highly reliable. Multiple-choice, true-false, and matching types, are classified as objective questions (5,6). Designing multiple-choice questions (MCQs) to be used repeatedly in final examinations is difficult and time consuming, even for professional educators; besides, with elimination of negative scores, association of examination  score with learners’ knowledge seems poor [7]. In designing these questions, it should be noted that these questions must include the cognitive educational purposes, which are mentioned in the lesson plan [8,9]. MCQs have high reliability, but their validity, particularly at the level of knowledge and considering the construct design principles, have been noticeable issues in educational studies. Clearly, if an examination is not well-designed in terms of questions’ taxonomy choice and/or structural rules, it will not yield an acceptable level of content validity or construct. Therefore, not only the main role of examination as a complementary and vital component of educational cycle will be lost, but also it will negatively affects learners’ motivation and threatens the efforts of teachers and educational system [1]. In comparing essay with MCQs regarding their predictive capability, various studies have reported different findings.  According to one, MCQs have a lower sensitivity to predicting students’ performance than essays [10]. However, as reported by another study, essay questions have higher validity than the MCQs, but can’t measure the qualification aspects of students more profoundly in contrast to MCQs [2]. Importantly, students’ awareness of conventional structure of questions has replaced question-reading with thinking during the learning process ; and also the approach to question-reading has frantically reduced welcoming class sessions which is an environment for communicating thoughts, analyzing data, and even teaching research [11].  One study reported that an important section of one’s preparation for examination is indeed his or her assessment of its method [12]. In Mother and Child 

Health course in Shahrekord University of Medical Sciences, the assessment of students’ performance is made with multi-choice questions. It seems that graduated nursing students, working in hospitals and health centers, have little knowledge about the mother and child health subject. The objective of the present study was to compare the scores obtained by the nursing students while answering essay and multiple choice questions about mother and child health subject. 

METHODS

 

 

In Shahrekord University of Medical Sciences, thirty or thirty-one students in nursing course are accepted each year who study the Mother and Child Health subject in the 3rd semester. To help generate the findings of this study, we studied the students of 4th  semesters; i.e. 121 nursing students between 2010 and 2012 who had taken the Mother  and Child Health subject.  In this course, the final evaluation is conventionally carried out through MCQs. In the current study, we compared the essay with MCQs. Data collection was performed by Personal Information Form and the questions of Mother and Child Health examination.  For this study, to examine mother and child health, first, the essay, consisting 40 questions, was given to the students to answer during the 40-minute allotted time.  At the end of this section and after the students had delivered their essay papers, a multiple-choice questionnaire composed of 40 questions that covered a similar content, was given to students to answer within 40 minutes. In essay and multiple choice questionnaires, 50% of questions were designed in taxonomy I (recalling information and memorizations) and 50% in taxonomy II (interpreting information). None of the questions in either exam had a negative score; this issue was inscribed on their papers. The value of essay questions and MCQs were equivalent; besides, each semester the questions were replaced with those of similar content. After the examination, the papers were scored by the teachers and controlled by other two colleagues. In analyzing questions, difficulty index, discrimination index of questions was detected and compared between the two exams. To consider the research ethics, only the scores of MCQ examination was noted for final evaluation as the real score of student based on 20 and was delivered to Education Deputy of Faculty. Analysis of the data was performed by using paired t-test and Pearson correlation coefficient and P

RESULTS

The mean age of all participants was 21.31±1.4. Sixty-Eight percent of them were female and Thirty-Two percent were male. There was not a significant difference with regards to difficulty and discrimination indices between the two kinds of questions for all the four semesters of study.  The comparison of students’ mean scores in two types of evaluation is presented in table 1 that indicates higher scores of MCQs compared to essay questions in all four years in a significant manner (P<0.001). The minimum score obtained in essay questionnaire was 11.50 and the maximum  was 16.50.  In MCQs questions, these scores were 14.00 and 18.00. A positive correlation was observed between the scores in the two types of questions [n=121, r=0.93, p

Table 1. MCQs and essay scores for nursing students in Mother and Child Health examination

study semester

 

Essay scores

MCQs scores

P Value

 

n

X±SD

X±SD

 

1st semester of 2009-2010

30

14.79±2.4

16.42±1.5

P<0.001

2nd  semester of 2009-2010

31

14.88±3.1

16.17±2.3

P<0.001

1st  semester of 2010-2011

30

13.27±5.3

16.33±4.3

P<0.001

2nd  semester of 2010-2011

30

13.91±4.3

17.21±3.5

P<0.001

Total

121

14.72±3.7

16.53±2.4

P<0.001

Values are means (SD); n, numbers of students.

DISCUSSION

 

The findings of present study showed a significant difference between the scores obtained by nursing students in essay and multiple- choice examination and the  MCQs’ scores were higher than the essay ones in all four semesters.  Moqattash et al ., reported similar findings and stated that the students with recalling ability may get a better score in MCQs while those with analysis, interpretation, organization of knowledge, and reporting skills obtain higher scores in essays [14]. On the other hand, the consistently lower scores in long essay questions may be attributed to the bias or more subjective marking schemes of different lecturers compared with the more quantitative nature of MCQs. A similar observation of higher scores in MCQs and short essay questions compared with structured, integrated long essay questions was reported by Moqattash et al. [14], who claimed that one of the reasons for higher scores was the elimination of examiners’ bias. However, we believe that long essay-type assessment is a sensitive test requiring students not only to recall facts but also to use higher-order cognitive skills, such as analytical, interpretive, and application skills.

In the present study, considering the fact that in designing essays we attempted each part of question to have a clear  

point for scoring, it seems the examiner’s bias could be minimized. Pepple et al ., also reported that  the essay questions with long answers not only make the students recall the facts, but also express their cognitive skills in higher areas such as analysis and applications [2]. One study compared medical students’ performance in essays with short answers and MCQs questions.  It was reported in physiology that an insufficient knowledge of physiology and students’ inability to establish a logical sequence among the learnt information is the main reason for their poor performance in essay questions compared to MCQs [15].  Another finding of the present study was the significant association between the scores of essay and MCQs questions. This relationship reveals that students with a good performance in essay questions will typically have an acceptable performance in MCQs. Similar findings were noted by Oyebok et al.,[15] and Mogattash et al., [14]. The analysis of multiple-choice questions in different studies indicate that the majority of MCQs have been designed to recall  information and memorization [16, -17].

Anyway, it is worthy considering that in MCQs examinations, the emphasis of test developers is on memorization rather than deep understanding and application of concepts, because this approach leads to memorizing subjects which in medical courses, it would not be satisfactorily borne in  mind and  is far from developing students’ ability [16,18]. Of course, the limitations of MCQs that were indicated by other studies should be noticed [19, 20], It is necessary for teachers to take advantage of other standard questions for assessing students’ knowledge. In brief, this study indicates that there was a strong correlation between students’ scores in MCQs and essay questions in general. The overall higher MCQ scores depict that without the essay examination format, some of the failing students are likely to pass the final examination in maternal and child Health subject. Conducting the study on just one subject of nursing lessons was the limitation of this study. The studies with more comprehensive samples and in other courses might help generalize the results and achieve a comprehensive analysis. 

The findings of the present study indicted a significant difference in students’ MCQ and essay scores; that is the scores of students in MCQs was higher than the essay 

CONCLUSION

questions. It reveals that if only MCQs are used, some students will pass the exams without cognition, analysis, thinking, and knowledge application abilities. Therefore, different methods of assessments are highly recommended.

ACKNOWLEDGEMENTS

The authors of the present study would like to thank  

Research and Technology Department of Shahrekord University of Medical Sciences and all students who helped to conduct the study.

 

Disclosures:

No conflicts of interest, financial or otherwise, are declared by the author(s)

  1. Moeen-Uz-Zafar, Badr-Aljarallah. Evaluation of mini-essay questions (MEQ) and multiple choice questions (MCQ) as a tool for assessing the cognitive skills of undergraduate students at the Department of Medicine.Int J Health Sci (Qassim). 2011 Jul;5(2 Suppl 1):43-4.
  2. Pepple DJ, Young LE, Carroll RG. A comparison of student performance in multiple-choice and long essay questions in the MBBS stage I physiology examination at the University of the West Indies (Mona Campus). AdvPhysiol Educ. 2010 Jun; 34(2):86-9.
  3. van Bruggen L, Manrique-van Woudenbergh M, Spierenburg E, Vos J.Preferred question types for computer-based assessment of clinical reasoning: a literature study. Perspect Med Educ. 2012 Nov;1(4):162-71. doi: 10.1007/s40037-012-0024-1. Epub 2012 Oct 2.
  4. Nnodim JO.Multiple-choice testing in anatomy.Med Educ. 1992 Jul; 26(4):301-9
  5. Bauer D, Kopp V, Fischer MR. Answer changing in multiple choice assessment change that answer when in doubt--and spread the word!BMC Med Educ. 2007 Aug 24; 7:28
  6. Karl M, Graef F, Eitner S, Beck N, Wichmann M, Holst S. Comparison between computer-aided testing and traditional multiple choice: and equivalence study. Eur J Dent Educ. 2007 Feb;11(1):38-41.
  7. Tractenberg RE, Gushta MM, Mulroney SE, Weissinger PA.Multiple choice questions can be designed or revised to challenge learners' critical 

    thinking.Adv Health Sci Educ Theory Pract. 2013 Jan 4. [Epub ahead of print]

  8. Bandaranayake RC. Setting and maintaining standards in multiple choice examinations: AMEE Guide no. 37. Med Teach. 2008;30(9-10):836-45. doi: 10.1080/01421590802402247.

  9. Beirer, SB, Dannefer, EF, Taylor, C, Hall, P Hull, A. Methods to assess students’ acquisition, application and integration of basic science knowledge in an innovative competency-based curriculum. Med Teach 2008; 30: e171–e177.

  10. Malone S, Brünken R. Assessment of driving expertise using multiple choice questions including static vs. animated presentation of driving scenarios. Accid Anal Prev. 2012 Nov 30; 51:112-119. doi: 10.1016

  11. Haddad M, Tylee A. The Development and First Use of the QUEST Measures to Evaluate School Nurses' Knowledge and Skills for Depression Recognition and Management. Sch Health. 2013 Jan;83(1):36-44. doi: 10.1111/j.1746-1561.2012.00745.x.

  12. Freestone, N. Drafting and acting on feedback supports student learning when writing essay assignments. AdvPhysiolEduc 2008; 33: 98–102.

  13. Rassaian N, Nakhaei S, Sadeghi GhandehariN. Comparison of three exam techniques in medical students:Multiple-choice, true-false and short answer questions. Hakim Research Journal 2003;5(4):271-278(Persian).

  14. Moqattash, S, Harris, PF, Gumaa, KA Abu-Hijleh, MF. Assessment of basic

  15. medical sciences in an integrated systems-based curriculum.Clin Anat1995; 8: 139–147.

  16. Oyebola, DDO, Adewoye, OE, Iyaniwura, JO, Alada, ARA, Fasanmade, AA Raji, Y. A comparative study of students’ performance in preclinical physiology assessed by multiple choice and short essay questions.Afr J Med MedSci 2000; 29: 201–205 .

  17. HaghShenas MR, VahidShahi K, Mahmoodi M, ShahbazNejad L, ParvinNejad N, Emadi A .Evaluation of multiple choice questions in the school of medicine, Mazandaran University of Medical Sciences, the first semester of 2007.Journal of Mazendaran University of Medical Sciences 2007;16(56): 126- 135(Persian).

  18. Rasolinejad SA, Vakihi Z, Fakharion E, Fakharian E, Mosayebi Z, Moniri R . Comparative survey of taxonomies of residents.Promotion examination, kashan Medical University 2006.The 8th National Congress of Medical Education.kerman: kermanUniverity of Medical Sciences; 2007: 68. (Persian).

  19. Collins J. Education techniques for long learning: Writing multiple choice question for continuing medical education activities and self assessment modules. Radio Graphics 2006: 26(2): 543-51.

  20. Sim kin MG, Kuechler WL. Multiple choice test and student understanding: What is the connection? Decision Scie J InnovatEduc 2005; 3(1): 73-98.

  21. Bush ME. Quality assurance of multiple choice tests.QualAssurEduc 2006; 14(4): 398-404.