Influence of Assessment Method Selection in Studying and Learning Approaches: Is It Necessary to Change Assessment Style?

INTRODUCTION

In education, examinations are almost exclusively used as assessment tools, thus teachers assess the efficacy of their curriculum and students’ learning outcomes in the course to assign grades (1).

Currently, medical school examinations comprise a range of different assessments, both written and performance-based, offering an opportunity to compare performance on different formats (2). The multiple-choice question (MCQ) type of tests is commonly used in assessment of medical knowledge acquisition (3). Objective testing, specifically MCQ is one of the approaches that may diversify the assessment approach in education. Significant commitment is required to prepare MCQ test items and examination formats that are reliable and consistent with curriculum objectives. Appropriately constructed MCQ examinations are efficient, objective, and capable of discrimination and can be combined with other assessment strategies to contribute to a comprehensive student assessment strategy for use in nursing education (4). Therefore, the MCQ examination is not a completely transparent examination tool and is supplemented by short-answer question (SAQ). However, since it is relatively cheap, easy to standardize and rapidly generates an objective score, it is still used to assess (5). Assessment is the process by which the teacher and the student gain knowledge about student progress. Assessment systems should aim at evaluating the desired learning outcomes. It was also demonstrated that a single assessment does not fulfill all aspects of assessment and that there is a need for an evaluating system with multiple ways of assessment (6). In another study, Rassaian showed that the most valid assessment tool was the SAQs. She concluded that it is recommended to use SAQs and true-false questions as the main components of examination, instead of MCQ alone, (7).

Tests that require effortful retrieval of information, such as short-answer tests, promote better retention and greatest gains in memory than tests that require recognition, such as multiple-choice tests (8,9). Therefore, tests can also directly affect learning by promoting better retention of information, a phenomenon known as the testing effect (10). Examination style may enhance study and learning approaches in students (11). Then, studying is different for different types of tests. It is important to learn how to study for multiple choice, true/false, short answer, and essay tests (12).

The purpose of the study is to explore whether the assessment method selection would significantly affect the studying and learning approaches of undergraduate nursing students based on comparing MCQ and SAQ examination scores.

This study was designed as a descriptive-analytical research. The study has been conducted according to the declaration of Helsinki. One hundred and ninety one first-year undergraduate nursing students enrolled in the study. Students have given the informed consent. The project was conducted in Mother School (Kerman Nursing) and two 

METHODS

pertaining nursing schools (Zarand Nursing and Bam Nursing Schools) and was composed of two separate second-semester student groups. This course has two credits with defined curriculum that have provided 34 hours of didactic lectures. In our study, the course relied primarily on lecture-based teaching with power point presentation and the main form of assessment used in the curriculum was MCQ and SAQ examinations. All of students were given mid and final term examinations with half of MCQs and half of SAQs, consecutively. The composite mid-term examination consists of a 10 multiple-choice questions paper and 10 SAQs paper and final-term examination consists of a 15 MCQs paper and 15 SAQs paper. The students were required to answer two examination papers separately for mid and final-term.

At the end of the final course examination, two structured questionnaires were distributed to ask the students to rank the two different assessment methods based on their preference. Apart from first structured questionnaire that composed of the demographic and academic characteristics of each nursing student, the second questionnaire sought information on student’s studying and learning approaches that may affect the student's learning outcome.

The second structured 12-item questionnaire (MASLAQ) (13) designed based on a modified 5-point Likert scale in outline of visual analogue scale to ask students’ studying and learning approaches. In our study, visual analogue scale including a horizontal line which was divided to two equal parts by a zero point. Choices of each query based on 5-point Likert scale were located on two equal parts of line that joint through a zero point. Each part of line related to an assessment method. On the Likert scale (1 = strongly disagree, 4 = strongly agree and 0= no difference), MCQ examination separate from SAQ examination was marked. To answer to each query of the second 12-item questionnaire, according to students’ personal viewpoint, students were then allowed place a mark on only one of square of related below line on the scale from 0 to 4 for only one exam.

Face validity of 12-item questionnaire was confirmed appropriate. One-week test-retest reliability of the 12-item questionnaire was measured by Cronbach’s alpha at 0.81 and 0.81, respectively. Reliability of the validated 12-item questionnaire was measured by Cronbach’s alpha at 0.77 (13). Range of agreement coefficient of kappa and Pearson’s correlation between pre and post-test questionnaire queries were computed in another article (13).

Statistical analysis was done with the Chi-square test, independent t-test, one way analysis of variance and ANCOVA. SPSS 16.0 was used for statistical analysis and p < 0.05 was considered significant.

Male students comprised 36.7% of the total participants. Mean (SD) admittance age to university for male and female students was 19.68 ± 3.26 and 19.06 ± 2.14 years, respectively. Baseline characteristics of nursing students according to schools are shown in Table 1.

There was a response rate of 99 percent for completing second questionnaire (189/191). Generally, students favorably

RESULTS

and significantly rated the queries of questionnaire on SAQ assessment part higher than in MCQ assessment part. In order to avoidance of confusing and more precise interpretation of collected data from second questionnaires, a shift is necessary from an assessment method to another method. Then, MCQ assessment’s viewpoints were accurately converted to SAQ assessment’s viewpoints. Subsequently, the scores of strongly disagree, disagree, agree, and strongly agree for multiple-choice option modified to strongly agree, agree, disagree, and strongly disagree short-answer option. Table 2 shows overview frequency of nursing students’ scores for SAQ assessment. With integration of disagree and strongly disagree under one item and integration of agree and strongly agree under another item, analytical analysis allowed us to interpret and deduce agreement between two assessment methods (Table 3).

Female students who participated in the study performed 

significantly better in the learning outcomes in SAQ and MCQ examinations grades and the total grade of both examinations was better than male students who participated in the study (p= 0.001, 0.001 and 0.030, respectively) (Table 4). Table 5 shows responses frequency to studying and learning approaches questionnaire according to lowest and highest quartiles of students’ MCQ and SAQ marks. Most students selected short-answer assessment method with attention to majority queries.

1. Motivation for more profound course study, 2. More time-consuming study for answering to examination, 3. More expanded study for answering to examination, 4. Better and more profound understanding of course meanings, 5. Thinking, deliberation, interpretation and analysis of knowledge in learning process, 6. Capability of retrieval of knowledge in future7. Appropriate evaluation method for nutrition course, 8. Appropriate evaluation method for knowledge level, 9. Appropriate evaluation