Real Time Electronic Formative Assessment in Medical Education and Its Impact on Motivational Beliefs and Self-regulation Strategies

Assessment is one of the important steps in any teaching-learning process [1, 2]. Brown and Knight consider assessment as the heart of student learning experience [3]. In new learning theories, classroom assessment is recognized as an important element in both teaching and learning processes [4, 5].According to Butler and Mcmunn (2006), assessment can be distinguished under three types of diagnostic, formative, and summative, which date back to Scriven (1967) who first made the distinction [6]. 

Formative assessment is defined as a continuous process of monitoring the learners' progress in obtaining knowledge [7]. In another definition, this kind of assessment is defined as "the process of seeking and interpreting evidence for the use by learners and teachers to decide where the learners are in their learning, where they need to go, and how best to get there" [8]. New learning environments are targeted toward the establishment and development of a deep and meaningful learning approach in students [9, 10]. In such learning environments, students have an active, constructive, self-regulated, targeted, and collaborative role, who engage themselves in these environments with the goal of self-promotion [11].

Formative assessment has a positive effect on the learner empowerment and learning self-regulation by helping students to find their strengths and weaknesses [12, 13]. On the other hand, the reduction of learning motivation in medical students is a challenge that has been considered in the recent years. Formative assessment can be used as an option for the enhancement of learner's motivation.

According to some evidence, the use of formative assessment, along with the summative assessment, can improve the quality of education, increase the learning motivation, and improve self-regulation learning in students [13]. In the same vein, Faber et al. reported that the use of electronic formative assessment tools exerts a positive impact on learners' academic achievement and motivation [14].

The advancement of technology has resulted in the establishment of many educational opportunities in the learning environments. One of these tools is the real-time electronic system in the classroom [15]. These technologies facilitate recurring opportunities for students to practice. Electronic formative assessment cannot only involve and motivate students, but also help them monitor their progress and learning speed [16]. These systems enjoy many merits, such as improvement of attention, enhancement of knowledge acquisition, possibility of performing secret surveys, tracking of individual responses, instant display of the responses of questions, creation of an interactive and entertaining learning environment, collection of information for reporting, and implementation of analysis [17].

It should be also noted that such assessments motivate the learners to reflect, discuss, and participate in the learning process [18]. One of the major challenges in the field of medical education is that in large classes, it is difficult to achieve high and acceptable lecturing standards and present materials while involving the students and increasing their participation and academic engagement. Individual response technology encourages active learning [19] and leads to the activation of higher levels of cognition in students.

It should be noted that motivation is one of the most important determinants of learning quality and success. Accordingly, the lack of this construct can well explain such questions as why professors sometimes encounter with discouraged students or learners who have lost their interest or have abandoned their studies and activities, as well as why sometimes the students feel weak or abandoned [20]. One of the objectives of the university environment is to encourage the students to engage in a social, sustainable, and non-threatening environment [21]

The medical education researchers should give special attention to the concept of motivation [20]. However, the mechanisms through which learners adjust their motivation and learning strategies are not fully understood yet [22]. According to Borman and Sleigh, learners can be significantly engaged in the learning environments by means of non-summative marking systems [23]. Electronic methods for feedback presentation can be also developed automatically and continuously to support learners' engagement in learning [15].

According to the literature, real-time electronic formative assessment facilitates the learners’ active participation in the learning environment and allow them to receive immediate feedback [21]. The technique investigated in this study specifically modifies the role of students from a passive position to the condition in which students take the responsibility of their learning process and seek to improve the process through active participation in the learning environment. The present study aimed to find a strategy to support and guide learners in real time [24].

This study also attempted to find out how electronic formative assessment could be provocative for learners. To this end, a learning environment was designed based on real-time electronic formative assessment that involved gamification elements as motivational aspects in the process of teaching and learning. Gamification refers to the mechanical and artistic application of ideas and aesthetic components of games (e.g., context, immediate feedback, competition, stages, achievements, and points) to engage the learners in the problem and motivational activities and promote their learning and problem solving ability [25]. Generally, the purpose of the present study was to investigate the effect of real-time electronic formative assessment-based medical education on students’ motivational beliefs and self-regulation strategies.

Materials and methods                                                                                                        

This randomized clinical trial was conducted on 323 students studying at the Medical School of Mashhad University of Medical Sciences, Mashhad, Iran, during the first semester of the academic year of 2017-2018 using a posttest control group design. The research hypotheses were as follows:

a) Students exposed to real-time electronic formative assessment-based learning model have a higher motivational belief score, compared to the non-exposed students.

b) Students exposed to the real-time electronic formative assessment-based learning model have a higher self-regulation strategies score, compared to the non-exposed students.

Study population and sampling

The study population corresponded to a group of 323 students selected from 1,620 MD students studying at Mashhad University of Medical Sciences using convenience sampling technique. The participants were randomly divided into two groups of intervention (n=159) and control (n=164).

Research process

Given that the difficulty of the educational content of each lesson could disturb the results of the study, there was a need to homogenize this variable. Regarding this, based on the opinions of the relevant experts and professors, the lessons of histology and trunk anatomy, which were similar in terms of difficulty level, were selected from the basic science courses for general medical students. The sessions were divided into two groups of control and intervention.

After the determination of the control and intervention groups, in the intervention sessions, real-time electronic formative assessment was adopted during the teaching process. In this regard, during each session, the instructor asked questions regarding the educational content taught at the same session in the form of PowerPoint slides. The students used keypads to answer the questions.

Based on the rate of the correct answers, the professor instantly (in the real time) decided on continuing the discussion or re-explaining the subject to resolve the gap between what is gained by the students and what they should know. In the control group, the sessions were implemented using the routine educational approach. At the end of the course, the two groups filled out the Motivated Strategies for Learning Questionnaire (MSLQ) developed by Pintrich and DeGroot (1990) [26]. This questionnaire measures the students' motivational beliefs and self-regulation strategies. The MSLQ consists of two main sections, namely motivational beliefs (including three subscales of self-efficacy, intrinsic value, and test anxiety) and self-regulation strategies (including two subscales of cognitive and self-regulation strategies).

Statistical analysis

Regarding the fact that the education was performed by two different lecturers in the control and intervention groups, the educator factor was in interaction with educational method; therefore, the design of the study could be considered with two independent variables.Therefore, a preliminary analysis was carried out separately, considering the roles of the teaching method and educator as independent variables in the subscales of motivational strategies for learning to use the independent t-test.

The results indicated that the intervention group exposed to the real-time electronic formative assessment-based learning showed higher mean scores in self-efficacy, intrinsic value, and self-regulation than the control group (P<0.05). In addition, the intervention group had a lower test anxiety mean score, compared to the control group (P<0.05). The only subscale that was not significantly different between the two groups was cognitive strategies (P>0.05). However, the investigation of educator factor as an independent variable revealed no significant difference between the two lecturers in terms of motivational strategies for learning. The performance of the two groups was comparable regardless of the grouping factor (i.e., lecturer) (P>0.05). Therefore, due to observing no difference in the subscales when considering the educator factor, this factor was ignored in all analyses of motivational strategies for learning subscales.

Since the five components (i.e., self-efficacy, test anxiety, intrinsic value, cognitive strategies, and self-regulation strategies) under study were aggregated in one scale, the separate analysis of the major two components (i.e., motivational beliefs and self-regulation strategies) was avoided, and they were analyzed collectively. Consequently, the research hypotheses were tested using the multivariate analysis of variance (MANOVA) with five dependent variables and one independent variable, namely teaching method.

Results

The distribution of the data related to motivational strategies for learning was tested using skewness and kurtosis. In addition, the Kolmogorov-Smirnov test was employed to investigate the normality of the data.

Table 1. Descriptive statistics and normality of motivational strategies for learning

Table 1 tabulates the descriptive statistics and normality of the data related to motivational strategies for learning. As indicated in this table, the dependent variable had a normal distribution in the subscales of motivational strategies for learning. It should be noted that the variables whose normality was not confirmed by the Kolmogorov-Smirnov test were evaluated using the skewness and kurtosis. Given the fact that there were five subscales in motivational strategies for learning and given the interdependence of these subscales, MANOVA was used to study the hypotheses considering the educational method as the independent variable.

Table 2. Effect of real-time electronic formative assessment-based learning environment on the subscales of motivational strategies for learning