Effect of levels of realism of mobile-based pedagogical agents on health e-learning

Document Type : Original Article

Authors

1 Department of Educational Sciences, Sayyed Jamaleddin Asadabadi University, Asadabad, Iran

2 M.A Student, Department of Educational Sciences, Islamic Azad University, Science and Research Branch, Tehran, Iran

Abstract

Background: One of the ways for effective communication between learners and instructional multimedia content in mobile learning systems is taking advantage of characters or pedagogical agents. The present study aimed to investigate the effect of the levels of realism in mobile-based pedagogical agents on health e-learning.
Methods: The study was quasi-experimental with a pretest-posttest design involving three experimental groups. The target population included those participants who themselves or one of their relatives suffered from digestive disorders. The sample consisted of 48 participants selected through a convenience sampling procedure and were randomly assigned to either of the groups. The instruments and materials included instructional multimedia lessons and learning tests (pretest and posttest). The instructional multimedia content consisted of instructional materials related to familiarity with the human digestive system, its function, and relevant disorders. The participants in each group were exposed to the same instructional content but with a different level of realism related to the pedagogical agent (i.e., iconic, semi-iconic, and realistic). The instructional multimedia lessons were delivered through a mobile-based health leaning management system. For the data analysis, an analysis of covariance (ANCOVA) was applied.
Results: The results showed that the group with the realistic pedagogical agent (M = 29.29, SD = 4.20), compared with the iconic pedagogical agent (M = 25.53, SD = 2.99), performed better on learning measurement (p = 0.20).
Conclusion: The employment of pedagogical agents, as one of the influential tools in improving learners' motivation and learning, should receive greater attention in designing and developing instructional multimedia, especially in the field of health learning services.

Keywords

Main Subjects


Introduction

The use of mobile learning systems, as one of various e-learning presentation formats, has taken on more importance in recent years. Easy and wide access to the Internet in general and wireless technologies in particular has influenced health care systems, resulting in e-health services (1, 2); among such services is health e-learning delivered in the form of mobile learning. Mobile learning, as one form of e-learning, could be performed anywhere and anytime via mobile means of communication such as cell phones (3, 4). While such learning systems have potentially provided users with a wide range of possibilities, they also demand increasingly various requirements that might pose serious challenges; these requirements included providing an interactive environment that helped learners benefit from the instructional content and other relevant services based on their own goals, knowledge levels, and preferences (5, 6).

The most obvious characteristic of a mobile learning system is its powerful and interactive user interface (7). Among the bewildering array of features that user interface could potentially provide, one of the most effective ways in establishing effective interaction between a learner and computer is the employment of pedagogical agents (8,9). Pedagogical agents are animated cartoon characters, talking video images or avatars on the screen that interact with users and help them through learning process during different parts of e-learning programs (10). Animated pedagogical agents feature greater ability to simulate real-world learning environments and engage learners in learning activities through creating multiple interaction instances or spending more time in the learning environments (11-14). Animated pedagogical agents are designed in a way to provide the opportunity for a learner’s interaction with a virtual teacher or learner (12). The use of pedagogical agents in instructional multimedia is based on different theories of psychology and learning including the cognitive theory of multimedia learning, social interaction theory, cognitive load theory, and social cognitive theory.

Based on the personalization principle in Mayer’s cognitive theory of multimedia learning, it is recommended that (a) the verbal content be provided in an interactive style, (b) screen characters be employed to increase learning levels, and (c) the author of the content be also visible to increase learning levels (15). Moreover, according to the social interaction theory, computers are interpreted as social companions of humans. This theory postulates that the inclusion of social cues, either verbal (e.g., spoken words) or non-verbal (e.g., gesture, gaze, and movement), can simulate human-to-human communication in multimedia environments, hence resulting in activating learners in the learning process (16). According to this theory, the use of visual and verbal social cues in computer-mediated environments can encourage learners’ sense of social participation through making them contemplate on their communication with computers (17).  Pedagogical agents, as a social companion with human-like voice and friendly behavior, engage learners in the process of constructing meaning and concept, which in turn, enhances the chances of learning transfer (7). Furthermore, according to the cognitive load theory, if the mental load imposed as a result of the instructional content is beyond the limited capacity of working memory, learning would be impeded (18). Relevant empirical research also recommends that a combination of auditory and visual presentation of materials via animated pedagogical agents (in contrast to speech or text alone displayed on the screen in the absence of agents) can be applied in order to stimulate learners’ deep processing of information (19). In his social cognitive theory, Bandura also saw most of learning as a result of observation; that is, the observation of events and activities in which one does not personally play any role but acquires many rules and knowledge throughout. According to this theory, the use of pedagogical agents in instructional multimedia seems necessary because learners can master many of the principles and concepts simply by observing another person (17).

Several studies have been conducted on pedagogical agents and their role in multimedia learning environments. Yılmaz and Kılıç (20) and Clark and Mayer (21), for example, observed relatively greater improvements in terms of academic achievement, retention, and attitude among the participants exposed to human-like pedagogical agents. Domagk (22) also showed that the mere presence of pedagogical agents did not lead to an increase in learning and motivation; rather, such an increase occurred when the agents were visually appealing. The results of their study suggested the important role of the voice and appearance of pedagogical agents in the enhancement of learning process. Similarly, in a series of studies, Baylor and Kim (23-25) revealed that the gesture and facial expressions of agents, like speech interactions, could positively affect learners’ learning, as well as their perceptions of the agents. Moreover, expert-like pedagogical agents could lead to greater access to information, motivator-like agents could increase self-efficacy, and mentor-like agents could improve learning and motivation. In addition, learners tend to learn better and more when they deal with pedagogical agents with more realistic characteristics and also when the agents play the role of experts. In this regard, Baylor and Ebbers (26) reported that the presence of two distinct pedagogical agents, representing the roles of expert and motivator, could produce a greater impact on learning and perceived value of the agents. Merrill (27) also indicated that the participants exposed to animated agents performed better than those not presented with any agents. Likewise, Baylor et al. (28) observed greater improvements in the participants’ motivation with machine-like voice on animated agents and also with human-like voice on non-animated agents. The results of Moundridou and Virvou’s (29) study also revealed that pedagogical agents used in an intelligent tutoring system could boost learners’ educational experience. In another study, Kim et al. (30) noted the significant effect of the image of agents on the perception of the role they played. In two experiments conducted by Atkinson (17), it was also shown that the learners who observed the agents providing oral explanations (i.e., agents with voice) performed better in transferring learning when compared with the learners in the control group. Gharabaghi (31) also reported that mentor-like pedagogical agents, compared with expert-like agents, could facilitate learning and motivation more in a science course. In a similar vein, social interaction between learners and agents has been studied in several studies (32, 33). Such research on different groups of learners suggested that factors such as learners’ age, gender, and educational grade might affect the choice of pedagogical agents (14, 34). Therefore, the selection of pedagogical agents should be carefully tailored to the specific groups of learners (35).

As mentioned earlier, pedagogical agents could have different features among which are the visual characteristics of agents. Along with the advancement of mobile educational systems, designers of such systems need to draw upon new empirical findings about the influential factors related to the impact of using pedagogical agents with different visual characteristics on learning. In this research, an attempt was made to evaluate the effect of such factors on learning by designing and producing three types of pedagogical agents with three levels of realism delivered through a mobile educational system.

Methods

The present study was quasi-experimental with a pretest-posttest design involving three experimental groups. The participants included in the study were among those who themselves or one of their relatives suffered from digestive disorders; they were selected through a convenience sampling procedure from a number of medical clinics in Hamedan, Iran. The data from the participants who did not complete all phases of the study (i.e., pretest, training sessions, posttest) were excluded from the final analyses. The final sample consisted of 48 participants (29 males, 19 females). The instruments and materials included instructional multimedia lessons and learning tests.  

The instructional multimedia content consisted of instructional materials related to familiarity with the human digestive system, its function, and relevant disorders. The instructional multimedia lessons were designed according to three different levels of realism of the pedagogical agents incorporated into the lessons. In lesson prepared for the first group (viz., iconic group), the pedagogical agent was iconic with the lowest level of visual realism; it was a pedagogical agent with only a plane and linear human representation lacking details of face lines. In the lesson prepared for the second group (viz., semi-iconic group), the pedagogical agent was semi-iconic, two-dimensional, and animated with relatively more details of face lines. And finally, in the lesson prepared for the third group (viz., realistic group), the pedagogical agent was similar to a real human with the highest level of realism and the greatest details. The instructional multimedia lessons were delivered through a mobile-based health learning management system. Two experts confirmed the validity of the multimedia lessons.

The researcher-developed learning test consisted of 48 items on the major concepts of the digestive system. The test was used as both pretest and posttest but with different item orders in order to reduce the practice effect; each correct answer was given one score. To determine the content validity of the test, three health professionals examined the items and provided modifications; the test was also piloted among a group of 12 people, who did not take part in the study but were from the same population. Accordingly, necessary changes were made. To determine the reliability of the test, the Cronbach’s alpha was used, resulting in the acceptable coefficient of 0.85.

The study was implemented in a computer lab in Hamedan, Iran during the winter of 2019. One week before the main phase of the study, the participants did the pretest. Following two training sessions, they completed the posttest. For the data analyses, descriptive and inferential statistics, including the Analysis of Covariance (ANCOVA) were conducted to compare the group means at p < 0.05 using the SPSS (version 21). Before the analysis, however, the main relevant statistical assumptions (i.e., normal distribution of data, homogeneity of variance, and homogeneity of regression slopes) were examined and no obvious violations were detected.

Results

First, the descriptive statistics of the participants’ pretest and posttest scores across different groups were examined (see Table 1).

 

Table 1. Descriptive statistics of participants’ pretest and posttest scores (N = 48)

Learning measure

Experimental groups

Iconic (n = 15)

 

Semi-iconic (n = 16)

 

Realistic (n = 17)

M

SD

M

SD

 

M

SD

 

 

 

 

 

 

 

 

 

Pretest

17.53

5.77

 

17.50

5.63

 

17.41

5.47

Posttest

25.53

2.99

 

 

26.94

3.77

 

29.29

4.20

 

 

 

 

 

 

 

 

 

 

 Further, the ANCOVA was conducted in order to investigate whether there was a significant difference among the different groups in terms of learning. The independent variable included group membership with three levels (i.e., iconic, semi-iconic, and realistic), and the dependent variable was the participants’ posttest scores; in addition, their pretest scores were included as a covariate to remove the participants’ pre-existing differences in terms of the knowledge of the instructional materials. Before the analysis, however, the main relevant statistical assumptions (i.e., normal distribution of data, homogeneity of variance, and homogeneity of regression slopes) were examined and no obvious violations were detected. Table 2 shows the results of the ANCOVA.

 

Table 2. Results of ANCOVA for participants’ posttest scores (N = 48)

Source

Sum of squares

df

Mean squares

 F

 p

Effect size a

 

 

 

 

 

 

 

Pretest

3.27

1

3.27

0.23

0.631

0.01

Group

116.83

2

58.41

4.17

0.022*

0.16

Error

616.93

44

14.02

 

 

 

 

 

 

 

 

 

 *p

a partial eta-squared.

As shown in the table, the participants’ posttest scores were significantly different, F (2, 44) = 4.17, p =0.022, partial eta-squared = 0.16. The effect size was large based on the Cohen’s guidelines (36), indicating that about 16 percent of the difference in posttest scores were explained by group membership. Furthermore, the results of Tukey’s HSD test revealed that the participants in the group with the realistic pedagogical agent (M =29.29, SD= 4.20), compared with those in the group with the iconic pedagogical agent (M =25.53, SD = 2.99), did better on the posttest (p=0.020). In contrast, in terms of their performance on the posttest, no significant difference was noticed between the iconic and semi-iconic groups (p=0.906) or between the realistic and semi-iconic groups (p=0.231). The results, thus, implied the effectiveness of realistic pedagogical agents in increasing learning levels.

 Discussion

Pedagogical agents are considered as one of the tools to communicate effectively between learners and the content of instructional multimedia in a mobile learning environment. This study was conducted with the aim of investigating the effect of the levels of realism in pedagogical agents on mobile health e-learning. The participants were assigned into three groups, each given an instructional multimedia lesson including a pedagogical agent with a different level of realism (viz., iconic, semi-iconic, and realistic). The lessons were delivered via a mobile learning system. The findings of the study suggested the effectiveness of the multimedia including the pedagogical agent with the highest level of realism (i.e., realistic) in the participants’ learning level. This was consistent with the results reported by Yılmaz and Kılıç (20), Domagk (22), and Baylor and Kim (24). These studies also showed the effects of higher levels of realism, in terms of visual appearance and the voice of pedagogical agents on learners’ academic achievement, motivation, attitude, and learning transfer. The findings of the current study were also in line with those reported in Baylor and Kim’s (23) study indicating that gesture and facial expressions, as well as speech interactions, tended to have an effect on learners’ level of learning and their perception of pedagogical agents. In the same line, Merrill (27) found that learners faced with animated pedagogical agents with more realistic human characteristics performed better on learning measures. Similarly, Moundridou and Virvou (29) showed that pedagogical agents with human-like faces could improve learning. Overall, the results of empirical research suggested that the amount of realism and the image of pedagogical agents could produce a profound effect on learners’ motivation and learning (6, 12); in fact, the more realistic and attractive the pedagogical agents are, the more attentive the learners become (23). According to Mayer’s (15) cognitive theory of multimedia learning, in the process of providing information by a pedagogical agent, the agent itself plays a very important role in attracting attention; as a result of greater attention, greater interaction is created; and consequently, motivation and learning are enhanced. The findings obtained in the present study could, thus, be explained by the fact that realistic and human-like pedagogical agents could produce greater plausibility, resulting in greater attention while non-realistic and strange-looking agents with unusual behavior might lead to distraction and diversion; therefore, the actual shape of pedagogical agents could also help increase motivation and learning. Moreover, based on Vygotsky’s theory, pedagogical agents could provide an enriched learning opportunity for engaging and motivating learners through creating an interactive environment between computer and user. In addition, pedagogical agents are able to instruct complicated tasks, make use of moves and gestures for drawing learners’ attention to the most important parts of the content, and transfer emotional reactions to the instructional environment. The use of pedagogical agents with the highest levels of realism in multimedia learning environments not only can simulate real-life learning situations, it can also result in greater attention to particular instructional points. In other words, human-like pedagogical agents play a significant role in guiding and directing learners’ attention to the main parts of the material, so that learners will be more interested and motivated to be engaged in instructional multimedia lessons enhanced with realistic and human-like pedagogical agents when compared to those with non-realistic, unreal, and cartoon-like pedagogical agents (14). One of the limitations of this study was the convenience sampling procedure that restricted the generalizability of the findings. Given the results of the current study, it is recommended that the designers and developers of mobile instructional multimedia content should draw more upon standards and principles in this regard and make greater use of human-like pedagogical agents with higher levels of realism in order to increase learners’ motivation and learning.

Ethical considerations: Ethical issues (Including plagiarism, informed consent, misconduct, data fabrication and/or falsification, double publication and/or submission, redundancy, etc.) have been completely observed by the authors.

ACKNOWLEDGMENTS

Authors would like to acknowledge all those who cooperated with them in this research project.

Financial Support:

This article is part of the Master's thesis for Educational Technology with the code number 63191 which was funded by Islamic Azad University, Science and Research Branch, Tehran, Iran.

Conflict of Interest: The authors declare that they have no competing interests.

  1. Chang CY, Lai CL, Hwang GJ. Trends and research issues of mobile learning studies in nursing education: A review of academic publications from 1971 to 2016. Comput Educ. 2018;1;116:28-48.

2.   Istepanian RS, Al-Anzi T. M-Health 2.0: new perspectives on mobile health, machine learning and big data analytics. Methods 2018; 151:34-40.

3.   Taylor JD, Dearnley CA, Laxton JC, Coates CA, Treasure‐Jones T, Campbell R, et al. Developing a mobile learning solution for health and social care practice. Distance Education 2010; 31(2):175-92.

4.   Mellow P. The media generation: Maximize learning by getting mobile. Proceedings of Conference of ASCILITE 2005: Balance, Fidelity, Mobility: maintaining the momentum?; 2005 Dec 4-7; Queensland, Australia. P. 469-76.

5.   Schroeder NL, Adesope OO. How does a contextually-relevant peer pedagogical agent in a learner-attenuated system-paced learning environment affect cognitive and affective outcomes?. Journal of Teaching and Learning with Technology 2013; 11:114-33.

6.   Rosenberg-Kima RB, Baylor AL, Plant EA, Doerr CE. Interface agents as social models for female students: The effects of agent visual presence and appearance on female students’ attitudes and beliefs. Comput Human Behav. 2008; 24(6):2741-56.

7.   Baylor AL, Kim S, Son C, Lee M. Designing effective nonverbal communication for pedagogical agents. In: Looi C-K, McCalla G,  Bredeweg B, Breuker J, editors. Proceedings of the 2005 conference on Artificial Intelligence in Education: Supporting Learning through Intelligent and Socially Informed Technology 2005; Amsterdam. The Netherlands: IOS Press; 2005. P. 744-46.

8.   Holmes J. Designing agents to support learning by explaining. Comput Educ. 2007; 48(4):523-47.

9.   Robertson J, Cross B, Macleod H, Wiemer-Hastings P. Children's interactions with animated agents in an intelligent tutoring system. Int J Artif Intell Educ 2004;14 (3, 4):335-57.

10.    Clark, R.C, Mayer, R.E. E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. John Wiley & Sons; 2016.

11.   Mayer RE, DaPra CS. An embodiment effect in computer-based learning with animated pedagogical agents. J Exp Psychol Appl. 2012; 18(3):239.

12.   Woo HL. Designing multimedia learning environments using animated pedagogical agents: factors and issues. J Comput Assist Learn. 2009; 25(3):203-18.

13. Darwish H. The “persona effect”: Shortcomings in the evaluation of pedagogical agents' embodiment. Proceedings of 2014 International Conference on Web and Open Access to Learning (ICWOAL); 2014 Nov 25-27; Dubai, United Arab Emirates. IEEE; P.1-5.

14.   Moreno R, Flowerday T. Students’ choice of animated pedagogical agents in science learning: A test of the similarity-attraction hypothesis on gender and ethnicity. Contemp. Educ. Psychol. 2006; 31(2):186-207.

15.   Mayer, R.E. Multimedia learning. Cambridge University Press; 2007.

16.   Louwerse MM, Graesser AC, Lu S, Mitchell HH. Social cues in animated conversational agents. Appl Cogn Psychol. 2005;19 (6):693-704.

17.   Atkinson RK. Optimizing learning from examples using animated pedagogical agents. Journal of Educational Psychology 2002; 94(2):416.

18.   Sweller, J. Cognitive load theory. In: Mestre JP, Ross BH, editors. The psychology of learning and motivation:Cognition in education. San Diego, CA, US: Elsevier Academic Press; 2011. P. 37-76. (The psychology of learning and motivation; Vol. 55). 

19.   Dunsworth Q, Atkinson RK. Fostering multimedia learning of science: Exploring the role of an animated agent’s image. Comput Educ. 2007; 49(3):677-90.

20.   Yılmaz R, Kılıç-Çakmak E. Educational interface agents as social models to influence learner achievement, attitude and retention of learning. Comput Educ. 2012;59(2):828-38.

21.   Clark RC, Mayer RE. Learning by viewing versus learning by doing: Evidence‐based guidelines for principled learning environments. Performance Improvement 2008; 47(9):5-13.

22.   Domagk S. Do pedagogical agents facilitate learner motivation and learning outcomes? J Media Psychol. 2010; 22(2):84-97.

23.   Baylor AL, Kim S. Designing nonverbal communication for pedagogical agents: When less is more. Comput Human Behav. 2009; 25(2):450-57.

24.   Baylor AL, Kim Y. Pedagogical agent design: The impact of agent realism, gender, ethnicity, and instructional role. In: Lester J.C., Vicari R.M., Paraguaçu F, editors. Proceedings of 7th International conference on intelligent tutoring systems; 2004 Aug 30-Sep 3; Maceio, Alagoas, Brazil. Lecture Notes in Computer Science, vol 3220. Springer, Berlin, Heidelberg; P. 592-603.

25.   Baylor A, Kim Y. Validating pedagogical agent roles: Expert, motivator, and mentor.  In: Lassner D, McNaught C, editors. Proceedings of ED-MEDIA + Innovate Learning 2003--World Conference on Educational Multimedia, Hypermedia & Telecommunications. Honolulu, Hawaii, USA. Association for the Advancement of Computing in Education (AACE); P. 463-66.

26.   Baylor A, Ebbers S. The pedagogical agent split-persona effect: When two agents are better than one. In: Lassner D, McNaught C, editors. Proceedings of ED-MEDIA + Innovate Learning 2003--World Conference on Educational Multimedia, Hypermedia & Telecommunications. Honolulu, Hawaii, USA. Association for the Advancement of Computing in Education (AACE); P. 459-62.

27.  Merrill MM. The role of animated pedagogical agent in multimedia learning environments. [PhD thesis]. Mississippi State University; 2003.

28. Baylor A, Ryu J, Shen E. The effects of pedagogical agent voice and animation on learning, motivation and perceived persona. In: Lassner D, McNaught C, editors. Proceedings of ED-MEDIA + Innovate Learning 2003--World Conference on Educational Multimedia, Hypermedia & Telecommunications. Honolulu, Hawaii, USA. Association for the Advancement of Computing in Education (AACE); P. 452-58.

29.   Moundridou M, Virvou M. Evaluating the persona effect of an interface agent in a tutoring system. J Comput Assist Learn. 2002;18(3):253-61.

30.   Kim Y, Baylor A, Reed G. The impact of image and voice with pedagogical agents. In: Rossett A, editor. Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2003; Association for the Advancement of Computing in Education (AACE); P. 2237-240.

31.  Gharabaghi S. The effect of pedagogical agent roles on learning, facilitating learning and learning motivation in Science course [MSc thesis]. Tehran: Allameh Tabataba’i University, 2010. Persian.

32.   Gulz A, Haake M. Social and visual style in virtual pedagogical agents. Proceedings of the Workshop on Adapting the Interaction Style to Affective Factors, 10th International Conference on User Modelling, (UM’05); 2005 Jul 24-27; Edinburgh, Scotland. Lund University; 2005.

33.   Kim Y, Baylor AL. Pedagogical agents as social models to influence learner attitudes. Educational Technology 2007; 1:23-28.

34.  Craig SD, Gholson B, Driscoll DM. Animated pedagogical agents in multimedia educational environments: Effects of agent properties, picture features and redundancy. J. Educ. Psychol. 2002; 94(2):428.

35.   Kim Y, Wei Q. The impact of learner attributes and learner choice in an agent-based environment. Comput Educ. 2011; 56(2):505-14.

36.   Cohen J. Statistical power analysis for the behavioral sciences. 2th ed. Newbury Park, CA: Lawrence Erlbaum Associates; 1988.