The Effect of Applying Portfolio on the Level of Learning among Nursing Students in Clerkship of Psychiatry Ward

Introduction

Clinical training is considered as one of the important elements of medical education curriculum (1) which leads to improve all three cognitive, emotional and psychomotor skills by applying theoretical training in real clinical environments (2,3). Indeed, clinical experience is an integral part of nursing education that leads to improve students' knowledge in "Doing" as much as "Knowing" (4). So that, it can be said that the quality of nursing education is highly dependent to the quality of clinical experience (5).

According to the nursing students’ points of view, due to the fear of facing with violent and unpredicted behaviors from patients, clinical experience is considered as the most stressful part of nursing education, especially in psychiatry wards (4, 6); as well as, this issue leads to lose development opportunities and clinical learning in psychiatry wards and prevents transplant between clinical theory and practice (7). However, today because of the increasingly rapid development of information, the existing gap between theory and practice in nursing is steadily getting deeper and the main purpose of the educational systems is to cover this gap (8, 9).

According to the pyramid of George Miller, theoretical knowledge is "knowing" (by cognitive activities) and practical knowledge is "how to know" that are acquired by practical training (10); so, the main purpose of the clinical training is to help nursing students "to learn how to learn" (11). So, during last decades, great innovations have been created in theoretical and practical trainings and education was promoted from teacher-center to student-center training (12).

On the other hand, one of the most prominent variables that influences on students' learning is assessment methods. So that, students use deep or flat learning approaches during their academic course and according to assessment demand (13). Therefore, in proportion to transformation of educational system, evaluation system has been changed as well and it was shifted from just measuring knowledge to the methods of measuring abilities and competence (12). Nursing education has focused on developing the quality of trainings in learning sequences to improve educational courses by data collection (14).

In most cases, due to the subjective judgments, lack of content validity and teachers' biases, the traditional methods of clinical assessment have been carried out non-standardly and by chance that is not able to reflect knowledge and clinical competence of students, correctly (15). On the other hand, by believing in modern educational methods (student-center), those assessment methods that reflect and encourage such beliefs should be used (16).

Portfolio is one of the modern assessment methods that first have been applied in USA in 1980 (14). It is one of the functional tests with two main purpose of education and evaluation and it can create a general attitude toward student's learning experience by providing some snapshots from his clinical learning (17). Some of the main features of portfolio in contrast with traditional tests are mentioned as: comprehensive active participation in evaluation, self-evaluation of knowledge and skills, developing reflective and critical thinking, become a professional and responsible student in learning (18).

In fact, thinking about what has been learned is an important part of providing a portfolio (16); so that applying portfolio in education make the student aware of his learning development and educational requirements of students would be studied and identified by observing the recorded works (in portfolio) by teacher (15). Furthermore, portfolio can transplant theory and practice in the process of clinical education and enhance learning by providing some evidence of actual performance in "doing" (19). In this regard, the results of the Karimian et al. research (2009) in Iran indicated that portfolio leads to enhancement of the level of learning in internship among midwifery students in contrast with traditional assessment methods (20), also the results of Khoram-Abadi et al. research (2010) showed that portfolio has more effect on increasing academic achievement on chemistry in psychomotor and cognitive area of pre-university students (21).

On the other side, using portfolio in education had some restrictions such as time-consuming maintenance (with details) for teachers and students, difficult standardization and scoring and decision making about the boundary between acceptability and refusal (16), that may be considered as the reason of its less using in educational system of Iran (22). So that, the results of Koohpaye-Zade et al. research (2012) indicated a limited use of clinical assessments in Medical Sciences Universities of Tehran that is considered as the least level of using portfolio (6%) (23).

Since studies in modern educational and measuring methods of psychiatry wards were limited, and due to unpredictability of events in psychiatry wards  (as a part to suggest and evaluate psychiatric nursing issues specifically) and previous conceptions of nursing students about patients' aggressive behaviors who suffer from psychological disorders (and it can be influenced by medias or family cultures), and on the other hand, the replacement of routine clinical experiences with some assignments such as therapeutic communication with patients, clinical training in psychiatric wards have been associated with many problems which requires a continuous evaluation and review in educational plans (6). So, the researcher was to do a research with the purpose of determining the effect of applying portfolio on the level of learning in the internship environment of psychiatric wards in contrast with routine method of evaluation.

Methods

It was a semi-experimental research and 60 nursing students who were passing internship in Ibne-Sina psychiatry Hospital in 2014 have been studied. Sample size was considered based on Karimian and Heidari research (2009) as "the effect of portfolio evaluation on the level of learning and satisfaction of midwifery students" and by using formula of "comparing the mean and standard deviation of both communities" with calculation of mean and standard deviation of learning score (practical test) in portfolio group (17.69±0.99) and control group (16.87±0.85) (20), with 95% reliability and 80% test ability has been estimated 32 individuals for each group; but 35 individuals were considered for each group (totally 70 individuals) by considering 10% dropping of the sample. So that, at first the qualified students for being studied have been selected by available sampling and then students who wanted to participate in the study were divided non-randomly (according to the pre-determined internship list) in two groups of educational purposes (routine educational method) (35 individuals) and portfolio (35 individuals). At the end of the research, 5 individuals of educational purposes (2 people due to the absence, 2 people due to student work and 1 individual because of group displacement) and 5 individuals of portfolio (2 people due to the group displacement, 1 individual due to absence and 2 people because of student work) were omitted form study, so final sample size included 60 individuals (30 individuals per group). The inclusion criteria for students had two parts that included the lack of passing psychiatric nursing course (failed students of this course), lack of student work experience in psychiatric ward, lack of psychiatric disorders and medicine consumption in this context. Also, inclusion criteria for teachers include having M.A. in psychiatric nursing, having the experience of teaching at least 5 theoretical courses in psychiatric nursing, and the experience of supervising at least 5 internship courses of psychiatric nursing.    

 Students' demographic information form, the questionnaire of assessing students' learning based on educational purposes by student, the questionnaire of assessing students' learning based on educational purposes by teacher, and written- functional test have been used to obtain data.  

Demographic information form consisted of 10 questions which have been completed by students. The questionnaire of assessing the students' learning based on achieving the educational purposes (by students and teachers) was consisted of 25 items which has been set based on educational purposes (and was matched previously) and also it was scored according to Likert 4-degree scale (1=very low, 2=low, 3=well, 4= very well). According to the students and teachers, 25 is the lowest level and 100 is the highest level of achieving educational purposes. Content validity of this questionnaire has been confirmed by 7 professors of nursing and midwifery faculty and its reliability was calculated by internal consistency with Cronbach's alpha coefficient as α=0.85 by students and α=0.93 by teachers.

The written-functional test was consisted of 10 items about performance of learners in psychiatric ward that was provided as two groups of questions with the same items but different orders and its range of scores was considered between 0 and 100. The validity content of the test was confirmed by 5 professors of psychiatric nursing.

At first, the students of other cities who were passing internship in Ibne-Sina psychiatric hospital of Mashhad in 2014 (Sabzevar, Neyshabour, Kashmar, Torbate-Heidarie) were identified after receiving the students' internship plan (from education department of nursing and midwifery faculty of Mashhad). Then, after attracting teachers' cooperation (three qualified teachers who had the responsibility of students' training), students attempted to develop common educational purposes of internship equally and in proportion to the training course. Afterwards, 2 groups were selected from each faculty, the first groups were managed by using routine evaluation methods and the second groups were managed by portfolio and it limited the exchange of information between two groups. In portfolio group, a blank form with students' profile, internship place, teacher's name, and a table that included date, student activity description, student method of learning, rethinking of student and teacher's opinion at the first day of internship was given to the students with a brief explanation about the way to complete the form along with the matched educational purposes. So, the students must provide their clinical activities according to this form, and in order to be evaluated at the end of their internship, they must deliver the form to the teacher (table 1). In the group of usual evaluation, the matched purposes were given to students at the first day of internship and it was carried out by usual evaluation.

Table1. Portfolio implementation

Demographic information form was completed at the first day of internship. The questionnaire of assessing the level of learning based on educational purposes by teachers and students, and short answer written-functional test to evaluate students' academic achievement (by teacher) were completed on the final day of internship. In all phases of the research, all ethical issues of the research such as obtaining written agreement from Ethic Committee of University, receiving a written introduction from faculty of nursing and midwifery and presenting to the head of Ibne-Sina psychiatric hospital of Mashhad, obtaining a consciously written consent from participants of the research and ensure them that they can leave the research if they do not want to continue the study, have been considered and are approved by the research department of University of Medical Sciences.

Data analysis was carried out by SPSS software version 11.5, statistical tests of Kolmogorov - Smirnov and Shapiro Wilk (to examine normal distribution of data), dependent t-test (to study quantitative variables after intervention), K², and exact Fisher (to compare the distribution of qualitative variables before intervention). In conducted tests, the level of confidence was 95% and the level of significance was calculated as α=0.05.

Results

According to table 2, there was no significant difference between the two groups of portfolio and educational purposes in terms of frequency distribution of students' demographic characteristics such as sex, age, marital status, city of education, total average, and the level of interest in psychiatric nursing (p>0.05) and both groups were equal. In portfolio group, the most frequency distribution of single students' demographic information with high rate of interest in psychiatric nursing was 70% (21 individuals), and the mean and standard deviation of their total average was 16.2±1.5 (table 1).

In educational purposes group, the most frequency distribution of single students' demographic information was 53.3% (16 individuals) and this percentage for those single students who had high interest in psychiatric nursing was 63.3% (19 individuals) and the mean and standard deviation of their total average was calculated as16.5±1.2 (table 2).

According to table 3, the mean score of learning among students of portfolio group by teachers (81.3±9.3) was more than mean score of learning among students of educational purposes group by students (62.4±4.8). The results of dependent t-test indicated that at the end of internship, there was significant difference between portfolio group (81.3±9.3) and educational purposes group (62.4±4.8) and in terms of mean score of nursing students' learning by teachers (p<0.001).

Also, the mean score of students' learning by students in portfolio group (72.7±13.6) was more than mean score of students' learning in educational purposes (59.1±16.3), and the results of dependent t-test showed that at the end of internship there was significant difference between portfolio group (72.7±13.6) and educational purposes group (59.1±16.3) in terms of mean score of nursing students' learning by students (self-evaluation) (p<0.001).  

Due to table 3, mean score of learning by teachers in portfolio group (72.7±13.6) was more than mean score of learning by students (72.7±13.6). Also, mean score of learning by teachers (61.9±4.8) was more than mean score of learning by students (59.1±16.3).

On the other hand, mean score of academic achievement (that was achieved from written-functional tests) in portfolio group (61.8±14.7) was more than mean score of academic achievement in educational purposes group (53.0±14.2). the results of dependent t-test indicated that there was significant statistical difference in terms of mean score of academic achievement between portfolio (61.8±14.7) and educational purposes at the end of internship course (p=0.02) (Table4).

Table2. The overall profile of studied nursing students separately in two groups of portfolio and educational purposes

Table3. The comparisons of nursing students' mean score (by teachers and students) in two groups of portfolio and educational purposes

Table4. The comparisons of nursing students' mean score of academic achievement in two groups of portfolio and educational purposes    

Discussion

The purpose of the present research was to determine the effect of applying portfolio on the level of learning among nursing students in internship course. According to the results, mean score of the level of learning among nursing students in internship course of portfolio group was significantly increased in contrast with educational purposes. Also, the results of the present research showed that the score of written-functional test in portfolio group was more than educational purposes group.

 The results of Karimian et al. research (2009) based on the fact that applying portfolio leads to increase the level of learning among 6^th semester midwifery students from teachers' point of view (20) was consistent with the results of the present research; however, self-evaluation has not been studied by students. The results of Tasdemir et al. research (2009) in India was also based on positive effect of both cooperative learning method and portfolio on improving the level of learning in clinical education (24); and it was consistent with the results of the present research. The results of Khoram-Abadi et al. research (2010) was based on the effect of performance measurement method and portfolio on improving male pre-university academic achievement in psychomotor and cognitive areas of chemistry (21) and the results of Hekmat-Poor (2012) was based on nursing students' attitude on the fact that assessment accuracy by portfolio was more than routine evaluation methods (25) and they confirm the results of the present research.

In fact, psychiatric internship was a good opportunity to promote clinical education and integrate theory and practice in issues relevant to care and treatment of patients who suffer from psychological disorders, therapeutic communication and eliminating social stigma for these patients. Since most of medical sciences students have negative attitude toward psychological disorders before clinical training, usually such negative attitudes may be increased in psychiatric internship and communicating with patients. So that, students may feel stress and anxiety when interact with patients and they may imagine that working on these patients is dangerous and treatment prognosis of them is bad (26). So, in comparison with other clinical parts, clinical education in psychiatric internship was very important in terms of learning development in emotional and psychomotor areas. So that, studies showed that knowledge and clinical experience in psychiatric wards can create a more positive attitude toward mental health disorders and taking care of patients who suffer from psychological disorders. Also, it is valuable to make students ready to take care of spiritual and psycho-social needs of patients who suffer from physical illnesses (6). In the present research, due to the fact that most students are interested in psychiatric nursing; portfolio method can lead to learning development and academic achievement by activating students in clinical training. Because motivation (as a mental preparation) is considered as a prerequisite for learning by educational psychologists and it leads to learning development. So that, if the students are not interested in this course; they would not do their assignments seriously, but if they are interested in the course, they would seek for more information about the course in addition to do educational assignments (27).

In the present research, the researcher confirms more on specific situations and psychiatric wards and because of having the least physical and clinical works, the most professors, regardless of increasing the students' motivation and making positive attitude toward psychiatric wards and without scientific documentations, and more based on their imagination about the students' performance during their internship, they would start scoring and evaluating the student in clinical environment; that it can lead to reduce the initial motivation among students. According to the results of the present research, applying portfolio through self-directed learning development and forcing students to discover, search and self-evaluation leads to maintain motivation and stimulate musing learning strategies among students and provide a situation to promote motivation for continuous learning and finally it leads to academic achievement.                     

The results of Belcher et al. research (2014) was based on medical sciences students' attitude about electronic portfolio that was performed in all four academic years and due to the lack of feedback from teachers it had no effect on their learning (28) and it had no consistency with the results of the present research. It can be mentioned that the duration of its performance and absolute self-evaluation of students through portfolio may be considered as reasons of this lack of consistency. On the other side, it was a qualitative study and was carried out on a quantitative sample size (15 individuals).

In order to mention the limitations of the present research, we can refer to impossibility of random replacing of students into two groups of portfolio and routine evaluation. Due to the lack of sample size of this plan in summer and according to its implementation on nursing students of other universities in different cities, internship groups has been determined previously.

According to the results, applying portfolio may lead to learning development and academic achievement among nursing students in psychiatric internship by teachers and students. It is suggested for the future studies to implement the effect of applying portfolio randomly by teachers and students on the level of learning among nursing students.

Acknowledgement

This article was extracted from a research plan with code of 930564 that was approved by Research Department of University of Medical Sciences of Mashhad in 3.9.2014. Thereby, we appreciate University of Medical Sciences of Mashhad for validating the research, and also we are grateful to officials of nursing and midwifery faculty of Mashhad and teachers and students that take part in the research.