The Effect of Applying Portfolio on the Level of Learning among Nursing Students in Clerkship of Psychiatry Ward

Document Type: Original Article

Authors

1 Evidence Based Care Research Centre, Nursing and Midwifery School, Mashhad University of Medical Sciences, Mashhad, IRAN

2 School of Nursing and Midwifery, Mashhad University of Medical Sciences, Mashhad, IRAN

3 Evidence Based Caring Research Centre, School of Nursing and Midwifery, Mashhad University of Medical Sciences, Mashhad, IRAN

4 Department of Medical Education, School of Medicine, Mashhad University of Medical Sciences, Mashhad, IRAN

5 Department of Medical Surgical Nursing, Birjand University of Medical Sciences, Birjand, IRAN

6 Department of Health, Neyshabur University of Medical Sciences, Neyshabur, IRAN

Abstract

Background: Due to unpredictability of events in psychiatry environment and replacement of routine clinical experiences with some assignments such as therapeutic communication with patients, clinical training in such environments has been associated with many problems which require continuous review and evaluation.  The purpose of the present research was to determine the effect of applying portfolio on the level of learning among nursing students in internship of psychiatry ward.
Methods: It was a semi-experimental study on 60 nursing students of the 4th semester who were passing internship in Ibne-Sina psychiatry hospital of Mashhad (summer, 2014), they have been studied in two groups of Portfolio and Educational purposes (the usual method of evaluation) non-randomly. Before intervention, the educational purposes were matched for both groups. The questionnaire of evaluating the level of learning among nursing students (by teachers and students), and written-functional test were used to obtain data. Data has been analyzed by version 11.5 SPSS software, COSCO statistical tests, Fisher's exact and independent t-test (p<0.05).
Results: The participants consisted 48.3% (29 individuals) female and 51.7% (31 individuals) male. The results of independent t-test showed that there was significant difference between the group of Portfolio (81.3±9.3) and Educational purposes (62.4±4.8) in terms of mean score of learning by teachers and also by the students, and there was significant difference between groups of Portfolio (72.7±13.6) and Educational purposes (59.1±16.3) (p<0.001).
Conclusions: According to teachers and students,applying portfolio on nursing students may leads to clinical learning improvement. So, it can be recommended to the nursing teachers to use portfolio in their clinical trainings. 

Keywords


Introduction

Clinical training is considered as one of the important elements of medical education curriculum (1) which leads to improve all three cognitive, emotional and psychomotor skills by applying theoretical training in real clinical environments (2,3). Indeed, clinical experience is an integral part of nursing education that leads to improve students' knowledge in "Doing" as much as "Knowing" (4). So that, it can be said that the quality of nursing education is highly dependent to the quality of clinical experience (5).

According to the nursing students’ points of view, due to the fear of facing with violent and unpredicted behaviors from patients, clinical experience is considered as the most stressful part of nursing education, especially in psychiatry wards (4, 6); as well as, this issue leads to lose development opportunities and clinical learning in psychiatry wards and prevents transplant between clinical theory and practice (7). However, today because of the increasingly rapid development of information, the existing gap between theory and practice in nursing is steadily getting deeper and the main purpose of the educational systems is to cover this gap (8, 9).

According to the pyramid of George Miller, theoretical knowledge is "knowing" (by cognitive activities) and practical knowledge is "how to know" that are acquired by practical training (10); so, the main purpose of the clinical training is to help nursing students "to learn how to learn" (11). So, during last decades, great innovations have been created in theoretical and practical trainings and education was promoted from teacher-center to student-center training (12).

On the other hand, one of the most prominent variables that influences on students' learning is assessment methods. So that, students use deep or flat learning approaches during their academic course and according to assessment demand (13). Therefore, in proportion to transformation of educational system, evaluation system has been changed as well and it was shifted from just measuring knowledge to the methods of measuring abilities and competence (12). Nursing education has focused on developing the quality of trainings in learning sequences to improve educational courses by data collection (14).

In most cases, due to the subjective judgments, lack of content validity and teachers' biases, the traditional methods of clinical assessment have been carried out non-standardly and by chance that is not able to reflect knowledge and clinical competence of students, correctly (15). On the other hand, by believing in modern educational methods (student-center), those assessment methods that reflect and encourage such beliefs should be used (16).

Portfolio is one of the modern assessment methods that first have been applied in USA in 1980 (14). It is one of the functional tests with two main purpose of education and evaluation and it can create a general attitude toward student's learning experience by providing some snapshots from his clinical learning (17). Some of the main features of portfolio in contrast with traditional tests are mentioned as: comprehensive active participation in evaluation, self-evaluation of knowledge and skills, developing reflective and critical thinking, become a professional and responsible student in learning (18).

In fact, thinking about what has been learned is an important part of providing a portfolio (16); so that applying portfolio in education make the student aware of his learning development and educational requirements of students would be studied and identified by observing the recorded works (in portfolio) by teacher (15). Furthermore, portfolio can transplant theory and practice in the process of clinical education and enhance learning by providing some evidence of actual performance in "doing" (19). In this regard, the results of the Karimian et al. research (2009) in Iran indicated that portfolio leads to enhancement of the level of learning in internship among midwifery students in contrast with traditional assessment methods (20), also the results of Khoram-Abadi et al. research (2010) showed that portfolio has more effect on increasing academic achievement on chemistry in psychomotor and cognitive area of pre-university students (21).

On the other side, using portfolio in education had some restrictions such as time-consuming maintenance (with details) for teachers and students, difficult standardization and scoring and decision making about the boundary between acceptability and refusal (16), that may be considered as the reason of its less using in educational system of Iran (22). So that, the results of Koohpaye-Zade et al. research (2012) indicated a limited use of clinical assessments in Medical Sciences Universities of Tehran that is considered as the least level of using portfolio (6%) (23).

Since studies in modern educational and measuring methods of psychiatry wards were limited, and due to unpredictability of events in psychiatry wards  (as a part to suggest and evaluate psychiatric nursing issues specifically) and previous conceptions of nursing students about patients' aggressive behaviors who suffer from psychological disorders (and it can be influenced by medias or family cultures), and on the other hand, the replacement of routine clinical experiences with some assignments such as therapeutic communication with patients, clinical training in psychiatric wards have been associated with many problems which requires a continuous evaluation and review in educational plans (6). So, the researcher was to do a research with the purpose of determining the effect of applying portfolio on the level of learning in the internship environment of psychiatric wards in contrast with routine method of evaluation.

Methods

It was a semi-experimental research and 60 nursing students who were passing internship in Ibne-Sina psychiatry Hospital in 2014 have been studied. Sample size was considered based on Karimian and Heidari research (2009) as "the effect of portfolio evaluation on the level of learning and satisfaction of midwifery students" and by using formula of "comparing the mean and standard deviation of both communities" with calculation of mean and standard deviation of learning score (practical test) in portfolio group (17.69±0.99) and control group (16.87±0.85) (20), with 95% reliability and 80% test ability has been estimated 32 individuals for each group; but 35 individuals were considered for each group (totally 70 individuals) by considering 10% dropping of the sample. So that, at first the qualified students for being studied have been selected by available sampling and then students who wanted to participate in the study were divided non-randomly (according to the pre-determined internship list) in two groups of educational purposes (routine educational method) (35 individuals) and portfolio (35 individuals). At the end of the research, 5 individuals of educational purposes (2 people due to the absence, 2 people due to student work and 1 individual because of group displacement) and 5 individuals of portfolio (2 people due to the group displacement, 1 individual due to absence and 2 people because of student work) were omitted form study, so final sample size included 60 individuals (30 individuals per group). The inclusion criteria for students had two parts that included the lack of passing psychiatric nursing course (failed students of this course), lack of student work experience in psychiatric ward, lack of psychiatric disorders and medicine consumption in this context. Also, inclusion criteria for teachers include having M.A. in psychiatric nursing, having the experience of teaching at least 5 theoretical courses in psychiatric nursing, and the experience of supervising at least 5 internship courses of psychiatric nursing.    

 Students' demographic information form, the questionnaire of assessing students' learning based on educational purposes by student, the questionnaire of assessing students' learning based on educational purposes by teacher, and written- functional test have been used to obtain data.  

Demographic information form consisted of 10 questions which have been completed by students. The questionnaire of assessing the students' learning based on achieving the educational purposes (by students and teachers) was consisted of 25 items which has been set based on educational purposes (and was matched previously) and also it was scored according to Likert 4-degree scale (1=very low, 2=low, 3=well, 4= very well). According to the students and teachers, 25 is the lowest level and 100 is the highest level of achieving educational purposes. Content validity of this questionnaire has been confirmed by 7 professors of nursing and midwifery faculty and its reliability was calculated by internal consistency with Cronbach's alpha coefficient as α=0.85 by students and α=0.93 by teachers.

The written-functional test was consisted of 10 items about performance of learners in psychiatric ward that was provided as two groups of questions with the same items but different orders and its range of scores was considered between 0 and 100. The validity content of the test was confirmed by 5 professors of psychiatric nursing.

At first, the students of other cities who were passing internship in Ibne-Sina psychiatric hospital of Mashhad in 2014 (Sabzevar, Neyshabour, Kashmar, Torbate-Heidarie) were identified after receiving the students' internship plan (from education department of nursing and midwifery faculty of Mashhad). Then, after attracting teachers' cooperation (three qualified teachers who had the responsibility of students' training), students attempted to develop common educational purposes of internship equally and in proportion to the training course. Afterwards, 2 groups were selected from each faculty, the first groups were managed by using routine evaluation methods and the second groups were managed by portfolio and it limited the exchange of information between two groups. In portfolio group, a blank form with students' profile, internship place, teacher's name, and a table that included date, student activity description, student method of learning, rethinking of student and teacher's opinion at the first day of internship was given to the students with a brief explanation about the way to complete the form along with the matched educational purposes. So, the students must provide their clinical activities according to this form, and in order to be evaluated at the end of their internship, they must deliver the form to the teacher (table 1). In the group of usual evaluation, the matched purposes were given to students at the first day of internship and it was carried out by usual evaluation.

Table1. Portfolio implementation

Description of activity

Students should explain their daily activities. It can be considered as one or more items to achieve educational purposes.

How we learn

Students specify how they have learned the activity (1. through observation, 2. through working along with the assistant teacher, 3. dependently). Also, students record the number of carried out items.

Rethinking

Students state the percentage of their ability for doing activities. They specify whether it is needed to repeat the technique? If they have to repeat the technique, do they have the ability to do it dependently or not?

Teacher's view

The teacher would record his point of view about the way of students' activity. The number of portfolio forms is equal with the number of internship days. These forms would be kept in separate folders and each folder is consisted of name, family name and group name.  

 

Demographic information form was completed at the first day of internship. The questionnaire of assessing the level of learning based on educational purposes by teachers and students, and short answer written-functional test to evaluate students' academic achievement (by teacher) were completed on the final day of internship. In all phases of the research, all ethical issues of the research such as obtaining written agreement from Ethic Committee of University, receiving a written introduction from faculty of nursing and midwifery and presenting to the head of Ibne-Sina psychiatric hospital of Mashhad, obtaining a consciously written consent from participants of the research and ensure them that they can leave the research if they do not want to continue the study, have been considered and are approved by the research department of University of Medical Sciences.

Data analysis was carried out by SPSS software version 11.5, statistical tests of Kolmogorov - Smirnov and Shapiro Wilk (to examine normal distribution of data), dependent t-test (to study quantitative variables after intervention), K2, and exact Fisher (to compare the distribution of qualitative variables before intervention). In conducted tests, the level of confidence was 95% and the level of significance was calculated as α=0.05.

Results

According to table 2, there was no significant difference between the two groups of portfolio and educational purposes in terms of frequency distribution of students' demographic characteristics such as sex, age, marital status, city of education, total average, and the level of interest in psychiatric nursing (p>0.05) and both groups were equal. In portfolio group, the most frequency distribution of single students' demographic information with high rate of interest in psychiatric nursing was 70% (21 individuals), and the mean and standard deviation of their total average was 16.2±1.5 (table 1).

In educational purposes group, the most frequency distribution of single students' demographic information was 53.3% (16 individuals) and this percentage for those single students who had high interest in psychiatric nursing was 63.3% (19 individuals) and the mean and standard deviation of their total average was calculated as16.5±1.2 (table 2).

According to table 3, the mean score of learning among students of portfolio group by teachers (81.3±9.3) was more than mean score of learning among students of educational purposes group by students (62.4±4.8). The results of dependent t-test indicated that at the end of internship, there was significant difference between portfolio group (81.3±9.3) and educational purposes group (62.4±4.8) and in terms of mean score of nursing students' learning by teachers (p<0.001).

Also, the mean score of students' learning by students in portfolio group (72.7±13.6) was more than mean score of students' learning in educational purposes (59.1±16.3), and the results of dependent t-test showed that at the end of internship there was significant difference between portfolio group (72.7±13.6) and educational purposes group (59.1±16.3) in terms of mean score of nursing students' learning by students (self-evaluation) (p<0.001).  

Due to table 3, mean score of learning by teachers in portfolio group (72.7±13.6) was more than mean score of learning by students (72.7±13.6). Also, mean score of learning by teachers (61.9±4.8) was more than mean score of learning by students (59.1±16.3).

On the other hand, mean score of academic achievement (that was achieved from written-functional tests) in portfolio group (61.8±14.7) was more than mean score of academic achievement in educational purposes group (53.0±14.2). the results of dependent t-test indicated that there was significant statistical difference in terms of mean score of academic achievement between portfolio (61.8±14.7) and educational purposes at the end of internship course (p=0.02) (Table4).

Table2. The overall profile of studied nursing students separately in two groups of portfolio and educational purposes

Variable

Portfolio

Educational purposes

Test result

sex

Number (percentage)

Number (percentage)

Number (percentage)

Female

(46.7)14

(50.0)15

Chi-Square: p=0.80

Male

(53.3)16

(50.0)15

Marital status

 

 

 

Married

(30.0)9

(46.7)14

Chi-Square: P=0.18

Single

(70.0)21

(53.3)16

City of education

 

 

 

Torbate-Heidarie

(20.0)6

(20.0)6

Chi-Square:

P=0.99

Sabzevar

(33.3)10

(30.0)9

Kashmar

(26.7)8

(26.7)8

Neyshabour

(20.0)6

(23.3)7

The level of interest in psychiatric nursing 

 

 

 

Very high

(3.3)1

(10.0)3

Fisher's Exact Test:   P=0.76

High 

(70.0)21

(63.3)19

Low

(26.7)8

(26.7)8

 

Mean ± standard deviation

 

Age (year)

22.3 ±1.3

23.1±2.2

P=0.10  df=58  t=1.69

Total average

16.2±1.5

16.5±1.2

P=0.34   df=58  t=0.96

 

Table3. The comparisons of nursing students' mean score (by teachers and students) in two groups of portfolio and educational purposes

Opinions

Group

Number

Standard deviation ± mean

The result of dependent t-test

Teacher

Portfolio

30

81.3±9.3

T=8.52   df=36.1   p<0.001

Educational purposes

30

62.4±4.8

Student

Portfolio

30

72.7±13.6

T=3.51    df=58    p<0.001

Educational purposes

30

59.1±16.3

 

Table4. The comparisons of nursing students' mean score of academic achievement in two groups of portfolio and educational purposes    

Variable

Group

Number

Standard deviation ± mean

The result of dependent t-test

Academic achievement

Portfolio

30

61.8±14.7

P=0.02    df=58    t=2.35

Educational purposes

30

53.0±14.2

 

Discussion

The purpose of the present research was to determine the effect of applying portfolio on the level of learning among nursing students in internship course. According to the results, mean score of the level of learning among nursing students in internship course of portfolio group was significantly increased in contrast with educational purposes. Also, the results of the present research showed that the score of written-functional test in portfolio group was more than educational purposes group.

 The results of Karimian et al. research (2009) based on the fact that applying portfolio leads to increase the level of learning among 6th semester midwifery students from teachers' point of view (20) was consistent with the results of the present research; however, self-evaluation has not been studied by students. The results of Tasdemir et al. research (2009) in India was also based on positive effect of both cooperative learning method and portfolio on improving the level of learning in clinical education (24); and it was consistent with the results of the present research. The results of Khoram-Abadi et al. research (2010) was based on the effect of performance measurement method and portfolio on improving male pre-university academic achievement in psychomotor and cognitive areas of chemistry (21) and the results of Hekmat-Poor (2012) was based on nursing students' attitude on the fact that assessment accuracy by portfolio was more than routine evaluation methods (25) and they confirm the results of the present research.

In fact, psychiatric internship was a good opportunity to promote clinical education and integrate theory and practice in issues relevant to care and treatment of patients who suffer from psychological disorders, therapeutic communication and eliminating social stigma for these patients. Since most of medical sciences students have negative attitude toward psychological disorders before clinical training, usually such negative attitudes may be increased in psychiatric internship and communicating with patients. So that, students may feel stress and anxiety when interact with patients and they may imagine that working on these patients is dangerous and treatment prognosis of them is bad (26). So, in comparison with other clinical parts, clinical education in psychiatric internship was very important in terms of learning development in emotional and psychomotor areas. So that, studies showed that knowledge and clinical experience in psychiatric wards can create a more positive attitude toward mental health disorders and taking care of patients who suffer from psychological disorders. Also, it is valuable to make students ready to take care of spiritual and psycho-social needs of patients who suffer from physical illnesses (6). In the present research, due to the fact that most students are interested in psychiatric nursing; portfolio method can lead to learning development and academic achievement by activating students in clinical training. Because motivation (as a mental preparation) is considered as a prerequisite for learning by educational psychologists and it leads to learning development. So that, if the students are not interested in this course; they would not do their assignments seriously, but if they are interested in the course, they would seek for more information about the course in addition to do educational assignments (27).

In the present research, the researcher confirms more on specific situations and psychiatric wards and because of having the least physical and clinical works, the most professors, regardless of increasing the students' motivation and making positive attitude toward psychiatric wards and without scientific documentations, and more based on their imagination about the students' performance during their internship, they would start scoring and evaluating the student in clinical environment; that it can lead to reduce the initial motivation among students. According to the results of the present research, applying portfolio through self-directed learning development and forcing students to discover, search and self-evaluation leads to maintain motivation and stimulate musing learning strategies among students and provide a situation to promote motivation for continuous learning and finally it leads to academic achievement.                     

The results of Belcher et al. research (2014) was based on medical sciences students' attitude about electronic portfolio that was performed in all four academic years and due to the lack of feedback from teachers it had no effect on their learning (28) and it had no consistency with the results of the present research. It can be mentioned that the duration of its performance and absolute self-evaluation of students through portfolio may be considered as reasons of this lack of consistency. On the other side, it was a qualitative study and was carried out on a quantitative sample size (15 individuals).

In order to mention the limitations of the present research, we can refer to impossibility of random replacing of students into two groups of portfolio and routine evaluation. Due to the lack of sample size of this plan in summer and according to its implementation on nursing students of other universities in different cities, internship groups has been determined previously.

According to the results, applying portfolio may lead to learning development and academic achievement among nursing students in psychiatric internship by teachers and students. It is suggested for the future studies to implement the effect of applying portfolio randomly by teachers and students on the level of learning among nursing students.

Acknowledgement

This article was extracted from a research plan with code of 930564 that was approved by Research Department of University of Medical Sciences of Mashhad in 3.9.2014. Thereby, we appreciate University of Medical Sciences of Mashhad for validating the research, and also we are grateful to officials of nursing and midwifery faculty of Mashhad and teachers and students that take part in the research.   

1.       Rowe M, Frantz J, Bozalek V. The role of blended learning in the clinical education of healthcare students: a systematic review. University of the Western Cape Research Repository 2012; 34(4): 216-21.

2.       Hayman-White K, Happel B. Nursing students’ attitudes towards mental health nursing and consumers: Psychometric properties of a self-report scale. Arch Psychiatr Nurs 2005; 19(4): 184-93.

3.       Moagi M, Rensburg EJ, Maritz J. Student nurses’ experiences of the clinical psychiatric learning environment in an education institution. J Psychol Afr 2013; 3(2): 359-62.

4.       Sharif F, Masoumi S. A qualitative study of nursing student experiences of clinical practice. BMC Nurs 2005; 4(6): 1-7.

5.       Henderson A, Twentyman M, Heel A, Lloyd B. Students’ perception of the psycho-social clinical learning environment: An evaluation of placement models. Nurse Educ Today 2006; 26: 564-71.

6.       Karimollahi M. An investigation of nursing students’ experiences in an Iranian psychiatric unit. Journal of psychiatric and mental health nursing 2012; 19: 738-45.

7.       Mabuda BT, Potgieter E. Student nurses experiences during clinical practice in The Limpopo Province. Curationis 2008; 3(1): 19-27.

8.       Roshangar F, Lotfi M, Zamanzadeh V, Abdollahzadeh F, Davoodi A. The effect of using logbook on nursing students' learning. Iranian journal of medical education 2010; 10(1): 64-70. [In Persian].

9.       Ajani K, Moez S. Gap between knowledge and practice in nursing. Procedia Soc Behav Sci 2011; 15(1): 3927-31.

10.     Craddock E. Developing the facilitator role in the clinical area. Nurse Educ Today 1993; 13(3): 217-24.

11.     Baker C. Problem-based learning for nursing integrating lessons from other disciplines with nursing experiences. J Prof Nurs 2000; 16(5): 258-66.

12.     Elango S, Jutti R, Lee L. Portfolio as a learning tool: Students perspective. Ann Acad Med Singapore 2005; 34(8): 511-4.

13.     Segers M, Nijhuis J, Gijselaers W. Redesigning a learning and assessment environment: The influence on students perceptions of assessment demands and their learning strategies. Studies in educational evaluation 2006; 32(3): 223-42.

14.     Chu MS, Cho EH, Hwang YY. Development of the teaching portfolio in nursing education. Int J Adv Comput Technol 2015; 10(1): 80-90.

15.     Joshi MK, Gupta P, Singh T. Portfolio-based learning and assessment. Indian Pediatr 2015; 52(15): 231-5.

16.     Karimi Moonaghi H. Practical guide to medical sciences students' assessment (focused on new methods in clinical assessment). Mashhad: Mashhad University of Medical Sciences; 2011. [In Persian].

17.     Coffey A. The clinical learning portfolio: A practice development experience in grontological nursing. J Clin Nurs 2005; 14(8): 75-83.

18.     Kavaliauskiene G, Anusiene L, Kaminskiene L. The challenges for Esp Learners: Alternative assessment of performance and usefulness of class activities. Socialinis Darbas 2007; 6(1): 134-44.

19.     GrantEmail AJ, Vermunt JD, Kinnersley P, Houston H. Exploring students' perceptions on the use of significant event analysis, as part of a portfolio assessment process in general practice, as a tool for learning how to use reflection in learning. BMC Med Educ 2007; 7(5): 1-8.

20.     Kariman N, Heidari T. The effect of portfolio’s evaluation on learning and satisfaction of midwifery students. Arak Medical University Journal (AMUJ) 2010; 12(4): 81-8.

21.     Zhoramabadi I, Delavari A. The combined impact of alternative assessment methods (performance and portfolio) on the attitudes and academic achievement of high school students' experimental science degree in chemistry in Khoramabad city. Journal of educational psychology 2010; 5(15): 87-107. [In Persian].

22.     Saeed-Alzakerin M, Zhorami-Markani A, Zhorami-Markani MH. Portfolio based learning in nursing education. Journal - Research School of Nursing and Midwifery, Shahid Beheshti University of Medical Sciences and Health Services 2008; 19: 9-43. [In Persian].

23.     Kouhpayezadeh J, Dargahi H, Arabshahi KS. Clinical assessment methods in medical sciences universities of Tehran-clinical instructors’ viewpoint. Journal of Hormozgan Medical University 2011; 16(5): 395-402. [In Persian].

24.     Tasdemir M, Tasdemir A, Yildirim K. Influence of portfolio evaluation in cooperative learning on student success. J Theory Pract Educ 2009; 5(1): 53-66.

25.     Hekmatpou D. Effect of portfolio based evaluation on accuracy of clinical evaluation of nursing students during internship in Arak University of Medical Sciences, Iran. Strides development medical education 2013; 10(1): 60-9. [In Persian].

26.     Lyons Z, Janca A. Impact of a psychiatry clerkship on stigma, attitudes towards psychiatry, and psychiatry as a career choice. Lyons and Janca BMC Med Educ 2015; 15: 2-10.

27.        Saif A-A. Modern Educational Psychology, Psychology of Learning and Instruction. 7th ed. Tehran: Dowran; 2015. [In Persian].

28. Belcher R, Jones A, Smith L-J, Vincent T, Naidu SB, Montgomery J, et al. Qualitative study of the impact of an authentic electronic portfolio in undergraduate medical education. BMC Med Educ 2014; 14: 1-17.