Satisfaction and Practice Assessment of Medical Interns in Application of New Methods of Evaluation

Document Type: Original Article

Authors

1 Department Of Community Medicine, Mashhad University of Medical Sciences, Faculty of Medicine, Mashhad, IRAN

2 University of Allameh Tabatabaee, Tehran, IRAN

Abstract

Background: Emphasis on revise and provide new methods of evaluation in medical colleges are widely increasing. Evaluation Methods based on clinical practice and students’ performance appraisement in health care system and in medical student is introduced as an effective method in some studies.
Methods: The aim of this study was to consider the usefulness of a new method to evaluate the medical interns’ practice, their satisfaction and their attitude toward evaluation process in community medicine course in Mashhad, Iran. Interns’ practice was evaluated by portfolio during their course, evaluation scores were categorized in four levels: High, good, moderate and less than expected. Intern satisfaction and attitudes was assessed via self-rating questionnaire.
Results: Ranking of the scores of interns’ practice appraisement, almost were in moderate to good ranks. Also the majority of interns reported their suitable satisfaction in this study.
Conclusion: This study is proposed applying evaluation methods based on students’ performance are as appropriate and practicable method for medical interns.

Keywords


Introduction

Evaluation is a process to determine the effectiveness, relevancy and effect of training activities in achievement of their objectives. A move from traditional paper-pencil form student evaluation to alternate and more creative types of assessment was made (1, 2). Evaluation may be classified as formative evaluation that involves the collection of useful and relevant tasks; Summative evaluation makes an overall assessment of its effectiveness in objective at the end of the course (3-4). One of the new method of evaluation that is task based evaluation and a described as systematic collection of learner work and related material that depicts students’ activities for describing their practice and achievement to educational aims is Portfolio (3-5). Portfolio is a set of writing matter and report and documentations of presentations, projects, seminars and research (3-5). Using portfolio can Promoting students self-evaluation, and their critical thinking, also in some situation it can apply for reporting od team working and group relationship during the time of education out of class and enable teachers and students to share the responsibility for setting learning goals in designing portfolio (4-6).

However portfolios usually reflect long term projects completed over the course of a semester, it is possible that portfolio used in short time periods like in student rotation in different educational wards or in a particular unit or just for particular topic of study (7, 9).

Developing portfolio needs to teacher and students clearly identify the educational aim, educational curriculum, portfolio contents and range of student work at first step (4-6). Develop evaluation procedures for grading the portfolio and follow of the portfolio contents at second step and organize a plan for holding portfolio conferences, in formal and informal meetings for review students’ work and their progress is next step (7-10). In the portfolio process student involvement is really important because in this process student understand the purpose of the portfolio, how it used to evaluate his work (3, 5). Although designing of portfolio requiring extra time for planning and conducting an assessment system and it needs to ensure that the assessment is reliable, Gathering all of the necessary data are difficult to manage (9,10).

Some studies introduced applying portfolio for evaluation as an effective method in evaluation student practice in health care system and in medical student (10, 11). This study assessed the medical interns’ practice by application of new methods of evaluation, their satisfaction and attitude in community medicine department of Mashhad, Iran.

Methods

In this cross-sectional study, portfolio based on educational medical interns’ curriculum purposes designed after gathering opinion of community medicine department faculties. After assessment of validity of designed portfolio, the pilot study was conducted.  Several corrections were done in proposed portfolio after pilot study. In each educational course of community medicine, head of medical interns’ education introduced aim of course, evaluation method and how student had to complete the portfolio to interns that enter to department.

Course of community medicine and attendance to health center in medical university of Iran is one month. Completed portfolio at the end of the each course was taken from interns students were in touch with their professors and especially with head of their education, during the course.  Interns encourage taking an active role in the development of their portfolios and they have possibility of what is placed in their portfolios in some tasks. Researcher tried to unify different evaluators scoring by making clear portfolio scoring items. Augmented portfolio as a set of interns’ activities that they could perform was applied for student evaluation.

Report of patients visit, logbook, educational poster or pamphlet designing, photos and power points of s interns’ presentation, results of interns’ self-evaluation and all that could express student learning ability were included in portfolio. Portfolio content was specified and after execution, interns’ evaluation practice scores were assessed and ranking in 4 groups (less than expected, moderate, good, and perfect). Intern satisfaction and attitudes by application of portfolio was assessed via self-rating questionnaire. Data enter in SPSS11.5 software and analyzed in descriptive and analytic manner.

Results

After summarizing results of pilot study and near to one year implementation of portfolio method, researcher conducted that despite of data gathering and student assignments in intern evaluation by portfolio method was difficult and time consuming but portfolio was a subjective useful type of assessment with significant satisfaction. Also every single activity and unnecessary details of the studying activity shouldn’t document in portfolio because of avoiding the problem in summing the scores and better management were difficulty.

Table 1 shows the interns’ satisfaction and their attitude from community medicine department evaluation process that applied portfolio method. Majority of medical interns reported positive attitudes and their satisfaction toward evaluation method applied in community medicine based on clarify of their tasks, reduce waste time and allowing some choices of what is placed in their portfolios.

Interns’ practice evaluation scores were categorized in four level: High (18-20), good (16-18), moderate (13-16) and less than expected (under 13).  Results of this study showed that scores of medical interns’ practice evaluation in community medicine department ranking of the scores of interns’ practice evaluation, almost were in moderate to good ranks (Table 2).

Table 1: Medical interns’ satisfaction and attitude from community medicine department evaluation process (Portfolio Method)

 

 

 

Level

Num

Percent

Interns’ Satisfaction

High

93

62

Moderate

39

26

Low

18

12

Interns’ Attitude

Agree

111

74

No comment

30

20

Oppose

9

6

Total

 

150

100

 

Table 2: Ranking of the scores of interns’ practice evaluation

 

 

Level

Num

Percent

Interns’ Practice Evaluation Score

High

19

13

Good

71

47

Moderate

57

38

Less than expected

3

2

Total

 

150

100

Discussion

This study showed by application of performance based evaluating method, ranking of the scores of interns’ performance, was in moderate to good ranks, also majority of interns reported their satisfaction in this study from current evaluation method. In Some research expressed that students usually have low information about what they are evaluated, studies showed applying portfolio provides appropriate criteria for evaluating of their performance and proper background for student from evaluation content (4,6).

One study which compares  nursery students’  satisfaction from clinical evaluation by using portfolio with compare traditional paper-pencil method, students satisfaction in portfolio method was more than other common methods, but students got lower scores in portfolio than o traditional evaluation method, which researcher believes that this is because of student had low familiarity with portfolio method (4).

Another study about effects of training and evaluation on cognitive learning in Iranian nursery students with applying portfolio, showed that in portfolio method students got higher scores and student evaluation by portfolio improved cognitive learning (9). Educational studies about portfolio application in evaluation of students declared that portfolio directly assesses students learning outcomes and it is a comprehensive method for evaluating clinical students (12). A systematic review study on effectiveness of portfolios in postgraduate students explained that portfolio has good support in evidences; although if it well implemented, and supported by professions, that study also expressed that electronic versions are better than paper based portfolios (13).

This study is proposed applying portfolio as a practicable evaluation method based on students’ performance for evaluation of medical interns and it increasing students' responsibility and satisfaction for self-learning.

Acknowledgments

The authors thank the Medical Interns whose participation made this study possible.


Research committee approval and financial support: Mashhad university of Medical Sciences.


Conflict of Interest: Non

  1. Kear M.E, Bear M. Using portfolio evaluation for program outcome assessment. Journal of Nursing Education 2007; 46(3): 109-14.
  2. Lettus M.K, Moessner P.H, Dooley L. The clinical portfolio as an assessment tool. Nursing administration quarterly J. 2001; 25(2): 74-9.
  3. Spence W, El-Ansari W. Portfolio assessment: practice teachers' early experience. Nurse Education Today J. 2004; 24(5): 388-401.
  4. Latifi M, Shaban M, Nikbakht A, Mehran A, Parsayekta Z. Comparison of the effect of clinical evaluation by two methods: Portfolio and popular, on satisfaction of nurse students. Iranian Journal of Nursing Research. 2011; 6 (21):15-28. [In Persian].
  5. Coffey A. The clinical learning portfolio: a practice development experience in gerontological nursing. International Journal of Older People Nursing in association with Journal of Clinical Nursing. 2005; 14:75-83.
  6. Schaffer M.A, Nelson P, Litt E. Using portfolio to evaluate achievement of population_ based public health nursing competencies in baccalaureate nursing students. Nursing Education Perspectives 2000; 26(2): 104-112.
  7. Brokhart S.M, Durkin D.T. Classroom assessment, student motivation, and achievement in high school social studies classes. Applied Measurement in Education J. 2003; 16(1).27-54.
  8. Liu E.Z, Zhuo Y, Yuan S. Assessing higher-order thinking using a networked portfolio system with peer assessment. International Journal of Instructional Media. 2004; 31(2), 139–149.
  9. Valizadeh S, Rahmani A, Davoodi A, Aminiaee N. Comparison of The Effect of Clinical Education and Assessment Using Portfolio and Blended Methods on Cognitive Learning of Nursing Students in Clinical Settings. Iranian Journal of Medical Education. 2011; 11 (3): 245-253. [In Persian].
  10. McCready T. Portfolios and the assessment of competence in nursing: a literature review. International Journal of Nursing Studies. 2007; 44(1): 143-51.
  11. Buckley S, Coleman J, Davison I, Khan K.S, Zamora J, Malick S, et al. The educational effects of portfolios on undergraduate student learning: a Best Evidence Medical Education (BEME) systematic review. BEME Guide, Med Teach J. 2009; 31(4): 282-98.
  12. McMullan M, Endacott R, Gray MA, Jassper M, Miller CML, Scholes J, et al. Portfolios and14assessment of competence: a review of the literature. Journal of Advanced Nursing. 2003; 41(3): 283-294.
  13. Tochel C, Haig A, Hesketh A, Cadzow A, Beggs K, Colthart I, et all. The effectiveness of portfolios for post-graduate assessment and education. BEME Guide, Med Teach J. 2009; 31(4):299-318. doi: 10.1080/01421590902883056.