Satisfaction and Practice Assessment of Medical Interns in Application of New Methods of Evaluation

Introduction

Evaluation is a process to determine the effectiveness, relevancy and effect of training activities in achievement of their objectives. A move from traditional paper-pencil form student evaluation to alternate and more creative types of assessment was made (1, 2). Evaluation may be classified as formative evaluation that involves the collection of useful and relevant tasks; Summative evaluation makes an overall assessment of its effectiveness in objective at the end of the course (3-4). One of the new method of evaluation that is task based evaluation and a described as systematic collection of learner work and related material that depicts students’ activities for describing their practice and achievement to educational aims is Portfolio (3-5). Portfolio is a set of writing matter and report and documentations of presentations, projects, seminars and research (3-5). Using portfolio can Promoting students self-evaluation, and their critical thinking, also in some situation it can apply for reporting od team working and group relationship during the time of education out of class and enable teachers and students to share the responsibility for setting learning goals in designing portfolio (4-6).

However portfolios usually reflect long term projects completed over the course of a semester, it is possible that portfolio used in short time periods like in student rotation in different educational wards or in a particular unit or just for particular topic of study (7, 9).

Developing portfolio needs to teacher and students clearly identify the educational aim, educational curriculum, portfolio contents and range of student work at first step (4-6). Develop evaluation procedures for grading the portfolio and follow of the portfolio contents at second step and organize a plan for holding portfolio conferences, in formal and informal meetings for review students’ work and their progress is next step (7-10). In the portfolio process student involvement is really important because in this process student understand the purpose of the portfolio, how it used to evaluate his work (3, 5). Although designing of portfolio requiring extra time for planning and conducting an assessment system and it needs to ensure that the assessment is reliable, Gathering all of the necessary data are difficult to manage (9,10).

Some studies introduced applying portfolio for evaluation as an effective method in evaluation student practice in health care system and in medical student (10, 11). This study assessed the medical interns’ practice by application of new methods of evaluation, their satisfaction and attitude in community medicine department of Mashhad, Iran.

Methods

In this cross-sectional study, portfolio based on educational medical interns’ curriculum purposes designed after gathering opinion of community medicine department faculties. After assessment of validity of designed portfolio, the pilot study was conducted.  Several corrections were done in proposed portfolio after pilot study. In each educational course of community medicine, head of medical interns’ education introduced aim of course, evaluation method and how student had to complete the portfolio to interns that enter to department.

Course of community medicine and attendance to health center in medical university of Iran is one month. Completed portfolio at the end of the each course was taken from interns students were in touch with their professors and especially with head of their education, during the course.  Interns encourage taking an active role in the development of their portfolios and they have possibility of what is placed in their portfolios in some tasks. Researcher tried to unify different evaluators scoring by making clear portfolio scoring items. Augmented portfolio as a set of interns’ activities that they could perform was applied for student evaluation.

Report of patients visit, logbook, educational poster or pamphlet designing, photos and power points of s interns’ presentation, results of interns’ self-evaluation and all that could express student learning ability were included in portfolio. Portfolio content was specified and after execution, interns’ evaluation practice scores were assessed and ranking in 4 groups (less than expected, moderate, good, and perfect). Intern satisfaction and attitudes by application of portfolio was assessed via self-rating questionnaire. Data enter in SPSS11.5 software and analyzed in descriptive and analytic manner.

Results

After summarizing results of pilot study and near to one year implementation of portfolio method, researcher conducted that despite of data gathering and student assignments in intern evaluation by portfolio method was difficult and time consuming but portfolio was a subjective useful type of assessment with significant satisfaction. Also every single activity and unnecessary details of the studying activity shouldn’t document in portfolio because of avoiding the problem in summing the scores and better management were difficulty.

Table 1 shows the interns’ satisfaction and their attitude from community medicine department evaluation process that applied portfolio method. Majority of medical interns reported positive attitudes and their satisfaction toward evaluation method applied in community medicine based on clarify of their tasks, reduce waste time and allowing some choices of what is placed in their portfolios.

Interns’ practice evaluation scores were categorized in four level: High (18-20), good (16-18), moderate (13-16) and less than expected (under 13).  Results of this study showed that scores of medical interns’ practice evaluation in community medicine department ranking of the scores of interns’ practice evaluation, almost were in moderate to good ranks (Table 2).

Table 1: Medical interns’ satisfaction and attitude from community medicine department evaluation process (Portfolio Method)

Table 2: Ranking of the scores of interns’ practice evaluation

Discussion

This study showed by application of performance based evaluating method, ranking of the scores of interns’ performance, was in moderate to good ranks, also majority of interns reported their satisfaction in this study from current evaluation method. In Some research expressed that students usually have low information about what they are evaluated, studies showed applying portfolio provides appropriate criteria for evaluating of their performance and proper background for student from evaluation content (4,6).

One study which compares  nursery students’  satisfaction from clinical evaluation by using portfolio with compare traditional paper-pencil method, students satisfaction in portfolio method was more than other common methods, but students got lower scores in portfolio than o traditional evaluation method, which researcher believes that this is because of student had low familiarity with portfolio method (4).

Another study about effects of training and evaluation on cognitive learning in Iranian nursery students with applying portfolio, showed that in portfolio method students got higher scores and student evaluation by portfolio improved cognitive learning (9). Educational studies about portfolio application in evaluation of students declared that portfolio directly assesses students learning outcomes and it is a comprehensive method for evaluating clinical students (12). A systematic review study on effectiveness of portfolios in postgraduate students explained that portfolio has good support in evidences; although if it well implemented, and supported by professions, that study also expressed that electronic versions are better than paper based portfolios (13).

This study is proposed applying portfolio as a practicable evaluation method based on students’ performance for evaluation of medical interns and it increasing students' responsibility and satisfaction for self-learning.

Acknowledgments

The authors thank the Medical Interns whose participation made this study possible.

Research committee approval and financial support: Mashhad university of Medical Sciences.

Conflict of Interest: Non