Investigating the Evaluators’ Correspondence Rate (Learners, Colleagues, Authorities) Regarding the Quality of Promoted Professors’ Educational Activities in the Medical Science University

Introduction

Training has been one of the basic and most important roles of the faculty. Therefore, we must consider its assessment and improvement with the same seriousness as the faculty skills and research activities improvement. Educational assessment has considerable applications in improving the quality of the educational system: 1. weutilize educational assessment as an ongoing, regular process in any educational activities or on the whole in any activity which designed to convey, simulate, gain knowledge and rehabilitate in order to conduct and ensure the quality of educational activities. 2. Assessment means collecting data regarding the activities, program characteristics and outputs so as to judge, improve the effectiveness of the program or inform to make decisions about future programs. 3. We can make use of resultant information for important eventual purposes. These information provides feedback to curriculum planner and professors so that they can take advantage of these feedbacks to make more conscious decisions regarding the improvement of their teaching styles and be informed about the degree of their success. On the whole, we utilize the resultant outcomes of assessment to provide a better decision making for the following cases:

• Considering the recruitment status of the faculty
• Faculty members’ yearly promotion
• Faculty members’ scientific rank promotion
• Delegating administrative responsibilities of successful faculty members.
• Granting particular privileges to successful faculty members.

Teachers’ evaluation is the most complicated type of assessment. The reason is low-credit and inaccuracy of assessment methods and tools. (4 and 5) evaluation by learners is one of the prevalent and regular procedures in most countries including Iran. (5, 6 and 7) according to some authors, learners’ evaluation is one of the best types of assessment. However, this type of assessment has not always been approved easily. Heads of departments, professors, and learners has sometimes claimed that these assessments are valid and acceptable and others claimed that they lack validity and stability. (8 and 9) in order to solve this problem, it is suggested that we utilize a combination of assessment information for the final decisions. (10) One of the sources to evaluate faculty, educational activities is resultant outcomes of learners, colleagues, heads and authorities opinion- seeking in the process of investigating the faculty member promotion case. To serve this purpose, assessment will be performed in Medical Science University from the above-mentioned sources in different areas (education, professional and social ethics, discipline and class management).

Regularly, each faculty member is involved in a vast area of these activities. Hence, it is essential to consider a huge amount of teachers’ activities while assessing the quality of their educational performance. Assessment must correspond to the degree of these activities and the audience share in it. Educational activities of the faculty members can be classified as follows:

1. Training teaching: including presentation, teaching practical courses, training small groups, clinical and nonclinical, clinical training, outpatient, hospital, operation room and educational area, contributed to morning report, educational seminars and conferences, designing and reviewing lesson plans and providing educational tools and materials for teaching the related courses (11).
2.  Educational programming: including designing and implementing the new educational programs (such as unit planning, clinical rotations, etc.). Reforming the current educational programs (according to resultant outcomes of assessment and/or programs strength and weakness points)
3. Guidance and consultation: including guiding learners in various levels to increase their feedbacks in learning activities, better understanding of skills, writing a proposal, report and essay related to thesis or research plan regardless of faculty member legal status in above-mentioned cases.
4.  Educational management: including programming, actively continuance pursuing of organizational excellence, continuous assessment of performing proceedings, publication of the outcomes of activities, getting necessary resources and other management areas.

Learners’ assessment includes testing in different levels, cooperation in holding the exams, designing and implementing methods and tools for modern assessment or test in various levels (with respect to weaknesses and strength of existing methods).

The validity rate of evaluating sources in teacher assessment has not examined yet. However, we can evaluate correspondence and harmony rate of survey scores as the first step. He and meaningful correspondence of scores displays that the related group evaluate the same issue. I.e. teachers’ success rate in achieving educational goals. Otherwise, it is necessary to plan further studies to separately investigate effective factors on assessment results from each one of the sources.

This study aims at determining the corresponding rate of evaluators’ opinions regarding the quality of education by promoting faculty members in the academic year 90 to 94.

Methods

Every year a number of Iran’s faculty members in Medical science universities got promoted to a higher scientific rank with respect to their history of administrative and scientific services and research and educational activities. One of the faculty promotion criteria is assessing the quality of their educational activities which include four areas (training, professional and social ethics, and class management).

Approved suggested method in the ministry of health and medical education rulebook is to evaluate the educational activities of faculty members, though learners, colleagues, head of the department, hospital and faculty educational assistant as well as self-assessment. The academic member promotion case will be examined by a selected committee and the faculty qualifying conditions will be approved. One of the procedures involved in the faculty promotion case is assessing his educational performance quality which will be performed through a survey from different evaluating groups. To serve this purpose, some questionnaires were designed and their validity and stability were approved by ministry of health and medical education and consider different areas of education, from professional and social ethics to discipline and class management and was adjusted by Likert Scale (from very good to very bad). These questions were confidentially emailed to heads of departments, colleagues and universities and hospitals’ assistants by education development group. Since the survey includes learners’ ideas on assessing the faculty members’ educational performance, quality, it is performed annually by fundamental science students for clinical faculty members before the residency promotion exam, and for fundamental faculty members at the end of the semester. Hence, its data exists after seeking the head of department for permission. In education development groups, which will make use of faculty, previous year average scores for promotion. After rendering the survey to an education development group by heads of departments, colleagues, hospital and faculty educational assistants, and eliciting learners’ ideas, all data were typed in Excel software. The teachers’ scores were calculated in different areas of education quality based on the priorities that were considered by the related ministry for each one of the evaluators in each evaluation area. It should be noted that this plan has been performed by an evaluation expert on the education development group. In order to access the information, they were confidentially analyzed by education, development group after asking the head of department for permission. Finally, the collected data were entered the ship's 11.5 software of the intended cases, i.e. scores considered for faculty members in each area and were analyzed through correlation coefficient in order to investigate the corresponding rate of evaluators (heads of departments, colleagues, hospital and faculty educational assistants) in different areas of assessing the quality of education In this study, the number of faculty members were 16- who were promoted through this process for the year 1390 to 1394. Since there were a limited number of teachers, the whole society was considered as a case study by the census.

Results

With 160 promoted clinical and fundamental faculty members, 134 (84%) were clinical and 26 (16%) were fundamental. Overall, from the 160 individuals, 118 were men (74%) and 42 were women (26 %.) Table number 1 displays average and standard deviation of learners, colleagues, heads of departments and hospital presidents and assistants’ scores in each evaluation area.

 Table 1. Evaluators’ average and standard deviation of learners’ scores in each assessment area

Considering the resultant outcomes of promoting faculty evaluation in each evaluation area, students, heads of departments, colleagues and hospital assistants’ views , reflect that there is meaningful correspondence among the above mentioned groups except in education area p=0.07 and in other evaluation areas there is meaningful correspondence.

Table 2.  Evaluators’ correlation coefficient scores in each evaluation area.

 Discussion

Without doubt assessment is one of the most controversial issues of the evaluation process in any educational system due to its specific nature and performance 12. As improving the educational system in universities and implementing educational activities based on researches were both cases related directly to teacher evaluation, evaluation of competence, performance, knowledge and ability of faculty members is considered an important issue in institutes of higher education and all the authorities and directors of education seek access to a suitable method of assessment 13. There are different methods of assessing the faculty, including the assessment by authorities, hospital, educational assistants, heads of departments, colleagues, students as well as self- assessment 14 and 15.

According to the results of this study, there is no correspondence between the views of students, teachers, heads of departments and hospital, educational assistants in the area of ethics, discipline and class management. However, there is correspondence among the ideas of students, teachers, heads of departments and hospital, educational assistants in the area of ethics, discipline and class management. Perhaps, the evaluators’ status and position are one of the reasons of lack of correspondence between the area of education and other areas. Learners directly observe teachers’ performances, but heads of departments, colleagues and universities and hospital presidents do not possess such situation. The main share of assessment belongs to professors’ theoretical and practical teaching from the viewpoint of students. However, heads of departments, colleagues and universities and hospital presidents consider faculty interactions in the workplace.

Tazakori et al. Research in Ardebil Medical Science, University displays that there is no considerable distinction regarding assessment score average from the heads of departments and university and hospital presidents’ point of view. However, teacher assessment scores were higher among learners than other groups. In this study correspondence rate of university presidents and heads of departments regarding teacher assessment was higher than their correspondence rate with learners 16. In another study, in Shahid Beheshti Medical Science University, examining the correlation coefficient among students, residents, and heads of departments shows that there is only an average meaningful statistical correlation between the president and educational assistant opinions and in other cases no correlation was observed. 17 By explaining advantages and disadvantages of different methods of teachers’ assessment from the teachers’ point of view in a research in Kerman, it is suggested that we can make use of a suitable combination of several information sources considering the statuesque and their shortcomings in order to achieve more justly results. Furthermore, this study shows that teachers accept students as a source of assessment, but not as the only source and they suggest other sources such as university presidents and heads of departments. However, they believe that university presidents cannot give an accurate opinion about all the teachers and their opinions could be biased. On the other hand, all teachers favored transferring a part of assessment to university presidents.

In a research, safari reviews advantages and disadvantages of each teacher assessment methods and suggests planning a comprehensive system of assessment in which each method is considered a component of this system. He also notes that it is necessary to make use of all the data source, including colleagues, heads of departments, university presidents and authorities so as to avoid bias and prejudice about judging the faculty performance. Such comprehensive and multilateral assessment renders a better and clearer view of the individual real performance as well as decreasing error probability in assessment 19.

Based on such researches, Medical science Universities utilize multilateral methods of assessment approved by the related ministry in order to assess and promote the academic rank. The extent to which these multilateral methods correspond and cover each other so that it results in having an effective and useful faculty assessment is a question which was dealt with in this study. Findings show that although the correlation coefficient of the evaluators’ views is 61%, there is no meaningful correspondence among students, heads of departments, colleagues, hospital, educational assistants’ opinions. it can be due to several reasons such as lack of attention to their importance and necessity of this process and as a result lack of attention in evaluators’ filling the assessment forms. Another reason, which can be considered as of the most important reasons of opinion in correspondence, is lack of accurate recognition of the teachers’’ teaching styles by evaluators. Since the students are in direct contact with the teaching methods, lack of correspondence and correlation regarding professors’ teaching may be the reason of in correspondence regarding teaching methods which require more consideration and thought. Regarding the limits of this studying we can refer to a limited number of cases, and being limited to specific university results. Therefore, the generalization of findings must be performed carefully. We suggest performing similar studies with more samples in other units.

There are correspondence and correlation of students, colleagues, hospital presidents and assistants in the areas discipline, ethics and management and there is no meaningful correspondence and correlation of evaluators’ opinions in the area of education.

Ethical Confirmation

Professors’ assessment score paper contain no name and this research is done by the help of Medical University Education development office.

Acknowledgment

This research has been done as an approved research project of Mashhad University of Medical Sciences with the code number 930311 and has been sponsored by the Research Department. Hereby, we thank Support and cooperation of the Research Department and university development group. In addition, we appreciate sincere cooperation of participants in the research.