Development of a Framework for Assessing Professionalism in Medical Students

Introduction

In recent years, the matter of professionalism has received the attention of medical faculties in Iran similar to other countries (1). Expectations of competency-based curriculum should upgrade medical students’ professionalism. It is obvious that gaining experiences and establishing goals to teach and learn professionalism are not enough alone, professionalism should be assessed appropriately, constructively and relatively to the content (2). Despite the great emphasis on the importance of professionalism, its assessment is still one of the difficult issues in medical education (3). It should be considered that although professionalism assessment confronts various challenges, measuring the changes in students is impossible without assessment (4). van Mook et al (2009) declares that professionalism assessment is like a stimulus that makes the students learn about professionalism, and the professors become aware of expectations and students’ learning process (5). In references such as Hodges et al (2011) study, the recommendation is on professionalism assessment according to status and attention to culture (6). Therefore, this study aim was to design a framework to assess professionalism among medical students which include assessing factors, appropriate method, assessor, place and time and considering cultural factors, and practicality in medical school of Mashhad.

Methods

According to the purposes, Q methodology and qualitative method were used in this cross-sectional study. In order to determine the factors of professionalism assessment, Q methodology was used; and the qualitative method was used to determine method, assessor, time and place. Q methodology is a combination of qualitative and quantitative methods because on one side the participants are selected through non-probability sampling techniques purposefully; moreover, a few numbers of participants are chosen which makes it similar to qualitative method. On the other hand, the data are analyzed via quantitative method and factor analysis (7).

The sample comprised 31 persons including 26 basic science and clinical professors of the Medical Faculty of Mashhad and a sociologist who were expert and interested in medical professionalism or has thought medical professionalism or medical ethics. Moreover, there were two postgraduate students of medicine and two intern of undergraduate medicine.

In Q methodology the data were gathered via review of literature, focus group discussions and individual interviews. The data about factors of professionalism assessment were thoroughly gathered after four focus group discussions and nine interviews. The participant freely consented to participate in focus group discussions and individual interviews. After each focus group or interview meeting the transcript of the records were written. The transcriptions were organized and coded via MAXQDA software in order to make Q sample. The codes were written in statements and then were reviewed. The statements that presented different dimensions of the subject were considered and 36 statements were selected and provided the Q sample. Numbers were assigned to the statements and presented in a table that was designed for this purpose, via quasi normal Q diagram which had 36 cells (similar to the numbers of the statements) and guidelines and instructions were emailed to twenty professors of basic and clinical science in the form of an excel file.

The people, who did not want to participate in focus group discussions and interviews or were not interested in arranging the statements in Q diagram, were excluded from the study. It was explained in the instruction that they should categorize the statements in three groups and arrange them in Q diagram (graded from -5 to +5). The first category included 15 significant statements that could be assessed at the school. The second category included 15  incidental statements that could not be assessed at school. The third category were the statements that did not have the significance criteria or could not be assessed, or the attendees had no ideas about them (6 statements). The three categories on the Q diagram are presented in figure 1. The data of Q diagram were analyzed via SPSS. In the first step, the data were analyzed by exploratory factor analysis of Q, based on Principal Component Analysis (PCA) and Varimax rotation. At this stage, 7 factors with eigenvalue more than 1 were gained, and the correlation between each matrix had significant relation up to 95%. In the second step, the reliability coefficient and variance of statements’ score of the participants were calculated and the absolute difference was specified between the scores which revealed that all of the statements were distinguishing. Therefore, this method of calculation did not provide sufficient information about the factor loading. As a result, the PCA and Varimax rotation methods were used to distinguish the correlation between the factor arrays. 

Determining the most appropriate method, assessor, time and place to assess professionalism among students, a questionnaire was developed via searching texts and extracting the most common and important of them. The questionnaires were completed through semi-structured interviews with 12 faculty members of Mashhad Medical School. These participants were aware of objectives of the study and have also been involved in determining the assessing factors in professionalism. During the interviews, the attendees were asked to express their opinions according to the determined factors obtained from Q methodology and the applicability to be performed at the Mashhad Medical School.

Results

According to the results, six factors were determined to assess professionalism (table 1). The important point is that all of the analyzed Q statements’ sample have significant factor loading and none of the statements were eliminated. Each factor were named based on the content. There were 8 statements related to the first factor including individual capacity and attention to culture; 6 statements relevant to the second factor named as knowledge, medical skill and responsibility; 5 statements related to the third factor called managerial skill and preserving human dignity; 5 statements connected with the fourth factor called as organizational commitment and excellence; 6 statements linked to the fifth factor named as respecting others’ rights and duty; and 6 statements related to the sixth factor named as communication skills, altruism and respecting law. Each of them allocated 21%, 18%, 17%, 16%, 14%, 12% variance, respectively.

Based on the interviewees’ opinion Direct Observation, Objective Structured Clinical Examination, Educational Portfolios, Professionalism Mini Evaluation Exercise, Videotape Analysis, Situational Judgment Test, Self-Assessment, Peer Assessment, Patient Evaluation and 360 degree were considered as assessing methods; professor, assistant, peer, patients and their family and the students themselves were regarded as the assessors; inpatient and outpatient setting, emergency and the simulated environment were perceived as the place of assessment; the end of clinical course, the beginning and the end of the clinical course and permanently during the clinical course were considered as the time for assessment. Summing the ideas up and including the applicability capacity direct observation, professor, hospital wards were chosen as the most appropriate method, assessor and place, respectively. The best time of professionalism assessment was the beginning, the end and during the clinical course. In other words, the factors of professionalism should be assessed in students during their clinical education courses in terms of time. Finally, based on the results, the overall framework of professionalism assessment was designed for the students of Mashhad Medical School. (Figure2).

Table 1: Factor loadings of related statements and professionalism fundamental factors

Discussion

Different parts of the framework include assessing factor, method, assessor, place and time to assess professionalism among medical students of Medical University of Mashhad.

In the qualitative study of Wagner et al (2007), medical students, assistants, professors of faculty members and patients concurred with three factors including knowledge and technical skills, doctor-patient relationship and character virtues about professionalism (3). Since in this study the majority of the samples were faculty members, the underlying factors of faculty professionalism map in Wagner study and ours were compared and they were similar in some factors such as duty, confidentiality, communication and the appearance. Al-Eraky et al (2012) added autonomy of professional factor to the American Board of Internal Medicine framework which consisted of altruism, accountability, excellence, duty, honor and integrity, and respecting others. The new factor was added due to the fact that the doctors were more powerful in decision-making process than the patients in Eastern societies (8). The framework that is designed in our study includes the factors of ABIM; however since it was designed for medical students, the researchers did not consider autonomy of professional factor because this factor is not related to students’ duties. In Morgan research (2009) those characteristics of the professional graduates were assessed that have the possibility to be measured actually (9) that is similar to the objectives of our study. Moreover, factors such as demonstrating empathy and compassion with the patients, good communicational skills, awareness of social, cultural and economic issues, demonstrating professionalism, demonstrating leadership and team spirit, demonstrating clinical competence, demonstrating lifelong learning skills, and demonstrating innovation except for critical thinking are considered in this study. While conducting the review of the literature, a few studies, similar to the mentioned study, considered critical thinking as a factor of professionalism; in interviews and focus group discussions, critical thinking was not considered as a factor of professionalism. Therefore, according to the mentioned reasons and as during the process of the study it was attempted to prevent the extension of the numbers of statements in Q samples in order to save time and improve accuracy, therefore, the critical thinking was not considered in the framework as a factor of professionalism assessment.

Rees& Shepherd (2005) conducted a research about assessing acceptability of 360 degree, they presented that this assessment has both positive and negative effects on learning and behavior; the causes are environmental influences and the related factors of assessment and assessors (10). Cottrell and his colleagues’ study (2006) indicated that although peer assessment is a practical method, several assessors are required to estimate reliable professional behavior of students (11). Since, in our study the practicality of the assessment method was important for the researchers, 360 degree method was not confirmed, although it is a perfect method to assess professionalism. Only one of the attendees mentioned this method as the most appropriate and practical method. Moreover, the interviewees mentioned that using standardized patient is an appropriate method to assess professionalism among medical students but it is expensive and peer assessment or patient assessment misleads the assessment. 

Different researches have been conducted about the factors of professionalism assessment in the universities of other countries. The crucial point of the study was that a practical framework was designed relevant to our culture which is practical in the present conditions and existing resources in the society.  The basic application of the result is to provide professionalism assessment among medical students and make appropriate tool. If the weaknesses of the framework are fixed, it could be used at other universities. The major limitation of the study is that the patients’ ideas are not considered in the framework because it was difficult to access them.

Acknowledgement

This article is taken from the MSc medical education thesis (code: 910287) and is confirmed by research deputy of Mashhad University of Medical Sciences. We highly appreciate the deputy to sponsor this study. Moreover, we appreciate the faculty members, students and the persons who participated in this research.