Development of a Framework for Assessing Professionalism in Medical Students

Document Type: Original Article

Authors

1 Medical Education Department , Faculty of Medicine , Mashhad University of Medical Sciences, Mashhad, IRAN.

2 Medical Education Department , Faculty of Medicine , Mashhad University of Medical Sciences , Mashhad , IRAN.

3 Internal Diseases Department , Faculty of Medicine , Mashhad University of Medical Sciences , Mashhad , IRAN.

4 Center for Social Determinants of Health , Kurdistan University of Medical Sciences , Sanandaj , IRAN.

Abstract

Background: professionalism assessment is one of the most difficult issues in medical education. In related references, it is recommended that professionalism assessment should be according to the status and attention to culture of each society. The purpose of the study was to design an appropriate framework in order to assess professionalism among medical students of Mashhad University of Medical Sciences. 
Methods: the professionalism assessment framework was developed in accordance with experts’ perspective. Q methodology, that is a mixed research method to study people’s viewpoint and subjectivity about the components of professionalism, was used to determine the relevant factors. The data were collected through focus group discussion and interview and analyzed quantitatively using exploratory factor analysis. In order to determine the most appropriate method, time, place and person to carry out the assessment, qualitative methods and interviews were used.
Results: the factors influencing professionalism assessment included individual capacity and attention to culture - knowledge, medical skills and responsibility - managerial skills and preserving human dignity - organizational commitment and excellence - respecting other people’s rights and duty – respecting laws, communication skills and altruism. Direct observation, professors, hospital wards, duration of clinical courses were considered as the most appropriate method, assessor, place and time to assess professionalism among medical students.
Conclusion: a framework was designed in the study that included assessing factors, the best assessor and the most appropriate method, time and place in order to assess professionalism among medical students. On the condition that the shortcomings of the framework are corrected by further researches and localization, it could be used in other medical schools.

Keywords


Introduction

In recent years, the matter of professionalism has received the attention of medical faculties in Iran similar to other countries (1). Expectations of competency-based curriculum should upgrade medical students’ professionalism. It is obvious that gaining experiences and establishing goals to teach and learn professionalism are not enough alone, professionalism should be assessed appropriately, constructively and relatively to the content (2). Despite the great emphasis on the importance of professionalism, its assessment is still one of the difficult issues in medical education (3). It should be considered that although professionalism assessment confronts various challenges, measuring the changes in students is impossible without assessment (4). van Mook et al (2009) declares that professionalism assessment is like a stimulus that makes the students learn about professionalism, and the professors become aware of expectations and students’ learning process (5). In references such as Hodges et al (2011) study, the recommendation is on professionalism assessment according to status and attention to culture (6). Therefore, this study aim was to design a framework to assess professionalism among medical students which include assessing factors, appropriate method, assessor, place and time and considering cultural factors, and practicality in medical school of Mashhad.

 

Methods

According to the purposes, Q methodology and qualitative method were used in this cross-sectional study. In order to determine the factors of professionalism assessment, Q methodology was used; and the qualitative method was used to determine method, assessor, time and place. Q methodology is a combination of qualitative and quantitative methods because on one side the participants are selected through non-probability sampling techniques purposefully; moreover, a few numbers of participants are chosen which makes it similar to qualitative method. On the other hand, the data are analyzed via quantitative method and factor analysis (7).

The sample comprised 31 persons including 26 basic science and clinical professors of the Medical Faculty of Mashhad and a sociologist who were expert and interested in medical professionalism or has thought medical professionalism or medical ethics. Moreover, there were two postgraduate students of medicine and two intern of undergraduate medicine.

In Q methodology the data were gathered via review of literature, focus group discussions and individual interviews. The data about factors of professionalism assessment were thoroughly gathered after four focus group discussions and nine interviews. The participant freely consented to participate in focus group discussions and individual interviews. After each focus group or interview meeting the transcript of the records were written. The transcriptions were organized and coded via MAXQDA software in order to make Q sample. The codes were written in statements and then were reviewed. The statements that presented different dimensions of the subject were considered and 36 statements were selected and provided the Q sample. Numbers were assigned to the statements and presented in a table that was designed for this purpose, via quasi normal Q diagram which had 36 cells (similar to the numbers of the statements) and guidelines and instructions were emailed to twenty professors of basic and clinical science in the form of an excel file.

The people, who did not want to participate in focus group discussions and interviews or were not interested in arranging the statements in Q diagram, were excluded from the study. It was explained in the instruction that they should categorize the statements in three groups and arrange them in Q diagram (graded from -5 to +5). The first category included 15 significant statements that could be assessed at the school. The second category included 15  incidental statements that could not be assessed at school. The third category were the statements that did not have the significance criteria or could not be assessed, or the attendees had no ideas about them (6 statements). The three categories on the Q diagram are presented in figure 1. The data of Q diagram were analyzed via SPSS. In the first step, the data were analyzed by exploratory factor analysis of Q, based on Principal Component Analysis (PCA) and Varimax rotation. At this stage, 7 factors with eigenvalue more than 1 were gained, and the correlation between each matrix had significant relation up to 95%. In the second step, the reliability coefficient and variance of statements’ score of the participants were calculated and the absolute difference was specified between the scores which revealed that all of the statements were distinguishing. Therefore, this method of calculation did not provide sufficient information about the factor loading. As a result, the PCA and Varimax rotation methods were used to distinguish the correlation between the factor arrays. 

Determining the most appropriate method, assessor, time and place to assess professionalism among students, a questionnaire was developed via searching texts and extracting the most common and important of them. The questionnaires were completed through semi-structured interviews with 12 faculty members of Mashhad Medical School. These participants were aware of objectives of the study and have also been involved in determining the assessing factors in professionalism. During the interviews, the attendees were asked to express their opinions according to the determined factors obtained from Q methodology and the applicability to be performed at the Mashhad Medical School.

Results

According to the results, six factors were determined to assess professionalism (table 1). The important point is that all of the analyzed Q statements’ sample have significant factor loading and none of the statements were eliminated. Each factor were named based on the content. There were 8 statements related to the first factor including individual capacity and attention to culture; 6 statements relevant to the second factor named as knowledge, medical skill and responsibility; 5 statements related to the third factor called managerial skill and preserving human dignity; 5 statements connected with the fourth factor called as organizational commitment and excellence; 6 statements linked to the fifth factor named as respecting others’ rights and duty; and 6 statements related to the sixth factor named as communication skills, altruism and respecting law. Each of them allocated 21%, 18%, 17%, 16%, 14%, 12% variance, respectively.

Based on the interviewees’ opinion Direct Observation, Objective Structured Clinical Examination, Educational Portfolios, Professionalism Mini Evaluation Exercise, Videotape Analysis, Situational Judgment Test, Self-Assessment, Peer Assessment, Patient Evaluation and 360 degree were considered as assessing methods; professor, assistant, peer, patients and their family and the students themselves were regarded as the assessors; inpatient and outpatient setting, emergency and the simulated environment were perceived as the place of assessment; the end of clinical course, the beginning and the end of the clinical course and permanently during the clinical course were considered as the time for assessment. Summing the ideas up and including the applicability capacity direct observation, professor, hospital wards were chosen as the most appropriate method, assessor and place, respectively. The best time of professionalism assessment was the beginning, the end and during the clinical course. In other words, the factors of professionalism should be assessed in students during their clinical education courses in terms of time. Finally, based on the results, the overall framework of professionalism assessment was designed for the students of Mashhad Medical School. (Figure2).

Table 1: Factor loadings of related statements and professionalism fundamental factors

Statement

                    Factors* and Factor Loadings                   

1

2

3

4

5

6

Students should respect the cultural traits of patients, professor, employees, and peers including language, accent, and the accepted customs.

947.-

_

180.-

179.-

_

172.

Students should be just to gender, urban, rural and tribal inequality.

875.-

208.

122.

343.

167.-

177.

Students should not select medical profession only because of economic reasons and must have an appropriate attitude toward this profession.

865.

193.

172.-

250.

128.-

325.

Students should prevent from becoming too much friendly with the patients.

804.

498.

272.

147.

_

_

Students should behave properly to professor, employees, peers, patients and their family.

770.-

235.

443.

230.-

269.

173.

Students should be acquainted with health system and be aware of the advantages and disadvantages.

749.

_

166.

619.

158.

_

Students should be aware of their competency shortcomings.

731.

464.

319.-

138.-

_

352.-

Students should provide the opportunity for patients to express their problems and express empathy with them.

623.-

145.-

504.-

417.

127.-

383.-

Students should have community oriented attitude.

100.-

954.-

_

197.-

169.

_

Students should report the medical errors to the professor or resident.

230.

915.

229.

117.-

153.-

143.-

Students should have holistic approach toward the patient.

_

878.-

392.

244.

_

120.-

Students should have conscience in working and be perseverant to do their duties.

_

734.-

187.

362.-

498.-

197.

Students should prevent plagiarism

_

673.

670.-

_

266.

163.

Students should be able to do teamwork in educational process.

541.-

649.

_

239.-

474.

_

Students should think more than their own properties, they should respect the identity of patient, professor and peers.

_

119.-

962.

226.

_

_

Students should have effective teamwork to provide the best services to the patients.

_

142.

911.-

225.-

312.-

_

Students should be aware of their own rights and defend them.

188.-

252.

797.

118.

501.

_

Students should be able to keep secret.

131.

_

749.

615.-

157.-

131.

Students should be able to decide appropriately in difficult situations.

533.

257.-

572.

177.-

304.

445.-

Students should be committed to the objectives of the university.

263.

174.-

134.

931.

105.-

_

Students should have sufficient medical knowledge.

171.

322.-

_

863.

165.

305.

Students should have creativity while facing with new situations.

117.

548.-

_

818.-

_

113.-

Students should use the opportunities to learn continuously both for their own and others.

232.-

271.-

391.-

790.-

272.

148.-

Students should have clinical and manual skills.

581.

_

314.-

595.-

455.

_

Students should respect the patients’ rights and diagnose and select the best treatment to provide healthiness for the patients.

214.

_

306.-

_

922.-

_

Students should be patient toward various different behaviors and ideas.

_

210.

365.-

167.

875.-

145.

Students should reply to patients’ questions after diagnosis and provide the necessary educations.

_

258.

_

159.-

805.-

507.-

Students should be honest in their behavior and speech.

116.-

305.-

110.-

515.

775.-

126.

Students should be aware of their duties and perform them properly.

550.

307.

275.-

_

677.

261.

Students should be up to date about new treatments and drugs.

343.

422.-

537.-

151.

596.

193.-

Students should be punctual and be present in educational sessions regularly.

_

163.-

_

237.

129.

947.

Students should wear professionally and be in harmony with educational norms.

276.-

276.

158.-

_

138.-

893.

Students should respect the values of the society.

110.-

198.

473.

144.-

_

838.

Students should have appropriate verbal and nonverbal skills in communication with patients.

329.-

321.

525.

_

295.-

653.-

Students should use the best evidences for treatment.

320.-

162.

136.-

521.

513.

564.-

Students should have self-confidence and can decide about disease by themselves and diagnose and treat properly.

267.

464.

_

453.-

444.

551.-

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Discussion

Different parts of the framework include assessing factor, method, assessor, place and time to assess professionalism among medical students of Medical University of Mashhad.

In the qualitative study of Wagner et al (2007), medical students, assistants, professors of faculty members and patients concurred with three factors including knowledge and technical skills, doctor-patient relationship and character virtues about professionalism (3). Since in this study the majority of the samples were faculty members, the underlying factors of faculty professionalism map in Wagner study and ours were compared and they were similar in some factors such as duty, confidentiality, communication and the appearance. Al-Eraky et al (2012) added autonomy of professional factor to the American Board of Internal Medicine framework which consisted of altruism, accountability, excellence, duty, honor and integrity, and respecting others. The new factor was added due to the fact that the doctors were more powerful in decision-making process than the patients in Eastern societies (8). The framework that is designed in our study includes the factors of ABIM; however since it was designed for medical students, the researchers did not consider autonomy of professional factor because this factor is not related to students’ duties. In Morgan research (2009) those characteristics of the professional graduates were assessed that have the possibility to be measured actually (9) that is similar to the objectives of our study. Moreover, factors such as demonstrating empathy and compassion with the patients, good communicational skills, awareness of social, cultural and economic issues, demonstrating professionalism, demonstrating leadership and team spirit, demonstrating clinical competence, demonstrating lifelong learning skills, and demonstrating innovation except for critical thinking are considered in this study. While conducting the review of the literature, a few studies, similar to the mentioned study, considered critical thinking as a factor of professionalism; in interviews and focus group discussions, critical thinking was not considered as a factor of professionalism. Therefore, according to the mentioned reasons and as during the process of the study it was attempted to prevent the extension of the numbers of statements in Q samples in order to save time and improve accuracy, therefore, the critical thinking was not considered in the framework as a factor of professionalism assessment.

Rees& Shepherd (2005) conducted a research about assessing acceptability of 360 degree, they presented that this assessment has both positive and negative effects on learning and behavior; the causes are environmental influences and the related factors of assessment and assessors (10). Cottrell and his colleagues’ study (2006) indicated that although peer assessment is a practical method, several assessors are required to estimate reliable professional behavior of students (11). Since, in our study the practicality of the assessment method was important for the researchers, 360 degree method was not confirmed, although it is a perfect method to assess professionalism. Only one of the attendees mentioned this method as the most appropriate and practical method. Moreover, the interviewees mentioned that using standardized patient is an appropriate method to assess professionalism among medical students but it is expensive and peer assessment or patient assessment misleads the assessment. 

Different researches have been conducted about the factors of professionalism assessment in the universities of other countries. The crucial point of the study was that a practical framework was designed relevant to our culture which is practical in the present conditions and existing resources in the society.  The basic application of the result is to provide professionalism assessment among medical students and make appropriate tool. If the weaknesses of the framework are fixed, it could be used at other universities. The major limitation of the study is that the patients’ ideas are not considered in the framework because it was difficult to access them.

 

Acknowledgement

This article is taken from the MSc medical education thesis (code: 910287) and is confirmed by research deputy of Mashhad University of Medical Sciences. We highly appreciate the deputy to sponsor this study. Moreover, we appreciate the faculty members, students and the persons who participated in this research.


* First factor: individual capacity and attention to culture; second factor: knowledge, medical skill and responsibility; third factor: managerial skill and preserving human dignity; fourth factor: organizational commitment and excellence; fifth factor: respecting others’ rights and duty; six factor: communication skills, altruism and respecting law.

1.         Aramesh K, Mohebbi M, Jessri M, Sanagou M. Measuring professionalism in residency training programs in iran. Med Teach 2009; 31(8): e356-e61.

2.         van Mook WN, van Luijk SJ, O'Sullivan H, Wass V, Schuwirth LW, van der Vleuten CP. General considerations regarding assessment of professional behaviour. Eur J Intern Med 2009; 20(4): e90-e95.

3.         Wagner P, Hendrich J, Moseley G, Hudson V. Defining medical professionalism: a qualitative study. Med Educ 2007; 41(3): 288-94.

4.         Lynch DC, Surdyk PM, Eiser AR. Assessing professionalism: a review of the literature. Med Teach 2004; 26(4): 366-73.

5.         van Mook WN, Gorter SL, O'Sullivan H, Wass V, Schuwirth LW, van der Vleuten CP. Approaches to professional behaviour assessment: Tools in the professionalism toolbox. Eur J Intern Med 2009; 20(8): e153-e7.

6.         Hodges BD, Ginsburg S, Cruess R, Cruess S, Delport R, Hafferty F, et al. Assessment of Professionalism: Recommendations from the Ottawa 2010 Conference. Med Teach 2011; 33(5): 354-63.

7.         Khoushgoyanfard A. Q Methodology. Tehran: Research Center of Islamic Republic of Iran Broadcasting (IRIB); 2007. [In Persian]

8.         Al-Eraky MM, CHandratilake M. How medical professionalism is conceptualised in Arabian context: A validation study. Med Teach 2012; 34: s90-s95.

9.         Morgan JH. Designing an assessment tool for professional attributes of medical graduates from a new medical school in Nepal. South East Asian Journal of Medical Education 2009; 3(1): 2-7.

10.       Rees C, SHepherd M. The acceptibility of 360-degree judgements as a method of assessing undergraduate medical students'personal and professional  behaviours. Med Educ 2005; 39(1):49-57.

11.       Cottrell S, Diaz S, Cather A, Shumway J. Assessing medical student professionalism: An analysis of a peer assessment. Med Educ Online 2006; 11: 1-8.