Assessing the clinical skills of interns and stagers in the department of obstetrics and gynecology using Direct Observation of Procedural Skills (DOPS) method and satisfaction level of learners and examiners

Introduction

Today, emphasis on clinical and communication skills training and creating a proper attitude in medical students has been widely considered (1). Possessing skills in conducting clinical examinations are the essence of the medical profession. Acquiring the right skills in performing clinical procedures requires time, patience and practice in a suitable environment. Clinical skills are slowly gained and are depleted by the lack of practice (2, 3). One of the most important duties of clinical faculty members is the assessment of students' clinical skills. The development of clinical education methods makes the use of new and appropriate clinical evaluation methods a necessity. The use of new evaluation approaches that reinforce learning at the same time, are being more widely considered (4). There are two approaches to university education: deep and superficial. If depth-based assessment is designed, it can lead to in-depth learning. The superficial approach is based on minimum effort and acquiring the passing score and the desire to learn the habitual selected content, while in an in-depth approach, student tries to understand the materials, integrate the components into the whole, and to learn with one's own knowledge and apply it in real situations. (5) Although the majority of medical students' evaluation is clinical evaluation, but finding the appropriate method which is satisfactory to the students, has always been challenging. (6) Student evaluation in a clinical setting not only requires the proper functioning of psycho-motor skills, but also reflects the science, knowledge, judgment, and the ability of the person to react to changes (7). Therefore, to judge the student's achievement to a level of competence in practical skill, one must show the ability to perform essential clinical skills (8, 9). However, clinical assessment is measured at the level of performance. In the categorization of the Miller’s pyramid, one of the various methods for evaluating performance is direct observation of procedural skills (10). This method involves observing an intern during an action procedure that takes place on a real patient in an environmentally responsible manner. (11) Direct observation of the practice of medical interns in real patients is critical to assessing performance-based clinical skills. (12) DOPS is a new assessment tool at the workplace of the student, and studies have shown that it is a useful tool for evaluating practical skills (13) and has a positive impact on the learning approach (14).

The results of the study by Sharemi in 2011 revealed that using the clinical skills training center and the objective methods of student assessment skills, especially before entering the real clinical setting, is necessary for observing patients' rights and meeting the needs of students (15). Considering that one of the important issues of health and development in societies is the need for women to have reproductive health services and health care, it is necessary for physicians to have adequate experience for initial treatment. Some of these problems are related to pregnancy (16). The use of virtual realities in the training of obstetrics and gynecology and midwifery skills was confirmed in a 2002 study by Letteri, quoted by Matari 2007 (17). Due to the fact that in many rural and remote areas, access to specialists is not readily possible, there is the likelihood that general practitioners confront maternal delivery and unforeseen situations during professional activities. Therefore, education of all medical students in this field, which does not contradict the principle of adaptation of educational environments with religious standards, is necessary (18). Considering the fact that in Birjand University of Medical Sciences, the evaluation of the end of the course for students in internship and training courses has long been based on the same questions as the theoretical questions that were asked at the end of the students' section and so the students were not evaluated in terms of practical learning. Also, since a study has not been done in Birjand University of Medical Sciences so far, we have decided to conduct a research aimed at assessing the clinical skills of interns and female students using the direct observation of procedural skills (DOPS) method.

Methods: This is a cross-sectional descriptive analytical study. The study population consisted of 60 interns and stagers of obstetrics and gynecology department who were surveyed by census method. According to the opinions of faculty members of medical universities, among the main procedures of obstetrics and gynecology departments, 11 procedures were selected for testing materials and a two-dimensional evaluation checklist was prepared to collect information about the clinical skills assessment of students in the department of obstetrics and gynecology. The first part of the checklist contains evaluation indicators that include 11 options. These options include: 1- Knowledge of the indications, anatomy and technique of the procedure 2- Patient satisfaction 3. Preparation stage before the procedure 4. Anesthetizing and proper sedation 5. Observance of the conditions sterility 6. Technical capability (e. g. normal delivery) 7. Requesting help if necessary 8. Follow-up procedures 9. Communication skills 10. Professional behavior and attention to the patient 11. The general skill of the procedure. For each of the options, the score is assigned from zero to 10, the score zero is assigned to the unacceptable item, between 1 to 3: lower than expected, 4 to 6: moderate, 7 to 9 and 10 are considered as higher than expected. The second part of the checklist is related to the degree of students and examiner’s satisfaction which consisted of two questions.

In order to assess the students' clinical skills, innovative stations for evaluating interns and stagers in Birjand University of Medical Sciences were used, which included 10 clinical stations, the student was required to attend all stations and assessed by the physician at the station. During the questionnaire, the student's current problems are presented simultaneously. The ten stations include a history station, a gynecologic station, a Pap smear station, and a Pap smear report, a fetal heart hearing station and Leopold’s maneuvers, a urinary catheter station, an IV Line station, an IUD deployment station, a station for natural delivery and episiotomy repair. The student was observed by the examiner during the procedure in the actual work environment and the results were recorded according to a structured checklist and presented in an objective setting environment. To determine the validity of the checklist, content validity was used and the checklist was approved by faculty members of obstetrics and gynecology. An 85% reliability was calculated based on Cronbach's alpha. Data were analyzed using SPSS version 18 and Chi-square and independent t-test were used at a significance level of 0.05.

 RESULTS

This research was conducted on 60 medical interns and stagers in the department of obstetrics and gynecology of Vali-e-Asr Hospital. The mean age of students was 23.43 ± 5.41 years with a minimum of 18 and a maximum of 32. The DOPS method was used at an average of 20 times by the examiner and the frequency of the procedure for the test was 20 times.

The results of the Chi-square test on evaluation indices at all stations showed that the student's clinical skills were as expected in the majority of cases. The highest expected and above expected were related to communication skills and requesting assistance if needed; while most below expected were related to knowledge of the indicators, anatomy, and procedure’s techniques. (Table 1).
Based on independent t-test, the results showed that there was a significant difference between learners' satisfaction and examiners’ satisfaction (p <0.05).

Discussion:

The aim of this study was to evaluate the clinical skills of interns and stagers using the Direct Observation of Procedural Skills (DOPS) method and satisfaction level of learners and examiners. The results showed that there was a significant difference between the scores of 11 clinical skills of the department of obstetrics and gynecology in the students, and the scores of the skills in all of the evaluation indicators at all stations were in the expected range in the majority of cases. The highest expected and above expected cases were related to communication skills and requesting assistance if needed; while most below expected cases were related to knowledge of the indicators, anatomy, and procedure’s techniques. The most disadvantage mentioned by the student was the technical ability and the most advantage was requesting help if needed.  Overall, the results showed that the majority of students had acceptable grades from the evaluated stations and this method was effective in learning practical skills in the students. In the study of Shah Gheibi et al., Who assessed the interns' skills at Kurdistan University of Medical Sciences with 7 stations based on the DOPS method, DOPS method was effective in students’ learning. (8) The Bazrafkan’s study in Shiraz also showed that the majority of the students obtained a high score from the DOPS test in assessing the practical skills of the lab, which shows that the DOPS test is a useful tool in evaluating practical skills (9) In a study by Habibi et al., the two methods of Mini- CEX and DOPS are more effective in promoting the level of nursing students' skills than the traditional evaluation method (1). Therefore, the use of this method is useful in evaluating practical skills in the department of obstetrics and gynecology. Using this method, the examiner focuses on the important points that need to be evaluated in the skill required. This approach facilitates providing feedback to the experts, because instead of general comment, feedback is based on the actual and objective behaviors. Also, the use of this approach is not only an incentive for learning, but also because the test method and content are directly related to clinical performance, it can lead to a comprehensive learning effort.

Other results the study showed that there was a significant difference between the mean score of satisfaction in learners of 3.5 ± 2.92 and the mean score of satisfaction of examiners in the test was 8.85 ± 0.93. In the study of Akbari et al, regarding the DOPS test in evaluating practical skills in restorative dentistry, 71% of the students were satisfied with the positive effect of this test on learning, which is consistent with the present study. (10) Therefore, the satisfaction degree with the evaluation method was higher in examiners than the learners. Despite the fact that in this assessment the presence of the professors at all stations is necessary and, unlike the OSCE method, it cannot be evaluated by another person, and it is practically difficult and time-consuming for faculty members to perform this test, but due to justice in the evaluation of the students’ practical skills, professors had a good and acceptable satisfaction with this method. One of the limitations of this study was the limited number of moulages in some of the skills and equipment needed for training at the center of clinical skills, and therefore, training required to spend more time and to express some skills in theory on the moulages.

Conclusion

Based on the DOPS method, the students' clinical assessment is cost-effective and highly suitable for assessing the practical skills of the obstetrics and gynecology interns, and leads them to direct their education towards clinical skills based on the curriculum of the department, and it can even be a suggested method for changing the educational curriculum of the obstetrics and gynecology internship.

Acknowledgments

The authors need to thank the faculty members of the department of obstetrics and gynecology who helped this project.

Ethical Considerations

All tests had no name and surname.

Conflict of interest

There is no conflict of interest between the authors and the Future of Medical Education Journal.

Funds

Funds have been provided by researcher.