Future of Medical Education Journal

Future of Medical Education Journal

Direct Observation of Procedural Skills (DOPS) in Ophthalmology Education: a Systematic Review of Learner Outcomes, Perceptions, and Implementation Considerations

Document Type : Systematic review/Meta analysis

Authors
Masomeh Kalantarion 1
Mohammad Mehdi Sadoughi 2
Arash Ziaee 3, 4
Siamak Zarei-Ghanavati 5
Amin Habibi 1
Amirsadra Zangouei 1, 6
1 Department of Medical Education, School of Medical Education and Learning Technologies, Shahid Beheshti University of Medical Sciences, Tehran, Iran
2 Ophthalmic Research Center, Research Institute for Ophthalmology and Vision Science, Shahid Beheshti University of Medical Sciences, Tehran, Iran
3 Infectious Diseases Research Center, Birjand University of Medical Sciences, Birjand, Iran
4 Sinus and Surgical Endoscopic Research Center, Mashhad University of Medical Sciences, Mashhad, Iran
5 Eye Research Center, Mashhad University of Medical Sciences, Mashhad, Iran
6 Farabi Eye Research Center, Farabi Eye Hospital, Tehran University of Medical Sciences, Tehran, Iran
10.22038/fmej.2026.96298.1740
Abstract
Background: Direct observation of procedural skills (DOPS) is a workplace-based assessment format that combines direct observation with feedback. This systematic review examined ophthalmology-specific studies reporting learner outcomes, perceptions, and implementation characteristics associated with DOPS.
Method: We searched PubMed, Scopus, Web of Science Core Collection, and ERIC through April 2025, with backward and forward citation tracking. Eligible studies were controlled or uncontrolled quasi-experimental studies and methodological comparisons of DOPS-related activities in ophthalmology trainees. Two reviewers independently screened records, extracted data, and assessed methodological limitations. Outcomes included procedural performance, learner and assessor perceptions, and implementation characteristics. Heterogeneous findings were synthesized narratively.
Results: Six studies met the inclusion criteria: one controlled pre–post comparison with alternating allocation, three uncontrolled quasi-experimental studies, and two methodological comparisons. The controlled comparison reported higher post-test OSCE scores after a DOPS-based teaching package than after traditional instruction (ophthalmic torch examination: 6.80 vs. 5.35; fundoscopy: 6.50 vs. 4.70; both p = 0.001). The uncontrolled studies reported higher scores or more favorable perceptions after programs that included DOPS; one program also included Peyton’s four-step teaching method, supervised practice, and feedback. The methodological comparisons found score correspondence between Video Observation of Procedural Skills (VOPS) and DOPS in small single-assessor samples; these findings do not establish validity or interchangeability. Several studies described assessor preparation and standardized forms.
Conclusions: Current evidence suggests that DOPS is a promising formative assessment strategy in ophthalmology education; however, the certainty of evidence is limited by small samples, single-center designs, lack of controls, heterogeneous interventions, and insufficient psychometric evaluation.
Keywords
Direct Observation of Procedural Skills
Competency-based Medical Education
Workplace-based Assessment
Ophthalmology
Ophthalmology Education
Subjects
1.         Swing SR. The ACGME outcome project: retrospective and prospective. Med Teach. 2007;29(7):648–54.
2.         Singh T, Modi JN. Workplace based assessment: a step to promote competency based postgraduate training. Indian Pediatr. 2013;50(6):553–9.
3.         Alkalash SH, Farag NA. Effect of Workplace-Based Assessment Utilization as a Formative Assessment for Learning Among Family Medicine Postgraduates at the Faculty of Medicine, Menoufia University: A Prospective Study. Cureus. 2023;15(2):e35246.
4.         Miller A, Archer J. Impact of workplace based assessment on doctors' education and performance: a systematic review. BMJ. 2010;341:c5064.
5.         Naeem N. Validity, reliability, feasibility, acceptability and educational impact of direct observation of procedural skills (DOPS). J Coll Physicians Surg Pak. 2013;23(1):77–82.
6.         Ali L, Ali S, Orakzai N, Ali N. Effectiveness of Direct Observation of Procedural Skills (DOPS) in Postgraduate Training in Urology at Institute of Kidney Diseases, Peshawar. J Coll Physicians Surg Pak. 2019;29(6):516–9.
7.         Farajpour A, Amini M, Pishbin E, Mostafavian Z, Akbari Farmad S. Using Modified Direct Observation of Procedural Skills (DOPS) to assess undergraduate medical students. J Adv Med Educ Prof. 2018;6(3):130–6.
8.         Sethi S, Badyal DK. Clinical procedural skills assessment during internship in ophthalmology. J Adv Med Educ Prof. 2019;7(2):56–61.
9.         Chen W, Liao SC, Tsai CH, Huang CC, Lin CC, Tsai CH. Clinical skills in final-year medical students: the relationship between self-reported confidence and direct observation by faculty or residents. Ann Acad Med Singap. 2008;37(1):3–8.
10.       Wang Y, Chen G, Chen J, Jiang F. Application of Mini-CEX combined with DOPS in standardized training of community outpatient residents. BMC Med Educ. 2024;24(1):780.
11.       Gupta SK, Srivastava T. Assessment in Undergraduate Competency-Based Medical Education: A Systematic Review. Cureus.16(4):e58073.
12.       Profanter C, Perathoner A. DOPS (Direct Observation of Procedural Skills) in undergraduate skills-lab: Does it work? Analysis of skills-performance and curricular side effects. GMS Z Med Ausbild. 2015;32(4):Doc45.
13.       Bansal M. Introduction of Directly Observed Procedural Skills (DOPS) as a Part of Competency-Based Medical Education in Otorhinolaryngology. Indian J Otolaryngol Head Neck Surg. 2019;71(2):161–6.
14.       Ebrahimpour-Sadagheyani H, et al. Investigating the impact of the DOPS assessment on the learning and satisfaction of operating room students. Iran J Nurs Midwifery Res.30(1):110–5.
15.       Acgme. Accreditation Council for Graduate Medical Education: Milestones Guidebook 2024.
16.       Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.
17.       Barker TH, Habibi N, Aromataris E, Stone JC, Leonardi-Bee J, Sears K, et al. The revised JBI critical appraisal tool for the assessment of risk of bias for quasi-experimental studies. JBI evidence synthesis. 2024;22(3):378–88.
18.       Hassanpour N, Chen R, Baikpour M, Moghimi S. Video observation of procedural skills for assessment of trabeculectomy performed by residents. J Curr Ophthalmol. 2016;28(2):61–4.
19.       ul Haq I, Jamil B, Durrani M. PROCEDURAL SKILLS;: LEARNING OF PROCEDURAL SKILLS IN OPHTHALMOLOGY THROUGH DIRECTLY OBSERVED PROCEDURAL SKILLS (DOPS) OR TRADITIONAL METHODS; A RANDOMIZED CONTROLLED TRIAL. The Professional Medical Journal. 2018;25(08):1207–12.
20.       Moudgil T, Bansal Y, Arora R, Bandhu SD. Assessment of Clinical Competence of Interns using Work Place Based Assessment in Ophthalmology Rotational Posting. Journal of Clinical and Diagnostic Research. 2019;13(6):NC1–NC8.
21.       Padgul SB, Choppara B. Assessment of IOP measurement skills in ophthalmology interns by direct observation of procedural skills (DOPS). 2017.
22.       Ghiasian L, Hadavandkhani A, Abdolalizadeh P, Janani L, Es'haghi A. Comparison of video-based observation and direct observation for assessing the operative performance of residents undergoing phacoemulsification training. Indian J Ophthalmol. 2021;69(3):574–8.
23.       Saedon H, Salleh S, Balakrishnan A, Imray CHE, Saedon M. The role of feedback in improving the effectiveness of workplace based assessments: a systematic review. BMC Medical Education. 2012;12(1):25.
24.       Pelgrim EA, Kramer AW, Mokkink HG, Van der Vleuten CP. Quality of written narrative feedback and reflection in a modified mini-clinical evaluation exercise: an observational study. BMC Medical Education. 2012;12:1–6.
25.       Ramani S, Krackov SK. Twelve tips for giving feedback effectively in the clinical environment. Med Teach. 2012;34(10):787–91.
26.       Mietzsch S, Bergholz R, Boettcher J, Klippgen L, Wenskus J, Reinshagen K, et al. Classical but Not Rap Music Significantly Improves Transferability and Long-Term Acquisition of Laparoscopic Suturing Skills: A Randomized Controlled Trial. Eur J Pediatr Surg. 2020;30(6):541–7.
27.       Aggarwal R, Grantcharov T, Moorthy K, Milland T, Darzi A. Toward feasible, valid, and reliable video-based assessments of technical surgical skills in the operating room. Ann Surg. 2008;247(2):372–9.
28.       Scott DJ, Rege RV, Bergen PC, Guo WA, Laycock R, Tesfay ST, et al. Measuring operative performance after laparoscopic skills training: edited videotape versus direct observation. J Laparoendosc Adv Surg Tech A. 2000;10(4):183–90.
29.       Driscoll PJ, Paisley AM, Paterson-Brown S. Video assessment of basic surgical trainees' operative skills. Am J Surg. 2008;196(2):265–72.
30.       Liao KC, Pu SJ, Liu MS, Yang CW, Kuo HP. Development and implementation of a mini-Clinical Evaluation Exercise (mini-CEX) program to assess the clinical competencies of internal medicine residents: from faculty development to curriculum evaluation. BMC Med Educ. 2013;13:31.
31.       Watson MJ, Wong DM, Kluger R, Chuan A, Herrick MD, Ng I, et al. Psychometric evaluation of a direct observation of procedural skills assessment tool for ultrasound-guided regional anaesthesia. Anaesthesia. 2014;69(6):604–12.
32.       Kamat C, Todakar M, Patil M, Teli A. Changing trends in assessment: Effectiveness of Direct observation of procedural skills (DOPS) as an assessment tool in anesthesiology postgraduate students. J Anaesthesiol Clin Pharmacol. 2022;38(2):275–80.
33.       Cantillon P, Sargeant J. Giving feedback in clinical settings. Bmj. 2008;337:a1961.