The Simulation Module for Assessment of Resident’s Targeted Event Response (SMARTER-Shift): a novel teaching method in Emergency Medicine education

Document Type: Original Article

Authors

1 Department of Emergency, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran

2 Department of Emergency Medicine, Faculty of Medicine, Azad University of Medical Sciences, Tehran, Iran

3 PhD student, School of Nursing & Midwifery, Shahid Beheshti University of Medical Sciences, Tehran, Iran

Abstract

Background: Attention to several clinical issues simultaneously and timeliness of proper treatment is of prime importance in Emergency Medicine. Simulation Based Training is a growing trend for clinical resident training which has been proposed for enhancing of residents multi-tasking abilities. The Simulation Module for Assessment of Resident’s Targeted Event Response is a methodology which derives specific scenario scripts from core competencies intended for residents. In this study we take that methodology one step further and use it to design multiple patient scripts hoping to enhance multiple knowledge and skills including multitasking.
Methods: Using qualitative study of literature we derive multiple patient scripts incorporating simulation technologies which aim to obtain multiple clear educational objectives including the skills required for multitasking. Furthermore we test this teaching method on first year emergency medicine residents. We analyze the effects using paired student T-test of pre/posttests. Also the students’ satisfaction and impression about the method is measured using a questionnaire. Results: The step by step development of a multiple patient scrip is described and a sample script is presented. Cumulative scores of skill and knowledge for resident improved significantly (P<0.001) before and after the course. Almost all students (96%) preferred SMARTER-shift to a clinical shift for teaching purposes while only 52% preferred it for practicing the skills they had already obtained.
Conclusion: The SMARTER-shift is an effective way for education of multitasking skills to residents of emergency medicine and it is strongly preferred by residents.

Keywords

Main Subjects


Introduction:

One of the most fundamental approaches in medical education is rooted in Halsted’s statement “See One, Do One, Teach One”. Although this emphasizes the clinical nature of medical education, it also raises concern about the quality of patient care in the educational setting (1). In recent years, this concern has grown. Simulation-based training (SBT) aims to reach a compromise between these two seemingly opposite goals (2-5). High-fidelity mannequin simulation provides medical students the opportunity for observation, evaluation, and feedback that may not occur in the clinical setting.

The Simulation Module for Assessment of Resident’s Targeted Event Response "SMARTER" methodology which was first described in 2008 is an Event-based approach to training which has been customized for using in healthcare.

Rosen showed the use of simulation in graduate medical education affords unique opportunities for increasing the quality of a resident's educational experiences (6).

Simulations are not only capable of reproducing the dynamic nature of real-world clinical experiences, but are also good news for those who blame education for poor and improper patient care (7). Some have even proposed a pedagogical framework which integrates simulation training into routine training of procedural skills for medical students (8). One approach in incorporating SBT into medicinal education is the SMARETR (simulation module for assessment of resident’s targeted event response) methodology (9), which uses an eight-step algorithm starting from Accreditation council of graduate medicinal education (ACGME) core competencies and ending in creation of a scenario script (10-12).

The SMARETR methodology provides a tool to guide learning experience and captures nearly all dynamic interactions in clinical interventions. In this process an ACGME core competency is selected (step one), based on which educational objectives are clarified (step two). Then a clinical context is chosen (step three) in which the Knowledge-Skill-Attitude (KSA) profile of good performance is defined (step four). Based on these, critical events (step five) and expected responses (step six) are designated and a measurement tool for the response is devised (step seven). In the last step all of the above are incorporated into a clinical scenario (step eight) (6). As one can see in step seven, the method can also be used as the base for education evaluation which is free of the limitations imposed by assessment at the bedside (6,13) .

Most of the available simulation technologies and most of the teaching processes that use simulation are designed to enhance procedural skills or at most, the management of a single patient, and thus lack the concept of multi-component problem solving (14-16) . In emergency medicine (EM), on the other hand, the very feature of multiple task management is deemed valuable. Using prioritizing techniques and reordering what needs to be done, an EM specialist must be able to face repetitious intrusions or different interruptions which might happen simultaneously in order to manage different tasks at the same time (17,18). When first introduced to EM, even the advocates of SMARTER declared that performance in EM is complex and covering its entirety within one scenario is difficult, impractical, and even questioned its necessity (6). Consequently, when SMARTER methodology is used, each scenario mainly focuses on a specific competency. Instead, we believe while each specific competency is important, the ability to perform in a competent manner in different dimensions simultaneously and under pressure is what separates a capable EM specialist from others. The fact that EM education has turned a blind eye to this issue means residents face additional stress and frustrations during a real shift clinical experience which drastically reduces their performance quality (19). Moreover, residents need to try and observe expert clinicians, handling multiple cases, and learn both the accuracy and timeliness of ordered medications through direct observation within the clinical setting (20). The problem with this approach is that each clinician may have his/her own attitude towards a particular treatment and this may in turn lead to a variety of treatments and a lack of homogeneity and standardized approaches.

However, it has been proposed that SBT has the potential to compensate for this drawback and help build a well-defined set of Knowledge Skill and Attitude (KSA) in the high-acuity multi-patient environment (21). In this study we describe and test the “SMARTER-Shift” as a teaching method in which several clinical scenarios based on the SMARTER approach are integrated to create a simulated setting in which several competencies are demonstrated alongside the ability to multitask.

In this study we take that methodology one step further and use it to design multiple patient scripts hoping to enhance multiple knowledge and skills including multitasking.

Methods:

This was a mixed study composed of two distinct parts. First the development of multi-patient SMARTER scenarios rooted in ACGME core competencies to cover major realms of knowledge, skill and attitude within the selected field, alongside being able to demonstrate the ability to manage multiple patients simultaneously. Second a quantitative before and after study aiming to assess the ability of this method to achieve its teaching objectives.

Scenario development

The eight steps of SMARTER (previously discussed) were used to create five patient scenarios for each SMARTER-Shift (6).A ninth step was added in which each patient’s introduction into the ED and also all the critical events of different patients were integrated into a single timeline. This was done so that resident would benefit more than one proper step to take and prioritize between these tasks. Also after finishing one task, the resident may be expected to resume a task which he/she had left unfinished; therefore the ability to task-switch was also demonstrated. A third possible situation was an unimportant interruption which needed to be deprioritized by the resident. Expected responses (step six) and measurement tools (step seven) pertinent to these new skills were also included in the assessment sheet. As the leader resident in each simulated shift was responsible for coordination of other team members, another demonstrated and rated skill was the ability to manage a treatment team.

Use of simulation technologies

Alongside the use of simulated case scenarios, simulation technologies where incorporated into cases to better teaching and assessment of resident skills. For this purpose five mannequins were available in each SMARTER-Shift. Based on the context of the scenarios these included a standard intubation mannequin, a rhythm and CPR simulation mannequin, a suturing mannequin, a pediatric intubation mannequin, a central line placement mannequin, a chest tube placement mannequin, basic airway management kit, a peripheral intravenous placement kit, a ventilator, digital radiologic results, EKG strips, and/or an ultrasonography devise. The equipment was initially covered by sheets and each mannequin or apparatus was revealed on the specified time on the simulated shift timeline and residents were expected to assess, perform procedures, or interpret data accordingly. 

Data gathering and assessment of KSAs

The residents’ knowledge and skills were assessed using pretest and posttests. Our pretest composed of a written and an OSCE exam both of which had been used for the evaluation of residents in our department on previous occasions in the related topic. Then the shift residents were asked to complete an anonymous questionnaire regarding their opinions on the use of SMARTER-Shifts as a teaching method and its ability to prepare them for a clinical shift.

In order to prevent the pretest acting as an educational vector, the posttest was delayed to 4 weeks after holding the simulated shift. The posttest again composed of both written and OSCE examinations on the relevant topic with questions covering the same domains but not the exact items covered on the pretest. In this study we aimed to evaluate the effects of the SMARTER-Shift on resident education, not as an assessment tool. But still assessment tools were devised as dichotomous checklists to help students realize their strengths and weaknesses during debriefing.

Statistical Analysis

The statistical software SPSS V21 was used for data analysis of the quantitative values. Student paired T-tests and standard deviations were used to analyze the pre/posttest scores.

Results:

The SMARTER-Shift

A sample SMARTER-Shift is described in Table-1. The critical events have been designated specific times on the multiple scenario timeline. Each shift was conducted with the participation of five residents all in their first year of postgraduate studies. One resident was determined as the lead physician. The instructor acted as the confederate nurse while the other four students could also be assigned tasks by the lead physician as nurses or staff. The data pertaining to each patient was announced by the instructor on specified times.

Table 1.  Sample SMARTER-Shift in the trauma ward

Core competency

Learning Objectives

Clinical context

KSAs

Critical events

Time

Targeted responses

Patient care

Develop and carry out patient treatment plans

 

Perform competently all medical and invasive procedures considered essential for the area of practice

Initially stable tension pneumothorax

Proper performance of the primary survey

Quick performance of a secondary survey

Recognizing the importance of frequent evaluations in a multiple trauma victim

Ordering the correct  splint

Severe leg pain

0 min

Perform primary survey

Order CXR and proper X-rays

Splint leg

Medical knowledge

Know and apply the basic and clinically supportive sciences that are appropriate to their discipline

Diagnosis of a pneumothorax on ultrasonography

Diagnosis of a tension pneumothorax based on clinical findings Recognizing the most critical patient

Switching task to this patient

Develops Severe dyspnea

15 min

Oxygen therapy

Order monitoring and frequent vitals

Consider e-FAST

Consider  Intubation

Develops hypotension

20 min

Needle thoracostomy

Chest tube placement

Medical knowledge

 

Know and apply the basic and clinically supportive sciences that are appropriate to their discipline

Pediatric head trauma and airway management

Ability to perform trauma triage

 

Decreased consciousness

10 min

Proper triage of mother and child Perform Primary Survey

Calculate GCS and decide to intubate

Step by step execution of RSI

Oral secretions

 

Attempts to open airway and suction secretions

Chose RSI for intubation

Name the drugs needed and sequence

Intubate with correct technique

Recognition of rising ICP and emergency treatment

Proper patient presentation on consult

Anisochoria develops

25 min

Hyperventilate

Emergent Neurosurgical consult

Consider manitol infusion

 

Professionalism

Demonstrate respect, compassion, and integrity; a responsiveness to the needs of patients and society that supersedes self-interest; accountability to patients, society, and the profession; and

a commitment to excellence and ongoing professional development

Familiarity with proper technique for bad news delivery

Family arrives

42 min

Proper presentation of bad news

Interpersonal and communication skills

Develop the ability to lead team members in a multiple patient setting

Intra-abdominal bleeding

Leading team members to care for the patient while attending to the child

 

Severe abdominal pain

 

10 min

Two large bore IV lines

Order frequent vitals

Perform e-FAST (fluid+)

Medical knowledge

Know and apply the basic and clinically supportive sciences that are appropriate to their discipline

Diagnosis of intra-abdominal free fluid

Recognize indications for emergent laparotomy of trauma patient

Correct fluid resuscitation

Familiarity with treatment of hemorrhagic shock

Tachycardia, diaphoresis

hypotension

12 min

Rapid fluid  therapy

Anticipate severe bleeding by demanding cross-match

Emergent Surgical consult

Consider permissive hypotension

Failure to respond to fluid bolus

32 min

Initiate O-negative blood

Consider FFP in 1:1 ratio

Professionalism

Demonstrate respect, compassion, and integrity; a responsiveness to the needs of patients and society that supersedes self-interest; accountability to patients, society, and the profession; and

a commitment to excellence and ongoing professional development

Familiarity with proper technique for bad news delivery

Family arrives

42 min

Proper presentation of bad news

Patient care

Develop and carry out patient treatment plans

Recurrent shoulder dislocation

Recognition of most critical case at hand

Attention to pain as a vital sign

Severe pain  in left shoulder

20 min

Proper triage between patient one and four

Splinting

Proper pain relief

Interpersonal and communication skills

Develop communication skills

Reassurance

Demands attention

30 min

Communicate lack of acuity or designate someone to do so

Medical knowledge

Know and apply the basic and clinically supportive sciences that are appropriate to their discipline

Ability to perform PSA including drug dosage

Knowledge of reduction techniques

All stable

35 min

Orders preparation of procedural sedation and analgesia

Able to describe a maneuver for reduction

Medical knowledge

Know and apply the basic and clinically supportive sciences that are appropriate to their discipline

Severe external hemorrhage

Recognize methods of external bleeding control

Severe scalp bleeding

 

30 min

Direct pressure on bleeding

 

Failure to control, vitals deteriorating

40 min

Consider suturing

Consider topical hemostatic

Initial hemodynamic support

Patient care

Gather essential and accurate information about their patients

Importance of proper information gathering in trauma

Announces Warfarin use (if not already asked)

40 min

Asks about the reason for warfarin use

Only partial correction with FFP ordered

Considers consult

After a briefing of about 10 minutes the original scrip ran without interruption regardless of resident responses for 45 minutes. During this time the instructor filled out the evaluation sheet regarding KSA competency. The four assisting students were also asked to note every error which is different in opinion with the lead physician, or with questions. During the shift, the lead resident was asked to document orders and progress notes as would be expected in an actual shift. Table-2 describes the presentation of each of the five simulated patients into the sample SMARTER-Shift described in Table-1.

Table 2. Final scenarios of SMARTER-Shift

Scenario

Initial description

Timeline

1

27 year-old male motorist with multiple trauma, alert and oriented, stable vital signs, mild respiratory distress, complaining of leg pain and deformity.

0 min

2,3

46 year-old woman with her five-year old girl brought by EMS after a pedestrian motor collision. The mother is alert complaining of abdominal pain, HR=128 RR=30, BP=90/50 and diaphoresis. The child moans in response to pain and withdraws from pain. Parietal scalp hematoma is evident.

10 min

4

53 year-old made walks in with severe pain and limitation of range of motion in left shoulder after stretching. If asked he has a history of recurrent shoulder dislocations.

20 min

5

78 year-old female with a scalp laceration and severe bleeding after she strikes her head on the sharp bottom edge of a kitchen cabinet. Vitals are stable. If asked the patient is on warfarin for a metallic heart valve.

30 min

 

After the shift and a 10 minute rest, an hour long debriefing session was held in which the strengths and weaknesses of the lead resident in responding to critical events, questions and opinions were discussed with the group. After that the same scenario script ran with a different lead physician, this time allowing for interruptions for questions by the instructor and aiding residents, real-time feedback, and clarifications upon request. The aiding residents were also given the chance to practice the included procedures. Overall each SMARTER-Shift lasted between 4 to 5 hours.

Quantitative analysis

Presented findings are based on five SMARTER-Shifts that were held for residents of emergency medicine at Mashhad University of Medical Sciences during the summer 2016. In total, 25 residents participated in the study voluntarily. Most participants were male (n=19, 76%). The mean age of participants was 32.93 (ranged 25-45). The students’ cumulative scores for knowledge and skills improved significantly before and after the SMARTER-Shift (P<0.001). Overall satisfaction with the SMARTER-Shift as indicated on a ten-point visual analogue scale was 9.96±0.20. Table-3 summarizes results of the pretest and posttests.

Table 3. Mean and standard deviation of residents’ scores before and after intervention

 

Participants

Min score

Max score

Mean ± Standard deviation

 

Pretest

25

4.5

12

7.82 ± 1.755

 

Posttest

25

6.5

14

9.66 ± 2.060

 

Paired student T-test (pretest-posttest)

P value <0.001

 

Most residents (96%) agreed that the SMARTER-Shift can boost their ability to think of multiple clinical issues simultaneously. Nearly all participants (96%) agreed that this method can speed up their management of critically ill patients during a clinical shift; while 84% of residents agreed that it can reduce their stress when faced with multiple patients during a real clinical shift. Moreover 84% agreed that the SMARTER-Shift can help them prioritize responsibilities and medical orders during a clinical shift. Also 88% of residents agreed the SMARTER-Shift was successful in transferring required knowledge for management of patients within the emergency department. As for clinical skills, 80% of residents believed the SMARTER-Shift was a more effective way to gain knowledge and learn clinical skills than a real clinical shift; but when they were asked which setting they find more effective for practicing the gained knowledge and skills only 52% of them believed the SMARTER-Shift was more effective and 48% still found a real clinical shift to be more effective when it comes to practicing previously acquired KSAs. 

Discussion

In this study we describe a new educational method which aims to provide a comprehensive experience for residents of emergency medicine which closely resembles their experience in the clinical setting.

The step by step development of a multiple patient scrip is described and a sample script is presented. Cumulative scores of skill and knowledge for resident improved significantly (P<0.001) before and after the course. Almost all students (96%) preferred SMARTER-shift to a clinical shift for teaching purposes while only 52% preferred it for practicing the skills they had already obtained. 

Evans and colleagues conducted a study “A Comprehensive, Simulation-Based Approach to Teaching Clinical Skills: The Medical Students’ Perspective “in the year 2014. The results of the study showed that a simulation curriculum can enhance the ability to manage acute clinical problems and switch well to the clinical experience (3).

We call this method “The SMARTER-Shift”. In this method we take advantage of previously described SMARTER methodology, and available simulation technologies to enhance resident experience in multiple knowledge and skill domains, while affecting their attitude hopefully and positively. Our results showed that the SMARTER-Shift, although time consuming, is effective in enhancing knowledge and skill proficiency and is rated highly successful in achieving all of its goals by residents. Also a score of 9.66 out of 10 is an impressive affirmation of participants’ satisfaction.

The changing face of technology and new expectations from residency programs around the world demand for development of new and more efficient methods of education. On one hand, the past two decades have seen the rise of simulation as a means of teaching and assessing physicians (22) through eliminating ethical concerns in order to rise clinical medical education (1). On the other hand, the core competencies defined by ACGME, which have been integrated into many residency training programs (12, 23), and are considered as corner stone of for broadening the medical education program.

The SMARTER methodology which was first described in 2008 is an Event-based approach to training which has been customized for use in healthcare (6,13).

Here we have updated the process by adding a ninth step. In this final step several patient scenarios are integrated into one scrip and create opportunities for residents to learn and display competency in multitasking and multiple patient care.

The ability to manage multiple patients simultaneously is a well-recognized characteristic of the Emergency Physician (EP) (18).There are several conscious and subconscious techniques that EPs use to achieve this responsibility. While real multi-tasking is rare, most EPs use task-switching, prioritizing, task-stacking, and willingness to welcome intrusions and interruptions, to do this (19).  It has been suggested that failure to learn these skills is a contributing cause to EP stress, burnout, fatigue, and sleep deprivation. Currently most EM residency programs do not officially include the teaching of these skills, while EM certifying institutions test them (17,19). In our study, 84% of participants agreed that the SMARTER-shift was successful in reducing their stress in real life clinical shifts and the same number found it successful in transferring the required knowledge and skills to residents.

It appears that the most common method for residents to gain the skills of managing multiple patients simultaneously is through direct observation of their supervisors which entails many limitations. Although primitive, this is the method that most EPs have acquired these skills and its importance must not be overlooked. If performed in a timely, graceful manner by the faculty, this onsite learning experience can be invaluable (20). Another method for teaching multi-tasking skills is multidisciplinary team simulation with scenarios focusing on multitasking, and multiple patient care (20). This can be achieved alongside the use of SBT or as a roundtable with the participation of faculty members and other students (24). Obviously the latter requires one to employ imagination for a fuller experience.

With the expansion of simulation technology, multiple encounter simulation scenarios have also been proposed for this purpose. These have been previously reported to create life-like educational settings for EM residents (21,24)but to our knowledge none have focused attention on teaching the skills of multitasking in the emergency department. Most literature have focused on alleviation of possible medical errors and improving team work. One review of SBT for trauma teams found that most of these studies have focused on the lower levels of education such as reaction, and learning, while fewer studies have looked at behavior and result levels. Yet, still improvement in knowledge, skill, and team work was generally significant in all levels after SBT (25).  Here we have combined the benefits of SMARTER methodology with SBT and multiple case scenarios. Although we too assess the lower levels of education in a before and after design too, our findings are very promising and high satisfaction of residents is impressive.

Another point that must be emphasized is the importance of debriefing as part of SBT (20,21). It provides a situation in which the residents can learn from their mistakes and reflect on their errors without the concerns of medico-legal and ethical issues. In our study we designed preliminary dichotomous checklists which helped the instructor determine which responses were presented in a timely manner and which were not. This was the basis for a one hour debriefing session which followed the initial 45 minute shift.

Although in our study we used the SMARTER methodology which has been tailored to meet the educational needs and objectives of residents, we believe that our methods may be adjusted to serve as a teaching platform in other clinical fields as well. The satisfaction of nursing students with simulation based training has been described extensively (26-29). We believe that a similar methodology based on nursing competencies may be an important aid in teaching nursing students, especially those training for advanced education and more specific positions in the emergency department.

In this study we described and tested the SMARTER-shift, as a novel educational method for teaching the care of multiple patients. Our results showed that while this is a time-consuming task the process is feasible and effective in improving knowledge and skills, with almost unanimous satisfaction amongst the residents taking part.

Ethical considerations 

Ethical issues (Including plagiarism, informed consent, misconduct, data fabrication and/or falsification, double publication and/or submission, redundancy, etc.) have been completely observed by the authors.

 Acknowledgement

The authors would like to thank the interns who participated in their study.

Financial support

This article is the result of a research project by Mashhad University of Medical Sciences with the code of 940313, sponsored by Mashhad University of Medical Sciences.     

Conflict of interest

The authors declare that they have no competing interests.

1.         Ziv A, Wolpe PR, Small SD, Glick S. Simulation-based medical education: an ethical imperative. Simulation in healthcare. Simul Healthc. 2006;1(4):252-6.

2.         Cheng A, Lockey A, Bhanji F, Lin Y, Hunt EA, Lang E. The use of high-fidelity manikins for advanced life support training—a systematic review and meta-analysis. Resuscitation. 2015;93:142-9.

3.         Evans LV, Crimmins AC, Bonz JW, Gusberg RJ, Tsyrulnik A, Dziura JD, et al. A comprehensive, simulation-based approach to teaching clinical skills: the medical students’ perspective. Yale J Biol Med. 2014;87(4):575.

4.         Von der Heyden M, Meissner K. Simulation in preclinical emergency medicine. Best Pract Res Clin Anaesthesiol. 2015;29(1):61-8.

5.         Ende J. Feedback in clinical medical education. Jama. 1983;250(6):777-81.

6.         Rosen MA, Salas E, Silvestri S, Wu TS, Lazzara EH. A measurement tool for simulation-based training in emergency medicine: the simulation module for assessment of resident targeted event responses (SMARTER) approach. Simulation in Healthcare. 2008;3(3):170-9.

7.         Hays RT, Singer MJ. Simulation fidelity in training system design: Bridging the gap between reality and training: Springer Science & Business Media; 2012.

8.         Sawyer T, White M, Zaveri P, Chang T, Ades A, French H, et al. Learn, See, Practice, Prove, Do, Maintain: An Evidence-Based Pedagogical Framework for Procedural Skill Training in Medicine. Acad Med. 2015 Aug;90(8):1025-33.

9.         Silvestri S, Wu T, Salas E, Rosen M, Lyons R, Lazarra E. Beyond the basics: brining simulation theory and technology together.  Council of Emergency Medicine Residency Directors (CORD) Academic Assembly 2007.

10.       Swing SR. Assessing the ACGME general competencies: general considerations and assessment methods. Academic Emergency Medicine. 2002;9(11):1278-88.

11.       Swing S, Bashook PG. ACGME Outcome Project—Toolbox of Assessment Methods. ACGME/ABMS Joint Initiative. 2000.

12.       Swick S, Hall MS, Beresin E. Assessing the ACGME competencies in psychiatry training programs. Academic Psychiatry. 2006;30(4):330-51.

13.       Rosen MA, Lazzara EH, Lyons R, Weaver SJ, Salas E, Wu T, et al., editors. SMARTER-Team: Adapting Event-based Tools for Simulation-based Training in Healthcare. Proceedings of the Human Factors and Ergonomics Society Annual Meeting; 2008: SAGE Publications.

14.       Salas E, Priest HA, Wilson KA, Burke CS. Scenario-based training: Improving military mission performance and adaptability.  Military life: The psychology of serving in peace and combat Vol 2: Operational stress. 2006. p. 32-53.

15.       Issenberg SB, Mcgaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005;27(1):10-28.

16.       McFetrich J. A structured literature review on the use of high fidelity patient simulators for teaching in emergency medicine. Emerg Med J. 2006;23(7):509-11.

17.       Heng KW. Teaching and evaluating multitasking ability in emergency medicine residents - what is the best practice? Int J Emerg Med. 2014;7:41.

18.       2011 EM Model Review Task Force, Perina DG, Brunett CP, Caro DA, Char DM, Chisholm CD, et al. The 2011 model of the clinical practice of emergency medicine. Acad Emerg Med. 2012;19(7):e19-40.

19.       Stephens RJ, Fairbanks RJ. Humans and multitask performance: let's give credit where credit is due. Academic emergency medicine: Acad Emerg Med. 2012;19(2):232-4.

20.       Wang EE, Dyne PL, Du H. Systems‐based Practice: Summary of the 2010 Council of Emergency Medicine Residency Directors Academic Assembly Consensus Workgroup—teaching and evaluating the difficult‐to‐teach competencies. Acad Emerg Med. 2011;18(s2):S110-S20.

21.       Kobayashi L, Shapiro MJ, Gutman DC, Jay G. Multiple encounter simulation for aigh‐acuity multipatient environment training. Acad Emerg Med. 2007;14(12):1141-8.

22.       Norcini J. Being smarter about SMARTER: A commentary on "a measurement tool for simulation-based training in emergency medicine: the simulation module for assessment of resident targeted event responses approach". Simul Healthc. 2008;3(3):131-2.

23.       Chapman DM, Hayden S, Sanders AB, Binder LS, Chinnis A, Corrigan K, et al. Integrating the accreditation council for graduate medical education core competencies into the model of the clinical practice of emergency medicine. Acad emerg med. 2004;11(6):674-85.

24.       Small SD, Wuerz RC, Simon R, Shapiro N, Conn A, Setnik G. Demonstration of high-fidelity simulation team training for emergency medicine. Acad emerg med. 1999;6(4):312-23.

25.       Gjeraa K, Moller TP, Ostergaard D. Efficacy of simulation-based trauma team training of non-technical skills. A systematic review. Acta Anaesthesiol Scand. 2014;58(7):775-87.

26.       Cant RP, Cooper SJ. Use of simulation-based learning in undergraduate nurse education: An umbrella systematic review. Nurse Educ Today. 2017;49:63-71.

27.       Lapkin S, Fernandez R, Levett-Jones T, Bellchambers H. The effectiveness of using human patient simulation manikins in the teaching of clinical reasoning skills to undergraduate nursing students: a systematic review. JBI Libr Syst Rev. 2010;8(16):661-94.

28.       Dubovsky SL, Antonius D, Ellis DG, Ceusters W, Sugarman RC, Roberts R, et al. A preliminary study of a novel emergency department nursing triage simulation for research applications. BMC Res Notes. 2017;10(1):15.

29.       Zakeri H, Najafi Z, Mousavi SM, Tabatabaey A. Effects of a virtual classroom in social media platform on the knowledge of emergency department interns: A randomized interventional trial. FMEJ. 2018; 8(3):3-8.