Western Journal of Emergency Medicine: Integrating Emergency Care with Population Health

Introduction: Resident achievement data is a powerful but underutilized means of program evaluation, allowing programs to empirically measure whether they are meeting their program aims, facilitate refinement of curricula and improve resident recruitment efforts. The goal was to provide an overview of available metrics of resident achievement and how these metrics can be used to inform program aims.

Methods: A literature search was performed using PubMed and Google Scholar between May and November of 2020. Publications were eligible for inclusion if they discussed or assessed “excellence” or “success” during residency training. A narrative review structure was chosen due to the intention to provide an examination of the literature on available resident achievement metrics. 

Results: 57 publications met inclusion criteria and were included in the review. Metrics of excellence were grouped into larger categories, including success defined by program factors, academics, national competencies, employer factors, and possible new metrics.

Conclusions: Programs can best evaluate whether they are meeting their program aims by creating a list of important resident-level metrics based on their stated goals and values using one or more of the published definitions as a foundation. Each program must define which metrics align best with their individual program aims and mission.

Introduction: Medical educators are constantly seeking methods to increase engagement in the era of coronavirus disease 2019 (COVID-19) where virtual and blended learning formats are increasingly common. Educational escape rooms have previously been used to motivate learners, enhance communication skills, and cultivate teamwork. However, it is not known whether escape rooms increase learner knowledge as compared to a lecture format. 

Methods: This quasi-experimental study included 30 emergency medicine residents at two programs who participated in both a virtual escape room and a lecture on infectious disease content. Learners completed a pre- and post-quiz and a tool to gauge resident motivation for each activity (the Intrinsic Motivation Inventory [IMI]). The primary objective was to determine a change in knowledge as a result of the activities, and a secondary objective was to determine resident motivation for each format.

Results: At both programs learners demonstrated a significant improvement in their pre- vs. post-quiz scores for the escape rooms (University of California Irvine [UCI]: 77.8% to 88.9%, p = 0.028, Prisma: 73.81% to 89.68%, p = 0.002), whereas the lectures did not impact a statistical improvement (UCI: 73.8% to 78.6%, p = 0.460, Prisma: 85.71% to 91.27%, p = 0.236). Learners at UCI noted equivalent results on the IMI for both formats, while residents at Prisma noted they were more motivated by the escape room.   

Conclusion: Emergency medicine residents at two programs participating in a virtual escape room demonstrated a statistical increase in knowledge on infectious disease content as compared to a lecture format and reported positive motivation ratings for both formats, with one program preferring the escape room.

Introduction: The United States Medical Licensing Examination (USMLE) Step 1 score is one of the few standardized metrics used to objectively review applicants for residency. In February 2020 the USMLE program announced that the numerical Step 1 scoring would be changed to a binary (Pass/Fail) system. In this study we sought to characterize how this change in score reporting will impact the application review process for emergency medicine (EM) program directors (PD). 

Methods: In March 2020 we electronically distributed a validated anonymous survey to EM PDs at 236 US EM residency programs accredited by the Accreditation Council for Graduate Medical Education. Results: Of 236 EM PDs, 121 responded (51.3% response rate). Overall, 72.7% believed binary Step 1 scoring would make the process of objectively comparing applicants more difficult. A minority (19.8%) believed it was a good idea, and 33.1% felt it would improve medical student well-being. The majority (88.4%) reported that they will increase their emphasis on Step 2 Clinical Knowledge (CK) for resident selection, and 85% plan to require Step 2 CK scores at application submission time. 

Conclusion: Our study suggests most EM PDs disapprove of the new Step 1 scoring. As more objective data is peeled away from the residency application, EM PDs will be left to rely more heavily on the few remaining measures, including Step 2 CK and standardized letters of evaluation. Further changes are needed to promote equity and improve the overall quality of the application process for students and PDs.

Introduction: Over the last several decades simulation, in both graduate and undergraduate emergency medicine education, has continued to develop as a leading and highly effective teaching modality. Limited research exists to evaluate the efficacy of low-fidelity (table-top) simulation, as compared to high-fidelity standards, as it relates to medical knowledge learning outcomes. We sought to assess the efficacy of a low-fidelity simulation modality in undergraduate emergency medicine education, based on quantitative medical knowledge learning outcomes.

Methods: A prospective, randomized, crossover-control study comparing objective medical knowledge learning outcomes between simulation modalities. Analysis was designed to evaluate for the statistical equivalence of learning outcomes between the two cohorts. This was done by comparing a calculated 95% confidence interval (CI) around the mean difference in post-test scores, between experimental and control modalities, to a pre-established equivalence margin. 

Results: Primary outcomes evaluating student performance on post-test examinations demonstrated a total cohort CI (95% CI, -0.22 and 0.68). Additional course-subject subgroup analysis demonstrated non-inferior CIs with: Shortness of Breath (95% CI, -0.35 and 1.27); Chest Pain (95% CI, -0.53 and .94); Abdominal Pain (95% CI, -0.88 and 1.17); Cardiovascular Shock (95% CI, -0.04 and 1.29). Secondary outcome analysis was done to evaluate medical knowledge acquisition by comparing the difference in pre and post-test examination between the cohorts. CI of the full cohort ranged from (95% CI, -0.14 and 0.96).

Conclusion: The student’s performance on quantitative medical-knowledge assessment was equivalent between the high-fidelity control and low-fidelity experimental simulation groups. Analysis of knowledge acquisition between the two groups also demonstrated statistical equivalence.

Introduction: Belief in a just world is the cognitive bias that “one gets what they deserve.” Stronger belief in a just world for others (BJW-O) has been associated with discrimination against individuals with low socioeconomic status (SES) or poor health status, as they may be perceived to have “deserved” their situation. Emergency medicine (EM) residents have been shown to “cherry pick” patients; in this study we sought to determine whether BJW-O is associated with a biased case mix seen in residency.

Methods: We assessed EM residents on their BJW-O using a scale with previous validity evidence and behavioral correlates. We identified chief complaints that residents may associate with low SES or poor health status, including psychiatric disease, substance use disorder (SUD); and patients with multidisciplinary care plans due to frequent ED visits. We then calculated the percentage of each of these patient types seen by each resident as well as correlations and a multiple linear regression.

Results: 38 of 48 (79%) residents completed the BJW-O, representing 98,825 total patient encounters. The median BJW-O score was 3.25 (interquartile range 2.81–3.75). There were no significant correlations observed between BJW-O and the percentage of patients with multidisciplinary care plans who were seen, or patients with psychiatric, SUD, dental or sickle cell chief complaints seen; and a multiple linear regression showed no significant association.

Conclusion: Higher BJW-O scores in EM residents are not significantly associated with a biased case mix of patients seen in residency.

Background: Acute stress impairs physician decision-making and clinical performance in resuscitations. Mental skills training, a component of the multistep, cognitive-behavioral technique of stress inoculation, modulates stress response in high-performance fields. 

Objective: We assessed the effects of mental skills training on emergency medicine (EM) residents’ stress response in simulated resuscitations as well as residents’ perceptions of this intervention.

Methods: In this prospective, educational intervention trial, postgraduate year-2 EM residents in seven Chicago-area programs were randomly assigned to receive either stress inoculation training or not. One month prior to assessment, the intervention group received didactic training on the “Breathe, Talk, See, Focus” mental performance tool. A standardized, case-based simulation was used for assessment. We measured subjective stress response using the six-item short form of the Spielberger State-Trait Anxiety Inventory (STAI-6). Objective stress response was measured through heart rate (HR) and heart rate variability (HRV) monitoring. We measured subjects’ perceptions of the training via survey.

Results: Of 92 eligible residents, 61 participated (25 intervention; 36 control). There were no significant differences in mean pre-/post-case STAI-6 scores (-1.7 intervention, 0.4 control; p = 0.38) or mean HRV (-3.8 milliseconds [ms] intervention, -3.8 ms control; p = 0.58). Post-assessment surveys indicated that residents found this training relevant and important. 

Conclusion: There was no difference in subjective or objective stress measures of EM resident stress response after a didactic, mental performance training session, although residents did value the training. More extensive or longitudinal stress inoculation curricula may provide benefit.

Introduction: Residency applicants use multiple resources to guide their application process including the Student Doctor Network (SDN), a publicly available online forum for the discussion of various topics in medical education. In recent years, specialty-specific forums for residency applicants to self-report their own application information have become popular. These forums allow other applicants to review self-reported data from their peers to inform their own application process. The accuracy of this resource is unknown. To determine whether the SDN is an accurate source of information for emergency medicine (EM) applicants, we compared self-reported SDN data to objective data from the National Resident Matching Program (NRMP).

Methods: We retrospectively reviewed self-reported SDN data by DO and MD candidates from EM forums for the 2014, 2016, and 2018 residency application cycles. These data were compared to the NRMP charting outcomes for each respective year.

Results: A total of 360 EM applicants self-reported data on the SDN during the years reviewed. The majority of these applicants (79%) posted for the 2018 application cycle following transition to a Google Docs spreadsheet. For the first two years of analysis, mean United States Medical Licensing Examination (USMLE) scores were similar to SDN reports. For the most recent year studied, applicants who posted to SDN reported higher mean (USMLE) Step 1 (234, 95% confidence interval [CI], 233-236) and Step 2 scores (250, 95% CI, 248-251) when compared to NRMP data (231 and 241). Reported contiguous residency program ranks were similar to NRMP in all years, and the proportion indicating Alpha Omega Alpha Honor Medical Society membership was similar to NRMP only for the most recent year studied. 

Conclusion: Self-reporting on SDN showed a slight bias toward higher USMLE step scores in the most recent year when compared to objective NRMP data. Self-reporting on SDN has increased in recent years, but it is unknown whether this increase will lead to more accurate information for EM applicants. Given the self-reported nature of the SDN, applicants should use SDN forums with caution.

Introduction: The Covid-19 pandemic limited educational and career development opportunities for medical students, requiring innovative programs to accelerate professional identity formation and clinical skills acquisition. 

Methods: We developed a brief coaching intervention that took place over the advanced (sub-internship) emergency medicine rotation at our institution. We trained coaches using a newly developed workshop, who met with students for an average of 4.5 hours over 3 weeks.

Impact/effectiveness: We showed that this coaching program was both feasible and impactful for faculty coaches and medical students.

Introduction: Safety concerns surrounding the coronavirus 2019 pandemic led to the prohibition of student rotations outside their home institutions. This resulted in emergency medicine (EM)- bound students having less specialty experience and exposure to outside programs and practice environments, and fewer opportunities to gain additional Standardized Letters of Evaluation, a cornerstone of the EM residency application. We filled this void by implementing a virtual clerkship.

Methods: We created a two-week virtual, fourth-year visiting clerkship focused on advanced medical knowledge topics, social determinants of health, professional development, and professional identity formation. Students completed asynchronous assignments and participated in small group-facilitated didactic sessions. We evaluated the virtual clerkship with pre- and post-medical knowledge tests and evaluative surveys.

Results: We hosted 26 senior medical students over two administrations of the same two-week virtual clerkship. Students had a statistically significant improvement on the medical knowledge post-tests compared to pre-tests (71.7% [21.5/30] to 76.3% [22.9/30]). Students reported being exposed to social determinants of health concepts they had not previously been exposed to. Students appreciated the interactive nature of the sessions; networking with other students, residents, and faculty; introduction to novel content regarding social determinants of health; and exposure to future career opportunities. Screen time, technological issues, and mismatch between volume of content and time allotted were identified as potential challenges and areas for improvement.

Conclusion: We demonstrate that a virtual EM visiting clerkship is feasible to implement, supports knowledge acquisition, and is perceived as valuable by participants. The benefits seen and challenges faced in the development and implementation of our clerkship can serve to inform future virtual clerkships, which we feel is a complement to traditional visiting clerkships even though in-person clerkships have been re-established.

Background: The COVID-19 pandemic has revealed the importance of teaching medical students pandemic preparedness and COVID-19 related clinical knowledge. To fill the gap of COVID-19 instruction backed by evaluation data, we present a comprehensive COVID-19 pilot curriculum with multiple levels of evaluation data. 

Methods: In the spring of 2020, the University of California, Irvine (UCI) School of Medicine piloted a two-week, primarily asynchronous COVID-19 elective course for medical students. The goal of the course is to provide a foundation in clinical care for COVID-19 while introducing students to emerging issues of a modern pandemic. Objectives align with institutional objectives, and instruction is delivered in thematic modules. Our curriculum utilizes numerous instructional strategies effective in distance learning including independent learning modules (ILM), reading, video lectures, discussion board debates, simulation and evidence-based argument writing. We designed a three-level, blended evaluation plan grounded in the Kirkpatrick and Kirkpatrick evaluation model that assessed student satisfaction, relevance, confidence, knowledge and behavior. 

Results: Our end of course survey revealed that students had high levels of satisfaction with the curriculum, and felt the course was relevant to their clinical education. Various assessment tools showed excellent levels of knowledge attainment. All respondents rated themselves as highly confident with the use of personal protective equipment, though fewer were confident with ventilator management. 

Conclusion: Overall our pilot showed that we were able to deliver relevant, satisfying COVID-19 instruction while allowing students to demonstrate knowledge and desired behaviors in COVID-19 patient care.

Introduction: Head computed tomography (CT) interpretation is a vital skill for emergency physicians. Existing literature shows poor concordance between emergency physicians and radiologists in head CT interpretation. Prior studies have used passive learning methods to address this knowledge gap. We created an active learning curriculum for teaching head CT interpretation to emergency medicine (EM) residents and compared its effectiveness to a passive learning strategy. 

Methods: We conducted a prospective, randomized controlled study of EM residents at a single institution. Three educational sessions were delivered over a three-month period via video conference. The active learning cohort (ALC) scrolled through head CT teaching cases we designed on Pascbin, a web-based radiology picture archiving and communication system. The passive learning cohort (PLC) watched instructional videos that scrolled through the same cases. Both cohorts were given equal time to review the cases and ask an instructor questions. Residents took pre-intervention and post-intervention tests on head CT interpretation. We analyzed scores using paired and unpaired t-tests.

Results: Forty-two residents took the pre-intervention test.  Mean pre- and post-test scores for the ALC were 43.8% and 59.0% (P <0.001), and for the PLC were 41.7% and 45.3% (P = 0.29). The difference in ALC and PLC post-test scores was statistically significant (P = 0.009) with a large effect size (Cohen’s d = 1.34). 

Conclusion: Our active learning head CT curriculum using Pacsbin showed superior learning outcomes when compared to a passive learning strategy and required no additional time or resources. This intervention offers a more effective and learner-centric method for implementing radiology curricula in EM residency programs.

Introduction: Residency didactic conferences transitioned to a virtual format during the COVID-19 pandemic. This format creates questions about effective educational practices, which depend on learner engagement. In this study we sought to characterize the competitive demands for learner attention during virtual didactics and to pilot methodology for future studies.

Methods: This was a prospective, observational, cohort study of attendees at virtual didactics from a single emergency medicine residency, which employed a self-report strategy informed by validated classroom assessments of student engagement. We deployed an online, two-question survey  polling across six conference days using random signaled sampling. Participants reported all activities during the preceding five minutes.

Results: There were 1303 responses over 40 survey deployments across six nonadjacent days. Respondents were residents (63.4%); faculty (27.5%); fellows (2.3%); students (2%); and others (4.8%). Across all responses, about 85% indicated engagement in the virtual conference within the last five minutes of the polls. The average number of activities engaged in was 2.0 (standard deviation = 1.1). Additional activities included education-related (34.2%), work-related (21.1%), social (18.8%), personal (14.6%), self-care (13.4%), and entertainment (4.4%). 

Conclusion: Learners engage in a variety of activities during virtual didactics. Engagement appears to fluctuate temporally, which may inform teaching strategies. This information may also provide unique instructor feedback. This pilot study demonstrates methodology for future studies of conference engagement and learning outcomes.

Introduction: Following resident requests, we created a public metrics dashboard to inform residents of their daily productivity. Our goal was to iteratively improve the dashboard based on resident feedback and to measure the impact of reviewing aggregate data on self-perceived productivity. 

Methods: A 10-question anonymous survey was completed by our postgraduate year 1-3 residents. Residents answered questions on the dashboard and rated their own productivity before and after reviewing aggregate peer-comparison data. Using the Wilcoxon signed-rank test we calculated summary statistics for survey questions and compared distributions of pre- and post-test, self-rated productivity scores. 

Results: All 43 eligible residents completed the survey (response rate 100%). Thirteen (30%) residents “rarely” or “never” reviewed the dashboard. No respondents felt the dashboard measured their productivity or quality of care “extremely accurately” or “very accurately.” Seven (16%) residents felt “very” or “extremely pressured” to change their practice patterns based on the metrics provided, and 28 (65%) would have preferred private over public feedback. Fifteen residents (35%) changed their self-perceived rank after viewing peer-comparison data, although not significantly in a particular direction (z = 0.71, P = 0.48). 

Conclusion: Residents did not view the presented metrics as reflective of their productivity or quality of care. Viewing the dashboard did not lead to statistically significant changes in resident self-perception of productivity. This finding highlights the need for expanding the resident conversation and education on metrics, given their frequent inclusion in attending physician workforce payment and incentive models.

Improving the recruitment, retention, and leadership advancement of faculty who are under-represented in medicine is a priority at many academic institutions to ensure excellence in patient care, research, and health equity. Here we provide a critical review of the literature and offer evidence-based guidelines for faculty recruitment, retention, and representation in leadership. Recommendations for recruitment include targeted recruitment to expand the candidate pool with diverse candidates, holistic review of applications, and incentivizing stakeholders for success with diversity efforts. Retention efforts should establish a culture of inclusivity, promote faculty develop-ment, and evaluate for biases in the promotion and tenure process. We believe this guide will be valuable for all leaders and faculty members seeking to advance diversity, equity, and inclusion in their institutions.