See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/354936122

Bias in recruitment: A focus on virtual interviews and holistic review to

advance diversity

Article · September 2021

DOI: 10.1002/aet2.10661

CITATION READS

1 33

4 authors, including:

Al'Ai Alvarez
Stanford Medicine
23 PUBLICATIONS   72 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Al'Ai Alvarez on 25 October 2021.

The user has requested enhancement of the downloaded file.

 | |
Received: 21 October 2020    Revised: 29 December 2020    Accepted: 2 February 2021

DOI: 10.1002/aet2.10661

C O M M E N TA R Y A N D P E R S P E C T I V E

Bias in recruitment: A focus on virtual interviews and holistic

review to advance diversity

Onyekachi Otugo MD, MPH, MPA1  | Al’ai Alvarez MD2  | Italo Brown MD, MPH3 |

Adaira Landry MD, MEd4
1
Department of Emergency Medicine, Brigham and Women’s Hospital, Boston, Massachusetts, USA
2
Department of Emergency Medicine, Emergency Medicine Residency Program, Stanford University School of Medicine, Stanford, California, USA
3
Department of Emergency Medicine, Stanford University School of Medicine, Stanford, California, USA
4
Department of Emergency Medicine, Brigham and Women’s Hospital, Harvard Medical School, Harvard Affiliated Emergency Medicine Residency, Boston,
Massachusetts, USA

*Correspondence
Onyekachi Otugo, MD, MPH, Department of Emergency Medicine, Brigham and Women’s Hospital, 75 Francis Street, Boston, MA 02215, USA.
Email: [email protected]

I NTRO D U C TI O N to seek a foundational understanding of the complexity and impact

of bias against UiM students and how the upcoming virtual interview
In 2019, nearly 45,000 medical students applied for the residency season format creates additional avenues for bias. Consequently, it
match.1 Interview invitations are commonly influenced by a variety is critical to discuss the importance of holistic review during this ap-
of factors such as academic performance metrics, personal state- plication cycle.
ment, and research interests. 2 On the 2020 survey of program Just as bias affects diversity and inclusion in medicine, the
directors, the application components that ranked highest were pandemic has a compounded effect on the landscape of medi-
quantifiers of academic achievement; among those were the United cal education with recruitment in particular. Across the country,
States Medical Licensing Examination (USMLE) Step 1 and 2 scores, routine clinical opportunities for students are now limited as an
clerkships grades, and class ranking. Other factors included were infection control measure and to preserve personal protection
the narratives accounting for student performance via letters of rec- equipment (PPE). Further, traditional subinternships, a critical
ommendation and the Dean's letter. 2 After interviews are complete, component of an application where medical students often travel
each candidate is ranked against their peers. Residency programs to different institutions, are also restricted. These rotations allow
heavily base ranking of candidates on performance during the in- medical students to better understand the practice variation of
terview day. All three of these review stages—­selection for inter- a field and an institution's culture, obtain a letter of evaluation,
view, performance during interview, and overall ranking compared and gain access to potential mentors and collaborators.7 For UiM
to peers—­leave room for bias. applicants, this opportunity offers them access to advocates in
The residency application and interview processes are subject that program. Historically Black colleges and universities (HBCUs),
to implicit and explicit bias.3,4 Furthermore, in 2020 national orga- who traditionally have higher percentages of UiM students, are
nizations such as The Coalition on Physician Accountability, which credited for training the majority of Black physicians in the United
includes the Association of American Medical Colleges (AAMC), States.10 HBCU medical schools, however, have limited affiliated
10 national emergency medicine organizations, and local hospital residency programs especially given hospital closures compared
institutions, recommended virtual interviews in response to the to large academic institutions, thereby limiting students’ access to
COVID-­19 pandemic.5,6 However, this new interview format may early advising specific to the specialty.11,12 Students from HBCU
potentiate bias and disproportionately affect students who histori- programs often use rotations to showcase their clinical strengths,
7–­9
cally have been underrepresented in medicine (UiM). It is essential gain exposure to the field, and build connections. The inability

Supervising Editor: Teresa Y. Smith, MD.

to travel for these rotations, therefore, affects every candidate's

© 2021 Society for Academic Emergency Medicine

AEM Educ Train. 2021;5(Suppl. 1):S135–S139. wileyonlinelibrary.com/journal/aet2    S135 |

 |
S136      COMMENTARY AND PERSPECTIVE

application, although much more so for students from HBCU pro- Bias in video interviews within medicine has been docu-
grams with limited internal residency programs. In addition to mented. In the 2017 to 2018 application cycle, standardized
the loss of away rotation opportunities, students also experience video interview (SVI) was created by the AAMC and was piloted
difficulty scheduling USMLE test dates due to travel restrictions as an opportunity for an applicant to “showcase the intangibles
and site closures.13 Because of these changes to the application about themselves” that were not highlighted in their application.
experience, it is imperative to develop and implement a true ho- The SVI aimed to offer a more holistic evaluation of an appli-
listic review that looks beyond inherently biased metrics (scores, cation by providing more information regarding communication,
evaluations, AOA, grades) and instead focuses on life experiences, professional, and interprofessional skills. 23,24 However, the data
social identities, and extracurriculars of the candidates to ensure received from the SVI were mixed, leaving programs unable to
equity, especially for UiM candidates. assess how to use this additional data point, and the SVI for-
mat was later abandoned. 24–­26 One study revealed no correlation
between USMLE scores and performance on the SVI. 27 Another
I NTE RV I E W B I A S study, while favoring women over men and US-­M D applicants
in SVI, also showed standardized evaluation ratings having bias
Biases can be formed and magnified within seconds of meeting against Black men. 28 Further an additional study found that
14
someone. Implicit biases place vulnerable groups at risk for pre- emergency medicine interviewees were concerned about bias
mature and unfair judgment.15 Also, while a person's name has toward their physical appearance, ethnicity, and video produc-
never been directly linked to economic success, data reveal that a tion quality. 29 Despite the AAMC having personnel trained to
person's given name can affect the likelihood of an interview offer recognize and avoid bias during the SVI scoring, standardizing
and success.16 Another source of bias is how candidates convey interviews across all programs and specialties for academic re-
their thoughts through language. A study from the University of cruitment cycle is challenging. 30 Furthermore, students with
Chicago has explored the possibility of linguistic and speech bias unreliable internet or are from low socioeconomic status might
during interviews and found that the ability to understand some- be judged unfairly based on connectivity issues or background
one's speech as a heavily desired factor—­t herefore, accents could decor. All of these, when left unaddressed will likely migrate
lower the desirability of a candidate. Subconsciously we make into the 2020 to 2021 application cycle due to heavy reliance
judgments on others based on accents, pitch, tone, and inflec- on the virtual format across all programs. Table 1 lists potential
tion points.17,18 In addition, it has been shown that interviewers interventions that can be incorporated to remove bias during the
typically look for applicants who remind them of themselves, the interview process.
concept of homophily.19 Bias can be gendered as well. We know
that women are questioned and interrupted more often during the
interview process and face more follow-­up questions than men, H O LI S TI C R E V I E W
who were given space and time to comment on their potential in
the business. 20 In the field of medicine, where diversity metrics The holistic review of applications humanizes each applicant beyond
continue to lag behind current estimates of the racial and ethnic quantitative metrics and narrative evaluations. Studies reveal that
composition of the U.S. population, these are essential details that holistic application reviews create a diverse pool of applicants in
must be proactively addressed. part due to a shift in emphasis from USMLE step scores alone to let-
ters of recommendations, life experiences, and extracurriculars.31,32
Unfortunately, as of 2020, program directors rated “other life ex-
V I RT UA L I NTE RV I E W B I A S perience,” “volunteer/extracurricular activities,” or “fluency in the
language spoken by [the] patient population” with lower importance
Vulnerable groups such as women and those applicants who iden- as it pertains to rank list placement. 2 However, all three of these
tify as UiM deserve special attention during virtual interviews. factors help create a picture of personhood—­the distance traveled
Compared to in-­p erson interviews (where unstructured, im- and lived journey—­and are therefore critical in performing a holistic
promptu conversations are natural), the transition to virtual inter- review that considers and values the entire application without un-
views limits downtime interactions and, therefore, the familiarity fairly placing weight on factors that have historically limited diver-
21
between the applicants and programs. Although safer and cost-­ sity (e.g., step scores, evaluations, AOA).
effective, virtual interviews in response to needs and restrictions
due to the pandemic introduce different additional biases. 8,22
When considering the intersectionality of various identities and C H A LLE N G E S I N H O LI S TI C R E V I E W
demographics including age, ethnicity, race, gender identity and
sexual orientation, abilities, and religion, these unanticipated bi- Holistic review is not without its challenges. Bias is present at multiple
ases may infringe on critical efforts to increase diversity, equity, junctures of the application, recruitment, and interview cycle. Even
and inclusion within medicine. when institutions have attempted to reduce bias through blinding
COMMENTARY AND PERSPECTIVE |
      S137

TA B L E 1  Approach and impact of potential interventions to limit bias

Intervention Approach Impact

Virtual background Create an expectation that all applicants use a Standardize the screen so that applicants are not
solid color for their background (a bare wall judged or graded by their surroundings.
without decor or a solid color image).
Offer several plain, standardized, virtual
background images to suit interviewees
preferences.
Pronunciation of names Invite students to submit audio recordings of their Removes potential discomfort that appears when
names via an online free software which will be unable to pronounce a name. A person's name
shared with interviewers. is a significant connection to one's identity.
Offer a technology check session Have an administrator host sessions for students Allows students to gain familiarity with software
unfamiliar with the technology, especially and a chance to explore compatibility and fix
if the program is hosting interviews on less technological mishaps.
frequently used software platforms.
Implicit bias training for interviewers Encourage the use of widely available open access While not sufficient to create change alone,
educational videos and incorporation of training starts the conversation and
self-­assessment. introduces the language and the concept of
bias.
Standardized questions Develop prospective core questions for Avoids stereotypical questions and also offers
interviewers to create consistency in areas the ability to compare among candidates.
asked and set agreed-­upon criteria that align Avoids personality questions and instead aligns
with departmental goals. with the needs of the program.
Focus questions on skills.
Multiple mini-­interviews Applicants get interviews from a series of multiple Applicants gain a better understanding of the
reviewers. institution by talking with different faculty
members and residents.
Allows programs to get to know the applicant
from an array of reviewers.
Mock interviews For students interviewing from schools without While mock interviews should have no bearing
an associated residency program, consider on the applicants’ final evaluation, it provides
offering mock virtual interviews with faculty an opportunity for applicants without home
or alumni. institution support.
Diverse Interviewers Increase UiM faculty and resident presence during Applicants can meet faculty and residents with
the interview day. shared experiences of being UiM.
Compensate UiM faculty and residents for the Avoids minority tax and also sets the model for
time given above minimum requirements or inclusion efforts as being part of the culture
expectations to be part of recruitment. rather than on a voluntary basis.
Anonymous post-­interview feedback Reaching out to UiM applicants after the Applicants have the opportunity to report
interview cycle for ongoing quality review and negative and positive experiences.
improvement. The program receives feedback to continue to
reiterate and reinforce positive experiences.

Abbreviation: UiM, underrepresented in medicine.

of an applicant's name and implementing automated interview pro- staggering difference that has persisted over the 25 years evaluated
33
cesses, the effects of biases persist. In one study, when an em- in the study.34
ployer blinded the applicants’ names in file reviews, they found other
ways of discriminating candidates such as looking at the applicant's
affiliations and participation in the Black student organization.33 In B E N E FIT S O F H O LI S TI C R E V I E W
response, studies have shown that Black and Asian applicants have
attempted to avoid job discrimination through the “whitening” of While one's application is vulnerable to bias at several junctures,
their applications, which yielded an increase in interview offers. This performing a holistic review offers a big picture about each candi-
was achieved by deliberately concealing aspects of the application, date and creates connections in each aspect of the application entry
including race-­related extracurricular activities, such as being presi- that can help overcome stereotypes.35 Programs that have applied
33
dent of a diversity and inclusion organization. In a meta-­analysis the use of holistic review have increased interviews for UiM and
performed regarding employer–­applicant callbacks, Black appli- the matriculation of these candidates.36 The AAMC has also shown
cants were less likely to receive a callback than White applicants, a that implementing a holistic review increases the interview and
 |
S138      COMMENTARY AND PERSPECTIVE

matriculation of first-­generation college, Black/African American, CO N C LU S I O N

35
and Hispanic students in medical school.
Advancing diversity in medicine continues to be a critical challenge and
heavily affects women and underrepresented in medicine applicants.
C H A LLE N G E S I N PE R FO R M I N G H O LI S TI C Virtual interviews will likely pose additional bias against women and
REVIEW underrepresented in medicine applicants. At the same time, there are
time and resource limitations that interfere with the implementation
The changes in the 2020 to 2021 application cycle also create a new of formal, standardized holistic reviews. These reviews would mini-
challenge for performing a holistic review of candidates given an an- mize bias when creating an environment of inclusion during the current
ticipated higher volume of applications with less time to review them recruitment season. Moving forward, it is necessary to prioritize the
thoroughly. Performing holistic reviews will be time-­ and resource-­ unique challenges in holistic reviews and virtual interviews in context
intensive. While restrictions on travel will lessen the financial and of diversity, equity and inclusion to advance representation in medicine.
time commitments for applicants, there is concern that applicants
will be financially available to apply to more programs, which will ul- C O N FL I C T O F I N T E R E S T
timately shift supply and demand for interview positions.37 Medical The authors have no potential conflicts to disclose.
schools have seen a 16% increase in their applicant pool this year.38
Residency programs anticipate a similar increase in trend.39 If pro- AU T H O R C O N T R I B U T I O N S
grams receive more applications to review over a shorter time al- All authors contributed to the writing, drafting and editing of this
lotment, especially since ERAS opens later this cycle on October manuscript concept and design, acquisition of the data, analysis and
21, 2020, performing a thorough holistic review of each applicant interpretation of the data, and critical revision of the manuscript for
will be challenging given that the deadline to the submit rank list important intellectual content.
is unchanged. Despite these high stakes decisions that are suscep-
tible to bias and variations in practice, programs will be expected ORCID
to do more in less time. To combat these challenges, implementing Onyekachi Otugo  https://orcid.org/0000-0002-9432-6161
quantitative, preweighted scoring pre-­ and postinterview can help Al’ai Alvarez  https://orcid.org/0000-0002-5438-2476
eliminate potential bias, and advocating for transparency can help Adaira Landry  https://orcid.org/0000-0002-5299-679X
reduce potential bias. In one study, they placed a cap on the num-
ber of applications evaluated per reviewer, selected interviewers to REFERENCES
eliminate bias, standardized interview questions, performed multi- 1. NRMP Main Residency Match Report Shows Record-­Highs in
ple mini-­interviews, and weighted z-­scores to create heat maps for Registrants, Positions. The Match, National Resident Matching
Program. May 7, 2020. Accessed September 16, 2020. https://
ratings to evaluate for outliers.40
www.nrmp.org/press​-­relea​se-­mrm-­resul​t s-­and-­data-­2020/
2. National Resident Matching Program, Data Release and Research
Committee. Results of the 2020 NRMP Program Director Survey.
LE T TE R S O F R E FE R E N C E S National Resident Matching Program; 2020.
3. Filippou P, Mahajan S, Deal A, et al. The presence of gender bias
in letters of recommendations written for urology residency ap-
Letters of recommendation have also been shown to have a linguistic plicants. Urology. 2019;134:56-­61. doi:https://doi.org/10.1016/j.
bias against women and racial minorities applying to residency. One urolo​g y.2019.05.065
study revealed that agentic words, descriptors related to leadership 4. Hauge LS, Stroessner SJ, Chowdhry S, Wool NL. Evaluating resi-
dent candidates: does closed file review impact faculty ratings? Am
and work ethic, were less likely to be used when describing Black
J Surg. 2007;193(6):761-­765. doi:https://doi.org/10.1016/j.amjsu​
and Latinx applicants when compared to White and Asian appli- rg.2007.01.020
cants. Female applicants were also more likely than male applicants 5. Conducting Interviews During the Coronavirus Pandemic.
to have both agentic and communal terms used in their recommen- Association of American Medical Colleges. Accessed December 12,
dations, with communal terms being used to describe stereotypes 2020. https://www.aamc.org/what-­we-­do/missi​on-­areas/​medic​al-­
educa​tion/condu​c ting​-­inter​views​-­durin​g-­coron​aviru​s-­pandemic
associated with being a woman such as kind and sympathetic.41 A
6. Consensus Statement on the 2020-­ 2021 Residency Application
way to overcome bias is to standardize letters of recommendations Process. Emergency Medicine Residents’ Association. 2020. Accessed
and evaluations. In emergency medicine, after the implementation December 21, 2020. http://www.emra.org/be-­invol​ved/be-­an-­
of the standardized letters of evaluation (SLOE), a study found no advoc​ate/worki​ng-­for-­you/resid​ency-­appli​c atio​n-­proce​ss/
7. Murphy B. Residency Match 2021: How COVID-­19 is forcing major
“pervasive differences” in linguistic categories in terms of gender
adjustments. American Medical Association. Accessed September
bias.42 Of note, the specialty also moved from using letters of recom- 16, 2020. https://www.ama-­assn.org/resid​ents-­stude​nts/match/​
mendations to SLOE to develop an objective focus on assessment. resid​ency-­match​-­2021-­how-­covid​-­19-­forci​ng-­major​-­adjus​tments
This example highlights the importance of standardizing the format 8. Nwora C, Allred DB, Verduzco-­Gutierrez M. Mitigating bias in
virtual interviews for applicants who are underrepresented in
for letters of references to limit bias and as a result provide a struc-
medicine. J Natl Med Assoc. 2020;113(1):74-­76. doi:https://doi.
tured format and minimize biasing language in evaluations. org/10.1016/j.jnma.2020.07.011
COMMENTARY AND PERSPECTIVE |
      S139

9. Capers Q, Clinchot D, McDougle L, Greenwald AG. Implicit racial 28. Hopson LR, Regan L, Bond MC, et al. The AAMC standardized video
bias in medical school admissions. Acad Med. 2017;92(3):365-­369. interview and the electronic standardized letter of evaluation in
doi:https://doi.org/10.1097/ACM.00000​0 0000​0 01388 emergency medicine: a comparison of performance characteristics.
10. Gasman M, Smith T, Ye C, Nguyen TH. HBCUs and the production Acad Med. 2019;94(10):1513-­1521. doi:https://doi.org/10.1097/
of doctors. AIMS Public Health. 2017;4(6):579-­589. doi:https://doi. ACM.00000​0 0000​0 02889
org/10.3934/publi​cheal​th.2017.6.579 29. Winfield AD, Chhabra N, Schindlbeck MA, Bowman SH. Applicant
11. Sullivan LW, Suez Mittman I. The state of diversity in the health attitudes toward the association of American Medical Colleges’
professions a century after Flexner. Acad Med. 2010;85(2):246-­253. standardized video interview. AEM Educ Train. 2020;4(1):5.
doi:https://doi.org/10.1097/ACM.0b013​e3181​c88145 doi:https://doi.org/10.1002/aet2.10355
12. Walker KO, Calmes D, Hanna N, Baker R. The impact of public 3 0. AAMC Standardized Video Interview Evaluation Summary.
hospital closure on medical and residency education: implications Association of American Medical Colleges. Accessed September
and recommendations. J Natl Med Assoc. 2008;100(12):1377-­1383. 16, 2020. https://stude​nts-­resid​ents.aamc.org/apply​ing-­resid​
doi:https://doi.org/10.1016/s0027​-­9684(15)31537​-­6 ency/artic​le/svi-­evalu​ation​-­summa​r y/
13. Rolak S, Keefe AM, Davidson EL, Aryal P, Parajuli S. Impacts and 31. Grabowski CJ. Impact of holistic review on student interview pool
challenges of United States medical students during the COVID-­19 diversity. Adv Health Sci Educ. 2018;23(3):487-­498. doi:https://doi.
pandemic. World J Clin Cases. 2020;8(15):3136-­3141. doi:https:// org/10.1007/s1045​9-­017-­9807-­9
doi.org/10.12998/​wjcc.v8.i15.3136 32. Spector AR, Railey KM. Reducing reliance on test scores re-
14. Willis J, Todorov A. First Impressions: Making Up Your Mind After duces racial bias in neurology residency recruitment. J Natl
a 100-­Ms Exposure to a Face. Psychological Science. May 6, 2016. Med Assoc. 2019;111(5):471-­474. doi:https://doi.org/10.1016/j.
Accessed September 11, 2020. https://journ​als.sagep​ub.com/doi/ jnma.2019.03.004
https://doi.org/10.1111/j.1467-­9280.2006.01750.x 33. Gerdeman D. Minorities Who “Whiten” Job Resumes Get More
15. Woods SE, Harju A, Rao S, Koo J, Kini D. Perceived biases and Interviews. Harvard Business School Working Knowledge. May 17,
prejudices experienced by international medical graduates in the 2017. Accessed September 11, 2020. http://hbswk.hbs.edu/item/
US post-­graduate medical education system. Med Educ Online. minor​ities​-­who-­white​n-­job-­resum​es-­get-­more-­inter​views
2006;11:1. doi:https://doi.org/10.3402/meo.v11i.4595 3 4. Quillian L, Pager D, Midtbøen AH, Hexel O. Hiring Discrimination
16. Bertrand M, Mullainathan S. Are Emily and Greg more employable Against Black Americans Hasn’t Declined in 25 Years. Harvard
than Lakisha and Jamal? A field experiment on labor market discrim- Business Review. October 11, 2017. Accessed September 10, 2020.
ination. Nat Bur Econ Res. 2003. doi:https://doi.org/10.3386/w9873 https://hbr.org/2017/10/hirin​g-­discr​imina​tion-­again​st-­black​-­ameri​
17. Grogger J. Speech and wages. J Hum Resources. 2019;54(4):926–­ cans-­hasnt​-­decli​ned-­in-­25-­years
952. doi:https://doi.org/10.3368/jhr.54.4.0617.8841R 35. Addams AN, Bletzinger RB, Sondheimer HM, White SE, Johnson
18. McWhorter J. The Biases We Hold Against the Way People Speak. LM. Roadmap to Diversity: Integrating Holistic Review Practices
The New York Times. July 21, 2020. Accessed September 15, 2020. Into Medical School Admission Processes. Association of American
https://www.nytim​e s.com/2020/07/21/books/​revie​w/how-­you-­ Medical Colleges; 2010.
say-­it-­kathe​rine-­kinzl​er.html 36. Aibana O, Swails JL, Flores RJ, Love LT. Bridging the gap: holistic re-
19. Rivera LA. Hiring as cultural matching: the case of elite professional view to increase diversity in graduate medical education. Acad Med.
service firms. Am Sociol Rev. 2012;77(6):999-­1022. doi:https://doi. 2019;94(8):1137-­1141. doi:https://doi.org/10.1097/ACM.00000​
org/10.1177/00031​22412​463213 00000​0 02779
20. Blair-­Loy M, Rogers L, Glaser D, Wong Y, Abraham D, Cosman P. 37. Fogel HA, Liskutin TE, Wu K, Nystrom L, Martin B, Schiff A. The
Gender in engineering departments: are there gender differences economic burden of residency interviews on applicants. Iowa
in interruptions of academic job talks? Social Sci. 2017;6(1):29. Orthop J. 2018;38:9-­15.
doi:https://doi.org/10.3390/socsc​i6010029 38. Kowarski I. How Coronavirus Could Affect Medical School Admissions.
21. Fuchs JW, Youmans QR. Mitigating bias in the era of virtual resi- US News & World Report. Accessed September 16, 2020. https://www.
dency and fellowship interviews. J Grad Med Educ. 2020;12(6):674-­ usnews.com/educa​tion/best-­gradu​ate-­schoo​ls/top-­medic​al-­schoo​ls/
677. doi:https://doi.org/10.4300/JGME-­D-­20-­0 0443.1 artic​les/how-­coron​aviru​s-­could​-­affec​t-­medic​al-­schoo​l-­admis​sions
22. Pourmand A, Lee H, Fair M, Maloney K, Caggiula A. Feasibility and usabil- 39. Hammoud MM, Standiford T, Carmody JB. Potential implications of
ity of tele-­interview for medical residency interview. West J Emerg Med. COVID-­19 for the 2020–­2021 residency application cycle. JAMA.
2018;19(1):80-­86. doi:https://doi.org/10.5811/westj​em.2017.11.35167 2020;324(1):29-­3 0. doi:https://doi.org/10.1001/jama.2020.8911
23. Bird SB, Hern HG, Blomkalns A, et al. Innovation in residency selection: 4 0. Paterson QS, Hartmann R, Woods R, Martin LJ, Thoma B. A
the AAMC standardized video interview. Acad Med. 2019;94(10):1489-­ transparent and defensible process for applicant selection within
1497. doi:https://doi.org/10.1097/ACM.00000​00000​002705 a Canadian emergency medicine residency program. CJEM.
24. King A, Mayer C, Starnes A, Barringer K, Beier L, Sule H. Using 2020;22(2):215-­223. doi:https://doi.org/10.1017/cem.2019.460
the Association of American Medical Colleges standardized 41. Grimm LJ, Redmond RA, Campbell JC, Rosette AS. Gender and racial
video interview in a holistic residency application review. Cureus. bias in radiology residency letters of recommendation. J Am Coll Radiol.
2017;9(12):e1913. doi:https://doi.org/10.7759/cureus.1913 2020;17(1):64-­71. doi:https://doi.org/10.1016/j.jacr.2019.08.008
25. Gallahue FE, Hiller KM, Bird SB, et al. The AAMC standardized 42. Li S, Fant AL, McCarthy DM, Miller D, Craig J, Kontrick A. Gender
video interview: reactions and use by residency programs during differences in language of standardized letter of evaluation narra-
the 2018 application cycle. Acad Med. 2019;94(10):1506-­1512. tives for emergency medicine residency applicants. AEM Educ Train.
doi:https://doi.org/10.1097/ACM.00000​0 0000​0 02714 2017;1(4):334-­339. doi:https://doi.org/10.1002/aet2.10057
26. Murphy B. Fade to Black: Why AAMC Scrapped the Standardized
Video Interview. American Medical Association. Accessed September
16, 2020. https://www.ama-­assn.org/resid​ents-­stude​nts/resid​ency/
How to cite this article: Otugo O, Alvarez A, Brown I, Landry
fade-­black​-­why-­aamc-­scrap​ped-­stand​ardiz​ed-­video​-­inter​view
A. Bias in recruitment: A focus on virtual interviews and holistic
27. Egan D, Husain A, Bond M, et al. Standardized video interviews do
not correlate to united states medical licensing examination step 1 review to advance diversity. AEM Educ Train. 2021;5(Suppl. 1):​
and step 2 scores. West J Emerg Med. 2019;20(1):87-­91. doi:https:// S135–­S139. https://doi.org/10.1002/aet2.10661
doi.org/10.5811/westj​em.2018.11.39730

View publication stats