14765381, 0, Downloaded from https://bpspubs.onlinelibrary.wiley.com/doi/10.1111/bph.16000 by Cochrane Colombia, Wiley Online Library on [18/01/2023].

See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Received: 25 August 2022 Revised: 27 October 2022 Accepted: 29 November 2022
DOI: 10.1111/bph.16000

RESEARCH ARTICLE

Identifying the core concepts of pharmacology education: A

global initiative

Paul J. White 1 | Clare Guilding 2 | Tom Angelo 3 | John P. Kelly 4 |

Laurel Gorman 5 | Steven J. Tucker 6 | Ashleigh Fun 1 | Jae Han 1 |
Guanliang Chen 7 | Yassmin Samak 1 | Anna-Marie Babey 8 |
Fabiana A. Caetano 9 | Sudhir Chandra Sarangi 10 | Jennifer Koenig 11 |
Haiping Hao 12 | Joseph Goldfarb 13 | Kelly Karpa 14 | Luciene Vieira 15 |
16 17 18
Carolina Restini | Margaret Cunningham | Patrik Aronsson |
19 20
Roisin Kelly-Laubscher | Mark Hernandez | Patangi K. Rangachari 21 |
Janet Mifsud 22 | Fatima Mraiche 23 | Ramzi Sabra 24 | Octavio Piñeros 25 |
Xuechu Zhen 26 | Helen Kwanashie 27 | Betty Exintaris 1 | Nilushi Karunaratne 1 |
Kuniaki Ishii 28 | Yannee Liu 1

Correspondence
Paul J. White, Faculty of Pharmacy and Background and Purpose: In recent decades, a focus on the most critical and funda-
Pharmaceutical Sciences, Monash University,
mental concepts has proven highly advantageous to students and educators in many
Parkville, VIC 3052, Australia.
Email: [email protected] science disciplines. Pharmacology, unlike microbiology, biochemistry, or physiology,
lacks a consensus list of such core concepts.
Experimental Approach: We sought to develop a research-based, globally relevant
list of core concepts that all students completing a foundational pharmacology course
should master. This two-part project consisted of exploratory and refinement phases.
The exploratory phase involved empirical data mining of the introductory sections of
five key textbooks, in parallel with an online survey of over 200 pharmacology educa-
tors from 17 countries across six continents. The refinement phase involved three
Delphi rounds involving 24 experts from 15 countries across six continents.
Key Results: The exploratory phase resulted in a consolidated list of 74 candidate
core concepts. In the refinement phase, the expert group produced a consensus list
of 25 core concepts of pharmacology.
Conclusion and Implications: This list will allow pharmacology educators everywhere
to focus their efforts on the conceptual knowledge perceived to matter most by
experts within the discipline. Next steps for this project include defining and

Abbreviations: PD, pharmacodynamic; PK, pharmacokinetic; TF-IDF, term frequency–inverse document frequency.

For affiliations refer to page 11

This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium,
provided the original work is properly cited.
© 2022 The Authors. British Journal of Pharmacology published by John Wiley & Sons Ltd on behalf of British Pharmacological Society.

Br J Pharmacol. 2022;1–13. wileyonlinelibrary.com/journal/bph 1

 14765381, 0, Downloaded from https://bpspubs.onlinelibrary.wiley.com/doi/10.1111/bph.16000 by Cochrane Colombia, Wiley Online Library on [18/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
2 WHITE ET AL.

unpacking each core concept and developing resources to help pharmacology educa-
tors globally teach and assess these concepts within their educational contexts.

KEYWORDS
core concepts, Delphi method, discipline-based educational research, global education,
pharmacology education, postgraduate, text mining, undergraduate

1 | I N T RO DU CT I O N
What is already known
1.1 | What are core concepts?
• Identification of the core concepts of disciplines have
Core concepts are big, important, fundamental ideas, which experts helped transform teaching and assessing student
agree are critical for all students in their discipline to learn, remember, understanding.
understand, and apply—in other words, to learn deeply. Core concepts
in other disciplines include ideas such as gravity in physics (Hestenes
et al., 1992) and homeostasis in physiology (Michael et al., 2017)—these What does this study add
concepts must be learnt and successfully applied by anyone claiming to
understand these disciplines. Over the past 30 years, disciplines such as • This study identifies 25 core concepts that can be applied
physics (Hestenes et al., 1992), statistics (Allen et al., 2004), information within all pharmacology education contexts.
technology (Porter et al., 2019), psychology (Landrum, 1993;
Zechmeister & Zechmeister, 2000), physiology (McFarland et al., 2017;
Michael et al., 2017), and microbiology (Marbach-Ad et al., 2009; What is the clinical significance
Merkel, 2012) have developed research-based lists of core concepts
and related assessments of concept attainment. • Pharmacology education can now focus on ensuring that
In biology, for example, a large, coordinated approach led to the graduates develop and apply the critical concepts.
development of a set of core concepts in the early 2000s. The US
National Science Foundation and American Association for the
Advancement of Science brought together many educators to
produce five core concepts of biology within a Vision and Change Mani-
festo (Brewer & Smith, 2011). Subsequent work led to the development contributed to the decreased time dedicated to teaching pharmacol-
of resources for biology educators to incorporate the teaching and ogy concepts. No pharmacology program, however well-resourced,
assessment of these core concepts into their curricula (Brownell has sufficient time to teach students all the knowledge in the disci-
et al., 2014). Subdisciplines within biology, including physiology and pline. While the sheer volume of the “potential curriculum” increases
microbiology (Hott et al., 2002; Marbach-Ad et al., 2009; Merkel, 2012), exponentially each year, available time for teaching continues to
have since identified further, more specific core concepts. decline overall (Lloyd et al., 2013). This is obvious in integrated health
professions education, where economic and administrative consider-
ations, medical education reforms shortening the preclinical years, and
1.2 | Why identify core concepts in pharmacology? a focus on competency-based models can all minimise time for teach-
ing foundational sciences, including pharmacology.
Pharmacology, defined as the science of drugs or medicines and their Studies of health professional graduates report perceived gaps in
interactions with biological systems, integrates knowledge from a pharmacology knowledge (Bullock & Leversha, 2019). Manias and
number of disciplines, including, but not limited to, physiology, pathol- Bullock (2002) conducted six focus groups with clinical nurses in
ogy, chemistry, biochemistry, structural biology, and mathematics. Australia and found that “all nurses experienced difficulties in under-
Pharmacology is taught across the breadth of health professional, bio- standing and demonstrating pharmacological concepts in the clinical
medical and basic science contexts, and draws upon basic science, practice setting.”
clinical pharmacology, and therapeutics concepts. It is taught at The authors of the well-cited investigation into prescriber error
undergraduate and graduate levels and via a range of instructional for the General Medical Council (London), known as the EQUIP study,
modes (Rubaiy, 2021). In these various programs, the enormous vol- argued “More could have been done during undergraduate education
ume of pharmacology content is often afforded limited time within to link theory with practice” (Dornan et al., 2009). In order to ensure
curricula. In recent years, the creation of integrated courses and the educational effectiveness and patient safety, we contend that it is
merging of physiology and pharmacology departments has essential to first identify the foundational concepts of pharmacology
 14765381, 0, Downloaded from https://bpspubs.onlinelibrary.wiley.com/doi/10.1111/bph.16000 by Cochrane Colombia, Wiley Online Library on [18/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
WHITE ET AL. 3

students require in order to provide them with the ability to link the- educators, of whom 23 engaged in follow-up workshop activities. Sub-
ory to practice. sequently, an expert group of 12 academics extracted a set of core con-
The COVID-19 pandemic has highlighted the need for a skilled cepts, which were then refined and confirmed by a survey of an
workforce across a range of industries to meet the healthcare require- additional 30 academics. The resulting core concepts were then
ments of our global communities. From the discovery of new medicines defined and unpacked (Santiago et al., 2021). This project provided
to safe and effective prescribing, our scientists and health professionals proof-of-concept that core concepts could be identified in pharmacol-
require the ability to apply these enduring ideas years after graduation. ogy, suggesting that such an approach might work at a broader interna-
In addition to providing consensus on the critical knowledge all tional level. Building on the experience and findings of this Australasian
students studying pharmacology need, core concepts provide a range approach, the authors were confident that a more international
of other benefits. Assessments that test their attainment, known as approach would have the potential to be impactful on a global scale.
concept inventories, provide students and educators with the tools to
measure their progress on the “knowledge that matters.” With such
tools, educators can compare the effectiveness of innovative paeda- 1.5 | Overview of a multi-step, international
gogical methods, and administrators can rigorously and reproducibly approach to identify the most important concepts
compare program learning (Sands et al., 2018). within a discipline

In late 2021, an international group of pharmacology education

1.3 | How have core concepts been identified in leaders committed to identifying current core concepts of pharmacol-
other disciplines? ogy education and to developing resources to help educators around
the world teach and assess those concepts. That initiative, described
Over the past 30 years, a range of approaches have been developed in detail below, set out to answer the following research questions.
to identify core concepts, mostly employing groups of disciplinary
experts to identify the critical ideas that all students need to master. • Which ideas do pharmacology educators across many countries
Most disciplines have used a Delphi method, involving a group of believe to be the core concepts of our discipline?
experts completing cycles of surveying and refinement until consen- • What are the most common conceptual terms used in pharmacol-
sus is reached (Boneau, 1990; Brownell et al., 2014; Landrum, 1993; ogy texts?
Merkel, 2012; Parekh et al., 2017; Wright & Hamilton, 2008). Core • Which of the most common terms from pharmacology texts do
concepts have also been extracted from textbooks, either via page- educators believe to be core concepts?
by-page expert analysis (Zechmeister & Zechmeister, 2000) or via • To what extent do lists of potential core concepts derived from
data-mining techniques (Foster et al., 2012). surveying educators agree with those produced via text mining?

1.4 | Pilot project to identify pharmacology core 2 | METHODS

concepts in Australasia
A two-phase approach was used to identify the core concepts of
Beginning in 2019, pharmacology educators based in Australia and pharmacology education (Figure 1). We used multiple approaches to
New Zealand engaged in research that identified 20 core concepts of ensure coverage of the entire field of pharmacology and a variety of
pharmacology education (White et al., 2021). This group, including sources of information to mitigate personal biases. The first, explor-
some authors of the present article, initially surveyed 41 Australasian atory phase included an empirical approach, specifically data mining of

F I G U R E 1 Summary of the two-phase approach used to identify the core concepts of pharmacology education. An n-gram is a contiguous
sequence of n items from a given sample of text or speech.
 14765381, 0, Downloaded from https://bpspubs.onlinelibrary.wiley.com/doi/10.1111/bph.16000 by Cochrane Colombia, Wiley Online Library on [18/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
4 WHITE ET AL.

five key textbook introductory sections. In parallel, an online survey 3 and 7—core concepts that are foundational for phar-
of pharmacology educators was used to complement the text mining, macology students in the text box below. Feel free to
and the terms identified using these two methods were merged to write as much or as little as you wish about your core
develop a list of 74 candidate terms. concepts. Your draft concepts should be big ideas that
The second, refinement phase involved the expert group of 24 in are useful to solve problems and enduring, and they
three rounds of a Delphi Method through which they analysed the should not be topics or facts.
candidate terms and produced a consensus list of 25 core concepts of
pharmacology education.
2.1.3 | Demographics

2.1 | Survey of pharmacology educators from The survey was distributed online via national pharmacology societies
17 countries and networks of pharmacology educators. A total of 201 pharmacol-
ogy educators voluntarily responded to the survey, resulting in
2.1.1 | Ethics 163 complete responses (81% response rate). Respondents were from
a total of 17 countries across six continents (Table 1). Almost two-
The survey was conducted under approved protocol #22727 of the thirds of the respondents identified as female (66%) and 32% identi-
Monash University Ethics in Human Research Committee. The data fied as male, with 1% preferring not to identify. The majority (73%) of
and statistical analysis comply with the recommendations of the Brit- respondents had been teaching pharmacology for more than 10 years,
ish Journal of Pharmacology on experimental design and analysis in while 18% had 3–10 years' experience and 9% had less than 3 years'
pharmacology. experience. Eighty-two percent recorded a PhD as their highest aca-
demic qualification, with 11% reporting a Masters qualification and
6% a combined MD/PhD or PharmD. The respondents taught a range
2.1.2 | Survey development of health professional and science students, and the disciplines taught
similarly encompassed the range of basic and clinical sciences
The research team developed an online survey to gather potential (Table 1). Most respondents taught students from more than one dis-
core concepts from a wide range of international pharmacology edu- ciplinary or professional cohort.
cators. The survey was based on that used for the pilot study (White
et al., 2021; see survey in Data S1). The survey consisted of three
parts: (i) theoretical information about core concepts; (ii) a series of 2.1.4 | Text mining of key pharmacology textbooks
demographic questions; and (iii) the key prompt and question to elicit
potential core concepts, shown below. The introductory sections of each of five key textbooks, selected by
the research team as the most commonly used texts in their local con-
Imagine your current/recent pharmacology students text, were mined to extract the pharmacology terms that were most
3 to 5 years after their graduation: What few essential commonly used. The relevant texts and sections were:
core concepts would you expect them to remember,
understand deeply, and apply effectively in their pro- 1. Rang & Dale's Pharmacology, 8th Edition Section I: General Princi-
fessional work? Please list a few—ideally between ples ISBN 978-0702053627

TABLE 1 Demographic and educator context for the survey respondents

Educator location (number of respondents in the Student cohorts (number of respondents Disciplines taught (number of respondents
location) who teach the cohort) who teach the discipline)
Japan (67) Medical (83) Basic pharmacology (100)
United Kingdom (27) Pharmacy (67) Systems pharmacology (28)
Canada (20) Nursing (39) Clinical Pharmacology (20)
United States (18) Dentistry (27) Pharmacotherapy/clinical management (18)
New Zealand (5) Physiotherapy (9) ADME/pharmacokinetics (8)
Ireland (4) Osteopathy (3) Physiology (8)
Nigeria, Australia (3) Science (27) Microbiology, medicinal chemistry and
epidemiology (1)
Brazil, Sweden, Colombia, Haiti, Hong Kong, Indonesia, Biomedical Science (58)
Qatar, South Africa, Zambia (1)
Post-graduate research (49)
 14765381, 0, Downloaded from https://bpspubs.onlinelibrary.wiley.com/doi/10.1111/bph.16000 by Cochrane Colombia, Wiley Online Library on [18/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
WHITE ET AL. 5

2. Goodman and Gilman's Manual of Pharmacology and Therapeutics, (De Villiers et al., 2005). In order to remove any conflict of interest,
13th Edition Section I: General Principles ISBN 978-0071624428 members of the research team led discussions regarding potential core
3. Katzung's Basic and Clinical Pharmacology, 14th Edition Section I: concepts but were not voting members of the expert group. To select
Basic Principles ISBN 978-1259641152 the expert group, the roles and experience needed were first identified.
4. Golan's Principles of Pharmacology, 4th Edition Section I: Funda- Each member of the research team was asked to invite experienced
mental Principles ISBN 978-1451191004 pharmacology educators to participate. All expert group members
5. Bryant and Knights' Pharmacology for Health Professionals, 4th selected met at least two of the inclusion criteria shown in Table S1.
Edition Unit II: Principles of Pharmacology ISBN 978-0729541701 Consequently, 24 pharmacology educators were recruited by
direct invitation from the research team to be members of the expert
Contents of the introductory chapters were converted to a raw text file group, comprising 14 women and 10 men, from a total of 15 countries
using PyPDF2 as the source texts were in PDF format. Pre-processing across six continents: Australia, Brazil, Canada, China, Colombia,
was conducted using the Natural Language Tool Kit (NLTK). To begin, India, Ireland, Japan, Lebanon, Malta, Nigeria, Qatar, Sweden, the
stop words (e.g., “the” and “an”), punctuations, numbers, tags, and spe- United States, and the United Kingdom. Twenty of the expert
cial characters were removed, and uppercase letters were converted to group members reported their highest qualification as PhD, two
lowercase. Words shorter than two characters were also removed. MBBS/MD, and one PharmD. The group included a number of
Next, additional user-defined keywords such as “chapter” and “section” practising health professionals and scientists from industry and
were excluded from the corpora. Artefacts arising from encoding (e.g., academia. Fourteen members reported a teaching qualification at
“\u02da” and “\n”) and ligatures (e.g., “ff” appearing as “©”) were also Graduate Certificate or higher level. Expert group members' pharma-
addressed. Subsequently, since the majority of the keywords were cology teaching experience ranged from 2 to 60 years, with a median
nouns, words with noun tags were extracted using parts-of-speech of 15 years.
tagging. Finally, all of the extracted words were normalised using stem- The expert group used the Delphi method to identify core con-
ming, which removes suffixes and prefixes from word roots, such that cepts of pharmacology education, conducted under the approved pro-
keywords “drugs” and “drug” would all be considered equal under the tocol #31379 of the Monash University Ethics in Human Research
word root “drug.” Key terms were extracted using scikit-learn's Term Committee. A multi-faceted approach was used, in which the expert
Frequency-Inverse Document Frequency (TF-IDF) Vectorizer with an group were provided with a range of resources to achieve the aim of
N-gram range between 2 and 3. The top 100 terms sorted by TF-IDF ensuring coverage of the entire field; deriving terms from a variety of
score were then selected for further analysis. sources to mitigate personal biases and including empirical data (text
mining) and expert opinion (educator survey).

2.1.5 | Delphi method

2.1.7 | Training of the expert group
The expert group used a criterion-based Delphi method to identify
core concepts of pharmacology education. The following six best- Expert group members attended an initial online workshop in which
practice elements of the Delphi method (see Bhandari & they received information about the project, core concepts, and the
Hallowell, 2021) were incorporated into our study. First, an expert Delphi method. Five criteria were presented to the group as a means
group technique was used to answer the question “What are the core by which to evaluate candidate core concepts for inclusion: funda-
concepts of pharmacology education?” Indirect interaction between mental, useful, enduring, complex and challenging. These criteria were
participants was maintained for Rounds 1 and 2. A predefined thresh- refined from those used in the Australasian study (White et al., 2021).
old was set at 80% agreement for any candidate term to be adopted
as a core concept. The anonymity of voting was maintained for the
expert group throughout the process. Controlled feedback was pro- 2.2 | Three rounds of Delphi method
vided in the form of a summary of voting after Round 2 and 3. Three
iterative rounds of analysis were conducted. In Round 1, expert group members were given the consolidated list of
74 candidate terms, which was produced by merging the outcomes of
text mining of five key textbooks with the survey responses from
2.1.6 | Research team and expert group 163 pharmacology educators. Individually and without consultation,
composition expert group members analysed each term using the five core con-
cepts criteria and then voted as to whether they perceived each term
The research team (PW, TA, CG, JK, and LG) was formed at the in the list to be a core concept of pharmacology education or not.
beginning of the process, and oversaw the Delphi process. The Members were also asked to identify any potential core concepts of
research team aimed to recruit a minimum of 20 expert group mem- pharmacology absent from the list of 74 terms. The terms that
bers based on best practice guidelines for “15 to 30 participants achieved a minimum threshold of 80% agreement by the expert group
from the same discipline” in health sciences education research members were accepted as core concepts and not discussed further.
 14765381, 0, Downloaded from https://bpspubs.onlinelibrary.wiley.com/doi/10.1111/bph.16000 by Cochrane Colombia, Wiley Online Library on [18/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
6 WHITE ET AL.

Terms that were agreed to be core concepts by less than 50% of and pharmacodynamics [PK]; JK and LG) separated the terms into PK
expert group members in Round 1 were rejected and not discussed and PD groupings (Table 2).
further. In Round 2, the terms deemed missing from the candidate
list within Round 1 were analysed and voted on using the same
criteria. 3 | RE SU LT S
Additional online workshops were held between Rounds 2 and
3. The intention of these workshops was to refine the list of core con- The exploratory phase of the project was designed to provide the
cepts, by clarifying areas of duplication, overlap, and hierarchical discon- expert group with a comprehensive list of proposed core concepts,
nect. Expert group members attended one of two online sessions into comprised of both the terms most frequently found in the five key
which they self-selected, depending on their area of expertise: textbook chapters and those identified through the survey of
(i) pharmacokinetic (PK, led by CG and ST) core concepts or 163 international educators.
(ii) pharmacodynamic (PD, led by JK and LG) core concepts. In the work-
shops, the co-leads facilitated a discussion on the small groups of related
terms, which we called clusters (for example, margin of safety and thera- 3.1 | Survey free-text responses: Development
peutic index) that had emerged during Rounds 1 and 2. The research and refinement of a list of terms
team recommended a single term from each cluster to be included in
Round 3 voting, and these recommendations were voted on by the Five hundred and ninety individual terms were submitted as core con-
experts prior to the workshop. Discussions within the workshop focused cepts of pharmacology education by 163 respondents who addressed
on recommendations for which there was less than 80% agreement. this survey question. A term was defined as one or more words that
In Round 3, expert group members analysed the terms that capture a single idea. Duplicates were removed by two raters (YS and
received between 50% and 79% agreement in Rounds 1 and 2. The PW), and clusters of words judged to have equivalent or similar mean-
co-leads for the two subgroups (pharmacokinetics [PK]; CG and ST, ings were grouped under the term that best reflected the central

TABLE 2 Purpose, response format, and the number of responses for each of the three rounds of the Delphi method

Round Purpose of survey Response format # of responses

1 Analyse list of terms from exploratory phase using Likert scale (ranging from 0 to 5) 21
criteria
Vote on inclusion of each of 74 terms from Exploratory Vote term to be core concept or other. Other included
phase as core concepts broad topic, simple concept, fact, term not specific to
pharmacology
Propose any terms that were missing from exploratory Free text
phase list
2 Analyse list of terms proposed by expert group during Likert scale 23
Round 1
Vote on inclusion of each term proposed during Round Vote term to be core concept or other. Other included
1 as core concept fact, simple concept, broad topic, no specific meaning
in pharmacology
3 Vote on inclusion of each term that reached 70%–79% Vote term to be core concept of pharmacology 20
agreement during Round 1 or Round 2 as core education or not
concept

Note: The number of responses was based on the availability of the 24 expert group members at that stage and that all 24 experts voted at least twice.

TABLE 3 Resolution of repeated terms and synonyms

Terms proposed by respondents Related terms proposed by respondents Representative term used in Delphi method
drug target (8); drug targets, targets, drug target drug-target interaction; the difference between a Drug target
and off-target; target identification; drug classical receptor and other types of drug Drug–receptor interaction
target interaction; drug targeting; molecular targets; target identification; drug/target
drug targets; targets for drug action interactions (especially G-coupled receptors);
approaches in terms of targeting a
physiological system; drug-receptor interaction

Note: The example of terms related to “drug target” illustrates the process used by the research team to choose a single term to represent closely related
synonyms. Respondents proposed 16 terms as core concepts that were identical or close synonyms of “drug target” and a further six terms related to this
term. The research team consolidated these 24 terms to two terms for inclusion in the Delphi process: “drug target” itself and “drug–receptor interaction.”
 14765381, 0, Downloaded from https://bpspubs.onlinelibrary.wiley.com/doi/10.1111/bph.16000 by Cochrane Colombia, Wiley Online Library on [18/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
WHITE ET AL. 7

meaning (see Table 3 for an example). The final list of 48 individual Thirteen such “clusters” were identified, and the research team
terms is shown in Table 4. came to agreement on a single term that met the criteria to be a core
concept and best represented each cluster. Seven of the 13 recom-
mendations were endorsed by 80% of the expert group. In the two
3.2 | Text mining: Generation of a list of terms workshops held between Rounds 2 and 3, single terms were identified
to represent each of the remaining six clusters in the final round.
The 100 most frequent terms specific to the discipline were mined
from the corpus produced by combining the five texts. Research team
members (CG, JK, LG, ST, and PW) performed the same consolidation 3.4.3 | Delphi Round 3
process as that used in the previous exploratory phase to yield a list
of 38 terms from the text mining (Table 4). In Round 3, expert group members voted online on each of the
25 terms that had reached the threshold for further consideration in
Rounds 1 and 2, and that had been chosen for inclusion during the
3.3 | Consolidation of survey and text mining consolidation of clusters. Eighteen terms were accepted as core
terms into a single list of terms concepts: 14 outright from Round 3 voting; and the remaining four by
consensus of the research team on the basis that they reached
The 48 terms produced by refinement of the survey responses and 70%–79% agreement in Round 3 and filled an essential gap. A total of
the 38 terms produced by refinement of the text mining terms were 25 terms were therefore accepted as international core concepts of
further refined by the research team, resulting in a final list of 74 pro- pharmacology education (Table 5).
posed core concepts.

3.5 | Comparison of final list of core concepts with

3.4 | Refinement phase initial survey and text mining results

3.4.1 | Delphi Rounds 1 and 2 Given that this is the first time that a combination of text mining and
survey have been used to elicit candidate core concepts, it was of
Of the 74 terms considered by the expert group members in Round interest to analyse the source of the final list of 25 terms. Nineteen of
1, seven were automatically adopted as core concepts, having reached the 25 core concepts (76%) were exact matches to terms in either the
the voting threshold (80% agreement), 35 terms were flagged for fur- text mining refined list or the survey refined list, and eight were pre-
ther discussion (50–79%), and 32 terms were automatically rejected sent in both. A further eight core concepts were present in the survey
(<50%). Note that two of the seven terms accepted during list but not the text mining list and three core concepts were present
Round 1 were later merged—concentration–response relationships in the text mining list but not the survey list (Data S1). Six terms had
and dose–response curve—to form Dose/concentration response no exact match in either text mining or survey list and were derived
relationships. During Round 1, expert group members collectively from the merging of synonyms during the workshops between
proposed an additional 97 terms that they felt were missing from Rounds 2 and 3.
the Round 1 list of terms. In Round 2, 23 terms were flagged for
further discussion, and the remaining 74 terms were rejected. As the
total number of responses for the Delphi rounds varied from 4 | DI SCU SSION
20 to 23, the 80% threshold for acceptance varied from 16 to
18 respondents. Our project has significance for the international pharmacology com-
munity, in that a consensus list of concepts central to the teaching and
learning of our discipline was produced. This is the first time to our
3.4.2 | Consolidation of “clusters” of related terms knowledge that a truly international initiative has produced a consen-
prior to Round 3 sus list of f-focused core concepts. Over 200 pharmacology educators
from 22 countries across six continents contributed to the effort: 201
Prior to Round 3, the research team worked with the expert group to responded anonymously to the survey; and there were 24 expert
consolidate a subset of some closely related terms that had reached group members and 6 research team members. Given the range and
the threshold for further discussion during Rounds 1 and 2. For exam- complexity of content and context, we employed a phased approach
ple, structure–activity relationship and structure–function relationship to the identification of core concepts, building on the lessons from
had both reached the threshold for discussion, and a single term to other disciplines (Hott et al., 2002; Merkel, 2012; Michael et al., 2017;
represent them was required to avoid the inclusion of close synonyms Parekh et al., 2017; Tweedie et al., 2020; Zechmeister &
in the final round of the Delphi method. Zechmeister, 2000) and our own pilot studies in Australia and
 14765381, 0, Downloaded from https://bpspubs.onlinelibrary.wiley.com/doi/10.1111/bph.16000 by Cochrane Colombia, Wiley Online Library on [18/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
8 WHITE ET AL.

TABLE 4 Synthesis of text mining and survey terms

Merged list
Text mining Survey
(refined list of 38) (refined list of 48) (74 terms)

Agonist ADME ADME Drug receptor

Amount of drug Agonists/antagonists Agonist Drug response
Concentration of drug Allosteric drugs Agonists/antagonists Drug safety
Dose response Bioavailability Allosteric drugs Drug selectivity/specificity
Dose–response curve Biologics Amount of drug Drug target
Drug absorption Common systemic pharmacology Bioavailability Drug therapy
Drug action Compartment models Biologics Drug tolerance
Drug administration Competitive/non-competitive inhibition Compartment models Drug–receptor interaction
Drug binding Concentration–response relationship Competitive/non-competitive inhibition Drugs
Drug bioavailability Dose regimens Concentration of drug Duration of action
Drug clearance Drug absorption Concentration–response relationship ED50
Drug compartment Drug action Dose regimens First-order kinetics
Drug concentration Drug affinity Dose–response curve First pass effect
Drug distribution Drug design Drug absorption Individual variation
Drug dose Drug development Drug action Integrative pharmacology
Drug effect Drug discovery Drug administration Ion channel
Drug effect Drug distribution Drug affinity LD50
Drug elimination Drug efficacy Drug binding Lead optimization
Drug interaction Drug elimination Drug bioavailability Mechanism of drug action
Drug mechanism of action Drug excretion Drug clearance Molecular pharmacology
Drug metabolism Drug half life Drug compartment Pharmacodynamics
Drug molecule Drug interactions Drug concentration Pharmacogenomics
Drug plasma concentration Drug metabolism Drug design Pharmacokinetic calculations
Drug receptor Drug potency Drug development Pharmacokinetics
Drug response Drug safety Drug discovery PKPD
Drug–receptor interaction Drug selectivity/specificity Drug distribution Plasma protein binding
Duration of action Drug target Drug dose Rate of drug elimination
First-order kinetics Drug therapy Drug effect Receptor antagonist
First pass effect Drug tolerance Drug efficacy Receptor type
Ion channel Drugs Drug elimination Routes of administration
Mechanism of drug action ED50 Drug excretion Signal transduction
Plasma protein binding Individual variation Drug half life Side effect
Rate of drug elimination Integrative pharmacology Drug interaction Structure–function relationship
Rate of drug elimination LD50 Drug mechanism of action Systems pharmacology
Receptor antagonist Lead optimization Drug metabolism Therapeutic window
Receptor type Mechanism of drug action Drug molecule Volume of distribution
Side effect Molecular pharmacology Drug plasma concentration
Volume of distribution Pharmacodynamics Drug potency
Pharmacogenomics
Pharmacokinetic calculations
Pharmacokinetics
PKPD Accepted Round 1
Receptors For Discussion Round 1
Routes of administration Rejected Round 1
Signal transduction
 14765381, 0, Downloaded from https://bpspubs.onlinelibrary.wiley.com/doi/10.1111/bph.16000 by Cochrane Colombia, Wiley Online Library on [18/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
WHITE ET AL. 9

TABLE 4 (Continued)

Merged list
Text mining Survey
(refined list of 38) (refined list of 48) (74 terms)

Side effects
Structure–function relationship
Therapeutic window

Note: A single combined list of terms produced by consolidation of the text mining and educator survey terms. Shading on the merged list shows the
outcome of Delphi Round 1 for each of the terms: green shading indicates Core Concepts with 80+% agreement, yellow shading indicates 50%–79%
agreement and brown shading indicates less than 50% agreement.
Abbreviations: ADME, absorption, distribution, metabolism, and excretion; ED50, effective dose in 50% of animals or participants; LD50, lethal dose
required to kill 50% of animals in the study; PKPD, pharmacokinetic/pharmacodynamic modelling.

TABLE 5 The terms accepted as core concepts of pharmacology pharmacology texts and 590 terms from large-scale survey. The
education research team then analysed and refined these large lists to create a
Round agreement manageable 74 potential core concepts for the expert group to
Core concept threshold reached % agreement resolve. To ensure genuine accord within the Delphi method, we set a
Drug elimination 1 90 high (80%) acceptance threshold, providing confidence in the final list
Dose/concentration–response 1 86 of 25 core concepts. For comparison, a median acceptance threshold
relationship of 75% was reported in a review of nursing Delphi studies (Foth et al.,
Drug bioavailability 1 81 2016).
Drug distribution 1 81
Drug tolerance 1 81
Drug metabolism 1 81 4.1 | The Delphi method ensured early
Drug half-life 3 100
contributions were made without the influence of
other group members
Drug absorption 3 95
Drug potency 3 95
The three rounds of Delphi method were critical to the development
Drug efficacy 3 95
of the final list. In particular, the comments from the experts in Round
Volume of distribution 3 90
2 and associated workshops were invaluable in dealing with the chal-
Steady state concentration 3 90 lenge of choosing a single core concept from a cluster of related
Mechanism of drug action 3 90 terms. For example, numerous expert comments were received
Agonists and antagonists 3 90 regarding the related terms margin of safety, therapeutic window, and
Adverse drug reaction 3 90 therapeutic index. These comments informed the final decision to pro-
Therapeutic index 3 90 pose therapeutic index in Round 3 and subsequent 90% endorsement
Drug affinity 3 85 of this concept. This is consistent with earlier findings supporting the
Drug selectivity 3 85 use of three rounds of expert analysis (Hallowell & Gambatese, 2010).

Drug clearance 3 80 An innovative part of our Delphi method was the use of workshops
between Rounds 2 and 3, with workshop discussions useful in shaping
Drug target 3 80
the clustering of terms around a central concept. The workshops also
Drug interaction 3 80
permitted discussion and input from a wider team outside of the core
Zero- and first-order kinetics 3a 75
research team. That said, the workshops may have affected the inde-
Drug–receptor interaction 3a 75
pendent nature of the expert ratings on the final Delphi round, so that
Individual variation in drug 3a 75
only the expert group ratings in the first two rounds were truly
response
independent.
Structure–activity relationship 3a 70
a
Between 70% and 80% agreement within the final Delphi round and
deemed by the research team to fill an essential gap in the overall list of
4.2 | Comparison to the ASCEPT list
concepts.

As the current study builds upon the foundation initiated by the

New Zealand (Santiago et al., 2021; White et al., 2021). The explor- Australasian pilot (White et al., 2021), it is of interest to compare and
atory phase ensured that we covered the full scope of concepts in contrast the two lists of core concepts. The pilot study was conducted
pharmacology, by identifying 100 terms used frequently in by a group of educators from Australian and New Zealand, only one
 14765381, 0, Downloaded from https://bpspubs.onlinelibrary.wiley.com/doi/10.1111/bph.16000 by Cochrane Colombia, Wiley Online Library on [18/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
10 WHITE ET AL.

of whom participated in the expert group in the current study. The core concepts work aims to provide focus for educators and program
present final list includes 15 of the 19 core concepts from the pilot directors/chairs to assure both students and stakeholders that
study, namely, drug absorption, concentration response relationships, graduates have acquired the fundamental knowledge necessary to be
drug distribution, drug target, drug metabolism, mechanism of drug successful in their careers.
action, drug efficacy, drug selectivity, drug affinity, drug tolerance, bio-
availability, drug potency, therapeutic window, drug elimination, and indi-
vidual variation. 4.4 | Limitations
Our current list does not include the pilot study core concepts
drugs and homeostasis, drug excretion, drug safety, and drugs and com- The total number of pharmacology educators around the world has not
plex systems. Our current list does however include 10 core concepts been rigorously determined, but is likely to be many thousands. Our
that were not included in the pilot study: drug half-life; volume of distri- study involved educators from every continent, but we would need a
bution; steady state concentration; agonists and antagonists; drug clear- far larger, more representative sample size to claim international con-
ance; drug interactions; zero and first order kinetics; drug–receptor sensus on our list of core concepts. Additionally, volunteer bias could
interaction; structure–activity relationship; adverse drug reaction. The have contributed to the outcomes of this study. These concerns are
observed differences in the lists may be linked to the different meth- offset by the high degree of similarity between the list of core con-
odological approaches taken, or the differences between Australasian cepts identified by rigorous, multi-stage methodology in this study with
and international expert views. Interestingly, the terms volume of dis- that of the pilot work in Australia and New Zealand. Perceptions held
tribution and drug–receptor interactions were both derived in our study by individuals regarding the core concepts for any discipline are likely
from the text mining process, and reached the consensus threshold in to be heavily influenced by their expertise, background, beliefs and
our study. The Australasian study did not involve text mining. experiences. While we did this work from the perspective of pharma-
cology education, our various health professional, industry and basic
science research backgrounds and current roles influenced our think-
4.3 | Comparison to other international resources ing. Future work specifically focusing on the core concepts of pharma-
cology utilised in industry, research and clinical practice will assist
Many disciplines have identified core concepts and developed con- educators to contextualise their concept-based teaching approach.
cept inventories over the past 30 years, with physics and biology Finally, the poor response rate to the survey from some countries
notable for the scope and depth of work. Hestenes and colleagues' and the overrepresentation of others has the potential to bias the
seminal work to develop the force concept inventory (Halloun & results towards the views of educators in those over-represented coun-
Hestenes, 1985; Hestenes et al., 1992) heralded a transformation of tries. Nevertheless, the use of the text mining approach and the high
learning across the discipline, with the rigour and reproducibility of degree of similarity between the pilot study and the present work both
the inventory enabling the identification of effective, evidence-based provide confidence that the list of core concepts we have developed
teaching practices (Deslauriers & Wieman, 2011; Hake, 1998). In the are broadly representative of the views of experts in the discipline.
United States, in biology, “Vision and Change” was a National Science Finally, we note that the list of core concepts generated in our
Association funded endeavour to develop core concepts and related work are foundational concepts. The majority of the concepts are
tools for educators (Brewer & Smith, 2011; Brownell et al., 2014). The relevant to drugs of all categories, including small molecule drugs,
current project aimed to follow in the footsteps of these seminal ini- biologics, nucleic acid medicines and emerging modalities. For exam-
tiatives, with the goal of providing educators with truly global pharma- ple, selectivity applies not only to drugs that target receptors and
cology core concepts, concept inventories, and related resources. enzymes, but also to antibodies and other more recent additions to
Of note, there are already a number of valuable resources our therapeutic arsenal. Some concepts are specific to sub-categories
for pharmacology educators. The Pharmacology Education Project of drugs, for example the efficacy that agonists display when bound
(Faccenda et al., 2019), an initiative of the International Union of Basic to receptors.
and Clinical Pharmacology, is a web-based resource with a wide range We are currently working to unpack each core concept to identify
of information, teaching, learning, and assessment tools. The underpinning sub-concepts. These sub-concepts will shed further light
“Knowledge Objectives in Medical Pharmacology” initiative spon- on the concepts that apply to all drugs, regardless of target, and those
sored by the Association of Medical School Pharmacology Chairs that are specific to sub-categories of drug classes or targets.
(AMSPC) has provided medical educators with extensive guidance on
their pharmacology curricula since 1985. The British Pharmacological
Society has several very extensive and paedagogically advanced core 4.5 | Future work
curricula and has published work on clinical pharmacology for medical
students (Ross & Maxwell, 2012) and undergraduate pharmacology The list of core concepts that we have produced in this study is a
(Wallace et al., 2021). These invaluable resources complement our new, evidence-based resource that will be of interest and use to
work in that they provide a breadth of material, allowing educators to pharmacology educators globally. Realising the full potential of this
ensure adequate coverage in their programs and courses, whereas our initiative will require resources to help educators around the world
 14765381, 0, Downloaded from https://bpspubs.onlinelibrary.wiley.com/doi/10.1111/bph.16000 by Cochrane Colombia, Wiley Online Library on [18/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
WHITE ET AL. 11

develop concept-based curricula, including definitions of each core Kwanashie: Investigation; methodology; writing-review and
concept, identification of underpinning sub-concepts, development of editing. Betty Exintaris: Investigation; methodology; writing-review
teaching methods and learning resources to embed attainment of and editing. Nilushi Karunaratne: Investigation; methodology;
these concepts into courses. Our global initiative will produce and dis- writing-review and editing. Kuniaki Ishii: Formal analysis;
seminate these resources, and more broadly initiate a conversation investigation; methodology. Yannee Liu: Data curation; formal
about concept-driven pharmacology education. Finally, we will con- analysis.
duct research to identify misconceptions held by students on each
core concept. The outcomes of this work will underpin development AF FILIATI ON S
1
of a concept inventory, a validated assessment of learner attainment Faculty of Pharmacy and Pharmaceutical Sciences, Monash
of each concept. University, Parkville, Victoria, Australia
2
School of Medicine, Faculty of Medical Sciences, Newcastle Upon
ACKNOWLEDGEMEN TS Tyne, UK
3
The authors acknowledge the contribution of the survey participants UNC Eshelman School of Pharmacy, University of North Carolina,
and expert group members who contributed their expertise to the Chapel Hill, North Carolina, USA
4
study. Open access publishing facilitated by Monash University, as Pharmacology and Therapeutics, School of Medicine, National
part of the Wiley - Monash University agreement via the Council of University of Ireland Galway, Galway, Ireland
5
Australian University Librarians. Department Medical Education, University of Central Florida College
of Medicine, Orlando, Florida, USA
6
AUTHOR CONTRIBUTIONS School of Medicine, Medical Science and Nutrition, University of
Paul White: Conceptualization; data curation; formal analysis; meth- Aberdeen, Aberdeen, UK
7
odology; project administration; visualization; writing-original draft; Centre for Learning Analytics at Monash, Faculty of Information
writing-review and editing. Clare Guilding: Conceptualization; formal Technology, Monash University, Clayton, Victoria, Australia
8
analysis; methodology; writing-review and editing. Tom Angelo: Faculty of Medicine and Health, University of New England,
Conceptualization; formal analysis; investigation; methodology; Armidale, New South Wales, Australia
9
visualization; writing-review and editing. John P. Kelly: Formal Department of Physiology and Pharmacology, Western University,
analysis; investigation; methodology; writing-review and editing. London, Ontario, Canada
10
Laurel Gorman: Formal analysis; investigation; methodology; writing- Department of Pharmacology, All India Institute of Medical
review and editing. Steven J. Tucker: Formal analysis; investigation; Sciences, New Delhi, India
11
methodology; writing-review and editing. Ashleigh Fun: Investigation; Division of Medical Sciences and Graduate Entry Medicine, School
methodology. Jae Han: Investigation; methodology; writing-review of Medicine, University of Nottingham, Nottingham, UK
12
and editing. Guanliang Chen: Investigation; methodology. Yassmin Department of Medicinal Chemistry, School of Pharmacy, China
Samak: Formal analysis; investigation; methodology. Anna-Marie Pharmaceutical University, Nanjing, China
13
Babey: Investigation; methodology; writing-review and editing. Department of Pharmacological Sciences and Institute for Systems
Fabiana A Caetano: Investigation; methodology; writing-review and Biomedicine, Icahn School of Medicine at Mount Sinai, New York,
editing. Sudhir Chandra Sarangi: Investigation; methodology; writing- New York, USA
14
review and editing. Jennifer Keonig: Investigation; methodology; Department of Medical Education and Family Medicine, East
writing-review and editing. Haiping Hao: Investigation; methodology; Tennessee State University, Johnson City, Tennessee, USA
15
writing-review and editing. Joseph Goldfarb: Investigation; methodol- Departamento de Farmacologia, Universidade Federal de Minas
ogy; writing-review and editing. Kelly Karpa: Investigation; methodol- Gerais, Belo Horizonte, Brazil
16
ogy; writing-review and editing. Luciene Vieira: Investigation; Pharmacology and Toxicology Department, College of Osteopathic
methodology; writing-review and editing. Carolina Restini: Investiga- Medicine, Michigan State University, Clinton Township, Michigan,
tion; methodology; writing-review and editing. Margaret Cunningham: USA
17
Investigation; methodology; writing-review and editing. Patrik Strathclyde Institute of Pharmacy and Biomedical Sciences (SIPBS),
Aronsson: Investigation; methodology; writing-review and editing. University of Strathclyde, Glasgow, UK
18
Roisin Kelly-Laubscher: Investigation; methodology; writing-review Department of Pharmacology, Institute of Neuroscience and
and editing. Mark Hernandez: Investigation; methodology; writing- Physiology, Sahlgrenska Academy, University of Gothenburg,
review and editing. Patangi K. Rangachari: Investigation; methodol- Gothenburg, Sweden
19
ogy; writing-review and editing. Janet Mifsud: Investigation; method- Department of Pharmacology and Therapeutics, College of
ology; writing-review and editing. Fatima Mraiche: Investigation; Medicine and Health, University College Cork, Cork, Ireland
20
methodology; writing-review and editing. Ramzi Sabra: Investigation; Division of Anatomy and Molecular Medicine, Alabama College of
methodology; writing-review and editing. Octavio Piñeros: Osteopathic Medicine, Dothan, Alabama, USA
21
Investigation; methodology; writing-review and editing. Xuechu Zhen: Department of Medicine, Faculty of Health Sciences, McMaster
Investigation; methodology; writing-review and editing. Helen University, Hamilton, Ontario, Canada
 14765381, 0, Downloaded from https://bpspubs.onlinelibrary.wiley.com/doi/10.1111/bph.16000 by Cochrane Colombia, Wiley Online Library on [18/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
12 WHITE ET AL.

22
Department of Clinical Pharmacology and Therapeutics, Faculty of De Villiers, M. R., De, P. J., & Kent, A. P. (2005). The Delphi technique in
Medicine and Surgery, University of Malta, Msida, Malta health sciences education research. Medical Teacher, 27(7), 639–643.
23 https://doi.org/10.1080/13611260500069947
College of Pharmacy, QU-Health, Qatar University, Doha, Qatar
Deslauriers, L., & Wieman, C. (2011). Learning and retention of quantum
24
Faculty of Medicine, American University of Beirut, Beirut, Lebanon concepts with different teaching methods. Physical Review Special
25
Department of Family Medicine, School of Medicine, Faculty of Topics-Physics Education Research, 7(1), 010101. https://doi.org/10.
Health, Universidad del Valle, Cali, Colombia 1103/PhysRevSTPER.7.010101
26 Dornan, T., Ashcroft, D., Heathfield, H., Lewis, P., Miles, J., Taylor, D., &
Jiangsu Key Laboratory of Neuropsychiatric Diseases Research and
Wass, V. (2009). An in-depth investigation into causes of prescribing
College of Pharmaceutical Sciences, Soochow University, Soochow, errors by foundation trainees in relation to their medical education: EQUIP
China study (pp. 1–215). General Medical Council.
27 Faccenda, E., Maxwell, S., & Szarek, J. L. (2019). The IUPHAR pharmacol-
Department of Nursing Science, Faculty of Health Sciences,
ogy education project. Clinical Pharmacology and Therapeutics, 105(1),
National Open University of Nigeria, Abuja, Nigeria
28
45–48. https://doi.org/10.1002/cpt.1278
Department of Pharmacology, Faculty of Medicine, Yamagata
Foster, J. M., Sultan, M. A., Devaul, H., Okoye, I., & Sumner, T. (2012).
University, Yamagata, Japan Identifying core concepts in educational resources. Paper presented at
the Proceedings of the 12th ACM/IEEE-CS joint conference on Digital
DECLARATION OF TRAN SPARENCY AND S CIENTI FIC Libraries.
Foth, T., Efstathiou, N., Vanderspank-Wright, B., Ufholz, L.-A.,
RI GOUR
Dütthorn, N., Zimansky, M., & Humphrey-Murto, S. (2016). The use of
This Declaration acknowledges that this paper adheres to the princi- Delphi and nominal group technique in nursing education: A review.
ples for transparent reporting and scientific rigour of preclinical International Journal of Nursing Studies, 60, 112–120. https://doi.org/
research as stated in the BJP guidelines for Design and Analysis, and 10.1016/j.ijnurstu.2016.04.015
Hake, R. R. (1998). Interactive-engagement versus traditional methods: A
as recommended by funding agencies, publishers and other organisa-
six-thousand-student survey of mechanics test data for introductory
tions engaged with supporting research. physics courses. American Journal of Physics, 66(1), 64–74. https://doi.
org/10.1119/1.18809
DATA AVAI LAB ILITY S TATEMENT Halloun, I. A., & Hestenes, D. (1985). The initial knowledge state of college
physics students. American Journal of Physics, 53(11), 1043–1055.
The data that support the findings of this study are available from the
https://doi.org/10.1119/1.14030
corresponding author upon reasonable request. Some data may not Hallowell, M. R., & Gambatese, J. A. (2010). Qualitative research: Applica-
be made available because of privacy or ethical restrictions. tion of the Delphi method to CEM research. Journal of Construction
Engineering and Management, 136(1), 99–107. https://doi.org/10.
1061/(ASCE)CO.1943-7862.0000137
ORCID
Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force concept inven-
Steven J. Tucker https://orcid.org/0000-0003-4854-8879 tory. The Physics Teacher, 30(3), 141–158. https://doi.org/10.1119/1.
Anna-Marie Babey https://orcid.org/0000-0003-3454-5204 2343497
Haiping Hao https://orcid.org/0000-0003-2522-7546 Hott, A. M., Huether, C. A., McInerney, J. D., Christianson, C., Fowler, R.,
Bender, H., Jenkins, J., Wysocki, A., Markle, G., & Karp, R. (2002).
Roisin Kelly-Laubscher https://orcid.org/0000-0002-6379-9943
Genetics content in introductory biology courses for non-science
majors: Theory and Practice. BioScience, 52(11), 1024.
RE FE R ENC E S Landrum, R. E. (1993). Identifying core concepts in introductory psychol-
Allen, K., Stone, A., Rhoads, T. R., & Murphy, T. J. (2004). The ogy. Psychological Reports, 72(2), 659–666. https://doi.org/10.2466/
statistics concepts inventory: Developing a valid and reliable instrument. pr0.1993.72.2.659
Paper presented at the Proceedings of the 2004 American Society for Lloyd, H., Hinton, T., Bullock, S., Babey, A.-M., Davis, E., Fernandes, L., &
Engineering Education Annual Conference and Exposition. Ziogas, J. (2013). An evaluation of pharmacology curricula in
Bhandari, S., & Hallowell, M. R. (2021). Identifying and controlling biases in Australian science and health-related degree programs. BMC Medical
expert-opinion research: Guidelines for variations of Delphi, nominal Education, 13(1), 1, 153–15. https://doi.org/10.1186/1472-6920-
group technique, and focus groups. Journal of Management in Engineer- 13-153
ing, 37(3), 04021015. https://doi.org/10.1061/(ASCE)ME.1943-5479. Manias, E., & Bullock, S. (2002). The educational preparation of undergrad-
0000909 uate nursing students in pharmacology: Clinical nurses' perceptions
Boneau, C. A. (1990). Psychological literacy: A first approximation. and experiences of graduate nurses' medication knowledge. Interna-
American Psychologist, 45(7), 891–900. https://doi.org/10.1037/0003- tional Journal of Nursing Studies, 39(8), 773–784. https://doi.org/10.
066X.45.7.891 1016/S0020-7489(02)00008-1
Brewer, C. A., & Smith, D. (2011). Vision and change in undergraduate biol- Marbach-Ad, G., Briken, V., El-Sayed, N. M., Frauwirth, K.,
ogy education: A call to action. American Association for the Advance- Fredericksen, B., Hutcheson, S., & McIver, K. S. (2009). Assessing stu-
ment of Science, Washington, DC, 81. dent understanding of host pathogen interactions using a concept
Brownell, S. E., Freeman, S., Wenderoth, M. P., & Crowe, A. J. (2014). Bio- inventory. Journal of Microbiology & Biology Education: JMBE, 10(1),
Core guide: A tool for interpreting the core concepts of vision and 43–50. https://doi.org/10.1128/jmbe.v10.98
change for biology majors. CBE—Life Sciences Education, 13(2), 200– McFarland, J. L., Price, R. M., Wenderoth, M. P., Martinková, P., Cliff, W.,
211. https://doi.org/10.1187/cbe.13-12-0233 Michael, J., & Wright, A. (2017). Development and validation of the
Bullock, S., & Leversha, A. (2019). Medical student perceptions of their homeostasis concept inventory. CBE—Life Sciences Education, 16(2),
knowledge and skills in pharmacology in their first and final clinical ar35. https://doi.org/10.1187/cbe.16-10-0305
years. MedEdPublish, 8(1), 42. https://doi.org/10.15694/mep.2019. Merkel, S. (2012). The development of curricular guidelines for introduc-
000042.1 tory microbiology that focus on understanding. Journal of
 14765381, 0, Downloaded from https://bpspubs.onlinelibrary.wiley.com/doi/10.1111/bph.16000 by Cochrane Colombia, Wiley Online Library on [18/01/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
WHITE ET AL. 13

Microbiology & Biology Education: JMBE, 13(1), 32–38. https://doi.org/ Wallace, M. J., Zecharia, A., Guilding, C., Tucker, S., & McFadzean, I.
10.1128/jmbe.v13i1.363 (2021). Developing a new undergraduate pharmacology core curricu-
Michael, J., Cliff, W., McFarland, J., Modell, H., & Wright, A. (2017). What lum: The British Pharmacological Society Delphi method. Pharmacology
are the core concepts of physiology? In The core concepts of physiology Research & Perspectives, 9(4), e00832. https://doi.org/10.1002/
(pp. 27–36). Springer. https://doi.org/10.1007/978-1-4939-6909-8_3 prp2.832
Parekh, G., DeLatte, D., Herman, G. L., Oliva, L., Phatak, D., White, P. J., Davis, E. A., Santiago, M., Angelo, T., Shield, A.,
Scheponik, T., & Sherman, A. T. (2017). Identifying core concepts of Babey, A. M., & Hinton, T. (2021). Identifying the core concepts of
cybersecurity: Results of two Delphi processes. IEEE Transactions on pharmacology education. Pharmacology Research & Perspectives, 9(4),
Education, 61(1), 11–20. https://doi.org/10.1109/TE.2017.2715174 e00836. https://doi.org/10.1002/prp2.836
Porter, L., Zingaro, D., Liao, S. N., Taylor, C., Webb, K. C., Lee, C., & Wright, T., & Hamilton, S. (2008). Assessing student understanding in the
Clancy, M. (2019). BDSI: A validated concept inventory for basic data molecular life sciences using a concept inventory. ATN Assessment.
structures. Paper presented at the Proceedings of the 2019 ACM Con- Zechmeister, J. S., & Zechmeister, E. B. (2000). Introductory textbooks and
ference on International Computing Education Research. psychology's core concepts. Teaching of Psychology, 27(1), 6–11.
Ross, S., & Maxwell, S. (2012). Prescribing and the core curriculum for https://doi.org/10.1207/S15328023TOP2701_1
tomorrow's doctors: BPS curriculum in clinical pharmacology and pre-
scribing for medical students. British Journal of Clinical Pharmacology,
74(4), 644–661. https://doi.org/10.1111/j.1365-2125.2012.04186.x SUPPORTING INF ORMATION
Rubaiy, H. N. (2021). Strategies to inspire Students' engagement in phar- Additional supporting information can be found online in the Support-
macology courses. Pharmacy, 9(2), 70. https://doi.org/10.3390/
ing Information section at the end of this article.
pharmacy9020070
Sands, D., Parker, M., Hedgeland, H., Jordan, S., & Galloway, R. (2018).
Using concept inventories to measure understanding. Higher Education
Pedagogies, 3(1), 173–182. https://doi.org/10.1080/23752696.2018. How to cite this article: White, P. J., Guilding, C., Angelo, T.,
1433546 Kelly, J. P., Gorman, L., Tucker, S. J., Fun, A., Han, J., Chen, G.,
Santiago, M., Davis, E. A., Hinton, T., Angelo, T. A., Shield, A., Samak, Y., Babey, A.-M., Caetano, F. A., Sarangi, S. C., Koenig,
Babey, A. M., & Musgrave, I. F. (2021). Defining and unpacking the
J., Hao, H., Goldfarb, J., Karpa, K., Vieira, L., Restini, C., … Liu,
core concepts of pharmacology education. Pharmacology Research &
Perspectives, 9(6), e00894. https://doi.org/10.1002/prp2.894 Y. (2022). Identifying the core concepts of pharmacology
Tweedie, J., Palermo, C., Wright, H. H., & Pelly, F. E. (2020). Using docu- education: A global initiative. British Journal of Pharmacology,
ment analysis to identify core concepts for dietetics: The first step in 1–13. https://doi.org/10.1111/bph.16000
promoting conceptual learning. Nursing & Health Sciences, 22(3),
675–684. https://doi.org/10.1111/nhs.12712