Abstract
Over a 31-year span as Director of the U.S. National Library of Medicine (NLM), Donald A.B. Lindberg, MD, and his extraordinary NLM colleagues fundamentally changed the field of biomedical and health informatics—with a resulting impact on biomedicine that is much broader than its influence on any single subfield. This article provides substance to bolster that claim. The review is based in part on the informatics section of a new book, “Transforming biomedical informatics and health information access: Don Lindberg and the US National Library of Medicine” (IOS Press, forthcoming 2021). After providing insights into selected aspects of the book’s informatics-related contents, the authors discuss the broader context in which Dr. Lindberg and the NLM accomplished their transformative work.
Keywords: US National Library of Medicine, Donald A.B. Lindberg, biomedical informatics, health informatics, review
INTRODUCTION: A REVOLUTION IN BIOMEDICAL AND HEALTH INFORMATICS
What the US National Library of Medicine (NLM) accomplished during the tenure of Donald A.B. Lindberg, MD as its Director amounts to a scientific revolution in biomedical and health informatics. This review supports that claim. It provides insight into and expands upon a new book about Lindberg and the NLM.1–12 In his landmark treatise, The Structure of Scientific Revolutions, T.S. Kuhn explained how paradigm shifts lead to scientific revolutions.13 He stated:
“The transition from a paradigm in crisis to a new one from which a new tradition of normal science can emerge is far from a cumulative process, one achieved by an articulation or extension of the old paradigm. Rather it is a reconstruction of the field from new fundamentals, a reconstruction that changes some of the field’s most elementary theoretical generalizations as well as many of its paradigm methods and applications. During the transition period, there will be a large but never complete overlap between the problems that can be solved by the old and by the new paradigm. But there will also be a decisive difference in the modes of solution. When the transition is complete, the profession will have changed its view of the field, its methods, and its goals.”13 [p. 84]
The foregoing quotation aptly characterizes the dramatic changes from 1984 to 2015 in the mission, roles, and functions of the NLM as an institute within the National Institutes of Health (NIH), and as a primary sponsor of biomedical and health informatics activities in the United States. Kuhn also described by analogy how those changes influenced professionals in our field:
“Led by a new paradigm, scientists adopt new instruments and look in new places. Even more important, during revolutions scientists see new and different things when looking with familiar instruments in places they have looked before. It is rather as if the professional community had been suddenly transported to another planet where familiar objects are seen in a different light and are joined by unfamiliar ones as well.”13 [p. 111]
EVOLUTION OF THE FIELD OF BIOMEDICAL INFORMATICS
New scientific disciplines do not emerge fully formed. They begin with pioneers, often isolated from one another while they work individually to effect substantial, but predominantly local changes. Morris F. Collen’s and Marion J. Ball’s History of Medical Informatics in the United States describes the isolated-silos nature of pre-1984 informatics efforts.14 Many modern clinical practice paradigms were established circa 1900, but computer technology did not become widely available until more than a half-century later. Only in 1962 did Purdue University establish the first US academic department of computer science. Support for the earliest university-based informatics laboratories typically came from internal institutional funds. Later, grants from private industry, for example, the IBM Corporation, and from the NIH Division of Research Resources (DRR) became available. While the DRR (founded in 1969–1970) primarily sponsored acquisition of computing equipment and physical laboratory space for informatics research, it took the NLM under Don Lindberg to produce a coherent single source for leadership, innovation, and funding that transformed the field. In 1990, when DRR merged with the NIH Division of Research Services to create the National Center for Research Resources (NCRR), the NLM, with Lindberg’s support, assumed responsibility for several of the research computing and informatics centers that had previously been created and funded by the DRR.
Early in his career, at the University of Missouri, Lindberg had become one of the influential isolated pioneers. Kingsland and Kulikowski1 have described Dr. Lindberg’s work on laboratory information systems, electrocardiogram (EKG) interpretation, an accessible computer fact bank, and knowledge-based systems. Lindberg and colleagues had to invent devices and equipment on the fly and to develop and deploy new software systems in innovative contexts. He had to recruit talented people from other professions (physics, mathematics, and electrical engineering) to help build systems. Initially, Lindberg had to vie for time on the main university IBM 1410 computer system but later he could use an NIH Regional Medical Program grant to obtain his own IBM 1620 computer. Nevertheless, he was able to construct several historically important systems that became national prototypes.1,15
Historians of science have shown that, as the number of individual pioneers in a nascent field grows, they begin to communicate, meet, and collaborate. At this stage, professional societies begin to form at a national level, and journals dedicated to the field emerge. Lindberg’s involvement in the formation of many early informatics organizations has been described by van Bemmel et al2 and Collen and Ball.14 In 1977, Helmuth F. Orthner, PhD and William Yamamoto, PhD at George Washington University initiated the Symposium on Computer Applications in Medical Care (SCAMC). It was one of the first annually recurring American meetings dedicated solely to clinical informatics. Thomas E. Piemme, MD played a key role in expanding SCAMC from a local to a national-level forum, where isolated members of the new field could meet and exchange ideas. The proceedings of the SCAMC meetings document the early history of American informatics activities. An international informatics community also was under development. In Germany, Gustav Wagner, MD established the first clinical informatics journal, Methods of Information in Medicine, in 1962; Homer R. Warner. MD, PhD established the first such journal in the United States, Computers and Biomedical Research, in 1967.2
Although small informatics-related professional societies were formed in the United States in the late 1960s and early 1970s (notably the Society for Advanced Medical Systems and the Society for Computer Medicine), the first large international organization was not created until 1974. Soon to be known as the International Medical Informatics Association (IMIA), it began a series of triennial Medinfo meetings that were held in different countries. The now biennial IMIA meetings continue to this day. IMIA was created as an organization of national organizations, and Lindberg was selected as the American representative to IMIA in 1983 and served in that role for several years. Meanwhile, the aforementioned early US professional groups merged to create the American Association for Medical Systems and Informatics (AAMSI), which became the dominant membership organization in the United States. In parallel, a group of informatics scientists, including Lindberg, created a college of elected leaders in the field in 1984. Known as the American College of Medical Informatics (ACMI), this organization held its own small meetings and conducted annual elections followed by the induction of new members.2 Lindberg also served on the AAMSI (1982–1986) and SCAMC (1981–1988) boards of directors. Thus, during the 1980s, he played leading roles in all 3 of these early US informatics organizations.
Ultimately, AAMSI membership grew, the SCAMC meetings became very large, and ACMI became an established society of fellows. The leaders of the 3 organizations came together to discuss a merger that would bring the large meeting, the membership organization, and the elected fellows under a single umbrella. The process took a few years to complete, but by 1990 the new organization had been incorporated as the American Medical Informatics Association (AMIA), with Lindberg selected as the founding President and Board Chair. ACMI continued as an elected college within AMIA and the SCAMC meetings evolved into what is now known as the annual AMIA Symposium. Further details have been provided in van Bemmel et al.2 As is stressed there, Lindberg was a major force in advancing all these organizations and the field itself. During Dr. Lindberg’s term as AMIA President (1988–1991), AMIA Board discussions began regarding creating an AMIA-sponsored Journal. By 1993, William W. Stead, MD had been selected to serve as Editor-in-Chief of the Journal of the American Medical Informatics Association (JAMIA). The first issue of JAMIA appeared in January 1994. By 2020, AMIA had grown to become the dominant US biomedical informatics professional society, with well over 5000 members, and JAMIA had become a top-tier journal in the field.
THE NATIONAL LIBRARY OF MEDICINE PROGRAMS UNDER DR. LINDBERG’S LEADERSHIP
As a field matures further, stable ongoing national-level funding sources appear that sponsor research and dissemination of resources. How the NLM under Lindberg’s leadership played a central role in transforming informatics support, and the substantial intramural accomplishments of the NLM during that period, have been detailed in refs.3–12
Intramural NLM Projects that transformed biomedical informatics
Ackerman3 has described the NLM’s ground-breaking Visible Human Project, drawing on his personal involvement with the work. He relates how Dr. Lindberg, as was often the case, enabled a highly talented colleague to develop an initial idea to fruition. The project created human anatomy datasets useful in teaching at the high school, college, medical school, and postgraduate medical training levels. The project’s segmentation algorithms have found widespread application beyond teaching anatomy—in radiological imaging, in pathology, and even in astronomy.
Humphreys and Tuttle4 have provided an account of how Lindberg was able to organize and inspire highly capable individuals—inside and outside of NLM—to address a problem that he had envisioned prior to becoming NLM Director. The result of this decades-long endeavor, the NLM’s Unified Medical Language System (UMLS), has served from 1987 to this day as a foundational resource supporting applied clinical and bioinformatics activities and active research.16,17 A quotation from Humphreys and Tuttle’s account illustrates how the UMLS work at NLM epitomizes Kuhn’s above-cited characterization of scientific revolutions:
“[It] had no precedent, and, thus, initially, application developers, and their end-users, had difficulty applying it. But, as Lindberg often said, ‘Things that are used tend to get better.’ Slowly the field adopted either the UMLS artifacts themselves, its content, such as the synonyms, or its ideas, such as concept-based representations. While computers still struggle to ‘understand’ biomedical meaning usefully, most would agree that Lindberg’s vision and development approach enabled substantial progress in this important area.”4
McDonald and Humphreys11 and Mo and Denny12 have also documented long-term, far-reaching UMLS project benefits.
Ackerman et al5 have described the inter-departmental governmental collaboration between the NLM and the White House Office of Science and Technology Policy during the High Performance Computing and Communications (HPCC) project. Lindberg was asked to serve as Director of the HPCC National Coordinator’s Office. The request was based on his international reputation and that of NLM. NLM and the other cooperating HPCC agencies (including DARPA, DOE, NASA, and NSF, which were the largest), had substantive, successful, national-scale applied high-performance computing and networking projects. In addition to substantive non-medical HPCC achievements, Lindberg extended HPCC’s reach to enhance computing and telecommunication capabilities at many NIH institutes. At NLM, HPCC helped to create advanced telemedicine projects; supported networked data-sharing projects that interlinked hospitals, clinics, medical schools, and libraries; and, developed early medical virtual reality applications. The NLM supported many other telemedicine/telehealth projects, including the National Telemedicine Initiative Awards that began in 2004; see ref.5
Masys and Benson6 have chronicled how the NLM’s National Center for Biotechnology Information (NCBI) became one of the most significant information resources for biomedical research and clinical practice worldwide. The impetus for creating NCBI arose early in Lindberg’s NLM tenure, based on 1985–1986 activities that produced the 1987 NLM Long-Range Plan.18 Lindberg attended one of the 1986 visioning sessions, attended by Nobelist Alan Maxam PhD.
“[Maxam] highlighted the lack of naming consistency and interconnections among research databases constructed by different organizations. The incompatibility of these closely related scientific resources thwarted a researcher’s ability to use similarities and insights from one database to explain findings recorded in another. … this ability to ‘reason by analogy’ often depended upon findings at different levels of the biologic hierarchy … and that there were few if any automated tools capable of finding such correlations across the dozens of databases storing molecular information and its interpretations … Lindberg was immediately and enduringly impressed by this presentation and the opportunity it portrayed for NLM to help guide, structure, and link related scientific resources in pursuit of better understanding human health and disease.”6
With the assistance of US Representative Claude Pepper, the ensuing efforts of Lindberg and colleagues culminated in November 1988 with President Reagan signing legislation authorizing creation of NCBI. As Masys and Benson explained, over the next quarter century,
“NCBI became not only a global resource for molecular biology and genetics, but also a brain trust for the redesign and modernization of NLM’s other services, including MEDLINE, its flagship literature resource. NCBI began with the subset of MEDLINE records linked to factual databases in biotechnology such as GenBank, and transformed the MEDLINE unit record design into a relational data model that enabled use of highly scalable relational database management systems.”6
NLM sponsorship of extramural projects that transformed biomedical informatics
Lorenzi and Stead7 have described far-ranging effects regarding the novelty and impact of NLM’s Integrated Academic [later “Advanced”] Information Management Systems (IAIMS) grants. Prior to Lindberg’s 1984 arrival, NLM contracted with the Association of American Medical Colleges to conduct a study that produced the landmark 1982 Cooper-Matheson report.19 That report recommended that academic medical centers network together information repositories, including those of their medical libraries and affiliated hospitals, and make those resources available to students, faculty, and practicing clinicians. The NLM’s initial request for IAIMS applications led to 4 prototype projects funded in 1983 via contracts to Columbia University, the University of Maryland at Baltimore, Georgetown University, and the University of Utah. Thereafter, NLM used competitive grant funding to support subsequent IAIMS activities. Dr. Lindberg integrated IAIMS-related topics into the charges he gave to panels creating NLM’s Long-Range Plans. The IAIMS institutions not only networked information resources together—they became leaders in informatics faculty development, automation of care delivery, and informatics research.20 For faculty and students who had not previously been exposed to the field, IAIMS projects increased understanding of, respect for, and reliance on informatics capabilities. Non-IAIMS academic medical centers were inspired to emulate the activities of IAIMS institutions.21 Lorenzi and Stead conclude:
“IAIMS essentially provided a strategic change management process for an organization to develop the needed integrated information, services, products and people. By the cooperation for planning, developing demonstrations and integrating external information, the people in an organization came to appreciate the benefit of IAIMS and the need to work differently and together.”7
Greenes et al8 have described how NLM’s institutional T15 training grants influenced successive generations of trainees in the field of biomedical informatics. Those grants launched the careers of many of the field’s subsequent leaders:
“Dr. Lindberg saw the NLM Institutional Training Program as a commitment and opportunity to build the nascent community of informatics professionals. NLM sponsored an annual training meeting that brought faculty and trainees from all the NLM-funded training sites together. The NLM annual training meetings were designed to encourage faculty and trainees to interact socially, to exchange ideas, and to build a sense of community among them. Dr. Lindberg also viewed the annual training meetings as a forum to introduce faculty and trainees to the opportunities provided by NLM for support of their future research.”8
After the initial 5 annual training meetings, on alternate years NLM rotated the site of the meetings between funded training site programs and the NLM. The rotations exposed trainees to different paradigms of informatics activities across the United States. While the initial focus of most training programs was clinical informatics, Dr. Lindberg and NLM circa 1990 encouraged training sites to engage in biotechnology informatics. In later years, funding to NLM from other NIH Institutes expanded training slots to include oncology-related informatics, dental informatics, public health informatics, and data science informatics. Through the funding cycle of 2017, a total of 24 universities/institutions had benefited from receiving NLM T15 training grants.
“The NLM Institutional Research Training Programs have been key to the expansion of biomedical informatics both academically and in organizations that rely on individuals well trained in the broad aspects of the field and its scientific underpinnings.”8
Cimino9 has illustrated how the NLM was able to bring a diverse set of individuals to idyllic settings for a week-long exposure to informatics concepts and capabilities. The diversity involved course faculty recruited from NLM, informatics organizations including academia and industry, and local experts from the host sites (Woods Hole, Massachusetts and Augusta University in Georgia). The diverse set of participants came from across the United States and internationally. They included librarians, clinicians, researchers, administrators, and social scientists, among others.22 Cimino noted:
“[Lindberg] firmly believed that the way to promote the adoption of informatics tools, resources and methods in the healthcare community was through outreach programs. Those programs exposed ‘change agents’ from participating institutions to available informatics resources and applications and demonstrated what they could do.”9
By 2018, the NLM Short Courses had served as a precursor to many virtual and live-format informatics short courses hosted by leading institutions, including AMIA’s 10 × 10 courses.
Kuo and Ohno-Machado10 have presented an analysis of NLM’s extramural grant sponsorship of biomedical informatics research during Lindberg’s term as Director. They note that beyond informatics per se, NLM profoundly enhanced biomedical research through indexing and rapidly disseminating peer-reviewed literature, curating scientific databases, and through developing and hosting ClinicalTrials.gov.
The NLM investigator initiated (R01) grants from 1984 through the 1990s predominantly focused on clinical informatics; after that, biotechnology informatics (which in time became known simply as bioinformatics) played a much larger role. Regarding the impact of NLM R01 funding in biomedical informatics, Kuo and Ohno-Machado conclude:
“The impact is palpable not only in terms of continued resources that expanded the depth and scope of the biomedical informatics community. The NLM programs expanded and supported the number of faculty members and trainees in biomedical informatics nationally.”10
Long-term effects of NLM operations during and after Lindberg’s tenure
McDonald and Humphreys have described a profound sea change in NLM’s involvement in informatics standards:
“When [Lindberg] became Director in 1984, the U.S. National Library of Medicine (NLM) was a leader in the development and use of information standards for published literature but had no involvement with standards for clinical data. When Dr. Lindberg retired in 2015, NLM was the Central Coordinating Body for Clinical Terminology Standards within the U.S. Department of Health and Human Services, a major funder of ongoing maintenance and free dissemination of clinical terminology standards required for use in U.S. electronic health records (EHRs), and the provider of many services and tools to support the use of terminology standards in health care, public health, and research.”11
The genesis and evolution of the NLM’s UMLS project is described in ref.4 Shortly, after the inception of the UMLS project, legislation creating the Agency for Health Care Policy and Research established a key role for NLM in supporting public health research. Specifically, NLM became involved in creating infrastructure to enable derivation of public health data from electronic record systems. In early 1992, the NLM adopted the following goals regarding governmental health data standards: “establish a U.S. federal mechanism for selecting standards applicable to all U.S. health care and public health entities; select the best available set of vocabularies as target U.S. standards; provide ongoing federal support for maintenance, enhancement, and free dissemination of the selected vocabularies; and, support testing and feedback from real clinical settings before any federal mandate for use.”11 As part of the HPCC project, the NLM became responsible for integrating advanced network and computing technologies into testbed healthcare settings.5 Clement J. McDonald, MD at the Regenstrief Institute in Indianapolis submitted a successful proposal to network 9 Indiana Hospitals together as a health information exchange (HIE) system. Over time,
“The modest Indianapolis HIE that began life in 1994 with NLM funding that Lindberg obtained from the HPCC program has continued to operate and grow. Today, the Indiana Health Information Exchange (IHIE) serves 20,000 care providers from Indiana and adjacent states and encompasses 12 billion structured observations and hundreds of millions of narrative reports and radiology images from more than 100 health care organizations - all in the service of better health care.”11
Use of the LOINC and SNOMED terminology systems (whose origins, like HL7, had antedated NLM involvement in clinical standards) facilitated creation of the Indiana HIE.23,24 During 2000–2003, NLM began to support the maintenance of LOINC and SNOMED as national standards and arranged for their subsequent free-of-charge distribution. Concurrently, NLM collaborated with the FDA and the Veterans Health Administration to develop and distribute standard medication information in the form of RxNorm and Structured Product Labels. NLM also created the DailyMed distribution mechanism. By 2015, the NLM-developed standards for representing healthcare data—and NLM’s ongoing maintenance and distribution of standard terminologies and related technical tools—had come to underpin important aspects of pharmacy, hospital, and clinic operations nationally.
Mo and Denny12 have noted that Lindberg in 1986–1987 envisioned the future NLM-based information landscape in predicting “general usage of reference resources but also collection of ‘big data’ primary data resources, which would be curated, searchable, and cross-indexed.” Over the ensuing 3 decades,
“Under Dr. Lindberg’s leadership, the NLM invested in three areas that enabled precision medicine to become a reality and begin to impact care: (a) curation of not just the literature but storage and cataloging of emerging digital data (especially of the genome), (b) electronic health records that supported clinical decision support, and (c) computational tools to link, search, compare, and analyze the resources described above.”12
Mo and Denny have pointed out that since 2010, clinicians regularly use NLM resources (MEDLINE, PubMed, OMIM, MedlinePlus Genetics) to derive clinically relevant information for patient care. Biomedical researchers (including informaticians) depend on GenBank, dbSNP, and ClinVar, PubMed, and OMIM among other NLM resources, to advance the field.23 Analytical tools that NCBI provides through application programming interfaces (APIs) assist researchers in accessing and utilizing the online databases. The one-stop-shopping, interconnected, cross-referenced nature of NLM resources has catalyzed remarkable progress in biomedical research. In addition to intramural repositories such as those at NCBI, NLM sponsored extramural research projects that have also advanced public health and personalized medicine capabilities on a national scale. Possibly the epitome of research projects that utilize NLM resources is the NIH-sponsored “All of Us” project (led since 2019 by Dr. Denny).25,26 The All of Us project:
“launched nationally in 2018 and has as its goal the recruitment of one million diverse participants from across the United States. Research participants share information surveys, EHR information, and collect samples for whole-genome sequencing. The EHR information is harmonized across more than 50 sites, 16 different vendor systems, and with participant-completed health survey data into a common data model. In addition, participants can share EHR information directly from their healthcare providers via Fast Health Interoperability Resource (FHIR) APIs. Researchers access the data via a web portal.”12
Mo and Denny conclude by citing examples of the tremendous advances in clinical practice enabled by NLM’s work. These include (but are not limited to) rapid diagnosis and treatment of rare, potentially life-threatening genetic disorders in newborn infants; novel, genetically targeted approaches to cancer therapy; pharmacogenomic testing to prevent adverse drug events before they occur; and, accelerating the evidence-based response to the 2019–2021 COVID-19 pandemic.
To summarize and augment what has been described thus far in this review, Table 1 chronologically lists important events in Donald A.B. Lindberg’s life and in the history of the National Library of Medicine related to Lindberg’s tenure there.
Table 1.
Key events related to Donald A.B. Lindberg, MD and the U.S. National Library of Medicine (NLM)
| 1836: Library begins as collection of books in the office of the Surgeon General |
| 1933: Donald Allen Bror Lindberg born on September 21, 1933 in Brooklyn, NY |
| 1954: Lindberg graduates Phi Beta Kappa and magna cum laude from Amherst College |
| 1957: Don marries Mary Musick, RN on June 8 |
| 1958: Lindberg receives MD from Columbia’s College of Physicians and Surgeons (P&S), New York |
| 1962: Lindberg completes split Pathology Residency, P&S and University of Missouri, Columbia |
| 1963: Lindberg begins multi-year Laboratory Information System Project at University of Missouri |
| 1964: Lindberg named Markle Scholar in Academic Medicine |
| 1965: Congress passes Medical Library Assistance Act (MLAA) authorizing new NLM functions/responsibilities |
| Lindberg receives NIH Regional Medical Program grant at University of Missouri |
| 1967: Lindberg develops CONSIDER diagnostic decision support program |
| 1968: Lindberg publishes his first book, The computer and medical care |
| Congress establishes the Lister Hill National Center for Biomedical Communications at NLM |
| 1971: NLM releases MEDLINE: online database available via nationwide commercial telecommunications networks |
| 1979: Lindberg publishes third book, The Growth of Medical Information Systems in the United States |
| 1982: Lindberg Collen Blois Piemme Shortliffe discuss creation of American College of Medical Informatics (ACMI) |
| NLM-AAMC sponsored Cooper-Matheson report recommends creation of IAIMS initiative |
| 1983: Lindberg begins term as American representative to International Medical Informatics Association (IMIA) |
| 1984: Lindberg begins term as NLM Director |
| First 4 IAIMS institutions begin operations funded via NLM Contracts (before Lindberg’s Directorship) |
| First 5 research oriented T15 Institutional Training Grants awarded (before Lindberg’s Directorship) |
| First group of ACMI Fellows elected |
| 1986: Lindberg chairs Organizing Committee for MEDINFO 86 (IMIA meeting held in Washington, DC) |
| Lindberg initiates NLM Long-Range Plan visioning sessions; first of many Long-Range Plans released 1987 |
| Lindberg initiates Unified Medical Language System (UMLS) project |
| Lindberg recruits Dan Masys, MD as Director, Lister Hill National Center for Biomedical Communication |
| 1988: Lindberg term as founding AMIA President begins (before organization’s legal incorporation completed) |
| Lindberg and van Bemmel begin term as co-Editors of Methods of Information in Medicine |
| Legislation authorizing creation of NCBI passed and signed into law |
| 1989: Lindberg recruits David Lipman, MD to become Director of NCBI |
| 1990: AMIA legally incorporated |
| NLM encourages T15 Training Sites to include Bioinformatics as key component |
| NLM initiates Visible Human Project, issues requests for proposals |
| NLM releases first version of UMLS Metathesaurus and Semantic Network |
| 1992: Lindberg Director of National Coordinating Office for High Performance Computing and Communications |
| NLM offers first Woods Hole Informatics Short Course |
| 1994: Visible Human male released |
| 1996: Lindberg U.S. National Coordinator G-7 Global Healthcare Applications Project on Information Infrastructure |
| 1997: NLM launches PubMed—first free public internet access to Medline |
| 1998: Lindberg receives ACMI Morris F. Collen Award for lifetime achievement |
| NLM releases MedlinePlus consumer health website |
| 2000: NLM initiates PubMed Central repository with free access to full-length journal articles |
| NLM releases ClinicalTrials.gov as free public registry of clinical trials |
| NLM and other federal agencies begin funding maintenance and free distribution of LOINC |
| NLM part of trans-NIH Biomedical Information Science and Technology Initiative (BISTI) |
| 2002: NLM releases the RxNorm clinical drug terminology within the UMLS |
| 2003: NLM licenses SNOMED for free nationwide use in U.S. |
| 2005: Robert Wood Johnson Foundation funds public health tracks in NLM informatics training programs |
| 2007: Lindberg chairs trans-NIH Biomedical Informatics Coordinating Committee (BMIC) |
| 2008: NIH Policy requires deposit of NIH-funded research articles in PubMed Central |
| ClinicalTrials.gov summary results submission system released (as required by 2007 legislation) |
| 2011: LOINC, RxNorm, SNOMED designated U.S. national standards; required in EHRs for meaningful use |
| 2015: Lindberg retires as Director of National Library of Medicine after 31 years at helm |
| 2019: Donald A.B. Lindberg dies at age 85 years; survived by Mary, his wife of 62 years |
GOVERNMENT-SPONSORED INFORMATICS PROJECTS EXTERNAL TO NLM INITIATED 1984–2015
Clearly, the NLM was not the only governmental source of informatics activity, standards activity, or productivity during the 3 decades on which this review has focused. It can be argued, however, that the NLM has been a prime mover in affecting the entire informatics ecosystem and, by inference, has played a role much broader than what can be directly documented. A few representative informatics activities (out of hundreds) developed through other agencies should be mentioned:
The Department of Veteran’s Affairs 1980s-initiated Decentralized Hospital Computer Program (DHCP)—later renamed VISTA—supported clinical care at over 180 hospitals and more than 1000 clinics for 3 decades.27
Begun in 2000, the NIH Cross-institutional (including NLM) Biomedical Information Science and Technology Initiative (BISTI) continues to apply computer science and technology to problems in biology and medicine by sponsoring research grants and training programs.28 One of the best-known BISTI-sponsored projects is Harvard’s Informatics for Integrating Biology & the Bedside (i2b2). It distributes modular tools that provide high-level uniform access to electronic health record data within and across institutions—suitable for research-related data mining. It is used at over 250 sites worldwide.29
Initiated in 2004, the ambitious National Cancer Institute (NCI) Cancer Biomedical Informatics Grid (caBIG) project supported cancer research and clinical care delivery on a national level. A 2011 NCI review found that the project had achieved at least partial success.30 Adoption of caBIG products had at that time been lower than expected because the early years of the project seemed driven more by technology than by needs of researchers and clinicians.30
The NIH National Center for Advancing Translational Science (NCATS) introduced its highly successful Clinical and Translational Science Awards (CTSA) Program in 2006. NCATS has supported CTSAs at over 60 academic medical centers and other institutions.31 The grant recipients participate in a research network geared to accelerate “bench to bedside” translation of basic science research to clinical care delivery.31 An exemplary informatics product of the CTSA initiative is the Research Electronic Data Capture (REDcap) project.32 As of 2021, REDcap has provided over 5400 institutions in more than 140 countries with an internet-based application that allows users to design and share surveys, launch them via Web interfaces, and then capture the data to support clinical and other forms of research.
The NIH National Human Genome Research Institute (NHGRI) eMERGE project, initiated in 2007, electronically interconnects research sites that have both electronic medical record systems and corresponding genetic data biobanks.33 The goal of eMERGE is to conduct cross-institutional studies that discover new phenotype-gene associations. For example, an early eMERGE study identified previously unknown genetic risk factors for hypothyroidism.
These and many other federally sponsored programs outside NLM involved researchers and developers who were also supported previously or concurrently by NLM, thereby leveraging NLM programs and resulting expertise in the execution of their own informatics-related activities.
INFORMATICS ACTIVITIES EXTERNAL TO THE NATIONAL LIBRARY OF MEDICINE, 1984–2015
During the 31 years when Don Lindberg served as NLM director, the world of digital technology and communications was simultaneously evolving in profound ways. Naturally, this evolution affected the NLM and its programs. Similarly, NLM had an impact on many innovations that either emerged from its intramural or extramural programs or had an influence on the ways in which the NLM and its grantees or trainees pursued their work. To provide context, a brief narrative follows to augment this review’s discussion of Lindberg and the NLM by describing some of the external informatics activities, spread over 3 decades, to which NLM contributed or from which it and the field benefited.
Advances in computer hardware and software
Lindberg arrived at the NLM just as microcomputer availability began having its revolutionary impact. The previous decade had been dominated by mainframe computing (eg, from IBM, Burroughs, Sperry-Rand) and by shared midsized or minicomputers (eg, from Digital Equipment Corporation). Nevertheless, with the introduction of the Apple personal computer in 1976 (and the Macintosh in 1984), complemented by the first IBM PC in 1981, minicomputers were destined to become less pertinent and eventually to disappear. They were replaced not only by Macs and PCs but by more powerful personal workstations running the UNIX operating system (eg, from Sun Microsystems, Hewlett Packard, and IBM). Along with the increasing availability of wide-area-networking (eg, the evolving ARPAnet as well as commercial networks), another key development was the introduction of local area networking (eg, Xerox Ethernet, WangNet).
All of these developments had an impact in the medical community, which in the 1970s and early 1980s had invested heavily in minicomputers to run departmental systems in hospital pharmacies, clinical laboratories, and myriad other sites.14 Hospital systems gradually moved to networked approaches using a mix of mainframe computers, personal computers, and high-performance workstations. Meanwhile, although the ARPAnet was managed for a few years as NSFNet by the National Science Foundation, by the late 1980s it became known solely as the Internet with the addition of the domain system (including .com, which was a controversial addition at the time). Whereas NLM’s early MEDLINE system had used commercial networks for access (largely by specially trained librarians), as did Grateful Med (the search system introduced in 1986,34,35 designed specifically for use by individual clinicians, scientists, and students), the Internet and the introduction of the World Wide Web in the 1990s allowed NLM to leverage broad network access to democratize access to the medical literature internationally, producing PubMed by the mid-1990s.8
More changes were afoot that began to arrive in the 1990s. These included the first mobile phones and tablet computers (although the technologies were not merged into single smartphone devices until after the turn of the century). The desire for mobility also drove the introduction of wireless networking, leading to Wi-Fi technologies that came to dominate consumer access to networking applications.
It is hard to imagine in 2021 how the world would have confronted the impact of the COVID-19 pandemic without the remarkable technological revolution briefly summarized here.36 Consider, for example, the explosive use of Internet-based telemedicine/telehealth, the ability to provide online education to students of all ages when schools or colleges were otherwise forced to shut down, the adoption of virtual meetings and working from home, connectivity among family members who could not visit loved-ones in person—all of which would have been inconceivable even a decade before.
Of course, the evolution in digital technologies was accompanied by software innovations—in many cases bringing to reality a variety of capabilities that had been futuristic research topics only a few decades earlier. This included commercialization of speech understanding in consumer products, hands-free computing environments, and a variety of new programming languages that have changed the way in which software developers think about how to solve problems. All of these have influenced informatics research and applied projects as well as their commercial implementations.
The NLM has carefully followed, encouraged, and often adopted the new hardware and software technologies, maintaining a modern, cutting-edge intramural environment, as well as a research grant portfolio that has explored, and often led, the refinement and biomedical application of new capabilities. For example, there is little doubt that the bevy of novel methodological capabilities and accomplishments in the areas of machine learning and artificial intelligence in the decade following 2010 were led by biomedical or healthcare applications, many developed by NLM or other NIH grantees.37
Growth of informatics as an academic priority
The NLM’s extramural programs—training grants and sponsored research projects—have played a key role in advancing informatics science and producing the next generation of researchers and informatics practitioners. When the current training grants program began in 1984, there were only a handful of centers with organized informatics research groups that had created academic programs (postdoctoral fellowships or graduate degrees). Such groups often strived for recognition and respect in their own institutions (typically medical or nursing schools), since informatics was a new discipline that was widely seen as being “outside” the traditional areas of biomedical scientific focus.38 Recognizing this problem, the NLM was proactive not only in funding training programs but in bringing educational opportunities to other interested individuals in the biomedical community. (see, for example, ref.9)
Lindberg himself was active on the faculty of annual informatics courses for medical school deans (and other senior administrators) that were organized by the Association of American Medical Colleges (AAMC). Academic units were gradually reorganized as formal divisions or departments in medical or nursing schools, which led to their recognition as relevant and collaborative elements in the academic environment. Many schools classified informatics units both as basic science departments (research organizations offering MS and PhD degrees) and as clinical departments (organizations managing and enhancing operations in the area of clinical systems and, in time, electronic health records). By the 1990s, biomedical informatics was increasingly recognized as an institutional priority for academic medical centers, and the demand for suitably trained faculty informaticians began to exceed the supply.39 NLM’s training programs became catalysts for increasing the number of trainees, even though not all trainees could be supported on the NLM T15 grants. Thus, NLM’s efforts had an amplification effect that enhanced the ability of programs to produce high-quality researchers, educators, and managers. The graduates of NLM training programs were able to lead new faculty units at other schools that wanted to create their own engagement with the field, including its inclusion in the education of medical and nursing students.
Today most US medical and nursing schools have an academic unit that deals with informatics topics (research, graduate education, training of clinical students, management of clinical systems, etc.). This change over 3 decades is a testament to the role that NLM has played, not only in training professional informaticians but also in solidifying the recognition of the field as a crucial element in the academic biomedical environment going forward.
Growth of informatics as a science
The discipline of biomedical and health informatics has always been driven by a desire to use computational methods to benefit biomedical science, clinical care, and the health of the populace. Thus applications matter—they are the ultimate goal of informatics work. However, the biomedical and health domains are sufficiently complex that it often is not practical or appropriate to build applied solutions that use standard methods that are already available. Tough challenges drive innovation, and the science of informatics is accordingly linked to the development of novel methods and approaches that not only enable a solution to a new problem but that also can be generalized and applied more broadly. The science of the field is thus ultimately linked to methodological innovation and the communication and demonstration of those methods so that others can also apply them or extend them further. The NLM and its grant review committees have understood this perspective on the field, which accounts for the remarkable amount of new science and innovation that has emerged through NLM’s intramural and extramural research programs.
It would be folly to attempt to summarize here the numerous novel methods that have resulted with NLM support, many of which have been communicated in the pages of this journal over the years. The full list would be very long but would demonstrate not only the breadth of the work but also the numerous examples where medically driven innovation has strengthened the science, and its applicability, in areas that extend beyond biomedicine itself. As an example from the 1980s, when Lindberg’s NLM directorship was just beginning, early research on medical expert systems led to a flurry of activity and computer science applications in far flung areas that at first glance appeared to have little in common with biomedicine or health.40,41 Other examples over the 3 decades include medically motivated innovations in natural language and text processing,42 knowledge representation,43 ontology design and management,44 machine learning and deep learning,45,46 image interpretation,47 data visualization,48 3-dimensional simulations,49 and large dataset analytics.50
The NLM also embraced the growing relevance of informatics methods for the basic sciences of human biology. This was a major topic for one of the original Lindberg planning panels between 1985-1987, which presaged the impact of the Human Genome Project during the 1990s.6 As genomics scientists realized that computational methods and analytics were crucial to their work, the NLM embraced the nascent field of bioinformatics, and especially its translational aspects in human health. Lindberg worked to assure that bioinformatics and clinical informatics evolved synergistically, sharing commonalities, learning from one another, and even creating a new name for the combined discipline (biomedical informatics as an umbrella term for bioinformatics, clinical informatics, and public health informatics). The creation of the National Center for Biotechnology Information at NLM, the assumption of responsibility for the management of GenBank, and support for precision and genomic medicine, were part of this effort.6,12,18
Due to the complexity and special natures of medicine and patient care, all the work described herein has proceeded in an environment that has distinct social, cultural, and regulatory requirements. Informatics scientists, and the NLM, have accordingly participated in the specification of privacy and confidentiality requirements, the development of standards for cross-platform fielding of systems, discussions of medical software regulation, ethics, the role of informatics in addressing the social determinants of health, and many more.51
CONCLUSION
As has been summarized, the NLM, with Don Lindberg’s leadership and drive—paired with an extraordinarily capable NLM team—had a remarkable impact on US biomedical science and clinical medicine during the years from 1984 to 2015 and beyond. The NLM leveraged its prominent and visible role as an NIH institute to garner a positive reputation, not only in the United States but internationally. Free online access to PubMed, and to the medical literature through PubMed Central, have benefited scientists, practitioners, and students globally, heightening international respect not only for the NIH but for the country itself. Always at the forefront of current technology, NLM has anticipated the future and driven the scientific and clinical communities toward adoption and use of informatics innovations. The NLM strived to identify and understand barriers to the use of scientific knowledge and new technologies. It has created solutions both through its intramural programs and its support for extramural research and development. Through all this effort and success, Lindberg himself served as a towering figure. His intellect, commitment to the national and global communities, effectiveness as a leader both at NIH and within the Department of Health and Human Services more broadly—coupled with a kindness and inspirational dedication—assured that NLM entered the 21st century as a model of what American ingenuity can achieve.
FUNDING
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
AUTHOR CONTRIBUTIONS
Both RAM and EHS contributed to the conceptualization of this work, analysis and review of references cited, writing, editing, revision, and final approval. We agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are investigated appropriately and resolved.
ACKNOWLEDGMENTS
This article summarizes and expands upon the informatics chapters in a forthcoming book (Transforming biomedical informatics and health information access: Don Lindberg and the U.S. National Library of Medicine, Humphreys BL, Logan RA, Miller RA, Siegel ER, IOS Press, 2021). We thank Betsy L. Humphreys MLS, other authors of those chapters, all of the book’s editors, and Mary Lindberg RN, who have reviewed previous versions of this manuscript and made helpful comments.
CONFLICT OF INTEREST STATEMENT
While RAM and EHS contributed to the writing and editing of the book acknowledged above, they did not and will not receive any compensation for doing so. RAM was 1 of the 4 Editors of the book. The other Editors of the book (Humphreys, Logan, and Siegel) were aware that we were writing this summary manuscript and encouraged us to do so. The publisher of the book, IOS Press (Netherlands) was also aware that we were writing this manuscript (and in doing so, avoiding any overlap with the book contents), and voiced no objections.
DATA AVAILABILITY
The generation of this manuscript did not involve creation or use of data other than straightforward bibliographic review processes. All bibliographic materials used in writing this manuscript have been duly cited in the References section below.
Contributor Information
Randolph A Miller, Emeritus Professor of Biomedical Informatics, Vanderbilt University School of Medicine, Alexandria, Virginia, USA.
Edward H Shortliffe, Department of Biomedical Informatics, Columbia University in the City of New York, New York, New York, USA.
REFERENCES
- 1. Kingsland LC 3rd, Kulikowski CA. A scientific mind embraces medicine: Donald Lindberg’s education and early career. In: Humphreys BL, Logan RA, Miller RA, Siegel ER, eds. Transforming Biomedical Informatics and Health Information Access: Don Lindberg and the U.S. National Library of Medicine. Amsterdam: IOS Press; Forthcoming, 2021. [Google Scholar]
- 2. van Bemmel JH, Ball MJ, Shortliffe EH.. Donald AB Lindberg, pioneer in biomedical and health informatics: his involvement in creating professional organizations. In: Humphreys BL, Logan RA, Miller RA, Siegel ER, eds. Transforming Biomedical Informatics and Health Information Access: Don Lindberg and the U.S. National Library of Medicine. Amsterdam: IOS Press; Forthcoming, 2021. [Google Scholar]
- 3. Ackerman MJ. The Visible Human project. In: Humphreys BL, Logan RA, Miller RA, Siegel ER, eds. Transforming Biomedical Informatics and Health Information Access: Don Lindberg and the U.S. National Library of Medicine. Amsterdam: IOS Press; Forthcoming, 2021. [Google Scholar]
- 4. Humphreys BL, Tuttle MS. Something new and different: the Unified Medical Language System. In: Humphreys BL, Logan RA, Miller RA, Siegel ER, eds. Transforming Biomedical Informatics and Health Information Access: Don Lindberg and the U.S. National Library of Medicine. Amsterdam: IOS Press; Forthcoming, 2021. [Google Scholar]
- 5. Ackerman MJ, Howe SE, Masys DR. Don Lindberg, high performance computing and communications, and telemedicine. In: Humphreys BL, Logan RA, Miller RA, Siegel ER, eds. Transforming Biomedical Informatics and Health Information Access: Don Lindberg and the U.S. National Library of Medicine. Amsterdam: IOS Press; Forthcoming, 2021. [Google Scholar]
- 6. Masys DR, Benson DA. Don Lindberg and the creation of the National Center for Biotechnology Information. In: Humphreys BL, Logan RA, Miller RA, Siegel ER, eds. Transforming Biomedical Informatics and Health Information Access: Don Lindberg and the U.S. National Library of Medicine. Amsterdam: IOS Press; Forthcoming, 2021. [Google Scholar]
- 7. Lorenzi NM, Stead WW. NLM and the IAIMS Initiative: cross-institutional academic/advanced systems contributing to the evolution of networked information and resources. In: Humphreys BL, Logan RA, Miller RA, Siegel ER, eds. Transforming Biomedical Informatics and Health Information Access: Don Lindberg and the U.S. National Library of Medicine. Amsterdam: IOS Press; Forthcoming, 2021. [Google Scholar]
- 8. Greenes RA, Florance V, Miller RA. Don Lindberg’s influence on future generations: the U.S. National Library of Medicine’s biomedical informatics research training programs. In: Humphreys BL, Logan RA, Miller RA, Siegel ER, eds. Transforming Biomedical Informatics and Health Information Access: Don Lindberg and the U.S. National Library of Medicine. Amsterdam: IOS Press; Forthcoming, 2021. [Google Scholar]
- 9. Cimino JJ. The biomedical informatics short course at Woods Hole/Georgia: training to support institutional change. In: Humphreys BL, Logan RA, Miller RA, Siegel ER, eds. Transforming Biomedical Informatics and Health Information Access: Don Lindberg and the U.S. National Library of Medicine. Amsterdam: IOS Press; Forthcoming, 2021. [Google Scholar]
- 10. Kuo T-T, Ohno-Machado L. NLM’s sponsorship of research in biomedical informatics (1985-2016). In: Humphreys BL, Logan RA, Miller RA, Siegel ER, eds. Transforming Biomedical Informatics and Health Information Access: Don Lindberg and the U.S. National Library of Medicine. Amsterdam: IOS Press; Forthcoming, 2021. [Google Scholar]
- 11. McDonald CJ, Humphreys Bl. The US. National Library of Medicine and standards for electronic health records: one thing led to another. In: Humphreys BL, Logan RA, Miller RA, Siegel ER, eds. Transforming Biomedical Informatics and Health Information Access: Don Lindberg and the U.S. National Library of Medicine. Amsterdam: IOS Press; Forthcoming, 2021. [Google Scholar]
- 12. Mo H, Denny JC, The US. National Library of Medicine’s impact on precision and genomic medicine. In: Humphreys BL, Logan RA, Miller RA, Siegel ER, eds. Transforming Biomedical Informatics and Health Information Access: Don Lindberg and the U.S. National Library of Medicine. Amsterdam: IOS Press; Forthcoming, 2021. [Google Scholar]
- 13. Kuhn TS. The Structure of Scientific Revolutions (50th Anniversary Edition). Chicago, IL: University of Chicago Press; 2012. [Google Scholar]
- 14. Collen MF, Ball MJ. The History of Medical Informatics in the United States. London: Springer; 2015. [Google Scholar]
- 15.Lindberg DAB, Rowland LR, Buch CR Jr, Morse WF, Morse SS. Consider: a computer program for medical instruction.In: Proceedings of the ninth IBM medical symposium. White Plains: IBM, 1968. Conference dates: 9th IBM Medical Symposium; October 24–26, 1969; Burlington, Vermont.
- 16. Lindberg DAB, Humphreys BL, McCray AT. The unified medical language system. Methods Inf Med 1993; 32 (04): 281–91. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Humphreys BL, Lindberg DAB, Schoolman HM, Barnett GO. The unified medical language system: an informatics research collaboration. J Am Med Inform Assoc 1998; 5 (1): 1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Smith K. A brief history of NCBI’s formation and growth. In: The NCBI Handbook. 2nd ed. Bethesda (MD: ): National Center for Biotechnology Information (US; ); 2013. Available from: https://www.ncbi.nlm.nih.gov/books/NBK148949/ Accessed October 2021. [Google Scholar]
- 19. Matheson NW, Cooper JA. Academic information in the academic health sciences center. Roles for the library in information management. J Med Educ 1982; 57 (10 Pt 2): 1–93. [DOI] [PubMed] [Google Scholar]
- 20. Lindberg DAB, West RT, Corn M. IAIMS: an overview from the National Library of Medicine. Bull Med Libr Assoc 1992; 30 (3): 244–6. [PMC free article] [PubMed] [Google Scholar]
- 21. Stead WW. The evolution of IAIMS: lessons for the next decade. J Am Med Inform Assoc 1997; 4 (2 Suppl): S4–9; Discussion S10–11. [PMC free article] [PubMed] [Google Scholar]
- 22. Patel VL, Branch T, Cimino A, Norton C, Cimino JJ. Participant perceptions of the influences of the NLM-sponsored Woods Hole medical informatics course. J Am Med Inform Assoc 2005; 12 (3): 256–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Bodenreider O, Cornet R, Vreeman DJ. Recent developments in clinical terminologies - SNOMED CT, LOINC, and RxNorm. Yearb Med Inform 2018; 27 (1): 129–39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. McDonald CJ, Overhage JM, Barnes M, et al. ; INPC Management Committee. The Indiana network for patient care: a working local health information infrastructure. An example of a working infrastructure collaboration that links data from five health systems and hundreds of millions of entries. Health Aff (Millwood) 2005; 24 (5): 1214–20. [DOI] [PubMed] [Google Scholar]
- 25. Denny JC, Rutter JL, Goldstein DB, Philippakis A, Smoller JW, et al. ; All of Us Research Program Investigators. The “All of Us” research program. N Engl J Med 2019; 381: 668–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.U.S. National Institutes of Health. All of Us Research Program. https://allofus.nih.gov. Accessed October 2021.
- 27. Lloyd SS. Automated information systems provide health information management support to veterans' healthcare. J AHIMA 1992; 63 (6): 63–7. PMID: 10119086 [PubMed] [Google Scholar]
- 28. U.S. National Institutes of Health. BISTI: Biomedical Information Science and Technology Initiative. https://bisti.nih.gov/Pages/About-Bisti.aspx Accessed October 2021.
- 29. Murphy SN, Weber G, Mendis M, et al. Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2). J Am Med Inform Assoc 2010; 17 (2): 124–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.National Cancer Institute. Report of the Board of Scientific Advisors Ad Hoc Working Group: An assessment of the impact of the NCI Cancer Biomedical Informatics Grid (caBIG). 2011. https://deainfo.nci.nih.gov/advisory/bsa/archive/bsa0311/caBIGfinalReport.pdf Accessed October 2021.
- 31.Committee to Review the Clinical and Translational Science Awards Program at the National Center for Advancing Translational Sciences; Board on Health Sciences Policy; Institute of Medicine. In: Leshner AI, Terry SF, Schultz AM, Liverman CT, eds. The CTSA Program at NIH: Opportunities for Advancing Clinical and Translational Research. Washington DC: National Academies Press; 2013. [PubMed] [Google Scholar]
- 32. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42 (2): 377–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Aronson S, Babb L, Ames D, et al. ; eMERGE Network EHRI Working Group. Empowering genomic medicine by establishing critical sequencing result data flows: the eMERGE example. J Am Med Inform Assoc 2018; 25 (10): 1375–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Dorsch JL, Faughnan JG, Humphreys BL. Direct access to MEDLINE for health professionals with personal computers. In: Humphreys BL, Logan RA, Miller RA, Siegel ER, eds. Transforming Biomedical Informatics and Health Information Access: Don Lindberg and the U.S. National Library of Medicine. Amsterdam: IOS Press; Forthcoming, 2021. [Google Scholar]
- 35. Smith K. Free MEDLINE access worldwide. In: Humphreys BL, Logan RA, Miller RA, Siegel ER, eds. Transforming Biomedical Informatics and Health Information Access: Don Lindberg and the U.S. National Library of Medicine. Amsterdam: IOS Press; Forthcoming, 2021. [Google Scholar]
- 36. Ye J. The role of health technology and informatics in a global public health emergency: practices and implications from the COVID-19 pandemic. JMIR Med Inform 2020; 8 (7): e19866. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Annapureddy AR, Angraal S, Caraballo C, et al. The National Institutes of Health funding for clinical research applying machine learning techniques in 2017. NPJ Digit Med 2020; 3: 13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Braude RM. A descriptive analysis of National Library of Medicine-funded medical informatics training programs and the career choice of their graduates. Med Decis Making 1991; 11 (1): 33–7. [DOI] [PubMed] [Google Scholar]
- 39. Greenes RA, Shortliffe EH. Medical informatics: an emerging academic discipline and institutional priority. JAMA 1990; 263 (8): 1114–20. [DOI] [PubMed] [Google Scholar]
- 40. Shortliffe EH. Medical expert systems—knowledge tools for physicians. West J Med 1986; 145 (6): 830–9. [PMC free article] [PubMed] [Google Scholar]
- 41. Saibene A, Assale M, Giltri M. Expert systems: definitions, advantages and issues in medical field applications. Exp Syst Appl 2021; 177: 114900. [Google Scholar]
- 42. Gonzalez-Hernandez G, Sarker A, O'Connor K, Savova G. Capturing the patient's perspective: a review of advances in natural language processing of health-related text. Yearb Med Inform 2017; 26 (1): 214–27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Riaño D, Peleg M, Ten Teije A. Ten years of knowledge representation for health care (2009-2018): topics, trends, and challenges. Artif Intell Med 2019; 100: 101713. [DOI] [PubMed] [Google Scholar]
- 44. Musen MA; Protégé Team. The Protégé project: a look back and a look forward. AI Matters 2015; 1 (4): 4–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45. Ghassemi M, Naumann T, Schulam P, Beam AL, Chen IY, Ranganath R. A review of challenges and opportunities in machine learning for health. AMIA Jt Summits Transl Sci Proc 2020; 2020: 191–200. [PMC free article] [PubMed] [Google Scholar]
- 46. Miotto R, Wang F, Wang S, Jiang X, Dudley JT. Deep learning for healthcare: review, opportunities and challenges. Brief Bioinform 2018; 19 (6): 1236–46. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47. Anwar SM, Majid M, Qayyum A, Awais M, Alnowami M, Khan MK. Medical image analysis using convolutional neural networks: a review. J Med Syst 2018; 42 (11): 226. [DOI] [PubMed] [Google Scholar]
- 48. Betzendahl L, Brown C, Rowell K. Visualizing Health and Healthcare Data. New York: Wiley; 2020. [Google Scholar]
- 49. Viglialoro RM, Condino S, Turini G, Carbone M, Ferrari V, Gesi M. Augmented reality, mixed reality, and hybrid approach in healthcare simulation: a systematic review. Appl. Sci 2021; 11 (5): 2338. [Google Scholar]
- 50. Ristevski B, Chen M. Big data analytics in medicine and healthcare. J Integr Bioinform 2018; 15 (3): 20170030. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51. Goodman KW, Miller RA. Ethics in biomedical and health informatics: users, standards, and outcomes. In: Shortliffe EH, Cimino JJ, Chiang MF, eds. Biomedical Informatics: Computer Applications in Health Care and Biomedicine. Cham, Switzerland: Springer Nature Switzerland AG; 2021. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The generation of this manuscript did not involve creation or use of data other than straightforward bibliographic review processes. All bibliographic materials used in writing this manuscript have been duly cited in the References section below.