Download PDF
Appl Clin Inform 2019; 10(05): 810-819
DOI: 10.1055/s-0039-1697905
State of the Art/Best Practice Paper
Georg Thieme Verlag KG Stuttgart · New York

Towards a Maturity Model for Clinical Decision Support Operations

Evan W. Orenstein
1   Department of Pediatrics, Emory University School of Medicine, Atlanta, Georgia, United States
2   Division of Hospital Medicine, Children's Healthcare of Atlanta, Atlanta, Georgia, United States
,
Naveen Muthu
3   Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
4   Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States
,
Asli O. Weitkamp
5   Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, Tennessee, United States
,
Daria F. Ferro
3   Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
4   Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States
,
Mike D. Zeidlhack
6   Phrase Health Inc., Philadelphia, Pennsylvania, United States
,
Jason Slagle
5   Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, Tennessee, United States
,
Eric Shelov
3   Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
4   Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States
,
Marc C. Tobias
6   Phrase Health Inc., Philadelphia, Pennsylvania, United States
› Author Affiliations
Further Information

Publication History

04 May 2019

14 August 2019

Publication Date:
30 October 2019 (online)

Also available at
   Permissions and Reprints

Abstract

Clinical decision support (CDS) systems delivered through the electronic health record are an important element of quality and safety initiatives within a health care system. However, managing a large CDS knowledge base can be an overwhelming task for informatics teams. Additionally, it can be difficult for these informatics teams to communicate their goals with external operational stakeholders and define concrete steps for improvement. We aimed to develop a maturity model that describes a roadmap toward organizational functions and processes that help health care systems use CDS more effectively to drive better outcomes. We developed a maturity model for CDS operations through discussions with health care leaders at 80 organizations, iterative model development by four clinical informaticists, and subsequent review with 19 health care organizations. We ceased iterations when feedback from three organizations did not result in any changes to the model. The proposed CDS maturity model includes three main “pillars”: “Content Creation,” “Analytics and Reporting,” and “Governance and Management.” Each pillar contains five levels—advancing along each pillar provides CDS teams a deeper understanding of the processes CDS systems are intended to improve. A “roof” represents the CDS functions that become attainable after advancing along each of the pillars. Organizations are not required to advance in order and can develop in one pillar separately from another. However, we hypothesize that optimal deployment of preceding levels and advancing in tandem along the pillars increase the value of organizational investment in higher levels of CDS maturity. In addition to describing the maturity model and its development, we also provide three case studies of health care organizations using the model for self-assessment and determine next steps in CDS development.

Keywords

decision support - clinical informatics - health information

Authors' Contributions

E.W.O., N.M., A.O.W., D.F.F., M.D.Z., J.S., E.S., and M.C.T. wrote the manuscript. N.M., M.D.Z., and M.C.T. performed primary data collection. E.W.O., N.M., D.F.F., M.D.Z., and M.C.T. analyzed the data and developed the proposed CDS maturity model.


Protection of Human and Animal Subjects

No human subjects were involved in this project.


 

  • References

  • 1 Bright TJ, Wong A, Dhurjati R. , et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med 2012; 157 (01) 29-43
    CrossrefPubMedGoogle Scholar
  • 2 Roshanov PS, You JJ, Dhaliwal J. , et al; CCDSS Systematic Review Team. Can computerized clinical decision support systems improve practitioners' diagnostic test ordering behavior? A decision-maker-researcher partnership systematic review. Implement Sci 2011; 6 (88) 88
    CrossrefPubMedGoogle Scholar
  • 3 Nieuwlaat R, Connolly SJ, Mackay JA. , et al; CCDSS Systematic Review Team. Computerized clinical decision support systems for therapeutic drug monitoring and dosing: a decision-maker-researcher partnership systematic review. Implement Sci 2011; 6: 90
    CrossrefPubMedGoogle Scholar
  • 4 Souza NM, Sebaldt RJ, Mackay JA. , et al; CCDSS Systematic Review Team. Computerized clinical decision support systems for primary preventive care: a decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci 2011; 6 (87) 87
    CrossrefPubMedGoogle Scholar
  • 5 Sahota N, Lloyd R, Ramakrishna A. , et al; CCDSS Systematic Review Team. Computerized clinical decision support systems for acute care management: a decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci 2011; 6 (91) 91
    CrossrefPubMedGoogle Scholar
  • 6 Hemens BJ, Holbrook A, Tonkin M. , et al; CCDSS Systematic Review Team. Computerized clinical decision support systems for drug prescribing and management: a decision-maker-researcher partnership systematic review. Implement Sci 2011; 6 (89) 89
    CrossrefPubMedGoogle Scholar
  • 7 Roshanov PS, Fernandes N, Wilczynski JM. , et al. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ 2013; 346: f657
    CrossrefPubMedGoogle Scholar
  • 8 Osheroff JA, Teich JM, Levick D. , et al. Improving Outcomes with Clinical Decision Support: an Implementer's Guide. 2nd ed. Chicago, IL: HIMSS; 2012
    Google Scholar
  • 9 Van de Velde S, Kunnamo I, Roshanov P. , et al; GUIDES expert panel. The GUIDES checklist: development of a tool to improve the successful use of guideline-based computerised clinical decision support. Implement Sci 2018; 13 (01) 86
    CrossrefPubMedGoogle Scholar
  • 10 Greenes RA, Bates DW, Kawamoto K, Middleton B, Osheroff J, Shahar Y. Clinical decision support models and frameworks: Seeking to address research issues underlying implementation successes and failures. J Biomed Inform 2018; 78: 134-143
    CrossrefPubMedGoogle Scholar
  • 11 Millard Krause T. CMIO/CHIO Summit: managing clinical decision support and improving workflow. Healthcare Informatics Magazine (2016). Available at: https://www.healthcare-informatics.com/article/ehr/cmiochio-summit-managing-clinical-decision-support-and-improving-workflow . Accessed November 9, 2018
    PubMedGoogle Scholar
  • 12 Carvalho JV, Rocha Á, Abreu A. Maturity models of healthcare information systems and technologies: a literature review. J Med Syst 2016; 40 (06) 131
    CrossrefPubMedGoogle Scholar
  • 13 Knosp BM, Barnett WK, Anderson NR, Embi PJ. Research IT maturity models for academic health centers: early development and initial evaluation. J Clin Transl Sci 2018; 2 (05) 289-294
    PubMedGoogle Scholar
  • 14 HIMSS Analytics - North America. Electronic Medical Record Adoption Model. Available at: https://www.himssanalytics.org/emram . Accessed November 12, 2018
    PubMedGoogle Scholar
  • 15 Analytics HIMSS. Continuity of Care Maturity Model. Available at: https://www.himssanalytics.org/ccmm . Accessed: November 23, 2018
    PubMed
  • 16 van de Wetering R, Batenburg R. Towards a theory of PACS deployment: an integrative PACS maturity framework. J Digit Imaging 2014; 27 (03) 337-350
    PubMedGoogle Scholar
  • 17 Corrigan D, McDonnell R, Zarabzadeh A, Fahey T. A multistep maturity model for the implementation of electronic and computable diagnostic clinical prediction rules (eCPRs). EGEMS (Wash DC) 2015; 3 (02) 1153
    PubMedGoogle Scholar
  • 18 Staggers N, Rodney M, Alafaireet P. , et al. Promoting Usability in Health Organizations: Initial Steps and Progress Toward a Healthcare Usability Maturity Model. Chicago, IL: Health Information Management Systems Society; 2011: 1-55
    Google Scholar
  • 19 Innovation Pathways Maturity Model HIMSS. Available at: https://www.himss.org/library/innovation/pathways-maturity-model . Accessed November 23, 2018
    PubMed
  • 20 Lipsitz LA. Understanding health care as a complex system: the foundation for unintended consequences. JAMA 2012; 308 (03) 243-244
    CrossrefPubMedGoogle Scholar
  • 21 Ash JS, Smith III AC, Stavri PZ. Performing subjectivist studies in the qualitative traditions responsive to users. In: Friedman CP, Wyatt JC. , eds. Evaluation Methods in Medical Informatics. 2nd ed. New York, NY: Springer-Verlag; 2006: 267-300
    Google Scholar
  • 22 Marshall MN. Sampling for qualitative research. Fam Pract 1996; 13 (06) 522-525
    CrossrefPubMedGoogle Scholar
  • 23 Tobias MC, Orenstein EW, Muthu N. Decision support for decision support: a novel system to prioritize improvement efforts, identify safety hazards, and measure improvement. AMIA Annu Symp Proc 2018; 2018: 2246
    PubMedGoogle Scholar
  • 24 Orenstein EW, Weitkamp AO, Rosenau P, Mallozzi C, Tobias MC. Towards a maturity model for clinical decision support operations: an interactive panel. In: American Medical Informatics Association Clinical Informatics Conference; April 30–May 2, 2019; Atlanta, GA
    PubMedGoogle Scholar
  • 25 Carayon P, Schoofs Hundt A, Karsh BT. , et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care 2006; 15 (Suppl. 01) i50-i58
    CrossrefPubMedGoogle Scholar
  • 26 Holden RJ, Carayon P, Gurses AP. , et al. SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals and patients. Ergonomics 2013; 56 (11) 1669-1686
    CrossrefPubMedGoogle Scholar
  • 27 Jones BE, Collingridge DS, Vines CG. , et al. CDS in a learning health care system: identifying physicians' reasons for rejection of best-practice recommendations in pneumonia through computerized clinical decision support. Appl Clin Inform 2019; 10 (01) 1-9
    Article in Thieme ConnectPubMedGoogle Scholar
  • 28 Aaron S, McEvoy DS, Ray S, Hickman TTT, Wright A. Cranky comments: detecting clinical decision support malfunctions through free-text override reasons. J Am Med Inform Assoc 2019; 26 (01) 37-43
    CrossrefPubMedGoogle Scholar
  • 29 Chan J, Shojania KG, Easty AC, Etchells EE. Does user-centred design affect the efficiency, usability and safety of CPOE order sets?. J Am Med Inform Assoc 2011; 18 (03) 276-281
    CrossrefPubMedGoogle Scholar
  • 30 Wright A, Hickman TT, McEvoy D. , et al. Analysis of clinical decision support system malfunctions: a case series and survey. J Am Med Inform Assoc 2016; 23 (06) 1068-1076
    CrossrefPubMedGoogle Scholar
  • 31 Yoshida E, Fei S, Bavuso K, Lagor C, Maviglia S. The value of monitoring clinical decision support interventions. Appl Clin Inform 2018; 9 (01) 163-173
    Article in Thieme ConnectPubMedGoogle Scholar
  • 32 Kassakian SZ, Yackel TR, Gorman PN, Dorr DA. Clinical decisions support malfunctions in a commercial electronic health record. Appl Clin Inform 2017; 8 (03) 910-923
    Article in Thieme ConnectPubMedGoogle Scholar
  • 33 Thayer JG, Miller JM, Fiks AG, Tague L, Grundmeier RW. Assessing the safety of custom web-based clinical decision support systems in electronic health records: a case study. Appl Clin Inform 2019; 10 (02) 237-246
    Article in Thieme ConnectPubMedGoogle Scholar
  • 34 Institute of Medicine. The Learning Healthcare System: Workshop Summary. Washington, DC: National Academies Press; 2007
    Google Scholar
  • 35 Lindgren H, Lu MH, Hong Y, Yan C. Applying the zone of proximal development when evaluating clinical decision support systems: a case study. Stud Health Technol Inform 2018; 247: 131-135
    PubMedGoogle Scholar
  • 36 Wright A, Ai A, Ash J. , et al. Clinical decision support alert malfunctions: analysis and empirically derived taxonomy. J Am Med Inform Assoc 2018; 25 (05) 496-506
    CrossrefPubMedGoogle Scholar
  • 37 Nielsen J. Corporate UX maturity: stages 5–8. 2006. Available at: https://www.nngroup.com/articles/ux-maturity-stages-5-8/ . Accessed November 23, 2018
    PubMedGoogle Scholar
  • 38 Wright A, Feblowitz JC, Pang JE. , et al. Use of order sets in inpatient computerized provider order entry systems: a comparative analysis of usage patterns at seven sites. Int J Med Inform 2012; 81 (11) 733-745
    CrossrefPubMedGoogle Scholar
  • 39 Shojania KG, Jennings A, Mayhew A, Ramsay CR, Eccles MP, Grimshaw J. The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database Syst Rev 2009; 8 (03) CD001096
    PubMedGoogle Scholar
  • 40 Mutafelija B, Stromberg H. Systematic Process Improvement Using ISO 9001:2000 and CMMI. Norwood, MA: Artech House; 2003
    Google Scholar
  • 41 Sanders D, Burton DA, Prott D. The healthcare analytics adoption model: a framework and roadmap. 2016 . Available at: https://downloads.healthcatalyst.com/wp-content/uploads/2013/11/analytics-adoption-model-Nov-2013.pdf . Accessed September 2, 2019
    PubMedGoogle Scholar
  • 42 Tarakci M, Ates NY, Porck JP, van Knippenberg D, Groenen PJF, de Haas M. Strategic consensus mapping: a new method for testing and visualizing strategic consensus within and between teams. Strateg Manage J 2014; 35: 1053-1069
    CrossrefPubMedGoogle Scholar
  • 43 Okoli C, Pawlowski S. The Delphi method as a research tool: an example, design considerations and applications. Inf Manage 2004; 42: 15-29
    CrossrefPubMedGoogle Scholar