Authors: Robyn Haas, Sarah C. McGill
Horizon Scan reports provide brief summaries of information regarding new and emerging health technologies; Heath Technology Update articles typically focus on a single device or intervention.
These technologies are identified through the CADTH Horizon Scanning Service as topics of potential interest to health care decision-makers in Canada.
This Horizon Scan summarizes the available information regarding an emerging technology, artificial intelligence algorithms, for the prediction of sepsis in adults.
Named as a global health priority in 2017 by the World Health Assembly and WHO, sepsis is a leading cause of morbidity and mortality around the world. Artificial intelligence (AI) algorithms could play important role in addressing this worldwide health threat.
Globally, sepsis affects about 49 million people each year.1 Sepsis is a life-threatening condition that occurs as a result of the body’s overwhelming systemic response to infection, and can cause tissue damage, organ failure, and death.2,3 There is a range in its severity from sepsis to septic shock.2 Sepsis can be difficult to detect because many people present with nonspecific symptoms, which can be easily mistaken for other illnesses (e.g., influenza), and clinical presentations of sepsis can vary from patient to patient.2,3 Timely and accurate detection of sepsis is crucial as every hour that treatment is delayed is associated with an increased risk of mortality.2 AI algorithms have the potential to predict sepsis hours before its onset.2 Harnessing this technology may help clinicians quickly and accurately diagnose sepsis, which could decrease time to treatment, and ultimately improve a person’s chance of recovery.
AI algorithms for the prediction of sepsis are machine learning models that are developed using clinical data collected in electronic health records (EHR).4 The use of AI predictive algorithms is meant to improve the accuracy and timeliness of sepsis prediction in a clinical setting.4 The technology may be particularly useful in intensive care units (ICUs) where sepsis is present in a notable number of patients.3 There are currently numerous AI algorithms in development for this purpose.4 The exact mechanisms of these algorithms vary from model to model, but most typically examine key variables that can play a role in the development of sepsis. These variables may include vital signs, demographics, laboratory tests, comorbidities, and administrative data (e.g., length of stay in the ICU).5 Predictive AI algorithms work by continuously monitoring a patient’s EHR data and assessing the aforementioned variables in real time to determine the likelihood that sepsis will occur.4 Earlier and more accurate sepsis prediction allows for timely clinical intervention, such as the administration of antibiotics.4 By optimizing the management of sepsis, AI predictive algorithms can contribute to a reduction in sepsis-associated morbidity and mortality.4
All individuals who have an existing infection are at risk for developing sepsis. As of 2011, sepsis was involved in 1 in 18 deaths in Canada.6 In 2020, sepsis was the 14th leading cause of death in Canada among all age groups,7 and the in-hospital rate of sepsis was 4.3 per 1,000 patients.8 Some populations are at a higher risk of sepsis and may experience greater benefits from AI predictive algorithms. Examples of risk factors include chronic conditions, immunodeficiency, pregnancy, and use of immunosuppressive drugs.6 Additionally, sepsis-associated mortality increases with age, except for in individuals under the age of 1, who have an increased risk of developing sepsis.6 Health care providers might also benefit from the use of AI algorithms as this technology is meant to assist in clinical decision-making regarding the management of sepsis, including timing of treatment initiation and choice of appropriate antibiotics.
In Canada, AI algorithms to predict sepsis are not currently authorized to be integrated into hospitals’ clinical workflows. There are several algorithms under development; however, this technology may still be some time away from real-world application. Researchers at McMaster University recently created an AI algorithm called BiLSTM (Bidirectional Long Short-Term Memory). They planned to launch a pilot project to investigate its clinical effectiveness in November 2021.9
Some of the AI algorithms currently in development outside of Canada are InSight, DeepAISE, and Sepsis Watch. InSight was developed by Dascena, a start-up company based in the US, and has been investigated in both a randomized controlled trial and a post-marketing study.10 DeepAISE was developed by a research lab at the University of California, San Diego, and is seeking prospective validation.11 Sepsis Watch was developed by Duke University and has been implemented in hospitals across the Duke University Health System.12
Information on the cost of AI algorithms for the prediction of sepsis was not found.
Sepsis itself is a costly condition. Health care costs for people with sepsis are generally higher than for those without sepsis.13 Additionally, rehospitalization rates and length of hospital stay in the ICU are higher among patients with sepsis than those without sepsis.13 There is evidence to suggest that early and accurate prediction of sepsis using machine AI models has the potential to decrease both rehospitalization rates and length of hospital stay.14,15 Therefore, the use of AI predictive algorithms may result in reduced health care costs.
In 2021, the Surviving Sepsis Campaign released updated international guidelines for the management of sepsis and septic shock. These guidelines include screening for patients with sepsis and septic shock; the guidelines also discuss prediction using rules-based tools or scoring systems.16 Common screening tools include systemic inflammatory response syndrome (SIRS) criteria, National Early Warning Score (NEWS), Modified Early Warning Score (MEWS), and quick Sequential Organ Failure Score (qSOFA) or Sequential Organ Failure Score (SOFA).16 As of 2021, the Surviving Sepsis Campaign recommends against the use of qSOFA as a single-screening tool.16 These screening tools use manual methods or are digital, automated assessment tools that use EHR data to assess the risk of sepsis occurring.16 These tools differ from AI algorithms, which go beyond risk assessment and aim to identify sepsis itself. They can be used in numerous locations, including the ICU, emergency departments, or in-patient wards.16 The guidelines note that sensitivity and specificity of the screening tools vary, but that their use is an important part of early sepsis identification.16 The strength of the recommendations for screening is strong, with moderate-quality evidence.
Several retrospective studies have investigated the ability of AI algorithms to predict sepsis hours before its onset in adult patients staying in the ICU. While results are generally favourable for the use of this technology, 2 systematic reviews of these studies have found a range in the predictive performance of these models.4,17 Even with the variation in performance, 1 review concluded that AI models could accurately predict sepsis onset in retrospective data.17 The other review found similar variation in AI model performance and concluded that the studied AI models had better predictive performance than the existing SIRS, MEWS, SOFA, and qSOFA scoring systems.4
One prospective clinical outcomes evaluation examined the effects of an AI algorithm for the prediction of severe sepsis on in-hospital mortality, length of stay, and 30-day readmission.14 Researchers collected data from EHR systems across 9 hospitals over 2 years.14 Following implementation of an AI algorithm, researchers found a 39.50% reduction in in-hospital mortality, a 32.27% reduction in length of stay, and a 22.74% reduction in 30-day readmission.14 A randomized controlled trial also used a machine learning–based sepsis prediction system to evaluate average length of stay and in-hospital mortality in adults.15 Researchers found that patients in the experimental group had a shorter average length of stay and reduced in-hospital mortality than those in the control group.15 Results from both studies suggest implementation of an AI algorithm to predict sepsis has the potential to improve patient-related outcomes.
A qualitative study examined the factors that influence the integration of an AI algorithm for the prediction of sepsis into clinical workflows.18 Through semi-structured interviews with physicians and nurses in the emergency department, researchers found that perceived utility and trust, the implementation process of the algorithm, and workforce considerations were notable factors influencing technology integration.18 Therefore, to maximize the adoption of the technology, these are areas that may be useful to target for education and learning.
No evidence on the safety of AI algorithms for the prediction of sepsis was identified in the literature.
While there is not yet information on the exact cost of AI algorithms for the prediction of sepsis, this technology will not be free. Given the widespread nature of sepsis, affordable technologies would be needed if AI algorithms are adopted for use. This would help to ensure hospitals across all settings could access predictive AI algorithms and ensure equitable patient care.
There may be technical limitations to integrating AI algorithms into hospital information systems. AI technologies tend to require a lot of data and a robust IT infrastructure to run, which existing information systems may not possess.19,20
Finally, clinician perception and acceptance of AI algorithms may be a barrier to the successful implementation of these systems. Results of a 2019 study found that overall perception of 1 machine learning–based early warning system were poor, where more than half of respondents did not report any change in their evaluation of a patient’s risk of sepsis.21 AI algorithms are intended to be decision support tools for clinicians; therefore, a lack of acceptance from clinicians may hinder both the implementation and intended utilization of this technology.
AI models to predict in-hospital mortality due to sepsis in patients the ICU are currently under development. Using retrospective data, researchers are investigating the predictive performance of various AI models.22-24 Research in this area is in the early stages of development and is focused on identifying the best-fitting type of model.22-24 Accurate prediction of mortality may help clinicians intervene and choose appropriate treatment methods.22 It may also lead to better prognoses for patients.22
AI technology for the prediction of sepsis is being investigated for infant populations as well as adult populations. In 2019, researchers performed a retrospective study of infants who were hospitalized for 48 hours or more in the neonatal intensive care unit.25 They used variables from electronic health record data to develop 8 machine learning models and found that 6 of them had strong abilities with respect to predicting sepsis.25 Researchers concluded that the models were able to identify infants with sepsis hours before clinical recognition was possible.25
The creation of AI algorithms to predict sepsis is a field undergoing rapid development. Numerous models are being created and tested; however, there are a notable lack of prospective studies. There is a need for further research on the integration of this technology in clinical settings. Prospective clinical studies can provide information regarding the effect of AI algorithms on patient-related outcomes. Studies that examine the implementation of this technology in a variety of settings (e.g., ICU, emergency departments, in-patient wards) would also be beneficial as they may demonstrate where the use of this technology is most effective. Clinical studies can also provide insight into how health care providers interact with this technology, and the degree to which it influences practice. Perspectives of clinicians can provide unique insight into the development and implementation of AI algorithms for the prediction of sepsis. Finally, while AI predictive algorithms are intended to be support tools for clinicians, who ultimately guide the course of treatment, it may also be valuable to explore patient perspectives on the use of this technology.
1.Rudd KE, Johnson SC, Agesa KM, et al. Global, regional, and national sepsis incidence and mortality, 1990-2017: analysis for the Global Burden of Disease Study. The Lancet. 2020;395(10219):200-211. PubMed
2.Persson I, Ostling A, Arlbrandt M, Soderberg J, Becedas D. A machine learning sepsis prediction algorithm for intended intensive care unit use (NAVOY Sepsis): proof-of-concept study. JMIR Form Res. 2021;5(9):e28000. PubMed
3.Moor M, Rieck B, Horn M, Jutzeler CR, Borgwardt K. Early prediction of sepsis in the ICU using machine learning: a systematic review. Frontiers in Medicine. 2021;8:607952. PubMed
4.Islam MM, Nasrin T, Walther BA, Wu CC, Yang HC, Li YC. Prediction of sepsis patients using machine learning approach: A meta-analysis. Comput Methods Programs Biomed. 2019;170:1-9. PubMed
5.Hassan N, Slight R, Weiand D, et al. Preventing sepsis; how can artificial intelligence inform the clinical decision-making process? A systematic review. Int J Med Inform. 2021;150:104457. PubMed
6.Navaneelan T, Alam S, Peters PA, Phillips O. Deaths involving sepsis in Canada. Statistics Canada - Health at a Glance. 2016. https://www150.statcan.gc.ca/n1/pub/82-624-x/2016001/article/14308-eng.htm. Accessed 2022 Feb 22.
7.Statistics Canada. Leading causes of death, total population, by age group. Table 13-10-0394-01. 2022 Jan 24; https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1310039401. Accessed 2022 Feb 17.
8.Canadian Institute for Health Information (CIHI). In-hospital sepsis details for New Brunswick. 2022; https://yourhealthsystem.cihi.ca/hsp/indepth?lang=en#/indicator/016/2/C300/. Accessed 2022 Feb 17.
9.McMaster DeGroote School of Business, Research Institute of St. Joe's Hamilton. Researchers create AI algorithm to improve timeliness, accuracy of sepsis predictions. 2021; https://www.degroote.mcmaster.ca/articles/researchers-create-ai-algorithm-to-improve-timeliness-accuracy-of-sepsis-predictions/. Accessed 2022 Feb 24.
10.Dascena. Predictive algorithms for better care: InSight. 2022; https://www.dascena.com/insight. Accessed 2022 Feb 15.
11.Nemati S, Holder A, Razmi F, Stanley MD, Clifford GD, Buchman TG. An interpretable machine learning model for accurate prediction of sepsis in the ICU. Crit Care Med. 2018;46(4):547-553. PubMed
12.Duke Health. Duke’s augmented intelligence system helps prevent sepsis in the ED. 2020: https://physicians.dukehealth.org/articles/dukes-augmented-intelligence-system-helps-prevent-sepsis-ed. Accessed 2022 Feb 17.
13.Farrah K, McIntyre L, Doig CJ, et al. Sepsis-associated mortality, resource use, and healthcare costs: A propensity-matched cohort study. Crit Care Med. 2021;49(2):215-227. PubMed
14.Burdick H, Pino E, Gabel-Comeau D, et al. Effect of a sepsis prediction algorithm on patient mortality, length of stay and readmission: a prospective multicentre clinical outcomes evaluation of real-world patient data from US hospitals. BMJ Health Care Inform. 2020;27(1). PubMed
15.Shimabukuro DW, Barton CW, Feldman MD, Mataraso SJ, Das R. Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial. BMJ Open Respir Res. 2017;4(1):e000234. PubMed
16.Evans L, Rhodes A, Alhazzani W, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021. Intensive Care Med. 2021;47(11):1181-1247. PubMed
17.Fleuren LM, Klausch TLT, Zwager CL, et al. Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy. Intensive Care Med. 2020;46(3):383-400. PubMed
18.Sandhu S, Lin AL, Brajer N, et al. Integrating a machine learning system into clinical workflows: Qualitative study. J Med Internet Res. 2020;22(11):e22421. PubMed
19.Garnacho-Montero J, Martin-Loeches I. Clinical management of sepsis can be improved by artificial intelligence: no. Intensive Care Med. 2020;46(2):378-380. PubMed
20.Sendak MP, Ratliff W, Sarro D, et al. Real-world integration of a sepsis deep learning technology into routine clinical care: implementation study. JMIR Med Inform. 2020;8(7):e15182. PubMed
21.Ginestra JC, Giannini HM, Schweickert WD, et al. Clinician perception of a machine learning-based early warning system designed to predict severe sepsis and septic shock. Crit Care Med. 2019;47(11):1477-1484. PubMed
22.Kong G, Lin K, Hu Y. Using machine learning methods to predict in-hospital mortality of sepsis patients in the ICU. BMC Med Inform Decis Mak. 2020;20(1):251. PubMed
23.Garcia-Gallo JE, Fonseca-Ruiz NJ, Celi LA, Duitama-Munoz JF. A machine learning-based model for 1-year mortality prediction in patients admitted to an Intensive Care Unit with a diagnosis of sepsis. Med Intensiva. 2020;44(3):160-170. PubMed
24.Wernly B, Mamandipoor B, Baldia P, Jung C, Osmani V. Machine learning predicts mortality in septic patients using only routinely available ABG variables: a multi-centre evaluation. Int J Med Inform. 2021;145:104312. PubMed
25.Masino AJ, Harris MC, Forsyth D, et al. Machine learning models for early sepsis recognition in the neonatal intensive care unit using readily available electronic health record data. PLoS One. 2019;14(2):e0212665. PubMed
ISSN: 2563-6596
Disclaimer: The information in this document is intended to help Canadian health care decision-makers, health care professionals, health systems leaders, and policy-makers make well-informed decisions and thereby improve the quality of health care services. While patients and others may access this document, the document is made available for informational purposes only and no representations or warranties are made with respect to its fitness for any particular purpose. The information in this document should not be used as a substitute for professional medical advice or as a substitute for the application of clinical judgment in respect of the care of a particular patient or other professional judgment in any decision-making process. The Canadian Agency for Drugs and Technologies in Health (CADTH) does not endorse any information, drugs, therapies, treatments, products, processes, or services.
While care has been taken to ensure that the information prepared by CADTH in this document is accurate, complete, and up to date as at the applicable date the material was first published by CADTH, CADTH does not make any guarantees to that effect. CADTH does not guarantee and is not responsible for the quality, currency, propriety, accuracy, or reasonableness of any statements, information, or conclusions contained in any third-party materials used in preparing this document. The views and opinions of third parties published in this document do not necessarily state or reflect those of CADTH.
CADTH is not responsible for any errors, omissions, injury, loss, or damage arising from or relating to the use (or misuse) of any information, statements, or conclusions contained in or implied by the contents of this document or any of the source materials.
This document may contain links to third-party websites. CADTH does not have control over the content of such sites. Use of third-party sites is governed by the third-party website owners’ own terms and conditions set out for such sites. CADTH does not make any guarantee with respect to any information contained on such third-party sites and CADTH is not responsible for any injury, loss, or damage suffered as a result of using such third-party sites. CADTH has no responsibility for the collection, use, and disclosure of personal information by third-party sites.
Subject to the aforementioned limitations, the views expressed herein are those of CADTH and do not necessarily represent the views of Canada’s federal, provincial, or territorial governments or any third-party supplier of information.
This document is prepared and intended for use in the context of the Canadian health care system. The use of this document outside of Canada is done so at the user’s own risk.
This disclaimer and any questions or matters of any nature arising from or relating to the content or use (or misuse) of this document will be governed by and interpreted in accordance with the laws of the Province of Ontario and the laws of Canada applicable therein, and all proceedings shall be subject to the exclusive jurisdiction of the courts of the Province of Ontario, Canada.
The copyright and other intellectual property rights in this document are owned by CADTH and its licensors. These rights are protected by the Canadian Copyright Act and other national and international laws and agreements. Users are permitted to make copies of this document for non-commercial purposes only, provided it is not modified when reproduced and appropriate credit is given to CADTH and its licensors.
About CADTH: CADTH is an independent, not-for-profit organization responsible for providing Canada’s health care decision-makers with objective evidence to help make informed decisions about the optimal use of drugs, medical devices, diagnostics, and procedures in our health care system.
Funding: CADTH receives funding from Canada’s federal, provincial, and territorial governments, with the exception of Quebec.
Questions or requests for information about this report can be directed to Requests@CADTH.ca.