Introduction: Acute hypoxemic respiratory failure (RF) is the hallmark symptom of COVID-19 infection that can lead to escalating oxygenation requirements and critical care utilization. Early identification of patients at risk for respiratory decompensation can facilitate timely resource deployment and mitigate care delays. To facilitate resource alignment with patient needs, a machine learning (ML) model designed to predict RF for COVID-19 tested patients who present to the emergency department (ED) was developed, integrated, and prospectively validated.
Methods: All patients ≥18 years of age presenting to any BJC ED with a COVID-19 PCR/antigen test performed ≤ 14 days before or 7 days after the visit were included (time criteria for model development). RF was defined by use of high humidity nasal cannula, non-invasive or invasive mechanical ventilation within 48 hours of ED arrival. The model was developed using encounter data from 3/6/20-6/8/20. Features were narrowed using a lasso regression. A logistic regression was trained using a 75%:25% test: train split with 5-fold cross validation. The model was implemented within Epic and prospectively ran from 12/23/20-4/8/21.
Results: Of the 11,558 patients in the development cohort, 566(4.9%) developed RF. These patients were more likely to be older (66.1[54.5-74.2] vs. 55.2[36.7-69.3], p<0.01), male (331[58.5%] vs. 5,183[47.2%], p<0.01) and have a higher BMI (34.9[27.0-40.9] vs. 28.1[23.6-34.0], p<0.01). Twelve features were selected by lasso regression and the retrospective area under the receiver operating curve ([AU]ROC) and AU the precision recall curve (AUPRC) was 0.80(0.794-0.811) and 0.28(0.26-0.29), respectively. Prospectively, the AUROC and AUPRC were 0.76 and 0.13, respectively. Patients exceeding the risk threshold were 3.5x and 3.9x more likely to develop RF (165[13.6%] vs. 401[3.9%], p<0.01) and require ICU transfer (238[19.6%] vs. 520[5.0%], p<0.01) within 48 hours.
Impact: Integration of simple ML-based prediction models within the EHR is feasible and yields relatively accurate, real-time predictions. Changes in prevalence, population and treatment may require ongoing algorithm monitoring and updating.
Organization – Washington University in St. Louis
Yu SC, Guo X, Haber G, Gupta A, Kannampallil T, Lai A, Payne PRO, Kollef M, Michelson AP