We included hospitalized patients over 18 years of age, with positive Reverse Transcriptase Polymerase Chain Reaction (RT-PCR) for SARS-CoV-2 and oxygen requirements that demanded HFO. Criteria for initiation of HFO were oxygen saturation lower than 92% with conventional oxygen therapy (COT) (reservoir mask at 15 L/min), and tachypnea (> 30 breaths per minute) and/or self-reported rest dyspnea despite COT. All patients received a protocolized treatment which consisted of dexamethasone (6 mg IV, QD) plus baricitinib (4mg PO, QD) for 10 days and those with interleukin-6 (IL-6) higher than 80 pg/mL at third day of hospital stay and C-reactive protein higher than 7.5 mg/dL without respiratory improvement received tocilizumab (8mg/kg/dose IV, BID).

We analyzed demographic characteristics that included: age, sex, body mass index (BMI), estimated glomerular filtration rate (eGFR), smoking status and comorbidities (type 2 diabetes, hypertension, chronic kidney disease and cardiac disease, COPD/Asthma, cancer, cardiovascular disease (CVD), and autoimmune conditions). eGFR was calculated using serum creatinine, with the CKD-EPI formula. All comorbidities were recorded by self-reported or by a proxy if patients were not able to provide their own history. Clinical characteristics included: number of days with symptoms prior to admission, oxygen saturation at room air upon admission, SAFI (saturation/fraction of inspired oxygen index), the CALL score, a COVID-19-specific score to predict disease progression that includes comorbidity, age, lymphocytes and lactate dehydrogenase (LDH),12 severity scores such as Sequential Organ Failure Assessment (SOFA),13 Pneumonia Severity Index (PSI),14 CURB-65,15 National Early Warning Score 2 (NEWS 2)16,17; days with HFO, length of hospital stay in days, ICU admission and length of stay in ICU, number of days since admission until HFO initiation. Respiratory parameters such as oxygen saturation, respiratory rate per minute, fraction of inspired oxygen (FiO2) and the Respiratory rate and Oxygenation (ROX)18 index were recorded upon admission, at the time of HFO, and 24 hours post HFO initiation. Superimposed bacterial infections were also documented at any time during hospitalization.

Admission laboratory tests that were analyzed included: complete blood count, LDH, C-reactive protein (CRP), procalcitonin, IL-6, D- dimer, brain natriuretic peptide (BNP), ferritin, and highly sensitive (HS) troponin.

Stata IC-16 was used to conduct statistical analyses. For categorical variables, frequencies and percentages are shown; for continuous variables, according to normal or non-normal distribution, mean and standard deviation or median and interquartile range (IQR), respectively, are shown. Chi-squared and t-test or U Mann-Whitney were used for comparisons between both groups. No imputation methods were utilized for data missingness. An alpha of 5% was set as threshold for statistical significance. The primary endpoint was IMV, or death while on HFO.

Demographic and clinical covariates associated with the endpoint were considered to construct the score. Cox proportional hazard models were employed to estimate the risk for the primary outcome by each category of the score. The proportional hazards assumption was tested. Kaplan-Meier (KM) estimates were used to display cumulative incidence of the primary endpoint, where time zero is the day the patient was placed on HFO. Results from the models are expressed as hazard ratios (HR) and 95% confidence intervals (CI). Receiver operating characteristic (ROC) curves were utilized to assess the performance of our score compared to other utilized scores. Area under the curve (AUC) for different scores were compared using DeLong's test. A classic 1,000- replication bootstrapping method was used to validate the score. Normal-based CI are reported in our HR estimate.

Sensitivity analysis using IMV as one endpoint and using all-cause mortality as another endpoint was performed. A cox proportional hazards model was employed to assess the average increment in risk for each endpoint by each increasing category in the R4 score.

Significant variables from Table 1 were imputed into a univariate model for predicting the primary endpoint. Due to low prevalence of comorbidities (CVD, COPD, cancer, autoimmune diseases, and hypothyroidism) were concatenated into one single variable. Clinical variables associated with the endpoint were selected to construct the score (Table 2). Bacterial infection was removed from the score because it is a consequence IMV. Six variables were included in the score. The score was optimized into 4 categories as follows: 0 criteria, 1 criterion, 2-3 criteria, and ≥4 criteria met. Event count and proportions per R4 score category are displayed in Supplement Table 1.

Median follow-up time in patients who met the primary endpoint was 3 (1-9) days. Each category increase in the R4 score increased likelihood for meeting the endpoint (HR 1.45 [1.31–1.61], p< 0.001). HR and CI by each category, as well as the bootstrapped sample CI are summarized in Table 3 and the KM estimate is displayed in Fig. 1. Sensitivity and specificity for each scoring component and for the R4 categories is summarized in Supplement Table 2. When breaking our primary endpoint into separate outcomes (VMI and death), the average HR for VMI and death by each increase in R4 category were 1.4 [1.26–1.56], p <0.001, and 1.62 [1.42–1.85] p<0.001, respectively.

Figure 1

(0.12MB).

We compared the R4 score against other scoring methods described in the literature using receiver operating characteristic (ROC) curves for our primary endpoint. When comparing the R4 score against other scores, it performed better than CALL (AUC 0.64), SOFA (AUC 0.63) and NEWS2 (AUC 0.57) scores (p= 0.005 and 0.001, respectively). However, it did not perform significantly different than PSI (AUC 0.69), ROX-24 (AUC 0.71) and CURB65 (AUC 0.68) (p= 0.206, 0.647, and 0.113, respectively). When using the traditional cutoff of 4.88 for the ROX score at 24 hours of HFO, the AUC was 0.61 with a sensitivity and specificity of 46 and 78%, respectively. The ROC curves and AUC for each score is provided in Supplement Fig. 2.

Traditional scoring tools have been repurposed for use among COVID-19 inpatients.25 In a cohort of 830 participants with COVID-19 pneumonia, the performance of qSOFA, NEWS2 and CURB-65 was evaluated. All tools lacked prognostic utility and underestimated the mortality rate in their population. The ROX score is a tool that predicts IVM among patients with HFO18 and has been validated in the setting of COVID-19.26,27 Moreover, a recent study demonstrated the value of the ROX in COVID-19 patients, and reiterates it's role in decision-making for clinicians to proceed to IMV.27 This study also suggests that repurposing the ROX score for COVID-19 patients deemed a reevaluation of cutoff values, and found that ROX at 12 hours best predicted IMV when using a ROX cutoff of 5.99.27 In agreement with this study, our data suggests using a higher cutoff value correctly classifies more patients to undergo IMV. Another cohort showed improved performance of the NEWS2 score,16 with an AUC of 0.82 for predicting mortality. The SOFA score has also been repurposed by different groups but its value is inconsistent among different cohorts.28,29 Evidence regarding the use of traditional scores to predict outcomes among COVID-19 inpatients is inconsistent, and likely sensitive to different populations and outcomes. Novel scores developed for COVID-19 disease like the CALL score12 claim high sensitivity and specificity for disease progression. The R4 score proved to perform similarly to PSI, ROX and CURB-65 among our study cohort, while performing better than the SOFA, NEWS2, and CALL scores to predict our primary endpoint. Despite the latter being designed for COVID-19 patients and claiming an AUC of over 0.9 the results did not hold in our cohort, probably in part because we used it for a different endpoint, and we did not have data on HIV status to incorporate into the score. These positive results suggest the R4 score may be used in lieu of, or as a complement of other traditional and novel scores. However, it is important to note this score was evaluated only in the setting of a developmental cohort and reevaluated among our population using bootstrapping, which limits our ability to compare it with other scoring methods.

Our study is limited in that it is a retrospective analysis, thus a casual pathway cannot be determined. Initiation of IMV could have been altered by clinician judgement, as well as ventilator shortage, and ICU availability. Our study does not consider therapies that participants might have received before admission, which may introduce selection bias into our study. Our score was only assessed in a single cohort; it is unclear how it will perform in other cohorts. Comorbidities were assessed by self- or proxy report, which may raise concern for recall bias, exaggerating the effect size. Moreover, with the emergence of SARS-CoV-2 variants, and the implementation of vaccine programs, it is unclear whether the score will perform similarly in populations infected with other variants, and whether vaccines might influence the effect estimates. However, this limitation is present in any scoring tool, until further studies are conducted to evaluate the impact of variants and vaccines on risk tools. The R4 score was not significantly different to some scoring methods in predicting the primary endpoint, although, this may also be seen as a strength, as it may be used in lieu of other scores if information to impute in other tools is insufficient in the clinical setting. Other strengths in our cohort include a large sample size with relatively high event rates, powering our analysis to develop multiple risk stratification categories. Moreover, reliable clinical data collection was accomplished despite having missingness among inflammatory markers, variables used in the score did not suffer from missingness. Additionally, this study employs survival analysis within each R4 score categories to predict our primary endpoint, giving it additional analytical strength.

Inpatients with COVID-19 pneumonia are complex and their outcomes are hard to predict as much about COVID-19 must still be learned. Different tools exist that may be used with limited reliability in predicting outcomes among these patients. The R4 score has proven to be a potentially useful tool that may be complementary to traditional tools to predict IMV or death among COVID-19 patients. It is imperative that this score be validated among another cohort to further understand the implications and utility of this tool. Therefore, we encourage other working groups to validate this tool among other cohorts to better understand the best ways to predict outcomes among COVID-19 patients with HFO.