The outcome to be predicted was ΔPes (in cmH2O). The candidate predictors were all measured within a few minutes of ΔPes by the same investigators. They included age, sex, diagnosis of the coronavirus disease 2019 (COVID-19), heart rate, mean arterial pressure, respiratory rate, arterial pH (pHa), arterial carbon dioxide tension (PaCO2), arterial oxygen tension (PaO2), arterial bicarbonate concentration (HCO3−a), arterial base excess concentration (BEa), arterial oxygen saturation (SaO2), and arterial tension to the inspiratory fraction of oxygen ratio (PaO2:FiO2). The product term or interaction between COVID-19 and PaO2:FiO2 was also considered based on scientific reasoning.42 Unfortunately, other potentially relevant predictors, such as the recruitment of accessory muscles or the severity of dyspnea, were unavailable in the dataset. Only 260 patients with complete data were considered for further analysis. The proportion of patients with missing data (5 %) was deemed negligible and may not have yielded significant biases.

Data are reported as median (Q1-Q3) and count (percentage).

Firstly, we studied the association between each candidate predictor and outcome separately with univariable linear regression. We used fractional polynomial regression analysis to explore potentially meaningful non-linear associations. We selected the most appropriate second-degree function by minimizing deviance in polynomial regression. However, we observed only a slight improvement in the model fit. To maintain simplicity and avoid overfitting, we opted to develop a model based on linear associations between the candidate predictors and the outcome. Afterward, we built a multivariable linear regression model. In cases of high correlation among candidate predictors (i.e., Spearman's rho >│0.80│), we excluded the variable that was least important based on mechanistic reasoning. Predictors were selected using a backward stepwise elimination strategy, with a p < 0.05 at the Wald test as the stopping rule. Based on the residual analysis, the linearity and normality assumptions were met, but the equal variance assumption was not. Therefore, we computed robust standard errors for the regression coefficients. The fit of the model was evaluated both in terms of calibration and overall performance. Calibration was assessed by plotting the observed ΔPes against the predicted ΔPes. In a well-fitting model, the predictions cluster around the 45° diagonal, or identity line, and the slope of the plot is close to 1.41,43 Overall performance was evaluated with the adjusted R2, which reflects the amount of outcome variability explained by the model adjusted for the number of predictors included.41,43

We built a multivariable logistic regression model using the same candidate predictors and variable selection technique mentioned previously. The dichotomous outcome to be predicted was strong breathing efforts, defined as those with a ΔPes >10 cmH2O as in our previous work.22 The apparent predictive performance of the model was assessed in terms of calibration, discrimination, and overall performance.41,43 Calibration was evaluated by plotting the observed proportion of patients with strong breathing efforts against the predicted risk. In a perfectly calibrated model, this plot has an intercept of 0 and a slope of 1. Discrimination was assessed with the area under the receiver operating characteristics curve (AUROC). This area can range from 0 to 1, where 0.5 indicates random guessing, and 1 indicates perfect discrimination. Overall performance was assessed with the scaled Brier score, computed as 1 – Brier score / Brier score max. It ranges from 0 % to 100 %, with higher values indicating a better-performing model.41,43

When testing performance, using the same dataset used to develop a model often leads to overly optimistic results. To obtain unbiased estimates, we used a bootstrap resampling technique. Firstly, we repeated the entire modeling process, including variable selection, on 200 bootstrap samples drawn with replacement from the original dataset. Secondly, we computed the optimism of each bootstrap model by comparing its performance in the same bootstrap sample or the original dataset. Finally, we subtracted the average optimism of the 200 bootstrap samples from the initial apparent performance of the model under validation to obtain its optimism-corrected performance.41,43

We created a score by dividing each regression coefficient by the smallest one and produced a table to transform each possible total point score into an outcome probability.41,43

All statistical analyses were run with Stata/SE 17.0 (StataCorp LLC; College Station, TX). A two-tailed p-value <0.05 was considered statistically significant.

Other details on methods are presented in the supplementary material.

In univariable linear regression analysis, ΔPes decreased with a diagnosis of COVID-19 but increased with a higher respiratory rate, higher heart rate, and lower PaCO2, PaO2, HCO3−a, BEa, SaO2, and PaO2:FiO2. It was not strongly associated with age, sex, mean arterial pressure, and pHa. The product term between COVID-19 and PaO2:FiO2 was significant (Table 2).

In multivariable model development, SaO2 and HCO3−a were excluded because of collinearity issues with PaO2 and BEa. Age, sex, heart rate, mean arterial pressure, pHa, PaCO2, and PaO2 were eliminated by backward selection. A diagnosis of COVID-19, BEa (mmol/L), respiratory rate (bpm), and PaO2:FiO2 (mmHg), and the product term between COVID-19 and PaO2:FiO2 remained in the model (Table 2). We found that in patients without COVID-19, ΔPes (in cmH2O) can be estimated as 14.25 – 0.52 × BEa + 0.36 × respiratory rate – 0.05 × PaO2:FiO2. In those with COVID-19, ΔPes (in cmH2O) can be estimated as 3.52 – 0.52 × BEa + 0.36 × respiratory rate – 0.01 × PaO2:FiO2. The calibration slope was 1, and the adjusted R2 was 0.39 (Fig. 1 and Fig. A.1).

Fig. 1.

Calibration plot of observed against predicted outcomes.

Panel A shows the calibration plot for the linear regression model for ΔPes (in cmH2O) tested on the development population, while Panel B shows the calibration plot for the logistic regression model for breathing efforts with a ΔPes >10 cmH2O tested on the development population. Each dot on the plot represents a tenth of the predicted values, each one based on data from 26 patients. The bars represent the 95 % confidence intervals, and the red line is the identity line.

(0.17MB).

On univariable logistic regression analysis, variables associated with the risk of strong breathing efforts were the same as those associated with ΔPes, except for heart rate and the product term between COVID-19 and PaO2:FiO2, which were not significant (Table 3).

During multivariable model development, BEa (mmol/L), respiratory rate (bpm), and PaO2:FiO2 (mmHg) remained in the model while all other variables, including a diagnosis of COVID-19 and the product term between COVID-19 and PaO2:FiO2, were removed (Table 3). We found that the individual probability of outcome can be calculated as 1 / {1 + exp [– (0.461 – 0.145 × BEa + 0.075 × respiratory rate – 0.014 × PaO2:FiO2)]}. The corresponding percentage risk can be obtained by multiplying this probability by 100.

When the fit of the model was tested on the same data set used to develop it (apparent performance), the calibration intercept was −0.00 (−0.29 to 0.29), the calibration slope was 1.00 (0.72 to 1.28), the AUROC was 0.79 (95 % CI, 0.73–0.85), and the scaled Brier score was 29 % (Fig. 1). On internal validation (optimism-corrected performance), the calibration intercept was 0.00 (−0.33 to 0.33), the calibration slope was 0.85 (0.60 to 1.12), the AUROC was 0.76 (0.71–0.81), and the scaled Brier score was 23 % (Fig. A.2). The prediction model was simplified into the following score: 33.7 – 10.6 × BEa + 5.5 × respiratory rate – PaO2:FiO2. The outcome probabilities of different total points scores are reported in Fig. A.3.

This was our first attempt at filling this gap in research. We developed two simple but composite models to estimate the effort of breathing during high-flow oxygen therapy when esophageal manometry is unavailable. One model predicts the actual ΔPes while the other predicts the risk of ΔPes being >10 cmH2O. The association between the selected variables and breathing effort is biologically plausible.1,2 Other variables, such as PaCO2, had some predictive value on their own but did not ameliorate model performance. We use the acronym BREF to refer to these models. BREF stands for BReathing EFfort but also reminds the common predictors: BEa (B), respiratory rate (RE), and PaO2:FiO2 (F). Doctors may use these models, as well as any updated version, as an aid for their decision-making process. For instance, to determine whether a patient requires admission to the intensive care unit, how often he or she should be reassessed, and whether to increase ventilatory support or proceed to intubation. Those with limited experience or working outside the ICU may find them particularly helpful.50 In other fields of medicine, clinical prediction rules are used this way.51,52 Results may be interpreted as follows: breathing efforts with ΔPes up to 10 cmH2O are probably safe; those with ΔPes from 11 to 15 cmH2O warrant close monitoring as they can lead to fatigue; those with ΔPes higher than 15 to 20 cmH2O should not be tolerated for long. These values reflect our current best practices and may be revised as new knowledge becomes available.

This work also has several limitations that need to be addressed. Firstly, our models require performing an arterial blood gas analysis, which is invasive and resource-consuming. Secondly, the performance was only moderately good and needs to be validated in a new study population.41,43 However, the models may be updated into a second version, including other predictors, and this will hopefully improve their accuracy. Thirdly, the clinical usefulness of any prediction model depends on the accuracy of the unaided doctor and the consequences of decisions based on it, which are yet to be determined. Fourthly, these models may not be suitable for patients excluded from our study population, including those with acute cardiogenic pulmonary edema or a history of chronic lung disease. Lastly, having a dichotomous outcome can facilitate the interpretation of the results but with many drawbacks.41,43 For instance, it carries the risk of characterizing patients with a ΔPes slightly lower or higher than 10 cmH2O as being very different rather than very similar. The exact threshold of ΔPes that separates normal and strong efforts, if such a threshold exists, remains unknown. Various suggestions have been proposed, ranging from 8 to 18 cmH2O (see Table A.1), but none of them is supported by solid evidence. Also, it may cause a loss of information, which may explain why a diagnosis of COVID-19 was associated with ΔPes at linear regression analysis but not with strong breathing efforts at logistic regression analysis. For these reasons, dichotomizing a continuous outcome is generally discouraged.41,43 Hence, our model for ΔPes in cmH2O should be preferred over the other.