The first report on ventilation in the prone position in ARDS was published in 1976.1 The approach was appreciated by intensivists because improved oxygenation was observed in most cases. However, no evidence was given to show that the method resulted in a higher rate of survival.2 Moreover, pronation of critically ill patients exposed them to several risks and required skilled specialists to be performed safely. For these reasons, risk and benefit assessments were called for.

The efficacy of prone ventilation In ARDS was investigated in several trials published between 2001 and 2009.3–6 Although all unsuccessful, they helped researchers to generate hypotheses concerning what patients would benefit the most from the treatment. They also helped them to greatly improve the treatment.7 Guerin and his collaborators designed the PROSEVA trial taking case-mix selection as well as pronation and ventilatory protocols into account, capitalizing on the experience gained through the above-mentioned previous trials.8 These specific features had not been sufficiently brought out by the standard meta-analyses used to investigate prone ventilation in ARDS after 2013. Nine of these meta-analyses were published between 2014 and 2021,9–17 each of which used between six and eleven randomized controlled trials (RCTs) selected from a pool of thirteen.3–6,8,18–25 The efficacy of this treatment, however, was only weakly supported by evidence from the meta-analyses carried out after the PROSEVA study was published. The findings of eight of these were non-significant, although in various subgroups (e.g., patients with more severe hypoxemia, subjected to prone ventilation for a longer time than others were, or treated with protective ventilation) a significant reduction in mortality was observed. In particular, the Cochrane meta-analysis provided only a rather feeble indication for prone ventilation in the subset of patients with severe hypoxemia. This is no surprise, given the exploratory nature of subgroup analyses in meta-analysis, where studies are assigned to a subgroup according to a specific feature, without accounting for the distribution of other confounders.

Using cumulative meta-analysis and an analysis of heterogeneity, we tested the hypothesis that the PROSEVA trial is an independent source of evidence for the use of prone positioning in ARDS

Prone ventilation is considered to be standard care for severe ARDS, and has been strongly recommended by recent guidelines based on subgroup analyses of negative meta-analysis and negative trials.37,38 However, the Cochrane handbook regards subgroup analyses as “observational by nature” (9.6.2 What are subgroup analyses?). Indeed, subgroup analyses in meta-analysis are exploratory in nature, since studies are assigned to a subgroup according to a specific feature, without accounting for the distribution of other confounders. Thus, when more confounders than one are present in the subgroup, we may be unable to recognize the ones affecting the outcome.

Meta-regression was also used in three studies to try to identify the subset of patients who responded to prone ventilation.13,14,17 However, it has the same limitations as subgroup analysis (Cochrane handbook 10.11.6 Interpretation of subgroup analyses and meta-regressions). Only one variable at a time is tested as a moderator when the number of studies is too small to adopt a multivariable approach (as a rule of thumb, ten studies are required for each moderator). Therefore, the information provided by these studies should not be regarded as evidence but preferably as a possible basis for an hypothesis.

Thus, evidence in support of prone ventilation in ARDS is inherently weak if we rely on meta-analytical approaches, implicitly prioritizing them with respect to the PROSEVA trial.

On the other hand, the standard assessment of heterogeneity by means of I2 is of limited usefulness when dealing with only a few studies, as in our case. If we only looked at the I2 estimates, which were high in the nine meta-analyses (Table 1), we could wrongly conclude that they showed high heterogeneity. Nevertheless, we considered heterogeneity assessment using the I2 to be inconclusive because confidence intervals ranged between low and high heterogeneity. In our case, lack of power diminished the usefulness of I2. However, when numerous large studies are meta-analysed, the opposite problem may come up, with clinically homogeneous studies (i.e., with similar case-mixes, treatments and controls) turning out to be statistically heterogeneous. Hence, clinical reasoning should always be used to support and integrate statistical analyses.

Given these premises, the use of a leave-one-out approach to investigate outliers turned out to be a winning strategy for testing our hypothesis that the PROSEVA trial was clinically too different from the previous trials to be meta-analytically combined with them. However, none of the nine meta-analyses adopted this strategy, although Cochrane's handbook does recommend meta-analyses both with and without clearly outlying studies (9.5.3 Strategies for addressing heterogeneity).

Only one study investigated outliers,17 using studentized residuals.39 However, seeing PROSEVA trial was an outlier did not substantially affect its conclusions.

The use of a leave-one-out strategy allowed us to investigate heterogeneity in greater detail, providing evidence in support of our hypothesis by showing that the higher heterogeneity was mainly attributable to the PROSEVA trial.

There were good clinical reasons for not combining the PROSEVA trial with the other studies. The main four RCTs that were published earlier had been unsuccessful in demonstrating improved survival (ESM Figs. 1 and 2).3–6 However, progress in physiological knowledge regarding ventilation in ARDS led to improvements in how the trials were designed.40 Thus, it was possible to hypothesize that patients with the most severe hypoxia would benefit the most from pronation,41 the duration of pronation needed to be prolonged, and protective ventilation needed to be associated with prone ventilation. The PROSEVA trial was designed on the basis of these hypotheses, which make it inherently different from previous RCTs, as Gattinoni et al. have shown.7

However, when trials bearing differences in case-mix, study treatment and other associated treatments that may modify the outcome are combined in a meta-analysis, the risk of generating spurious findings is high.42

Our study supports the hypothesis that the PROSEVA trial is an outlier compared to the RCTs that preceded it. Our cumulative meta-analyses show that the PROSEVA trial changes the estimate of effect strikingly. Moreover, given the strong protective effect of the PROSEVA trial compared to the other trials, the exclusion of the PROSEVA trial from any meta-analysis markedly increases the Cochran's Q p value and the p value for effectiveness. Finally, the protective effect of the PROSEVA trial was formally confirmed by interaction tests (Table 2).

Several topics in medicine have been investigated over long periods of time as knowledge of these topics evolved. Thus, the case of the PROSEVA trial should not be considered an exception. When it does happen, attention should be paid to clinical heterogeneity before studies are selected for meta-analysis. The problem of heterogeneity across studies, typical for aggregated data, could be offset by sharing trial datasets making individual-patient granular information public. Individual-patient data could be analysed with powerful multivariable and machine-learning tools, as has recently been done for ARDS phenotypes.43

Weak evidence provided by meta-analyses may have fostered the low rate of prone ventilation prescriptions in severe ARDS reported in literature.45 Our study brings out the limitations of the meta-analytical approach in treating ARDS and strongly reinforces Cochrane's recommendation that a one-by-one study exclusion analysis be performed when potential outlier studies are identified.

For clinical recommendations, the findings of the PROSEVA trial should be regarded as an independent source of evidence.

Research for the present paper has not been funded by any grant from funding agencies.

This research has not received any grant from funding agencies in the public, commercial or not-for-profit sectors. non-profit sectors.

Daniele Poole had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Daniele Poole, Roberto Fumagalli and Andrea Pisa contributed to the interpretation of studies and the writing of the manuscript.

D. Poole: Conceptualization, Writing – review & editing, Data curation, Formal analysis, Methodology, Writing – original draft. A. Pisa: Conceptualization, Writing – review & editing. R. Fumagalli: Conceptualization, Writing – review & editing.