Today, EBM is the gold standard for validating treatments and is transforming medicine into a scientific discipline.1 Despite ethical and logistical considerations associated with placebo controls,2 there is an ever-growing reliance on randomized double-blind placebo-controlled (RDBPC) trials over observational studies and case series. However, the fact that EBM suggests that RDBPC trials are necessary for high quality treatment recommendations, whereas observational studies and case series receive only low quality ratings, merits further scrutiny.3–5

Sackett et al. define EBM as, “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.”5 Thus, even if a cited treatment is unsupported by a placebo-controlled trial but is still the “best available evidence,” then the treatment would still be supported by EBM. Unfortunately, Sackett and the Cochrane group go on to classify “no controls, case-series only” as Level V evidence that “…contain useful information about clinical course and prognosis but can only hint at efficacy.”4 They further describe the quality of this evidence as low.3 This can lead to errors of interpretation by clinicians who, at least in the field of pulmonary medicine, often justify their overlooking of new, beneficial, and even life-saving approaches because they do not consider the evidence supporting them to be high quality.

Interventions that replace the function of a vital organ (or, for that matter, any organ) cannot be placebo controlled. Such interventions can only be compared for secondary benefits or for their capacity to more effectively normalize physiological parameters. For instance, although invasive continuous tracheostomy mechanical ventilation and continuous noninvasive ventilatory support (CNVS) cohorts can be compared to each other for secondary benefits and untoward effects, in the absence of measurable vital capacity (VC), they cannot be matched with placebo controls any more than can parachute use be studied in this manner for skydivers.6

Prolongation of survival by CNVS for patients with Duchenne muscular dystrophy and “…a mean [VC of] 176±102mL, or 3% of predicted normal, which is not compatible with survival without continuous ventilator support,”7 has been known since 1979.8 The same has been reported for amyotrophic lateral sclerosis patients with VCs as low as unmeasurable9 and other neuromuscular disease populations with less than 2% of normal VC. Submissions of these papers consistently draw comments from reviewers that there was no control group, so nothing can be generalized from them. Reviewers have gone so far as to state that prolongation of life by a few months with use of sleep-only bi-level positive airway pressure that has been documented by controlled studies is high quality evidence, but prolonging life by as much as 60 years with CNVS is not because there were no controls.

Placebo controls also cannot be used when substituting for or restoring the function of any organ unless the intervention is so minimally effective that it is difficult to determine any positive treatment effect at all. For example, placebo is not possible and controlled trials were never performed for dialysis or renal organ transplantation, mechanical ventilator support, cochlear implants or hearing aids, prosthetic legs or arms, hip and knee replacements, or parachutes in the absence of wings.6 Remarkably, this stark limitation has yet to be considered in the literature.

Besides mechanical ventilation, expiratory muscles must be supported or substituted for by alternately applying positive and negative pressures to the airways to generate effective cough flows to prevent episodes of acute respiratory failure, and if one is intubated, to avoid extubation failure. In this manner, mechanical insufflation-exsufflation (MIE) expulses airway debris. While the cough-enhancing effectiveness of different devices and techniques can be compared, such interventions cannot ethically be placebo controlled; yet, intensivists are reticent to use MIE, despite high rates of extubation failure in many patient populations.10 Scrupulous adherence to Cochrane EBM criteria can, and do, result in studies11 being discredited. Consequently, acute respiratory failure and resort to tracheotomy occur because there are no “high quality” RDBPC studies of CNVS or MIE.

Modern medicine evolves by minor EBM-supported improvements on major breakthroughs. As EBM establishes the efficacy and safety of new approaches, it is nonetheless crucial to understand its limitations. As long as there exists an expectation that all innovations be grounded in RDBPC trials, the institution of life-saving innovations may remain unsubstantiated. Sackett supports the use of both individual clinical expertise and best available evidence as “evidence-based,” but too many tend to forget this.5 Indeed, some medical journals offer free publication of the latter but not for observational studies.