Confounding by indication in non-experimental evaluation of vaccine effectiveness: the example of prevention of influenza complications. (Theory and Methods).

Randomised allocation of vaccine or placebo is the preferred method to assess the effects of the vaccine on clinical outcomes relevant to the individual patient. In the absence of phase 3 trials using clinical end points, notably post-influenza complications, alternative non-experimental designs to evaluate vaccine effects or safety are often used. The application of these designs may, however, lead to invalid estimates of vaccine effectiveness or safety. As patients with poor prognosis are more likely to be immunised, selection for vaccination is confounded by patient factors that are also related to clinical end points. This paper describes several design and analytical methods aimed at limiting or preventing this confounding by indication in non-experimental studies. In short, comparison of study groups with similar prognosis, restriction of the study population, and statistical adjustment for dissimilarities in prognosis are important tools and should be considered. Only if the investigator is able to sho w that confounding by indication is sufficiently controlled for, results of a non-experimental study may be of use to direct an evidence based vaccine policy.

**********

The health economic impact of influenza epidemics is considerable. (1-3) In most western countries, the use of inactivated influenza vaccines by vulnerable patient groups is advocated to prevent complications. (4) However, uptake of the vaccine remains low, especially in those who need it most. (4-6) Disbelief in the vaccine's effects on clinical outcomes relevant to the individual patient--that is, post-influenza complications--may be one of the major reasons for disappointing immunisation rates. (3,4,6-8)

THE PREVENTION OF INFLUENZA COMPLICATIONS BY VACCINATION: RANDOMISED CONTROLLED TRIALS

The clinical effects of influenza vaccines on reduction of major symptomatic events or death should preferably be studied in phase 3 randomised controlled trials (RCT). (9) Provided that the sample size is large enough, randomised assignment of patients to vaccine or placebo enables valid assessment of vaccine effects through comparing the occurrence of outcomes in both patient groups with similar prognosis. Such trials can be conducted among various segments of the patient population and may give insight into positive as well as negative clinical consequences of immunisation in daily practice. Results of large enough trials in which the primary end point is a clinical outcome rather than a surrogate end point (for example, immune response) provide crucial information on the true impact of these preventive measures and are best suited to guide healthcare decisions. (10,11)

However, scientists face many obstacles when planning a RCT for clinical evaluation of influenza vaccines. Firstly and foremost, as the incidence of influenza related complications or adverse effects is low these trials would entail great expense because large numbers of patients are required. (1,12) Secondly, several influenza seasons may need to be observed as the virulence of circulating influenza viral types is highly variable and unpredictable. (1,6,13) Finally, once the vaccine has been licensed ethical concerns may be raised to further evaluate its effectiveness in placebo controlled studies, especially when persons at high risk for complications are involved. Because of these limitations, post-licensing or phase 4 studies evaluating the vaccine's clinical effectiveness or safety usually use a non-experimental approach, notably a case-control or cohort design. (9) The vaccine's effectiveness is interpreted as the percentage reduction in risk of influenza associated complications attributable to vaccina tion, given in percentage by 1-RR in cohort studies or 1-OR in case-control studies. (3) The main difference between experimental and non-experimental designs lies in the absence of random allocation of the intervention, for example, vaccination, by the investigator.

EFFECTIVENESS OF INFLUENZA VACCINATION: NON-EXPERIMENTAL STUDIES

One of the important problems encountered in non-experimental evaluation of intended drug effects is the "natural" presence of incomparability of prognosis among subjects receiving the drug and those who do not. (14) In non-experimental influenza vaccine studies, the vaccine group typically comprises patients with more severe disease or (perceived) higher risk, either as a result of self selection or physician preference, than the non-vaccinated (control) group. (15,16) In contrast, those with a contraindication for the intervention will usually be found in the control group only. Thus, selection of exposure is confounded with patient factors, both clinical and non-clinical, which are also related to (detection of) the outcome. This phenomenon may equally apply to qualitative (absence/presence) as well as quantitative (dosing schedule) aspects of exposure and is usually referred to as "confounding by …