Weighting by inverse variance or by sample size in meta-analysis: a simulation study.

Meta-analysis aims to quantitatively integrate the results of several studies on a research topic (Cooper & Hedges, 1994; Glass, McGaw, & Smith, 1981). To this aim, an effect-size index (Kirk, 1996) is chosen as the standard measure to express the results, enabling them to be compared between the studies. To obtain a global index of the magnitude of the relation, the effect sizes are averaged, their homogeneity is tested, and if homogeneity is not assumed, possible variables or characteristics influencing the heterogeneity of effect sizes are explored (Bangert-Drowns, 1986; Hedges & Olkin, 1985; Hunter & Schmidt, 1990; Johnson, Mullen, & Salas, 1995; Rosenthal, 1991; Sanchez-Meca and Marin-Martinez, in press).

One of the most widely used effect-size indexes in meta-analysis is the standardized mean difference, d, defined as the difference between two group means (usually experimental vs. control) divided by the within-group standard deviation. The use of this index is indicated especially when the studies to be integrated are experimental or quasi-experimental. Hedges and Olkin (1985) have shown that the best procedure to average a set of independent ds is a weighted average, with the inverse variance of each d as the optimal weight factor.

However, the variance of d depends on the population standardized mean difference, [Delta], a parameter unknown in practice. Therefore, an estimate of the optimal weight is required. Hedges and Olkin proposed to substitute the sample standardized mean difference, [d.sub.i], for the [Delta] parameter in each single study. On the other hand, Hunter and Schmidt (1990) proposed a simpler procedure consisting of weighting by the sample size of each study. Hunter and Schmidt argued that their procedure is less biased than Hedges and Olkin's (1985), even in the case of nonhomogeneous effect sizes.

The statistical properties of these estimators have not been compared. In the present study, a Monte Carlo simulation is carried out to assess the bias and efficiency of the two estimators for averaging independent ds in conditions similar to those of real meta-analyses.

Two Procedures for Averaging ds

Conceptually, we assume that a set of k independent studies estimates the same population effect size. Let us also assume that for each study, a standardized mean difference is obtained by

(1) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

where [g.sub.i] is a positively biased estimator of the population standardized mean difference, [Delta]; [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] and [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] are the experimental and control group means, respectively, of the ith study; and [S.sub.i] is the ith within-group standard deviation computed through

(2) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] and [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] being the sample sizes and ([MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] and [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] the unbiased variances of the two groups in the ith study.

To remove the bias in the [g.sub.i] index, the accurate correction factor includes the gamma …