Percentage of all non-overlapping Data (PAND): an alternative to PND.

Although single-case researchers are not accustomed to analyzing data statistically, standards for research and accountability from government and other funding agents are creating pressure for more objective, reliable data. In addition, "'evidence-based interventions" movements in special education, clinical psychology, and school psychology imply reliable data summaries. Within special education, two heavily debated single-case research (SCR) statistical indices are "'percentage of non-overlapping data" (PND) and the regression effect size, [R.sup.2]. This article proposes a new index--PAND, the "percentage of all non-overlapping data"--to remedy deficiencies of both PND and [R.sup.2]. PAND is closely related to the established effect size, Pearson's Phi, the "'fourfold point correlation coefficient." The PAND/Phi procedure is demonstrated and applied to 75 published multiple baseline designs to answer questions about typical effect sizes, relationships with PND and Re, statistical power, and time efficiency. Confidence intervals and p values for Phi also are demonstrated. The findings are that PAND/ Phi and PND correlate equally well to [R.sup.2]. However, only PAND/Phi could show adequate power for most of the multiple baseline designs sampled. The findings suggest that PAND/Phi may meet the requirement for a useful effect size for multiple baseline and other longer designs in SCR.

**********

Single-case researchers traditionally have relied on visual analysis of graphs for judging intervention success. Despite the well-documented unreliability of visual judgments (Brossart, Parker, Olson, & Lakshmi, 2006; DeProspero & Cohen, 1979; Harbst, Ottenbacher, & Harris, 1991 : Ottenbacher, 1990; Park, Marascuilo, & Gaylord-Ross, 1990), most published single-case research (SCR) continues to rely on visual judgments, assisted by comparisons of phase means, medians, or percentages. Visual analysis also commonly includes judging the amount of data overlap between phases, which helps capture the important concept of data dispersion or variability around a center. A recent study of 124 published SCR datasets (Parker et al., 2005) found statistical analyses in only 11%, which is comparable to the 10% rate cited in earlier studies of 10 and 25 years ago (Busk & Marascuilo, 1992; Kratochwill & Brody, 1978). Thus, in acceptance of statistical analysis, the SCR field has changed little over recent decades.

For the 75 multiple baseline designs (MBDs) sampled in this study, most authors (87%) relied solely on visual analysis, with phase percentages or means calculated, but no tests of differences between these indices. Data variability around the means and percents (e.g., standard deviations) was not presented, nor were reliabilities (standard error or confidence intervals [CIs]). The summary statistics served as nonessential additions to the primary visual analysis. Of the eight designs (11%) that were evaluated statistically, six used the student t test, one a Freidman two-way nonparametric test, and one a repeated measures ANOVA. Several of these authors referred to visually apparent data overlap between phases, but without quantifying the overlap.

Recent changes within the fields of education and psychology bolster the arguments for more objective and reliable results. These are the movements for evidence-based interventions, practices, or treatments in fields such as special education (Odom et al., 2005), school psychology (Kratochwill & Stoiber, 2000), and clinical psychology (Chambless & Ollendick, 2001). In school and clinical psychology, these movements are setting standards for reporting intervention efficacy, including objective and statistically reliable summaries that can be interpreted across the field and that permit comparisons between studies. The field of special education also is setting design standards, but has been slower to accept statistical summaries.

The evidence-based movements in education and psychology were accelerated (or spawned) by the federal legislation No Child Left Behind (NCLB; 2001) and the Education Sciences Reform Act (ESRA; 2002). These laws are played out in the new Institute of Education Sciences (IES: Whitehurst, 2004), which has set higher standards for funded educational research, including SCR. Corresponding policies also are reflected in statements such as the National Research Council's Scientific Research in Education (Shavelson & Towne, 2002).

The momentum for more objective and reliable SCR results also is accelerated by the need to have SCR studies included in meta-analyses and other non-SCR publications (Homer, Carr, Halle, McGee, Odom, & Wolery, 2005). Meta-analyses of single-case studies generally are separate from those of group research, and results from the two methodologies may not agree (Forness, 2001). This inconsistency may be because in the SCR field, "the synthesis of single-participant studies remains a controversial topic" (Forness, 2001, p. 190). The meta-analyses of special education research summarized by Forness failed to use standard effect sizes, instead employing methods that discard much of the data. Mostert (2001) evaluated all special education meta-analyses to that date against minimum standards and found most to be deficient. One missing critical piece of information was "accounting for the amount of total variance explained by the treatment effect...." The higher the proportion of variance accounted for, the stronger the evidence for the efficacy of the treatment or intervention" (Mostert, p. 215). The summaries of treatment effect used by most SCR meta-analyses do not meet this minimum criterion.