Exploring the validity of the Oregon Extended Writing Assessment.

Research into the technical adequacy of statewide alternate assessments is limited. In this study, the authors analyzed 2 years of data from one state's alternate assessment in written language in an attempt to validate current test score interpretations. More than 1,000 students were included in each year. Findings support the test's technical adequacy on two major dimensions: (a) strong convergent and discriminant evidence for tasks on the assessment and (b) evidence that a collection of writing subtasks contribute unique information to the assessment of writing among students in special education. Implications include the use of the data (a) on subtasks to inform instruction in written language and (b) for accountability purposes.

**********

The 1997 amendments to the Individuals with Disabilities Education Act required states to have alternate assessments for students with disabilities in place by 2000; today, every state has either an alternate assessment or guidelines allowing districts to develop their own assessments (Olson, 2004). As researchers at the National Center on Educational Outcomes indicated, however, "Alternate assessments are still very new, and taking the time for thoughtful reexamination is critical" (Quenemoen, Thompson, & Thurlow, 2003, p. 4). After 6 years of implementation, many states are engaged in "thoughtful reexamination" that is focused on improving their alternate assessments. Unfortunately this reexamination, including related research, tends to be "descriptive rather than evaluative" (Browder et al., 2003, p. 51), and technical adequacy data on the assessment of students with disabilities in statewide systems are lacking (Koretz & Barton, 2003).

Challenges Facing Alternate Assessment Systems

Alternate assessments for students with disabilities must yield technically adequate information about individual student performance and contribute to overall accountability efforts at the district and state levels (Johnson & Arnold, 2004). An alternate assessment system designed exclusively for accountability purposes falls short of the individualization required by the Individuals with Disabilities Education Improvement Act of 2004; on the other hand, data provided by a purely diagnostic assessment cannot be aggregated for purposes of accountability. Designing an assessment system to achieve both purposes is daunting and "requires rethinking traditional assessment methods" (No Child Left Behind Act of 2001, [paragraph] 3). Unfortunately, research supporting the effectiveness of current alternate assessment systems in meeting this dual challenge is limited (Johnson & Arnold, 2004).

Weak Reliability

Currently, states use various types of alternate assessment systems, including checklists, direct observations, samples of student work, portfolios, and performance assessments (Roeber, 2002). Portfolios are the most frequently used technique in the field of alternate assessments (Thompson & Thurlow, 2003), although there is evidence indicating questionable reliability levels (Kleinert, Kearns, & Kennedy, 1997). Traditional state-wide portfolio systems also have encountered similar challenges in regards to establishing reliability (Klein, McCaffrey, Stecher, & Koretz, 1995; Koretz & Barron, 1998).