The validity of using a content-specific reading comprehension test for college placement.

Publication: Journal of College Reading and Learning

Publication Date: 22-MAR-05

Bookmark this article Print this article Link to this article Email this article Digg It! Add to del.icio.us RSS

COPYRIGHT 2005 College Reading and Learning Association

This study provides empirical evidence to support the validity of using a content-specific reading test for college placement decisions. A content-specific reading test presents passages exclusively from the subject area for which the placement decision is intended. Forty-nine students in a human anatomy class were administered a content-specific reading test, a content-general reading test, and a test of prior domain knowledge on the first day of class. In a forward-solution multiple regression, the content-specific reading test was a significant predictor (p <. 01) of course grades, but neither the content-general reading test nor the knowledge test added significantly to the prediction. Thus, neither domain knowledge nor generic reading ability provided an independent contribution to the prediction, after partialing out the effects of content-specific reading ability.

**********

The scope and importance of reading placement testing at American colleges cannot be underestimated. Reading placement tests often determine whether incoming students will be allowed to pursue degree-level course work immediately or be required first to enroll in developmental (remedial) courses. Each year a large proportion of entering college students are thus assigned to noncredit-level developmental reading courses. Of the 2.4 million freshmen attending 2-year and 4-year colleges in the United States in 2000, 11 percent or about 260,000 students were required to take a developmental reading course (Parsad & Lewis, 2003).

Unfortunately, despite the common wisdom that general reading ability should be related to academic achievement, reading placement tests have shown a negligible to modest relationship to grades in credit-level college courses (American College Testing Program, 1990; Armstrong, 1994; Brown, Fishco, & Hanna, 1993, citing Guidan; College of the Canyons, 1994; Feldt, 1989; Kessler, 1987). Reading tests in common use, such as ACCUPLACER, APS, ASSET, and Nelson-Denny, are grounded in a domain-generic model of comprehension that assumes "a good reader is a good reader," no matter the content. These content-general reading tests present passages from a variety of subject areas and yield a global comprehension score. However; research suggests that learning is based on both domain-specific and domain-generic factors, with emerging evidence that domain-specific factors may have primacy (for reviews, see Alexander & Judy, 1988; Byrnes, 1995). Further, both schema theory (Anderson, 1984; Anderson & Pearson, 1984; Mason & staff, 1984; Rumelhart, 1981; Wilson & Anderson, 1986) and the construction-integration model of reading (Kintsch, 1986, 1988; Kintsch & vanDijk, 1978; Mannes & Kintsch, 1987; Moravcsik & Kintsch, 1993; vanDijk & Kintsch, 1983) support the domain-related nature of understanding and learning from text.

As an alternative to using content-general reading tests that mask the influence of domain-specific knowledge and domain-specific reading strategies on comprehension, it has been recommended that content-specific reading placement tests might be more valid predictors of course success (Behrman, 2000). A content-specific reading test would measure the reader's ability to comprehend text in a particular subject area, such as history, psychology, literature, or biology. Such a test would present passages exclusively from the academic discipline for which the placement decision would be made, and the comprehension score would indicate the examinee's ability to understand text in that subject area. The purpose of the present study is to explore the validity of using content-specific reading tests for college placement decisions by examining the relationship among scores on a content-general reading test, a content-specific reading test, a test of prior domain knowledge, and grades in an introductory human anatomy class.

Theoretical Framework

Establishing Validity of Placement Decisions

Because the intent of placement testing is to predict whether or not a student will be successful in credit-level coursework, the proper external measure of a placement test's validity is the relationship between placement test scores and grades in the target credit-level course rather than the developmental course (Sawyer, 1989, 1996). In the ideal placement situation we would be able to accurately predict which students are academically prepared for the demands of college study. Using Guilford's (1956) classification, each placement decision therefore falls into one of four quadrants: successful predictors, successful non-predictors, unsuccessful predictors, and unsuccessful non-predictors. A good placement test would minimize the proportion of successful non-predictors and unsuccessful predictors. Put another way, the majority of students placed directly in the credit-level course would be successful without the need for developmental coursework; and the majority of those placed into the developmental course would not have been successful if placed directly into the credit-level course.

However, the strength of the placement test-criterion relationship is not by itself sufficient for validity claims. AERA/APA/NCME standards emphasize that internal evidence, such as a conceptual framework underlying the test's development, may also be required to establish validity (Joint Committee, 1999). Although dissatisfied with current notions of reliability and validity, Schoenfeld (1999) echoes this same point: "If you are going to test for students' understanding of something, then (a) you have to have an adequate characterization of what it is you're assessing, and (10) you need to have a good idea of how performance on the assessment corresponds to being able to do whatever it is that's supposedly being assessed" (p. 11). Content-general reading tests may serve poorly as placement instruments because they score low in both areas: (a) they are founded on an assumption that reading comprehension is not mediated by the nature of the reading content; and (b) they attempt to predict performance in a particular course by presenting examinees with passages from different subjects altogether. An "adequate characterization" of reading comprehension would require attention to four areas of psychological and educational inquiry that build a theoretical basis for the role of content-specific factors in comprehension, and by extension, to...

Read the full article for free courtesy of your local library.