|
COPYRIGHT 2007 Delmar Learning
The study's purpose was to investigate the item-level psychometric properties of the Communicative Effectiveness Survey (CES) using Rasch analysis, an item response theory model. The CES (8-item 7-category rating scale) was designed to elicit ratings of communicative effectiveness in everyday situations from individuals with dysarthria. A speech-language pathologist (SLP) blind to the study's purpose administered the CES to 95 individuals (58% male), mean age 62.5 (SD = 14.0, range 18-87), with neurodegenerative diseases and a wide range of speech competency. Rasch analysis showed three of the seven rating scale categories were used with such low statistical probability that they were not useful. All subsequent analyses used the 4-category rating scale. Twelve participants (five males, seven females; eight with Parkinson's disease, four with other neurodegenerative diseases) demonstrated erratic response patterns and were removed from analysis. Positive item-level psychometric properties of the CES included: (1) item hierarchy conformed to an a priori theoretical hierarchy (indicator of content validity); (2) principal components analysis showed that the items represented a unidimensional construct; (3) 3.92 statistically significant ability levels; and (4) high person-to-measure consistency of 0.94 (analogous to Cronbach's alpha). We suggest that the CES could be used to add an additional dimension to a clinician's repertoire of pre- and posttreatment outcome measures.
**********
Researchers and clinicians increasingly recognize that the patient perspective is essential in understanding health status and healthcare (Arpinelli & Bamfi, 2006). Instruments that assess the patient perspective are referred to as "patient-reported outcomes" (PROs) (Arpinelli & Bamfi, 2006; Fries, Bruce, Bjorner, & Rose, 2007; Willke, Burke, & Erickson, 2004). PROs provide important information from the patient's perspective that might be lost otherwise. For example, some treatment effects can only be perceived by the patient, or treatment effects demonstrated from standardized assessments or physiologic measures may not correspond to changes perceived by the patient. In addition, PROs reduce the risk of interobserver bias or variability that may occur when patient responses are filtered through a clinician (FDA, 2006). Currently, the National Institutes of Health (NIH) Patient-Reported Outcomes Measurement Information System (PROMIS) is in the process of developing PROs to be used as outcome measures in clinical treatment trials for many chronic illnesses (Fries et al., 2007). As with any diagnostic instrument, the conceptual development of a PRO must be sound, and it must have demonstrated validity and reliability before it can be used with confidence (Coyne, Tubaro, Brubaker, & Bavendam, 2006; FDA, 2006; Fries et al., 2007; Ross, 2006).
Researchers and clinicians in the field of communicative disorders (CD) have a need for measures that describe the patient's communication status from the patient's perspective. In the treatment of dysarthria, adding a PRO to the traditional assessment battery may provide clinicians with a more complete picture of the patient's condition before treatment, assist in determining optimal treatment, demonstrate the outcomes of the treatments, and justify what more needs to be done. One such instrument under development is the Communicative Effectiveness Survey (CES; Yorkston, Beukelman, Strand, & Bell, 1999), a patient-reported outcome measure of communicative effectiveness designed for individuals with motor speech disorders.
The purpose of this study was to investigate the item-level psychometric properties of the CES using Rasch analysis, an item response theory model. The relevance of this purpose and justification for the design arose from three sources: models of objective measurement; the World Health Organization (WHO) International Classification of Functioning, Disability, and Health (ICF); and the identified need for an outcome measure of this kind in motor speech disorders.
OBJECTIVE MEASUREMENT
There is a difference between an assessment and an objective measure. Stevens (1946) set the stage for the use of classical test theory methodology in the social sciences when he defined measures as nominal, ordinal, interval, and ratio, based on the observed relationship among items and the properties of the numbers assigned to the items. Most of the assessments used in CD were developed using the classical test theory where ordinal data is gathered, summed, or tallied but then erroneously treated as interval data. Using ordinal data in this way is problematic because ordinal scales are not based on equal (i.e., interval) units of measurement. For example, if one person responds "1" on a 1-to-4-category rating scale (lowest to highest ability), and another person responds "2," we cannot say that the second person is twice as able as the first. We can only say that the second person is more able than the first. Wright and Linacre (1989) suggest that when ordinal data are treated as interval measurement clinicians run the risk of making improper treatment decisions and erroneous interpretations about improvement over time.
An alternative to classical test development is to use item response theory (IRT) for measure development. IRT has been used extensively in education and psychology since the 1940s to describe and order observed behaviors. It has gained acceptance in rehabilitation outcomes research within the past decade as well (Haley, McHorney, & Ware, 1994; Jette & Haley, 2005; McHorney, Haley, & Ware, 1997; Velozo, Magalhaes, Pan, & Leiter, 1995; Ware et al., 1995). The American Psychological Association Task Force on Statistical Inference proposed that the psychometric properties of scales be assessed using IRT prior to publication (Wilkinson, 1999). Currently, the NIH PROMIS project has chosen to utilize IRT to develop a number of physical function/disability PROs for chronic illnesses (Fries et al., 2007). Clinician researchers in CD have also recognized IRT's utility for both new measure development (Doyle, Hula, McNeil, Mikolic, & Matthews, 2005; Doyle et al., 2004) and analysis of existing instruments (Donovan, Rosenbek, Ketterson, & Velozo, 2006; Hula, Doyle, McNeil, & Mikolic, 2006).
Rasch analysis, the one-parameter IRT model, attempts to apply principles of objective measurement found in the natural sciences to social behaviors. (1) Objective measurement (e.g., length, weight, temperature) is based on certain fundamentals (Wright, 1997). First, the instrument must measure a single construct (termed unidimensionality). For example, a ruler measures one construct, length; while a scale measures a different construct, weight. The ruler would be an invalid measure of weight. A second principal of objective measurement is that a construct be ordered hierarchically from less to more, in stable units of measurement (termed linearity). This aspect of measurement allows for comparisons between the items being measured. Third, the unit of measurement remains the same across the instrument (termed invariance). Continuing the ruler example, the "inch" is an invariant unit of measure of length. In Rasch analysis the unit of measure is the "logit," defined as the change in person ability or item difficulty score needed to change the odds of a given response by a constant factor (2.718, the base of the natural logarithm) (Wright & Stone, 1979). The premise of Rasch analysis is that the most understanding about an individual's level of functioning is gained at the point where the individual's level of ability is matched to the difficulty of items he can perform. Calibrating...
Read the full article for free courtesy of your local library.
|
Bookmark this article
Print this article
Link to this article
Email this article
Digg It!
Add to del.icio.us
RSS
