The Communicative Effectiveness Survey: investigating its item-level psychometric properties.(Survey)

The study's purpose was to investigate the item-level psychometric properties of the Communicative Effectiveness Survey (CES) using Rasch analysis, an item response theory model. The CES (8-item 7-category rating scale) was designed to elicit ratings of communicative effectiveness in everyday situations from individuals with dysarthria. A speech-language pathologist (SLP) blind to the study's purpose administered the CES to 95 individuals (58% male), mean age 62.5 (SD = 14.0, range 18-87), with neurodegenerative diseases and a wide range of speech competency. Rasch analysis showed three of the seven rating scale categories were used with such low statistical probability that they were not useful. All subsequent analyses used the 4-category rating scale. Twelve participants (five males, seven females; eight with Parkinson's disease, four with other neurodegenerative diseases) demonstrated erratic response patterns and were removed from analysis. Positive item-level psychometric properties of the CES included: (1) item hierarchy conformed to an a priori theoretical hierarchy (indicator of content validity); (2) principal components analysis showed that the items represented a unidimensional construct; (3) 3.92 statistically significant ability levels; and (4) high person-to-measure consistency of 0.94 (analogous to Cronbach's alpha). We suggest that the CES could be used to add an additional dimension to a clinician's repertoire of pre- and posttreatment outcome measures.

**********

Researchers and clinicians increasingly recognize that the patient perspective is essential in understanding health status and healthcare (Arpinelli & Bamfi, 2006). Instruments that assess the patient perspective are referred to as "patient-reported outcomes" (PROs) (Arpinelli & Bamfi, 2006; Fries, Bruce, Bjorner, & Rose, 2007; Willke, Burke, & Erickson, 2004). PROs provide important information from the patient's perspective that might be lost otherwise. For example, some treatment effects can only be perceived by the patient, or treatment effects demonstrated from standardized assessments or physiologic measures may not correspond to changes perceived by the patient. In addition, PROs reduce the risk of interobserver bias or variability that may occur when patient responses are filtered through a clinician (FDA, 2006). Currently, the National Institutes of Health (NIH) Patient-Reported Outcomes Measurement Information System (PROMIS) is in the process of developing PROs to be used as outcome measures in clinical treatment trials for many chronic illnesses (Fries et al., 2007). As with any diagnostic instrument, the conceptual development of a PRO must be sound, and it must have demonstrated validity and reliability before it can be used with confidence (Coyne, Tubaro, Brubaker, & Bavendam, 2006; FDA, 2006; Fries et al., 2007; Ross, 2006).

Researchers and clinicians in the field of communicative disorders (CD) have a need for measures that describe the patient's communication status from the patient's perspective. In the treatment of dysarthria, adding a PRO to the traditional assessment battery may provide clinicians with a more complete picture of the patient's condition before treatment, assist in determining optimal treatment, demonstrate the outcomes of the treatments, and justify what more needs to be done. One such instrument under development is the Communicative Effectiveness Survey (CES; Yorkston, Beukelman, Strand, & Bell, 1999), a patient-reported outcome measure of communicative effectiveness designed for individuals with motor speech disorders.

The purpose of this study was to investigate the item-level psychometric properties of the CES using Rasch analysis, an item response theory model. The relevance of this purpose and justification for the design arose from three sources: models of objective measurement; the World Health Organization (WHO) International Classification of Functioning, Disability, and Health (ICF); and the identified need for an outcome measure of this kind in motor speech disorders.

OBJECTIVE MEASUREMENT

There is a difference between an assessment and an objective measure. Stevens (1946) set the stage for the use of classical test theory methodology in the social sciences when he defined measures as nominal, ordinal, interval, and ratio, based on the observed relationship among items and the properties of the numbers assigned to the items. Most of the assessments used in CD were developed using the classical test theory where ordinal data is gathered, summed, or tallied but then erroneously treated as interval data. Using ordinal data in this way is problematic because ordinal scales are not based on equal (i.e., interval) units of measurement. For example, if one person responds "1" on a 1-to-4-category rating scale (lowest to highest ability), and another person responds "2," we cannot say that the second person is twice as able as the first. We can only say that the second person is more able than the first. Wright and Linacre (1989) suggest that when ordinal data are treated as interval measurement clinicians run the risk of making improper treatment decisions and erroneous interpretations about improvement over time.