Using multiple measures to evaluate positive behavior support: a case example.

Abstract: In recent years, calls to expand the criteria by which behavior support efforts are evaluated have increased. Success is now said to depend on outcomes that transcend a reduction in the occurrence of problem behaviors and include the achievement of new competencies and improvements in one's quality of life. This single-case investigation was conducted as an effort to evaluate the effects of a positive behavior support intervention with multiple measures that included experimental analyses of the participant's problem behavior, engagement, happiness, and efficiency in completing transitions, as well as adult and peer perceptions of aspects of the participant's quality of life. Multiple baseline analyses indicated that the assessment-based intervention was effective in producing durable improvements in all of the measured variables and that the procedures were socially valid. The results are discussed in the context of the growing number of empirical case studies in positive behavior support, and the need to develop more efficient strategies for evaluating the essential outcomes of intervention.

**********

Positive behavior support (PBS) is a rapidly evolving approach for meeting the needs of people who experience challenges associated with behavioral adaptation. PBS began in the 1980s as a set of nonaversive strategies designed primarily to reduce serious problem behaviors (Horner et al., 1990) and has developed in the past decade to become a comprehensive discipline that incorporates individualized and systems interventions to enhance participants' behavioral repertoires. Whereas the primary emphasis was initially on problem behavior, the focus has shifted to lifestyle change and an enhanced quality of life, with reductions in problem behavior becoming an important but secondary or intermediate goal (Carr et al., 2002). This vital development in the conceptualization of PBS has implications for research and for the evaluation of PBS interventions.

This major tenet of PBS, that lifestyle improvements represent the overarching goal of intervention, suggests that outcomes must be measured not simply in terms of levels of problem behavior but also in terms of the participants' social relationships, productivity, opportunity, affect, and personal satisfaction (Carr et al., 2002; Risley, 1996; Turnbull & Ruef, 1997). Quality of life is a notion that fits well with the goals of PBS, and there have been a number of laudatory efforts to define and even quantify the construct (Hughes, Hwang, Kim, Eisenman, & Killian, 1995; Knoster, 1999a; Schalock, Keith, Hoffman, & Karan, 1989). However, a single measure that can efficiently and reliably be used to assess with sensitivity the lifestyle changes that might be sought or anticipated as a function of individualized PBS programs has not yet been developed. This is not surprising given the highly idiosyncratic nature of behavior support efforts, the circumstances in which they are applied, and the diversity of individual, family, and setting factors that must be conspicuous ingredients in evaluation plans.

Traditionally, analyses of behavioral interventions in the research literature have been limited to direct observations of problem behavior and, perhaps, one targeted alternative, such as on-task responding or a specific communicative response. This focus was understandable given the constraints on applied research (i.e., collecting valid and reliable data in natural settings) and the research emphasis on internal validity and the delineation of functional variables. However, the new emphasis of PBS to produce broad, lifestyle benefits has led authors to advocate that more flexible methodologies and, in particular, broader measurement strategies are needed if research and evaluation are to match the proclaimed mandate of PBS (e.g., Carr et al., 2002; Dunlap, Fox, Vaughn, Bucy, & Clarke, 1997). Indeed, researchers have increasingly been including multiple measures in order to provide broader validation of their interventions with participants in school, home, and community settings (e.g., Lucyshyn, Olson, & Horner, 1995; Stiebel, 1999; Todd, Horner, & Sugai, 1999; Vaughn, Dunlap, Fox, Clarke, & Bucy, 1997). For instance, investigations of PBS in the past few years have included a variety of social validation assessments (e.g., Carr, Horner, et al., 1999; Jensen, McConnachie, & Pierson, 2001; Lucyshyn, Albin, & Nixon, 1997; McConnachie & Carr, 1997), as well as measures of child affect (e.g., Moes, 1998), activity patterns and social integration (e.g., Lucyshyn et al., 1995; Luchyshyn et al., 1997), cooperation (e.g., Vaughn, Dunlap, et al., 1997), contextual fit (e.g., Lucyshyn et al., 1997; Moes & Frea, 2000), and task performance and task completion (Todd et al., 1999; Vaughn, Clarke, & Dunlap, 1997).

The current study was intended to further the trend toward an expanded evaluation of PBS interventions. By collecting numerous measures pertinent to the lifestyle of the participant, the purpose was to assemble a broader picture of relevant outcomes than could be managed by more limited data collection. In this case, a female middle school student with very serious problem behaviors was supported by a team that adopted an assessment-based, PBS approach to intervention. Measures were collected using time series and pre-post schedules to assess levels of problem behavior, engagement, efficiency in completing routines, and the participant's affect, as well as the perceptions of adults and peers regarding the participant's quality of life and the social validity of the intervention process.

Method

PARTICIPANT, SETTING, AND FOCUS ACTIVITIES

Mindy, a 12-year-old student, served as the participant in the investigation. Mindy was a slender girl who liked to laugh, listen to music, play on the computer, and interact with others. Mindy lived at home with her teenage brother and parents. The family had moved to the area from a different state at the beginning of the school year. Mindy's family was close-knit, and the primary language spoken in the home was Polish. She was enrolled in a classroom for students with severe and profound mental retardation within a regular public school campus. Mindy's classroom included four other female classmates, her teacher, and a paraprofessional. Mindy had been diagnosed with autism spectrum disorder and several medical/developmental conditions, including hyperthyroidism, dysmorphic syndrome, hypotonia, asthma, temporomandibular joint syndrome, and a visual impairment. Mindy communicated with a few one-word vocalizations, gestures, and a variety of problem behaviors. Mindy often displayed problem behavior when she was given requests or demands to complete an activity and during transitions from one activity to another. At such times, she often engaged in self-injurious behavior (biting), physical resistance, aggression, property destruction, and screaming. Mindy also had difficulty staying on task during preacademic activities.

Mindy's classroom teacher (the fourth author) requested consultative assistance from our university-based research group in PBS. A first step in the consultation was to establish a collaborative team to conduct assessments, develop a behavior support plan, and provide the intervention (Hieneman & Dunlap, 1999). The team included all members of Mindy's family as well as her classroom teacher and paraprofessional, her speech teacher, the adaptive physical education coach, and two behavioral consultants (the first and second authors). The team was responsible for developing all steps of the assessment and intervention process.

The study was conducted on Mindy's middle school campus in her special education classroom and in the context of transitions from one school location to another. The team identified specific activities and routines that were especially problematical. Four of the routines were daily school preacademic activities that took place in the classroom in a one-to-one instructional context. These activities included putting a puzzle together, a match-to-sample task, an assembly task, and a sorting activity. All activities occurred consecutively each morning at Mindy's desk located in the corner of the classroom. Three transition activities, in which Mindy was required to physically move from one area to another, were also selected for intervention. The transition activities selected for intervention included transition to the cafeteria, transition to preacademics, and transition to physical education. All of these activities were selected for intervention by the team because of their association with significant problem behavior and disruptions to the entire classroom.

GENERAL PROCEDURES AND EXPERIMENTAL DESIGN

The study was conducted as a detailed, experimental case study with multiple measures. The procedures followed the general PBS protocol (Hieneman et al., 1999) including team formation, functional assessment of problem behaviors, development of a support plan, and assessment-based intervention. Intervention was conducted by the classroom teacher and paraprofessional, with consultation from the behavioral consultants.

Two multiple baseline across activities designs were implemented to demonstrate the efficacy of the independent variable, which was defined as the individualized, assessment-based intervention plan constructed for each activity. One multiple baseline design was conducted across the four preacademic activities, and a second multiple baseline design was conducted across the three transition routines. The two multiple baselines were run concurrently. Follow-up measures were obtained following the implementation of the multiple baseline protocols. Maintenance was also assessed by conducting probes one year following intervention, in the subsequent school year. Dependent variables included levels of problem behavior, student engagement, affect, and duration of transitions. Additional measures sampled the interaction behaviors of classroom personnel, perceptions of student quality of life, and adults' satisfaction with the PBS strategies. All measures are described in a subsequent Measurement section of the methodology.

FUNCTIONAL ASSESSMENT

Following the establishment of the collaborative team, a functional assessment was conducted in order to produce a full description of Mindy's problem behavior, a precise description of the contexts in which the behaviors occurred, a statement of the presumed operant functions of the behaviors in each context, and a delineation of Mindy's preferences for stimuli and activities that might be utilized in her school curriculum (O'Neill, Vaughn, & Dunlap, 1998). The functional assessment was conducted over a period of approximately 4 weeks and included interviews, direct observations, and a review of archival records (Foster-Johnson & Dunlap, 1993; O'Neill et al., 1997). Interviews were conducted with Mindy's parents and all school staff involved in the team. Direct observations were conducted by the consultants during all routines throughout the entire school day. The interviews and observations were valuable, but the information available from Mindy's previous school records was scant, and none of it was judged ultimately to be pertinent to the functional assessment objectives.

The assessment …