Reproducibility and validity of the secondary level School-Based Nutrition Monitoring student questionnaire. (Research).

Determination of child nutrition behaviors, attitudes, and knowledge is crucial for monitoring trends and evaluation of nutrition interventions (1,2). However, there are few readily available questionnaires; of these, many do not have published estimates of validity and reproducibility in school-aged children (3). In addition, most do not adequately assess key nutrition and diet-related topics (2,4-6). For example, the Youth Risk Behavior Survey (YRBS) addresses nutrition and body weight issues with 14 items (3,7), whereas other instruments (5,6) have been designed for specific purposes, such as assessment of fat, fruit, and vegetable intake or calcium (4). Many nutrition-related questionnaires do not include physical activity items, although it is important to address energy expenditure as well as energy intake (8,9). Finally, more precise methods of dietary assessment, such as 24-hour recalls food records, food frequency questionnaires, and direct observation, have high cost and respondent burdens and canno t be easily administered in a school setting (2,10).

The National Nutrition Monitoring and Related Research Act of 1990 (1) mandated the development of a 10-year Comprehensive Plan to establish a nutrition monitoring and related research program. As part of this plan, an interagency work group was formed in August 1992 to plan for the development and testing of a model school-based nutrition monitoring system to assess the health and nutritional status of school-aged children. The federal participants of the interagency school nutrition monitoring work group included the following: the Department of Health and Human Services (DHHS)/Centers for Disease Control and Prevention (CDC)INationai Center for Chronic Disease Prevention and Health Promotion (NCCD-PHP)/National Center for Health Statistics (NCHS) (co-lead agency), the United States Department of Agriculture (USDA)/Food and Nutrition Service (FNS) (co-lead agency), the USDA/Cooperative Extension Service, the DHHS/Health Resource and Services Administration, DHHS/Indian Health Service, and the Department of Education. Three professional associations also participated in the work group: the Association of State and Territorial Public Health Nutrition Directors (ASTPHND), the National Association of State Nutrition Education and Training (NASNET) Coordinators, and the American School Food Service Association (ASFSA).

As part of this initiative, the CDC together with USDA funded the School-Based Nutrition Monitoring (SBNM) Project to design, develop, and evaluate the psychometric properties of surveillance tools to be used in elementary, middle, and high schools. The purpose of this paper is to present reproducibility and validity results for the secondary level surveillance instrument developed to assess nutrition-related behaviors, attitudes, and knowledge and some physical activity behaviors for children at middle and high school levels.

METHODS

Instrument Development

The development of the survey was an iterative process (11,12) that included the following: needs assessment; development of a Rationale, Goals, and Scope statement; development of questionnaire items; review by an expert panel; focus group testing; and cognitive interview testing of questionnaire items. The initial needs assessment data were collected by an interagency work group assembled by the two lead agencies: the CDC and the USDA. The University of Texas-Houston investigators completed the remainder of the developmental steps.

Needs assessment A needs assessment was conducted in 1993 with representatives from the three professional organizations that participated in the SBNM interagency work group to collect information about each organization's specific needs for child and adolescent nutrition information and to develop specific aims for the student level questionnaires. Based on results from the needs assessment, the specific aim of this questionnaire would be to provide a surveillance instrument that focused primarily on a broad range of nutrition-related constructs in school children and could provide data for local entities on a timely basis. This questionnaire was developed to fill a gap in nutrition surveillance that was not addressed with information currently obtained from surveys such as the YRBS and NHANES. It was determined that the questionnaires should address nutrition attitudes and behaviors, specifically dietary fat intake, consumption of fruits and vegetables, consumption of high calorie/low nutrient dense foods, and consumption of grain products. It was recommended that the questionnaire be administered in the fall with a time length of approximately 30 to 45 minutes and no more than once every two years.

Rationale, goals, and scope statement Data from the initial needs assessment together with national Dietary Guidelines (13-15) and current dietary intake data (16-22) were used to draft a Rationale, Goals, and Scope statement. This statement became the guiding document for the questionnaire development and consisted of specific goals for the questionnaire, example uses of the data, and the scope of the questionnaire administration. An expert panel consisting of professionals with nutrition, anthropometry, physical activity, food service, and health education backgrounds reviewed the Rationale, Goals, and Scope statement before development of the questionnaire items.

Development of questionnaire items and protocol for administration Three versions of the questionnaire were initially developed: one for elementary schools (4th grade level), one for middle schools (8th grade level), and one for high schools (11th grade level). Because of the scope of information covered on the questionnaire and the time constraints on its administration, questionnaire items were designed to provide epidemiologic data that can distinguish between groups of students for surveillance and planning purposes rather than providing precise levels of nutrient intake at the individual level.

The questions were derived from a variety of references and survey instruments (3,23-35). The investigators selected and/or adapted items from these tools that best addressed the specific goals of the SBNM questionnaire as outlined in the Rationale, Goals, and Scope statement. For some survey goals, no specific survey items were found, and new questions were developed.

Questions were modified for each grade level, with consideration for reading level and cognitive ability. Although all content domains were covered at the three grade levels, the number of questions addressing the content domain varied among the questionnaires because of reading ability constraints and to ease respondent burden.

The reading level of the questionnaires was determined using the Dale-Chall formula (36). The elementary level survey was determined to be at a US grade level of 4.84 (nine years of age), and the upper level survey instruments were determined to be at a US grade level of 5.41 (approximately 10 to 11 years of age).

Survey questionnaires were reviewed by the SBNM Expert Panel and Interagency Work Group for content face validity, feasibility, and questionnaire layout. Protocols for administration of the questionnaires at school were also developed, based on similar protocols found in the literature (24-26), as well as the experience of the investigative team.

Focus group testing The questionnaires were tested for clarity and interpretation using focus groups and a cognitive interviewing technique in a convenience sample of elementary, middle, and high schools in central Texas. The feasibility of the protocol for questionnaire administration was also assessed during this phase of the project.

Most of the suggestions and comments by the students in the focus groups consisted of clarification of food terms. For example, students in the focus groups were unsure of whether or not sugar-flavored drinks were counted as 100% juice. Another example is that they knew which foods were high in fat, but did not generally consume fat-modified products, so it was unnecessary to include them on the instrument. Based on teacher feedback during the focus groups, question items and administration protocol was streamlined.

Cognitive interviews After focus group testing and revisions to the instruments, the questionnaires were further tested using a cognitive interview procedure (37,38). Questions were similar to those used for the focus groups but were updated to reflect changes that had been made to the survey as a result of the focus group data, and they also included specific concepts or items that seemed to be difficult for the students in the focus groups.

The interview process allowed clarification of the wording of several of the survey questions. For example, subjects were unclear about the

potato choices, so that question was reworded to reflect mashed potatoes, fried potatoes, and others. It was also found that students did not read instructions embedded within the questionnaire itself. Thus, instructions were shortened, and the majority of …