Respondent non-cooperation in surveys and diaries: an analysis of item non-response and panel attrition.

This paper analyses the impact of questionnaire design and length of participation on item non-response and panel attrition. Based upon the results of previous studies, the authors propose a framework that involves the mediating variables of participation fatigue and level of difficulty. From this framework, they develop a set of hypotheses on item non-response and panel attrition, which then are tested using a large database collected by AT & T. The results show that difficult questions in surveys and diaries lead to higher rates of item non-response. The study also finds that participation fatigue causes increased item non-response and, eventually, panel attrition. Thus, high initial rates of item non-response are warning signs for high subsequent attrition. However, those who stay on the diary panel do not necessarily show increasing rates of item non-response over time, because the effect of increasing participation fatigue is counterbalanced by the impact of decreasing difficulty, caused by answering the same questions repeatedly. In contrast, it was also found that because the diary panel increasingly comprises people who are relatively attrition-resistant, the rate of attrition decreases over time.

**********

Introduction

Marketers often rely on consumer surveys and diary panels to obtain a variety of information on consumer characteristics and behaviour. Among various problems associated with these methods is respondent non-cooperation, because these methods depend on the cooperation of respondents. The three specific types of respondent non-cooperation are under-coverage, attrition, and item non-response. The first problem of under-coverage occurs when units with certain characteristics are underrepresented in the survey or initial diary panel, usually because of refusals. This issue has received plenty of attention from researchers, and there exist well-established statistical techniques (e.g. stratified random sampling and quota sampling) to overcome the problem. In this study, we focus on the other two types of the respondent non-cooperation problem.

The second problem of attrition occurs when panellists drop out from diary panels, admittedly unavoidable unless operating under tightly controlled conditions (Winer 1983). This is especially true in the case of static (i.e. no replacement of leaving members), continuous-wave (i.e. frequently intermittent recording) diary panels. Numerous studies indicate that panel attrition is found everywhere and is non-random (Bucklin & Carman 1967; Winer 1983; Aneshenshel et al. 1989; Toh & Hu 1996; Falaris & Peters 1998), raising the concern that its cumulative effect attacks the internal validity of the study (Greenland 1977; Criqui 1979) by producing biased parameter estimates (Sussman 1964; Zabel 1998). Attrition also attacks the external validity of the study (Norris 1987), since selective attrition affects the characteristics of the sample, restricting the generalisability of the findings (Snow et al. 1992).

The third problem of item non-response occurs when surveys and diary records are returned with missing answers to some questions. Unlike attrition, item non-response has received relatively little attention from researchers, as pointed out by Ferber (1966). Since then, many researchers (Houston & Ford 1976; Craig & McCann 1978; Durand et al. 1983; Durand & Lambert 1988; Gronhaug et al. 1988) have expressed similar concerns about the lack of research on item non-response. Why is item non-response an important problem that merits further investigation? It is because the few studies conducted on item non-response reveal that it is, like attrition, also found everywhere and is seldom random (Francis & Busch 1975; Craig & McCann 1978; Douglas & Shoemaker 1981; Durand & Lambert 1988; Gronhaug et al. 1988; Kupek 1988). Therefore, to the extent that statistical modelling usually incorrectly assumes that incomplete answers are missing at random (Woolsen et al. 1992), systematic item non-response is an important issue that needs greater attention.

Besides the lack of adequate attention given to item non-response, the literature on respondent non-cooperation is descriptive in nature, without a single major article with a formal statement of hypotheses. Thus, the literature provides evidence of certain groups of people being less cooperative, but fails to explain why they return incomplete questionnaires, or drop out from panels. Furthermore, most studies on panel attrition and item non-response have analysed only one of the two issues without considering the other, leaving a lack of literature linking the two.