The impact of material incentives on response quantity, response quality, sample composition, survey outcome, and cost in online access panels.

Two incentive experiments were conducted in different online access panels. Experiment 1 was carried out in a commercial market research panel. It examined whether three different types of promised incentives (redeemable bonus points, money lottery and gift lottery), four different amounts of bonus points or raffled money, and two different denominations of raffled money influenced response quantity, sample composition, response quality and survey outcome. Type of incentive and number of bonus points mildly influenced dropout and sample composition. Moreover, response was higher with bonus points than with the two types of lotteries. Response quality and survey outcome were not affected. Experiment 2 was conducted in a non-profit panel, which holds one half self-selected and one half non-self-selected participants. Incentives were two different amounts of raffled money in two different denominations. Response, dropout, response quality, survey outcome and sample composition were not affected. Based on a cost-benefit analysis, recommendations for employing incentives in online access panels are given.

**********

Introduction

This article deals with theoretical and methodological issues surrounding the use of incentives in online access panels. An online access panel is a pool of people who have agreed to repeatedly take part in web surveys (Goritz et al. 2002). Online panels are an important, if not the dominant, form of reactive web-based research in the medium term (Couper 2000). A panel can be used as a sampling source for thematically and methodologically diverse studies. In contrast to ad hoc recruitment, online panels reduce the cost associated with locating appropriate respondents and ensure their immediate availability, along with additional benefits such as easy identification of key sample segments, increased response, augmented response quality, shorter field times and ethical advantages (Goritz 2002). Moreover, on the basis of previously collected data, respondents' answers can be cross-referenced and thereby validated (Goritz & Moser 2000) and questionnaires can be limited to relevant items, that is, panellists do not need to answer the same items again and again (for example age and sex).

Economic and research success with online access panels go hand in hand with learning how to induce panellists to participate in the surveys. A popular form of motivating people to take part in a survey is to offer incentives. In offline situations, researchers widely agree that if incentives are to increase response, they need to be given in advance, instead of being made contingent on the return of the questionnaire (Armstrong 1975; Linsky 1975; James & Bolstein 1992; Church 1993). If this holds with online panel surveys, researchers need to find pre-paid incentives appropriate for the web, for instance inclusion of electronic vouchers from online shops or redeemable loyalty points with the invitation email. Even money or credit can be sent by using products such as PayPal (www.paypal.com). Online money, however, is not 'in the hand' and collecting it can be cumbersome (depending on the user's web literacy and willingness to register with a transaction party). However, perhaps in online panels promised incentives work better than offline, because most panellists, have agreed to participate in advance and are accustomed to receiving an incentive after their participation.

There are other differences between the use of incentives in offline surveys and those conducted in online access panels. Unless panellists have unlimited internet access (for example for a fixed monthly fee or at university/work) they have to pay for their online session (e.g. provider fees, telephone charge), whereas in most cases offline, researchers incur the monetary cost of participation. Thus, when collecting data online, incentives might be necessary to counteract a bias toward respondents who have a free or flat-rate internet connection. Another difference is that, on the one hand, panellists in the relatively anonymous medium of the internet might be more distrustful that, for instance, a lottery actually exists and will be run fairly (Porter & Whitcomb 2003). On the other hand, it can be assumed that with online panellists, a certain degree of trust has already been established. Furthermore, every panellist's participation history is recorded. As this information can be used to decide who is invited to future studies, panellists might think twice whether they decline an invitation (Goritz 2004a). Another difference is that participation in online panel surveys compared to face-to-face studies is often more appealing, comfortable, easy, and saves time (Goritz 2002). Consequently, incentives might have less impact in online panels than in offline surveys. Yet, incentives are probably important because panellists are requested to participate repeatedly and one cannot avail oneself of the curiosity-bonus of one-time studies.

However, incentives are not only beneficial, but can also bring about a series of potential dangers to survey validity. Some challenges are the same offline and online: incentives might attract particular respondents and thus alter the sample's composition. But researchers can also take advantage of such knowledge and tailor incentives to particular target groups. In the same vein, the survey's outcome can be influenced by incentives. For instance, incentives can affect the participants' mood, which might lead to mood-congruent or mood-incongruent answering. Or, certain incentives (for example contributions to charity) might alter participants' attitudes toward the interviewing agency and thereby influence their statements. Finally, offering material incentives can drive intrinsically motivated participants away from the survey (Deci 1971).