Response rates in HRM/OB survey research: norms and correlates, 1990-1994. (Human Resource Management and Organizational Behavior)

Low response rates are a substantial problem to many survey researchers. Researchers find that the responses obtained from only a portion of their sample may not represent the full sample (Fowler, 1984; Viswesvaran, Barrick, & Ones, 1993) due to problems such as selective returns, volunteer bias, and other phenomenon. This leads to potentially serious concerns about the external validity of the entire research project. External validity concerns have led researchers in marketing, sociology, and public opinion measurement to analyze many response-enhancing techniques. Most of the research has utilized mail surveys to assess consumer sentiments (e.g., Armstrong & Lusk, 1987). Unfortunately, there is substantially less information about how industrial samples react to these response enhancing techniques. This leaves Human Resources Management and Organizational Behavior researchers with little understanding of the phenomenon and few guidelines to enhance response rates in their projects.

The purposes of this article are three. First, it examines response rate research in marketing, sociology, and public opinion measurement. Second, it develops and tests a model of response rates in HRM/OB research. Third, it examines normative levels of response rates in HRM/OB research to define high, medium, and low levels of response.

The Need for Response Rate Research

There has been relatively little analysis of response rates in HRM/OB research. l Most of the information available to researchers is the conventional wisdom set down in texts on surveys. These texts suggest that a 50% response rate is usually considered adequate (Babble, 1990; Dillman, 1978; Rea & Parker, 1992), though some authors suggest 60% is needed (Fowler, 1984). Sixty percent is usually considered to be a good response rate, while 70% is considered very good (Babbie, 1990; Rea & Parker, 1992). Other authors suggest that good sampling and questionnaire design should lead to an 80% response rate (De Vaus, 1986) and several authors give readers few, if any, guidelines (Converse & Presser, 1986; Fink & Kosecoff, 1985; Rossi, Wright, & Anderson, 1983).

The lack of guidelines leads to a practice in most articles of justifying response rates with references to obscure articles or other articles in the same area with lower response rates. The conclusion of such articles is usually that "this is regarded as a relatively high response rate for a survey of this type" (Babbie, 1990, p. 182).

One of the problems associated with the conventional wisdom is that it does not appear to be based on empirical investigation. One is hard pressed to find any systematic examination of response rates to justify the conventional norms. Neither is there a strong review of literature in other areas to guide the research procedures and understanding of survey researchers in HRM/OB.

Prior Research on Response Rates

Most research to understand response rates has taken place in marketing, sociology, and public opinion measurement. The dominant methodology is to examine mail-out surveys to consumers, rather than industrial populations. Much of the research has focused on enhancing response rates by modifying research design or looking at the characteristics of issues or respondents.

Primary Variables

The research has led to a number of interesting ideas about techniques associated with increased response rates in fields other than HRM/OB. Table 1 summarizes some of the major conclusions from areas such as marketing, sociology, and public opinion measurement.

Advance notice. Higher response rates are associated with sending advance notice to subjects (Duncan, 1979; Harvey, 1987; Heberlein & Baumgartnet, 1978; Linsky, 1975; Martin, Duncan, Powers, & Sawyer, 1989; Nowack, 1990). [TABULAR DATA FOR TABLE 1 OMITTED] Meta-analyses show that advance notification is associated with increased response rates over conditions in which there is no advance notification (Bruvold, Comer, & Rospert, 1990) from approximately 7.7 percentage points (Fox, Crask, & Jonghoon, 1988) to 17.6 percentage points (Yammarino, Skinner, & Childers, 1991). This suggests that the addition of advance notice could raise response rate in a given study from 50% to between 58% and 68%.

Follow-ups. Higher response rates are also associated with follow-up reminders to respond (Duncan, 1979; Kanuk & Berenson, 1975; Harvey, 1987; Heberlein & Baumgartner, 1978; Linsky, 1975; Salant & Dillman, 1994). One meta-analysis of follow-up post cards found that cards were associated with an increase of 3.5 percentage points (Fox, et al., 1988), while other meta-analyses suggest increases of up to 18 percentage points (Yammarino, et al., 1991) above control groups without various forms of follow-ups. One researcher suggests that advance notice and follow-ups are somewhat interchangeable. The advance notice should be counted as one follow-up. The researcher also notes that the usefulness of additional contacts diminishes markedly after the third contact (Martin et al., 1989).

Monetary incentives. Higher response rates are associated with the use of monetary incentives such as including a dollar bill in the questionnaire (Biner, 1988; James & Bolstein, 1990; Harvey, 1987; Heberlein & Baumgartner, 1978; Kanuk & Berenson, 1975; Linsky, 1975; Nowack, 1990). Meta-analyses suggest that monetary incentives can be associated with markedly higher response rates. Monetary incentives were associated with increased response rates 12% to 19% over control groups depending on the size of the incentive (Church, 1993; Yammarino, et al., 1991). The enclosure of $1 was associated with a response rate 19% higher than no reward (Hopkins & Gullickson, 1993).

Personalization. Personalization of the cover letter and/or address of mail surveys has been associated with higher response rates (Bruvold, et al, 1990; Dillman, 1978; Duncan, 1979; Nowack, 1990; Salant & Dillman, 1994). Studies suggest that the various methods of personalizing surveys are associated with response rates four percentage points (Yammarino, et al., 1991) to nine percentage points (Martin, et al., 1989) over control groups.

Salience. The salience of the issues investigated is associated with higher response rates. For example, a survey addressing worker satisfaction with pay and benefits might be more noteworthy than a survey addressing norms about feedback. One investigation found that salience had the strongest relationship of any issue or research design decision including advance notice, follow-up contacts, and monetary incentives (Heberlein & Baumgartner, 1978). These investigators coded the salience of the issue to subjects in each of 155 surveys. Salience was defined as a topic that dealt with an important issue that was also current or timely. For example, a survey mailed to veterans groups dealing with revisions to educational benefits would be considered salient. Topics were coded as possibly salient if the topic was important but not necessarily current or timely. For example, a survey of the occupational mobility of former students would be possibly salient.

Heberlein & Baumgartner (1978) found that the average response rate was 77% …