Margin of Error.(Mobocracy: How the Media's Obsession with Polling Twists the News, Alters Elections, and Undermines Democracy)

Mobocracy: How the Media's Obsession with Polling Twists the News, Alters Elections, and Undermines Democracy, by Matthew Robinson (Prima, 377 pp., $24.95)

Polls can provide useful information, but only if readers understand their limitations. The caveat is increasingly important as the media take polls with increasing frequency and decreasing quality. Where reporters once used polls in their stories, the polls now often make headlines themselves. But reporters rarely provide the information necessary to weigh the worth of poll results.

That failure and its consequences are the subject of Matthew Robinson's book. The problem he identifies is not merely the ubiquity of polling, but what happens when bad polls combine with sloppy or biased reporting.

In early 2001, for example, several media outlets reported that a Newsweek poll had found that a plurality (41 percent) of the public thought the Senate should reject John Ashcroft's nomination as attorney general. Few of these outlets reported how Newsweek had reached this conclusion: by asking respondents, "Do you think Congress should approve Bush's choice of John Ashcroft for Attorney General, or reject Ashcroft as too far to the right on issues like abortion, drugs, and gun control to be an effective Attorney General?" It was a loaded question. Only the charge against Ashcroft was provided. When ABC ran a poll mentioning only that Bush had nominated Ashcroft, it found 54 percent support for his confirmation. A poll is only as good as its wording.

Readers also need to know the order in which the questions were asked. An unbiased question can get skewed results if the previous question was loaded. It's also important to know who was polled. When Al Gore picked Joe Lieberman as his running mate, George W. Bush's lead over him dropped from 17 points to 2 points in one day. But the shift was illusory. Gallup had switched from questioning only "likely voters" in the 17-point poll to questioning "registered voters." Among registered voters, Republicans are more likely to vote than Democrats; hence the discrepancy. Within a week, a Gallup poll confirmed that even post- Lieberman, Bush had a 16-point lead among likely voters.

As Robinson writes, lazy reporting on Gallup polls produced a similar effect in 1992. There are still Republicans who insist that the first President Bush was surging toward the end of that year's campaign, and lost momentum only when Iran-contra prosecutor Lawrence Walsh decided to indict a prominent Republican the weekend before the election. But Robinson shows that the apparent surge-and-collapse of Bush's numbers was an artifact of changes in poll methods.

The Lewinsky scandal was a milestone for polling -- had the polls gone against Clinton, he might well have been removed. The poll results, in turn, were affected by the media's decision to ask the public whether Clinton should be removed from office, rather than whether Starr should submit a report, whether the House Judiciary Committee should hold ...