The red flags

The Zogby International polling firm has been using controversial methods for years, and this election cycle is no different. The Zogby Interactive Battleground Poll -- regularly cited in the blogosphere and published on the Wall Street Journal's Web site -- is an online poll conducted via e-mail. If this sounds dubious, that's because it is. Online polling has a spotty track record and remains an unproven method for gauging public opinion. Nancy Belden, president of the standard-setting American Association of Public Opinion Researchers, advises wariness about "any organization that uses online polling which necessarily excludes people who are not online." CNN polling chief Keating Holland agreed, saying online polling is more self-selecting and therefore less accurate. "When you're polling by telephone there's this big thing that makes this noise when I want to ask you questions. When you're polling by the Internet, there's no big bell." For the same reason, watch out for Harris Interactive polls.

Zogby does conduct telephone surveys that are less unorthodox, though they are still controversial.

Rasmussen Reports uses another questionable technique to gather its polling data: interactive voice technology (IVT), in which a computer does the calling and the interviewing. Though Rasmussen himself said that it is "easier to get people to talk to a computer than it used to be," polling units that use IVT have a reputation for low response rates. "It's a far cry from having an interviewer," Belden said. Holland does not let CNN report results from IVT polls. "I find [IVT] polls unreliable," he said. "I've actually been polled, and it was far too easy to screw around with it, which I did." He added, "People feel a bigger obligation to tell the truth to a real person." SurveyUSA, another prolific polling organization, also uses IVT.

Also watch out for Rasmussen's tracking polls, which combine the IVT trouble with the problems of a tracking poll. Tracking polls -- which are updated daily -- tend to be volatile and unreliable because "different kinds of people are at home on different nights of the week," Blendon explained. Be wary of any tracking poll you see -- even from the big names.

American Research Group (ARG) polls are out all the time with fresh numbers from battleground states. The organization, however, would not release the most vital piece of information about its work -- who pays for it -- when Salon asked. ARG spokesman Dick Bennett told Salon that "news organizations" generally subscribe to fund ARG polls. But without full disclosure, the survey data cannot be completely trusted.

Local state polls, often conducted by cash-strapped local newspapers or television stations, are also often unreliable. "Doing it and doing it well is expensive," Blendon said. Pinkus said that there are some good local polling organizations, such as Quinnipiac University in Connecticut. But sticking with the more reputable national organizations, even for local numbers, is safer. Also suspicious: Polls sponsored by a political campaign or interest group. Local news organizations sometimes pick these polls up, but they are notoriously biased. "Sometimes they almost beat the [respondents] into answering a certain way," Blendon said.

All the experts Salon contacted agreed that the worst polls out there are the fantastically useless Internet surveys like those CNN's Lou Dobbs asks his viewers to fill out on his Web site. When he says they are unscientific, he means it. They're really just for fun.

Note that a lot of the red-flagged organizations come out with polls early and often. That's because IVT or online survey technology makes polling cheap and quick -- and too good to be true. But because of the seductive frequency of the polls, they show up all over, particularly in the blogosphere. But take their results with a grain of salt.

Polls not included...

These ratings are not comprehensive; there are other good and bad polls out there. In general, trust a poll only as much as you trust the news source, and be wary of most polls published online -- even on big-name sites like the Wall Street Journal's, which regularly posts Zogby Interactive polls with a lot of flashy graphics. Polls that would never make it into print or on TV regularly show up online, just for the amusement of readers. Finally, good polls should be digested for what they are worth -- they are just snapshots of public opinion at one time, not a prediction about November.

If polling is to drive much of the election coverage this year, as it already has, becoming an informed poll watcher is more vital than ever. The bottom line: Choose a few solid polls to follow and ignore the rest. Be sure to look for agreement among those polls and be wary of outliers -- that is, polls that disagree with the pack. Because even the best pollsters are wrong sometimes.

Recent Stories