Tuesday, September 21, 2004

Polls Fluctuate Wildly. Does Our National Media Explain Why? Of Course Not.

What happens when three national polls show Kerry and Bush separated by one percentage point or less, and two polls show Bush leading by eight percentage points or more?

Well, if you're a television talking head or a millionaire pundit, you do nothing.

Oh sure, they report the obvious. Poll A shows this and Poll B shows that. Maybe they offer their own take on things, suggesting that the truth may fall between the wildly disparate polls -- maybe Bush is actually leading by six or eight percentage points. But then they yawn and move on to the rest of the news day.

At the newsletter I work for, Real Estate Alert, about half of our stories each week regard an effort to sell a major commercial property, such as an office high-rise or a regional mall. To do my job, I might talk to a half-dozen sources for a particular story, fetching background information on a property, with a top concern being how much the property may sell for.

Now, let's say hypothetically I spoke with three sources, and two of them speculated that such-and-such office tower in New York was worth $200 million, and the third guesstimated it was worth $300 million, I would have a decision to make. I could a) ignore the disparate price and go with the majority opinion, b) split the difference and say the building was worth $250 million, or c) go back to the source saying $300 million, and find out why they came to that conclusion.

Obviously, I'd try to find out why the third source's opinion differed so greatly from the other two. Even if I thought the third source shed new light on the building's value, I might go to a fourth or fifth source, as well as going back to the sources speculating $200 million. Ultimately, I would try to get everyone on the same page, so that I could form a consensus opinion.

But that's me writing about an office building. Can the media actually determine which of the disparate polls -- the three showing a close race, or the two show a hefty Bush lead -- are accurate?

Of course they can, if they were willing to do some homework.

Polling is a science, of course. And every science is studied.

Just as quickly as the media was able to find knowledgeable sources who could provide reasons why the Killian documents were fakes, they should have been able to find knowledgeable sources who could provide reasons why the two polls showing hefty Bush leads -- from USA Today/CNN/Gallup and CBS News/New York Times -- were inaccurate gauges of voter opinions.

Several knowledgeable sources -- some partisan, some not -- found that those two polls oversampled Republicans.

What does that mean? Nonpartisan pollster John Zogby points out that in 1996, 39% of voters identified themselves as Democrats. Meanwhile, 34% identified themselves as Republicans, and 27% said they were independents. In 2000, the results were similar: 39% Democrats, 35% Republicans, 26% independents.

Most polls, therefore, shoot for those percentages when they conduct national polls. But the Center for American Progress reports that the CBS/NYT poll included 4% more Republicans than Democrats. And Gallup told blogger Steve Soto, of TheLeftCoaster.com, that it surveyed 7% more Republicans than Democrats.

When the data conforms to the party alignment in 2000, the CBS/NYT poll would have given Bush a one-point lead, 47-46, and the Gallup poll would have resulted in a tie, 48-48.

***

Now, I'm not suggesting that Bill O'Reilly or Bob Schieffer should understand the intracacies of polling science. That wouldn't be reasonable. But before they go offer their opinions on the Gallup and CBS/NYT polls, shouldn't they have bothered to do their homework? Find someone who understands polling and can explain it to them before they go on the air?

Hey, this isn't some liberal fantasy. Even the Wall Street Journal, in its issue Monday, reported that Gallup and CBS/NYT surveyed more Republicans than other polls showing the race to be a dead heat.

But you know those television talking heads and millionaire pundits. They aren't paid to do their homework. They're paid to state the obvious (if they are a television talking head) and offer a snap judgment (if they are a millionaire pundit). Why should they actually be responsible when discussing some major issue with the American people?

Listen to your media:

-- Fox News' Bill O'Reilly: "Gallup is pretty reliable."
-- Fox News' John Gibson said of Gallup: "That is a very steady poll over the years; it's not given to wild fluctuations. And it's considered pretty reliable."
-- CBS' Bob Schieffer: "George Bush has now opened a nine-point lead over John Kerry. You don't have to be an expert to figure that out. Voters may be less than enamored with President Bush but they are even more uneasy about John Kerry, whose plans for the country remain a mystery to them, according to this poll."
-- MSNBC's Pat Buchanan, who offered that while he didn't believe the Gallup or CBS/NYT numbers, opined that Bush's real lead was probably "six, seven, eight points."

... and on an on.

Maybe it's not coincidental that the ones trumpeting the faulty polls the loudest were conservatives.

Still, at some point the television big-shots -- not the Buchanans and Gingriches of the world, but rather folks like Chris Matthews and Judy Woodruff -- have to earn their money, don't they? If they are going to report on polls, don't they have to do a little homework first?

2 Comments:

Anonymous Anonymous said...

Such reports, which disputably show Bush with a sizeable lead, are said to reduce press coverage for the perceived loser in the crucial final weeks of the campaign, cause a sense hopelessness in the party, and force voters who might have otherwise ventured out on Nov. 2 to stay home, convinced the thing is already over.
In the name of objectivity, the media should require of, and broadcast, from each poll careful descriptions of how it has arrived at its numbers, including disqualifiers. Any poll and media outlet which refuses to include such data should be immediately discredited.

9:13 AM  
Anonymous Anonymous said...

Just to throw in two cents, dailyhowler.com adds this CNN transcript featuring their polling guru, William Schneider.

Before you read the transcript, be aware of two things: 1) Schneider is linked with the conservative American Enterprise Institute, a fact that I don't remember CNN ever offering. 2) as was pointed out in the original post, CNN is a co-sponsor of the Gallup poll.

Anyway, read along as Schneider shows off his cluelessness ...

WILLIAM SCHNEIDER (9/17/04): We have three national polls that show the race a sudden dead heat. One shows Bush ahead by a point, one shows Kerry ahead by a point and one shows an absolute tie...One, the Gallup poll, does show Bush significantly ahead, 13 points among likely voters, eight points among registered voters, but that poll appears to be the outlier.

FREDERIKA WHITFIELD: One point versus 13 points, huge difference there. Is there a reasonable explanation for these disparities?

SCHNEIDER: Well, the most reasonable explanation that I can figure out is that the polls that show a close race all ask people how they would vote after they ask voters a series of questions about the issues, how they feel about the economy, about the war in Iraq, about health care, about the candidates and at the end of that, they ask people, “Now, how will you vote in this election if you had to vote right now, Bush or Kerry?” In those polls, the race is very close. The Gallup poll, which is the only one that shows Bush significantly ahead, asks people how they would vote right at the outset. They ask people before they investigate the issues.

What does that suggest? It suggests that when people think about the issues, they’re more inclined to vote for John Kerry and we've seen that before. Voters who say their decision is based on the issues, tend to favor Kerry. Voters who say they are voting for candidates on person grounds tend to favor Bush.

12:49 PM  

Post a Comment

<< Home

Listed on BlogShares