Tuesday, June 29, 2010

Why you shouldn't trust polls

The Daily Kos has been running a series of State of the Nation polls for the last year and a half. Today they announced that those polls appear to have been complete bunk, the results at least manipulated if not completely made up. This was only discovered by statistical analysis of the data about the polls and polling process the Daily Kos published along with the poll results.

Now, the polls weren't from some fly-by-night operation. Research 2000 was one of the most respected polling houses in the business. But the one thing that strikes me is that, despite that, they simply had a lousy track record at actually correctly predicting outcomes. To me it seems obvious that if the polls are being done right and are accurate then they should closely match the actual outcome, and if they don't match the outcome then the rest doesn't matter because the polls are useless for prediction. If anyone's reporting those polls knowing about their track record, they're using those polls for something other than simply tracking actual sentiment on an issue.

My advice: never trust a poll unless the data underlying it is published for independent review. And never trust a poll or the use of that poll when the polling house doesn't have a good track record of getting the outcomes right.

No comments: