Polling

There are two related problems with political polling that have emerged over the last 8 years.  They both stem from what is otherwise a positive development:  the widespread use of polling averages to assess the state of political races.  The first problem is that pollsters have a strong incentive to bias their poll results toward the current average.  The counterintuitive result is that this makes each individual poll more accurate but makes the average of polls less accurate (and probably slower to react to real changes in the underlying data).

The second problem is that the rewards for conducting polls (which are expensive and are often paid for by news organizations) have declined significantly.  Again, I want to emphasize, this is partly for good reason:  individual polls were previously given too much attention, and it is healthy for them to be deemphasized.  But the downside is that it makes less and less sense to invest resources in high-quality polls.  In a sense, Nate Silver's work in 2008 may have been "out of equilibrium"—news organizations were still conducting lots of high-quality polls out of habit/inertia, giving Silver plenty to work with.  Over time, Silver's approach may become less powerful as it becomes starved of raw materials.

By the way, a related point is that news organizations are often criticized for hyping their own polls:  why are you reporting candidate X is up by 4 (as your poll indicates) when it is probably more accurate to report that candidate X is up by 7 (as the polling average indicates)?  There's some logic to this criticism, but it has the bizarre implication that after shelling out $50,000 (or whatever) you can't even report your own poll as straight news!  You can see why news organizations would start having second thoughts about how often to poll the field, and at what level of quality.

(You may be wondering if I have any evidence that news organizations are cutting back on the resources going into polling.  The answer is, a little.  Here's Nate Silver in 2014 saying "there are fewer high-quality polls than there used to be."  And there is Pew's shift away from horse-race polling, which Harry Enten of 538 called "unfortunate."  I am not sophisticated enough about polling to understand whether this is related to the issues I'm writing about, but it seems as though it might be.  In any case my worry is about where things are headed—the current level of polling seems adequate, though it is not everything we might wish for.)

[Update:  this piece by Nate Cohn describes the relative scarcity of polls in the 2016 race.]

Here's my (admittedly imperfect) solution.  News organizations that conduct high-quality polls could form a consortium and hire someone to create an average of their polls, all of which would meet some specified quality threshold.  When a new poll comes out, it is fed into the average before it is published, and then the organization that paid for the poll is allowed to be the first to publish both the poll results and the new, updated polling average (which is what everyone really wants to see).  The other media organizations need not wait long to post stories about the new average (I'm thinking minutes or hours at most), but the organization that actually shelled out the money for the poll would get some advantage for its money.  The organizations would have to stagger the release of their poll data for this to work, but not by much.

It's pretty clear how this would encourage media organizations to spend more on high-quality polling.  I also think it would discourage "herding" (the tendency to skew polls toward the pre-existing average) because the headline is less the new poll number and more the new, updated polling average.  If anything there might be reason to worry that the pollster would skew away from the existing average in order to make a more dramatic announcement.

I suppose there are some anti-trust type concerns.  Who gets to be in the consortium?  However, I'd observe that a monopolist might have better incentives than players in a competitive market.  We could use a high-quality poll of Missouri more than we could use the 20th poll of Ohio, right?  The consortium could make resource-allocation decisions in a more rational way for the collective good of the model.

Anyway look it's not a great idea but it seems as though it might work.