Oct. 26: State Poll Averages Usually Call Election Right
By NATE SILVER
915 Comments
The FiveThirtyEight forecast model has found the past several days of battleground state polling to be reasonably strong for Barack Obama, with his chances of winning the Electoral College increasing as a result. The intuition behind this ought to be very simple: Mr. Obama is maintaining leads in the polls in Ohio and other states that are sufficient for him to win 270 electoral votes.
Friday featured a large volume of swing state polling, including three polls of Ohio, each of which showed Mr. Obama ahead by margins ranging from two to four percentage points.
Between Ohio and the other battleground states, Mr. Obama held leads in 11 polls on Friday, against four leads for Mitt Romney’s and two ties. Mr. Romney’s leads came in North Carolina and Florida, two states where the FiveThirtyEight forecast already had him favored.
To the extent that there was a trend in the state polls, it was slightly favorable for Mr. Obama. Among the eight polls that had previously published numbers after the first presidential debate in Denver, Mr. Obama gained about one percentage point, on average.
Mr. Romney made gains in four of the five polls that had last surveyed the race before Denver. Nevertheless, his average gain in the polls – 2.4 percentage points – was less than the 4-point bounce he was seeing in the immediate aftermath of the Denver debate. This suggests that Mr. Romney’s bounce has receded some since his post-Denver peak.
The national polls out on Friday were not terribly newsworthy. Mr. Obama had a miniscule lead of 0.2 points between the eight national tracking polls that were published, reversing an equally small 0.2-point advantage for Mr. Romney in the same surveys on Thursday.
You can see here my thoughts on reconciling the differences between state and national polls. They may be reflective of a potential split outcome between the popular vote and the Electoral College, but there are other plausible hypotheses as well. Specifically, it could be that the national polls slightly underrate Mr. Obama’s position, that the state polls slightly overrate it, or both.
The FiveThirtyEight forecast has Mr. Obama leading the popular vote along with the Electoral College, because it uses both state and national polls to calibrate its estimate of where the vote stands. Also, however, Mr. Obama’s state polls were adjusted slightly downward because his national polls remain middling.
Still, our state-by-state forecasts are extremely similar to those issued by our competitors.
For example,
we had Mr. Obama projected to win Ohio by 2.4 percentage points as of Friday. That compares to a 2.3 percentage-point lead for Mr. Obama in the Real Clear Politics average of Ohio polls, a 2.9-point advantage for him in the Huffington Post Pollster model, and a 2.7-point edge for him according to Talking Points Memo’s Poll Tracker.
How often does a lead of two or three points in the polling average, with 10 days to go until the election, translate into a victory in the state?
This is the sort of question that the FiveThirtyEight forecast is designed to address. But a simpler method is to just look at what happened when candidates held similar advantages in the past.
In the table that follows, I have attempted to recreate a simple polling average for competitive states in past elections, using about the same rules that Real Clear Politics applies.
In particular, I’ve looked at all states in our database in which there were at least three distinct polling firms that conducted surveys in the window between 10 days and three weeks before the election. Like Real Clear Politics, I used only the most recent poll (the one closest to the 10-day cutoff) if the polling firm surveyed the state multiple times during this period. I used the version of the poll among likely voters if it was available, defaulting to registered voter numbers otherwise.
In the table, I’ve listed all cases in which the race was within the single digits in the polling average. If you focus on those cases where a candidate held a lead of two to three percentage points, he won the state in all six out of six cases, although the sample size was small.
Historically, this two- to three-point range has been something of an inflection point. Poll leads of 1.5 percentage points or less have been very tenuous and have not conveyed much advantage.
On the other hand, there was not a single instance in the database where a candidate lost a state when he held a lead of more than 3.5 points in the polling average at this point in time. (Bill Clinton, in 1992, lost Texas despite leading George H.W. Bush there by that margin.)
It is possible to generalize these findings by means of a probit regression model, where the independent variable is the candidate’s lead in the polling average and the dependent one is whether he won or lost the state.
That analysis implies that a lead of 2.4 percent in the polling average (Mr. Obama’s current edge in Ohio in the FiveThirtyEight model) would translate to a win in the state 82 percent of the time. This percentage is similar to, but slightly higher than, the FiveThirtyEight forecast, which gave Mr. Obama a 76 percent chance of winning Ohio as of Friday.
It is important to emphasize that this analysis covers cases in which there were at least three distinct polling firms active in a state; you will find more frequent misses in cases where there were just one or two polls.
In Ohio, however, there are not just three polls: roughly a dozen polling firms, rather, have surveyed the state over the past 10 days.
There are no precedents in the database for a candidate losing with a two- or three-point lead in a state when the polling volume was that rich.
Instead, the biggest upsets in states with at least five polls in the average came in 2000, when George W. Bush beat Al Gore in Florida, and in 2008, when John McCain beat Mr. Obama in Missouri. Mr. Obama and Mr. Gore had held leads of 1.3 percentage points in the polling averages of those states.
If you look at the actual track record of state polling averages, it may even seem as though the FiveThirtyEight forecast is being conservative in giving Mr. Obama “only” a 76 percent chance of winning Ohio. I do not necessarily think that is the case.
The state-by-state polling averages have performed very well in recent years, but that is not likely to have been the case in, for example, 1980, when Ronald Reagan substantially beat his polls on Election Day. Years like 1980 are not represented very well in the tables above, because there were few states with rich polling that year. But they are considered by the FiveThirtyEight model, which calibrates its estimates of uncertainty based on the performance of state and national polls dating back to 1968.
Still, it is misinformed to refer to Ohio as a toss-up. Mr. Obama is the favorite there, and because of Ohio’s central position in the Electoral College, he is therefore the overall favorite in the election.
The FiveThirtyEight forecast model has found the past several days of battleground state polling to be reasonably strong for Barack Obama, with his chances of winning the Electoral College increasing as a result. The intuition behind this ought to be very simple: Mr. Obama is maintaining leads in the polls in Ohio and other states that are sufficient for him to win 270 electoral votes.
Friday featured a large volume of swing state polling, including three polls of Ohio, each of which showed Mr. Obama ahead by margins ranging from two to four percentage points.
Between Ohio and the other battleground states, Mr. Obama held leads in 11 polls on Friday, against four leads for Mitt Romney’s and two ties. Mr. Romney’s leads came in North Carolina and Florida, two states where the FiveThirtyEight forecast already had him favored.
To the extent that there was a trend in the state polls, it was slightly favorable for Mr. Obama. Among the eight polls that had previously published numbers after the first presidential debate in Denver, Mr. Obama gained about one percentage point, on average.
Mr. Romney made gains in four of the five polls that had last surveyed the race before Denver. Nevertheless, his average gain in the polls – 2.4 percentage points – was less than the 4-point bounce he was seeing in the immediate aftermath of the Denver debate. This suggests that Mr. Romney’s bounce has receded some since his post-Denver peak.
The national polls out on Friday were not terribly newsworthy. Mr. Obama had a miniscule lead of 0.2 points between the eight national tracking polls that were published, reversing an equally small 0.2-point advantage for Mr. Romney in the same surveys on Thursday.
You can see here my thoughts on reconciling the differences between state and national polls. They may be reflective of a potential split outcome between the popular vote and the Electoral College, but there are other plausible hypotheses as well. Specifically, it could be that the national polls slightly underrate Mr. Obama’s position, that the state polls slightly overrate it, or both.
The FiveThirtyEight forecast has Mr. Obama leading the popular vote along with the Electoral College, because it uses both state and national polls to calibrate its estimate of where the vote stands. Also, however, Mr. Obama’s state polls were adjusted slightly downward because his national polls remain middling.
Still, our state-by-state forecasts are extremely similar to those issued by our competitors.
For example,
we had Mr. Obama projected to win Ohio by 2.4 percentage points as of Friday. That compares to a 2.3 percentage-point lead for Mr. Obama in the Real Clear Politics average of Ohio polls, a 2.9-point advantage for him in the Huffington Post Pollster model, and a 2.7-point edge for him according to Talking Points Memo’s Poll Tracker.
How often does a lead of two or three points in the polling average, with 10 days to go until the election, translate into a victory in the state?
This is the sort of question that the FiveThirtyEight forecast is designed to address. But a simpler method is to just look at what happened when candidates held similar advantages in the past.
In the table that follows, I have attempted to recreate a simple polling average for competitive states in past elections, using about the same rules that Real Clear Politics applies.
In particular, I’ve looked at all states in our database in which there were at least three distinct polling firms that conducted surveys in the window between 10 days and three weeks before the election. Like Real Clear Politics, I used only the most recent poll (the one closest to the 10-day cutoff) if the polling firm surveyed the state multiple times during this period. I used the version of the poll among likely voters if it was available, defaulting to registered voter numbers otherwise.
In the table, I’ve listed all cases in which the race was within the single digits in the polling average. If you focus on those cases where a candidate held a lead of two to three percentage points, he won the state in all six out of six cases, although the sample size was small.
Historically, this two- to three-point range has been something of an inflection point. Poll leads of 1.5 percentage points or less have been very tenuous and have not conveyed much advantage.
On the other hand, there was not a single instance in the database where a candidate lost a state when he held a lead of more than 3.5 points in the polling average at this point in time. (Bill Clinton, in 1992, lost Texas despite leading George H.W. Bush there by that margin.)
It is possible to generalize these findings by means of a probit regression model, where the independent variable is the candidate’s lead in the polling average and the dependent one is whether he won or lost the state.
That analysis implies that a lead of 2.4 percent in the polling average (Mr. Obama’s current edge in Ohio in the FiveThirtyEight model) would translate to a win in the state 82 percent of the time. This percentage is similar to, but slightly higher than, the FiveThirtyEight forecast, which gave Mr. Obama a 76 percent chance of winning Ohio as of Friday.
It is important to emphasize that this analysis covers cases in which there were at least three distinct polling firms active in a state; you will find more frequent misses in cases where there were just one or two polls.
In Ohio, however, there are not just three polls: roughly a dozen polling firms, rather, have surveyed the state over the past 10 days.
There are no precedents in the database for a candidate losing with a two- or three-point lead in a state when the polling volume was that rich.
Instead, the biggest upsets in states with at least five polls in the average came in 2000, when George W. Bush beat Al Gore in Florida, and in 2008, when John McCain beat Mr. Obama in Missouri. Mr. Obama and Mr. Gore had held leads of 1.3 percentage points in the polling averages of those states.
If you look at the actual track record of state polling averages, it may even seem as though the FiveThirtyEight forecast is being conservative in giving Mr. Obama “only” a 76 percent chance of winning Ohio. I do not necessarily think that is the case.
The state-by-state polling averages have performed very well in recent years, but that is not likely to have been the case in, for example, 1980, when Ronald Reagan substantially beat his polls on Election Day. Years like 1980 are not represented very well in the tables above, because there were few states with rich polling that year. But they are considered by the FiveThirtyEight model, which calibrates its estimates of uncertainty based on the performance of state and national polls dating back to 1968.
Still, it is misinformed to refer to Ohio as a toss-up. Mr. Obama is the favorite there, and because of Ohio’s central position in the Electoral College, he is therefore the overall favorite in the election.
No comments:
Post a Comment