Oct. 21: Uncertainty Clouds Polling, but Obama Remains Electoral College Favorite
By NATE SILVER475 Comments
The bad news for President Obama: it’s been almost a week since the second presidential debate, in Hempstead, N.Y., one that instant-reaction polls said was a narrow victory for him. But there is little sign that this has translated into a bounce for Mr. Obama in his head-to-head polls against Mitt Romney. Instead, the presidential race may have settled into a period of relative stability.
There is bad news for Mr. Romney as well, however. The “new normal” of the presidential campaign is considerably more favorable for him than the environment before the first debate, in Denver. However, it is one in which he still seems to be trailing, by perhaps 2 percentage points, in the states that are most vital in the Electoral College.
The FiveThirtyEight forecast was essentially unchanged again on Sunday, with Mr. Obama retaining a 67.6 percent chance of winning the Electoral College, little different from his 67.9 percent odds on Friday and Saturday.
These estimates might seem to be incongruous with national polls that show a nearly tied race. But the FiveThirtyEight method is, principally, an Electoral College simulation, and therefore relies more heavily on state-by-state polls. Our estimates of the popular vote in the critical states are highly similar to those of other Web sites that use different methods to calculate them.
This is expressed in the chart below, which compares the current FiveThirtyEight “now-cast” in each battleground state (our snapshot of what would happen in an election held today) against those of three of our competitors, RealClearPolitics, HuffPost Pollster and Talking Points Memo.
There are only three states that are “called” differently by the different methods. In Colorado, RealClearPolitics gives a slight edge to Mr. Romney, while the other three sites have Mr. Obama just ahead. However, it would be easy to overestimate the importance of these differences. RealClearPolitics has Mr. Romney only 0.2 percentage points ahead there, while the most favorable result for Mr. Obama, from The Huffington Post, has him up by 1.3 percentage points.
In Virginia, RealClearPolitics and The Huffington Post show an exact tie, while FiveThirtyEight and Talking Points Memo give a slight advantage to Mr. Romney.
Finally, in New Hampshire, FiveThirtyEight and The Huffington Post have Mr. Obama ahead, while the other two sites have the race going for Mr. Romney.
Still, these states — Virginia, Colorado and New Hampshire — are not quite at the electoral tipping point. Instead, Mr. Obama could win the Electoral College by winning Ohio, Wisconsin, and either Iowa or Nevada.
All the methods have Mr. Obama just ahead in each of these states. In Iowa, his margin runs from 1.5 percentage points to 2.5 percentage points, according to the different methods, with FiveThirtyEight representing the lowest point in the consensus. In Nevada, it is between 2.1 points and 5.2 points, with FiveThirtyEight again reflecting the lowest end of the range.
FiveThirtyEight is somewhere in the middle of the pack in Ohio, where the different methods give Mr. Obama an advantage of between 1.4 and 2.6 points, and in Wisconsin, where the estimates run between a 2.8- and a 3.8-point advantage.
One can certainly debate how to translate Mr. Obama’s apparent edge in each state into his overall probability of winning the Electoral College. It is a challenging problem from a mathematical standpoint, particularly given that the states cannot be expected to behave independently from one another. (That is, if Mr. Obama underperforms his polls in one swing state like Ohio, he is also more likely to do so in similar ones like Iowa.) It is a problem that we thought about very carefully in designing the model this spring, and in advance of the 2008 election. But because there have been only about eight presidential elections in which there has been a rich amount of state-by-state polling data, any approach will inherently be speculative to some degree.
At the same time, off-the-cuff or “gut-feel” estimates of these probabilities are probably not a good substitute for an effort to determine them through a more rigorous, empirical and rule-driven means. Our intuitive estimates of probabilities are normally quite poor, especially in situations that involve a relatively large degree of mathematical complexity and in which we have relatively little hands-on experience.
To be sure, these problems also present difficulties for anyone seeking to design a forecast model.
But if you accept the premise that Mr. Obama is ahead by some (small) margin in the tipping-point states, something that all the different methods agree on, it then becomes a question of how much doubt you should have about that advantage given the intrinsic uncertainty in polling.
Saying that the race “could go either way” is an obviously correct statement — but also one devoid of insight.
We dare to pose a more difficult question, the one that a gambler or an investor might naturally ask: What are the odds?
We calculate Mr. Obama’s odds as being about two chances out of three.
Not only will the underdog — Mr. Romney — win some of the time, but heshould win some of the time if we have estimated the odds correctly. If the set of candidates you have listed as 67 percent favorites in fact win 95 percent of the time, or 100 percent of the time, you’ve done something wrong. Over the long run, such candidates should win two out of three times — no less and no more.
Of course, it takes a very long time to realize the long run in presidential elections, since there is only one of them every four years. To the extent that one is evaluating the accuracy of political forecast models — whether theycalibrate the odds correctly — it is probably better to look at something like races for the Senate. In that case, there are roughly 35 races held every other year, as opposed to just one every four years. Although these races are not completely independent from one another (there have been years in which Democrats or Republicans overperformed their Senate polls across the board), they are substantially more informative on balance for measuring the effectiveness of a series of forecasts.
Having indulged myself and imposed on your patience with this philosophical detour, let me turn your attention back to the presidential race.
Sunday’s Polls
Four of the five national tracking polls published on Sunday showed a gain for Mr. Romney, but there was one clear exception: a poll for Investor’s Business Daily, which showed Mr. Obama surging to a 5.7-percentage-point lead from 2.6 percentage points previously.
What to make of the Investor’s Business Daily poll? Probably about the same thing that you make of the Gallup poll, which had Mr. Romney up by 7 points among likely voters, and had him gaining an additional point on Sunday.
You should be highly skeptical that these polls represent the true condition of the race: polls that look like outliers normally prove to be so. This is certainly the case with the Gallup poll, which has performed quite poorly in the past when it has diverged from the consensus of other surveys.
But the Investor’s Business Daily poll, like the Gallup poll, also has a history of sharp swings to and fro (sometimes before conveniently settling in with the consensus at the very end of the race).
When I point out that polls like Gallup or Investor’s Business Daily have a history of behaving in this way, my goal is not to browbeat them into changing their methods. Instead, as strange as this might sound, I would much rather that the pollsters paid no attention to FiveThirtyEight. When pollsters feel as though they are under pressure to conform to expectations about the race, they may herd toward the poll averages, which would reduce their independence and would therefore reduce the benefit of aggregating different polls together.
My intention, rather, is to provide a counterweight to what seem like misinformed views about the presidential race. Sometimes the outlying polls receive more attention from the news media, when they probably deserve less.
On Sunday afternoon, I looked up how often each tracking poll had been cited in the Lexis-Nexis news database over the course of the past week. (The criterion I used for this search was to look for instances in which the pollster’s name appeared within 25 words of the term “tracking” and also either of the terms “poll” or “survey.”)
It turns out that the Gallup national tracking poll was cited in the news media more often than the other six national tracking polls combined. Although the national tracking polls show, on average, a race that is about tied, they would have conveyed the impression of a four-point lead for Mr. Romney if weighted based on how often they were cited in news accounts.
The simplest and best defense against this is merely to take the average of the polls.
Even averaging methods can conceal a fair amount of intrinsic uncertainty, however.
The nine national polls included in the RealClearPolitics average as of Sunday evening, for instance, contained about 12,000 interviews between them. (Collectively, they showed Mr. Obama ahead by a trivial margin of 0.2 percentage points.)
The margin of sampling error on a 12,000-person sample is larger than you might think: about plus or minus 2 percentage points in measuring the difference between the two major candidates. So the national polls reflected in the RealClearPolitics average could in fact be consistent with anything from a two-point lead for Mr. Romney to one of about the same margin for Mr. Obama.
And that calculation is based on just one type of error, that which is unavoidable from taking a random subsample of a larger population.
In fact, there are many other types of ambiguities in polling. Whether polling firms include cellphones in their samples seems to make a difference of about 2 percentage points, for instance. (The firms that include cellphones normally show stronger results for Mr. Obama.)
The likely-voter models that polls employ vary substantially from one another, suggesting everything from a situation in which Mr. Romney is performing about one point better among likely voters than among the broader universe of registered voters to one in which he has as much as a six- or seven-point advantage instead.
Some demographic groups are harder to get on the phone than others, and the techniques used to correct for this vary in quality. Polls that are especially “bouncy” may use poorly designed demographic weighting algorithms.
These same issues also apply in polls conducted at the state level, of course.
There are three major defenses that the FiveThirtyEight model employs in recognition of these problems.
First, some of the “noise” in polling is predictable, since some polling firms consistently show results that are more favorable for one or the other candidate. In the cases in which the pattern is most consistent, we correct for this by means of our house effects adjustment.
Second, we seek to use state polling data and national polling data in a holistic way. It might seem, if you are looking at the state polls and the national polls in a very literal-minded way, that Mr. Obama is quite likely to win the Electoral College despite losing the popular vote. We certainly do allow forsome possibility of this, and it has tended to increase over the course of the year. (Our estimate is that the chances would be about 7 percent in an election held today, although there is also a 3 percent chance of the reverse occurring and Mr. Romney winning the Electoral College despite losing the popular vote.)
To regard the probability as being substantially higher than that — taking the RealClearPolitics state and national polling averages might imply a 20 percent probability of a split outcome, for instance — may be to overestimate the degree of precision in both state and national polling averages.
What appears to be a tied race in national polls, as I mentioned, could fairly easily be a two-point lead for Mr. Obama just based on statistical variance alone, in which case there would be no appreciable difference between the national polls and the polls in the tipping-point states. By the same calculation, of course, the national polls could also be consistent with a two-point lead for Mr. Romney.
But our finding is that both state and national polls provide useful information about the overall condition of the race. The state polls can be aggregated, and weighted based on each state’s projected turnout, to provide an estimate of the national popular vote. In the past, this technique would have provided an estimate of the national popular vote that would sometimes have been more accurate than the one from national polls themselves.
Mr. Obama’s swing state polls are consistent with the hypothesis that he holds some sort of Electoral College advantage, but they are also consistent with the hypothesis that the national polls have the race slightly wrong. If you start out with the premise that the national polls are unimpeachably precise, then you might not give much weight to this hypothesis. But they are not all that precise as a matter of either theory or practice.
The opposite hypothesis — that the state polls are overestimating Mr. Obama’s standing — is also a valid one to consider. If you return to the first chart in this article, you’ll find that the FiveThirtyEight “now-cast” provides the most optimistic estimate for Mr. Obama’s standing in the national popular vote (it puts him ahead by 0.8 percentage points). But we are generally on the lower end of the consensus on Mr. Obama’s standing in swing states like Ohio, Iowa and Nevada. This is because, when the model detects an overall discrepancy between state and national polls, it adjusts both of them in opposite directions to bring them closer to one another. If after that adjustment the model still finds evidence that one candidate is performing especially well in the most important states, it will then attribute the remainder of the difference to a potential popular vote and Electoral College split.
Following a similar theme, we also use both national and state polls to inform our view about the overall trend in the race. The state polls published over the past several days, including on Sunday, have shown some stronger and weaker numbers for the candidates, but they are generally consistent with a race that has not changed much since Mr. Romney made his gains after the Denver debate.
Be wary of analyses that suggest that one candidate is surging or declining by an especially large amount in one or two particular states in a way that diverges from the overall trend in the race. Many of these apparent changesmerely reflect statistical noise. We are at the point, certainly, where if Ohio moves a little less than the national polls, or Florida moves a little more, that is potentially significant. But most people’s first instinct is to attribute too much meaning to these changes.
That brings up the third line of defense in the FiveThirtyEight model: our estimate of the uncertainty in the calculation. These estimates are based on how well polling averages, and presidential forecasting models, have performed under real-world conditions, rather than under idealized assumptions that, for instance, the uncertainty in polls is limited to statistical sampling error alone, or that the vote in each state is independent of the next one.
Right now we’re at a point where, for better or for worse, these methodological choices will make quite a lot of difference in your evaluation of the race. The best I can do is to explain why we made the choices that we did in designing the model, and how they are playing out given the information that is coming our way.
There remains an outside chance that the race will break clearly toward one or the other candidate, after the third debate on Monday or because of some intervening news event, but the odds are strong that we will wake up on Nov. 6 with a reasonable degree of doubt about the winner. For that matter, we may wake up on Nov. 7 still uncertain about who won.
Nonetheless, stipulating that the race is clearly very close is not an adequate substitute for placing any kinds of odds on it at all. And the central premise behind why we see Mr. Obama as the modest favorite is very simple: he seems to hold a slight advantage right now in enough states to carry 270 electoral votes.
472 Comments
I have new for you, Nate: once you stop assuming a D+10 voter turnout, the world starts looking a lot less favorable to you silly liberals. When Romney wins by 8 points on election day, maybe you can fold up your silly blog and get a real job.
Oh, and Iowa early voting 2-1 Dems, so are you one of those folks that thinks mass number of Dems are crossing over? Lol
Women will decide this election, especially Conservative Wives and Daughters.
Poker players know this stuff intuitively. 2:1 is nothing. If you bet all your money twice as a 2:1 favorite, you'll probably go broke.
At 70-30, running the election twice, you get the following percentages:
49% of the time Obama would win twice.
42% of the time they would split, one each.
9% of the time Romney would win both.
So maybe this was meant to read "the most likely outcome would be that Romney would win *at least* one", though this is only barely true.
Your second paragraph refers to 2:1 (a little worse than 70-30), so I'll examine its claim on that basis. There is also a subtle difference that changes the stakes, in that the second scenario implies that chronological order comes into play, meaning that if the first bet is lost there is no second bet because the player is broke; it also implies that if the first bet is won, the player wagers twice as much (everything they have after winning) on the second bet.
Thus the "you'll probably go broke" claim is true, given the 2-1 parameters and the implied assumptions I mentioned. Nearly 56% of the time you will go broke. (However, the rest of the time you will quadruple your money, meaning that your average "expected value" is positive, averaging a 78% profit, meaning you should make these bets if you can afford to!) This is known within poker circles (and in some other quarters as well, I'm sure) as a "high variance" proposition.
(By the way, if the second paragraph used 70-30 instead of 2-1, it would still barely be true: 51% of the time the bettor would go broke.)
Final Rasmussen poll (10/25): Angle +4
Actual result: Reid +5.6
Buck-Bennet 2010 Senate Race in Colorado
Final Rasmussen (10/30): Buck +4
Actual result: Bennet +0.9
Boxer-Fiorina 2010 California Senate
Final Rasmussen (10/27): Boxer +3
Actual result: Boxer +9.8
I could go on and on.
What mental gymnastics are required to consider these results as within 1%?
What other article, johndouglas? Written by whom? Published where?
What else could explain this annoying disconnect between the national and state level polling? Here is a question... Suppose for a moment that some Republicans are correct and the national numbers for Romney are shifting toward him like the polls suggest. If that were the case and we were to believe Gallup and Rasmussen that Romney were truly ahead by +3, then why isn't the map for Obama worse than it was during the Gore-Bosh election? I mean if you recall, during the Gore vs Bush election, the only contested states were PA, OH, and FL. Now, PA and NM are solid blue and NV, VA, and CO are contested states which were formerly solid red.
http://www.oddschecker.com/specials/politics-and-election/us-presidentia...
66.7 to 67.1 % and Nate is at 69% today.
What I found
"As directed by the 12th Amendment, the 435 -- many of them brand new -- members of the House of Representatives would find as their first official duty the selection of the next President of the United States. "Welcome to Congress!"
Unlike the Electoral College system, where larger population equals more votes, each state in the House gets exactly one (1) vote when selecting the president. Even California, with its 53 Representatives, get one vote. The first candidate to win the votes of any 26 states is the new president. The 12th Amendment gives the House until the fourth day of March to select a president. Should the House fail to meet the deadline, well, more on that later.
It is up to the group, or "delegation" of representatives from each state to decide among themselves how their state will cast its one and only vote. Suddenly, smaller states like Wyoming, Montana and Vermont, with only one representative wield as much power as California or New York.
Senate Selects the New Vice President
As the House is selecting the new president, the Senate is busy selecting the new vice-president. In the Senate, each of the 100 Senators gets one vote, with a simple majority -- 51 -- Senators required to select the vice-president. Unlike it does on the House, the 12th Amendment places no time limit on the Senate's selection of a vice president."
Your arguments seem to more a defense of your model as the numbers tighten and a cheerleading session for other like minded liberals who want to believe so badly that their man cannot possible lose.
I don't have a 538 model or any kind of statistical analysis. Just my observations of keen political interest. Romney is going to win and my bet is at least 301 EV.
If I am wrong, then I am wrong. You did very well last election, but really, did you predict what everyone else did not? Not really. I was pulling for McCain, but I knew he would lose and lose big. He lost big.
On November 7th, I will be expecting a Why Romney won and How my 538 Model is flawed.
As they say in baseball, your data set is not very large yet. Lets talk in 2020.
And if Mr.Silver is right and you are wrong on November 7th, may we expect you to be back here explaining how statistics is superior to "observations of keen political interest" and how you were wrong?
Undecideds break for the challenger.
Nate cannot wish away reality anymore than he can turn the trends in the polls. The trend is moving to Romney.
If the undecided break anyway it is ineffectively toward the eventual loser. More significant is who are the undecided. In 1972 they broke for the challenger because Nixon was poaching the usual Democratic voters, and the undecided were liberal-leaning voters -- and that about every Presidential nominee has a ceiling of about 62% of the popular vote. In 1976, Ford gained late as the incumbent because he started to campaign more effectively -- too late. (The 1976 election didn't have much ideological difference between nominees). In 1980 and 1984 Ronald Reagan was picking off some usual Democratic voters as he approached the ceiling. In 1988 the challenger collapsed as a campaigner.
This is an ideological election with the undecided near the political center. Random chance implies a 50-50 split among the undecided of this moment who will cast a vote. Many won't . If President Obama is up 49-47 the law of large numbers favors him.
For all the difference your deluge of numbers and fancy graphs makes, you would be better off spending your time and energy on open cheerleading. Gimme an O...Gimme a B.....
That may overstate the case, as they might not be so random. The states are scattered enough that it won't be easy to campaign in them all and fit a nationwide campaign pitch that wins them all without picking off a bunch of reasonably-secure Obama states and winning.
At least pretend to be halfway fair. Gallup has had Romney leading by 5% or 6% the last three days. Florida, NC and VA are pretty much given now in Romney's camp. Ohio is tied (within the margin of error). Colorado, NV, WI, NH and even Ohio are now toss-ups.
It shouldn't surprise me but it still amazes me that liberals like this refuse to report facts and still think they can sway public opinion. Not anymore...
Is Nevada truly a tossup? No poll has shown Romney with a lead there since April (according to RCP)http://www.realclearpolitics.com/epolls/2012/president/nv/nevada_romney_....
Even if Romney is granted Florida and NC, VA is a "tossup" given the RCP average.
Mr. Silver has used an public, consistent methodology. He's not altering it as things get closer. He's not refusing to "face facts." Where exactly does Mr. Silver's model err?
The entire election is basically Ohio and Ohio is within the margin of error with Romney leading big among Independents. Only a former Daily Kos blogger would somehow twist that into a 70-30 chance of re-election for the president.
You exist to reassure Democrats that The Chosen One is somehow winning. You are a run-of-the-mill left wing partisan and you deserve to have your money taken from you.