Oct. 23: The Virtues and Vices of Election Prediction Markets

 

By NATE SILVER

October 24, 2012, 5:30 pm138 Comments

The Electoral Map: Building a Path to Victory    An interactive you canplay with to see how each candidate could reach the magic number of 270

Tuesday featured what, on the surface, would seem to be a fairly neutral day of polling. Of the 11 national polls published on Tuesday, five showed President Obama ahead, four gave a lead to Mitt Romney and two had a tied race. On average among the polls, Mr. Romney led by 0.3 percentage points, the same as his average margin in the previous edition of the same polls.

The volume of battleground state polling was light on Tuesday, and what polls there were require a few cautions. For instance, an Old Dominion University poll of Virginia put Mr. Obama seven points ahead there – but the poll had an extremely long field period, having been started in September, with about two-thirds of its interviews coming before the first presidential debate in Denver. It thus receives little weight in the forecast.
The best-looking result of the day for Mr. Romney was probably an American Research Group poll of New Hampshire, which had him up two points there – although its prior poll of the state had Mr. Romney up four points, so this isn’t necessarily a sign of a change in his favor.
Similarly, a SurveyUSA poll of Ohio had Mr. Obama three points ahead there – a decent result for him, but the same as in its previous poll of the state.

Mr. Obama’s chances of winning the Electoral College were down slightly in the FiveThirtyEight forecast, to 68.1 percent from 70.3 percent on Monday. Some of the decline was because of the sharp decrease in the stock market on Tuesday, which still has some influence on the FiveThirtyEight economic index. Some of the rest is because among the various national polls, the one that the model rates most highly is the national tracking poll run by ABC News and The Washington Post, and it showed a slight decline for Mr. Obama.
At the political betting market Intrade, however, traders seemed to have a much stronger opinion about the day’s news. Mr. Romney’s stock, which can be read as a forecast of the chance that he will win the Electoral College, rose to 45.4 percent from 39.7 over the course of the day.
The probabilistic forecasts issued by FiveThirtyEight have been quite close to Intrade and those at other trading and betting markets over the course of the election. The spread as of Tuesday night – with Intrade implying that Mr. Obama has a 55 percent chance of winning the election, and FiveThirtyEight a 68 percent chance – is much wider than usual.
However, Intrade is also something of an outlier relative to other prediction markets. Another prediction market, Betfair, gave Mr. Obama about a 63 percent chance of winning as of early Wednesday morning. The consensus of sports books, similarly, implied that he has about a 65 percent chance. These values are not significantly different from the FiveThirtyEight forecast.
The general question of whether FiveThirtyEight ought to be better than prediction and betting markets is an interesting one. I am far from an efficient-market hypothesis purist, but markets are tough to beat in most circumstances. Furthermore, the FiveThirtyEight forecasts are public information, and bettors can use our forecasts along with those of our competitors to calibrate their estimates of the outcomes.
A 2009 paper by David Rothschild found that FiveThirtyEight’s forecasts were somewhat better than Intrade’s over the course of the 2008 presidential election cycle. Mr. Rothschild wrote, however, that Intrade’s forecasts would have outperformed FiveThirtyEight’s had they been corrected to adjust for the market’s tendency to overrate the chances of the trailing candidate (what is known as a favorite-long shot bias). Of course, if that bias still applies in Intrade’s pricing, it would imply that the market is overrating Mr. Romney’s chances now as he is the underdog in the race.
The more pertinent question for the time being may be why Intrade differs from the other betting markets, and from the sports books.
It is unusual for differences like these to persist for very long in well-functioning markets, because they would potentially enable a trader to make a risk-free profit by selling Mr. Romney’s stock short at Intrade, where it is expensive, and then betting on him at one of the sports books or at another prediction market, where it is relatively cheap. The fact that these relatively wide spreads exist between the different markets suggest that they are relatively illiquid and that they entail high transaction costs, which may limit their efficiency.
In fact, these differences between Intrade and the other prediction markets have persisted throughout much of the past two months. Intrade has consistently shown slightly more favorable odds for Mr. Romney than other prediction and gambling sites.
According to Brandon Adams, a professional poker player and a teaching fellow at Harvard, Intrade’s prices are not necessarily considered more reliable by professional gamblers. Mr. Adams says that Betfair and the sports book Pinnacle, both of which put Mr. Obama’s odds at about 63 percent as of early Wednesday morning, feature more sophisticated market participants.
However, Intrade is cited far more frequently by the American news media. That at least opens up the possibility that someone could place a wager on Mr. Romney (or Mr. Obama) at Intrade in order to influence the news media’s perceptions about which candidate has the momentum.
At several points on Tuesday and early Wednesday, what appeared to be a single trader bought a large number of Mr. Romney’s shares at Intrade, at one point boosting Mr. Romney’s chances to about 49 percent from 41 percent over the span of a few minutes. The betting patterns echo similar ones in the pricing of John McCain and Mr. Obama’s stock at Intrade late in the 2008 cycle.
It is not necessarily safe to conclude that represents an effort to manipulate the market, or public perceptions about Mr. Romney’s chances. Instead, it could just be someone who thinks Mr. Romney is a good bet relative to the price being offered.
A sophisticated bettor, however, would probably have sought out somewhere else to place his bets on Mr. Romney, since even before the recent market action, they offered a more favorable price than Intrade did.
Some people are also inclined to conclude that the “rogue trader” may have inside information about the state of the race. Personally, I think this is unlikely. The overall amount of the bet early on Tuesday equaled about $18,000. That is a large bet in everyday terms, but not by the standards of a professional gambler or investor who felt he had a sure thing.
A truly sophisticated trader, for instance, could make bets in the actual stock market, buying up certain stocks in industries that might be expected to perform well under a Romney presidency (for instance, those of oil or coal companies) while shorting those that would probably benefit from Mr. Obama’s remaining in office (for example, certain health care stocks or green-technology companies). Although it would require some skill to build such portfolios, the potential payoff would be much greater – measured in the millions rather than the thousands of dollars.
My conclusion, then, is that the recent bets in Intrade are unlikely to represent the behavior of a sophisticated trader. Beyond that, it is hard to infer much about the motivation behind them.
Markets of any kind are potentially subject to herding behavior. Even if the “rogue trader” does not have superior information about the campaign, other traders might alter their behavior assuming that he did.
Or traders could be betting on speculation – buying up Mr. Romney’s stock, for example, in the hope of selling it at a higher price later, even if that price diverged from their estimate of the “fundamentals” in the race.
Coming up with a probabilistic assessment of the chances of each candidate’s winning the Electoral College is intrinsically a very challenging task. ”Gut-feel” probability estimates are not likely to be much good. It is the sort of problem, in my view, that benefits from a more rigorous approach, i.e. building a model. Yet, models are constrained by the lack of historical data and the infrequency of presidential elections – only since about 1980 has state-by-state polling of the election been common.
In these circumstances, traders may be looking for an anchor to calibrate their assessment of the probabilities. It is certainly possible that FiveThirtyEight itself serves something of this function.
The FiveThirtyEight model is sometimes perceived as being incredibly bold for having Mr. Obama as a two-to-one favorite despite what is certainly very close polling, but the few other models that seek to frame the election in probabilistic terms tend to give Mr. Obama an even clearer advantage, putting his odds at between 80 and 95 percent.
I would be happy to engage those other forecasters in a (good-natured) argument about why I think their models are considerably too confident about Mr. Obama’s chances. But if the FiveThirtyEight model is thought of as “Obama-leaning” relative to the conventional wisdom, it is “Romney-leaning” relative to the other models that use a similar approach based on state polling data.
For betting markets like Intrade, it would be useful to have a dozen FiveThirtyEight-like models in the public sphere. The consensus of models or forecasts is very often better than even the best-performing members of the group. (This is the sense in which I remarked on Twitter on Tuesday that I thought FiveThirtyEight was “overrated.”)
But I also think that efforts to look at the polls in a systematic way are likely to be superior to much of what passes for polling analysis, which looks at them on an ad-hoc basis. Trying to decode the meaning of an individual poll is not a terribly useful exercise when 20 or 30 of them are released every day.
Instead, debates about overall, macro-level strategies for processing the polls and other information about the election – for instance, how one might reconcile state polls against the national ones – are liable to be much more informative.
They could be especially informative to betting markets, serving as a better anchor against which hard-to-quantify elements about the election could be weighed. Betting markets certainly have some advantages over model-based approaches, like being able to react to news events before their effects show up in the polls.
But these advantages may be outweighed if most participants in the market are prone toward herding, speculation and similar behaviors – and if the markets are not liquid enough, and do not offer high enough stakes, to be attractive to highly sophisticated traders.
So is FiveThirtyEight better than Intrade? I don’t know about that as a general proposition.
But I’d be happy to bet on FiveThirtyEight relative to the price that Intrade offered on Mr. Obama on Tuesday, which was not characteristic of rational market behavior.