Four years ago Merlin Heidemanns and I worked with Elliott Morris at the Economist magazine to produce a state-by-state election forecast, combining national polls, state polls, economic and political “fundamentals,” and a hierarchical Bayesian model allowing for correlation among states, variation over time, and sampling and nonsamplng error of surveys. The model, which built off that of Lock and Gelman (2010), was described in this journal by Heidemanns et al. (2020), with further discussion of communication in Gelman et al. (2020). We fit the model in Stan (Carpenter et al., 2017), and our forecast updated daily as polls came in during the summer and fall and, with some hiccups, it performed reasonably well, albeit with some concerns regarding the quantification of uncertainty (Gelman, 2020), issues that arose with poll-based forecasts more generally (Gelman, 2021).
This year, Ben Goodrich, Geonhee Han, and I accepted the invitation of Dan Rosenheck of the Economist to help with their 2024 forecast. We started with the code from 2020, which we altered in several ways, including: (i) improving the fundamentals-based model to better account for the declining importance of the economy as a predictive factor in an increasingly polarized electorate; (ii) more carefully estimating the state-level correlations of polling errors and time trends in opinions; (iii) accounting for more nonsampling error in polling. As before, we checked our model by fitting it to data from the 2012, 2016, and 2020 campaigns along with existing polls from 2024, to check that it produced inferences that seemed reasonable given our current political understanding.
It might seem silly to check a model by comparing its inferences to reasonable expectations—if we knew what to expect, what is the purpose of the model at all?—but there are two reasons why this procedure seems reasonable to us. First, we are forecasting a multivariate outcome—50 state elections—and it require a lot of care to construct a full forecast with all its correlations. Second, we are constructing a sort of robot–a forecast that should be able to update itself over time as new polls and economic and political information arrive—so our checking is not just on the current forecast probabilities but also on how they develop over time. For example, if a new poll comes in from Ohio showing a stronger-than-expected support for the Democratic candidate, how much should this shift the forecast in Ohio and in other states, and how does that map to the probability of each candidate winning?
Our model currently gives the Republican candidate an expected 51% share of the national two-party vote and a 3/4 probability of winning the Electoral College (Economist, 2024). With the current state of public opinion and the expected relative distribution of votes among the states, it makes sense that the Republicans have the Electoral College edge, and a probability of 75% expresses an appropriate uncertainty given the closeness of the polls and the possibility of large polling errors and national swings between now and November.
Here are a few possible failures that we anticipated with our forecast going forward:
– What if one candidate or another takes a solid lead in the national polls? It would not take much for that party to get assigned a probability of 90% or more of winning–but then what if there is a big swing in the other direction, leading to that candidate’s win probability going below 10%? A probabilistic forecast should be a martingale—that is, if the forecast at time t of a certain future event has a probability of X(t), then E(X(t+s)), given all information available at time t, should be equal to X(t). So a swing in predicted probability from 90% to 10%, while possible, should be very unlikely, and a forecasting procedure that regularly shows such swings has problems (Taleb, 2018). We do not expect this to happen, but it could! Polling has been very steady during the past several election campaigns, but large swings were common in decades past (Gelman and King, 1993). The relevant parameter in our model is the standard deviation of the random walk of national vote preference over time. When implementing our model for the Economist, we set this scale to a value that seemed high enough to allow for plausible changes during the half year leading up to the election while still allowing informative inferences during those early months. But large enough variation over time could break this model and yield overconfident predictions.
– What about third parties? Following our practice in previous elections, we model preferences for the Democrat and the Republican, ignoring other candidates, which has seemed reasonable given that no third-party nominee has won any states since 1968. For awhile, though, RFK Jr. appeared to be a strong alternative to Biden and Trump, which could affect our forecast directly if Kennedy were to win any states and indirectly to the extent that changes in his support were to go unevenly to the major-party candidates. Presumably other minor parties won’t matter much, at least not compared to 2016, when the Libertarian and Green candidates did not win many votes despite widespread discontent with the options of Clinton and Trump.
– Actuarial concerns. Biden and Trump are both around 80 years old, with a nontrivial risk of death or disability between now and election day. What happens if one or the other candidate needs to be replaced? Even before the recent presidential debate, this was a vigorously-discussed topic, with pundits arguing that both parties were hobbled by weak candidates; see Gelman (2024). We did not have anything on this on our model, implicitly assuming that any replacement candidate would do about as well as the existing nominees. Ever since Rosenstone (1983), there has been a consensus in political science that candidates don’t matter so much for presidential voting, except that there is a slight advantage to political moderation. Given that most prominent alternatives within their parties are no more politically moderate than Biden or Trump, it seemed safe to not worry about specific candidate effects. That said, if, as it now seems likely, Biden is replaced on the Democratic ticket, we could well see some fluctuation in the polls beyond what might be expected from our default time-series model.
– Concerns specific to 2024. This is the first presidential election where either major-party candidate has been convicted of a felony, and the first since 1984 where there are serious concerns about either candidate’s mental deterioration. Pundits have also noted the unusual disconnect between relatively strong economic performance and the president’s low approval ratings. Another noteworthy feature, with effects already apparent in the 2022 midterm elections, has been a series of controversial Supreme Court decisions on issues ranging from abortion to presidential immunity. On the other hand, other recent campaigns have had historically unique features: the 2020 election was complicated by covid, early voting, two already elderly candidates, and justified concerns that one of these candidates would not accept the election outcome; and the three elections before that had the first African-American, Mormon, and female nominees, all of which might seem commonplace today, but at the time many people polled expressed resistance to voting for candidates with these attributes. This is not to say that it is a bad idea to adjust for what we can, just that we would hope our existing error terms to capture some of the unexpected. The Supreme Court issue is related to concerns about partisan balance, another tricky feature this year, with both houses of congress up for grabs.
– Polling errors were major concerns in 2016 and 2020. What about 2024? It’s hard to say. Our model allows for systematic errors at the national and state level. A study of state-level polling errors since 2000 found a positive correlation among successive elections—that is, if state polls are biased toward the Republicans or Democrats one year, they are likely to have a similar bias in the next election. Our model does not include this autocorrelation (because we assume that pollsters are trying to correct for such biases), so we may be leaving some information on the table. We hope that a reasonable range of possible polling bias is included in our predictive uncertainties.
Traditionally, the general election campaign is said to begin on Labor Day, after the two parties’ nominating conventions. This year, neither party’s candidates faced serious primary challenges, the two candidates appeared to be set in the spring, and observers were anticipating a long slog through November. Recently we have seen two shocks—Trump’s felony conviction and subsequent erratic performance in campaign events, and concerns about Biden’s age culminating in his calamitous debate performance—and here we are at the beginning of the summer with a new and potentially volatile race. In the modern era of extreme political polarization, we expect our state and national forecasts to still be reasonable, but ultimately they are conditional on model assumptions, hence the importance of transparency in methods and data.
References
Bob Carpenter, Andrew Gelman, Matt Hoffman, Daniel Lee, Ben Goodrich, Michael Betancourt, Marcus Brubaker, Jiqiang Guo, Peter Li, and Allen Riddell (2017), Stan: A probabilistic programming language. Journal of Statistical Software 76 (1).
The Economist (2024). US Election 2024. Accessed 3 Jul. https://www.economist.com/interactive/us-2024-election/prediction-model/president
Andrew Gelman (2020). Concerns with our Economist election forecast. Statistical Modeling, Causal Inference, and Social Science, 28 Oct. https://statmodeling.stat.columbia.edu/2020/10/28/concerns-with-our-economist-election-forecast/
Andrew Gelman (2021). Failure and success in political polling and election forecasting. Statistics and Public Policy 8, 67-72.
Andrew Gelman (2024). How would the election turn out if Biden or Trump were replaced by a different candidate? Statistical Modeling, Causal Inference, and Social Science, 12 Jun. https://statmodeling.stat.columbia.edu/2024/06/12/how-would-the-election-turn-out-if-biden-or-trump-were-not-running/
Andrew Gelman, Jessica Hullman, Christopher Wlezien, and G. Elliott Morris (2020). Information, incentives, and goals in election forecasts. Judgment and Decision Making 15, 863-880.
Andrew Gelman and Gary King (1993). Why are American Presidential election campaign polls so variable when votes are so predictable? British Journal of Political Science 23, 409-451.
Merlin Heidemanns, Andrew Gelman, and G. Elliott Morris (2020). An updated dynamic Bayesian forecasting model for the 2020 election. Harvard Data Science Review 2 (4).
Kari Lock and Andrew Gelman (2010). Bayesian combination of state polls and election forecasts. Political Analysis 18, 337-348.
Steven J. Rosenstone (1983). Forecasting Presidential Elections. Yale University Press.
Nassim N. Taleb (2018). Election predictions as martingales: An arbitrage approach. Quantitative Finance 18, 1-5.