How Reliable Is the 'Nostradamus' of US Polling?

Allan Lichtman’s "Keys to the White House" model has been widely recognized for its accuracy in predicting U.S. presidential elections. Introduced in 1981, the model uses a series of 13 true/false questions, referred to as "keys," to assess the incumbent party's position. If six or more of these keys are false, the model forecasts a loss for the incumbent party. Despite its impressive track record of correctly predicting nine of the last ten elections, the model has faced criticism, with some detractors labeling it as "superficial" and "prone to bias."

The model has recently faced intensified scrutiny in light of President Joe Biden's decision not to run for re-election, in favor of Vice President Kamala Harris. Lichtman criticized the pressure on Biden to withdraw from the race as a "foolish, self-destructive escapade," asserting that the president still had sufficient "keys" to secure victory. In an interview with Newsweek, Lichtman defended his model, stating, "Any successful forecasting model, especially one that challenges the approach taken by pundits and pollsters, will attract critics."

Critics of the model, however, have raised several concerns. Lars Emerson and Michael Lovito, both reporters and alumni of American University, where Lichtman teaches, published a detailed critique of the model in The Postrider. They accused Lichtman of dishonesty, arguing that he misrepresented his model’s predictions regarding the 2016 election. Although Lichtman’s model was originally designed to forecast the winner of the popular vote—correctly predicting the outcome until 2016, when Donald Trump won the Electoral College despite Hillary Clinton winning the popular vote—Emerson and Lovito claimed that Lichtman had obscured the model’s limitations by revising its purpose to predict election winners more broadly.

Emerson and Lovito also criticized Lichtman’s model for its perceived lack of empirical rigor. They argued that the model is "prone to bias and subjectivity," citing James E. Campbell, Distinguished Professor Emeritus of Political Science at the University of Buffalo, who described many of the keys as highly subjective. For example, assessing whether there has been significant social unrest or if a candidate is charismatic often depends on personal interpretation rather than objective criteria.

In response to these critiques, Lichtman defended the model’s robustness and historical consistency. He acknowledged that critics frequently demand changes to the Keys in response to unique election circumstances, such as the historic candidacies of an African American or a woman. Lichtman maintains that the model’s strength lies in its consistency and its ability to endure through various changes in the political landscape. "You cannot change a successful model on the fly," he argued. "The Keys are a robust system that has withstood much greater changes in America’s economy, politics, demography, and society than any recent campaign."

The debate over Lichtman’s model also includes a notable exchange with Nate Silver, a prominent election forecaster. In a 2011 New York Times article, Silver dismissed Lichtman’s model as "superficial," suggesting that Lichtman was less a discoverer of accurate keys and more a locksmith capable of continually creating new keys. The tension between Lichtman and Silver resurfaced in July when Silver criticized Lichtman’s model on X (formerly Twitter), implying that it failed to account for critical factors such as President Biden’s performance issues. Lichtman responded by dismissing Silver as lacking a fundamental grasp of historical and electoral dynamics.

Amidst these debates, Emerson and Lovito questioned the uniqueness of Lichtman’s model. They suggested that traditional polling methods could match the model’s accuracy. "If you took whoever led in the polls from every election from 1984 to 2020 and predicted they would win, you would have predicted 9/10 elections correctly, which is the exact same record Lichtman can claim," they argued. This comparison, they suggested, undermines the perceived superiority of the Keys model and raises doubts about Lichtman’s dismissal of polling data.

Despite the criticisms, Lichtman remains steadfast in his defense of the Keys model. He views it as a guiding standard in election forecasting, asserting, "The Keys stand alone among all forecasting models with a 40-year track record of successful presidential predictions." He likens the model to the "North Star" of prediction models, arguing that it remains effective despite the unique challenges of individual campaigns. Lichtman is open to the possibility of future changes but emphasizes that any such adjustments would need to be based on new patterns rather than transient events.

Julia Azari, a political scientist at Marquette University, challenges the notion that Lichtman’s Keys should be considered a model at all. She argues that the Keys are more useful as a starting point for exploring questions about elections rather than a comprehensive forecasting tool. Azari contends that a robust model should be parsimonious, specific, and grounded in a theoretical framework of electoral dynamics. She suggests that the media would benefit more from focusing on Lichtman’s broader insights into electoral history rather than fixating on the Keys model itself.

In summary, while Lichtman’s "Keys to the White House" model has garnered acclaim for its predictive success, it continues to face significant scrutiny and debate. Critics challenge its empirical validity and subjective nature, while proponents, including Lichtman himself, argue for its enduring value and consistency in the realm of election forecasting.