The 2020 election presented another distinct challenge — it took place amid the pandemic. Pollsters found that some Americans, stuck at home and lonel
The 2020 election presented another distinct challenge — it took place amid the pandemic. Pollsters found that some Americans, stuck at home and lonely, were more likely to respond to surveys. While that was initially seen as a boon, it might have led to even more bias if it meant the uneven adherence to stay-at-home orders added another source of bias to who picked up the phone.
Weighting on recalled vote is not without its concerns.
Voters have been shown to have poor recall of whom they voted for or even whether they voted at all, typically being more likely to recall voting for the winner. One study of Canadian voters found up to a quarter of voters were inconsistent when recalling whom they had voted for.
This misrepresentation of past vote can push polls in different directions depending on who won the most recent election. In 2022, that meant respondents were more likely to say they had supported Joe Biden, and pollsters using recalled vote would end up giving them less weight, meaning Republican support was bolstered.
But with a prior winner from a different party, the effect would be reversed. An assessment by The Times found that weighting its 2020 polls using recalled 2016 vote would have made them even more biased toward Mr. Biden. And a report from the American Association for Public Opinion Research examining how 2020 polls could have been improved found that polls that weighted on recalled vote were no better than those that didn’t.
Similarly, in 2022, weighting by recalled vote would have made Times/Siena polls less accurate. As published, without weighting on recalled vote, the final polls of Senate, governor and House races had an average error of less than two percentage points and zero bias toward Democrats or Republicans. When weighted using recalled vote to 2020 election results, average error would have increased by a percentage point, and overall the polls would have been slightly biased toward Republicans.
But that might have been a consequence of other decisions The Times makes, which includes weighting to demographic information available on the voter file that is not always available to other pollsters.
Other pollsters have found the method to yield significant improvements over typical weighting schemes. SSRS used a range of weighting methods in 2022, including recalled vote for some of its polls, and also experimented with weighting on political identification. Its post-election analysis found that using recalled vote as a weight would have been the most accurate overall approach, increasing average accuracy by more than three percentage points over just weighting on standard demographics.
www.nytimes.com