Post Mortem Part 1: Why did the Polls forecast so Wrong?

One week after the General Election 2015 the Opposition of Cameron lays crushed on the ground, while the Tories have to reflect about the long-term prospect of their victory at the polls. The obvious question about the election outcome is about, why got the polls it so wrong?

Well, the history of British Politics is a never ending story of complete surprises coming out of the ballot boxes. The Blair era was, compared to previous election cycles, a time of rather astonishing poll accuracy. There was a overstated Labour vote share here and there [General Election 2005 & 2001], but nobody doubted a Labour win before and so the impact on public opinion was pretty small.

The first complete surprise outcome of a General Election was beyond to the unexpected defeat of Churchill in 1945, the General Election in 1970. The ambitious young Member of the shadow Cabinet of Edward Heath, Margaret Thatcher fought the Election with few illusions about ending up in government. She did so like the rest of the political establishment. Most pundits even expected Harald Wilson to increase his majority of 96 in Westminster. Then on election night with constituency after constituency declared, the swing away from Labour to the Tories stacked the establishment and with so the entire United Kingdom.

Swing after swing coming out from the reported numbers of each Returning Officer suggest not only a substantial drop of the majority of Wilson, but lead finally to a complete unexpected victory of Edward Heath. In 1974 with the tensions of the unions and after the famous U-Turn on the domestic economic agenda, Heath requested the Queen to resolve Parliament, expecting a strong public support and an increase in his majority. The public expected the same result as well until Election Day thanks to the polls. But then with each Returning Officer reporting his or her numbers, the confidence of the Heath camp went further and further down. Her Majesty even visited foreign shore, expecting along with the Central London political class, a save return of Heath to Downing Street, indeed.

But then with a Hung Parliament more and more emerging, her Majesty headed back to Buckingham Palace and Wilson ended up with a four seats edge. The Tories went into opposition preparing for a soon coming new General Election. Eight months later at the ballot boxes, Wilson went into election night anticipating from public polls a huge or at least significant majority. But then optimism vanished again on the actual election night and he ended up with a tiny majority, which he soon lost due to by-elections. The then emerging Lib-Lab Pact sacked the Liberals from almost 20 percent to normal size and though Thatcher’s lead in the polls right before election night in 1979 almost vanished, she ended up with a more than sufficient majority and an far better result than final pre-election polls suggested.

From all of these cases emerges one central pattern: Every time the main party in opposition did far better than the polls suggested in terms of swing in their favour. These pattern even repeated itself during the Thatcher era. Giving her between 45 to 47 percent in the opinion polls but only 42,4% on election night in the true landslide General Election in 1983.

All of this occurred in stark contrast to the continent, where in many European nations the government benefited from a last minute swing in their favor. Meaning they moved their numbers in the last two weeks, but then even outperformed their poll numbers in the election result itself. This pattern occurred especially in consociational democracies or consensus driven political systems such as Germany, Austria, across Scandinavia or the Netherlands. For instance in Germany occurred often a stronger share of the vote for the governing coalition than final polls predicted right before the election, whereas in the British Westminster model more often than not the opposition emerged stronger after the election than final polls across the bank suggested (with the exception of 1992, 1997 and 2015).

This effect can be attributed to in terms of politics completely disinterested undecided voters making up their mind right before the election and often supporting the party of the standing government in consensus orientated proportional representation systems. In contrast to the Anglo Saxon Majority Model, where undecided voters moved to the main opposition Party in Britain and the United States, where Dick Morris did his famous study about the vast majority of undecided voters moving in every Presidential Race with a sitting President on the ballot against the incumbent, with the exception of 2012 thanks to Hurricane Sandy.

The picture turned in 1987, with the polls matching the Conservative showing and overestimating the Labour vote significantly. Then came the famous 1992 Election, where driven by the polls, the public expected a Labour win but the Exit Poll indicated a Hung Parliament with John Major Tories as largest Party. Eventually the Tories ended up, after all acting Returning Officers reported, with a small majority in Westminster. But this time a very opposite pattern emerged like in the 1987 election in less significant size: The polls underestimated to a significant extend the numbers of governing Tóry party and pretty heaviy overestimated the share in terms of the popular vote and seats in Westminster of the opposing Labour Party across the United Kingdom. Then came the Blair era, which was mentioned above and the 2010 General Election, where the polls placed Labour and Conservatives very close to the final outcome, but overestimated the vote of Liberal Democrats on a four to six point level.

By going back to the General Election from last week, David Cowling states on the online BBC article: “Why did pollsters get it so wrong?” that Labour underperformed on every General Election since 1992. Well, one can add: Actually Labour dramatically underperformed in the 1887 General Election, too. Going into Election day under the moderate appealing Neil Kinnock with figures between 34 and 35 of the vote and awaking the next morning with a 30,8% percent share of the popular vote. Additionally Labour also underperformed in the 1979 and October 1974 Election in this regard. But most stunning was surely the General Election in 1970, where many pundits expected Harold Wilson to increase his large majority of 96, only to wake up with a substantial Tory majority the next day.

This pattern is pretty similar to experiences from the continent. In my own country Germany, Social Democrats in the best cast matched their poll numbers in General Elections or European Elections. These Election are counted as huge success in their party history. In most cases Social Democrats constantly slightly underperformed in General Elections, but eventually massively underperformed in the Euro Poll Elections, with a far lower turnout.

Poll companies tried to predict the numbers of Social Democrats from a model designed for the General Election with a turnout between 70 and 80 percent, only to get it in on the Euro Poll in all cases since 1979, except of 2013, extremely wrong, with a turnout between 40 and 50 percent.

Back to Britain where the popular vote is a Pretty Amazing Contest and only MP figures count, one could state the following poll perception thesis: Nobody cared about the constant Labour underperforming in the popular vote that occured, with the exception of 2010. Between two to four points in 2005 and on average seven points in 2001. And even in the year of the Labour Asteroid hitting Tory Britain in 1997, the polls overestimated the Labour share by some points. But only the General Election of 1992 and 2015 count as complete disasters on Pre Election Polling. Both General Elections were the closest fought ones since 1974. So the regular occurring pattern made here the difference between a Labour or Tory or Hung Parliament and tuned most marginals in the other direction than expected by the public anticipation.

Where lays the difference between an Exit Poll and a final Pre-Election Poll? The Exit Poll is math in working, with field staff of a huge sample (20 000 on the General Election 2015) asking real life voters directly after they left the polling stations. In a telephone poll an anonymous voice is hitting the respondent on an unchosen and often unwelcomed time. Whereas many real life voters fill out the gven sheet of paper after leaving the polling station, most of the phoned one’s hang up from the start. So the contrast with respect to the poll attendance rate between an exit poll and a telephone poll is pretty stark. YouGov Online Polling still bases on free will and some engagement in order to attend. In contrast to a Exit Poll, where every twentieth or tenth or fifth is directly approached with the assurance that he also went to the polls and not tentative will show up on his or her Polling Station.

In the balance a general tendency of overestimating the Labour vote, because so called downscale working class voters sometimes don't show up at the polling stations plus the impact of a Coalition Government and a fixed five Year term, lead to an effect otherwise experienced on the continent, where undecided voters break finally heavily for the leading party in government. There are also several pre-election studies showing Cameron leading with undecidets and this group prefering least a Labour SNP Deal in order to get through a Queen Speech. A ComRes poll from April 24th concentrated on the undecided vote and found the following remarkble views in this otherwise clouded group: "ComRes found 93 per cent of undecided voters said the economy will be the most important factor determining who they are going to vote for. More than 40 per cent of those polled said the Tories will do most to boost economic growth, compared with just 15 per cent who trusted Labour to grow the economy."

If there is one consequence for the next General Election, only a Labour lead of some proportion should anticipate in the Labour headquarter the real prospect of becoming the largest party in Westminster. And sometines the domestic state of the economy can tell us more about the reelection prospect than any poll on this very Earth.