by Rob Marchant
Much has been made during 2012 of Labour’s solid poll lead, which has been of the order of 10% ever since the government’s disastrous Budget. For some it seems difficult to refrain from mentally converting this into a line on Peter Snow’s election night model of the House of Commons, showing a majority for Labour.
What this fails to account for, quite apart from the changed circumstances which may prevail in two years’ time (and which may be much more attractive for the Tories, as Peter Kellner cheekily points out here), is that incumbents tend to have a dip at mid-term anyway. Quantifying this effect would obviously help us to make more accurate forecasts, to the extent that this is possible.
Last week at Uncut, Atul Hatwal showed the numerical arguments against a win for any party which failed to establish a poll lead on the economy. In a complementary way, we can try and allow for this mid-term effect, to try and get a better idea of where things might end up, all other things being equal.
There are various ways of trying to gauge it*. A year ago, an excellent piece of analysis was carried out by Leo Barasi (and for which I am indebted to John Rentoul for pointing out) on post-war polls both eighteen months into a parliament and two years out from the next, which concluded, rightly, that Labour’s prospects were numerically better than many seemed to think.
Interestingly, this conclusion now looks a little less secure, not because Barasi’s model has changed or become less applicable, but because the polls have actually improved for Labour in the last year and therefore many commentators, who were very negative at the end of last year, have gravitated towards a much warmer view of Labour’s prospects.
To revisit the second part of that analysis – that of two years out from the next election – is now both timely and illustrative, which we do now in this graph:
It is interesting to look at the regression line but the data points, too, tell a story. It is also useful to apply a little political history to the figures.
First, the most striking thing to note is that over four-fifths of postwar oppositions have had a lead in the polls at this point, two years out from an election. Irrespective of whether they go on to win or lose. So having a lead in itself is hardly remarkable: in fact, allowing for outliers**, they almost all do. It is the size of the lead that counts. Conclusion: don’t get excited about being ahead in the polls at this point. It seems to be a necessary, not a sufficient condition. Not being ahead, now that would be something.
Second, since the war, no opposition has won without polling at least 6% two years out. In fact, this lowest-polling election-winner from opposition, two years out, was Labour in 1964 (which Barasi classifies as an outlier). The regression implies that, on average, it should be at least 12% – comparable to today’s lead of 11% – for a marginal win on vote-share, and in fact the other four cases were all 14% or more.
Moreover, historically, 1964 was clearly an “only just” election; Wilson won by a whisker, and if he had not had the anomalous effect of no small parties (outside the main parties, there were only an unprecedentedly small number of 9 Liberal MPs; no unionists, no nationalists, no independents), he would have had to form a coalition. In the end, he had to call another election two years later to get a proper working majority.
Third, we can get a rough measure of the “mid-term effect” by looking at the line of best fit, which is interesting. In general, the lead not only narrows very significantly at the election, it also tends to swing a few percentage points in favour of the incumbent.
So, in the case where the party ahead in the polls has gone on to win, we can see that final lead has been, on average, around a third of the mid-term lead, less 3.64%. Our mid-term lead of 11%, now projected out into a general election lead, is not in fact 11% – as our intuition might suggest – but close to zero.
There are some significant caveats to all this. For a start, there is obviously a significant margin for error on either side. And, as a general rule, we must be wary of the argument “no-one has ever won an election without X”. As this cartoon and article brilliantly point out (h/t: Anthony Wells), you can always apply this argument, irrespective of the relevance of X as a factor. For example, no African-American had ever become US President – until Obama.
However, if we can accept the idea that this case is different and that there is at least some fairly strong correlation between polling levels and election results – which seems eminently reasonable, given that one is supposed to be a direct, if error-prone, measure of the other – it should surely give us some pause for thought (there are also some more technical caveats***).
Conclusion: it is clear that the next election is far too close to call and, as that Wells points out at the excellent UK Polling Report, despite Labour and the Tories both playing down their chances, someone has to win it. It is highly questionable to try and make a prediction now on who will win or lose. It is also true that there is still time to upset these historical precedents and for Labour to secure a full majority, or for 2015 to be an outlier, like 1964.
However, one thing that we can conclude is that it is a highly unsafe bet that Labour will win a majority, despite what its current 11-point lead says, because of this mid-term effect. When we combine it with Atul’s results on economic polling, the picture for Labour looks still more tenuous.
So, Labour needs to establish a much stronger poll lead if it is seriously aiming for a majority in 2015 and allowing a sufficient margin for error. Currently, we are most likely to be right on the edge of winning (that is, winning the highest proportion of the vote, not a majority) – no more.
In fact, because of Labour’s higher standing in the polls and commentators’ more favourable perceptions, it is arguable that the reverse is now true of Barasi’s (informal) conclusion, that many seemed to be underestimating Labour. It now seems that some commentators are now seeing Labour’s win as somehow a strong probability, when its position is in fact rather more tenuous. On the basis of this model, one slip and we would not even have the highest vote-share, let alone a majority.
In short: hold the champagne.
Notes:
*It might also be interesting to use the base data to regress the average “mid-term dip” for incumbent governments against (i.e. the % reduction in polling between an election win and the polling two years before the next one) another variable, something which I confess I have not had time to do.
**There were only three exceptions to this out of 16 data points, and two of these were the elections of 2001 and 2005, where the previous landslide victories for Labour meant the Tories were starting from an exceptionally low base of support and did not have time to recover. We are clearly not in that scenario because of the relative closeness of the 2010 election. So if we exclude these as outliers, it is only one exception out of 14 data points.
***Other caveats:
- The R2 is a respectable 0.4, but Barasi himself points out that, when at the margin, the polls two years out are potentially not great predictors. For this reason we don’t try to predict who will win.
- The number of data points is clearly less than we would like.
- A few odd general election data points have been skipped, for very good reasons to do with the inapplicability of two elections in quick succession such as 1974 – see Leo Barasi’s source for more details.
Rob Marchant is an activist and former Labour Party manager who blogs at The Centre Left
Tags: election results, Leo Barasi, mid-term blues, polls, regression analysis, Rob Marchant
According to the graph, there’s only been two occassions when an opposition lead of more than 10% resulted in election loss (bottom right block).
1981 (1983 election) labour lead was negated by Falklands victory in 1982.
1985 (1987 election) labour lead was somehow lost by combination of looney-left faction and slick Tory election machine.
Other labour loss in 1992, that appears to have had a 7-8% mid-term lead was reversed when the Tories changed leader.
I can’t see any of those factors reversing the 10% lead this time. I can’t see the ecconomy and jobs picking up that much.
This is interesting, and you’d expect the gap between parties to close as the election approaches.
However, it is worth noting that (given the probable electoral arithmetic and current boundaries) a tie in the percentage vote at the general election would make Labour the biggest party by a significant margin – even if it didn’t quite manage an overall majority. And even a very small lead of just one or two percentage points in the national polling could deliver a working majority.
It would also mark a remarkable comeback given the mauling we suffered in 2010.
It looks like being 1992 all over again, which isn’t as bad as being 1983.
So some consolation.
“two of these were the elections of 2001 and 2005, where the previous landslide victories for Labour meant the Tories were starting from an exceptionally low base of support and did not have time to recover. We are clearly not in that scenario because of the relative closeness of the 2010 election.”
dunno about relative closeness of 2010. we lost by 7%, compared to the Tories losing by 9% in 2001, but our share of the vote was almost 2.7% below what Hague managed.
@Scott, sorry but you’re reading the graph wrongly, I think you’ve mixed up the axes. The 10% you have quoted refers to the election result, not the polling, so you’re right about there being two occasions but they are 1992 and 1959.
And meanwhile there only four occasions when we went on to win. So it’s two versus four on a sample space of six (where they polled >10% at midterm), you can conclude very little from this. My conclusion is based on one exception out of 14 data points.
@Martin: an interesting point: however, it’s fairly difficult to directly convert vote-share into seats, especially with a three-party dynamic. There is probably some truth in this effect, but I’d like to see some hard data.
Wow, a Rob Marchant article with a pessimistic approach to Labour?!
For me, there are two questions for which the answers make me very optimistic when looking at past election trends.
1) When was the last time a government increased their representation in the Commons?
2) When was the last time the British electorate wanted a Conservative majority?
As both answers require an s at the end of the word decade, the champagne is on ice.
While this historical data is of interest I don’t think it is of great deal of use in projecting forward to 2015 because:
– There are huge variations in the graph. Events matter and can go against the government as well as for it.
– Polling methodology has changed significantly. Polls overestimated Labour at every GE from at least 1979 to 2005. In particular, polls up to 1992 significantly overestimated Labour. 2010 was the first time polls underestimated Labour. So comparing polls today to those in previous decades is not especially relevant.
– There are very few data points. In any form of scientific investigation drawing conclusions on a sample size of 14 would be seen as foolhardy.
All we can say for any certainty is that it is better to be ahead than behind, and it is better to be ahead by more than less.
For that reason I find analyses based on the particular political conditions now (the effect of the coalition, the importance of party leadership and economic competence ratings, tactical voting, the effect of UKIP etc.) to be more persuasive than these historical analysis that can only tell us very little.
@Martin – I think you are wrong to assume Labour would win a majority with a national vote share lead of only 1 or 2 percent. On a national uniform swing Labour would need a lead of 4 percent. In addition, even if the Lib Dems collapsed Labour would win relatively few of these seats providing little help in winning a majority with a lower lead.
Finally there is the issue of incumbency effect. The conservatives will likely do 2 – 3 percent better on average in the 100 seats they gained in 2010 because of this (it is strongest at the first election, and counts double if a Labour MP lost seeking re-election). This is something often forgotten in the debate about the boundary changes and should be factored in. It will not help the Conservatives win a majority as they need to win new seats to do that but it will make it more difficult for Labour to.
I think Labour are likely to need at least a 5 point national lead at the next GE to win an outright majority.
@swatantra: well hopefully not! But it is illustrative to look at the sheet size of the midterm lead we had in 1992 (double what we have now, and we still lost) as a worst-case scenario.
@TomP: no, I’m actually making a different point here. My point is that if and only if the previous election was a landslide, you can argue that the opposition simply didn’t have time to recover in 2-3 years to have a positive lead. We certainly didn’t have a landslide in 2010, hence this doesn’t apply. When I talk about closeness, I mean relative to a landslide (i.e. practically all election victories, except 1997, 2001 and 1945).
1,danny When was the alst time A govenment increased it’s prescence in parliament, In 1983, when the Tory lead wnet up from 44 to 144, also in 1955, and 1959, and 1966 and October 1974
2o.k 1992, but when was the last time the Public wanted a Labour Govenment who admitted they couldn’t control the economy , answer 1974
Hi Rob
Thanks for the reply, I take your point.
2010 was a weird one because it was a major rejection of Labour, but not a thumping endorsement of the Tories. Therefore I think we have done ‘better’ than Tories post 2001 election, but largely down to Lib Dem switchers. And it also superficially looks better for us because the Tories didn’t get a monster share of the vote in 2010, and so it’s relatively easier to establish a lead when they shed a few %.
@Danny: ah, I’m sorry for depressing you with those pesky, uncomfortable facts. Now, about your argument: you have fallen into the precise trap outlined in the piece, of “there has never been a black president” – until Obama. You can claim an awful lot of things which have never or scarcely happened, therefore X, but it’s not an argument. It’s hope.
@Kieran: I would accept a lot of your caveats (indeed, stated most of them in the piece, as did Leo in his piece), but they are caveats, not facts which completely destroy our ability to draw any conclusions. You will note that I have specifically not attempted to say who will win, because I don’t think that’s possible, at least at this point.
The conclusion I make is simply that a Labour victory is extremely uncertain, so those who are taking it as such are wrong to do so. Especially those who think we are likely to have a majority. I think that’s a quite reasonable conclusion to draw on this evidence, even after taking into account these caveats.
Also some interesting thoughts on your answer to Martin. I’d agree that 1-2% seems unlikely to be enough, and even then a majority needs to be sustainable, not marginal (as John Major found out to his cost).
Since this article will be read by relatively few Labour supporters, an interesting question is how most MPS/activists/bloggers will react as reality sets in, in 2 years time. Will we see a return to the venerable Labour tradition of blaming The Leader/other factions & demands that Labour shift rapidly left/right or will you all hold your nerve & tongues ?
@john P reid: Exactly. Decades.
@Rob: It didn’t depress me, it did little to change my hypothesis on the outcome of the 2015 General Election. I can’t remember where the article was, but I read something last week which suggested bookmakers are actually a good measure as unlike the bloggers and article authors (most of whom will have a personal agenda), they have to put their money where their mouth is.
And with the accusation of me falling into a “trap”, are you not adopting the exact same approach with your fancy graph? Pointing out that the poll lead we have now does not always lead to a majority come election day. That’s as much of an argument as my assertion that because it has been over 20 years since the electorate have returned a Tory majority and will have been over 30 years since an incumbent government increased their number of seats in the Commons (although when Thatcher did this, she did it despite losing nearly three quarters of a million votes) by the next election, then a Labour victory in 2015 is the most likely.
I agree that complacency would be very, very dangerous and the last thing that anyone wanting a Labour majority should be doing is resting on their laurels. I just find it particularly peculiar that the majority of Labour doom-mongers appears to be their own “supporters”. You cannot help feeling that certain advocates of the Labour party read the positive signs with regards to opinion polls and are disappointed. Worse than disappointed, they make an effort to turn the positive signs on their head and make out we’re struggling.
The reality is the current government are useless, Ed Milliband is doing a decent job and the public seem to be prepared to be coming round to the idea that the current shadow cabinet will not cause mishandle the economy in the way the previous Labour administrations did. Yes there is still work to do, but as things stand, we should only have to put up with this current administration for another 2 and a half years.
No amount of Rob Marchant and Peter Watt articles are going to stop me feeling pretty pleased about that thought!
Rob – I’m not disputing the fact that a Labour majority at the next election is uncertain I would agree with you. It is probably about a 50/50 proposition at best as reflected in the bookies odds. And 2 years is a long time in politics. However, I think we can say Labour are more likely than not to be the largest party in parliament. That might only require being within 1 / 2 points of the Conservatives.
My point is that as interesting as the historical analysis is I just don’t think it adds much at all to an analysis of the current situation. In the comments you have started to ‘exclude’ elections because of specific circumstances. Once you start doing that it all starts to seem an exercise to do too much with too little data. For example, we have very few examples of what effect a coalition will have on election outcomes. And while you say we should not let the caveats stop us from drawing some conclusions you point particularly to the 1992 result. But that is hugely distorted by the polling. Polling then massively overestimated Labour support. Polling today does not (on the latest evidence of 2010).
My own view is that while a Conservative majority is quite unlikely – probably just a 15 – 20% shot. I think a worst case scenario for Labour is likely to be 35%. If you look at the trend of the polls Labour jumped to the high thirties pretty much as soon as the coalition was formed and has never dropped below this. Even if some of the Lib Dem –> Lab are lost 35% seems fairly achievable. Added to this traditional supporters are probably going to be more motivated to vote in 2010 and while this may be concentrated in safe seats it will also help in marginals (Corby is an example of this, a marginal with some areas of very strong Labour support where turnout differentials are important). That would mean the Tories would need to break 40% and possibly need to get as high as 43% / 44%. Their best chance is a Lib Dem collapse in which they pick up 20 / 25 seats.
Past that though it is all to play for and I view those who can read with a crystal ball the 2015 result with great scepticism.
Yes Kieran, it is all to play for in 2015, which is very good considering Labour’s low vote in 2010. I actually quite like the idea of a Lab-Lib coalition, which would avoid Blairite insanity.
@TomP: Yes, it’s true we’re in a far better place than the Tories post-2001, however that’s largely because they were soundly thrashed in 2001 (Labour landslide) and two years later couldn’t even manage a poll lead. That said, it sadly doesn’t always translate into a win (e.g. ’92 when we had 22% lead two years out and Tories ended up with clear, if small, majority.
@Danny: good luck with that!
@Kieran: Hmm, you make some decent points, but I’m not sure if you’re entirely consistent. “I think we can say Labour are more likely than not to be the largest party in parliament. That might only require being within 1 / 2 points of the Conservatives.” If you mean are Labour at least 51% probable to be the largest party, you may be right. But it doesn’t tell us very much, and it’s also a guess rather than a calculation. And I’m not sure I buy the 1/2 pt away from the Tories part.
I point to the 1992 result, but only in the comments, and only to show what is possible. It’s an extreme case, but I also think your “flawed polling” is the wrong conclusion. The problem then was not the polling (which said “too close to call”) but the interpretations put on it by feverish journalists, desperate for a prediction. It also missed some one-off last-minute effects, like the Sun’s vicious, personal campaign against Kinnock.
“I think a worst case scenario for Labour is likely to be 35%” – now who’s making predictions! Seriously, I just can’t reach that point of estimation. My only objective for this piece was to show what an *unsafe* verdict from polling is. There is not much of a safe one, except that a decent majority for either party is (as of today) less likely than a hung parliament. This may well change as circumstances change approaching the election. In the meantime, we can note that there is a clear tendency for polls to narrow substantially and, on top of that, to swing a little against the challenger.
@Rob I think you slightly misunderstand my point re: prediction. I’m not saying all forecasting is wrong but that:
a) Basing this on history is dangerous – and I think we agree on this to a great extent as your Obama example shows.
b) Expressing certainty about the outcome is also foolhardy.
However, that does not mean we cannot try to estimate the likelihood of various result based on the huge array of political factors in play. In these terms the graph does move us forward in that it shows in the vast majority of cases the government does better at a GE than in polls 2 years before. But the extent of this improvement varies markedly. You are therefore right to say those who blindly equate polls today to a likely result in 2015 are being naive.
The other point is that when I think Labour is likely to get at least 35% next time I am not trying to claim any ‘scientific’, or even evidential support for this position, it is based on my assessment of the evidence and my subjectivity. It is based on the fact that the Labour share in the vote has very rarely dipped below this point since 2010 – and there has been essentially 8% of voters which have switched directly from LD to Labour. My question is what will cause them to return to the LDs? This switch happened even while the government was generally quite popular and has remained stable for over 2 years. I therefore think it is quite conservative to assume even in a worst case scenario that Labour will keep their 2010 voters (plus and minus churn) and keep 5% of the LD vote getting them to 35%. Of course this might be wrong, but seems a reasonable assessment. And it is from that basis that we can talk about what the Conservatives need to do to get a majority.
And given the distance until the GE I think the best way to look at the likely outcomes is to assess the probable bands of support for each party. Currently my assessment of this would be Lab 35 – 42, Conservatives 30 – 42, LD 12 – 18. Clearly, other people will have different assessments but those are mine – and they are not based on what happened in 1992 or 1955 or 1970. NB – The Conservative share is most unpredictable for me because of the UKIP wildcard. My view is they are in reality likely to have a similar floor to Labour at 35, but there is a small, but not negligible, chance that UKIP could push them down lower.
@Kieran: it’s difficult to comment on one’s gut feel for polling, because it might be bang-on or it might be rubbish, but we won’t really know till election night. For this reason (and also because I’m about to leave on holiday!) I’d just comment on one thing: my point about Obama is a very specific one, not quite what you meant, and I feel I ought to explain it better.
Basing analysis on history is not dangerous per se: the danger is when you start basing your analysis on things which have no real connection between them, but just happen to follow a similar pattern. The Obama example is one of millions of such spurious correlations. But historical analysis is perfectly valid, up to a point: for example, in extremis, if a pattern has repeated itself 99 times (and the events are not random events entirely independent from each other), it’s pretty likely that the 100th will be similar.
Now, I believe it is reasonable to specifically exclude Leo Barasi’s figures from this effect, for the obvious reason that polls are clearly closely related to vote share, if not a 100% accurate measure, they are a measure. If on the other hand we were to choose vote share against, say, number of MPs who have ginger hair, we might just end up with a +ve correlation by chance, but it wouldn’t mean anything. So that’s the difference: whether or not the events are necessarily related.
Have a good holiday Rob! I get your point about drawing conclusions about things not necessarily related and clearly polls two years before an election have some relationship with the eventual results (oppositions almost never do better) but that still leaves a very wide range of outcomes.
And I’m not sure the obama example works as you say. There is clearly a reason why the first 43 presidents were all white – ethnicity is clearly an important part of the us political system – there is a real connection.
Finally, you’re right that we don’t know whether a gut feeling will be right until the election. But that is true of any forecast! My point is that in trying to make a forecast I think it is more useful to look at the circumstances today than try to find historical parallels.