Post by pim on Jul 29, 2020 0:12:50 GMT 10
Morrison's election upset forced a reckoning on pollsters. Here's what we've changed
Peter Lewis 28 July 2020 www.theguardian.com/australia-news/commentisfree/2020/jul/28/morrisons-election-upset-forced-a-reckoning-on-pollsters-heres-what-weve-changed?CMP=Share_iOSApp_Other
It seems like ancient history now. But before the pandemic and before the bushfires, there was another event that invoked adjectives like shock, unprecedented and “once in a generation”. In May 2019 Scott Morrison conjured up an against-the-odds election win that even many of his most loyal followers didn’t see coming. When the prime minister defied the projections of all the major polls to hold onto power it was not just the Labor opposition that was forced into a period of existential soul-searching. We political pollsters, and the media organisations that relied on them, were also forced to confront our own performance.
Essential’s final poll, like all the major published polls, was on the outer edges of the margin of error in projecting a narrow Labor victory. While all polls state that their results fall within a range of about 2% for a sample of 1,000, the success of most polls in landing close to previous results, particularly in 2010 and 2013, had imbued the two-party-preferred vote (2PP) with an almost mystical sense of certainty. This time, however, we were not on the money. There was clearly a need to review our methodology, particularly the way we weighted our sample and distributed preferences.
Over at Guardian Australia, editor Lenore Taylor was also reflecting on how her news site reported on the polling results. She told me that, despite taking care in how polls were reported, most observers had made assumptions based on the 2pp results indicating an enthusiasm for change which jarred with the disaffection and uncertainty her reporters were picking up when out in the field. We decided to suspend publication of primary voting intention and the two-party preferred and spend some time reflecting – not just on how we poll, but also why. In the past decade, four prime ministers have been removed by their peers, in large part on the basis of their poor polling, whereas only one had been removed by the people. From the dual vantage points of editor and pollster, we asked ourselves whether horse-race polling had done its dash and whether we need to imagine a better way of reflecting public attitudes.
Since the election, the Guardian Essential Report has continued to canvass the public, collecting the data but only releasing broader attitudes on leadership, on climate and more recently charting our reaction to this unprecedented crisis. Free of the tyranny of voting intention, we have been able to draw rich insights from the public as we have grappled to make sense of a world turned upside down. This has allowed us to look past the partisan battle to understand the real story of this pandemic: in crisis we put our faith in our institutions and, as they proved resilient, our regard for them has grown. Now we are ready to re-engage with voting intention, but conscious of the lessons of 2019. Like other pollsters, we have updated our calculations to incorporate the 2019 election results, including the shift of preference votes from minor parties and factoring in United Australia party, which was absent prior to the 2019 election. We have also updated our demographic information, which is now representative by individual states, rather than just the country as a whole. We believe this will better reflect the views of the population as a whole.
Secondly, we have changed the way we allocate preferences when calculating the two-party preferred votes. Usually pollsters allocate preferences for voters of minor parties on the basis of the split at the previous election. Last election, this created a distortion because One Nation (which had allocated preferences against all sitting MPs in 2016) decided to preference the Coalition. We will now be asking participants who vote for a minor party to indicate a preferred major party. Only when they do not provide a preference will we allocate based on previous flows. But these technical tweaks only go part of the way towards addressing the issues we have been grappling with.
One of our key reflections from last year was the large number of voters surveyed who could not indicate a voting preference at all. Even in the final week of the campaign 8% of respondents could not even say who they were leaning towards. Following normal polling practice, these voters were removed for our sample so we could report a clean split that added up to 100%. This is a totally understandable practice in countries where voting is optional and the “don’t knows” are unlikely to vote. But in a system of compulsory voting, the elegance of 2PP has a distorting effect. In fact, Labor was never ahead 51-49% 2PP as was reported. They were ahead about 47-46%, with seven undecided. Reporting this figure, as a political work in progress, rather than 51-49 as a prediction, would have created an entirely different perception of the contest. In removing the undecideds from the sample we had disenfranchised the disengaged. Keeping them in would have, more accurately, placed a spotlight on them and raised their importance to the final days of the campaign.
With this in mind, the Guardian Essential poll is today launching what we will call “2PP+”, an allocation of major party preferences that also includes those who are unable to give an indication of their voting intention. We don’t think it takes anything away from the insight of polling, apart from the assertion that we have a winner. The below table shows how the new 2PP+ can tell a compelling story that would not be surprising to anyone who has been awake over the past seven months. The government’s narrow win was consolidated after the election, before the summer bushfires saw it head south. Then, as our institutions kicked in, the government recovered.
(Go to link to see graphic)
What’s also striking in these numbers is that, while we know support for the government’s handling of Covid-19 has been in the 70s, this has not been reflected in party political vote. Governing constructively has not been a zero sum game where one side of politics has to lose. Labor voters have given the government credit but not, at this stage at least, their vote. Indeed, in the past month as the reality of second waves and the looming economic crisis has become more real, the gap has narrowed, with more voters indicating support for Labor. Indeed, if we were currently in a horse race we would say that Labor has its nose in front. Which, two years out from an election – and 8% unable to indicate a choice – is neither particularly constructive nor illuminating.
This brings us to the final change we propose making. While 2PP+ is back, we do not intend to release its findings week on week. Instead, we will collect the figures and release them each quarter, giving the numbers the space to take form and build a story over time, rather than running a real-time political scoreboard. This seems more honest – it provides a rear-view mirror view of how public sentiment is changing, rather than purporting to predict where it will be in the future. More significantly, it leaves space for us to focus on the issues and attitudes that inform votes, rather than seeing the vote as the end in itself. In the middle of a global pandemic, the last thing the public wants is horse race analysis, but there will be an election in the future and understanding shifts in voter sentiment is part of seeing this context with clear eyes. Hopefully our new approach to reporting on voting intention will contribute to that.
Peter Lewis 28 July 2020 www.theguardian.com/australia-news/commentisfree/2020/jul/28/morrisons-election-upset-forced-a-reckoning-on-pollsters-heres-what-weve-changed?CMP=Share_iOSApp_Other
It seems like ancient history now. But before the pandemic and before the bushfires, there was another event that invoked adjectives like shock, unprecedented and “once in a generation”. In May 2019 Scott Morrison conjured up an against-the-odds election win that even many of his most loyal followers didn’t see coming. When the prime minister defied the projections of all the major polls to hold onto power it was not just the Labor opposition that was forced into a period of existential soul-searching. We political pollsters, and the media organisations that relied on them, were also forced to confront our own performance.
Essential’s final poll, like all the major published polls, was on the outer edges of the margin of error in projecting a narrow Labor victory. While all polls state that their results fall within a range of about 2% for a sample of 1,000, the success of most polls in landing close to previous results, particularly in 2010 and 2013, had imbued the two-party-preferred vote (2PP) with an almost mystical sense of certainty. This time, however, we were not on the money. There was clearly a need to review our methodology, particularly the way we weighted our sample and distributed preferences.
Over at Guardian Australia, editor Lenore Taylor was also reflecting on how her news site reported on the polling results. She told me that, despite taking care in how polls were reported, most observers had made assumptions based on the 2pp results indicating an enthusiasm for change which jarred with the disaffection and uncertainty her reporters were picking up when out in the field. We decided to suspend publication of primary voting intention and the two-party preferred and spend some time reflecting – not just on how we poll, but also why. In the past decade, four prime ministers have been removed by their peers, in large part on the basis of their poor polling, whereas only one had been removed by the people. From the dual vantage points of editor and pollster, we asked ourselves whether horse-race polling had done its dash and whether we need to imagine a better way of reflecting public attitudes.
Since the election, the Guardian Essential Report has continued to canvass the public, collecting the data but only releasing broader attitudes on leadership, on climate and more recently charting our reaction to this unprecedented crisis. Free of the tyranny of voting intention, we have been able to draw rich insights from the public as we have grappled to make sense of a world turned upside down. This has allowed us to look past the partisan battle to understand the real story of this pandemic: in crisis we put our faith in our institutions and, as they proved resilient, our regard for them has grown. Now we are ready to re-engage with voting intention, but conscious of the lessons of 2019. Like other pollsters, we have updated our calculations to incorporate the 2019 election results, including the shift of preference votes from minor parties and factoring in United Australia party, which was absent prior to the 2019 election. We have also updated our demographic information, which is now representative by individual states, rather than just the country as a whole. We believe this will better reflect the views of the population as a whole.
Secondly, we have changed the way we allocate preferences when calculating the two-party preferred votes. Usually pollsters allocate preferences for voters of minor parties on the basis of the split at the previous election. Last election, this created a distortion because One Nation (which had allocated preferences against all sitting MPs in 2016) decided to preference the Coalition. We will now be asking participants who vote for a minor party to indicate a preferred major party. Only when they do not provide a preference will we allocate based on previous flows. But these technical tweaks only go part of the way towards addressing the issues we have been grappling with.
One of our key reflections from last year was the large number of voters surveyed who could not indicate a voting preference at all. Even in the final week of the campaign 8% of respondents could not even say who they were leaning towards. Following normal polling practice, these voters were removed for our sample so we could report a clean split that added up to 100%. This is a totally understandable practice in countries where voting is optional and the “don’t knows” are unlikely to vote. But in a system of compulsory voting, the elegance of 2PP has a distorting effect. In fact, Labor was never ahead 51-49% 2PP as was reported. They were ahead about 47-46%, with seven undecided. Reporting this figure, as a political work in progress, rather than 51-49 as a prediction, would have created an entirely different perception of the contest. In removing the undecideds from the sample we had disenfranchised the disengaged. Keeping them in would have, more accurately, placed a spotlight on them and raised their importance to the final days of the campaign.
With this in mind, the Guardian Essential poll is today launching what we will call “2PP+”, an allocation of major party preferences that also includes those who are unable to give an indication of their voting intention. We don’t think it takes anything away from the insight of polling, apart from the assertion that we have a winner. The below table shows how the new 2PP+ can tell a compelling story that would not be surprising to anyone who has been awake over the past seven months. The government’s narrow win was consolidated after the election, before the summer bushfires saw it head south. Then, as our institutions kicked in, the government recovered.
(Go to link to see graphic)
What’s also striking in these numbers is that, while we know support for the government’s handling of Covid-19 has been in the 70s, this has not been reflected in party political vote. Governing constructively has not been a zero sum game where one side of politics has to lose. Labor voters have given the government credit but not, at this stage at least, their vote. Indeed, in the past month as the reality of second waves and the looming economic crisis has become more real, the gap has narrowed, with more voters indicating support for Labor. Indeed, if we were currently in a horse race we would say that Labor has its nose in front. Which, two years out from an election – and 8% unable to indicate a choice – is neither particularly constructive nor illuminating.
This brings us to the final change we propose making. While 2PP+ is back, we do not intend to release its findings week on week. Instead, we will collect the figures and release them each quarter, giving the numbers the space to take form and build a story over time, rather than running a real-time political scoreboard. This seems more honest – it provides a rear-view mirror view of how public sentiment is changing, rather than purporting to predict where it will be in the future. More significantly, it leaves space for us to focus on the issues and attitudes that inform votes, rather than seeing the vote as the end in itself. In the middle of a global pandemic, the last thing the public wants is horse race analysis, but there will be an election in the future and understanding shifts in voter sentiment is part of seeing this context with clear eyes. Hopefully our new approach to reporting on voting intention will contribute to that.