The Sun on Sunday Attacked for Misleading Climate Change Polling

picture-25876-1571179299.jpg
on

Itโ€™s as clear as your iced latte on a hazy heatwaveย day.

Most people think this summerโ€™s hot weather was due to climate change. Or, confusingly, most people think climate change had nothing to do this yearโ€™s searingย summer.

You were likely greeted with one of those headlines on Monday morning, though which you saw largely depended on which newspaper you happened to pickย up.

In a poll that mainly focused on Brexit and party politics, Deltapoll, on behalf of the Sun on Sunday, slipped in a question on climate change. The results led the tabloid to present a bright blue box claiming โ€œBrits Cool on Global Warmingโ€, with most respondents apparently believing โ€œBritainโ€™s sizzling summer was not the result of climateย changeโ€.

This was in stark contrast to a poll by YouGov on behalf of environmental lawyers ClientEarth, which found 82 percent of respondents thought climate change was responsible for causing heatwaves now and in theย future.


Image: Coverage of the DeltaPoll poll in the Sun on Sunday (left) and coverage of the ClientEarth/YouGov poll in Energy Voiceย (right)

The hot weather appears to have led to a race to weaponise climate change polling, so who toย trust?

Well, according to academics, Deltapollโ€™s effort, reported by the Sun on Sunday, was โ€œone of the worst climate change questionsโ€ everย seen.

Pollingย Problems

Deltapoll constructed a long, thorough survey on the state of the UKโ€™s politics. But when it came to asking about whether the heatwave was caused by climate change, โ€œnot really in line with standard practiceโ€, Stuart Capstick, a Research Associate at Cardiff Universityโ€™s School of Psychology told DeSmog UK.

And that probably had a significant bearing on theย result.

The issue is that the climate question is โ€œmulti-barrelledโ€, Capstick said, with multiple answers embedded within each of the two options presented.

โ€œFor example, the first answer has embedded within it: did climate change cause the heatwave, is climate change human-caused, will this continue to get worse, unless a lot isย done.โ€

โ€œHow am I supposed to answer if I agree with the first two points, but not the secondย two?โ€

Nick Pidgeon, a professor also at Cardiff University, agreed the question was poorlyย presented.

โ€œIt’s particularly hard to answer honestly yes to all of these in one go with a general knowledge level, and anybody who does you can’t unambiguously ascertain why the answer is yes or no. They should have asked four separate questions on each of theseย elements.โ€

โ€œThe Sun on Sunday question is about one of the worst climate questions I have everย seenโ€.

And worse than that, neither of the two options presented actually reflect the balance or nuance of what most scientists are saying, according to Leo Barasi, a polling expert and author of The Climate Majority.

โ€œThe issue here is that neither of the options are the correctย one.โ€

โ€œItโ€™s fairly common in polling to do polarity questions like this where you sketch out two extreme positions and ask them which one theyโ€™re closerย to.โ€

โ€œBut when itโ€™s understanding of a scientific issue, itโ€™s not a helpful way ofย polling.โ€

โ€œThis is trying to understand how far people recognise the science, so not presenting the thing that scientists say is the case is a pretty poor way of phrasing theย questionโ€.

Attributing any particular extreme weather event to climate change is tricky, and no-one is certain how rising emissions will specifically affect things like heatwaves or cold snaps and hurricanes in theย future.

But a rapid assessment by scientists found this yearโ€™s heatwave in Northern Europe was as much as five times more likely to have occured thanks to climateย change.

And the level of uncertainty around one-shot questions that appear in surveys that are largely about other issues mean the result may not be reliable anyway, Zorzeta Bakaki, a lecturer at the University of Essex, told DeSmog UK.

Because itโ€™s a one-off question, โ€œthe margin of error, and the uncertainty surrounding it, is likely to be way higher than the +/- 4 percent of the voting question. Under those circumstances, a difference of 13 percentage points may be of little informative valueโ€, she told DeSmog UK.

Deltapoll declined to comment when contacted by DeSmog UK.

Oppositeย Conclusions

In contrast to the Sun on Sunday poll, a YouGov poll commissioned by environmental lawyers Client Earth and released the same day found a very large majority of respondents attributed the recent heatwave to climateย change.

While the experts generally agreed that the YouGov polling was more reliable, there were a couple of factors that may have driven up the number of people agreeing that climate change was responsible for theย heatwave.

For example, because it appeared in an environmental survey, people were less likely to want to appear to be โ€œskepticalโ€, Bakaki said. And the question is framed in such a way so as to passively โ€œinformโ€ respondents that weather and climate change may be related, sheย explained.


Image: The heatwave question in the ClientEarth/YouGovย poll

A ClientEarth spokesperson pointed out that the method YouGov had chosen deliberately tried to minimise โ€œsocial desirability biasโ€, where people give answers that make them look good. And they said the poll had deliberately only included options that reflected the scientific literature, but had also provided people with the option to say that none of the suggested impacts were due to climate change (which 11 percent agreedย with).

โ€œWe wanted to investigate public sentiment towards the deeper issues of climate change and ask detailed questions around who should pay, how the government should respond, the role of the financial sector and how we power our future,โ€, theyย said.

โ€œThe YouGov survey proves that the British public are engaged with the issue of climate change and overwhelmingly support concerted action from the government and want to see those responsible held toย account.โ€

Ultimately, polling is always political in some way, with the responses sometimes proving inconvenient for policymakers. The UK government recently dropped a fracking question from its regular energy survey after years of opposition for shale gas development consistently outstrippingย support.

But on the whole, journalists have become more aware and skeptical of interpreting too much from single polls, particularly in the post-Brexit, Trumpย era.

So the issue may be more with the presentation of polling results, than the polling itself. Which is why it might be time to move beyond simple polling questions, Chris Shaw, Senior Researcher with thinktank Climate Outreach, told DeSmog UK.

โ€œOn a polarised issue like climate change it follows that on the same day polls can reveal quite differing insights depending on who funded them and what they hoped to reveal,โ€ heย said.

โ€œThe whole polling and survey process can be quite unnatural, and doesn’t properly reflect the way people make sense of the world, which is a social process, mediated through conversation and peer-to-peerย interaction.โ€

โ€œPolling which is repeated over time with the appropriate academic rigour, and which synthesises qualitative and quantitative results, can generate useful insights. But in the end there is no substitute for sitting down and listening toย people.โ€

Main image: Lets Go Out Bournemouth/Flickr CC BYย 2.0

picture-25876-1571179299.jpg
Mat was DeSmog's Special Projects and Investigations Editor, and Operations Director of DeSmog UK Ltd. He was DeSmog UKโ€™s Editor from October 2017 to March 2021, having previously been an editor at Nature Climate Change and analyst at Carbon Brief.

Related Posts

on

A Peterson Academy lecture by Marian Tupy frames notorious killers as โ€˜extreme environmentalistsโ€™ who want to reduce Earthโ€™s population.

A Peterson Academy lecture by Marian Tupy frames notorious killers as โ€˜extreme environmentalistsโ€™ who want to reduce Earthโ€™s population.
on

DeSmog estimates raise questions over climate benefits as EU officials consider whether the technology should qualify for billions of euros in subsidies.

DeSmog estimates raise questions over climate benefits as EU officials consider whether the technology should qualify for billions of euros in subsidies.
Analysis
on

Experts accuse Farageโ€™s party of a โ€˜deliberate campaign of misinformation about climate changeโ€™ in the House of Commons.

Experts accuse Farageโ€™s party of a โ€˜deliberate campaign of misinformation about climate changeโ€™ in the House of Commons.
on

A Conservative peer and former UK trade advisor were among those who spoke at the summit.

A Conservative peer and former UK trade advisor were among those who spoke at the summit.