It’s as clear as your iced latte on a hazy heatwave day.

Most people think this summer’s hot weather was due to climate change. Or, confusingly, most people think climate change had nothing to do this year’s searing summer.

You were likely greeted with one of those headlines on Monday morning, though which you saw largely depended on which newspaper you happened to pick up.

In a poll that mainly focused on Brexit and party politics, Deltapoll, on behalf of the Sun on Sunday, slipped in a question on climate change. The results led the tabloid to present a bright blue box claiming “Brits Cool on Global Warming”, with most respondents apparently believing “Britain’s sizzling summer was not the result of climate change”.

This was in stark contrast to a poll by YouGov on behalf of environmental lawyers ClientEarth, which found 82 percent of respondents thought climate change was responsible for causing heatwaves now and in the future.

Image: Coverage of the DeltaPoll poll in the Sun on Sunday (left) and coverage of the ClientEarth/YouGov poll in Energy Voice (right)

The hot weather appears to have led to a race to weaponise climate change polling, so who to trust?

Well, according to academics, Deltapoll’s effort, reported by the Sun on Sunday, was “one of the worst climate change questions” ever seen.

Polling Problems

Deltapoll constructed a long, thorough survey on the state of the UK’s politics. But when it came to asking about whether the heatwave was caused by climate change, “not really in line with standard practice”, Stuart Capstick, a Research Associate at Cardiff University’s School of Psychology told DeSmog UK.

And that probably had a significant bearing on the result.

The issue is that the climate question is “multi-barrelled”, Capstick said, with multiple answers embedded within each of the two options presented.

“For example, the first answer has embedded within it: did climate change cause the heatwave, is climate change human-caused, will this continue to get worse, unless a lot is done.”

“How am I supposed to answer if I agree with the first two points, but not the second two?”

Nick Pidgeon, a professor also at Cardiff University, agreed the question was poorly presented.

“It’s particularly hard to answer honestly yes to all of these in one go with a general knowledge level, and anybody who does you can’t unambiguously ascertain why the answer is yes or no. They should have asked four separate questions on each of these elements.”

“The Sun on Sunday question is about one of the worst climate questions I have ever seen”.

And worse than that, neither of the two options presented actually reflect the balance or nuance of what most scientists are saying, according to Leo Barasi, a polling expert and author of The Climate Majority.

“The issue here is that neither of the options are the correct one.”

“It’s fairly common in polling to do polarity questions like this where you sketch out two extreme positions and ask them which one they’re closer to.”

“But when it’s understanding of a scientific issue, it’s not a helpful way of polling.”

“This is trying to understand how far people recognise the science, so not presenting the thing that scientists say is the case is a pretty poor way of phrasing the question”.

Attributing any particular extreme weather event to climate change is tricky, and no-one is certain how rising emissions will specifically affect things like heatwaves or cold snaps and hurricanes in the future.

But a rapid assessment by scientists found this year’s heatwave in Northern Europe was as much as five times more likely to have occured thanks to climate change.

And the level of uncertainty around one-shot questions that appear in surveys that are largely about other issues mean the result may not be reliable anyway, Zorzeta Bakaki, a lecturer at the University of Essex, told DeSmog UK.

Because it’s a one-off question, “the margin of error, and the uncertainty surrounding it, is likely to be way higher than the +/- 4 percent of the voting question. Under those circumstances, a difference of 13 percentage points may be of little informative value”, she told DeSmog UK.

Deltapoll declined to comment when contacted by DeSmog UK.

Opposite Conclusions

In contrast to the Sun on Sunday poll, a YouGov poll commissioned by environmental lawyers Client Earth and released the same day found a very large majority of respondents attributed the recent heatwave to climate change.

While the experts generally agreed that the YouGov polling was more reliable, there were a couple of factors that may have driven up the number of people agreeing that climate change was responsible for the heatwave.

For example, because it appeared in an environmental survey, people were less likely to want to appear to be “skeptical”, Bakaki said. And the question is framed in such a way so as to passively “inform” respondents that weather and climate change may be related, she explained.

Image: The heatwave question in the ClientEarth/YouGov poll

A ClientEarth spokesperson pointed out that the method YouGov had chosen deliberately tried to minimise “social desirability bias”, where people give answers that make them look good. And they said the poll had deliberately only included options that reflected the scientific literature, but had also provided people with the option to say that none of the suggested impacts were due to climate change (which 11 percent agreed with).

“We wanted to investigate public sentiment towards the deeper issues of climate change and ask detailed questions around who should pay, how the government should respond, the role of the financial sector and how we power our future,”, they said.

“The YouGov survey proves that the British public are engaged with the issue of climate change and overwhelmingly support concerted action from the government and want to see those responsible held to account.”

Ultimately, polling is always political in some way, with the responses sometimes proving inconvenient for policymakers. The UK government recently dropped a fracking question from its regular energy survey after years of opposition for shale gas development consistently outstripping support.

But on the whole, journalists have become more aware and skeptical of interpreting too much from single polls, particularly in the post-Brexit, Trump era.

So the issue may be more with the presentation of polling results, than the polling itself. Which is why it might be time to move beyond simple polling questions, Chris Shaw, Senior Researcher with thinktank Climate Outreach, told DeSmog UK.

“On a polarised issue like climate change it follows that on the same day polls can reveal quite differing insights depending on who funded them and what they hoped to reveal,” he said.

“The whole polling and survey process can be quite unnatural, and doesn’t properly reflect the way people make sense of the world, which is a social process, mediated through conversation and peer-to-peer interaction.”

“Polling which is repeated over time with the appropriate academic rigour, and which synthesises qualitative and quantitative results, can generate useful insights. But in the end there is no substitute for sitting down and listening to people.”

Main image: Lets Go Out Bournemouth/Flickr CC BY 2.0