Interval measures – Part 5 of "Ew, I just stepped in a Heartland study!"

picture-8-1346574554.jpg
on

This is part 4 in a series on the Heartland Institute’s supposedly rigorous study (pdf) on the state of global warming science. This flawed paper has been distributed to 10,000 Utahns by the Utah-based Sutherland Institute, a “sister” of the Heartland Institute.

Paul T. Mero, the president of the Sutherland Institute claims that, “for skeptics, we went out of our way to include a special analysis of the methodology used to create this study. This report is an honest reflection of the international scientific community…”

Let’s see how that holds up.

Flaw #4: Heartland’s study makes incorrect assumptions about levels of agreement and disagreement

Probably the most mind-numbing courses I’ve taken are in the area of measurement theory and levels of measurement, so I’ll try to keep this short and sweet. The Heartland Institute study asks respondents to record their responses on a scale of 1 to 7 – 1 being “strongly agree” with 7 being “strongly disagree.” After the data was collected, the “researchers” then interpreted scores of 1, 2 and 3 to mean “agree,” scores of 4, 5 and 6 to mean “disagree” and a score of 4 to mean “uncertain.”

The problem here is that you are making a leap of faith that this is what your respondents meant when choosing any given number. Marking a 5 in one respondents mind might mean something very different in the minds of another respondent.

This is why a well done survey is usually uses a scale of measurement that looks something like this: “Strongly Agree — Agree — Somewhat Agree – Somewhat Disagree – Disagree — Strongly Disagree — Uncertain.” A scale like this is simple and leaves almost no interpretation of what the respondent actually means.

The Heartland Institute’s use of arbitrary measurement levels leaves its claims of agreement and disagreement up to a certain amount of interpretation – something any rigorous research tries to avoid as much as possible.

picture-8-1346574554.jpg
Kevin is a contributor and strategic adviser to DeSmog. He runs the digital marketing agency Spake Media House. Named a “Green Hero” by Rolling Stone Magazine and one of the “Top 50 Tweeters” on climate change and environment issues, Kevin has appeared in major news media outlets around the world for his work on digital campaigning. Kevin has been involved in the public policy arena in both the United States and Canada for more than a decade. For five years he was the managing editor of DeSmogBlog.com. In this role, Kevin’s research into the “climate denial industry” and the right-wing think tank networks was featured in news media articles around the world. He is most well known for his ground-breaking research into David and Charles Koch’s massive financial investments in the Republican and tea party networks. Kevin is the first person to be designated a “Certified Expert” on the political and community organizing platform NationBuilder. Prior to DeSmog, Kevin worked in various political and government roles. He was Senior Advisor to the Minister of State for Multiculturalism and a Special Assistant to the Minister of State for Asia Pacific, Foreign Affairs for the Government of Canada. Kevin also worked in various roles in the British Columbia provincial government in the Office of the Premier and the Ministry of Health. In 2008 Kevin co-founded a groundbreaking new online election tool called Vote for Environment which was later nominated for a World Summit Award in recognition of the world’s best e-Content and innovative ICT applications. Kevin moved to Washington, DC in 2010 where he worked for two years as the Director of Online Strategy for Greenpeace USA and has since returned to his hometown of Vancouver, Canada.

Related Posts

on

From South Africa to Ukraine, five industrial chicken companies that supply KFC have benefited from financing from the World Bank Group and the European Bank for Reconstruction and Development.

From South Africa to Ukraine, five industrial chicken companies that supply KFC have benefited from financing from the World Bank Group and the European Bank for Reconstruction and Development.
on

It’s an effort by the oil sands group to deflect attention away from a long record of misleading climate claims, disinformation expert argues.

It’s an effort by the oil sands group to deflect attention away from a long record of misleading climate claims, disinformation expert argues.
on

The former prime minister attacks flagship climate deals and makes false claims about electric vehicles, Russia’s influence on energy policies, and net zero.

The former prime minister attacks flagship climate deals and makes false claims about electric vehicles, Russia’s influence on energy policies, and net zero.
on

The conference featuring Nigel Farage and Suella Braverman descended into chaos as police were called.

The conference featuring Nigel Farage and Suella Braverman descended into chaos as police were called.