Flaws in Environmental Defense Fund's Methane Study Draw Criticism from Scientists

1-DSC09675
on

Perhaps the single most consequential and controversial issue at the center of the onshore natural gas drilling boom is the question of methane leaks. Natural gas is primarily made of methane, a powerful greenhouse gas, and if enough escapes into the atmosphere, these leaks could potentially make natural gas a worse fuel for the climate thanย coal.

In mid-September, researchers from the University of Texas published a study that was hailed by a triumphant oil and gas industry, which claimed it definitively showed that methane leaks from fracking are minimal. Major news outlets largely fed this excitement, proclaiming that the study showed EPA had dramatically overestimated methane leaks from the drillingย boom.

But as the celebrations died down and more sober and rigorous analysis of the study has begun, scientists are finding that the University of Texas study is riddled with flaws.

The backers of the report cherry-picked the oil and gas wells included in the study, selecting smaller wells that had less capacity to leak and ones that used leak controls that are not currently used at many of the nationโ€™s wells. The authors systematically ignored more recent federal research indicating that as much as 17 percent of natural gas โ€“ more than 10 times the estimate indicated by the UT study โ€“ leaks from gas fields, and overlooked serious methodological flaws that were pointed out in similar studies dating back as far asย 1996.

As scientists have raised these concerns, the Environmental Defense Fund, one backer of the study which was 90 percent funded by the oil and gas industry, have tried to tamp down some of the media excitement surrounding the result and said that their research was misrepresented.

For example, the Wall Street Journal headlined its coverage of the report: โ€œU.S. Overstates Leaks by Gas-Drillers, Says Study.โ€ But some at EDF have said The Journal got it flatย wrong.

โ€œNo. The WSJ headline is not an accurate reflection of what the study found,โ€ wrote EDFโ€™s Sam Parry in an online discussion. โ€œWhat the study found was that when you use green completion technologies, you can mitigate the problem of methane leakage. If you don’t use these technologies, the problem is real and veryย serious.โ€

Despite Mr. Parryโ€™s allegation, no retractions or corrections have yet been made to the mainstream press coverage of theย study.

EDFโ€™s complaints that the study was misreported are finding limited sympathy from some in the research community, who say that EDF officials have allowed their study to be misrepresented in the public eye by failing to mount an effective response to counter the inaccurate press coverage. EDF, they say, actually made things worse by promising that their 16-part research series, of which the UT study is the first, would provide the exclusive and definitive answer on methaneย leaks.

โ€œIt is very bad form to allege that the contributions of others are mere โ€˜snapshotsโ€™, and then offer up a very few snapshots of your own as somehow โ€˜definitive,โ€™โ€ said Prof. Anthony Ingraffea, one of the foremost experts on methane and climate change, and co-author of a seminal paper that first called into question the claimed climate benefits of natural gas. โ€œIt is for other stakeholders to judge the value of this research project, not the scientists and engineers performingย it.โ€

The University of Texas studyโ€™s many flaws have cast further doubt on the claim that EDFโ€™s series will provide a definitive โ€“ or even reliable โ€“ assessment of natural gasโ€™s climateย impacts.

The stakes are enormous. Over the next two decades, the greenhouse effect from methane that leaks today will be up to 86 times as powerful as todayโ€™s carbon dioxide emissions. The single largest source of methane emissions in America is the natural gas industry โ€“ and right now that industry is experiencing a gold rush that has created a vast new network of wells, pipelines, and storage facilities across the U.S. โ€“ all holding methane, the main ingredient in naturalย gas.

If natural gas leaks at more than 3.2 percent overall, any climate change benefit gained by switching away from coal for electricity generation and burning natural gas instead will be lost, although that critical threshold may soon be lowered, as the Intergovernmental Panel on Climate Change recently concluded that methane is even more potent than previouslyย believed.

The UT study had a single narrow aim: to measure the amount of methane that leaked during certain stages of drilling and fracking a natural gasย well.

A team of researchers descended upon 190 oil and gas wells owned by 9 major oil and gas companies, studying 27 hydraulic fracturing events and several other parts of the completion process. They used this data to estimate that on average, 0.42 percent of natural gas from a gas well could be expected to leak out at the well pad during a stage called โ€œcompletingโ€ the well. Thatโ€™s consistent with EPAโ€™s current official but controversial estimate that 1.5 percent of natural gas leaks overall, after accounting for leaks from places like pipelines (the agencyโ€™s own Inspector General called for more data and a closer review of thatย figure).

Some of the earliest criticism of the UT study focused on a major internal contradiction within the study itself. The well sites were selected with substantial input from the oil and gas industry, which volunteered specific sites, and the vast majority of the wells studied used leak-control technology that has yet to be adopted at many, if not most, oil and gas wells, while others were wells that produced very little gas and consequently even serious leaks would produce relatively small emissions โ€“ specifically, the authors noted, those wells had the potential to emit only 0.55% as much as an average well.

Although the studyโ€™s authors acknowledged that their measurements were by no means representative of the average gas well nationwide, they nonetheless chose to use that skewed data to estimate gas leaksย nationwide.

The methodology that UT chose for making that estimate also has drawn heavy fire from others in the research community. The study, and its principal investigator David Allen, chose to use the same method that EPA used in its 1996 national inventory of methane emissions โ€“ and came to a nearly identical conclusion.

That 1996 methodology has been widely criticized, in part because it leaves a large margin of error. And with so much on the line, a large margin of error can obscure vital information. At its high end, the UT estimate of 1157 Gigagrams is roughly 1.5 times its low-end estimate, 757ย Gigagrams.

The 1996 methodology uses an unreliably small percentage of wells to make sweeping predictions, researchers have complained. Like the 1996 study, which tested 0.14 percent of wells nationwide, the UT study relied on a tiny sample set of 0.11% of wells nationwide โ€“ in other words, less than 500 out of a half millionย wells.

โ€œThis is a grossly inadequate sampling, given that the United States is now home to more than half a million active natural gas wells โ€” and more and more every day; 25,000 new wells were drilled last year alone,โ€ย wroteย Dr. Alison Stine, a research fellow at Ohio University.

As in the 1996 study, key uncertainties were left unaccounted for. The UT authors wrote that they simply ignored uncertainties surrounding โ€œnational counts of wells or equipment and the issue of whether the companies that provided sampling sites are representative of the nationalย population.โ€

โ€œUnfortunately, these are exactly the most important parameters on which to base a truly representative, nation-wide assessment,โ€ Physicians Scientists & Engineers for Healthy Energy wrote in their assessment of the UT research.

The wells tested by UT belonged to major oil and gas companies like Shell and Chevron โ€“ and those same companies have long argued that they have a better environmental track record than the littleย guys.

In addition, the UT study leaves out key information, researchers charge. The paper itself does not even indicate what type of formation was being drilled at each site โ€“ meaning that itโ€™s not clear whether these wells are shale gas wells, which have been at the heart of the onshore drilling boom, or other types of fracked wells, like coal bed methane or tight sand โ€“ and industry data shows that there can be a 10-fold difference between the varying types ofย wells.

Most importantly, the UT research completely ignores conflicting estimates in peer-reviewed literature. A string of studies have found vastly higher leaks than UTโ€™s estimated 0.42 percent leak rate would support. Field testing by the National Oceanic and Atmospheric Administration reported cumulative leaks as high as 12 percent of production in Colorado and 17 percent in the Los Angeles basin.

No effort to address these findings or to explain the discrepancy between the differing estimates was made by the UTย researchers.

โ€œIt is disappointing that Allen and colleagues seem to have failed to employ basic scientific rules including transparent criteria for the selection of study sites to measure, sufficient sample sizes, and the attempt to place their results in the context of other scientific studies to date,โ€ said Seth B. Shonkoff, Executive Director of Physicians Scientists & Engineers for Healthy Energy.

โ€œThis study falls short in its attempt to help answer questions about methane emissions from modern gas development beyond the small number of gas industry-selected wells where measures wereย taken.โ€

For its part, EDF has argued that the study does not prove that only small amounts of natural gas are currently leaking, especially because the wells studied nearly all used the leak control technology, called โ€œgreen completions,โ€ saying that the study merely shows the industry can control 99 percent of methane emissions during the completion stage โ€“ not that it does control them.

They argue that many wells will be required to adopt the technique in 2015 when new regulations go into effect, and so understanding its effectiveness isย important.

Others have said that something doesnโ€™t add up about the fact that the 1996 study and the UT study happen to reach nearly identical conclusions โ€“ despite intervening research that shows the true number is likely much higher and the fact that there was no large-scale shale gas development in 1996. They charge that the coincidence could be because both suffer from the sameย flaws.

โ€œTo allege that emissions have not changed since 1996 is absurd for at least two reasons,โ€ said Prof. Ingraffea.

โ€œEDF and industry cannot say, as they are now boasting, that new technologies have led to reduced emissions and then say they are the same now as in 1996; and there was no large scale commercial shale gas development in 1996, and the present issue and their study is all about shale gasย development.โ€

Photo credit: Detail of an oil-refinery plant, viaย Shutterstock.

1-DSC09675
Sharon Kelly is an attorney and investigative reporter based in Pennsylvania. She was previously a senior correspondent at The Capitol Forum and, prior to that, she reported for The New York Times, The Guardian, The Nation, Earth Island Journal, and a variety of other print and online publications.

Related Posts

The EU and many member states have set limits for how much manure farmers are allowed to apply in their fields, but crucial oversight is lacking.

The EU and many member states have set limits for how much manure farmers are allowed to apply in their fields, but crucial oversight is lacking.
on

Robert Wilkie was speaking at a conference co-hosted by the group behind the radical Project 2025 agenda.

Robert Wilkie was speaking at a conference co-hosted by the group behind the radical Project 2025 agenda.
on

Scope of corporate influence underscores concerns the technology will be used to prolong demand for planet-heating natural gas.

Scope of corporate influence underscores concerns the technology will be used to prolong demand for planet-heating natural gas.
on

A 1961 oil and gas well is the suspected source of a geyser eruption in the region where Permian wastewater disposal is causing a flurry of earthquakes.

A 1961 oil and gas well is the suspected source of a geyser eruption in the region where Permian wastewater disposal is causing a flurry of earthquakes.