Energy markets have gotten more volatile and harder to predict

Oil, gas and electricity markets in the U.S. showed more yearly volatility and proved tougher for Energy Department analysts to predict from 2005 to 2014 compared to preceding decades, new research published in the journal Nature Energy shows.

Why it matters: The results underscore the challenges facing policymakers, and companies making investment decisions that "influence the cost, and environmental and health impacts of the US energy system for decades," the paper states.

• "Understanding historical changes in the projected and actual values of key energy quantities can help decision-makers create robust strategies for a deeply uncertain future," write the three Carnegie Mellon University authors.
• "This turbulence may or may not continue. However, this analysis should serve as a stark reminder of the importance of considering the possibility of further surprises when planning for the future," they conclude.

What they did: The paper explores decades of U.S. data on prices, production and use of oil, gas and coal; electricity prices and demand; transportation fuels and more, totaling 17 separate categories.

• Their analyses looked at two types of "extreme" changes in these categories. One was the largest cases of year-over-year volatility. The other was the biggest misses in annual Energy Information Administration long-term outlooks — in which the errors were outside of the 95 percent range of prior errors for a given topic.

What they found: "Both volatility and unpredictability have increased in the past decade, compared to the three and two decades before it."

• In terms of volatility, data since the mid-20th century shows there were nine extreme changes between 2005 and 2014, compared to just seven from 1975 to 2004.
• On the unpredictability front, 2005-2014 showed major forecasting errors that included over-projections of coal production; under-projections of oil production toward the end of the study period; and over-projection of total electricity sales.

To be sure: Some of the findings can be traced to a single, if broad, seismic change — the oil and gas fracking boom over the last decade. It has, for instance, led to much more production than forecast as well as much lower natural gas prices. And the 2007-2008 financial crisis also shook things up.

• But that's not enough to explain the results. "[H]igh concentrations of extreme errors begin before both the massive expansion of hydraulic fracturing and the great recession," the authors write.

Their bottom line: "The observed increase in the volatility and unpredictability of key energy-related quantities may suggest complex structural shifts in the U.S. and world economies and energy systems."

The paper's authors are Inês Azevedo, a professor of engineering and public policy; Evan Sherwin, a grad student in that program; and Max Henrion, the CEO of Lumina Decision Systems who is also an adjunct professor at Carnegie Mellon.