Experts consistently underestimate U.S. oil production

A new report this week underscores a wider trend: The Energy Department's data arm and private forecasters alike have consistently underestimated the U.S. crude oil production surge in recent years.

Why it matters: There are geopolitical, economic and environmental repercussions from the faster-than-expected growth, which is driven largely by surging output in the Permian Basin region of Texas and New Mexico.

What's new: The EIA's latest monthly forecast sees U.S. crude production, already at record levels, averaging 12.4 million barrels per day this year and 13.2 million bpd in 2020. That's a substantial uptick from projections just a month earlier, and if history is any guide, the latest increase won't be the last upward revision.

The intrigue: EIA is hardly alone in low-balling the surge, which saw U.S. output climb 2 million barrels per day last year alone.

• The price recovery that began in 2016 and saw consistent gains from mid-2017 until last October has spurred things along.
• But the known unknowns of price forecasting aren't enough to explain why U.S. growth has confounded banks, the International Energy Agency, and consultancies.

By the numbers: Harry compared EIA's forecasts in their monthly short-term energy outlook with actual production data that arrives later. He found...

• U.S. crude oil production has outpaced EIA forecasts every month since January 2017. The one-year forecasts have been too low by an average of more than one million barrels.
• During the same time period, the 6-month forecasts were always revised upwards but still proved too low — by an average of about 465,000 barrels per day.

Between the lines: This faster-than-expected growth has several effects, including...

• It complicates the joint OPEC-Russia efforts to continue propping up prices through their supply-cutting pact.
• It brings economic gains to Texas and elsewhere, but the speed has also left local officials playing lots of catch-up on housing, infrastructure and other civic matters.
• It could boost global greenhouse gas emissions in the long-term if it keeps going higher and higher, based on findings from a recent study released via the think tank Resources for the Future.

What we're hearing: Analysts offered a suite of reasons why growth has exceeded forecasts, such as the decline in technology costs, which helped U.S. producers expand output at lower costs.

• "Since the end of 2016 through now, operators have really shown that they can optimize their operations and kind of get the most out of different prices far more than we have previously anticipated," Meg Coleman, a senior EIA analyst, tells Harry.

Quick take: One overarching thing is that it's just hard to get a handle on how to model the characteristics of the shale resource, given the complicated geology and the limited history of the U.S. boom.

• “It is just a dynamic resource theme, a dynamic time in the industry where so much is changing so fast. That it makes it hard for anyone to take a snapshot and say this is what informs me about the future," says Wood Mackenzie analyst R.T. Dukes.
• After all, the boom began relatively recently — in the mid-late 2000s — as advances in fracking and horizontal drilling unlocked huge resources in Texas, North Dakota and elsewhere.

Another big challenge is that the U.S. industry is very fragmented, with players of all shapes and sizes undertaking scattered developments. It's harder to analyze than big conventional projects planned far in advance, like big Gulf of Mexico projects.

The big picture: “This is hundreds of operators utilizing hundreds of service companies, accessing hundreds of markets and making their own individual decisions, and the markets have largely rewarded growth," Dukes said.

What's next: Coleman said EIA is working to improve its forecasting models. "Every month we're looking to update parameters, to find that predictor that is a leading indicator of growth," Coleman said. "We're constantly updating and improving models to capture a very very dynamic industry."