The statistics crisis
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Aïda Amer/Axios
If you don't know how broken something is, you're not going to be able to fix it. That's the crisis facing policymakers trying to repair a devastated economy without knowing the true degree to which the pandemic has hurt the country.
Why it matters: Some parts of what ails America, like the nascent mental-health crisis, are by their nature hard to measure. But other aspects of the recession, like the unemployment rate or national GDP, are foundational statistics upon which multi-trillion-dollar decisions are made.
- Never in living memory have those statistics been less reliable.
How it works: The unemployment rate — the single most important statistical data point in America — is derived from a survey of a representative sample of Americans who are asked whether they worked in the past week.
- Historically, more than nine out of 10 Americans answered the questions. That ratio has been falling in recent years, but during the pandemic it has plunged to just two in three.
The survey is also long overdue for a fundamental overhaul. The last such revamp took place in 1994, before the internet transformed the nature of work.
- The catch: The Bureau of Labor Statistics has been perennially underfunded, and now employs fewer than 2,000 full-time-equivalent staffers.
- That's a decline of more than 20% from 1990s levels, even as the complexity of the workforce — and the difficulty of accurately measuring it — has increased dramatically.
Other data series are similarly troubled. The national homeownership rate increased a stunning and unprecedented 3.3 percentage points in the first half of this year, for instance — something no analyst even thought possible. That might reflect a fast-changing reality, or it could also be a statistical anomaly.
- The Census Bureau, which calculates the homeownership rate, reports a response rate of just 64.9% for its June survey, down from 82.7% a year previously. If renters were significantly less likely to respond than homeowners, that could account for all or most of the rise in the official statistic.
The intrigue: International comparisons have become more difficult than ever.
- America's GDP officially shrank by 9.5% in the second quarter, while the U.K.'s shrank more than twice as fast, at 20.4%.
- It's impossible to know whether that reflects a much more gruesome economic reality in Great Britain, or whether it, too, is a weird statistical artifact.
The bottom line: "There's nothing like a crisis to shine a light on inadequacies that need addressing," says Erica Groshen, a former BLS leader who's now at Cornell University.
- "There's a little hole in the roof that was sometimes damp, and then a storm comes through. A crisis like this reveals a lot of those things. It is stressing many of our systems, and you can see that in the statistics."
Charted: The decline of government statistics
There's something quixotic about trying to find accurate statistics that measure the decline in statistical accuracy. There's no perfect proxy, but these two charts are highly suggestive.
Why it matters: The government's official statistics provide the benchmark against which all private surveys are calibrated.
- The proliferation of private data sources, far from obviating the need for accurate government statistics, in fact makes a solid and reliable baseline more necessary than ever.
