Sign up for our daily briefing

Make your busy days simpler with the Axios AM and PM newsletters. Catch up on what's new and why it matters in just 5 minutes.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Catch up on the day's biggest business stories

Subscribe to the Axios Closer newsletter for insights into the day’s business news and trends and why they matter.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Sign up for Axios Pro Rata

Dive into the world of dealmakers across VC, PE and M&A with Axios Pro Rata. Delivered daily to your inbox by Dan Primack and Kia Kokalitcheva.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Sports news worthy of your time

Binge on the stats and stories that drive the sports world with the Axios Sports newsletter. Sign up for free.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Tech news worthy of your time

Get our smart take on technology from the Valley and D.C. with Axios Login. Sign up for free.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Get the inside stories

Get an insider's guide to the new White House with Axios Sneak Peek. Sign up for free.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Catch up on coronavirus stories and special reports, curated by Mike Allen everyday

Catch up on coronavirus stories and special reports, curated by Mike Allen everyday

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Want a daily digest of the top Denver news?

Get a daily digest of the most important stories affecting your hometown with Axios Denver

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Want a daily digest of the top Des Moines news?

Get a daily digest of the most important stories affecting your hometown with the Axios Des Moines newsletter.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Want a daily digest of the top Twin Cities news?

Get a daily digest of the most important stories affecting your hometown with Axios Twin Cities

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Want a daily digest of the top Tampa Bay news?

Get a daily digest of the most important stories affecting your hometown with the Axios Tampa Bay newsletter.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Want a daily digest of the top Charlotte news?

Get a daily digest of the most important stories affecting your hometown with Axios Charlotte

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Want a daily digest of the top Nashville news?

Get a daily digest of the most important stories affecting your hometown with the Axios Nashville newsletter.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Want a daily digest of the top Columbus news?

Get a daily digest of the most important stories affecting your hometown with the Axios Columbus newsletter.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Want a daily digest of the top Dallas news?

Get a daily digest of the most important stories affecting your hometown with the Axios Dallas newsletter.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Want a daily digest of the top Austin news?

Get a daily digest of the most important stories affecting your hometown with the Axios Austin newsletter.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Want a daily digest of the top Atlanta news?

Get a daily digest of the most important stories affecting your hometown with the Axios Atlanta newsletter.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Want a daily digest of the top Philadelphia news?

Get a daily digest of the most important stories affecting your hometown with the Axios Philadelphia newsletter.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Want a daily digest of the top Chicago news?

Get a daily digest of the most important stories affecting your hometown with the Axios Chicago newsletter.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Sign up for Axios NW Arkansas

Stay up-to-date on the most important and interesting stories affecting NW Arkansas, authored by local reporters

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Want a daily digest of the top DC news?

Get a daily digest of the most important stories affecting your hometown with the Axios DC newsletter.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Illustration: Eniola Odetunde/Axios

In the most exclusive AI conferences and journals, AI systems are judged largely on their accuracy: How well do they stack up against human-level translation or vision or speech?

Yes, but: In the messy real world, even the most accurate programs can stumble and break. Considerations that matter little in the lab, like reliability or computing and environmental costs, are huge hurdles for businesses.

Why it matters: Some stumbles that have tarred high-profile AI systems — like facial recognition that fails more often on darker faces, or medical AI that gives potentially harmful advice — have resulted from unanticipated real-world scenarios.

The big picture: Research often guns for incremental advances, juicing an extra percentage point of accuracy out of a previously proposed model.

  • "We feel like this is the only thing that gets rewarded nowadays in the community," says Roy Schwartz, a researcher at the Allen Institute for Artificial Intelligence. "We want to argue that there are other things to consider."
  • In a paper posted this summer, Schwartz and several co-authors proposed that researchers present not just the accuracy of their models but also the computing cost it took to get there — and the resulting environmental toll.

That's one of several hidden considerations that can drive up the cost of creating an AI model that works in the real world. Among the other important factors accuracy doesn't capture:

  • The cost of labeling training data, which is still mostly done by humans.
  • The reliability of a system or, conversely, its tendency to make critical mistakes when it sees a new situation it hasn't been trained to deal with.
  • Vulnerability to adversarial examples, a special kind of attack that can cripple certain kinds of AI systems, potentially making an autonomous car blind to a fast-approaching stop sign.
  • Bias, or whether a model's accuracy varies depending on the kind of person or thing it is evaluating.
  • Interpretability of a model's results, or how easy it is to understand why an AI system made a particular prediction.

"The machine learning model is just a tiny piece of the machine learning product," says Andrew Ng, founder of Landing.ai, a startup that helps companies set up AI processes.

  • "One of the challenges is that you ship a system and then the world changes in some weird and unpredictable way," says Ng, who previously started Google's and Baidu's AI operations.
  • "Anyone can download code from GitHub, but that's not the point," Ng tells Axios. "You need all these other things."

One of the byproducts of the hidden costs associated with increasingly accurate AI systems is that they can make it hard for a new startup or cash-strapped university lab to compete.

  • "The AI models that are being designed and developed don't always contemplate the application of the model," says Josh Elliot, director of AI at Booz Allen Hamilton.
  • Big companies offer pre-packaged AI programs — like one for detecting street signs, say — but users often lack the computing firepower required to tweak them.
  • That means the reigning players get to decide the important problems worth trying to solve with AI, Schwartz says, and everyone else has to go along.

What's next: Schwartz, Ng and others propose putting more effort toward solving problems more efficiently rather than making a slightly more accurate model at an enormous cost.

Go deeper: How not to replace humans

Go deeper

FAA clears more planes after 5G fears

Photo: David McNew/Getty Images

The Federal Aviation Administration said Thursday it had approved nearly 80% of the U.S. commercial fleet to perform low-visibility landings at airports with new 5G services after fears of signal interference limited 5G rollout.

Why it matters: The FAA approvals will help provide more certainty after the agency raised fears that 5G signals could reduce the accuracy of certain equipment, known as radio altimeters, that helps planes land and take off in inclement weather.

Dan Primack, author of Pro Rata
3 hours ago - Economy & Business

Peloton stock tanks on report of production halt

Illustration: Aïda Amer/Axios

Peloton stock fell by as much as 25% on Thursday, following a CNBC report that the connected fitness company will temporarily halt production on its bikes and treadmills.

Why it matters: Peloton is viewed by many as a proxy for consumer behavior in the pandemic era, as its popularity surged when gyms closed and people wanted to exercise at home.

Updated 3 hours ago - Politics & Policy

Omicron dashboard

Illustration: Brendan Lynch/Axios

  1. Health: Omicron pushes COVID deaths toward 2,000 per day — The pandemic-proof health care giant.
  2. Vaccines: No evidence that healthy children, teens need boosters, WHO says — Starbucks drops worker vaccine or test requirement after SCOTUS ruling — Kids' COVID vaccination rates are particularly low in rural America.
  3. Politics: Biden concedes U.S. should have done more testing — Arizona says it "will not be intimidated" by Biden on anti-mask school policies.
  4. World: American Airlines flight to London forced to turn around over mask dispute — WHO: COVID health emergency could end this year — Greece imposes vaccine mandate for people 60 and older.
  5. Variant tracker