Illustration: Sarah Grillo/Axios

New research offers strategies to prevent algorithms used in business from pushing unethical policies.

Why it matters: Machine-learning algorithms are increasingly being deployed in commercial settings. If they are optimized only to seek maximum revenue, they can end up treating customers in unethical ways, putting companies at reputational or even regulatory risk.

How it works: In a paper published on Wednesday in Royal Society Open Science, researchers formulated what they call the "Unethical Optimization Principle."

  • It essentially boils down to the idea that "if there is an advantage to something that will be perceived as unethical, then it is quite likely the machine learning is going to find it," says Robert MacKay, a mathematician at the University of Warwick and an author of the paper.
  • MacKay uses the example of an algorithm that prices insurance products. If it is optimized only to maximize revenue, it's likely to treat customers unfairly and even unlawfully, selecting a higher price for users whose names code as non-white.
  • In their paper, MacKay and his colleagues lay out complex mathematics that can help businesses and regulators detect the unethical strategies an algorithm might pursue in a given space and identify how the AI should be modified to prevent that behavior.

The big picture: As increasingly sophisticated algorithms take more decisions out of the hand of humans, it becomes even more important for programmers to set initial clear limits.

  • Unfortunately, as a new survey from the data science platform Anaconda demonstrates, while data scientists are increasingly concerned about the ethical implications of their work, 39% of those polled say their team has no plans in place to address fairness or bias.
  • "Businesses using algorithms need to ask questions of 'ought,' rather than just 'can,'" says Peter Wang, Anaconda's CEO.

Go deeper

Updated 2 hours ago - Politics & Policy

Coronavirus dashboard

Illustration: Aïda Amer/Axios

  1. Politics: Pence chief of staff Marc Short tests positive for coronavirus — COVID-19 looms over White House Halloween celebrations
  2. Health: Fauci says maybe we should mandate masks if people don't wear them — America was sick well before it ever got COVID-19
  3. World: Polish President Andrzej Duda tests positive for COVID-19.

Pence chief of staff Marc Short tests positive for coronavirus

Marc Short with Katie Miller, Vice President Pence's communications director, in March. Photo: Doug Mills/The New York Times via Reuters

Marc Short, Vice President Mike Pence’s chief of staff, tested positive for the coronavirus Saturday and is quarantining, according to a White House statement.

Why it matters: Short is Pence's closest aide, and was one of the most powerful forces on the White House coronavirus task force.

8 hours ago - World

Opposition leader Leopoldo López flees Venezuela

Venezuelan opposition politician Leopoldo López outside the Spanish embassy in Caracas, in 2019. Photo: Juan Barreto/AFP via Getty Images

Leopoldo López, a former political prisoner and prominent Venezuelan opposition leader, has left the country, his Popular Will party confirmed in a statement Saturday.

Why it matters: He's been highly influential in the push to oust President Nicolás Maduro's regime and a mentor to opposition leader Juan Guaidó. He'd been in the Spanish ambassador's Caracas residence since escaping house arrest in April 2019 following a failed military uprising.