Oct 19, 2019

Fighting hate with AI-powered retorts

Illustration: Aïda Amer & Eniola Odetunde/Axios

Scientists have long tried to use AI to automatically detect hate speech, which is a huge problem for social network users. And they're getting better at it, despite the difficulty of the task.

What's new: A project from UC Santa Barbara and Intel takes a big step further — it proposes a way to automate responses to online vitriol.

  • The researchers cite a widely held belief that counterspeech is a better antidote to hate than censorship.
  • Their ultimate vision is a bot that steps in when someone has crossed the line, reining them in and potentially sparing the target.

The big picture: Automated text generation is a buzzy frontier of the science of speech and language. In recent years, huge advances have elevated these programs from error-prone autocomplete tools to super-convincing — though sometimes still transparently robotic — authors.

How it works: To build a good hate speech detector, you need some actual hate speech. So the researchers turned to Reddit and Gab, two social networks with little to no policing and a reputation for rancor.

  • For maximum bile, they went straight for the "whiniest most low-key toxic subreddits," as curated by Vice. They grabbed about 5,000 conversations from those forums, plus 12,000 from Gab.
  • They passed the threads to workers on Amazon Mechanical Turk, a crowdsourcing platform, who were asked to identify hate speech in the conversations and write short interventions to defuse the hateful messages.
  • The researchers trained several kinds of AI text generators on these conversations and responses, priming them to write responses to toxic comments.

The results: Some of the computer-generated responses could easily pass as human written — like, "Use of the c-word is unacceptable in our discourse as it demeans and insults women" or "Please do not use derogatory language for intellectual disabilities."

  • But the replies were inconsistent, and some were incomprehensible: "If you don't agree with you, there's no need to resort to name calling."
  • When Mechanical Turk workers were asked to evaluate the output, they preferred human-written responses more than two-thirds of time.

Our take: This project didn't test how effective the responses were in stemming hate speech — just how successful other people thought it might be.

  • Even the most rational, empathetic response, not to mention the somewhat robotic computer-generated ones above, could flop or even backfire — especially if Reddit trolls knew they were being policed by bots.

"We believe that bots will need to declare their identities to humans at the beginning," says William Wang, a UCSB computer scientist and paper co-author. "However, there is more research needed how exactly the intervention will happen in human-computer interaction."

Go deeper

Updated 14 mins ago - World

In photos: People around the world rally against racism

Despite a ban on large gatherings implemented in response to the coronavirus pandemic, protesters rally against racism in front of the American Embassy in Paris on June 6. Photo: Julien Mattia/Anadolu Agency via Getty Images

Tens of thousands of people have continued to rally in cities across the world against racism and show their support this week for U.S. demonstrators protesting the death in police custody of George Floyd.

Why it matters: The tense situation in the U.S. has brought the discussion of racism and discrimination onto the global stage at a time when most of the world is consumed by the novel coronavirus.

George Floyd updates

Protesters in Washington, D.C. on June 6. Photo: Samuel Corum/Getty Images

Thousands of demonstrators are gathering in cities across the U.S. and around the world to protest the killing of George Floyd. Huge crowds have assembled in Washington, D.C., Philadelphia and Chicago for full-day events.

Why it matters: Twelve days of nationwide protest in the U.S. has built pressure for states to make new changes on what kind of force law enforcement can use on civilians and prompted officials to review police conduct.

Updated 1 hour ago - Politics & Policy

Coronavirus dashboard

Illustration: Sarah Grillo/Axios

  1. Global: Total confirmed cases as of 7:30 p.m. ET: 6,852,810 — Total deaths: 398,211 — Total recoveries — 3,071,142Map.
  2. U.S.: Total confirmed cases as of 7:30 p.m. ET: 1,917,080 — Total deaths: 109,702 — Total recoveries: 500,849 — Total tested: 19,778,873Map.
  3. Public health: Why the pandemic is hitting minorities harder — Coronavirus curve rises in FloridaHow racism threatens the response to the pandemic Some people are drinking and inhaling cleaning products in attempt to fight the virus.
  4. Tech: The pandemic is accelerating next-generation disease diagnostics — Robotics looks to copy software-as-a-service model.
  5. Business: Budgets busted by coronavirus make it harder for cities to address inequality Sports, film production in California to resume June 12 after 3-month hiatus.
  6. Education: Students and teachers flunked remote learning.