Illustration: Aïda Amer & Eniola Odetunde/Axios

Scientists have long tried to use AI to automatically detect hate speech, which is a huge problem for social network users. And they're getting better at it, despite the difficulty of the task.

What's new: A project from UC Santa Barbara and Intel takes a big step further — it proposes a way to automate responses to online vitriol.

  • The researchers cite a widely held belief that counterspeech is a better antidote to hate than censorship.
  • Their ultimate vision is a bot that steps in when someone has crossed the line, reining them in and potentially sparing the target.

The big picture: Automated text generation is a buzzy frontier of the science of speech and language. In recent years, huge advances have elevated these programs from error-prone autocomplete tools to super-convincing — though sometimes still transparently robotic — authors.

How it works: To build a good hate speech detector, you need some actual hate speech. So the researchers turned to Reddit and Gab, two social networks with little to no policing and a reputation for rancor.

  • For maximum bile, they went straight for the "whiniest most low-key toxic subreddits," as curated by Vice. They grabbed about 5,000 conversations from those forums, plus 12,000 from Gab.
  • They passed the threads to workers on Amazon Mechanical Turk, a crowdsourcing platform, who were asked to identify hate speech in the conversations and write short interventions to defuse the hateful messages.
  • The researchers trained several kinds of AI text generators on these conversations and responses, priming them to write responses to toxic comments.

The results: Some of the computer-generated responses could easily pass as human written — like, "Use of the c-word is unacceptable in our discourse as it demeans and insults women" or "Please do not use derogatory language for intellectual disabilities."

  • But the replies were inconsistent, and some were incomprehensible: "If you don't agree with you, there's no need to resort to name calling."
  • When Mechanical Turk workers were asked to evaluate the output, they preferred human-written responses more than two-thirds of time.

Our take: This project didn't test how effective the responses were in stemming hate speech — just how successful other people thought it might be.

  • Even the most rational, empathetic response, not to mention the somewhat robotic computer-generated ones above, could flop or even backfire — especially if Reddit trolls knew they were being policed by bots.

"We believe that bots will need to declare their identities to humans at the beginning," says William Wang, a UCSB computer scientist and paper co-author. "However, there is more research needed how exactly the intervention will happen in human-computer interaction."

Go deeper

Virginia Gov. Ralph Northam tests positive for coronavirus

Photo: Zach Gibson/Getty Images

Virginia Gov. Ralph Northam (D) and his wife, Pamela, both tested positive for coronavirus, his office announced on Friday.

The state of play: The Northams were tested after one of their staff "who works closely within the couple's living quarters" tested positive. The governor is asymptomatic, while his wife is "experiencing mild symptoms." They plan to isolate at home for 10 days.

Ina Fried, author of Login
38 mins ago - Technology

Amazon wants to flood America with Alexa cameras and microphones

Photo: Amazon

In a Thursday event unveiling a slew of new home devices ahead of the holidays, Amazon made clearer than ever its determination to flood America with cameras, microphones and the voice of Alexa, its AI assistant.

The big picture: Updating popular products and expanding its range to car alarms and in-home drones, Amazon extended its lead in smart home devices and moved into new areas including cloud gaming and car security. The new offerings will also fuel criticism that the tech giant is helping equip a society built around surveillance.

Ben Geman, author of Generate
2 hours ago - Energy & Environment

Oil's turbulent long-term future

Illustration: Aïda Amer/Axios

The oil sector is facing risks from all sides.

Why it matters: Risk in the industry is nothing new. But these are especially turbulent and uncertain times. The industry's market clout has waned, the future of demand is kind of a mystery, and future U.S. policy is too, just to name three.

Get Axios AM in your inbox

Catch up on coronavirus stories and special reports, curated by Mike Allen everyday

Please enter a valid email.

Subscription failed
Thank you for subscribing!