Sixth graders at Flint Hill School in Oakton, Va. Photo: Katherine Frey/Washington Post/Getty

Jolted by rising suicide and self-injury among young Americans, schools are using software to monitor student browsing history and logs for signs of distress, in what they hope will curb the problem.

What's going on: Kids and teens increasingly rely on the internet to answer their mental-health questions, creating browsing patterns that schools hope could identify potential harm before it happens. But that requires sweeping online surveillance that critics say could leave a lasting mark on students.

The context: Between 2006 and 2016, the suicide rate for white children rose 70%, to more than five per 100,000 children, according to the CDC. The rate for black children increased 77%, to 2.62 suicides per 100,000 children.

The details: Many monitoring tools are on the market, including one by GoGuardian, a Los Angeles-based education-tech company.

  • GoGuardian's early filtering products were aimed at keeping students away from online porn, Tyler Shaddix, the company's chief product officer, tells Axios.
  • After a student committed suicide at a school testing the software, an administrator asked whether better monitoring could have prevented it.
  • That led to a new system that evaluated students’ online activity for signs they may harm themselves.

At first, GoGuardian used keyword matching, a standard approach among companies like it. This generated thousands of extraneous alerts, so in its latest version — due out in October — GoGuardian has added artificial intelligence techniques that it says improves the software's accuracy.

How it works: GoGuardian installs an extension on school-issued computers running Chrome OS or Windows. When the system detects a potential problem in chat conversations, emails, Google Docs, or browsing, it sends an alert to whomever the school has designated to receive them.

  • With the help of mental-health experts, the company trained a machine-learning algorithm so it flags content most closely associated with potential harm, like searches for suicide methods.
  • The system now sends a much more manageable 2,000 alerts a week across 5 million students, within seconds of being triggered.
  • A human makes the final call, reading over the alert — including five minutes of browsing history before and after the incident — to decide whether or not to act.

The software can stumble either by sending sending schools unneeded messages, drowning out real flags, or by missing important signs and failing to send a critical alert.

  • GoGuardian CEO Advait Shinde says the system errs on the side of alerting more often, but the company refused to share stats about its software's failure rate.
  • Gennie Gebhart, a researcher at EFF and co-author of a 2017 report on student privacy, said companies that help monitor students are not transparent enough about the accuracy of their software.

Electronic snooping always raises weighty ethical questions, even when it’s meant to safeguard young people.

  • Privacy experts worry about early, all-encompassing monitoring. Gebhart said pervasive surveillance normalizes electronic snooping, and can keep kids from testing out new ideas and identities as they grow.
  • "This piece of software is going beyond ed-tech and toward a new kind of surveillance in the classroom," Gebhart said.
  • When I asked GoGuardian if students have a right to privacy, Shaddix said the benefits of monitoring "outweigh the negatives."

Go deeper: Wired reports on schools that monitor students’ social media feeds for threatening posts or cyberbullying.

Go deeper

Felix Salmon, author of Capital
18 mins ago - Economy & Business

Wall Street: Recession is over

Illustration: Aïda Amer/Axios

U.S. economic activity fell more sharply in the second quarter of this year than during any other quarter in history. It's also going to grow more sharply in the third quarter of this year than during any other quarter in history.

  • The recession is over, according to Wall Street, with current forecasts showing sustained economic growth through 2021 and beyond.
27 mins ago - Health

If Trump replaces Ginsburg, the ACA really is at risk

Illustration: Aïda Amer/Axios

The Supreme Court vacancy created by Ruth Bader Ginsburg’s death leaves the Affordable Care Act in much greater jeopardy than it was just a few days ago.

The big picture: Conventional wisdom had held that Chief Justice John Roberts would likely join with the court’s liberals to save the ACA once again. But if President Trump is able to fill Ginsburg’s former seat, Roberts’ vote alone wouldn’t be enough to do the trick, and the law — or big sections of it — is more likely to be struck down.

Ben Geman, author of Generate
32 mins ago - Energy & Environment

Nikola founder steps aside amid allegations of fraud

Illustration: Lazaro Gamio/Axios

Nikola announced Monday that executive chairman Trevor Milton, who is also the company's founder, is out as the electric and fuel cell truck startup reportedly faces federal inquiries into a short-seller's allegations of inaccurate or misleading statements.

Why it matters: It's the latest move in a head-spinning series of events for Nikola.