Rise of the new radical internet

• Now, these gaming platforms are drawing new scrutiny for harboring hate and exploitation in closed forums that remain hidden until problems spill out, often too late.

Zoom out: Mainstream social media apps focus on elevating content publicly, making them ideal for spreading ideologies, rumors or disinformation. But often of those ideas have their roots in smaller forums on gaming platforms.

• Extremist groups that have been booted from mainstream platforms have found new homes in gaming and gaming-adjacent spaces.
• Unlike public-facing apps like Instagram and TikTok, users of these apps are much more accustomed to using pseudo-identities to connect, making it easier to share radical and taboo ideas anonymously.

Zoom in: The gaming-focused architecture of these platforms is key to the proliferation of dangerous content on them, says Mariana Olaizola Rosenblat, a policy advisor on tech and law at NYU Stern.

• “Extremists and predators go to these gaming spaces to find highly-engaged, susceptible young people, many of whom are yearning for connection,” she says.

Smaller chat rooms where harmful conversations unfold are typically sealed off. “Most researchers are basically blind to all of this. You can’t enter these rooms,” Rosenblat says. Users also exploit the gaming context, using “gamespeak” to mask extreme or dangerous ideas — blurring the line between play and reality.

• The platforms themselves can access this content, but finding it is like searching for a drop in the ocean — and most haven’t invested enough in safeguards or moderation to protect young people, Rosenblat notes.

Reality check: Most conversations on these platforms are ordinary — ranging from gaming and study groups to sports fandoms and neighborhood discussion groups.

• But these same spaces have also become fertile ground for radicalization and exploitation.

That isn’t a new problem, but examples are piling up and spotlighting the issue.

• Discord: There's fresh scrutiny on the popular platform after the suspect in Charlie Kirk's murder appeared to confess in a Discord chat. Discord was also used by organizers of the 2017 Unite the Right rally in Charlottesville to spread the word and set up car pools and lodging. The shooter who killed 10 people in a Black neighborhood of Buffalo in 2022 recorded months of preparation in a Discord chat. The Daily Beast also uncovered hundreds of examples of revenge porn across discord in a 2018 investigation.
• Roblox: The platform is marketed to kids but has drawn sharp criticism for the sexual, predatory and extremist content that bubbles up on it. The company faces multiple lawsuits — including one from Louisiana alleging it failed to protect children, and another from an Iowa family after their 13-year-old daughter was kidnapped, trafficked and raped by a predator she met on the platform. "While we cannot comment on claims raised in litigation, at Roblox, we strive to hold ourselves to the highest safety standards. We invest significant resources in advanced safety technology, including a combination of machine learning and human moderation teams working 24/7 to detect and address inappropriate content and behavior," a Roblox spokesperson told Axios.

• Twitch: The aforementioned 2022 Buffalo shooting was livestreamed on the Amazon-owned gaming platform Twitch. The company very quickly condemned the attack and took down the video. It said it was working closely with law enforcement to investigate the incident.
• Steam: Researchers in 2021 found that Steam had become a place where far-right ideologies came to connect, including groups that promoted neo-Nazi organizations.

What to watch:: Gaming platforms are getting some attention in Washington after Kirk's killing.

• House Oversight Chair James Comer (R-Ky.) has asked the CEOs of Discord, Steam, Twitch and Reddit to testify before Congress about user radicalization on Oct. 8.