OpenAI finds new Chinese influence campaigns using its tools
Add Axios as your preferred source to
see more of our stories on Google.
A Spanish-language article promoting anti-U.S. opinions generated by ChatGPT as part of a suspected Chinese foreign influence campaign. Image: OpenAI
OpenAI spotted and disrupted two uses of its AI tools as part of broader Chinese influence campaigns, including one designed to spread Spanish-language anti-American disinformation, the company said.
Why it matters: AI's potential to supercharge disinformation and speed the work of nation state-backed cyberattacks is steadily moving from scary theory to complex reality.
Driving the news: OpenAI published its latest threat report on Friday, identifying several examples of efforts to misuse ChatGPT and its other tools.
- One campaign, which OpenAI labeled "sponsored discontent," used ChatGPT accounts to generate both English-language comments attacking Chinese dissident Cai Xia and Spanish-language news articles critical of the U.S.
- Some of the short comments were posted on X, while the articles found their way into a variety of Latin American news sites, in some cases as sponsored content.
What they're saying: "As far as we know this is the first time a Chinese influence operation has been found translating long-form articles into Spanish and publishing them in Latin America," Ben Nimmo, principal investigator on OpenAI's intelligence and investigations team, said in a briefing with reporters.
- "Without our view of their use of AI, we would not have been able to make the connection between the tweets and web articles."
Another campaign, which OpenAI dubbed "peer review," consisted of accounts using ChatGPT to generate marketing materials for a social media listening tool that its creators claimed had been used to send reports of protests to the Chinese security services.
- OpenAI banned the related accounts, saying they violated company policies that "prohibit the use of AI for communications surveillance, or unauthorized monitoring of individuals."
- Other campaigns called out in the latest report include several scams as well as influence campaigns tied to North Korea and Iran and an effort to influence an election in Ghana.
Between the lines: OpenAI, which started publishing threat reports last year, says that it's doing so "to inform efforts to understand and prepare for how the P.R.C. or other authoritarian regimes may try to leverage AI against the U.S. and allied countries, as well as their own people."
- As the new report shows, AI tools can be used at various points in a disinformation campaign, sometimes revealing other aspects of a group's techniques, aims and weaknesses.
- "Threat actors sometimes give us a glimpse of what they are doing in other parts of the internet because of the way they use our models," Nimmo said.
Yes, but: As open source tools become more powerful — and are able to be run locally — threat actors may use them for more of their tasks, making it harder for such efforts to be detected.
- In the "peer review" case, for example, OpenAI noticed that while ChatGPT was used to edit and debug some code, there were also references to the use of open-source models, including DeepSeek and a version of Meta's Llama 3.1.
"This was a really interesting case where it looks like a threat actor at least mentions the use of a bunch of different models," Nimmo said, noting it's not clear what motivated the use of so many tools.
- "Maybe they wanted to break up their signal," he said. "There's a bunch of different reasons that some of this could be going on."
The bottom line: As AI continues to ratchet up attackers' capabilities, AI providers are having to put more effort into tracking and foiling them — often with the help of their own tools.
