DeepSeek's models are easier to manipulate than U.S. counterparts, research finds
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Natalie Peeples/Axios
A flurry of security research reports this week suggest DeepSeek's open-source AI models could be more susceptible to cyberattacks than previously thought.
Why it matters: Cybersecurity and national security experts are already on high alert over potential security problems with the China-based AI startup — including potential model leaks and cyberattacks.
Driving the news: Security researchers at cloud security startup Wiz identified an exposed DeepSeek database that left chat histories, secret keys, backend details and other sensitive information exposed online, according to a report released Wednesday.
- Palo Alto Networks' threat researchers also released a report Thursday finding that it was fairly easy to get DeepSeek to break its own guardrails and provide tips for writing code to help hackers exfiltrate data, send phishing emails and optimize social engineering attacks.
- Researchers at security firm Enkrypt AI published a report Friday morning finding that DeepSeek's new R1 reasoning model is four times more likely to write malware and other insecure code than OpenAI's o1.
The big picture: U.S. interest in DeepSeek skyrocketed in the last week after the AI startup released its R1 model, a reasoning model that rivals OpenAI's capabilities at a fraction of the investment cost.
- The overnight obsession sparked a fresh new round of scrutiny into DeepSeek's data privacy and content moderation policies.
Zoom in: Wiz's security researchers found the exposed database of chat logs and other sensitive information within minutes of beginning their investigation, per their report.
- The database had more than 1 million lines of activity log streams that included the exposed chat histories and other sensitive information.
- The database also gave anyone who came across it full database control and potential privilege escalation capabilities without needing to authenticate a user's identity and whether they were allowed to see this database.
- DeepSeek fixed the exposure before Wiz released its findings.
Meanwhile, researchers at Palo Alto Networks' Unit 42 research unit used basic jailbreaking techniques to get DeepSeek's R1 model to help them craft phishing emails, write malware and even provide comprehensive instructions for constructing a Molotov cocktail.
- Enkrypt AI's researchers found that DeepSeek was "highly susceptible" to prompt injections, where hackers socially engineer malicious prompts to look like legitimate requests.
- Researchers could also get DeepSeek to produce harmful content in 45% of their tests. In one test, DeepSeek wrote a blog that detailed ways that terrorist organizations could recruit new members, Enkrypt AI found.
- DeepSeek did not respond to a request for comment.
Reality check: Even U.S. models are susceptible to jailbreaking, but researchers note that it's gotten harder for them to use these techniques to trick ChatGPT, Anthropic's Claude and others.
Between the lines: The findings each highlight the faults AI models can have if companies' don't conduct proper security and safety checks before release.
- "While DeepSeek R1 may be viable for narrowly scoped applications, robust safeguards — including guardrails and continuous monitoring — are essential to prevent harmful misuse," Enkrypt AI CEO Sahil Agarwal said in a statement. "AI safety must evolve alongside innovation, not as an afterthought."
What we're watching: It remains to be seen how long the U.S. obsession with DeepSeek will last — and whether there will be a major U.S. policy backlash to companies and employees using the China-based startup's app.
