Inside the startup market trying to capitalize on AI security

As Washington and Silicon Valley rush to mitigate AI's security risks, a new crop of entrepreneurs and investors are clamoring to monetize the latest emerging security category.

Why it matters: AI security startups are just the latest cohort trying to capitalize on the craze around generative AI and large-language models.

• And the interest in their offerings exists as AI operators and government officials host meeting after meeting to figure out how to best regulate AI before it becomes even more widespread.

The big picture: Security experts are worried about a long list of threats to AI models, including prompt injection (where users trick large-language models to go against their rules and share malicious outputs); data leaks of sensitive corporate information that the models ingest; and run-of-the-mill hacks of AI models' training data.

• The solutions AI security startups are offering either tackle a subset of these problems or try to solve all of them.
• But just like the industry's overall understanding of AI security threats, these startups are still quite early in their quest to secure artificial intelligence, Avivah Litan, distinguished VP analyst at Gartner, told Axios.

By the numbers: Investors are increasingly jumping at the chance to pour money into the next big AI security company.

• In the first three quarters of 2023, AI security startups have raised roughly $130.7 million, according to PitchBook data shared with Axios — already surpassing the $122.2 million raised in all of 2022.

Driving the news: HiddenLayer, an AI startup that emerged from stealth last year, announced a $50 million Series A funding round Tuesday led by M12 and Moore Strategic Ventures.

• The company is just the latest in a long string of startups promising to protect AI models — including CalypsoAI, Protect AI, and others — that have raised money in recent months.

Between the lines: Many of these startups are tackling AI security in a slightly different way.

• CalypsoAI focuses on auditing the sensitive data in an enterprise and preventing that data from being sucked into outside AI models. Their customer base is largely in the U.S. government, including the Defense Department and parts of the intelligence community.
• HiddenLayer provides a solution similar to endpoint security tools to review the outputs from AI models and ensure malicious actors didn't tamper with the algorithms through prompt injection or other misuse.
• Lakera AI, a security startup based in Switzerland, employs a similar idea and offers a firewall-like tool for AI model inputs and outputs to detect AI "hallucinations," prompt injections and other misuses.

The intrigue: Some of the AI security startups catching investors' eyes are attracting more demand than they originally anticipated since OpenAI's ChatGPT became available to the public.

• While HiddenLayer CEO Chris Sestito told Axios his company's approach hasn't changed, he said potential buyers have become more aware and educated about the risks that AI models pose.
• CalypsoAI raised its recent $23 million round to further fund the development of its large-language model security solutions.
• Lakera AI originally started in 2021 by securing biometrics and medical imaging algorithms, but pivoted to securing AI models at the end of 2022 due to customer demand, David Haber, founder and CEO of the company, told Axios.

Zoom out: The exit strategy for these startups is still up in the air.

• Some could sell their products to larger cybersecurity vendors, like CrowdStrike, Litan said.
• But others told Axios they see a market for AI security to become its own standalone product vertical, much in the same way that companies buy from standalone cloud security vendors.

Yes, but: Enterprises are still in the early stages of figuring out how they'll use AI internally, and until they land on an answer, they're not going to know what kinds of AI security startups to buy from, Litan said.

• Gartner estimates that the market of AI security and risk management companies will be worth $150 million by 2025, Litan said.
• "It's very much an influx market," she added. "There's definitely demand, it's just early."

Sign up for Axios' cybersecurity newsletter Codebook here.