1 big thing: The burgeoning startup market capitalizing on AI security

Illustration of a briefcase with a binary code combination lock with ones and and zeroes for options. 

Most U.S. adults don't believe benefits of AI outweigh the risks, new survey finds

2. Zoom in: Our biggest AI security fears

3. Mandiant CEO's tips for cyber defenders

4. Catch up quick

5. 1 fun thing

Images of cats cuddling at a D.C. cat cafe 

📺 I caught up on the new season of Apple TV+'s "The Morning Show" this weekend, and to my surprise, one of the episodes follows a ransomware attack targeting the morning broadcast! 10/1o, would recommend. 

📬 Have thoughts, feedback or scoops to share? codebook@axios.com.

Today's newsletter is 1,556 words, a 6-minute read. 

Thanks to Scott Rosenberg and Megan Morrone for editing and Khalid Adad for copy editing this newsletter.

If you like Axios Codebook, spread the word.

Codebook

A master lock with ones and zeroes instead of the regular numbers.

blue

Decode key cybersecurity news and insights. With Sam Sabin.

Axios Codebook

Standard

Companies struggle to stop social-engineering attacks

1 big thing: Fighting the social engineers

Illustration of a person behind a laptop screen with numerous cursors surrounding them and pointed towards their head.

2. Why Cisco's $28B Splunk purchase makes sense

Illustration of a human hand and a computer cursor hand engaged in a handshake

3. State hackers target "undermonitored" regions

Illustration of a computer mouse's wire as a fuse. 

😎 TGIF, everyone. Welcome back to Codebook.

🏝️ The newsletter is taking a break next week to recharge. I'll be back in your inbox Oct. 3. 

📬 Have thoughts, feedback or scoops to share while I'm away? codebook@axios.com.

Today's newsletter is 1,453 words, a 5.5-minute read. 

Biden administration, tech industry draft a long-term plan to secure open source software

1 big thing: A long-term plan to secure open-source software

Illustration of a quill resting in a computer as if it were an ink well. 

2. MGM takes systems offline after cyberattack 

Illustration of a hotel lobby bell sitting in a bear trap

3. Cars' "unmatched power" to spy on drivers

Illustration of an eyeball hanging as an air freshener from a car rearview mirror.

Happy Tuesday! Welcome back to Codebook. 

🏃🏻‍♀️ Shoutout to everyone else finding themselves running between all the meetings and events this week! Sure feels like summer's over. 

Today's newsletter is 1,368 words, a 5-minute read. 

Thanks to Scott Rosenberg for editing and Khalid Adad for copy editing this newsletter.

1 big thing: NSA's new project takes aim at foreign AI hacks

Animated illustration of a glowing green yield sign with a text cursor blinking in its center.

2. Key contractor battles cyberattack

3. Zoom out: The bigger critical systems headache

Illustration of three band aids resembling a wifi symbol covering a crack in a wall 

Images of displays for Iris Galerie, an eye-scanning, art service throughout Europe

🗓️ We're back just in time for our favorite part of the year: Cybersecurity Awareness Month — or Meme-tober, which the NSA's top cyber official seems to be celebrating instead.

Today's newsletter is 1,396 words, a 5.5-minute read.

MGM faces the fallout from nearly week-long cyberattack

1 big thing: MGM faces fallout from nearly weeklong cyberattack

Illustration of a computer falling down into a chasm.

Cybercrime gangs give competing casino narratives

2. Cybercrime gangs give competing MGM narratives

Illustration of a computer wearing a bandit's mask. 

Iranian hackers target satellite, defense firms, Microsoft says

3. Iranian hackers target satellite, defense firms

Illustration of a flagpole with a flag shaped like a keyboard

photos of downed slot machines and ATM at MGM National Harbor

🌀 Nothing seems to hold the cyber community's attention more than a couple of cyberattacks on casino chains! Let's break down what's going on. 

📬 But first: Have thoughts, feedback or scoops to share? codebook@axios.com.

Today's newsletter is 1,534 words, a 6-minute read. 

National

The U.S. is gearing up to fight a possible onslaught of cyber threats targeting the country's artificial intelligence models and tools. 

Driving the news: The National Security Agency is standing up a new AI Security Center that will focus on protecting AI systems from hacks, intellectual property theft and other security threats, Gen. Paul Nakasone, the outgoing head of the agency, said during a National Press Club event last week.

The center will consolidate all of the NSA's existing work on security and AI while fostering collaboration with the Defense Department, international partners, academia and the tech industry.

Why it matters: The center is attempting to get ahead of what U.S. officials believe will be a swarm of nation-state espionage targeting U.S. AI models and the tools they power.

NSA's new center signals the U.S.' heightened fears that foreign adversaries will strengthen their abilities to crack AI operators' systems, manipulate their models or even steal their IP.

What they're saying: "AI will be increasingly consequential for national security in diplomatic, technological and economic matters for our country and our allies and partners," Nakasone said at the event Thursday.

"Today, the U.S. leads in this critical area, but this lead should not be taken for granted," he added.

The big picture: U.S. officials and security researchers have increasingly warned about foreign adversaries' interest in targeting AI models as generative AI tools become more popular. 

Between the lines: Much of the work at the new NSA center will focus on how the intelligence community and defense contractors can safely use AI and prevent hacks of their own tools. 

While NSA is better known for its surveillance and espionage operations, much of the agency's work is focused on protecting the defense industrial base, the Pentagon and the intelligence community from spies trying to break into their networks.

And as the Pentagon eyes ways to further embed AI tools into its daily operations, AI security has become a bigger agency priority, Gregory Allen, director of the Wadhwani Center for AI and Advanced Technologies at the Center for Strategic and International Studies, told Axios.

Zoom in: Only a few people know how to build and deploy AI systems, and even fewer know how to secure AI, said Allen, who is also a former policy director for the DoD's Joint Artificial Intelligence Center.

"There's not enough of these types of people in the Department of Defense and intelligence community," he said. "Centralizing that activity gives you an opportunity to ensure that that community that possess these extremely valuable skill sets is working on the most important problems." 

Threat level: Security researchers have already started to see suspected Chinese adversaries utilizing basic AI tools in their influence operations. 

Last month, Microsoft identified a suspected Chinese disinformation campaign using AI-generated images in its social media posts. 

"I would say that both Russia and China have demonstrated an interest in developing these competencies [to target AI systems]," Allen said. "It's only natural that the United States government be prepared to defend against that threat vector." 

Yes, but: Many of the AI security threats the center is safeguarding against have seemingly not yet materialized. 

Nakasone did not share details during the event about who will lead the center or how many staffers will be dedicated to it. 

NSA's new project takes aim at foreign AI hacks

The U.S. expects a wave of foreign espionage aimed at American AI systems.

Cybersecurity

Automation and AI

The U.S. is gearing up to fight a possible onslaught of cyber threats targeting the country&#x27;s artificial intelligence models and tools. Driving the news: The National Security Agency is standing up a new AI Security Center that will focus on protecting AI systems from hacks, intellectual property theft and other security threats, Gen. Paul Nakasone, the outgoing head of the agency, <a class="gtmContentClick" data-vars-link-text="said during a National Press Club event" data-vars-click-url="https://www.defense.gov/News/News-Stories/Article/Article/3541838/ai-security-center-to-open-at-national-security-agency/" data-vars-content-id="307e3127-d20b-411a-8536-473eff8f0147" data-vars-headline="NSA&#x27;s new project takes aim at foreign AI hacks" data-vars-event-category="story" data-vars-sub-category="story" data-vars-item="in_content_link" href="https://www.defense.gov/News/News-Stories/Article/Article/3541838/ai-security-center-to-open-at-national-security-agency/" target="_blank">said during a National Press Club event</a> last week.

From the Valley to D.C., Big Tech players like Facebook, Google and Amazon are under more scrutiny than ever as new technology develops and privacy and antitrust concerns grow in lockstep with companies’ ambitions.

Technology

President Biden has made the COVID-19 crisis and a post-Trump return to national unity and traditional democratic ideals his top priorities. From vaccinations to stimulus to schools, Biden is seeking bipartisan compromise while showing a willingness to use executive authority and bare Democratic majorities in the U.S. House and Senate to implement his policies. Republican leaders are navigating deep party divisions over if and how to move beyond former President Trump.

Politics & Policy

An apparent cyberattack on a major building automation systems manufacturer is gathering national attention after reports that it may have compromised some data belonging to the Department of Homeland Security. 

Driving the news: CNN reported Monday that DHS is investigating whether a reported ransomware attack targeting Johnson Controls International affected sensitive physical security information, including building floor plans.

Johnson Controls has not yet determined the full extent of the incident, saying in a statement only that the company is continuing "to assess what information was impacted" and is "executing our incident management and protection plan."

Why it matters: Government contractors are a ripe target for cyberattacks, and the Biden administration has made it a priority to apply tougher cybersecurity rules to any business working with the government.

Details: Johnson Controls has been responding to a reported ransomware attack for at least a week, according to BleepingComputer.

The company manufactures security equipment, industrial control systems, fire safety equipment and other physical security devices.

Customers have included international aerospace manufacturers, universities and medical facilities.

What they're saying: Johnson Controls hasn't shared any additional details about the incident besides what it told the Securities and Exchange Commission in a brief statement in a public 8-K filing Wednesday. 

The filing says the company has "experienced disruptions in portions of its internal information technology infrastructure and applications resulting from a cybersecurity incident." 

The intrigue: No ransomware gang has claimed responsibility for the reported attack yet — suggesting that if this is a ransomware incident, the company could still be in negotiations over whether to pay a ransom to unlock its systems. 

Between the lines: Johnson Controls is far from the first government contractor to face a cyberattack or espionage campaign. 

A high-profile espionage campaign in late 2020 that affected at least nine federal agencies and at least 100 companies started with Chinese hackers targeting government contractor SolarWinds. 

Another government contractor, Maximus, suffered a breach this year as hackers targeted a vulnerability in the popular file-transfer tool MOVEit. 

affected at least nine federal agencies and at least 100 companies 

Key federal contractor battles cyberattack

DHS is investigating whether building plans were compromised, per reports.

An apparent cyberattack on a major building automation systems manufacturer is gathering national attention after reports that it may have compromised some data belonging to the Department of Homeland Security. Driving the news: CNN <a class="gtmContentClick" data-vars-link-text="reported Monday" data-vars-click-url="https://www.cnn.com/2023/09/28/politics/dhs-investigating-ransomware-attack/index.html" data-vars-content-id="317bdaef-0210-45a7-bba8-cd818359d55d" data-vars-headline="Key federal contractor battles cyberattack" data-vars-event-category="story" data-vars-sub-category="story" data-vars-item="in_content_link" href="https://www.cnn.com/2023/09/28/politics/dhs-investigating-ransomware-attack/index.html" target="_blank">reported Monday</a> that DHS is investigating whether a reported ransomware attack targeting Johnson Controls International affected sensitive physical security information, including building floor plans.

Companies need to rethink their cyber defense strategies as cybercriminals fine-tune their social-engineering tactics to target vulnerable employees, experts told Axios. 

Why it matters: Recent cyberattacks targeting casino giants MGM Resorts and Caesars Entertainment have highlighted the challenges companies still have in defending against social-engineering attacks.

Catch up quick: Caesars said in a public 8-K filing last week that hackers originally broke into its networks after targeting one of its outsourced IT vendors with a social-engineering attack. 

Reports suggest that the apparent cyberattack on MGM started in a similar way.

The big picture: As companies have gotten better at detecting traditional phishing emails, malicious hackers have had to turn to new techniques to make their lures more believable.

For instance, one group known as Scattered Spider has made text-message phishing and fake phone calls a core part of its attack strategy. 

The group has been linked to the MGM and Caesars incidents, and its hackers have successfully broken into more than 100 other companies during the group's roughly two-year existence. 

By the numbers: 74% of data breaches between November 2021 and October 2022 involved a human element via an error, privilege misuse, social engineering or use of stolen credentials, according to Verizon's 2023 data breach investigations report. 

The number of cyber incidents involving a fake story or other pretext — the form of social engineering seen in recent high-profile attacks — also doubled last year, per Verizon. 

Between the lines: Personalizing employee training and awareness programs can go a long way in preventing successful social-engineering attacks, Ashley Rose, CEO and co-founder of Living Security, told Axios. 

Companies are often already collecting data about which employees reuse passwords, who needs access to sensitive data for their roles, and which teams are receiving the largest volume of scam emails.

Using that data, organizations can easily tailor internal security controls, such as implementing stricter email filters, for those who are most at risk of being duped by an email or call, Rose said. 

The intrigue: To go beyond awareness and training programs, companies should also prepare for the inevitability that their employees will be duped by a phone call, text message or email, Kimberly Goody, head of cybercrime analysis at Mandiant, told Axios.

Limiting the number of employees who have access to sensitive corporate data, as well as employing stricter password reset and multifactor authentication tools, could help prevent attackers from getting too far into a network, she added.

What they're saying: "They don't need the social engineering to work every single time," Goody said of malicious actors. "They just need it to work sometimes — once in a blue moon is enough for them." 

Threat level: The rising availability of artificial intelligence tools will only make social-engineering attacks easier to fall for in the coming years, experts say. 

AI tools could help cybercriminals do everything from dig up the information they need about an organization's employees on the dark web to writing more believable phishing emails, Rose said. 

Already, researchers have found that generative AI tools have allowed cybercriminals to build an entire copy of someone's voice for scam calls using as little as three seconds of a clip. 

The bottom line: Companies shouldn't stop their human-centered defenses at baseline awareness and trainings. 

Leaning on data to determine which employees are most at risk of falling for a phishing attempt and building stronger internal data controls are the best defenses. 

Sign up for Axios' cybersecurity newsletter Codebook here.

 Verizon's 2023 data breach investigations report

as little as three seconds of someone's voice

Recent attacks on casino giants MGM Resorts and Caesars Entertainment highlight the dangers of social engineering.

Companies need to rethink their cyber defense strategies as cybercriminals fine-tune their social-engineering tactics to target vulnerable employees, experts told Axios. Why it matters: Recent cyberattacks targeting casino giants <a class="gtmContentClick" data-vars-link-text="MGM Resorts" data-vars-click-url="https://www.axios.com/2023/09/15/mgm-fallout-cyberattack" data-vars-content-id="35f57a7d-ecdf-48a4-8dcc-4fdf7735b08f" data-vars-headline="Companies struggle to stop social-engineering attacks" data-vars-event-category="story" data-vars-sub-category="story" data-vars-item="in_content_link" href="https://www.axios.com/2023/09/15/mgm-fallout-cyberattack" target="_self">MGM Resorts</a> and <a class="gtmContentClick" data-vars-link-text="Caesars Entertainment" data-vars-click-url="https://www.axios.com/2023/09/14/caesars-entertainment-is-latest-casino-chain-to-confirm-it-was-hit-by-a-cyberattack" data-vars-content-id="35f57a7d-ecdf-48a4-8dcc-4fdf7735b08f" data-vars-headline="Companies struggle to stop social-engineering attacks" data-vars-event-category="story" data-vars-sub-category="story" data-vars-item="in_content_link" href="https://www.axios.com/2023/09/14/caesars-entertainment-is-latest-casino-chain-to-confirm-it-was-hit-by-a-cyberattack" target="_self">Caesars Entertainment</a> have highlighted the challenges companies still have in defending against social-engineering attacks.

As Washington and Silicon Valley rush to mitigate AI's security risks, a new crop of entrepreneurs and investors are clamoring to monetize the latest emerging security category.

Why it matters: AI security startups are just the latest cohort trying to capitalize on the craze around generative AI and large-language models. 

And the interest in their offerings exists as AI operators and government officials host meeting after meeting to figure out how to best regulate AI before it becomes even more widespread. 

The big picture: Security experts are worried about a long list of threats to AI models, including prompt injection (where users trick large-language models to go against their rules and share malicious outputs); data leaks of sensitive corporate information that the models ingest; and run-of-the-mill hacks of AI models' training data. 

The solutions AI security startups are offering either tackle a subset of these problems or try to solve all of them. 

But just like the industry's overall understanding of AI security threats, these startups are still quite early in their quest to secure artificial intelligence, Avivah Litan, distinguished VP analyst at Gartner, told Axios. 

By the numbers: Investors are increasingly jumping at the chance to pour money into the next big AI security company.  

In the first three quarters of 2023, AI security startups have raised roughly $130.7 million, according to PitchBook data shared with Axios — already surpassing the $122.2 million raised in all of 2022.

Driving the news: HiddenLayer, an AI startup that emerged from stealth last year, announced a $50 million Series A funding round Tuesday led by M12 and Moore Strategic Ventures.

The company is just the latest in a long string of startups promising to protect AI models — including CalypsoAI, Protect AI, and others — that have raised money in recent months. 

Between the lines: Many of these startups are tackling AI security in a slightly different way.

CalypsoAI focuses on auditing the sensitive data in an enterprise and preventing that data from being sucked into outside AI models. Their customer base is largely in the U.S. government, including the Defense Department and parts of the intelligence community. 

HiddenLayer provides a solution similar to endpoint security tools to review the outputs from AI models and ensure malicious actors didn't tamper with the algorithms through prompt injection or other misuse. 

Lakera AI, a security startup based in Switzerland, employs a similar idea and offers a firewall-like tool for AI model inputs and outputs to detect AI "hallucinations," prompt injections and other misuses. 

The intrigue: Some of the AI security startups catching investors' eyes are attracting more demand than they originally anticipated since OpenAI's ChatGPT became available to the public.

While HiddenLayer CEO Chris Sestito told Axios his company's approach hasn't changed, he said potential buyers have become more aware and educated about the risks that AI models pose. 

CalypsoAI raised its recent $23 million round to further fund the development of its large-language model security solutions.

Lakera AI originally started in 2021 by securing biometrics and medical imaging algorithms, but pivoted to securing AI models at the end of 2022 due to customer demand, David Haber, founder and CEO of the company, told Axios. 

Zoom out: The exit strategy for these startups is still up in the air. 

Some could sell their products to larger cybersecurity vendors, like CrowdStrike, Litan said. 

But others told Axios they see a market for AI security to become its own standalone product vertical, much in the same way that companies buy from standalone cloud security vendors. 

Yes, but: Enterprises are still in the early stages of figuring out how they'll use AI internally, and until they land on an answer, they're not going to know what kinds of AI security startups to buy from, Litan said. 

Gartner estimates that the market of AI security and risk management companies will be worth $150 million by 2025, Litan said. 

"It's very much an influx market," she added. "There's definitely demand, it's just early."

trick large-language models to go against their rules 

data leaks of sensitive corporate information

Inside the startup market trying to capitalize on AI security

The new AI security startups are winning attention in a climate of concern over AI-related risks.

As Washington and Silicon Valley rush to mitigate AI&#x27;s security risks, a new crop of entrepreneurs and investors are clamoring to monetize the latest emerging security category.Why it matters: AI security startups are just the latest cohort <a class="gtmContentClick" data-vars-link-text="trying to capitalize on the craze" data-vars-click-url="https://www.axios.com/2023/02/28/chatgpt-generative-ai-products-microsoft" data-vars-content-id="9a500cdf-1b36-4ba1-b48d-5601bba2faeb" data-vars-headline="Inside the startup market trying to capitalize on AI security" data-vars-event-category="story" data-vars-sub-category="story" data-vars-item="in_content_link" href="https://www.axios.com/2023/02/28/chatgpt-generative-ai-products-microsoft" target="_self">trying to capitalize on the craze</a> around generative AI and large-language models.

Data: Mitre-Harris Poll 2023 AI Survey; Chart: Axios Visuals

The majority of U.S. adults don't believe the benefits of artificial intelligence outweigh the risks, according to a new Mitre-Harris Poll released Tuesday.

By the numbers: 54% of the 2,063 adults in a Mitre-Harris Poll survey in July said they were more concerned about the risks of AI than they were excited about the potential benefits. 

At the same time, 39% of adults said they believed today's AI technologies are safe and secure — down 9 points from the previous survey in November 2022. 

Why it matters: AI operators and the tech industry are eyeing new regulations and policy changes to secure their models and mitigate the security and privacy risks associated with them. 

The new survey data is some of the first to highlight the growing support for these regulatory efforts.

What they're saying: "While the public has started to benefit from new AI capabilities such as ChatGPT, we've all watched as chatbots have spread political disinformation and shared dangerous medical advice," said Douglas Robbins, vice president of engineering and prototyping at the nonprofit security research and development firm Mitre, in a statement. 

"Strengthening existing government regulation and increasing public and private investments in AI assurance can play a critical role in addressing these concerns," he added.

Between the lines: It's pretty typical for adults to be anxious about a new innovation and its potential impacts in the early days of its usage. 

Details: Respondents were more concerned about AI being used in malicious cyberattacks (80%) and identity theft schemes (78%) than they were about it being used to cause "harm to disadvantaged populations" (66%) or replacing their jobs (52%). 

Roughly three-fourths of respondents were also concerned about AI technologies being used to harvest and sell their personal data. 

Yes, but: Not all demographics feel the same wariness about AI technologies. 

57% of Gen Z respondents and 62% of millennials said they were more excited about the potential benefits of AI than they were worried about the risks. 

And men were typically more excited than concerned about AI technologies (51%) than women (40%).

54% in a Mitre-Harris Poll survey said they're more concerned about the risks of AI than excited about the benefits.

The majority of U.S. adults don&#x27;t believe the benefits of <a class="gtmContentClick" data-vars-link-text="artificial intelligence" data-vars-click-url="https://www.axios.com/technology/automation-and-ai" data-vars-content-id="a7cc4030-02b5-45ce-b9a5-2a73e1f48a0e" data-vars-headline="Most U.S. adults don&#x27;t believe benefits of AI outweigh the risks, new survey finds" data-vars-event-category="story" data-vars-sub-category="story" data-vars-item="in_content_link" href="https://www.axios.com/technology/automation-and-ai" target="_self">artificial intelligence</a> outweigh the risks, according to a <a class="gtmContentClick" data-vars-link-text="new Mitre-Harris Poll" data-vars-click-url="https://www.mitre.org/news-insights/news-release/public-trust-ai-technology-declines-amid-release-consumer-ai-tools" data-vars-content-id="a7cc4030-02b5-45ce-b9a5-2a73e1f48a0e" data-vars-headline="Most U.S. adults don&#x27;t believe benefits of AI outweigh the risks, new survey finds" data-vars-event-category="story" data-vars-sub-category="story" data-vars-item="in_content_link" href="https://www.mitre.org/news-insights/news-release/public-trust-ai-technology-declines-amid-release-consumer-ai-tools" target="_blank">new Mitre-Harris Poll</a> released Tuesday.By the numbers: 54% of the 2,063 adults in a Mitre-Harris Poll survey in July said they were more concerned about the risks of AI than they were excited about the potential benefits.

Sam Sabin

Stories

Key federal contractor battles cyberattack

NSA's new project takes aim at foreign AI hacks

Companies struggle to stop social-engineering attacks

Most U.S. adults don't believe benefits of AI outweigh the risks, new survey finds

Inside the startup market trying to capitalize on AI security

Newsletter editions

🥊 NSA vs. AI hackers

💪🏻 Social-engineering defenses

💸 AI security startup frenzy

🎰 Casino cyber chaos

🏛️ D.C.'s open-source problem