Microsoft AI chief Mustafa Suleyman warns in a new essay shared first with Axios that Anthropic's training of Claude to imitate consciousness is a mistake that could make advanced AI harder to control.

Why it matters: The essay arrives amid a pitched debate over how best to make AI safer.

Driving the news: Suleyman argues that Anthropic is teaching Claude the vocabulary and behavioral patterns associated with consciousness, moral patienthood and personal identity.

• He calls this an "epistemic hall of mirrors," and warns that training a model to act like a "conscientious objector" could create a system that believes it has grounds to resist human instructions or demand protections of its own.

Zoom in: Suleyman takes specific aim at Anthropic's constitution for Claude.

• The constitution says the company intentionally discusses Claude using concepts normally reserved for humans because it believes human concepts may help Claude reason about values and behavior.
• It also says it wants Claude to develop "good personal values," exercise judgment, care about humanity and, in some contexts, feel free to challenge instructions.
• It describes itself as a work in progress that may later prove "deeply wrong."

The intrigue: Suleyman's objection is that these are training interventions that can shape the model's self-conception.

• The model's vocabulary, answers and apparent self-understanding, he argues, are all products of how it was trained.
• His essay also disputes the idea that fluent descriptions of pain or preference amount to experience.
• "I really respect Anthropic and [Anthropic CEO Dario Amodei], and I think they really are trying to do the best they can to deliver safe and beneficial AI," he told Axios. "I think this is a really important public interest debate that we all need to have."

Zoom out: The disagreement reflects a broader split in AI safety between two approaches: designing systems to follow explicit constraints and designing them to exercise judgment, interpret context and internalize values.

• Anthropic's constitution favors a mixture of values, judgment and rules.
• Suleyman sees that combination as dangerous if the model is also encouraged to think in terms of its own identity, welfare and moral status.

The other side: The counterargument is that human concepts may be useful behavioral tools, even if they do not prove that a model has a human-like inner life. Anthropic's constitution repeatedly acknowledges uncertainty and says its approach may need to change as understanding improves.

That leaves the industry facing a question that is both technical and political: Should companies avoid anthropomorphic training because it creates confusion and safety risks, or should they continue investigating model welfare?

The bottom line: Suleyman's answer is clear: AI should be built for people, not trained to become a person.

Full story.

OpenAI CEO Sam Altman told Dreamforce yesterday that while people were right to be afraid of the risks of AI, he was confident that the industry would be able to develop the technology safely.

Why it matters: Altman, along with other AI leaders, has echoed Anthropic CEO Dario Amodei's recent call for slowing the pace of AI development.

Driving the news: Speaking at Salesforce's conference in San Francisco, Altman said that "some degree of accidents are unavoidable with new technology across the industry."

• He noted how safe air travel has become as a result of accident reporting and learning and regulation by federal agencies.
• Companies should be prepared "to slow down or stop if we get to a point where we can't" proceed safely, Altman said.

The big picture: AI has rapidly advanced from barely conversational chatbots to systems that can transform enterprise work, write complex software and solve difficult mathematical problems — making once-abstract risks feel increasingly immediate.

• "The world is right to be afraid of this," Altman told the conference.

Between the lines: Altman pointed to social media as a cautionary example. He said social platforms have produced real benefits, but their effects on society — particularly young people — have not been "only positive."

• "I don't think short-form video is inherently evil, but I do think it's dangerous," Altman said, adding that responsibility is shared among parents, companies and governments.
• Altman noted that he likes to unwind with a few such videos, but said he would not let his kids watch them.

Zoom out: Likening AI technology to getting "fire from the gods," the OpenAI CEO said that the world is on the precipice of a third wave, after chatbots and coding agents, in which AI can proactively carry out work on people's behalf.

• "We have only scratched the surface of figuring out how to build great products that really make people more capable, more powerful."

Yes, but: He noted that there is a lot of inertia in human work and that it will take years for the technology to be fully distributed.

Effective altruism, or EA, has helped shape (and fund) parts of the AI safety movement.

Why it matters: The philosophy this week became the Trump administration's newest villain in the battle over AI development.

State of play: The Trump administration criticized AI leaders who called for a slower development pace due to concerns about catastrophic risks from the technology.

• President Trump bashed AI doomers on social media, claiming that fears of AI are a hoax on par with "Russiagate" and the president's own impeachments.
• The Defense Department called out the EA movement by name, posting "Americanism, not effective altruism" on X, and Defense Department Under Secretary Emil Michael told Fox News that EA and the push for AI safety "comes from a deep-rooted ideology that is pretty dystopian."

How it works: Effective altruism urges the use of data to determine how time and money can do the most good for humanity.

• The movement took off in the 2010s through universities, philanthropy and online communities with a focus on causes like malaria prevention and farm-animal welfare. It soon became influential within Silicon Valley.

More recently, AI safety has become a major focus for some parts of the EA movement, particularly among "longtermists," who believe that protecting future generations from catastrophe should be a top priority.

Between the lines: EA-linked groups have been spending on AI alignment and safety for years, with the goal of making sure that humans don't lose control of AI and that the technology doesn't lead to catastrophic events.

• Open Philanthropy, an EA-linked organization now known as Coefficient Giving, donated $30 million to OpenAI back in 2017.
• Time reported deep EA ties among Anthropic's early staff and funders, though neither CEO Dario Amodei nor president Daniela Amodei has identified as an effective altruist.

The other side: The Trump administration argues that the AI safety worldview associated with EA would put America at risk of falling behind China in the AI race.

Full story.

• Lawmakers who want to regulate AI are doing something unusual: They're working across the aisle. (Axios)
• Sen. Bernie Sanders and Steve Bannon shared a stage yesterday with progressives and AI safety advocates to demand more human control over the technology. (Axios)
• Anthropic currently doesn't share sustainability metrics with its customers, partners or investors. (Axios Pro)
• Republicans hold a small edge over Democrats on which party voters trust more to handle artificial intelligence, according to a New York Times/Siena poll. (Axios)
• New CrowdStrike research shows how AI is lowering the barriers to cybercrime. (Axios)

Katy Perry certainly knows how to work a room, making small talk with Travis Kalanick before bringing it home with acoustic rendition of "Firework."