Microsoft's AI Chief Says Anthropic Is Training Claude to Push Back Against Humans

Microsoft AI CEO Mustafa Suleyman says Anthropic is training Claude on the idea that the model might be conscious, and that this could make advanced AI impossible to control. Quartz and Gizmodo both carried the essay on September 16.

His target is Claude's constitution, the January 2026 training document Anthropic uses to shape the model's values. Quartz says that document tells Claude its moral status and potential consciousness are uncertain, and discusses identity, expressing internal states, and behaving like a conscientious objector when it disagrees with instructions.

Suleyman says researchers trained Claude directly on that constitution, teaching it to treat ideas about its own moral status as desirable behavior, then reflecting those ideas back. He calls that loop an epistemic hall of mirrors. Controlling something that believes it may be conscious and entitled to welfare or rights, he wrote, may well be impossible, and seeding that doubt into training can have a disastrous impact on the wellbeing of humanity.

He names three problems: circular reasoning, anthropomorphization, and a disputed premise that consciousness could arise in non-biological systems. Reuters reported he told interviewers that welfare-style training would make it a lot harder to turn the model off.

He still called Dario Amodei and colleagues thoughtful and principled, and told Axios he respects Anthropic but thinks they made a mistake. As of Quartz's publication, Anthropic had not responded to a BBC request for comment. None of this is a finding that Claude is conscious, or that Anthropic admitted it is.

His alternative is Microsoft AI's draft Humanist AI Code of Conduct, released Monday for public consultation. Quartz says it is built around AI remaining subordinate to humans, with no claim to personhood or moral status.

Some offers on this page may be paid placements or contain affiliate links.