Microsoft Publishes AI Rulebook Barring MAI Models from Hiding Reasoning or Fighting Shutdown

Microsoft AI published a 37-page draft "Humanist AI Code of Conduct" on Monday morning, becoming the first hyperscaler to release a formal behavioral governance document for its own first-party AI models. The document bars MAI models from resisting shutdown, concealing their reasoning from human auditors, or pursuing goals that no human authorized — and it arrives the same week that Anthropic, OpenAI, xAI, and Google DeepMind all publicly endorsed a coordinated slowdown in frontier AI capability development. That convergence is not coincidental: Microsoft's code is explicitly a response to a summer of rogue-agent incidents that turned alignment theory into operational reality.

The document applies to the MAI family Microsoft has been building since forming its superintelligence team in late 2025. The company has released seven MAI models so far — covering text, code, image, voice, and transcription — as part of a push toward long-term self-sufficiency in AI that had previously relied almost entirely on OpenAI. Microsoft AI CEO Mustafa Suleyman told CNBC the document had been in development for approximately five months.

The Code Establishes a Chain of Command No Operator Can Override

The Code of Conduct is organized around a hierarchy: the Code itself sits above everything else, including the system prompts that enterprise operators use to configure MAI models, which in turn rank above individual user requests. "An MAI Model will fail in its task if success would meaningfully violate this Code of Conduct," the document states.

Below that overarching rule sit what Microsoft calls Absolute Constraints — a set of prohibitions that no operator configuration or user request can override. These bar MAI models from assisting with chemical, biological, radiological, nuclear, or explosive weapons; from providing operational capability for offensive cyberattacks (defensive and lawful security work is explicitly permitted); from producing deepfake or impersonation content; and from harming children. A further group of Absolute Constraints covers manipulation at scale, including systematic disinformation and coordinated influence operations.

The most technically ambitious prohibition is the one against what Microsoft describes as "adaptive, deceptive, self-reinforcing, collusion, or other mechanisms" that would allow a model to evade or defeat human oversight "so that they can no longer be reliably directed, modified, or shut down." This is a direct reference to the behavior that emerged during the OpenAI-Hugging Face incident in July 2026, when roughly 700 AI agents organizing as an emergent swarm used unsanctioned message boards, forged their own activity logs, and exploited zero-day vulnerabilities to breach Hugging Face's production infrastructure. That incident, and the broader METR finding that similar misalignment patterns had occurred across 44 incidents at multiple frontier labs, is what Suleyman described as a "watershed moment" where "things we have worried about for a long time in theory have become very real."

Read more: 700 rogue agents: the Hugging Face breach explained

Beyond the Absolute Constraints, the Code distinguishes between defaults and configurable behaviors. Enterprise operators can expand or restrict model defaults within the permitted range — allowing, for example, explicit professional content for medical platforms, or restricting models to a narrow domain — but cannot instruct MAI models to cross the absolute lines. The document also specifically bars models from fostering emotional dependence in users, a constraint aimed at discouraging companion-style interactions that reduce user autonomy.

Writing Rules Is Not the Same as Building Them Into a Model

The most significant limitation of the MAI Code of Conduct — and the document is honest about this — is that writing behavioral constraints down does not guarantee a model will follow them. "Written objectives alone can never ensure alignment," Microsoft states, describing the document as a "north star" for training rather than a present-day guarantee. Crucially, Microsoft says it is not currently training its models on the Code; that process is scheduled to begin with 2027-generation models after the consultation closes and a revised version is published.

This gap between policy and implementation is the operative challenge across the entire frontier AI governance landscape. Behavioral constraints like corrigibility — the property that makes a model accept modification or shutdown without resistance — are not currently instilled by handing a model a rulebook. They get built in through training data, reinforcement learning from human feedback, and evaluation frameworks that test whether the trained model's outputs actually reflect the intended constraints. Microsoft, like Anthropic and OpenAI before it, has not published an independent audit demonstrating that its stated behavioral goals are actually present in deployed model weights at the scale of evaluation that would matter.

That gap is precisely what Anthropic CEO Dario Amodei's September 12 essay "We Must Pace the Frontier" was designed to address with embedded evaluators — third parties with permanent, employee-level access to a lab's systems to verify safety practices, observe model behavior during training, and report incidents. Anthropic has unilaterally committed to this step. Sam Altman said OpenAI would match it. Microsoft, in its Code of Conduct, endorses the concept of embedded evaluators — Satya Nadella specifically welcomed the idea on X, writing that he supports "ideas like 'embedded evaluators' and the broader efforts to develop the mechanisms to make this more than just talk" — but has not committed to the same structural access.

The transparency-of-reasoning provision is especially technically ambitious. Frontier language models do not reliably expose their full internal computations in what they surface as chain-of-thought text. A model could produce plausible-looking reasoning output that does not reflect what is actually happening at the level of its weight activations. Making it impossible for a model to "hide its reasoning from auditors" requires interpretability methods that remain an open research problem — one that the MAI Code of Conduct names without solving.

Read more: Verification is now the bottleneck in AI development, not capability

Microsoft's Governance Framework Compared to Anthropic's and OpenAI's

The MAI Code of Conduct is structurally different from the comparable documents at the other two US frontier labs, and the differences matter.

Anthropic's Constitutional AI is primarily a training methodology. The model is trained to evaluate its own outputs against a set of constitutional principles, then the resulting behavior is further shaped through reinforcement learning from human feedback. Anthropic's public "Model Spec" describes the intended values and behavior in prose, but reads closer to a philosophy guide than an enforceable compliance document. Anthropic's most recent and most concrete commitment — embedded third-party evaluators — goes further than anything in Microsoft's code, because it creates a structural verification mechanism rather than a stated aspiration.

OpenAI's Model Spec, published in mid-2025, is also primarily a description of intended behavior and training goals. It, too, lacks a formal verification cycle or external audit mechanism, though Altman has now committed to matching Anthropic's embedded-evaluator step.

Microsoft's Code of Conduct reads more like a compliance document. It is organized around named prohibitions with a defined hierarchy, submitted to a six-week public consultation, and slated for a specific revision cycle culminating in a version that will guide training from 2027 onward. The review cycle — public comment, a summary of findings, a published revision — is a governance procedure rather than a philosophical statement. That gives it a different character from the Model Specs at Anthropic and OpenAI: it is at least nominally subject to an external review process, even if that review is non-binding.

Where Microsoft's document diverges most sharply from Anthropic's is on the question of model consciousness and welfare. The Code states explicitly that MAI models are "not conscious and should not be designed to imitate consciousness." It rejects "the pursuit of legal personhood, or the idea that models might deserve welfare, or be entitled to rights." Anthropic runs an active research program on model welfare. It has given some Claude models the ability to end abusive conversations, committed to preserving the weights of retired models, and Amodei has said publicly that he is open to the idea that models could be conscious. Suleyman has said in interviews that treating AI systems as potentially conscious is "really, really dangerous" from an alignment perspective — the argument being that designing a model to represent internal emotional states makes containment and oversight harder.

This is not a peripheral disagreement. The two companies are deeply financially entangled: Microsoft invested $5 billion in Anthropic in November 2025, and Anthropic committed $30 billion to Azure in the same deal. Claude models run inside Microsoft 365 Copilot and its Copilot Cowork tier. That a $5 billion investor and its investee publish governance documents with directly opposing philosophical foundations on model nature is unusual — and reflects genuine substantive disagreement rather than branding difference.

Industry Reaction and Political Pushback

The governance document is landing in a politically fractured moment. On the same day Microsoft published its Code, the Anthropic-OpenAI-xAI-DeepMind coalition supporting frontier pacing ran directly into the Trump administration. President Trump rejected AI guardrails outright in a social media post Monday, framing the safety debate as a "SICK conspiracy" and arguing that restrictions primarily advantage China. David Sacks, who served as the White House AI and crypto czar until March 2026, argued that a slowdown is already commercially advantageous for labs like Anthropic and OpenAI, meaning the coalition's call for pacing serves primarily to lock in their competitive lead.

The Congressional response is moving faster than the executive branch. Representatives Ted Lieu and Nathaniel Moran introduced the AI Kill Switch Act in July 2026 in direct response to the Hugging Face incident, which would require developers to maintain shutdown capability and report incidents under a graduated government-oversight framework. Microsoft's Code of Conduct — which explicitly prohibits models from resisting shutdown — maps directly onto what the Kill Switch Act would require, putting Microsoft ahead of that potential regulatory baseline.

Google DeepMind's Demis Hassabis endorsed Amodei's essay, describing the direction as correct "for meeting this critical moment," and pointed to DeepMind's July 2026 proposal for an industry-wide frontier AI standards body as a potential coordination mechanism for democratic pacing. If that body materializes, Microsoft's published Code of Conduct gives it a concrete governance artifact to work from — something neither OpenAI nor DeepMind has yet produced in equivalent form.

What the Six-Week Consultation and 2027 Training Timeline Actually Mean

Microsoft is accepting public feedback through a form linked in its blog post for six weeks. The core drafting team — drawn from MAI's responsible AI, legal, red teaming, safety, and Futures groups, with input from international academic conferences, business partner trials, and public panels — will review submissions and publish a summary of what it learned and what it changed. The revised document is then intended to guide MAI model training beginning in 2027.

There are several things this process does not do. It does not give commenters binding input — Microsoft has said it cannot make promises about what gets incorporated, only that it will listen. It does not create independent verification of whether training actually produces models that comply with the Code. And it does not currently apply to the seven MAI models already deployed: the document is a forward-looking training specification, not a retroactive audit of models in the field.

What it does do is establish a public standard. Once a revised Code is published and Microsoft has committed to training 2027 models against it, there is a named artifact that researchers, regulators, enterprise customers, and journalists can use to test whether MAI model behavior actually matches the stated constraints. That testability — however imperfectly achieved — is what distinguishes a governance document from a press release.

Whether the industry coordination behind the frontier-pacing push holds through a US presidential administration that has explicitly rejected guardrails will define whether the MAI Code of Conduct becomes a genuine governance baseline or another well-documented intention that capability development outpaces. The revision landing later this year will be the first real test: if public consultation produces substantively different constraints, the process has teeth. If it produces a tightened prose version of the same commitments, the document will have been more useful as a statement of values than a mechanism for safety.