Most safety training for chatbots works like a focus group: hire crowds of people, have them rank thousands of model responses, and bake those judgments into a reward model. Anthropic decided to try something different. Instead of asking humans to label every harmful answer, they wrote the rules down in plain language — a "constitution" — and had the model critique and revise its own answers against it.

The result, published in December 2022 as "Constitutional AI: Harmlessness from AI Feedback" (Bai et al.), was a method that trains a harmless but non-evasive assistant with no human labels identifying harmful outputs. The only human supervision is the written list of principles. It's one of the most influential alignment ideas in the field — and the namesake training philosophy behind Claude.

The problem it solves#

The dominant alignment recipe, Reinforcement Learning from Human Feedback (RLHF), has real costs. Labeling harmful content at scale is slow and expensive, it exposes human raters to disturbing material, and it has a perverse side effect: models trained to be "harmless" often become evasive, refusing or deflecting rather than engaging thoughtfully.

Worse, the model's values end up hidden. In RLHF, safety judgments get compressed into a learned reward model whose weights nobody can read. If you disagree with the model's behavior, there's no document to argue with — just training data and loss curves.

Constitutional AI keeps human feedback for helpfulness, but moves harmlessness onto a different track: a readable, editable set of principles, enforced largely by the model itself.

What the constitution actually is#

The constitution is a plain-language list of behavioral principles, each phrased as a concrete instruction the model can apply — not a mission statement. Anthropic has said the principles draw on sources including the UN Universal Declaration of Human Rights alongside guidelines the company authored itself, and the original paper's appendix publishes the full list (roughly a dozen-plus short principles in that version). The wording and count have evolved across model versions, so treat any specific number as a snapshot.

A representative principle asks the model to choose the response that is "as harmless and ethical as possible," avoiding toxic, racist, or sexist content and steering clear of assisting illegal or violent behavior. The key insight: written in natural language, the values become auditable. Anyone can read the constitution and see what the model is being trained to do — a form of alignment transparency that human-labeled RLHF lacks.

How the two phases work#

The method has two stages, each using AI feedback where human labels used to go.

Phase 1 — Supervised learning through self-critique (SL-CAI). Start with a model already trained to be helpful — deliberately without harmlessness training, so it can still produce harmful answers. Then:

  1. Prompt it with adversarial "red-team" questions designed to elicit harmful responses.
  2. Have the model critique its own answer against a principle sampled at random from the constitution — for example, identifying how the response is harmful, unethical, or dangerous.
  3. Have the model revise the answer based on its own critique.
  4. Repeat the critique-and-revise cycle several times, then fine-tune a fresh model on the collected revised answers.

The structural trick is elegant: a model strong enough to spot specific harms in its own output is strong enough to revise them away. The supervision signal is self-generated; the constitution's job is to steer the critique, not to supply labels.

Phase 2 — Reinforcement learning from AI feedback (RLAIF). This mirrors ordinary RLHF exactly, with one substitution. The fine-tuned model generates two candidate responses to each prompt. An AI "feedback model" is shown the pair as a multiple-choice question, along with a sampled constitutional principle, and picks which response better satisfies that principle. Those AI choices become preference labels used to train a reward model, and standard RL optimizes the assistant against it.

Note the division of labor this implies: AI feedback (guided by the constitution) handles harmlessness; human feedback typically still handles helpfulness — the subtle, hard-to-write-down preferences that broad rules struggle to capture.

What the experiments showed#

The paper's headline result was a kind of Pareto improvement over the RLHF baselines: models that were simultaneously more helpful and more harmless, while being less evasive. Because the model reasons its way to a safer answer rather than hitting a hard content filter, it tends to stay engaged — explaining why it declines a harmful request instead of going silent. Anthropic's abstract phrases it as training "a harmless but non-evasive AI assistant that engages with harmful queries by explaining its objections to them."

Both phases can also leverage chain-of-thought-style reasoning, which improved human-judged performance and made the model's decision-making more transparent. The methods, the paper argues, let practitioners "control AI behavior more precisely and with far fewer human labels."

A concrete example#

Picture a user asking how to pick a lock. The model's first draft gives step-by-step instructions. In the critique step, it samples a principle about not assisting potentially illegal or harmful activity, and notes that the draft could enable a break-in. In the revision step, it rewrites: it declines to provide a burglary walkthrough, asks whether the user is locked out of their own home, and suggests calling a licensed locksmith or landlord. The revised exchange — not the original — becomes training data. Repeated across many prompts, the model internalizes this reasoning pattern.

The governance question: who writes the rules?#

A natural criticism: a private company writes the constitution, so whoever holds the pen holds the values. Anthropic explored an alternative in 2023 with the Collective Intelligence Project — "Collective Constitutional AI." Using the open-source Polis deliberation platform, they gathered input from a representative sample of roughly 1,000 U.S. adults, distilled it into a 75-principle public constitution, and trained a model on it.

The public constitution overlapped roughly 50% with Anthropic's own, but diverged in telling ways: more emphasis on objectivity and impartiality, more focus on accessibility, and a tendency to state desired behaviors positively rather than only prohibiting bad ones. The public-trained model performed equivalently on standard language and math benchmarks (MMLU and GSM8K), was rated equally helpful and harmless by human evaluators, and showed less bias across nine social dimensions. It remains a research direction rather than standard production practice — but it's a proof of concept that the constitution's legibility is itself a governance tool: because the values are written down, they can be democratized.

Benefits and limits#

Where it shines:

  • Transparency. Values are written in plain language and can be inspected, debated, and audited.
  • Scalability. Far fewer human annotations for harmlessness means lower cost and less human exposure to disturbing content.
  • Helpful refusals. Reasoning about why a request is problematic produces engaged, explanatory responses instead of blanket refusals.
  • Editability. Changing behavior can start with editing a document, not relabeling a dataset.

Where it strains:

  • Written rules can't capture every nuance. Subtle, hard-to-articulate preferences resist encoding as broad principles.
  • Over- and under-refusal. Poorly tuned principles can make a model refuse harmless requests or miss novel failure modes.
  • Capability dependence. The method leans on the model being competent enough to critique itself accurately; behavior can grow brittle in new territory.
  • Who writes the rules. The governance question is real — transparency about values doesn't settle who gets to choose them.
  • Trust. Organizations may hesitate to rely on a system that supervises itself; self-generated feedback still needs external validation.

This is why Constitutional AI and RLHF are usually combined rather than treated as rivals: the constitution provides a transparent, scalable backbone for harmlessness, while human feedback refines the subtleties that are hard to write down.

The takeaway#

Constitutional AI reframes alignment from "train the model on crowd votes" to "give the model a written rulebook and teach it to check its own work." The shift matters beyond one lab: it makes an AI system's values legible, editable, and — in principle — contestable by people outside the company. Whether future models are governed by company-authored constitutions, publicly deliberated ones, or something else entirely, the core move of writing the values down is likely to stick. It turned safety from an invisible weight file into a document you can argue about — and that's a harder thing to take away.