Model Cards, System Cards and What They’re Quietly Becoming
What are AI model cards, and why are they becoming the documents regulators will turn to first? I read a few and it taught me more than I expected.
"I think everyone interested in AI should read the model cards for the frontier models, especially the safety sections, which give you a sense of immediate concerns."
— Ethan Mollick, 4 August 2025
View on X
A few days ago, I came across a post by Ethan Mollick on X that stopped me in my scroll. He simply said that anyone interested in AI should read the model cards of the big frontier models — and especially the safety sections. He even linked to a few: OpenAI’s o3, Google’s Gemini Deep Think, Anthropic’s Claude 4. One was missing: Grok. Just question marks.
Mollick’s post wasn’t technical, but it hit a nerve. It made me realise how little I had actually looked at these cards myself. I work with AI models, talk about AI policy, and build things with them — but I hadn’t taken the time to really read the source documents that describe what these systems are capable of, where they might go wrong, and how (or if) they’re being kept in check.
At the same time, I had been reading up on AI benchmarks and evaluations — trying to make sense of what all these tests (MMLU, ARC, GSM8K, TruthfulQA) actually measure. That helped me understand what I was seeing in the cards. Benchmarks are not just numbers. They’re signals too: of what the labs choose to measure, and what they might prefer to avoid.
So I started digging.
What Are Model Cards — and Why Do They Matter?
In short: model cards (and their broader cousins, system cards) are documents published by the makers of large AI models to describe what those models can do, how they’ve been tested, and what their limitations are.
They usually include:
- What kind of data the model was trained on (or a vague description of it)
- What benchmarks it has been tested against (like MMLU, ARC, TruthfulQA)
- Where it tends to fail (hallucinations, bias, factuality)
- What the developers did to reduce harm (red teaming, refusal behaviour)
- And how they see the model in a broader societal context (misuse risks, alignment strategies)
Some are technical. Some are polished. All of them are political in the sense that they reveal (or conceal) the design choices and values behind these increasingly powerful systems.
System Cards and Safety: The Frontier Disclosure
OpenAI calls theirs system cards. So does Anthropic. Google went with model cards. The terms overlap, but the purpose is similar: they’re a kind of disclosure. A way to say: "Here’s what we’ve built, and here’s what you should know about it."
When you read through them, as I did this week, you start to see patterns:
- Benchmarks are everywhere. Some models outperform humans on complex tests, but still hallucinate simple facts.
- Safety claims are layered: technical fixes, refusal behaviour, internal red teaming, sometimes external audits.
- Many disclaim any true autonomy. Even the most capable models are described as "not agentic," "not self-aware," or "not capable of long-term planning."
- And yet, the cards often reveal that these same models can reason, plan, and follow goals in scaffolded environments.
It’s like looking at a machine that almost drives itself and seeing all the little disclaimers stuck to the dashboard.
How This Relates to Policy
In the EU AI Act, which I wrote about recently here, foundation models like GPT, Claude and Gemini are being put in their own regulatory category. The Act introduces specific documentation and transparency obligations — including the need to describe:
- Training data sources,
- Evaluation results,
- Risks and mitigations.
If you’ve read a good system card, you’ll recognise the format.
Model and system cards are no longer merely proto-regulatory documents. Under the EU AI Act, providers of general-purpose AI models must maintain technical documentation, provide information to downstream providers, adopt a copyright policy and publish a summary of training content. System cards often make some of that work visible, but they are not the compliance record itself. They show where the lines are being drawn — not just between companies, but between what’s considered safe, responsible, or questionable.
In the US, there is still no law equivalent to the EU AI Act. NIST’s AI Risk Management Framework remains a voluntary framework for thinking about documentation, testing and disclosure. Model and system cards are one useful public expression of that work, rather than its natural container.
Rob Hoeijmakers
And Then There’s Grok
At the time of Mollick’s post, Grok by xAI was the only one listed without a link. "????", he wrote. That absence was telling.
That has changed. xAI has since published system cards for Grok, including one for Grok 4.20 in April 2026. The contrast now is not between disclosure and silence, but between the detail, consistency and practical usefulness of the documents different labs choose to publish.
Why I’ll Keep Reading Them
This week, I’ve learned more from reading these cards than from any number of AI blog posts or press releases. They’re not perfect. They’re not always fully honest. But they are informative.
More importantly, they give you a vocabulary for thinking about capabilities, failures, responsibility and for asking better questions about the tools we’re integrating into our work, schools, and societies.
If you haven’t read one yet, start with the Claude 4 system card[¹]. Or OpenAI’s recent o4 mini system card[²]. They’re not just technical documents. They’re early signals of how AI governance is taking shape from the inside out.
¹ Claude 4 System Card, Anthropic: https://www.anthropic.com/index/claude-4
² o4 Mini System Card, OpenAI: https://cdn.openai.com/papers/o4-mini-system-card.pdf
³ Gemini 2.5 Deep Think Model Card, Google DeepMind: PDF💡