Model Card Best Practices for SOC 2 Mapped AI Vendors
Learn how to build model cards that satisfy SOC 2 audits and EU AI Act requirements with technical, evidence-driven documentation.
Model cards are compliance evidence, not marketing pages
Model card best practices for vendors pursuing SOC 2 come down to one shift: treat the card as audit evidence, not as a product page. That means documenting precise metrics, data lineage, and risk guardrails in a versioned format that an auditor can sample, trace, and verify — and that maps cleanly onto the technical documentation expectations emerging under the EU AI Act. A marketing-grade model card says the model is accurate and safe. An evidence-grade model card says which metric, on which evaluation set, at which model version, measured when, with which known failure modes. The difference decides whether the document helps you in an audit or embarrasses you in one. This post covers the fields that matter, the SOC 2 control themes they support, and the versioning discipline that keeps the card trustworthy.
A model card is a structured factsheet for one model
Model cards began as a research proposal for standardized model reporting and have since become a de facto industry norm — most public model releases now ship with one. The standard skeleton covers model details (name, version, type, training approach), intended use, evaluation data and results, training data, performance limitations, and ethical or safety considerations. For compliance purposes, add three fields the research template underweights: the named owner accountable for the card's accuracy, the date of last review, and a change log. A card without an owner and a review date is unverifiable by construction, and auditors treat unverifiable documents as absent. Keep one card per deployed model version, not one card per product; a product-level card that blends several models cannot answer version-specific questions.
State intended use and out-of-scope use explicitly
The intended-use section is where legal exposure concentrates, so write it with the same care as a contract. State what the model is designed to do, in which domains, for which user populations, and under what human-oversight assumptions. Then state what it must not be used for, concretely: not for consequential decisions without human review, not for populations absent from the evaluation data, not for inputs in unsupported languages. Vague out-of-scope language ("should not be used for harmful purposes") protects no one. Specific out-of-scope language does two jobs — it steers deployers away from misuse, and it documents that the vendor identified the risk in advance, which matters in any post-incident review. If the model is a component inside a larger system, say which system-level mitigations the intended-use claims assume.
Data provenance is the section auditors probe first
Auditors and enterprise buyers go to the training-data section first because it is where vague cards fail fastest. Document where training and fine-tuning data came from: named datasets with versions, collection methods and date ranges, licensing or consent basis, and what preprocessing or filtering was applied. Where data is proprietary or sensitive, describe it at the level of category, source type, and governance controls rather than not at all — "we cannot disclose" reads very differently from "customer support transcripts, collected under contract terms permitting model improvement, with personal data removed by a tested pipeline." Record data lineage as a chain: source, transformation, storage, use. When a data problem surfaces later — a license dispute, a contamination claim — this section determines whether you can scope the damage or must assume the worst.
Report evaluation results someone else can reproduce
An evaluation claim is only evidence if it can be checked. For every headline metric, record the evaluation dataset and its version, the metric definition, the model version tested, the date, and the harness or configuration used. Report performance disaggregated across the segments that matter for the model's intended use — languages, demographic groups, input types, difficulty bands — not just a single aggregate number, because aggregates hide exactly the failures that become incidents. Include results the marketing team would rather omit; a card that shows only wins signals selection, and auditors are trained to notice. Where an evaluation set is private, describe its construction and size so the result can at least be interrogated. Reproducibility is the property that separates a measurement from an anecdote.
Write limitations as engineering facts, not disclaimers
The limitations section fails when it is written defensively. "The model may sometimes produce incorrect output" is a disclaimer; "accuracy degrades measurably on inputs longer than the training distribution, and the model produces confident output on out-of-domain inputs rather than abstaining" is an engineering fact a deployer can act on. List known failure modes with the conditions that trigger them, quantified where you can. Document the guardrails that exist — input validation, output filtering, refusal behavior, rate limits — and what each one does and does not catch. This section also carries the honest boundary of your own knowledge: state which failure classes have not been tested. Auditors do not penalize documented limitations; they penalize limitations discovered in production that the vendor plainly knew about and left out.
Model cards map onto SOC 2 control themes
SOC 2 is an attestation, performed by a licensed CPA firm, against the AICPA Trust Services Criteria; it does not name model cards anywhere. The mapping works at the level of control themes. A versioned model card supports change-management controls, because it shows that material changes to the system are documented and reviewed. The limitations and intended-use sections support risk-assessment controls, because they show the organization identifies and evaluates risks arising from its own technology. Evaluation results tied to model versions support monitoring controls. During a Type II audit — which tests whether controls operated over a review period, not just at a point in time — auditors sample evidence across that period. A card created the week before fieldwork covers nothing; cards versioned continuously alongside releases cover everything.
Version the card with the model, under change control
A model card that lags the model it describes is worse than no card: it is affirmative documentation of the wrong system. Put cards under the same change control as the model artifacts. A model version bump that changes training data, evaluation results, or intended use must not ship until the card is updated and reviewed, and the card's change log should record what changed, who approved it, and when. Store cards in a system with history — a repository, not a shared document — so any past state can be reconstructed, because audits and incident investigations ask about the system as it was, not as it is. Assign ownership to a named role. Review on a fixed cadence even without changes, and record the review; a dated "reviewed, no changes" entry is itself evidence.
One card can serve SOC 2 and EU AI Act documentation
The EU AI Act requires providers of high-risk AI systems to maintain technical documentation covering, in general terms, the system's design and purpose, its data and data governance, its performance characteristics, and its human-oversight and risk-management measures. That list overlaps heavily with an evidence-grade model card, which is the point: build the card once, rigorously, and let it feed both the SOC 2 evidence base and the Act's documentation requirements rather than maintaining parallel documents that drift apart. The Act's implementation details and timelines have continued to move, so treat specific deadlines as a question for counsel, not a blog post. The practice that survives every regulatory revision is the same one that survives every audit: precise claims, versioned records, named owners, and nothing in the card you cannot prove.