How to run an AI compliance audit
A working playbook for auditing AI systems against the EU AI Act, NIST AI RMF and ISO 42001 — without burning a quarter of consulting hours.
What is an AI compliance audit?
An AI compliance audit is a structured review of every AI system your organization builds or uses, measured against a chosen control framework. The output is a written opinion (internal or third-party) on whether those systems meet legal, contractual and risk obligations — plus a remediation plan for the gaps.
Unlike a SOC 2 audit, AI audits cover model behavior, training data provenance, inference-time data residency and human oversight — not just access controls and change management.
The five phases
1. Scope every AI surface
Catalog every model, vendor, prompt path and dataset. Shadow AI usage by junior staff is the single biggest audit gap we see.
2. Map controls to frameworks
Align each AI system to EU AI Act risk tiers, NIST AI RMF GOVERN/MAP/MEASURE/MANAGE functions and ISO/IEC 42001 clauses.
3. Collect evidence
Model cards, DPAs with US vendors, prompt logs, human-in-the-loop reviews, incident reports and access controls.
4. Run the gap analysis
Score each control. Flag high-risk gaps (missing DPAs, undocumented model changes, no rollback plan).
5. Remediate and re-test
Close gaps, document fixes, and schedule continuous monitoring — not just a once-a-year scramble.
Common audit failures
- Treating AI policy as a static PDF nobody updates after kick-off
- No DPA in place with US-based model providers
- Vendor risk register that hasn't been touched since onboarding
- Zero documented human-in-the-loop for high-risk decisions
- No audit log of prompt changes or model version upgrades