Assurance and Control Objectives for AI
How to Assess AI Control
Five types. Two axes. No badge before a system of control.
Most AI "assurance" conversations start at the wrong end. They ask which certificate to buy, which annex to tick, which self-assessment template to fill. Those are later questions. The first question is whether a system of internal control exists at all - control objectives, then controls, then a map of scope. Only after that work exists does it make sense to ask how the system will be examined.
We use five assessment types. They are not five logos and they are not five strengths of the same test. Four of them sit on a reliance ladder: how much may the assessor take from management? The fifth sits on a different axis: whose criteria are applied? Mixing those axes is how a young AI estate certifies itself.
What we refuse to treat as a control framework
A Statement of Applicability is a scope artefact. It records which determined controls apply to which use, and which do not. It is related to ISO practice. It is not the system of internal control, and it is not an assessment of that system. Writing a SoA before the controls exist is how the map becomes the territory in the file and nowhere else.
A vendor badge, a locked catalogue claim and a completed questionnaire are likewise not assessments. They may be outputs of an assessment. They do not create what is to be assessed.
Two axes
Every engagement we take is classified on both axes before fieldwork starts. The file states the assumed condition of the local control environment, what may be relied on, and whose criteria will be used. If those three lines cannot be written honestly, the type drops.
| Axis | The question |
|---|---|
| Reliance | How much may the assessor rely on management's process and assertions? |
| Criteria | Are we testing this organisation's own outputs, or applying published criteria that exist whether or not this organisation asserted them? |
The reliance ladder
Control self-assessment
CSA is management examining its own controls against stated control objectives. It relies on the first and second lines. That is the point of it, not a weakness to be dressed as independence. Policy, regulation and standard checks are a subset of a CSA. They are not the whole.
We treat CSA as an assertion, not as assurance. It belongs with internal assessment. The people who write it do not write the independent opinion.
Attestation audit
Used when the local control environment is already known to be good. The auditor relies on management's risk-and-control process and attests to the adequacy and reasonableness of that process. Attestation is not a first engagement for a new agent. It is what becomes available after the process has already been confirmed.
Confirmation audit
Used when the environment is fair. Management's assertions are the starting point. They are tested. The work is on completeness and accuracy of this estate's outputs: the residual-risk profile, the SoA after controls exist, the logs, the evaluation packets for this use. Confirmation does not apply a foreign catalogue. It asks whether what this organisation said about itself holds.
Direct audit
Used when the environment is poor or unknown - which is the normal condition of a first AI estate, a first agent, or a first Annex III use. No reliance is placed on management's assertions. The auditor builds an independent view of whether the control objectives are met. Direct is not a punishment. It is the honest type while the environment is still being built.
The other axis: independent evaluation
Independent evaluation is not Confirmation with a different label. Confirmation tests this organisation's asserted outputs. Independent evaluation applies generally applicable published criteria - a locked control catalogue, a harmonised standard, a scheme protocol - that exist whether or not this organisation asserted anything.
The preconditions are independence and published criteria, not a mature environment. A catalogue campaign is often commissioned precisely because the environment is not yet known to be good. The evaluator does not write the CSA, does not accept residual risk, and does not occupy the overseer seat. A badge is an output of this type. It is not a substitute for it.
How we choose
| Type | Environment | Reliance | What is examined |
|---|---|---|---|
| CSA | Known, adequate for self-assessment | Management's process and supervisory controls | Design and operating effectiveness against control objectives |
| Attestation | Good | Management's process | Reasonableness of that process |
| Confirmation | Fair | Assertions, after testing | Completeness and accuracy of this estate's outputs |
| Direct | Poor or unknown | None | Whether control objectives are met |
| Independent evaluation | Criteria exist; evaluator is independent. Environment may still be forming. | Not management's criteria | Baseline against the published set |
The choice follows the environment you have, not the logo you want. For most organisations first implementing AI control, the honest starting internal type is closer to direct than to attestation. Independent evaluation may run in parallel if a catalogue is claimed. It does not upgrade the environment.
What we sample when the system is an agent
An agent that plans across steps, calls tools and keeps memory will produce a coherent final draft. That draft is not evidence of oversight. Whatever type we use at runtime, the sample includes reserved decisions the agent cannot skip, a reconstructable trace, and a halt the system cannot override. If hardship or discretion never reached a human, Article 14 has already failed - even if a name is on the last page.
Evaluative review after the fact cannot compensate where the harm is irreversible. That is an implementation fact. It is also an assessment fact. We do not treat a quarterly log review as a substitute for a missing checkpoint.
Order of work
- Set control objectives.
- Apply the intended-purpose, state-of-the-art and proportionality lens before controls are chosen.
- Determine operating, supervisory, administrative and fiduciary controls. Prefer design-time prevention.
- Embed those controls in running processes.
- Then map scope. That is the SoA.
- Then choose the assessment type from the environment that actually exists.
Assessment last is not a courtesy to the assessor. It is the only sequence in which an assessment can be about something real.
What we will not do
- Attest a process that has never been confirmed.
- Treat a CSA written by the team that shipped the agent as independent assurance.
- Let the authors of the CSA perform the independent evaluation.
- Accept a SoA as proof that controls exist.
- Take a vendor's claim as residual-risk acceptance. That acceptance stays here.
A working test
Can you name the control objective, the controls that serve it, the human who can stop the system, and the assessment type that matches the environment you actually have? If the last of those is "attestation" and the first three are still being written, the type is wrong. Change the type. Do not decorate the file.
This is Our approach. The rest is commentary on it.
