Why boards are being offered confidence about AI when what they need is assurance
A director asks a reasonable question at a board meeting: are we comfortable with how AI is being used across the business?
Management answers yes. The answer is sincere. It is delivered by people who are competent, who have thought about the issue, and who have no intention of misleading anyone. A slide may accompany it. The minute records that the board discussed AI usage and was satisfied.
Six months later, a regulator, an external auditor or a significant client asks a narrower question: show us.
At that moment the difference between two words becomes expensive. Confidence is what management feels. Assurance is what an independent party can evidence. Most organizations have been quietly accumulating the first while assuming they were building the second.
1. A word doing too much work
“Assurance” has become one of the most overused words in corporate reporting, and its meaning has drifted to the point of being decorative. It now routinely describes a management update, a vendor’s marketing claim, a policy document, or a verbal comfort given at a meeting.
None of those is assurance in any sense a board can rely on. Assurance has a specific structure, and it has always had one:
Somebody with no stake in the outcome examined the position, against a defined standard, using evidence, and reported what they found — including what they could not conclude.
Strip out any element and something else remains. Remove independence and you have management reporting. Remove the standard and you have opinion. Remove the evidence and you have assertion. Remove the willingness to report what could not be concluded and you have advocacy.
The question a board should ask about any AI comfort it receives is not “is this reassuring?” It is “who says so, on what basis, and could they have said otherwise?”
2. Three positions, frequently confused
In a well-run organization three distinct things happen, and they are performed by different people for different reasons.
Management operates and asserts. The people running the process say it is working. This is necessary and legitimate. It is also, structurally, the least independent statement available.
A second line monitors. Risk, compliance and control functions check that the process is being followed. Closer to objective, but still inside the reporting line that owns the outcome.
A third position verifies independently. Internal audit, or an external party, examines whether the assertion holds when tested against evidence — and is organizationally able to report that it does not.
With AI, most organizations currently have only the first. Innovation moved faster than the control functions, the second line has not yet developed the competence to challenge model behaviour, and internal audit is frequently being asked to cover AI without having been resourced to do so.
That is not a criticism of anyone. It is a description of where the region is, and it is exactly why the gap is worth naming rather than papering over.
3. Can you reconstruct the decision?
Here is the single most useful test we apply, and any director can apply it too.
Pick one decision in the last quarter where an AI system contributed to the outcome. A credit assessment. A customer communication. A fraud alert that was dismissed. A shortlist of job applicants. Then ask for the file.
To reconstruct that decision properly, seven things must be retrievable:
The seven links in an AI decision record
- The input data — what actually went in
- The model version — which system, at which version, on that date
- The parameters — how it was configured or prompted
- The output — what it actually said or scored
- The human review — who looked at it, and what they did
- The action taken — what changed as a result
- The record — all of the above, still retrievable now
Break any link and the decision cannot be reconstructed. And a decision that cannot be reconstructed cannot be evidenced, cannot be reviewed, and cannot be defended — to a regulator, to a court, to an auditor, or to the customer it affected.
Most organizations discover, on trying this exercise once, that they can produce the output and the action, and almost nothing else. The model has since been updated. The prompt was never captured. The reviewer is remembered but not recorded.
That gap is not theoretical. It is the difference between a defensible decision and an indefensible one, and it is invisible until somebody asks.
4. The seven questions
An organization’s assurance position over AI can be diagnosed with seven questions. They are the questions a board is entitled to have answered, and the ones an audit committee should be putting in the papers rather than raising in discussion.
- Do we know where AI is being used?
- Are AI systems properly governed?
- Are AI outputs validated?
- Are human oversight controls effective?
- Are audit trails maintained?
- Are AI vendors properly assessed?
- Can management evidence responsible AI adoption?
The seventh question is the one that matters, and it deliberately does not ask whether adoption is responsible. It asks whether it can be evidenced — because that is the only version of the question anyone outside the organization will ever ask.
5. Human oversight that isn’t
Almost every AI policy we review contains a sentence to the effect that AI outputs are subject to human review. It is the control everyone reaches for first, because it sounds decisive and costs nothing to write down.
Tested, it is frequently the weakest control in the environment.
Human oversight is meaningful only when three conditions hold. The reviewer must have the information needed to disagree — if they cannot see why the system produced the output, they cannot evaluate it. They must have the time; a reviewer processing two hundred items an hour is a formality, not a control. And they must have the standing to overturn the output without it counting against them, which is a cultural question rather than a procedural one.
Where those conditions fail, what exists is not oversight but ratification — and ratification transfers accountability to a human being without transferring any real control.
The test of human oversight is not whether a person reviewed the output. It is whether that person could realistically have reached a different conclusion, and whether the record would show it if they had.
Proportionality matters too. A person tidying the language of an internal memo does not need a documented review trail. A person approving an AI-assisted credit decline does. Policies that apply the same review standard to both produce reviewers who skim everything.
6. Validated against what?
The second common gap is output validation, and it is usually absent rather than weak — because validating a model requires a standard to validate against, and most organizations have never set one.
Accuracy against what benchmark? Reliability across what range of inputs? Completeness by whose definition? These are answerable questions, but they must be answered before deployment, not after an incident.
Then there is drift — the quiet problem. A model that performed acceptably at implementation will not necessarily perform acceptably eighteen months later, because the data it meets has changed, the vendor has updated the underlying system, or the business has started using it for something adjacent to its original purpose. Nothing announces this. Performance degrades silently, and the first evidence is usually a complaint.
An organization that validated once, at go-live, and has not tested since does not have a validated model. It has a model that was validated on a date that is receding.
7. The monitoring illusion
Continuous monitoring is the third area where the language outruns the reality.
Dashboards exist. Exception reports are generated. Incidents are logged somewhere. And yet, when we ask who reviewed the AI exception report last month and what they did about it, the answer frequently reveals that the report is produced, distributed, and read by nobody with the authority to act.
Monitoring is a control only when three things are true: something is measured, someone is accountable for looking at it, and something happens when the measure moves. Most AI monitoring in the region satisfies the first condition and stops there.
The corrective is unglamorous. Name the metric. Name the person. Define the threshold that triggers action. Put the exceptions — not the dashboard — into the management review, and put the unresolved ones into the board pack.
8. Where internal audit stands
There is a particular pressure on internal audit functions right now that deserves saying plainly.
Audit committees are asking for AI coverage. The IIA’s Cybersecurity Topical Requirement has raised the bar on what internal audit is expected to address around governance, risk management and control processes. And most internal audit functions in the region were sized and skilled for a world that did not include model validation, prompt-level exposure or vendor AI clauses.
The honest position is that many CAEs are being asked to provide assurance over something they have not been resourced to examine. Pretending otherwise serves nobody.
There are three workable responses. Build the capability, which is right long term and slow. Co-source a specialist review that internal audit scopes, owns and reports — capability arrives immediately and stays partly behind. Or decline coverage explicitly, in writing, so the audit committee knows the area is unassured rather than assumed to be covered.
The third option is far more respectable than it sounds. An audit plan that quietly implies coverage it cannot deliver is worse than one that names the gap.
9. What a readiness review actually produces
An AI Assurance Readiness Review is a bounded exercise with a specific output. It is not a strategy engagement, and it does not implement anything.
It produces a readiness report setting out what was examined and what was found. A control maturity assessment rating each area — governance, inventory, data, cybersecurity, validation, human oversight, auditability, monitoring — on a five-level scale, so the position is a location rather than a mood. A governance gap analysis. An auditability and evidence trail review, which in our experience surfaces more than any other section. Cybersecurity and data protection findings. A board-ready assurance summary — one page the audit committee can read in two minutes. And a practical remediation roadmap with named owners and dates.
Illustrative control maturity assessment across the eight review areas
Three to six weeks, delivered through interviews, documentation review and control testing.
The purpose of the maturity rating deserves emphasis. Its value is not the score. Its value is that it converts an anxious, open question — are we exposed? — into a bounded one: we are here, the next level requires these specific things, and they are owned by these people.
A three-to-six week engagement with a defined end point
10. Before you scale, not after
The timing argument is the practical one.
Most organizations in the region are in the same position: AI is present, growing, and about to be extended into something more consequential than it currently touches. The controls that were adequate for a drafting assistant will not be adequate for a system that scores customers.
Testing the control environment before that step is cheap. Testing it afterwards, following an incident, a complaint or an examination, is expensive — and by then the review is being conducted by someone else, on their timetable, with their questions.
There is a version of this that ends well. The board asked for assurance. Someone independent examined the position against a standard, using evidence. The findings were uncomfortable in places, owned by name, and closed on a date. And the next time anyone asked show us, the answer took an afternoon rather than a quarter.
That is not a technology outcome. It is a governance one — and it starts with a board being willing to ask, of every comfortable answer it receives about AI: who says so?
Ten questions for your next board or audit committee meeting
- When we are told AI use is under control, who is making that statement — and are they independent of it?
- Against what standard has our AI control environment been assessed?
- Could we reconstruct one AI-supported decision from the last quarter, end to end?
- Where human review is a stated control, could the reviewer realistically have disagreed?
- What were our AI models validated against, and when were they last retested?
- Who reads the AI exception report, and what happened the last time it flagged something?
- Does our internal audit plan cover AI — and is it resourced to?
- Which AI use cases will become more consequential in the next twelve months?
- What is our current AI control maturity, and what would move us up one level?
- If a regulator asked us to evidence responsible AI adoption tomorrow, how long would it take?
Frequently asked questions
How long does an AI Assurance Readiness Review take?
Typically three to six weeks, depending on the number of AI use cases and business units in scope. Delivery is through interviews, documentation review, control testing and executive reporting.
Is this an audit?
It is an independent readiness review conducted with audit discipline — findings are tested, evidenced and rated. It can be structured to feed directly into an internal audit opinion, or to stand alone as a management and board deliverable.
We have an internal audit function. Does this duplicate it?
No. It is most often run as a co-sourced review that internal audit scopes and owns, bringing specialist capability to an area the function may not yet be resourced for. The output strengthens internal audit’s position rather than competing with it.
Do we need mature AI systems before this is worthwhile?
No — and earlier is better. Reviewing the control environment before AI is scaled is materially cheaper than remediating after it has been.
What if the findings are poor?
Then you have learned it from us rather than from an examiner, on your timetable, with a remediation roadmap attached. A review that finds nothing has usually not looked hard enough.
Who typically commissions this?
Audit committees, boards, chief audit executives, chief risk officers and chief executives. The most common trigger is a request for assurance that nobody can currently satisfy.
Download the brochure
Dawgen Global’s AI Assurance Readiness Review covers eight review areas — governance and accountability, AI inventory and risk classification, data governance and privacy, cybersecurity controls, model and output validation, human oversight, auditability and evidence trails, and continuous monitoring — and delivers a board-ready assurance summary with a control maturity rating and a practical remediation roadmap.
Download the service brochure for the full scope, deliverables and engagement format.
| REQUEST THE REVIEW
[email protected] | dawgen.global/contact-us 47 Trinidad Terrace, New Kingston, Jamaica | 876-929-3670 | 876-665-5926 | US 855-354-2447 |
About Dawgen Global
Dawgen Global is an independent, integrated multidisciplinary professional services firm headquartered at 47 Trinidad Terrace, New Kingston, Jamaica, serving more than 15 territories across the Caribbean. Founded and led by Dr. Dawkins Brown, Executive Chairman, the firm is independent and not affiliated with any international network. It delivers a full suite of professional services under one roof: audit and assurance; tax advisory; IT and digital transformation; risk management; cybersecurity; actuarial and insurance regulatory advisory; HR advisory; mergers and acquisitions; corporate recovery; business advisory and strategy; accounting BPO and virtual CFO services; and legal process outsourcing.
The proposition is simple: big-firm capability without the big-firm price. Dawgen Global’s integrated approach is built for the specific complexities and opportunities of the Caribbean market, helping organizations make sharper, better-informed decisions that drive measurable progress.
To explore a partnership, reach out:
- Website: dawgen.global
- Email: [email protected]
- WhatsApp (Global): +1 555-795-9071
- Caribbean offices: +1 876-665-5926 | +1 876-929-3670 | +1 876-926-5210
