When Your AI Has an Incident, Who Calls the Board?

The Headline Your Board Saw

Gallagher released its third annual AI Adoption and Risk Survey this week. The lead number: 63% of organizations say they’ve fully operationalized AI, up from 45% just a year ago. A lot of production systems. A lot of real business decisions running through models that, twelve months ago, were still in pilots.

Here’s the second number from the same report: only 43% have an AI incident response plan.

So more than six in ten organizations are running AI in core operations, and fewer than half have written down what they’d do if something went badly wrong. The Stanford AI Index recorded 233 harmful AI incidents in 2024, a 56% increase year-on-year. The systems are scaling. The preparedness is not keeping pace.

What It Actually Means

The preparedness gap is real, but it’s the confidence gap sitting underneath it that tells the more interesting story.

According to the same Gallagher survey, 93% of executives say they understand AI risks “quite well” or “very well.” Only 43% have an incident response plan. That’s a 50-point spread between feeling prepared and having done the preparation work. Organizations are measuring governance by intent, not by operational reality.

A second report, published this week in eSecurity Planet, found the same pattern in AI visibility. 90% of organizations believe they have complete visibility into their AI systems. Meanwhile, 59% of those same organizations acknowledge that employees are using unapproved AI tools. Those two numbers cannot both be true. And yet.

Gallagher’s managing director Ben Warren put the root cause plainly in the survey:

When AI governance sits inside the IT team, it gets evaluated against technology criteria: uptime, access controls, security patches. But when an AI system produces a harmful output for a customer, an employee, or a regulated process, what follows is not a technology incident. It’s a business incident. The legal response, the PR response, the regulatory response, and the leadership response all need to be in motion before the call comes in, not improvised after.

One more signal worth noting. According to Conference Board data published this week on the Harvard Law School Corporate Governance blog, 46% of new S&P 500 directors now have technology experience, up from 17% in 2021. The board is getting smarter about AI governance faster than most management teams are getting ready for the questions. That gap is narrowing, and not in the direction that favors an executive who hasn’t prepared.

Three Questions Your Board Will Ask

I’ve been in enough board prep sessions to know which AI governance questions are coming right now. These three have near-universal attendance.

“Do we have an AI incident response plan, and have we tested it?”

Having a document is not having a plan. A tested plan means someone can describe concretely: what triggers a response, who gets called in the first 24 hours, what gets paused or rolled back, what communication goes to customers and regulators, and who owns the post-mortem. Walking through those steps in a prep session reveals gaps much faster than any audit. Better to find them now.

“What’s in our AI inventory, and who is accountable for each system?”

The 86% of organizations claiming a complete AI inventory are about to collide with the 59% shadow AI problem. The board won’t just want the official list. They’ll want to understand whether it reflects what’s actually running. Any AI system causing harm that wasn’t on the inventory is one that leadership cannot credibly claim it was managing. That’s a distinct kind of exposure, and it can’t be patched retroactively.

“When something goes wrong, where does the accountability sit?”

This is the question that produces the longest silence in most executive teams. The answer cannot be “the vendor” and it cannot be “the model.” Someone in the organization owns the outcome when an AI makes a consequential mistake. The Conference Board now recommends that AI governance competence be a formal factor in CEO performance evaluation. That recommendation is new this year. It’s worth taking seriously.

The 60-Second Brief

If you had one minute with your board on this topic: AI incident preparedness is the governance gap that matters most right now. Most organizations have operationalized AI. Fewer than half have a tested response plan for when something goes wrong. Boards that are actively governing AI are asking about incident protocols, accountability structure, and inventory completeness. I’d recommend spending 30 minutes this quarter walking through what the first 48 hours would look like if your most consequential AI system produced a harmful output. Not because an incident is inevitable. Because rehearsing the answer now is materially better than improvising it later.

The companies that handle AI incidents best are the ones who walked through the first 48 hours before anything went wrong.

What to Watch

The 46% of new S&P 500 directors with technology experience are the ones writing the governance questions of the next board cycle. Watch for AI incident response becoming a standard board disclosure item, the way cyber incident response did after the major breaches of 2017 and 2018. That normalization took about three years from early adopters to industry expectation. The timeline on this one looks shorter.