Most AI frameworks fail the operating test
- benjamin. brl
- 7 juil.
- 5 min de lecture
Most AI frameworks fail the operating test
I spent last weekend going deep on AI maturity frameworks.
NIST AI RMF. ISO/IEC 42001. TOGAF. DAMA-DMBOK. DCAM. Microsoft, AWS and Google cloud adoption frameworks. Gartner AI TRiSM. MuShuHaRi.
If some of those names mean nothing to you, start with my last issue.
The surprising part was not how many exist. The surprising part was how few survive contact with real operations.
Most frameworks are useful for the question they were designed to answer. Risk frameworks help leaders think about trust, monitoring and accountability. Data frameworks bring the foundation back into the discussion. Architecture frameworks connect ambition to systems. Cloud adoption frameworks help translate strategy into platform and deployment capabilities. Maturity frameworks help companies understand where they are.
But after reading through them, I kept coming back to a different question.
Can this framework help a leadership team decide what should change inside the company next?
Not what should be assessed. Not what should be documented. Not what should appear in a maturity score.
That is the operating test.
A framework passes the operating test if it changes what the company funds, stops, or sequences next.
Fund, stop, sequence. If a framework can't move one of those three, it isn't strategy yet.
Structure is not strategy
The trap is believing that a complete framework creates a complete strategy.
It does not.
A company can have a roadmap, a steering committee, a risk policy, a data governance program, a cloud architecture and a portfolio of AI use cases. It can still have no AI strategy.
It has structure. That is different.
Structure tells you that the right topics are on the table. Strategy tells you which decision changes first.
This matters because AI does not fail in neat categories. It fails in the spaces between them. The business wants the use case now. The data team says the source is not reliable. The legal team asks who is accountable. IT says integration will take six months. The pilot team says the model works. Finance asks where the value appears.
Everyone may be right. And the initiative still cannot scale.
That is the operating reality most frameworks underestimate. They help the company sound mature before the company is ready to operate differently.
Applying the test
I watch leaders get lost between use cases. Excited, but unsure what to start first. Quick wins, or revenue? That choice is the strategy, and most frameworks stay silent on it.
The question is not only "where are we on the maturity curve?" The question is "what does this maturity level allow us to do now?" The question is not only "which controls should exist?" The question is "who has the authority to stop the system if the risk changes?" The question is not only "which data capabilities are missing?" The question is "which use cases should wait until those capabilities exist?"
This is where many AI strategies become fragile. They start with the model, the tool, the platform or the use case. They ask what AI can do before asking what the organization is ready to absorb.
The result is familiar. A customer service team launches a support copilot that drafts replies. In the pilot it handles seven out of ten tickets well, and everyone celebrates. Then it meets the real system. No one owns the rule for when it should hand a conversation to a human. The CRM integration that would let it see a customer's order history is six months out. Legal has not signed off on what it is allowed to promise. So the copilot stays in the pilot environment, impressive and idle, while the team keeps answering tickets by hand.
It looks like progress.
But activity is not capability.
The operating test is designed to make that visible. If a framework cannot tell you what to prioritise, fund, stop, or sequence, it may still be useful. But it is not yet a transformation tool.
What fails
The frameworks that fail the operating test are rarely weak. Most of them are intelligent.
They describe what maturity looks like, but not the order in which an organization earns it. They separate risk, data, architecture, governance and adoption, while real companies have to make those layers work together. They interest the CTO more than the CEO, which is usually the moment AI becomes a technical workstream instead of a business transformation agenda.
They also assume readiness too quickly. They move from ambition to scale without testing whether the business can actually absorb the change. But AI maturity is not the number of initiatives running. It is the organization's ability to make AI repeatable inside the business.
That difference matters.
A pilot can work outside the system. A strategy cannot. To create value, AI has to enter workflows. It has to connect with applications, processes, roles, controls and decisions. It has to be monitored, updated, secured and audited. It has to keep working when the pilot team is no longer in the room.
This is why so many AI roadmaps feel convincing in slides and fragile in execution. They describe ambition. They do not test readiness.
What survives
The frameworks that survive do one thing well: they tell you what to prioritise, fund, stop, and sequence, and in what order. And they do not isolate a single topic from the rest of the company.
They do not let leaders confuse activity with capability. They force the company to ask what must be true before the next AI move is allowed.
Before a use case scales, what business process does it actually change? Before a model supports a decision, who owns the decision? Before customer-facing AI is launched, what happens when it fails? Before another pilot is funded, where is the verified value from the last one? Before the board is asked to approve the next AI budget, which number can the CFO already see?
The sequencing discipline I find most useful here comes from one of the frameworks on that list. More on that soon.
That is the discipline many companies are missing. They do not need another framework to tell them AI is strategic. They need a way to decide what should happen first, what should happen later, and what should not happen yet.
Where this is taking my practice
This is also where my own advising practice is becoming clearer.
I am less interested in helping companies collect more AI frameworks. I am less interested in maturity assessments that produce another score. I am less interested in AI strategies that become technical workstreams.
The work is different. It is translating AI maturity into operating sequence.
Which business priority comes first? Which data foundation is missing? Which information system pathway blocks scale? Which governance owner must exist before deployment? Which operating rhythm needs to change? Which initiative should wait because the organization is not ready? Which number should the CFO be able to verify?
That is where AI strategy becomes real.
Not in the model. Not in the framework. In the order of decisions.
A CEO does not need another AI maturity score. A CEO needs to know what to fund first, what to stop, who to hold accountable, which risk to accept, and what it takes to reach the next AI stage and why.
That is the difference between an AI framework and an AI strategy.
One organizes thinking. The other changes the company's next move.
One question for your next executive committee
Look at your current AI priorities.
Which order are you following, and why?
Not because the model works. Not because the pilot is promising. Not because the roadmap says scale.
Because it builds the foundation for the next move, and your CFO can verify the value.
What remains is probably your AI strategy.
The rest may only be AI activity with a framework around it.
Commentaires