- AI maturity is not how much AI you use. A use case is mature when it can prove what it delivers.
- Measure it on the six branches of our AI Metrics Tree: Impact, Cost and Quality (what AI delivers), and Adoption, Data and Control (what it runs on).
- Start in four steps: Map what’s running, Gate every use case, Score it on the six branches, Plan what to scale, fix or switch off.
In this article
If your CEO asked tomorrow what your AI is actually delivering, could you answer? In most organisations we talk to, the honest answer is “not really”. There are tools, pilots and licences everywhere, but no clear answer to the question every board asks sooner or later: what is it worth?
At Joyful Craftsmen we’ve spent more than 12 years building the data and reporting foundations companies run on, and AI is now part of that work. Now, we dive deep into a practical guide: why the value gap matters, why AI agents make it urgent, and how to measure AI maturity with our six-branch model.
What is AI maturity?
AI maturity is the degree to which an AI use case, or a company’s AI as a whole, can prove what it delivers. Readiness tells you whether you can start, and where. Maturity tells you whether it’s working.
“
A use case is mature when it can prove what it delivers.
Why AI activity doesn't show AI value
Most organisations can tell you how much AI they use. Far fewer can tell you what it does for them. When we ask leaders what worries them, the same three things keep coming up:
- It’s all moving too fast. New models, rising costs and new rules land every month, and keeping an eye on it all gets harder in every department.
- People aren’t sure about it. Employees need to use these tools well, often while quietly wondering what it means for their jobs.
- Shadow AI. People, and increasingly AI agents, are doing things nobody signed off on. Who’s using what? With which data? And who’s accountable when it goes wrong?
Here’s what we’ve learned: when AI projects stall, it’s rarely the technology. It’s a missing owner, a sponsor who moves on, or a value case nobody wrote down. AI projects fail on people long before they fail on models.
That’s why we’d never start a use case without three things in place: an agreed return, a named owner and a plan for getting it into production. A slick demo built because someone senior asked for one doesn’t count.
And “value” needs to mean more than a ROI number. Think about what a use case costs, how much time it frees up, how good its output is, how good the data behind it is, what it does for customers and what new risks it brings. If you don’t know where you started, you’ll never be able to show how far you’ve come.
Why it matters now: AI agents are moving into everyday work
The era of endless pilots is over. The conversations we have with clients have moved from “should we try AI?” to “how do we run AI agents alongside our people?” AI agents are leaving the lab and taking on real work: drafting, checking, reconciling, routing.
That changes the shape of a job. Less doing every step yourself, more orchestrating. People bring the judgment, the context, the accountability and the creative spark. AI agents do the analysis, the first drafts and the heavy lifting. And clear rules sit in between: when to escalate, how often to review, what good quality looks like.
There’s a leadership question hiding in here too. AI frees up capacity, but it doesn’t decide what to do with it. You do: save costs, grow, or rethink how the work gets done altogether.
As AI agents scale, we see three management jobs becoming more important than ever:
- Deciding which data and systems each agent can touch.
- Knowing what every agent costs and what it delivers.
- Keeping an eye on the quality of what agents produce.
In other words, the question is no longer “are we using AI?” It’s “do we know what it’s worth, who’s using it and who’s in charge?”
The AI Metrics Tree: 6 ways to measure AI value
The AI Metrics Tree measures AI value on six branches: three for what AI delivers (Impact, Cost and Quality) and three for what it runs on (Adoption, Data and Control). Look at it from two angles: each individual use case, and the company as a whole. Here’s what good looks like on each branch, what to do about it, and what to keep an eye on.
What AI delivers: Impact, Cost and Quality
Branch | What good looks like | What to do | What to measure |
Impact | Every use case proves its effect on the business | Write the value case before you build; set a baseline at day 0; track outcomes, not logins | Realised vs planned impact |
Cost | You know what each use case and agent costs to run | Assign model, platform and people costs to each use case; review the most expensive ones regularly | Cost per use case, cost per outcome |
Quality | Output is good enough to rely on, and it gets checked | Define what good output looks like per use case; build in human review and escalation thresholds | Error and rework rate, escalation rate |
What AI runs on: Adoption, Data and Control
| Branch | What good looks like | What to do | What to measure |
| Adoption | The right people use it well and trust it | Train by role, including how to review AI output; share use cases across teams; talk openly about roles and job fears | Active use in target roles, reuse of shared use cases |
| Data | Data that is accessible, trusted and owned | Fix data quality and access for the priority use cases first; keep one inventory of every AI system and agent | Data quality in priority domains, inventory coverage |
| Control | Clear ownership and guardrails people can work within | Give each use case an owner and a sponsor; approve tools and data; give agents least-privilege access; register Shadow AI rather than block it | Use cases with an owner, agents with defined access, unregistered tools found |
Of the six branches, Data quietly decides the other five. If you can’t trust the data an agent works with, you can’t trust its output, and you certainly can’t prove its value. Most of the “AI problems” we see turn out to be data problems wearing a new hat.
How to start measuring AI maturity in 4 steps
You don’t need a big program to get a first honest picture. Four steps will take you a long way.
1. Map what AI is running
What AI is actually running across your organisation, on which platforms, and who built it? Most companies are building AI agents on more than one platform, so get them all on one list, ideally automatically. And set guardrails rather than bans, so your teams can experiment without wandering off the map.
2. Gate every use case
Does each use case have an owner, a value case, a deployment plan and a sponsor who’ll stay? Every use case needs someone who owns it, a clear goal and foundations that can actually carry it.
Treat this as a gate, not a survey. A readiness check only earns its keep if it’s allowed to say no. It takes nerve to hold that line, and it pays off every time.
No owner, no return, no deployment plan, no project.
3. Score it on the six branches
How does each use case, and the company as a whole, score on Impact, Cost, Quality, Adoption, Data and Control? Measure to make decisions, not to fill a report. Knowing how many people have a Copilot licence tells you nothing about value. Look beyond ROI, and set your baseline before you go live, not after.
Score the portfolio, not just the pilot. One successful pilot tells you very little. The real questions sit across use cases: which three deliver most of the value, which ones overlap, and which agent got built twice in two departments.
4. Plan: scale, fix or switch off
What did things look like before AI, and which use cases should you scale, fix or switch off? Design for people and AI agents together. Automate a bad process and you just get a faster bad process, so fix the process first. Decide where the freed-up time goes. And get your experts ready for a new job: more and more, they’ll be reviewing what AI produces. Checking is a skill, and it’s worth investing in.
Get a first picture of your AI maturity
People and AI agents working side by side isn’t the future. It’s already here. The organisations that do well in it won’t be the ones with the most AI, but the ones that know what their AI is worth, and keep asking.
We’re working through these questions with clients across Europe. If you’re on the same journey, book a call: we’ll map your AI use cases and score them on the six branches.
The post How to Measure AI Value: A 6-Branch AI Maturity Model for Use Cases and Agents appeared first on Joyful Craftsmen.