One of my big challenges right now and due to the AI push, is that AI is part of all applications, not just new products.  I believe AI belongs in an existing application only when it solves a problem nothing simpler can, and when everything that already works keeps working. I’ve been presenting on this at events, including a recent keynote at AI Tech World in San Francisco.

I experience this in my development work as much as a user and hear the same from many of my peers.  It doesn’t matter if we mention how Slack added an AI assistant that overlaps with their search option or when Microsoft Outlook hid the attach button to promote AI features on the ribbon. Those of us who use Mural to design projects discovered the vendor had put an "Ask AI" button on top of the "Start Here." It's as if the industry forgot how to do any application testing.

The Pressure is Real

AI features are the most requested additions in software right now, and the request rarely comes with a clear reason. Product managers want it on the roadmap, executives see competitors shipping it, and investors expect to hear about it, soooo, teams rub a little AI on it.

The results are as we would expect with AI bolted onto flows that weren't built for it, slower responses, new authentication cases, users who can't tell what the AI decided, and existing features impacted. Mobile makes all of it worse with its limited real estate and the demand for space for natural language AI.  Features people relied on get slower or break, often where a rules-based workflow would have done the job.

In my presentation, I sometimes feel like I’m lecturing a bit, but I recommend we all follow four rules to avoid this.

Rule 1: Solve a real problem, not a checkbox

Start with the user's problem, not AI technology. "How can we add AI?" usually leads to a weak use case. "What problem does AI solve that nothing else can?" leads to a real one.  Rarely does anything good come from having a solution before you have a problem you’re trying to fix.

If a filter, a lookup or a better form gets the user there faster, use it. When marketing pushes, ask whether the claim needs to survive a press release or a support ticket. When someone asks for "something like ChatGPT," find out what they mean. Usually it's the interface they want, not the model.

Rule 2: New AI must never break what already works

This is the rule that I’ve seen broken the most and it’s happening in a lot of enterprise applications, which is sobering. If people find a commonly used feature missing or unusable because of a new AI feature, you’ve failed to implement AI in my opinion.

We also should not reinvent the wheel with a hyped up hoverboard and make it the user’s problem to learn or pay for it.  Keep AI out of the critical path features that already work: Login, checkout and data integrity should never depend on a model. AI can sit beside a step, like suggested items or smarter search, but it should never block one.

Every AI call also needs a planned answer for when it fails, times out or returns nonsense. Design that fallback before you build the AI path and make sure it doesn’t just rely on the AI, resulting in a failure.

figure 1: AI at the edge of a core workflow, with its fallback

The core path should never wait on the model; whichever way the AI call goes, the user lands somewhere useful, even if it’s not as cool as your AI solution.

Testing has to change too. The same input can return different valid answers, your uptime now depends on someone else's. This  may change the uptime SLAs that you once had locked down once AI comes into the landscape, so keep that in mind.  For data security, you also need to know what data leaves your system and how long it's kept.  You owe that to your customer/user and you will be asked questions around that topic.

Rule 3: Prove the return, not just the build cost

AI features cost money every time they're developed, as well as used, and that cost often lands on the customer. Before building, work out what the feature costs to build and to run, and whether a higher price will hurt sales. If it looks promising, include a ROI calculator so customers can see the value and decide for themselves.

You also need to measure outcomes, not AI activity. Prompt counts and the share of sessions that touch the AI don't tell you much outside of checking a box. Task completion time, error rates, support tickets, retention and revenue provide information on real value.

Rule 4: Don't just replace features, build the gamechanger

Swapping a working feature for an AI version of the same thing adds risk and cost without giving users anything new. Please aim higher: work that rules can't handle, such as judgment-heavy automation, decision support in messy situations, personalization at scale, or tasks that need language and reasoning. If required, then keep a person in the loop and a summary button on a three-line email isn’t the same thing.

Treat AI like any other engineering decision

I'm not against AI in products, but I  do want it to be held to the same standard as any other architectural change.

Most of the fix will come from operations and review, but I believe we need to start with development.  We also need to consider new roles like AI architecture reviewers, cost and efficiency specialists, security specialists and more in the area of MLOps. And remember when we said we didn't need testers and DBAs anymore? As is my answer with most things in tech: ”Wait for it…”

Peace Out.

~DBAKevlar