Product design used to have clear boundaries. A UX designer would build screens in Figma, test them with users, then handed them to developers to build. They’d go through refinement, join the backlog and eventually reach customers. This was a process with a clearly-defined start and a clearly-defined ending.
With AI, and especially AI agents, transformation teams can work differently. You can deploy individual capabilities rather than closed products. There’s still loads of decisions to make that require individual judgement calls – guardrails around what an agent can do, what data sources it can draw from, which skills and tool calls it can use. But these say much less than before about where a workflow begins and ends. For longstanding problems that used to produce bloated software, that should be a welcome change.
Shorter feedback loops
In the old world, a customer’s request had a long way to travel. Research revealed the need, then designers, researchers, product managers and engineers turned that need into something real, that people use. Realistically, that could take months (or worst case, the request could sit stuck at the bottom of the backlog).
With agent-driven features, a customer can simply ask your app for a report, or to compare two sources, or anything. So long as the data’s there and the agent understands the request, there are no longer so many steps between the user wanting it to do a thing, and the app doing a thing.
That freedom brings new responsibilities for product teams, delivery leads and leadership. The biggest one is observability: analytics built for graphical interfaces, such as clicks, funnels and drop-off, tell you very little about how people are using an agentic app.
So teams are moving towards evals. These are structured tests that show how a change to a prompt or product affects the outcomes you’re working towards. They also need to see what people are asking the agent to do, particularly when your customers do things you didn’t expect them to.
Reading the signal
One unexpected request could just be a one-off. If a user keeps making it, it might be a sign there’s something there. When several users start asking for the same thing, there’s probably a use case worth designing for. Or - better still - the agent may already do what they need, and you don’t have to build anything.
This matters twice as much for regulated firms. “People are using it in ways we didn’t expect” is both an opportunity and a risk question. The same observability that brings new use cases to light also lets you evidence how a capability behaves, catch misuse early and show you understand what your AI is doing for customers.
If you’re interested in exploring how agentic AI might fit with the processes and platforms in your business - and what needs to remain human - drop us a line. We’re currently working on exactly these questions with a number of regulated businesses across the financial sector.
