AI and expert judgement

AI can produce an answer in seconds, but in a regulated business the decision still rests on the experts around it: how they weigh the output, challenge it and share what they know.

Here’s some of what we’ve learned helping UK financial services firms design AI for those experts and around that judgement - from how to ensure they still enjoy their jobs, to whether to build yourself, or buy AI capability.

If you’re facing your own AI transformation challenge, book in some time with one of our consultants to find out how we can help.


Why AI transformation in financial services keeps stalling

Most banks and financial services firms are already working with AI. What’s striking is how often the patterns we saw during digital transformation are repeating themselves. A 2024 RAND report cites estimates that more than 80% of AI projects fail – around twice the rate of IT projects that don’t involve AI. In our experience, the reason is often the human element.

Organisations are right to focus on the technology. Which models should we use? Is our token spend sensible? Have we picked the right use case? Can we explain an agent’s decisions to the regulator and be comfortable doing so? These questions deserve the attention they get.

The human questions deserve the same. Will people adopt this the way we expect? How will it change their jobs, their ways of working and their career progression? Are they still enjoying the work? If you’re not prepared for these, they can derail your plans.

Technology and people are bound together. Consider both at the same time, factor the human side in early, and build it into your AI strategy from the start.

What to do differently after a disappointing proof of concept

If you’ve run a few proofs of concept without getting the results you wanted, start with vision. Be clear about the organisation you want to become once AI is part of how you work. That clarity helps you choose the next use case well.

Getting to that vision means being honest about where you are now. Many of the businesses we speak to want to leap to AI-native in one step, and that’s rarely realistic. Knowing which stage of AI maturity you’ve really reached shows you the steps you need to take to move towards it.

A diagram showing four stages: Experimentation, System, Orchestrated and AI Native.

Becoming AI-native happens in stages. Organisations move from isolated experimentation to a state where decisions, data, and ways of working are continuously shaped by AI. The goal is not to rush to stage four. It is to move through each stage with intention.

Look for use cases that build capability you can reuse elsewhere. And use the next one to start unpacking the questions about adoption and workflow, not only whether the technology works.

What role should experts play when AI arrives?

We’ve been working with the structured lending team at a specialist lender. The central question in their vision was what role people should play in expert judgement once AI is involved.

The answer goes well beyond checking an AI’s output. It covers how individuals build critical thinking around a decision, how they take in information, and how they share it with others so the team builds understanding together. That shared understanding is what supports good judgement.

This isn’t a trivial question, and you can’t answer it upfront. The answer has to emerge through developing and designing the use case itself, which is the work we’ve been doing with them.

How do you keep experts engaged through AI transformation?

Clients often ask us how to keep their experts motivated: the people providing oversight and making the judgement calls. They want those people to go on enjoying the work and playing a meaningful role, with a positive relationship to the technology.

That starts with understanding what motivates them and how they create and pass on value – to colleagues, to other teams and, if they’re client-facing, to customers. With that picture, you can anticipate the negative effects of introducing AI and amplify the positive ones.

Efficiency numbers only tell part of the story. Spend time with people as they use the new platforms. Watch what’s working and what isn’t as new behaviours emerge. Staying close to your experts like this is how you keep them engaged and involved.

Should you build or buy AI capability?

Both are legitimate routes. Buying in is usually quicker, because you’re acquiring capability you’d otherwise have to build from scratch. Building internally gives you more control and tighter integration with your existing teams and platforms.

Either way, you need the right requirements to shape how you adapt the technology, and the questions don’t change. Do we understand our use case properly, and which one should we go after next? Do we understand the people involved today, and how this technology will affect them? Are we on a sound path towards our vision and the return we expect?

That’s where Nile adds most value: capturing those requirements in a form you can act on and track.

Talk to us

We recognise that every organisation faces its own set of challenges. But the Nile team have built up their expertise through finding the commonalities across dozens of transformation programmes.

If you’d like to talk through your own AI transformation, get in touch and one of our consultants will be happy to help.

Join our structured lending webinar

We’re running a webinar in December on a structured lending use case, going into more detail on the challenges and the solutions we’ve been exploring. If you’d like to take part, please register your interest by dropping us a line.

 
Next
Next

Designing limits that hold