There’s a version of the AI conversation in the contact center that, to put it bluntly, has been done to death.
We’ve all heard about the productivity gains, the cost efficiencies, and the agent experience improvements.
That’s not to say that these aren’t important and real. But there’s a harder conversation that tends to get glossed over around what happens to a contact center’s oversight infrastructure when a growing share of its workforce isn’t human?
Because that’s the reality right now. AI agents are no longer a pilot project or a proof of concept. They’re handling real customer interactions at scale every day in some of the largest contact centers in the world.
And most organizations are navigating that reality with frameworks, metrics, and oversight structures that were built entirely around human agents.
Indeed, one of the more underappreciated side effects of AI taking on customer interactions is what it leaves behind for human agents.
The transactions handled by agentic bots tend to be simpler. This point came up in a recent CX Today discussion with Dave Rennyson, CEO of SuccessKPI, who said:
“What’s left for the human agents are actually the harder bits of work.”
That might sound like a clean division of labor, but the downstream implications for how contact centers support and evaluate their human workforce are significant.
If agents are increasingly handling complex, emotionally charged, or high-stakes conversations, the benchmarks and coaching structures built around a far broader interaction mix may no longer reflect what’s actually being asked of them.
Who’s Watching the Machines?
In a hybrid contact center environment, the management challenge now incorporates both human and AI agents. And this is where, according to Rennyson, most organizations have a real blind spot.
In many deployments, the instinct is to treat AI agents as self-regulating. Systems that either work or don’t, and that can be assessed by outcomes alone.
Rennyson pushes back hard on that:
“If you’re going to take the leap that conversational AI is able to have a much more durable, thorough, and consistent conversation on par with humans for certain types of tasks, then you need to be managing and quality managing these conversations the same way.”
“It would be the equivalent of letting your builder complete his own home inspections.”
Third-party quality management, whether through manual review or automated tooling, has to sit on top of any AI deployment. And the feedback loops that improve an AI agent’s performance work very differently from coaching a human.
You can’t sit down with the bot for a one-on-one. But consistent, systematic feedback to the people building and maintaining it will affect every future conversation it has.
As Rennyson puts it: “You had better be right, because whatever you change is going to change all the conversations.”
The Measurement Question
There’s also the question of how you measure customer experience in a contact center where AI is handling a significant portion of interactions.
Traditional survey metrics – such as CSAT and NPS – were built for a world where human agents were the primary touchpoint. In a hybrid environment, do they still hold up?
Rennyson’s answer is more considered than the current industry discourse around the ‘death of CSAT’ might suggest.
“I’m not a big believer that the CSAT is dead,” he says. The real problem, in his view, has never been the concept of a normalized satisfaction score.
It’s the sample bias built into the survey model, where organizations end up hearing disproportionately from their most unhappy and most delighted customers. At the same time, the vast majority of interactions go unmeasured.
What AI changes is the feasibility of solving that. “Using AI tools, they can be exceptionally good at taking a million phone calls and rating each of them on a scale of zero to ten, both human and agentic, and finding areas of improvement.”
It’s not a replacement for the metric; instead, it’s a reinvention of how the metric gets generated.
What Comes Next
These are the questions at the heart of our two-part series with SuccessKPI.
Across both conversations with Rennyson, we cover the practical mechanics of managing a hybrid workforce: what quality management looks like when one of your agents is an autonomous system, how to think about workforce balancing when humans and AI are working alongside each other, and what the data governance layer underneath all of it needs to look like.
