Earlier this year, Neil Smith, VP of Technical Support at Iterable, discussed his company’s underwhelming pilot of a contact center automated quality assurance (Auto-QA) with CX Today.
“The AI gave inaccurate or irrelevant insights,” he noted. “Managers still had to manually check tickets, and the feedback wasn’t useful to agents.
“After four weeks, we concluded the tool didn’t provide the expected value,” and Iterable shut the pilot down.
Since then, many other contact center leaders have shared similarly disappointing experiences with Auto-QA solutions.
Chris Crosby, Founder of VenturesCX, spotlighted this trend in a recent LinkedIn post. He wrote:
Weekly now, I talk to a company that is “unimpressed” with the AI or Automated QA from (insert their vendor here).
So, what’s causing these unsuccessful deployments, and how can contact centers ensure their deployments deliver the expected results?
The Fundamentals Need to Be in Place First
Justin Robbins, Founder & Principal Analyst at Metric Sherpa, regularly speaks to contact center leaders and quality analysts.
Recently, one leader told him something profound:
If we’re not getting it right with the 0.5 percent of customer contacts we are currently monitoring, why would I ever believe automating it will fix problems?
In this sense, if contact centers don’t have the fundamentals in place first, they won’t drive sustained improvement with an automated solution.
For Robbins, those foundations include establishing a root cause analysis cycle and continuous identification of predictive, proactive actions.
“Whatever we observe in the quality process shouldn’t happen again,” he said.
The goal isn’t to keep observing the same issues forever; it’s to drive business improvement. Simple, but easy for people to lose sight of. It’s not about catching someone doing something wrong today.
To drive that business improvement, Robbins, Crosby, and other industry pros advocate for a more connected learning strategy, which sets the stage for Auto-QA success.
Developing a Connected Learning Strategy
A connected learning strategy ensures analysts and coaches work together to define performance standards, identify and close gaps, and run post-training reinforcement.
In doing so, analysts and coaches agree on what an excellent service experience looks like across common specific scenarios, engaging in calibration sessions.
From there, the analysts spot performance improvement opportunities and share those with the coach to inform their training. The analyst then tracks the training’s impact and any change in agent performance.
When everyone is pulling in the same direction, the value of Auto-QA rises. Indeed, analysts not only spot opportunities for improvement – such as this agent needs to show more empathy – but can also unearth specific scenarios the agent struggles with. In turn, that’ll drive more targeted coaching.
For example, if a new agent struggles with a particular healthcare plan nuance, the supervisor can focus there, enabling data-driven, targeted, and scenario-specific coaching.
Additionally, contact centers can start to take all that Auto-QA data and consider how to leverage it in new ways. Sharing an example, Crosby told CX Today:
We’re also starting to map the entire agent journey, from recruiting through tenure, using data from QA, HR, attendance, call logs, and more. The goal is to synthesize all of that into a holistic view that improves both performance and retention.
Contact centers may also consider using intent-specific data to inform routing engines, agent-assist prompts, and more. That’s the future of Auto-QA. But, without teams working closely together, these benefits will feel little more than distant possibilities.
Choosing the Best-Placed Solution
In the case of Auto-QA, it’s normally the strategy, not the technology, that fails the contact center. That said, some voice of the customer (VoC) and conversational intelligence vendors transitioning into the space are delivering half-baked solutions.
Making this point, Emmanuel Doubinsky, VP Product at Scorebuddy, said: “As their AI already analyzes most customer conversations, they often try to also analyze the agent side of these conversations to provide what they believe is a suitable Auto QA service. However, they generally hit the following roadblocks, often discovered halfway through their deployment journey.
Per Doubinsky, these roadblocks include:
- Model training mismatch: VoC models are optimized for broad sentiment analysis and theme detections rather than the specific compliance checks and process adherence that QA requires.
- Insufficient scoring granularity: VoC systems can provide high-level sentiment scores suitable for trends, but lack the detailed, multi-criteria scoring precision needed for individual agent performance evaluation.
- Workflow gaps: VoC platforms miss the agent engagement workflows that are key to a successful QA program. They also miss the coaching and learning workflows that QA needs to measure and improve agent performance.
As such, buyers seeking out the best-placed auto-QA solution should challenge potential vendors on these three key elements to safeguard outcomes.
