Japanese supermarket chain AEON has become the first company to leverage AI to monitor the tone and demeanor of its customer-facing staff.
Created by InstaVR, the “Mr Smile” solution aims to gauge and standardize the smiles of customer service employees across AEON’s 240 stores.
The tool claims to be able to analyze over 450 elements, including facial expressions, voice volume, and greeting tone. The AI-powered program then uses this data to rate a customer-facing employee’s overall attitude.
Moreover, Mr Smile also contains a competitive game element that encourages staff to achieve higher scores by improving their attitudes.
In discussing the implementation of the new tech, AEON stated that it would help to boost customer experience:
[The goal is to] standardize staff members’ smiles and satisfy customers to the maximum.
Indeed, in a three-month trial conducted by AEON across eight of its stores, the supermarket chain reported that the attitudes of its customer service staff had improved by 1.6 times.
However, despite the positive trial feedback, the move raises serious questions about the morality of pressuring workers to express themselves in a particular way.
Sinister or Revolutionary?
While AEON has stated that its employees’ attitudes have improved, it does not specify if this information is garnered from the My Smile solution or from direct feedback from AEON employees.
This, in a nutshell, illustrates the issue of measuring customer service agents’ attitudes.
By pressuring agents to act in a particular manner that is deemed agreeable by the company, they are losing their individuality and freedom to express themselves as they see fit.
The inauthenticity that the solution encourages has drawn criticism from several unnamed Japanese customer service workers in a union survey.
“When the service industry workers are forced to smile according to a ‘standard’, it looks to me like another form of customer harassment,” one employee said.
“Smiles should be a beautiful, heartfelt thing, and not be treated like a product,” another claimed.
“People are different, and they also express their affections differently. Using a machine to ‘standardize’ people’s attitude sounds cold and silly,” commented a third.
A Step too Far?
My Smile might be the first example of AI being used to measure employees’ attitudes, but it is far from the first time that the tech has been deployed to provide personal support to agents.
Earlier this year, in an attempt to combat the persistent issue of contact center agent stress, Cisco Webex introduced an Agent Burnout Detection tool designed to identify high stress levels and offer a one-minute break for agents to relax.
Testing of the tool reported a 13-15% reduction in stress, a 40-50% smaller increase in average handling time (AHT), and improved customer satisfaction.
However, when piloting the solution, First Horizon Bank faced social media criticism for some of the additional features that it included. The bank chose to show agents a montage of family photos with their favorite songs during the breaks, which some social media users described as “bleak,” “grim,” and reminiscent of a “Black Mirror plot.”
Indeed, following the outcry, First Horizon released the following statement on X:
These initiatives, driven by a technology partner of First Horizon’s, have not been rolled out to our contact center or client care teams. These are just some of the options we have explored to improve the lives of our associates across the bank.
Elsewhere, in another attempt to tackle agent stress, Japanese telecoms organization SoftBank Corp is testing AI software that moderates the tone of irate customers.
This technology – combining voice processing and AI-enabled emotion recognition – also aims to improve customer retention.
However, there is a concern that if agents misjudge the customer’s tone, they might not respond with enough empathy or urgency.
