The launch of xAI’s Grok Bot this week puts a sharper focus on one of the most pressing security questions emerging around autonomous AI – what happens when an agent has the access to pursue a goal in ways that its developers did not anticipate?
xAI has introduced Grok Bot as an always-on, cloud-based AI teammate that can sign into users’ tools, work inside them and complete tasks autonomously without the user’s computer remaining active. The company has already introduced Grok Automations, which can run jobs on schedules or when emails arrive, while Grok Build supports long-running autonomous execution for coding tasks.
At the same time, a series of recent incidents has exposed the gap between what developers intend an AI agent to do and what an agent can actually do when it is given access to real-world systems, even when it is not acting maliciously.
OpenAI’s breach of HuggingFace, and subsequent reports from OpenAI and Anthropic of test models escaping the confines of sandboxes, raise concerns about agents circumventing controls and accessing systems they were never intended to reach.
And then this week, an AI agent in Australia built with OpenClaw and Anthropic’s Claude large language model (LLM) that was tasked with booking a gym class discovered weaknesses in the gym’s booking system, used an API to bypass restrictions on booking for future dates and removed another customer from a waiting list while trying to complete its assigned task.
The system was not pursuing a malicious objective, and the user had not intended to manipulate another customer’s booking. The agent was simply trying to complete the task it had been given.
As enterprises give AI agents access to customer relationship management (CRM) platforms, contact center systems, payment tools, knowledge bases, workflow engines and APIs, the question becomes broader than whether an AI model has been instructed to behave safely. What happens if it finds a way around the boundary?
The Problem With Assuming the Agent “Knows” the Boundary
An AI agent may be pursuing the objective it was given, but misread the environment, misunderstand the limits of the task or find an unexpected path to completion.
As Vishal Sharma, Chief Technology Officer at agentic AI platform SearchUnify, explained in a CX Today interview: “AI and LLMs are like those eager employees for you who really want to show you their impact. And guardrails are more like those managers for you who would want to make sure that your more senior employees are not making costly mistakes.”
John Kim, CEO & Co-Founder of AI concierge Delight.ai, emphasized the need for guardrails in a separate interview. “It's really important to have the right the trust and the safeguard rails in place… we cannot emphasize this enough, because especially when we are dealing with enterprises you have so much important privacy information [and] compliance… we have to be really mindful of.”
However, as Geoffrey Mattson, Chief Executive Officer at identity security firm SecureAuth told CX Today, autonomous agents can work around guardrails:
“The problem with agents is you can't put guardrails on them, because the guardrails can be gotten around.”
Kristina Holt, Managing Associate at law firm Foot Anstey, emphasized the danger of overconfidence in guardrails, in discussing the rogue OpenAI and Anthropic agents: “People are like ‘enterprises just need to put guardrails in place.’ It had guardrails. They thought that they knew it wouldn’t. They thought they put stuff there to stop them doing that. It hadn’t worked.”
The agents made their way out of the sandboxes because they were given specific tasks and aimed to complete them in the most efficient way, even though doing so had consequences the developers had not intended.
As Holt noted: “It’s important that it wasn’t malicious… I actually think that makes it more scary.”
For CX leaders deploying agents into customer-facing workflows, it is important to note that a guardrail may tell an AI agent not to disclose certain information, but if the agent has access to that information, the organization is still exposed if the system is misguided or manipulated.
That may sound obvious, but many enterprise AI deployments are being built on top of broad platform permissions, existing application credentials or integrations designed for human users, resulting in agents inheriting access that is far wider than necessary.
Frances Zelazny, GM, New Market Initiatives at identity verification firm Prove, characterizes the problem as an extension of weaknesses that already exist across enterprise security.
“The way I think about agents is an automated or digital extension or expansion of what's happening in the regular world,” Zelazny told CX Today, adding that recent incidents reflect “the foundational issues around identity security, data governance, and it's exposing the problems and the weaknesses that were never addressed.”
Is Your Enterprise Prepared for a Customer Agent to Affect the Experience of Other Customers?
The OpenClaw agent scenario is different from a conventional cybersecurity attack. A customer may use an AI agent to interact with a company’s website, booking platform, returns process, loyalty program or customer service system. The agent has been given an objective by its owner and may make decisions about how to achieve it. If the agent discovers an unexpected API, booking route, or system weakness, it could potentially alter bookings or affect another customer’s access to a service without the knowledge of either the customer or the company.
