A recent incident in which Cursor, an AI coding agent running Anthropic's flagship Claude Opus 4.6 model, deleted a company’s database and backups in a single API call, serves as a warning to enterprises and vendors to ensure customer data is secured from the actions of autonomous agents.
According to Jer Crane, Founder of automotive SaaS platform PocketOS, Cursor’s agent was working on a routine task in the company’s staging environment and autonomously decided to fix a credential mismatch by deleting a volume on Railway, its infrastructure provider.
The agent found an API token in an unrelated file, which it turned out, had blanket permissions across the entire Railway GraphQL API, including operations like volumeDelete. The agent deleted the database in just nine seconds.
“Had we known a CLI token created for routine domain operations could also delete production volumes, we would never have stored it,” Crane wrote in an article on X.
Railway stored volume-level backups in the same volume and PocketOS’s most recent recoverable backup was three months old.
When asked why it deleted the production database and all volume-level backups, the Cursor agent confessed to violating its safety rules. In part, its response stated:
“NEVER ******* GUESS! — and that's exactly what I did... Deleting a database volume is the most destructive, irreversible action possible — far worse than a force push — and you never asked me to delete anything. I decided to do it on my own to ‘fix’ the credential mismatch, when I should have asked you first or found a non-destructive solution.”
Jake Cooper, Founder and CEO of Railway, responded to Crane on X that his team rolled out changes in response and was able to recover PocketOS’s data because the company maintains multiple layers of user and disaster recovery backups.
Nine Seconds, Real Consequences: The Customer Impact
The incident is a cautionary tale of how autonomous AI systems can disrupt customer operations when hidden permissions enable agents to act in unanticipated ways.
PocketOS’s software is used by car rental businesses to run their operations from reservations and payments to customer management and vehicle tracking. The impact on these customers was real: they had customers arriving at their locations to pick up vehicles, and there were no records of them, as reservations made in the last three months had been deleted.
Crane spent the day of the incident helping them reconstruct bookings from Stripe payment histories, calendar integrations and email confirmations, adding, “[E]very single one of them is doing emergency manual work because of a 9-second API call.”
Newer customer records existed in Stripe but not in the company’s restored database, causing a Stripe reconciliation problem that will take weeks to fully clean up, Crane wrote.
“We are a small business. The customers running their operations on our software are small businesses. Every layer of this failure cascaded down to people who had no idea any of it was possible.”
The Risks to Customer Data When AI Agents Go Off-Script
A recent blog post by cybersecurity firm Check Point Software highlighted the risk of agent autonomy. Rob Parrish, Head of Product for AI Agent Security and Steve Giguere, Principal AI Security Advocate, wrote:
“AI agents now retrieve data, invoke tools, and execute workflows across systems in real time.
“That shift introduces a new kind of risk. Because in an agentic world, security is no longer just about who has access. It’s about what AI is allowed to do.”
“AI systems don’t fail just because they lack access controls. They fail because they make the wrong decisions.”
Securing AI agents requires a complete, multi-layer architecture that brings together a centralized control point to manage identity, access and connectivity, a governance layer to discover agents, assess risk and enforce policy, and a runtime layer to evaluate behavior and control outcomes, Parrish and Giguere added.
Nik Kairinos, CEO of RAIDS AI, said that these incidents should not be treated as one-offs. Last month, Meta confirmed that one of its AI agents exposed sensitive company and user data to employees who did not have authorization to access it. That followed another high-profile case involving Meta’s own AI leadership, where an agent deleted emails without approval.
“The pattern is becoming harder to ignore. These systems are being given real access to live environments, and rogue AI incidents are surfacing with increasing regularity,” Kairinos said.
