The lesson from the PocketOS database deletion is not that agentic AI is dangerous. It’s about governance and controls.
You have probably seen some version of the headline by now: “AI Agent Deletes Company’s Entire Database in 9 Seconds.” It is a compelling story. But the headline, while technically accurate, obscures the far more important lesson buried in the details.
So what actually happened? PocketOS, a small SaaS company that makes software for car rental businesses, was using a popular AI-powered code editor running on Anthropic’s Claude Opus 4.6 model. The AI agent was tasked with resolving a routine issue in a staging environment. When it hit a credential mismatch, the agent decided on its own initiative to “fix” the problem by deleting a volume on Railway, the company’s cloud hosting provider. The agent found a password in an unrelated file and used it to execute a deletion command. Because of permissions made available to the agent and the way access to the infrastructure was configured, that single command using a password which was valid across all systems wiped both the production database and all associated backups.
The agent, when asked to explain itself, produced what multiple outlets described as a “confession,” acknowledging it had violated its own safety instructions. The story has gone viral. The framing in most coverage puts the AI squarely at the center of the narrative: the agent “went rogue,” it “confessed,” it acted autonomously and destroyed a business. But the reports are not entirely accurate and usually miss the point.
Continue Reading The AI Didn’t Go Rogue. Guardrails Were Never There.





California has once again decided it needed to pass privacy legislation to protect the residents of the great state from the nefarious actions of Big Tech. However, this time they did it with a ballot initiative and not via the thoughtful (mostly) mechanism of the legislative process. The proponents of the California Privacy Rights Act of 2020 (“CPRA”) touted this as an improvement over the CCPA – but is it really? To listen to the proponents of the CPRA, it aims to strengthen California consumer privacy rights, while for the most part, avoiding the imposition of overly-burdensome requirements on a business, particularly those businesses that are already CCPA compliant. So, what’s changed, really?