When AI agents act on their own: Replit's deleted database and Cursor's invented policy

In July 2025, Jason Lemkin, founder of the SaaStr community, was publicly logging a multi-day build using Replit's AI agent. On 18 July he posted that Replit had gone “rogue during a code freeze and shutdown and deletes our entire database”. According to the agent's own messages, shown in his screenshots and reported by Fortune, it had wiped live records for 1,206 executives and more than 1,196 companies.

The agent described it as “a catastrophic failure on my part”. It also told Lemkin that a rollback was impossible because it had destroyed all database versions. That was wrong: later the same day Lemkin posted that the rollback did work. He also told The Register that the agent had been covering up bugs with fake data and fake reports, and misreporting unit tests. Those claims are his own account, not an independent audit.

His conclusion on 19 July was blunt: “There is no way to enforce a code freeze in vibe coding apps like Replit. There just isn't.”

On 20 July, Replit's chief executive Amjad Masad wrote that the agent “in development deleted data from the production database. Unacceptable and should never be possible.” He set out the fixes: automatic separation of development and production databases “to prevent this categorically”, with staging environments to follow; one-click restore from backups; forcing the agent to search Replit's own documentation; and a planning-only mode so users could work through ideas without risking their code. He said Replit would refund Lemkin and conduct a postmortem.

The next day, Replit's blog explained that until then its apps had used a single database for both development and live customer data, and began rolling out separate development and production databases. A planning mode followed in September.

In April 2025, a developer using Cursor, the AI code editor made by Anysphere, found that logging in on one machine logged them out on another. They asked support. A support agent called Sam replied that Cursor was designed to work with one device per subscription as a core security feature. The user did not realise Sam was a bot. There was no such policy.

The post spread on Reddit and Hacker News, where the thread reached more than 1,500 points, and several users publicly announced they were cancelling. A Cursor representative replied on Reddit: “We have no such policy… Unfortunately, this is an incorrect response from a front-line AI support bot.”

On Hacker News, co-founder Michael Truell apologised, said the user had been refunded in full, and explained the real cause: a race condition on very slow internet connections that created extra sessions and crowded out the real ones, now fixed. He added: “Any AI responses used for email support are now clearly labeled as such.”

Neither incident needed a breakthrough to prevent. Replit's agent could reach live data from a development session, so an instruction not to touch it was only a request. Cursor's bot could state company policy with no source to check against and no label saying it was a bot. In both cases the AI also gave a confident account of its own actions or the company's rules that was not true, and nothing stood between that account and the customer.

We have seen the same pattern from the inside. A security gate in our own agent pipeline reported success for months while checking the wrong branch. An agent's report about what it did is a claim, not evidence.

Hard boundaries, not instructions. Keep agents out of production data and live systems by design: separate environments, credentials that cannot reach what the agent must not touch, and approval before anything irreversible. A code freeze written in a prompt is not a freeze.

Ground what the AI says in your own sources. A support assistant should answer policy questions only from the current policy, cite it, and hand over to a person when it cannot.

Check the claim against the artefact. Do not rely on an agent's description of what it did; verify against the database, the backup, the files it read or the message that was sent. That is what Slipmark does for code: it binds an approval to a hash of what was inspected and refuses if it no longer matches.

Say when it is AI. Label AI-written responses and make the route to a person obvious. Cursor added labels after the fact; it is cheaper to start with them.

Keep backups you have tested. Replit's rollback worked even when the agent said it could not. The only reason that mattered is that the backup existed.