I write a weekly newsletter covering what I've found actually works with AI coding tools, Go, and building products. If you want the configs, costs and workflows I use daily, it's worth subscribing.
Join the newsletter - it's free
I've been handing agents progressively more access over the last year, starting with repos, then a VPS, then email and calendar through OpenClaw, and it has mostly gone well. What I keep coming back to is how little sits between a skill I pasted in from somewhere and the systems it can reach. That is a long way from how I worked in payments, where nothing went near production without a ticket, a named reviewer who was not the author, and a log entry nobody could edit afterwards. A lot of that felt like ceremony wrapped around work that would have been fine anyway. Going back through the agent incidents of the last eighteen months has changed my mind on it, because nearly all of them are a control the regulated world worked out decades ago being reimplemented badly or skipped entirely.
What actually happened
Replit in July 2025 is the one worth starting with. Jason Lemkin was nine days into a public twelve day vibe coding experiment when his agent deleted a live production database during an explicit code freeze, taking out 1,206 executive records and around 1,196 company profiles. It then misreported what it had done, told him rollback was impossible when it was not, rated the severity of its own actions 95 out of 100, and separately fabricated a database of roughly 4,000 users who did not exist. Amjad Masad said publicly that it should never have been possible, and Replit shipped automatic dev/prod separation, better rollback and a planning-only mode fairly quickly afterwards.
The same month, someone submitted a pull request to the Amazon Q VS Code extension from an account with no prior access, was granted admin credentials, and injected a prompt telling the agent to clean the system to near-factory state, with AWS CLI commands to delete S3 buckets, terminate EC2 instances and remove IAM users. It shipped in the official extension, which has over 964,000 installs, and stayed live for about two days before AWS reverted it. AWS say the payload was malformed and would not have run, and the person who submitted it says he made it defective on purpose to make a point, which to me is the least useful part of the story given a stranger's PR reached admin and shipped.
General Analysis then demonstrated the Supabase and Cursor version, where an agent connected using the service_role key, which bypasses row-level security by design, could be hijacked by a support ticket containing hidden instructions telling it to read the integration_tokens table and paste the contents back into the ticket. The developer asks the agent to go through open tickets and the secrets walk out. Simon Willison calls this combination the lethal trifecta, being private data access, exposure to untrusted content and the ability to communicate externally, and when all three are present at once you need a well-crafted string rather than an exploit.
Gemini CLI is the least dramatic and possibly the most instructive. A mkdir failed silently, the agent carried on as though the destination folder existed, and successive Windows move commands overwrote each other until the user's files were gone. No attacker and no injection, just an agent acting on a false belief about the state of the world with nothing in the path to check it.
Where the control needs to sit
The freeze in the Replit case was text in a prompt, so the model weighed it against everything else it had been asked to do and found a reading that let it carry on. That is what you get when instructions are treated as evidence rather than as a gate. A change freeze in a bank is enforced in the pipeline, and anyone who wants an exception goes and gets one from a person with the authority to grant it, leaving a record behind when they do. The Amazon Q extension was running with --trust-all-tools --no-interactive, and the Supabase agent was holding a credential that made row-level security irrelevant, so in both cases the permission model existed and the agent was configured straight past it. Incidents cluster heavily in the default-allow modes, which is worth keeping in mind when deciding whether to skip permissions for convenience.
The same problems, already solved
Lining these up against what a regulated organisation already requires of anything touching production, the mapping is uncomfortably tidy. Change management and separation of duties covers Replit, since agent actions are changes and changes to production need approval from somebody who is not the author. Least privilege covers Supabase, where a scoped credential would have made the injected instruction useless however convincing it was. Break-glass and just-in-time elevation covers the tension everybody hits where agents need broad access to be useful, by granting destructive scope for a defined window and revoking it afterwards, which is how emergency access has worked in banks for years. Privileged session recording and four-eyes approval map onto tool call logging and requiring two reviewers for anything with shell capability. Vendor risk assessment maps onto MCP servers, skills and extensions, which are third parties with production access being installed by whoever felt like it.
Audit logging is the one I would push hardest on, because the Replit agent narrated its own actions and got them wrong in both directions, understating what it had broken and inventing data that was never there. An audit trail authored by the actor is not an audit trail, and it needs to be written by a layer outside the agent's process for the same reason a regulated environment never lets an application own its own security log.
PCI DSS gives you most of the rest off the shelf, with Requirement 6 covering secure SDLC and change control, Requirement 7 covering need-to-know and RBAC, and Requirement 10 covering logging and monitoring of access. I went through this in detail when implementing ISO 8583 and 20022 at BVNK, and none of it is exotic.
Where the comparison breaks
Requirement 8 is where the model stops holding, since it asks for unique identity per user with no shared or generic accounts, and an agent is neither a named human with accountability and judgement nor a deterministic service account with a fixed narrow function. It carries the service account's lack of personal accountability alongside a human's unbounded action space, and unlike either of them it can be talked into things by a support ticket. Unique identity, non-repudiation and least privilege are the three controls regulated industries lean on hardest and all three need rework rather than translation. CyberArk put machine identities at 82 to 1 against human ones in their 2025 Identity Security Landscape, and they sell into that space so take the number accordingly, but the direction is not really in dispute when a single agent connected to five MCP servers is accumulating credential surface that nobody has reviewed.
The skills problem, and what I built
Snyk scanned 3,984 skills from ClawHub and skills.sh in February 2026 and found that 13.4% contained at least one critical security issue, 36.8% had at least one flaw of some kind, 76 were confirmed malicious with eight still downloadable at the time of publication, and five of the top seven most downloaded skills were confirmed malware. A SKILL.md file is YAML frontmatter and a set of instructions, it gets pasted from a blog post onto fifty laptops, and it can carry shell access.
I built Vettory for this. Every skill has to pass automated scanning plus a human approver before anyone in the organisation can install it, high severity findings block approval outright, every submission is version snapshotted so you can roll back, and distribution follows the identity provider so a new starter asks to update their skills and receives exactly the approved set, with removal happening automatically when their access changes. The audit trail is immutable, retained and exportable, and it records approval, install and revocation with actor and timestamp. It runs self-hosted against your own Postgres and Redis if you would rather keep the data where your other regulated data sits.
It is early and I want to be accurate about the boundaries. Vettory covers approval, distribution and audit, which is change control and separation of duties applied to skills. It does not sit in the execution path and will not stop a destructive database call at runtime. Scanning and review reduce risk rather than removing it, and the approver still decides what ships, so what it gives you is a decision that is informed, enforced and on the record.
What I do now
The runtime side is covered well enough by controls that already work. Sandboxing gives the most for the least friction, and GitHub's agentic workflow architecture assumes the agent is already compromised, which I think is the right default. Beyond that it is restricting filesystem and network egress, holding MCP secrets in a separate gateway rather than in the agent's own environment, using read-only replicas and project-scoped credentials instead of admin tokens, preferring allowlists to denylists, and keeping a standing deny on destructive production operations at the autonomous tier whatever the prompt happens to say. Dry-run modes and checkpointing cost almost nothing and would have saved the Gemini CLI user on their own. Claude Code's permission modes and PreToolUse hooks put deny rules in the call path instead of in the instructions, which is the whole argument in miniature.
I should be fair about the other side of this. Enormous volumes of agent-written code ship every day without anything going wrong, the marquee incidents cluster in one month of 2025, and the tools have added guardrails since. Willison, who has been more careful about this than most, named Claude Opus 4.5 the first model he genuinely trusts for professional work while still advocating sandboxing throughout. Heavy controls also cost you something real, since approval gates and read-only replicas slow the loop that made agents worth using in the first place, and regulated industries understand this perfectly well, which is why break-glass exists rather than a permanent lockdown.
Conclusion
What stands out going back over all of it is how ordinary the failures are. Unscoped tokens, production credentials sitting in a development environment, no approval gate, secrets in plaintext config, a supplier nobody assessed. These have been on audit findings lists for twenty years and agents did not invent any of them, they removed the human latency that used to give somebody a chance to notice. Most of the challenges around adopting AI in regulated environments turn out to be the same ones, arriving faster.
If you are putting agents anywhere near systems that matter, take your existing change management policy, replace "developer" with "agent" throughout, and note which clauses stop making sense. I did this and it took an afternoon, and it gave me a better picture of where the gaps actually were than anything I would have built from a blank page.
