Why Structural Backpressure Prevents Security Bugs Better Than Smarter AI Agents
By
pyrex41
The kind of bagel that ruins lesser bagels for you.
Summary
The article argues that the most serious software bugs, like broken access control (OWASP #1), persist not because developers disagree on the rules but because security invariants are placed in the wrong part of the system—in prompts, review checklists, or shared expectations. The author advocates for "structural backpressure": embedding security constraints directly into the system architecture and tooling (e.g., formal verification gates in AI coding loops) so that violations are structurally impossible rather than relying on human or model memory. This approach beats trying to build "smarter agents" that might still forget or misapply invariants.
Key quotes
· 3 pulledA user should not be able to read another tenant's data. Nobody disagrees with this, nobody stands up in a design review to defend Alice reading Bob's records, and yet broken access control remains the #1 category on the OWASP Top 10.
These bugs ship because the rule has been placed in the wrong part of the system. It lives in a prompt, in a review checklist, in the shared expectation that every future engineer, and now every future model invocation, will remember the invariant and reapply it correctly.
That assumption was already fragile with human engineers; with AI-generated code, it becomes untenable.
You might also wanna read
AI bug-finding systems uncover real vulnerabilities at DARPA cybersecurity challenge
The article discusses the DARPA AI Cyber Challenge (AIxCC) held in Las Vegas, where top cybersecurity teams demonstrated AI-powered bug-find
The Verge·1mo ago
AI-assisted vulnerability discovery raises concerns about Linux kernel security
This opinion article discusses a troubling trend in Linux security where AI-powered tools are being used to discover and exploit kernel vuln
theregister.com·1d ago