Blog · ← All posts
Vibe Code Audit: What AI Coding Assistants Get Wrong
Published: November 6, 2026 · 8 min read
The short answer
A vibe code audit is a security review aimed at the specific ways AI coding assistants fail — which are not the ways human juniors fail. Assistants produce code that runs: it passes the happy path, demos beautifully, and carries a characteristic set of security defects that come from how generation works. Vibe-Code Audit runs 62 code detectors (11 core patterns on every scan, plus 51 advanced detectors) and a 32-pattern secret scan, and its Advanced Patterns engine covers 19 next-level families — JWT key confusion, deserialization gadgets, second-order injection, advanced SSTI, GraphQL abuse, cache poisoning, request smuggling, LLM prompt-injection sinks, RAG poisoning, tool-call injection, and more. The assistants are not careless. They are average: trained on millions of repos written under deadline, they reproduce the average repo's security posture at machine speed. Detector counts as documented in Vibe-Code Audit's own tier reference, November 2026.
Why assistants get these wrong
Three mechanics explain almost every finding. Generation optimizes for "runs," not "is safe." A model rewarded for code that executes will choose the API call that works in the demo — the permissive deserializer, the string-built query, the token check that accepts the header it was shown. Prompts specify features, not invariants. Nobody writes "and verify the JWT signature against the right key algorithm," because it feels like plumbing; the model fills the silence with the most common pattern in its training data, which is frequently the insecure one. Context is local. The assistant hardening your login route cannot see that another generated file trusts a header your proxy never sets — the header-vs-URL auth dispatch bug class exists precisely in that seam. Our 30-codebase study documented the quality-side version of the same story (swallowed errors in 29 of 30 codebases); this page is the security side.
The 19 advanced families, plainly
| Family | The mistake it catches |
|---|---|
| JWT key confusion | Token verification that accepts the wrong algorithm or key, so forged tokens validate |
| Deserialization gadgets | Untrusted data fed to native deserializers that can be weaponized into code execution |
| Second-order injection | Payloads stored safely, then executed later in a different context that trusts them |
| DOM clobbering | Page markup that overwrites the script's own variables and references |
| Advanced SSTI | Server-side template injection beyond the obvious sandbox escapes |
| GraphQL abuse | Introspection left open, query depth and batching unpunished, field-level auth missing |
| Cache poisoning | Unkeyed headers and inputs that let one user's response be served to everyone |
| Request smuggling | Front-end/back-end disagreement about where one request ends and the next begins |
| Build-artifact secret leaks | Keys and tokens baked into shipped bundles, images, and generated files |
| TOCTOU races | Check-then-act gaps an attacker walks through between the check and the act |
| LLM prompt-injection sinks | User text flowing into prompts that drive tools or privileged actions |
| Insecure LLM output handling | Model output rendered or executed as if it were trusted code or markup |
| RAG poisoning | Instructions planted in retrieved documents the model treats as orders |
| Tool-call injection | Attacker-shaped arguments reaching the agent's tools unvalidated |
| Advanced prototype pollution | Deep-merge and assignment paths that let input rewrite object defaults |
| Unguarded image generation | Generation endpoints with no auth, quota, or content boundary — an open spend and abuse surface |
| Unsafe ML model loading | Pickle-style model files loaded from untrusted sources — code execution by download |
| Header-vs-URL auth dispatch | Authorization decided by headers or paths that don't agree, leaving a route guarded in one view and open in another |
| Path-traversal file writes | User-shaped filenames writing outside the intended directory |
Underneath sit the 11 core detectors (the OWASP basics — injection, broken access control, misconfiguration shapes) that run on every scan including the free tier, and the secret scanner: 15 patterns free, the full 32 on Pro. The complete tier arithmetic is in our tier and detectors reference.
What an audit is not
Detectors find detector-shaped problems. They do not judge whether your business logic should allow a refund, whether your architecture trusts the right parties, or whether the feature itself is a liability — the honest boundary we draw in why AI code passes tests but fails human review. The audit's job is narrower and, for generated code, decisive: the known-bad patterns, found in minutes, with an exploitability ranking (the Top-risks view) and rotation runbooks for the secrets — so the human review you also need starts from findings instead of from a blank repo.
Frequently asked questions
What is a vibe code audit?
A security audit designed for AI-generated code: it checks the defect families coding assistants characteristically produce. Vibe-Code Audit runs 62 code detectors (11 core + 51 advanced), a 32-pattern secret scan on Pro (15 on Free), taint tracking, and an Advanced Patterns engine covering 19 families from JWT key confusion to LLM prompt-injection sinks.
What do AI coding assistants get wrong most?
They optimize for code that runs, so they reproduce the average training repo's security posture: permissive defaults, string-built queries, weak token verification, secrets committed as literals, and — newest — LLM-era sinks like prompt injection into tools and unsafe handling of model output.
How many detectors does Vibe-Code Audit run?
62 code detectors in total on the Pro path: 11 core patterns that run on every scan, plus 51 advanced detectors. The secret scanner checks 32 patterns on Pro versus 15 on Free.
Does an automated audit replace a human code review?
No. Detectors find known-bad patterns fast and completely; humans judge architecture, business logic, and intent. For high-stakes applications you want both — the audit first, so the expensive human hours start from a findings list.
What does the free audit include?
The 11 core detectors, the 15-pattern secret scan, security header checks, URL scans, and a shareable report — enough to find the OWASP-basics class of problems in a generated codebase at no cost.