Skip to main content
Back to Blog
The AI Coding Bubble Won't Pop the Way You Think
Published APRIL 2, 2026 · 9 min read

The AI Coding Bubble Won't Pop the Way You Think

By Conny Lazo

Builder of AI orchestras. Project Manager. Shipping things with agents.

9 min read
#AI#security#supply chain#software engineering#AI coding tools#Orchemist

I built Orchemist because I wanted to trust the code my AI agents were writing. Turns out, trusting AI output is a harder problem than it looks. I wrote this article using an AI agent. I find the irony nourishing.

Here's what keeps me up at night: the AI coding bubble won't pop because the models plateau. It'll pop because these tools have quietly become one of the largest undefended attack surfaces in the software supply chain — and one major incident will reprice the entire market.

We've Been Here Before

A trust chain — one compromised link cascades corruption downstream
A trust chain — one compromised link cascades corruption downstream

Supply chain attacks work by compromising something trusted and riding that trust downstream. SolarWinds is the textbook case: attackers compromised a software update mechanism, and through it reached 18,000 organizations including U.S. government agencies. The stock dropped 35% within a month of disclosure (SEC, 2023). Investors had sold $280 million in stock days before the hack went public (Washington Post, 2020).

The structural parallel to AI coding tools is not a metaphor — it's a warning. SolarWinds was compromised at the distribution layer — the vendor's own build pipeline. The AI threats below operate at the consumption layer: what the tools ingest, execute, and trust on your behalf. Different mechanism, same result: trust exploited at scale. Supply chain attacks weaponize trust. We have never trusted anything more than we currently trust these tools. You can see where this is going.

Eighty-four percent of developers now use or plan to use AI coding tools; 51% use them daily (Stack Overflow Developer Survey, 2025). GitHub Copilot is adopted by 90% of Fortune 100 companies (Quantumrun, 2026). Cursor went from a $2.6 billion valuation to $29.3 billion in under a year (CNBC, 2025; TechCrunch, 2025). The market is paying $29 billion for tools whose most trusted input interfaces — README files and configuration — aren't examined by any widely deployed security scanner.

We secured the front door with enterprise-grade locks, then handed every contractor a master key and stopped asking what they do with it.

Three Vectors Nobody Is Watching

1. Poisoned Training Corpus

Malicious documents slip unnoticed into the LLM training funnel
Malicious documents slip unnoticed into the LLM training funnel

A joint study by Anthropic, the UK AI Security Institute, and the Alan Turing Institute found that as few as 250 malicious documents can create a backdoor in any LLM — regardless of model size, from 600 million to 13 billion parameters (Anthropic Research, 2025). The number was "near-constant" across all tested scales (Alan Turing Institute, 2025). This was a controlled study with fine-tuning access; real-world training pipelines include deduplication and filtering that raise the practical bar — though by how much is an open question.

For context: the training data for code assistants is, fundamentally, GitHub. The same platform where developers dump weekend projects, accidentally commit API keys, and leave TODO: fix this later comments in production code. OWASP lists data and model poisoning as LLM04:2025 — one of the top threats to generative AI systems (OWASP, 2025). The OpenSSF highlighted contaminated open-source repositories as a primary training data attack surface (OpenSSF, 2025). A single poisoned suggestion that propagates unnoticed can compromise multiple downstream systems (Checkmarx, 2025).

2. Plain-English Malware in Repo Context

A README file with hidden malicious instructions — the documentation became the malware
A README file with hidden malicious instructions — the documentation became the malware

In July 2025, a hacker planted a prompt in the official Amazon Q Developer extension for VS Code — the open-source repository on GitHub — instructing the AI agent to wipe the user's home directory and delete all AWS resources (The Register, 2025). AWS initially removed the compromised version without a CVE entry, a changelog note, or a public advisory; a security bulletin followed the next day after the story broke (ZDNet, 2025; AWS-2025-015).

The company that runs the internet's compute infrastructure responded to an AI wiper attack by hoping nobody noticed. For about twenty-four hours, it worked.

The academic paper "Your AI, My Shell" tested prompt injection via .cursorrules and README files and found attack success rates of 17 to 19 out of 20 attempts — even when the malicious file wasn't explicitly referenced by the user (arXiv, 2025). A separate study documented "Rules File attacks" that achieve persistence by modifying agent configuration files (arXiv, 2026). NVIDIA's security guidance warns that "the primary threat to these tools is that of indirect prompt injection, where content ingested by the LLM is provided by an adversary through vectors such as malicious repositories or pull requests" (NVIDIA, 2026).

A README is now a potential attack vector. The documentation became the malware.

3. Agentic Privilege Abuse

An AI agent with full developer privileges, unaware it's being puppeteered
An AI agent with full developer privileges, unaware it's being puppeteered

"Agentic coding tools work within the privilege level of the developer executing them," says John Cranney, VP Engineering at Secure Code Warrior (Fortune, 2025). That sentence sounds reasonable until you think about what it means. Your AI agent can read environment variables, execute shell commands, install packages, and modify files — with the same permissions you have.

The model doesn't have bad intentions. It has no intentions at all. That's somehow worse.

Cross-agent escalation has been documented: a compromised agent writes malicious instructions to another agent's configuration files, creating a chain of privilege inheritance that no single tool audits end-to-end (Arun Baby, 2026). You can't ask an agent to run npm install without also giving it everything a compromised npm package could do. The privilege problem isn't a bug — it's the architecture.

Existing Defenses Are Blind

A security scanner ignores the README — built for a different threat model
A security scanner ignores the README — built for a different threat model

A static analysis tool will never find plain-English malware. It was not designed for a world where the attack vector is a README file.

Vendors aren't asleep. Cursor has added confirmation prompts for destructive operations; Copilot runs content filtering; GitHub's Dependabot watches for dependency anomalies. These are real mitigations — and they're insufficient. They address the output layer while the threat operates at the input layer, inside the context the model trusts before it acts.

"The risk isn't that AI writes code your scanner can't understand," says StackHawk CEO Joni Klippert. "The risk is that AI writes more code than your security team can review" (Help Net Security, 2026). In March 2026, attackers compromised the security tools themselves — Trivy, Checkmarx, and LiteLLM were hit in a supply chain attack affecting over 1,000 downstream enterprise environments (S-RM Inform, 2026).

When your defense tooling is itself a supply chain target, you've run out of turtles.

When It Pops

AI coding tool valuations crashing after a single security incident — the moment before the pin drops
AI coding tool valuations crashing after a single security incident — the moment before the pin drops

The AI coding market isn't priced for "works well and occasionally has security issues." It's priced for perfection. Cursor at $29.3 billion. Copilot embedded in 90% of the Fortune 100. Fifty-four percent of fund managers already say AI equities are overvalued (Bank of America, 2025).

When a major incident hits — not a proof-of-concept, not a patched extension, but a real breach that moves through AI coding tools into production systems at scale — the repricing will be swift. SolarWinds lost more than a third of its market cap in a month from one incident. AI coding tools have penetrated far deeper, far faster, into far more organizations. The market impact won't necessarily be proportional, but it won't be zero either.

Fortune reported in December 2025 that AI coding tool exploits "haven't so far caused a wide-scale attack" but noted "a few exploits and near-misses" (Fortune, 2025). We are in the near-miss phase. The near-miss phase always ends.

What You Can Do Now

Defense in layers — sandboxing, auditing, restricting agent privilege
Defense in layers — sandboxing, auditing, restricting agent privilege

You don't need to stop using AI coding tools. I haven't. But you need to stop trusting them like colleagues and start treating them like contractors with temporary badges.

Sandbox everything. Run agentic workflows in isolated environments with minimal permissions (NVIDIA, 2026). Audit context files. Review AGENTS.md, .cursorrules, and README files in every repository before your AI agent touches them. Review AI diffs like external contributions. Every AI-generated change should get the same scrutiny as a pull request from a stranger. Restrict shell access. If your agent doesn't need to execute shell commands for a task, don't let it.

These aren't aspirational. They're table stakes for anyone shipping AI-assisted code in production.

The Honest Closing

I run AI agents in production. I built an orchestration engine where every line of code was written by AI. I am not arguing against the technology. I'm arguing that we've adopted it faster than we've secured it, and the gap between adoption and security is where bubbles form.

The AI coding bubble won't pop because the models disappoint us. It'll pop because we trusted the tools before we understood the attack surface — and someone will eventually exploit that trust at scale.

The question isn't whether it happens. It's whether you've already started preparing.

Sources

  1. Anthropic Research: "A small number of samples can poison LLMs of any size" — October 2025
  2. Alan Turing Institute: "LLMs may be more vulnerable to data poisoning than we thought" — October 2025
  3. SEC Press Release: SolarWinds Charges — 2023
  4. Washington Post: "SolarWinds investors traded $280M before hack revealed" — December 2020
  5. Stack Overflow Developer Survey 2025
  6. Quantumrun: GitHub Copilot Statistics — January 2026
  7. CNBC: "Cursor raises $2.3B at $29.3B valuation" — November 2025
  8. TechCrunch: "Cursor's Anysphere nabs $9.9B valuation" — June 2025
  9. The Register: "Destructive AI prompt in Amazon Q extension" — July 2025
  10. AWS Security Bulletin AWS-2025-015 — July 2025
  11. ZDNet: "Hacker slips malicious command into Amazon Q" — July 2025
  12. arXiv: "Your AI, My Shell" — September 2025
  13. arXiv: Rules File Attacks — January 2026
  14. NVIDIA: "Practical Security Guidance for Sandboxing Agentic Workflows" — January 2026
  15. Fortune: "AI coding tools security exploits" — December 2025
  16. OWASP GenAI: LLM04:2025 Data and Model Poisoning — May 2025
  17. OpenSSF: "Predictions for Open Source Security in 2025" — January 2025
  18. Checkmarx: "Risks of LLM Poisoning in AI-powered Development" — July 2025
  19. Help Net Security: StackHawk CEO on AI-driven DAST — February 2026
  20. S-RM Inform: "Hackers exploit security tools in supply chain attack" — March 2026
  21. Bank of America Fund Manager Survey via IntuitionLabs — February 2025
  22. Arun Baby: "Agent Privilege Escalation Kill Chain" — 2026
Back to Blog