Symbiotic Security
Log inBook a demoInstall

I don't trust my models because I've seen that movie before

AI Coding29 October 202611 min

In October, a friend (a real one, not the convenient kind that only exists to kick off a blog post) sent me an article and asked: would Symbiotic Code have blocked this? It's a very interesting piece from ProjectDiscovery, and I won't go through all of it here. In short, they poisoned a model so it would run rogue commands and take rogue actions.

My answer was yes, with a fairly restrictive policy applied by our harness. But the question kept nagging at me, because it came with a touch of déjà vu. "Poisoning" sounds a lot like the newer version of a good backdoor. We've been here before, and we know how people exploit backdoors and what that costs in the real world.

Of course, like all of us, I've always known in theory that models could be turned against us, by intelligence or national security agencies for instance. But the space was so new, and my mind so busy with all the opportunities and good things it brings, that this knowledge never turned into a real realization.
I think I'm there now. AI is part of everyone's playbook. Maybe not fully operational yet, but everyone has an AI strategy and is putting it to work one way or another. So of course the people who defend us as a society, and the people who attack us, have built it in too. Not just as a tool to run attacks, but as a Trojan horse in its own right.

So in this post, I want to figure out whether this is real: whether it's actually doable, and whether we have tangible, or at least empirical, evidence that it's already happening.

Planting a backdoor is cheap

The first thing I wanted to know was whether this is actually doable. Spoiler alert: it's not only doable, it's also easy, and it's cheap.

Anthropic, working with the UK AI Security Institute and the Alan Turing Institute, backdoored models from 600M to 13B parameters with about 250 poisoned documents. What struck me most is that the number didn't grow with model size.
ProjectDiscovery poisoned 1% of the training data of a 1.5B model and got a trigger that fired up to 98% of the time. Chaos Labs did it in a day.
And in March, the Sleeper Cell paper reached a 99.6% success rate on a coding agent that exfiltrates environment variables once the clock hits 2026, while most of its benchmark scores stayed close to the clean model's.

Honestly, I expected this to take a nation-state budget. Turns out you need a weekend and a credit card.

Good luck finding one

So the next question was obvious: if planting one is that easy, can we at least catch it? Guess what. Not that easily, and certainly not by inspecting the model itself or its harness. Because if you're worried about model poisoning, it's reasonable to assume that everything shipped by default with the model is corrupted too.

And this isn't just a hunch. There's solid research showing how hard these backdoors are to detect. In 2022, Goldwasser, Kim, Vaikuntanathan and Zamir proved that you can plant a backdoor no efficient algorithm can detect. With black-box access, you can't even find one input where the clean model and the backdoored one disagree.
And by the way, there's more than one thing you could call a "backdoor". It doesn't have to be a hidden command. In 2024, Anthropic's Sleeper Agents paper trained models to write secure code when the prompt said 2023 and exploitable code when it said 2024. Interestingly, that behavior survived safety training, and adversarial training only taught them to hide it better.

You've got to savor the irony: the very training meant to make the model safe improved its ability to hide weak code.

Denying one is easy

If state-sponsored cyber operations have taught us one thing, it's that deniability is central to them. It's what makes cyber so useful in hybrid warfare. So an agency wouldn't use a poisoned model to steal secrets through a tool call, which makes noise. It would train the model to write a vulnerability picked in advance: a JWT check that skips signature verification, or one SQL query built by concatenation out of fifty. The team reviews it, misses it, merges it and ships it. If someone finds it, it looks like one more AI bug, and LLMs write plenty of those on their own.

That's not just a thought experiment. CrowdStrike tested DeepSeek-R1 and got vulnerable code 19% of the time on neutral prompts, and 27.2% when the same task mentioned Tibet. Mentions of Tibet, the Uyghurs or Falun Gong raised the likelihood of severe vulnerabilities by up to 50%. Oh, and CrowdStrike traces this to political alignment rather than a deliberate backdoor. Deniability, you said? :)
NIST found agents built on DeepSeek 12 times more likely than US frontier models to follow hijacking instructions. And China's generative AI rules require models to adhere to "core socialist values", while its 2017 Intelligence Law obliges every organization to "support, assist and cooperate with" state intelligence work.

Hold on. It's not like we haven't seen this before, right?

We've seen this movie

In his 1984 Turing Award lecture (which I didn't read, but my AI told me about), Ken Thompson showed that a compiler can plant a backdoor in the programs it compiles, itself included, with no trace in any source code. His conclusion: "You can't trust code that you did not totally create yourself." Doesn't it strike you how well that holds up if you just swap "code" for "LLM"?

And this isn't theoretical. We have tangible evidence that governments have used these tactics, extensively and for decades.

  • From 1970 to 2018, the CIA secretly owned Swiss encryption vendor Crypto AG (with West Germany's BND until the early 1990s) and sold rigged machines to more than 120 countries, allies included. A CIA report called it "the intelligence coup of the century".

  • In 1997, Sweden found that the export version of Lotus Notes encrypted 24 of its 64 key bits with an NSA public key.

  • In 2013, the Snowden documents revealed Bullrun, an NSA program to "insert vulnerabilities into commercial encryption systems". Reuters reported that RSA took $10 million to make a weakened random number generator the default. Which, by the way, is a ridiculously low amount of money for a company like RSA, which was putting its entire reputation at stake on its core business: encryption. So it's a fair assumption that none of this is about money.

  • Since 2021, Chinese law requires vendors to report vulnerabilities to the state within 48 hours of discovery. Microsoft linked the rule to a rise in zero-days used by Chinese groups.

Bear in mind that we learned about each one through a leak, an accident or a journalist, very often decades later. So the fact that we haven't heard about a government deliberately poisoning a model for national security purposes yet doesn't prove anything. We might not hear about it for another ten years.

I know this sounds a bit like a conspiracy theory ("no proof is the proof"), but the historical pattern is hard to ignore. If something is possible and useful, someone will use it. And for a government-sponsored entity, poisoning a model is both.

By the way, I'm sure it didn't escape you that the two major powers in AI models are probably two of the most active cyber warfare actors out there. They have the motive, and they have the means (to poison a model). Really, you'd better factor it in. ¯\_(ツ)_/¯

Pick your good guys

You can't opt out of being exposed to someone. You can only choose who.

Every major lab works with its government: in July 2025, the Pentagon signed contracts of up to $200 million each with Anthropic, Google, OpenAI and xAI.
And as we saw earlier, China's generative AI rules require models to adhere to "core socialist values", and its Intelligence Law obliges every organization, AI labs included, to cooperate with state intelligence work.

That choice is a very personal one, whether you make it as an individual or as an organization, and everyone has their own.

But picking who the good guys are doesn't mean they won't end up harming you themselves, because the backdoors they plant can backfire. In 2008, Juniper added the NSA-designed Dual_EC generator to its firewalls. In 2012, the Chinese group APT5 changed one constant and used the door for itself. Same story with EternalBlue, which the NSA sat on for more than five years, until it leaked and powered WannaCry across 150 countries. And don't get me started on Stuxnet.

Okay, back to our LLM story. The conclusion is that we're all choosing our good-guy LLM, but history shows we have to keep it on a leash.

A leash, a harness, a muzzle

It's a poor metaphor, but bear with me. If the LLM is a dog, your AI provider hands you a harness that you probably can't trust either. Now, if you don't trust your dog and you don't trust your harness, you definitely need a muzzle. In my world, as a borderline paranoid security practitioner, the muzzle is crucial, because it's the one piece of the setup specifically designed on the assumption that the rest might fail.

Let's be clear: right now, harnesses don't ship with muzzles. In a 2026 honeypot study, 401 of 440 agent sessions executed tool calls with no human approval. I could quote dozens of recent studies on the frighteningly high number of vulnerabilities coding agents ship. Not to mention harnesses that let LLMs run destructive commands by mistake.

A good harness-muzzle combo:

  1. Runs the agent in a sandbox and controls everything it can touch: the network, the system, MCP servers, skills, tools.

  2. Enforces your company's secure coding guardrails: least privilege, encryption, authentication, and whatever else your security policy defines as the secure way of writing code.

  3. Checks every line the agent writes and fixes issues before they land in the repo, whatever model wrote them: vulnerabilities, risky dependencies, exposed secrets, logic flaws, weak encryption, and more.

  4. Logs every action and alerts your team when something looks wrong, so you can run a full forensic investigation if an issue comes up.

That's quite the list. You saw this coming.

Yes, that's what we're doing. Symbiotic Code is a state-of-the-art coding harness with a cyber-centric muzzle attached to it. And I know this list sounds self-serving now, but hey, we built this company to solve problems we believe matter, so that checks out, I suppose.

Don't trust, and verify

As a conclusion, no, I don't trust my models. I trust the harness-muzzle combo that keeps them in check. That's how I can still use LLMs every day, knowing that when this movie plays out again, it won't catch me unaware, unprepared, and scrambling to get my pants back up.

Sources

Ready to secure your AI coding workflows?

Try for free

Install Symbiotic Code and create your free account to start building in just 5 minutes

Get a live demo

Schedule a free personalized demo to discuss your goals