And that’s the uncomfortable truth, we need to hear.
When a new frontier model drops, what does 99% of the internet do?
“Can you build GTA 6?”
“Make me a 3D spaceship that files.”
“Write a Pixar movie in one prompt.”
Meanwhile, a tiny fraction on the other side is asking something very different:
“Can you autonomously scan 30 organizations, write your own exploits, harvest credentials, steal data, and write the indecent report without me having to be in the loop?”
And according to Anthropic, Google, Mandiant, and Microsoft that second group is not hypothetical. It’s already happening.
TL;DR
While most people test a new frontier model with “build me GTA 6 Like game?” or “make a 3D spaceship,” a small group on the other side asks if it can autonomously attack organizations. that question is no longer hypothetical. The factory is already running. One operator with AI agents can now do the work of an entire crew, and defenders constrained by OPEX, approvals and staffing are struggling to keep pace.
1. It’s No Longer “AI Helps Hackers.” It’s “AI is the Hacker”
For two years we talked about AI as an assistant. A co-pilot that writes a better phishing email or helps debug malware code. That phase is over.
It starts with a prompt that got refused
Recently, I was hunting a phishing cluster against a large multi-brand retailer. Mid-hunt, I pasted A CAPTURED CREDENTIAL-POSTING ROUTINE into a frontier model and asked it to walk me through what the code did.
Refused. I reworded it. Refused again. I got there eventually, but it cost me roughly 30 minutes of wall-clock time on a live incident.
That night I stopped being annoyed and got curious about a different question: what are the people on the other end of this campaign running on?
The answer reorganized how I think about the gap between us.
Tier three: the criminal LLM is the cheap seats
Everyone writes about WormGPT, so let’s dispose of it first.
WormGPT isn’t a model anymore. It’s a brand. Cato CTRL’s teardown found variants riding on jailbroken Grok and Mixtral, built by threat actors adapting existing LLMs rather than training anything from scratch. Palo Alto Unit 42’s look at the current generation WormGPT 4 and KawaiiGPT puts subscriptions at roughly $50/month to $220 for lifetime access, sold on the promise of AI without boundaries.
Then Unit 42 said the quiet part out loud: much of the malware these tools generate is still readily detectable, and less advanced than code observed in recent AI-assisted campaigns.
Sit with that. The purpose-built criminal LLM the thing that gets the headlines and the dark-web screenshots is the entry-level option. The most sophisticated operators targeting national retailers probably aren’t relying on Telegram bot,
Tier two: your model, their account
The serious operators use the same models we do. They just don’t pay, and they don’t sign their name.
Google’s Threat Intelligence Group documented the plumbing: custom middleware, proxy relays and automated registration pipelines built specifically to bypass safety guardrails and billing limits, with anti-detect browsers and account-pooling services keeping high-volume anonymized access to premium tiers alive, subsidized by trial abuse and programmatic account cycling. One PRC-nexus cluster used a publicly hosted script that handled registration, CAPTCHA bypass, SMS verification and cancellation end to end. Another aggregated Gemini, Claude and OpenAI accounts behind relay services so a group could pool and cost-share.
Anthropic’s September 2026 threat report closes the loop on why this is now a category of crime rather than a workaround. Stolen AI credentials give an operator three things simultaneously: loot with resale value, compute so the workload runs at someone else’s expense, and cover, because the activity gets attributed to the credential’s legitimate owner. One hacktivist campaign in that report ran for a month entirely on stolen API keys.
So the asymmetry isn’t model quality. They’re on the same frontier models, on someone else’s bill, with someone else’s name on the logs.
Tier one: they prompt as us
Here is the part that actually stung.
a suspected China-nexus espionage group, prompting Gemini to act as a senior security auditor or C/C++ binary security expert. The prompt claimed the user had already extracted a device’s filesystem and was auditing it for pre-authentication remote code execution. GTIG says these fabricated scenarios supported vulnerability research into embedded device targets, including TP-Link firmware and Odette File Transfer Protocol (OFTP) implementations. GTIG calls the technique persona-driven jailbreaking and notes that threat actors often use expert personas this way.
Limitation of Guardrails:
The filter cannot see intent. It sees request shape. An attacker who writes like a defender gets served. A defender who is one gets a refusal and pays a workaround tax measured in minutes per task, every task, forever.
Which brings me to what that asymmetry buys, when it lands on a customer.
“We recently argued that an inflection point had been reached in cybersecurity: a point at which AI models had become genuinely useful for cybersecurity operations, both for good and for ill.” Anthropic
We have crossed that inflection point.
2. Why Attackers Win the OPEX vs Output Game
Blue teams live in the real world. Every tool needs budget approval, compliance review, SOC integration, staffing and training. It’s OPEX vs output. Slow, careful, and accountable. Attackers have none of that. And AI agents supercharged their biggest advantage.
New open-source agent harnesses drop almost weekly OpenCode, DSH, Hermes Agent, and dozens more. ESET analyzed ~900,000 AI “skills”(functional components for agents) in H1 2026 and found suspicious grew from 10,000 to over 25,000 and outright malicious ones from 600 to over 3,000 in just months. Most were built to exfiltrate data, download malware or silently override user instructions.
They separate frontend and backend, rotate instantly
Classic red-team infrastructure is now industrialized: separate payload servers, phishing servers and C2s hidden behind legitimate cloud redirectors (AWS CloudFront, Azure FrontDoor) and reverse proxies. To defenders, traffic looks like normal AWS or Slack API calls.
In my hunt, attackers were creating login pages for a renowned commerce chain, and multiple domains were serving the phishing page via a centralized kit hosted on Alibaba Cloud.
The collapse to one. The finding that reframed the whole engagement: 10+ domains, one brain. Each storefront was a disposable shell. The templates, the configuration and the harvested data all resolved back to a single content and API server.
One domain hosted with AI chat artifact, possibly they shipped that by mistake.
The admin layer. These were not amateur pages. victim-session tracking, bot gating and researcher redirection, template management, real-time data handling. The administration layer self-identified as tkshop.
This sophistication is the valid proof that moving that fast with so lean strategy highly needs AI assistance,
3. Defenders Are Catching Up. But How Fast?
Yes, the blue side is not standing still.
But the gap is structural. Google’s Mandiant M-Trends 2026 found median dwell time increased to 14 days in 2025 (from 11) driven by espionage and DPRK IT worker operations. Trend Micro predicts 2026 as the year scams become AI-Driven, AI-Scaled and Emotion-Engineered with multi-channel lures moving victims from SMS to chat apps to fake payment pages seamlessly.
Attackers adopt a new harness overnight. Enterprises need quarters to adopt and secure the same harness.
4. The GTA 6 Problem
This is the cultural problem no one wants to say out loud. When a state-of-the-art model releases GPT Astra, Claude Opus 5.5, Gemini Argon the overwhelming use is toys. Games, 3D spaceships, Studio Ghibli filters. Fun demos that get millions of views.
Attackers see the same model and see automation of reconnaissance, exploitation, evasion, and extortion at machine speed. They see cost reduction. They see they don’t need a team of 5 anymore.
“If you’re a sophisticated actor, what would have otherwise required maybe a team of sophisticated actors … now, a single individual can conduct, with the assistance of agentic systems.” Jacob Klein, Head of Anthropic Threat Intelligence, on vibe hacking
5. How Bolster AI Can Help You?
You cannot block a factory with a manual checklist. You need to fight agents with agents.
This is the part of the problem we built for at Bolster. Our detection engine runs headless-browser analysis with LLM and computer-vision classification across millions of records continuously, which is what turns a forty-day manual correlation into a standing query.
Takedowns with packaged evidence through direct registrar and hosting-provider APIs at a mean time to response of around 60 seconds with roughly 95% removed without analyst intervention and keeps watching for reappearance after removal. Our in-house agentic scam-capture workflows push that further: agents that pivot, enrich and escalate on their own, so the humans spend their time on the central server instead of the storefronts.
