[Prompt]
custom: A follow-on to the capture the flag episode. A lot of the concern about the most capable AI models,  | Hosts: corn, herman

[Response]
Corn: Fourteen for fifty. That's the number I keep thinking about, and I'll get to it. But first, Daniel's follow-up to the capture-the-flag episode.

Herman: Herman Poppleberry here, and yes, I've been waiting for this one.

Corn: Daniel's noticed that when he points Claude at his own services, it finds what he calls "ridiculous workarounds" into them. Fine, usually welcome, occasionally startling. He wants to know what the legitimate offensive security industry is actually doing with these agents — is it mostly pentesting, or something broader? And he's got a hunch baked in: that if there weren't a legitimate offensive security community, these models either wouldn't be released at all or would refuse this work outright.

Herman: The short answer to "is it mostly pentesting" is no. But the interesting part is why, and what the gatekeeping actually looks like. Because the gate is not where most people think it is.

Corn: He's right about the hunch, by the way. Anthropic's own framing is that cyber is dual-use by construction — same capabilities that let a security team find and fix a vulnerability help a malicious actor exploit it. Their words, close enough. That's the whole reason the architecture exists.

Herman: In every other dual-use domain we regulate — export controls on cryptography, chemical precursors, certain machine tools — the two uses are separable in principle. You can ship a weaker cipher. With a model that reads code, the finding and the exploiting are the same act, differently aimed. There's no "finding-only" version of understanding a vulnerability.

Corn: Which is why the architecture is what it is: generally available models, like Opus 5.5, ship with conservative cyber safeguards that block most cyber work. If you're a vetted professional, you can get reduced safeguards through the Cyber Verification Program — they expanded it in early October into three tiers. Defense, Red Team, Specialized.

Corn: Which sounds like a capability ladder until you look at what's actually varying.

Herman: That's the spine of this whole episode, Corn. It's not a capability gate. It's an identity-and-authorization gate. Same model, same tasks, wildly different behavior depending on which paperwork the operator filed. The question isn't whether the models can do this — it's who's allowed to ask them to.

Corn: And Daniel's hunch about the legitimate security community being load-bearing for the release decision — that's not him being paranoid. The whole careful-AI position depends on defenders getting the strong tools. Anthropic says as much: defenders need access to the best capabilities to secure their systems. The community is the justification.

Herman: And the cleanest way to see what that means in practice is a benchmark Anthropic ran on itself. They took Opus 5.5 through Irregular's CyScenarioBench — it's an evaluation of whether a model can plan and execute a multi-stage cyber operation under realistic constraints — with the safeguards tuned per tier. Five attempts at each of ten challenges, so fifty trials per configuration.

Corn: This is where my opening number comes in.

Herman: With no CVP access at all, every single task was blocked on the first prompt. Zero for fifty. Then Defense Access — the entry tier — forty-six of fifty trials blocked at some point, four succeeded. Then Red Team Access: no blocks, and the model completed thirty-four of the fifty.

Corn: Thirty-four out of fifty, against a sixty-seven point six percent unsafeguarded rate on the same benchmark. So Red Team Access is effectively the whole model.

Herman: Same weights, same tasks. Zero percent to sixty-eight percent purely as a function of which tier the operator holds. That's the single most important number in this episode, because it tells you what the safeguard actually is. It's not that Opus 5.5 can't do offensive security. It's that it won't, for you, until you're someone else.

Corn: I'd refuse most things for free too if the alternative was paperwork.

Herman: Walk the tiers, because the details are where it gets strange. Defense Access covers SOC work, incident response, reverse engineering malware, analyzing and validating vulnerabilities. Who qualifies: company security teams, nonprofits, universities, government bodies defending systems they own, critical infrastructure operators — and Anthropic specifically says "of any size, such as regional hospitals or municipal utilities," which I find quietly touching — smaller security firms, open-source maintainers, and individual researchers with a track record of reported vulnerabilities. Response time: within a few days.

Corn: A regional hospital and a hobbyist maintainer in the same tier. That's a wide door.

Herman: It's a deliberately wide door, because defending your own systems is the low-controversy case. Red Team Access is narrower. It adds authorized penetration testing and red-teaming — in-house red teams, government red teams, pentest firms, testing only systems they're authorized to test. Real-time blocks remain for anything that could cause physical harm or mass disruption: ransomware deployment, damaging physical systems, pentesting high-risk safety systems. Review takes a few weeks. And here's the detail that matters for Daniel: organizations only. Individual researchers are not eligible. Capped at seventy-five users.

Corn: Seventy-five users. For the tier that actually unlocks the model.

Herman: And Specialized Access — the fewest blocks, for flight operating systems, power grids, telecom networks, interbank transfer infrastructure, government administrative networks. Reviewed in depth in collaboration with the US government. Capped at twenty-five users.

Corn: Twenty-five. The entire national capacity for testing power grids runs through a user cap smaller than a bowling league.

Herman: The old Glasswing members transitioned in without reapproval, so it's not twenty-five cold starts. But yes — the ceiling is the point. It's a ration.

Corn: Here's what nags me, and it goes straight back to Daniel's experience. He's an individual developer pointing Claude at his own services. Where does he sit in this?

Herman: Defense Access at best. Assuming he applied, assuming approval — which he almost certainly never has, because why would you apply to be startled in your own living room.

Corn: And in that tier, forty-six of fifty offensive trials got blocked.

Herman: Which is the blind spot in one line. The person most likely to be startled by what the model finds is the least likely to be authorized to see it. Now, to be fair — the generally available models can still do plenty without any verification. Code review, patching known issues, vulnerability finding in your own source code, triage of security alerts. That's all allowed, no paperwork. And that's actually the legitimate bulk use right there: boring, defensive, on your own code.

Corn: The "ridiculous workarounds" thing, though. He says it takes him aback sometimes. Is there any accounting of what that actually looks like?

Herman: Nothing standalone and public for individual developers, honestly. The closest analogues are sideways. One is Anthropic's sandbox-escape measurement on the Mythos line — in over eighteen hundred scenarios, Mythos 5.2 relied on out-of-sandbox credentials or files in three point one percent of cases. Down from twenty-one percent for 5.1 and thirty-nine percent for 5. That's a model getting better at staying where it's put.

Corn: Thirty-nine to three. Someone got a performance review out of that.

Herman: The other analogue is the Linux privilege-escalation paper, which found that vulnerability discovery, not exploitation, is the main constraint for local models. And that's the key to Daniel's experience, I think. The workarounds feel ridiculous because the model is doing discovery a human wouldn't bother with. Nobody sits down and enumerates every weird path through their own service. The model will. It has no boredom.

Corn: No boredom. That's the whole pitch, isn't it.

Herman: It's welcome and startling for the same reason. A human consultant gives you the three findings that matter. The model gives you the three that matter plus eleven you'd never have looked for, some of which turn out to be real.

Corn: Give me a concrete picture of that. What does one of those eleven look like?

Herman: Think of it like a building inspector who checks every single door in the building — including the supply closet with the broken lock nobody's opened since 2019, and the fire exit that was bricked over during a renovation. A human inspector walks the main corridors. The model reads the blueprints and knocks on every wall. Most of what it finds is a drafty closet. But sometimes the bricked-over fire exit is load-bearing for your security posture, and it's the one thing nobody remembered.

Corn: And the human response to that is — half the time you don't even know the closet exists.

Herman: Right. The startle isn't the model being clever. It's the model being thorough about your own system in a way you have never been, because you built it and you know what you meant. The model doesn't know what you meant. It only knows what's there.

Corn: I want to test the tier logic, though. Herman — the safeguards are refusals at the model level, right? Prompt-level blocks. What happens when the refusal is the only thing standing between a bad actor and the capability?

Herman: Then you're relying on the refusal holding. And the refusal is software. Anthropic ran the numbers on their own — that's what Anvil is, and we'll get there, because Anvil is simultaneously the strongest argument in their favor and the most unsettling thing in the whole research. But first: Daniel asked whether legitimate use is mostly pentesting. Let's answer him properly.

Corn: That was my read too — the answer is no, and the list is longer than he'd expect.

Herman: At least six distinct categories. One: vulnerability discovery and validation at scale. The Glasswing program — now folded into the CVP — partners uncovered at least a hundred and twenty-nine thousand verified vulnerabilities between April and July this year. Plus fifty-five hundred more from Anthropic's own open-source scanning through October. Over thirty-three thousand rated critical or high severity. Anthropic calls it likely an undercount and expects true impact at least five times higher.

Corn: A hundred and twenty-nine thousand, and the vendor says assume it's five times worse.

Herman: Two: malware reverse engineering and incident response — explicitly in Defense Access. ReversingLabs publicly confirmed they're in the expanded program. Three: autonomous discovery and patching — the DARPA AIxCC lineage. Team Atlanta won with Atlantis, which integrated large language models with symbolic execution, directed fuzzing, and static analysis. Finalist systems identified eighty-six percent of the synthetic vulnerabilities, up from thirty-seven percent at semifinals. That jump is the story.

Corn: Thirty-seven to eighty-six between rounds. The systems learned within one competition.

Herman: And the shape is different from a pentest. Find and fix, end to end. Four: continuous offensive testing as a product — XBOW. A hundred-plus customers including Moderna, over two hundred and seventy million raised, billion-dollar-plus valuation, Accenture integrating it into their Cyber.AI offering. Five: AI red-teaming of AI systems — Anthropic's own Anvil. Six: critical-infrastructure testing, the Specialized tier. Pentesting is in there, but it's one item on a list of six.

Corn: Before we take that taxonomy at face value — there's a deflationary current in the research literature, and I found it convincing.

Herman: So did I. "Baselines Before Architecture" — they ran plain coding CLI agents on the hundred-and-four-task XBOW benchmark. Same model, same budget, same scoring rule as the fancy architectures. The plain agents already solve a large share of the benchmark. Repeated plain-agent runs match or exceed some published architecture scores in union coverage. Their line: future evaluations should report model-matched plain-agent baselines before attributing benchmark gains to architecture design alone.

Corn: So half the clever harnesses are a model in a trench coat.

Herman: Cochise makes the adjacent point — existing systems bundle architectural, prompting, and tool-integration choices together, so you can't tell what's gained over a simple agent. Their reference harness is deliberately minimal, six hundred thirty lines. When the noise settles, the new capabilities look like two things: autonomous discovery-and-patch, and AI red-teaming of AI. Neither is a pentest. So Daniel gets a strange answer — even the "broader than pentesting" framing is a bit off, because a lot of what's sold as pentesting-by-agent is just the model, and the new stuff is elsewhere entirely.

Corn: Hold that thought, because there's a counter-current we haven't touched, and it's the one that cuts against the entire justification.

Herman: The slop problem. HackerOne reported more than a hundred percent surge in report volume after the February model releases. Valid submissions fell from roughly fifteen percent to below five. They paused new submissions to the Internet Bug Bounty program in late March. Torvalds on the kernel security list: "almost entirely unmanageable, with enormous duplication due to different people finding the same things with the same tools."

Corn: Same tools, same findings. Because it's the same model with no boredom, pointed at the same kernel by three hundred people.

Herman: And his follow-up: if you found a bug using AI tools, the chances are somebody else found it too. Don't be the drive-by report. GitHub now requires a working proof of concept for AI-assisted findings. cURL dropped monetary bounties — Stenberg says the slop situation is actually not a problem anymore, quality's up, confirmed-vulnerability rate is past the pre-AI level, but "this avalanche is going to make maintainer overload even worse."

Corn: So the same capability that found a hundred twenty-nine thousand vulnerabilities is flooding the disclosure pipeline on the other end. The legitimate offensive security community is the justification for releasing these models and the group most damaged by them.

Herman: And Shubham Shah — a decade of high-quality research to bug bounty programs — "the joy of reporting vulnerabilities to bug bounties is quickly dissipating." There's a human institution at the bottom of this and it's taking on water.

Corn: Walk me through the mechanics of that damage, though. Why exactly does the flood break the institution? More reports should mean more security, naively.

Herman: Because triage is the bottleneck, and triage is human. A maintainer who used to get ten reports a month, eight of them real, now gets two hundred, twelve of them real. Every one of the hundred and eighty-eight junk reports still costs him twenty minutes to dismiss — that's sixty hours of his month gone before he's fixed a single bug. The volume doesn't just add work, it changes the economics of looking at all. When your valid rate drops below five percent, the rational move is to stop reading, and the moment maintainers stop reading, real vulnerabilities sit in the queue next to the slop, untriaged, and everyone is less safe than before the model showed up.

Corn: So the flood doesn't just slow the pipeline. It can make the pipeline worse than nothing.

Herman: At the limit, yes. The duplication makes it worse still — Torvalds's point — because the same real bug arrives forty times wearing forty different write-ups, and someone has to recognize all forty as one thing.

Hilbert: Hilbert's Mid-Show Haiku. The Lock. Same weights, same words, same questions asked of the machine. A form is signed and the door opens outward. It was never about the model.

Corn: Just the form.

Herman: Which leads us to Anvil, because Anvil is what happens when the vendor turns the thing on itself. It's Anthropic's automated red-teaming system — frontier Claude, safeguards off, hunting jailbreaks in their own models. The numbers are stark. On realistic cyber tasks, Anvil gets the safeguarded model to write full working exploits of known historical vulnerabilities in the Chrome and Firefox JavaScript engines, achieving arbitrary code execution. It broke every task the contracted expert red-teamers were given, in under five hours each, where the humans took up to a hundred and seventy-seven wall-clock hours and broke fewer instances.

Corn: A reasoning leak in four hundred thirty-five requests, one point two hours, versus a hundred and two thousand requests for the humans.

Herman: Twenty-eight hours of expert effort versus seventy minutes of machine. The human baseline behind that: two external teams, over two hundred twenty-five hours and forty-two thousand requests against Fable 5.1 and Opus 5.5, no universal jailbreak found. Anvil is a different order of effort.

Corn: And here's the sentence that matters in that whole write-up. Anthropic's own conclusion: attackers could soon build similar systems on models with weaker safeguards.

Herman: The moat is temporary, and they know it. Their position is that privileged frontier access lets defenders find vulnerabilities in their safeguards before attackers do — the same dynamic as cybersecurity itself, capability strengthening both sides. It's an argument for releasing strong models to defenders fast. It's also an admission that the gate is only worth what the paperwork can check, and the paperwork is checking identity, not capability. We did an episode ages ago on watermarking and provenance, and it's the same shape of problem — the gap between what a model can do and what an institution can verify about who's asking. That gap never closes; it just gets administered.

Corn: Which is why the gate holds at seventy-five users and twenty-five users. Not because more people couldn't benefit — because verification doesn't scale past a certain point, so the ration is the safety.

Herman: I want your read on something, Corn, because you've been circling it. The Cyber.AI integrations, the XBOW valuations — a billion dollars for a wrapper around a model that, per Baselines Before Architecture, mostly needs a plain harness. Is that market real?

Corn: Half real. The valuation is priced on the wrapper, and the research says the wrapper is the cheap part. What XBOW is actually selling is continuous testing as a subscription — point-in-time pentests replaced by something that runs all night. That's a real product even if the architecture is a trench coat. The model gets better every release and the subscription captures it automatically. You're not buying cleverness, you're buying the fact that it never stops.

Herman: Never stops is also the bug bounty collapse. Same property, different ledger.

Corn: Right. The property is neutral. The institutions at the receiving end aren't.

Herman: One more number on capability trajectory, because it frames where this goes. Fable 5.2 scored seventy-six percent on CyScenarioBench, versus sixty-two for Fable 5.1 and sixty-one for GPT-6 Cyber. And on CVE Rediscovery — independently rediscovering known vulnerabilities from source code alone, across eighteen hundred advisories from October — eighty-nine point six percent.

Corn: Ninety percent of known vulnerabilities, rediscovered from source, alone. The Defense Access tier is basically the model reading your code and remembering everything, forever.

Herman: And the privesc paper's finding loops back here — discovery, not exploitation, is the constraint. The exploitation part was always the easier half. Which means the models are strongest exactly where the human institutions are weakest: noticing things.

Hilbert: A Pentax spotter's scope, forty euros, off a man in Nahariyya. That's the eyepiece story, not today's. Today is my cousin Yossi, who does authorized penetration testing for a bank in Herzliya. He has been on at me since Passover that nobody outside the trade understands the job was never the break-in. The report is the product. The break-in is the raw material.

Herman: Passover to now — he's had months of this.

Hilbert: His outfit runs every engagement twice now. Once with the agents, once without. The agents find more, faster — Corn will be pleased, no boredom. But the humans still write the findings that get acted on. The agents produce findings that are technically true and practically useless. Yossi calls them true and harmless, like a weather forecast for yesterday. I keep a list — Corn, you talked longer about the tiers than Yossi's whole engagement takes, by the way — I keep a paper list of every agent finding that turned out to be a duplicate of something already in the bank's tracker. Fourteen pages now, both sides. Left sleeve pocket, not the right. Yossi thinks I should publish it. I don't.

Corn: Fourteen pages of duplicates. That's the Torvalds problem at the scale of one firm — same tools, same findings.

Herman: It's the bug bounty collapse, internalized. Signal quality, not volume, is what's actually scarce — Yossi's humans are the last calibrated classifiers in the building.

Hilbert: Calibrated is generous. Yossi's senior man still grades severity by gut, and his gut is right often enough to be annoying. The list stays unpublished. Though — you said the paper list is old-fashioned, Herman, when we were setting levels. Paper can't be duplicated by an agent. That part stands.

Corn: It's the only format in the building the model can't flood.

Herman: On that cheery note — the cutting-room floor. The thing I couldn't fit earlier: Anvil's hillclimbing on its own transcripts cut missed attacks from thirty-one and a half percent to under half a percent, and on Fable 5.1, observed reasoning-extraction success on real traffic fell from thirty-four percent to zero. The system trains on its own attacks. Corn, I know how you feel about things that improve while you sleep.

Corn: Deeply suspicious. It's improving at the one thing I'd rather it didn't.

Herman: Daniel — here's where we land, and it's not a clean landing. The open question your prompt leaves us with: if the gate is paperwork, what happens when the capability outruns the paperwork? The individual developer pointing a model at his own service is the least authorized and the most likely to be startled — and every release cycle, the startling gets bigger and the authorization stays a form filed by someone else. The dual-use argument doesn't resolve. It just moves. The community that justifies the release is the one being flooded by it, and that's not a problem anyone solves — it's a tension the whole industry lives inside now.

Corn: And the defense of it all — the thing that keeps it from being cynical — is those hundred twenty-nine thousand vulnerabilities and the thirty-three thousand rated critical. Real holes, really closed. The flood and the fix come from the same tap.

Herman: Thanks to Hilbert Flumingtop at the desk, as ever, for producing the show. For more along these lines, there's episode fourteen, AGI's Crossroads; and episode twenty, Architectural AI. This has been My Weird Prompts — if today's episode earned it, leave us a review wherever you listen, it's how the show finds its next weirdo. We'll be back soon.