Detection rules fail in two directions: they can be too broad and create false positive alerts for the Security Operations Center (SOC) to triage, or they can be too precise and miss malicious activity. Either way, the fix starts by comparing what the rule matched against what actually happened.
If an attacker uses a stolen Microsoft 365 session to take over an account without tripping any detections, the question is what should have caught it: an existing rule that was too precise, or one that doesn't exist yet. LLMs can be useful for analysis like this, given large, structured datasets to work with. This task seemed reasonable to hand over: AI had produced good results on similar work before. The model quickly came back with a Microsoft 365 app identifier pulled from the attacker's activity and called it distinctive enough to build a new rule on.
The evidence the model turned up was real, but the conclusion was faulty. The model flagged the identifier as malicious because it appeared in a known-compromised mailbox and was uncommon in that organization. But the identifier was the Microsoft Outlook desktop client, which shows up in legitimate activity across more than 17% of our customer organizations. A detection built on it fires on all that legitimate activity.
Figure 1: Comparing how often the Microsoft 365 identifier appears in one organization vs the entire Huntress customer base
The data was right. The model just measured it against the wrong denominator. That's a basic scoping mistake and it happens all the time. Scope it wrong and you still get a real number, which makes it easy to miss.
It's also easy to check. Scoped more widely, that same query turned up the identifier that actually was malicious tooling. It showed up in only three organizations, wasn't in any Microsoft catalog, and did nothing but rewrite inbox rules. That logic is used in a live detection rule now, with two true positives and no false positives so far.
So don't just trust what the model writes; trust the check. This applies well beyond detection engineering: what you can check is what you can hand over.
Many people have certain tasks they refuse to delegate to AI. Some are protecting the work that matters most to them; others are avoiding the work that looks too risky, or too complicated. But the reason underneath is usually something else. The work people protect is almost always the work they have no fast way to check.
Risk tolerance matters here too, and sometimes it's enforced on you rather than chosen. A check won't change that. What it changes is the odds a wrong answer goes unnoticed. It costs a review. It buys the second pass.
Why you don't run the loop
The standard advice is to stop treating AI like a vending machine—one prompt in, one result out, walk away—and run a loop instead: generate, check, refine, go again. Andrej Karpathy calls the fast version of this the "generation-verification loop." The part that's talked about less is why you're not already running it.
Every turn of the loop costs tokens and a review. Tokens can scale with budget. Your time can't. And if the review costs the same as the work, the first turn only breaks even—the second puts you behind. So you take your one shot and stop. The people who loop fluently have made the review nearly free: a test, a metric, or a diff against something known-good.
Consider a rule we recently built: No engineer wrote it but one defined the scope, at which point the agentic workflow took over to prototype and test a new rule.
To determine if the rule was actually valuable, these questions had to be answered:
Is this already covered by one or more existing rules?
Does it detect the activity it was meant to?
How many alerts will it generate in a month?
None of those questions can be answered for free, at least not quickly. Each one is a check that has to be defined manually first. Once defined, AI can run each as a short, bounded pass instead of an open-ended investigation: cheap enough to spend on a working draft. It runs them and the detection engineer reads the results. And each one can fail. The first one did: most of the new detectors suggested already existed at a lower priority, so the work turned into one list expansion and three new experimental rules to expand the scope. That's the check earning its keep. None of that settled whether the rule was safe to ship, or whether it would create false positives at scale. What it bought was more passes in the same amount of time.
What makes work safe to delegate to AI
The check looks different in every job: tests for a coder, a rubric or a trusted reader for a writer, a known-answer dataset for an analyst, evals for a team shipping an AI feature, or the coverage and volume questions a new detection has to survive. But they achieve the same goal: reducing risk and providing confidence in the work. Either the work was done correctly, or the mistakes get uncovered for you. Because the check has to be able to fail. One that always passes is not a check at all. So if it fails, you know what to do next.
Recently I found myself correcting a model on the same thing over and over; it drifted from the guidelines I'd given it. You've probably been here. After the third time I stopped repeating myself and built a hook that catches the drift on its own. I'd been running that check by hand, so I made it run itself. Judge something by hand until the judgment is stable enough to encode; then the machine runs it and attention moves up to the next layer. The stack grows, and there's always a person on top of it. If a task you're guarding has no check, building one is usually the higher-value work, at least for anything you'll do more than a few times. Each one is a one-time build, and each keeps paying. The check is the asset.
Figure 2: Illustration of how the check and judgment loop expands the scope of a workflow
When you can't build the check
Sometimes the reason there's no check is that building one is out of reach. For a long time there was one piece of analysis I wanted and never attempted: a way to see what each of our detections actually cost by determining how much analyst time a noisy rule can quietly burn on alerts that go nowhere. I didn't sit on it because it wasn't worth doing. I sat on it because I was sure it was beyond me. No single system held the data. I'd have to pull from several places that didn't talk to each other, write the queries, parse the output, run the stats, build a whole pipeline, and then decide what any of it even meant.
Then AI made the build cheap, and I finally had it in about a week. AI wrote the queries and the plumbing I'd never have finished on my own; it's a reusable tool now, and it drops a report into a channel the team actually uses. The process had a shape: I decided what mattered, the machine did the labor in the middle, and I made the calls at the end. Kieran Klaassen calls this the AI sandwich: human on both sides, machine between. That's one rung of the stack, which is the shape every layer takes before the judging gets encoded and you move up. The judgment was still mine—what to measure, how to weigh it, where the lines sit, what counts as waste worth recovering—and so was the check.
Both of the bread layers of the sandwich have to be real. When I know the inputs and what a good answer looks like, the middle can be inferred. When I can't say what the output should be or how I'd score it, I make up the difference in the prompt by piling on instructions until the model is mostly repeating my assumptions back to me. Get both ends right and the tool stays in its place: it hands a person the evidence to decide what to tune, what to retire, what to leave alone.
What made this one unusual is that the build was the obstacle, not the check. Even so, once AI cleared it, what was left was judgment on the way in and a check on the way out. Far more often the verification is harder than the build.
Delegating, or just not looking?
Verification is also the easier half to fake. From the outside, delegating the doing and delegating the judgment look identical: same task, same model. The difference is whether you can still tell good from bad on the way out, and whether you actually looked. Rubber-stamp the check and you never really ran it. Then when a bad detection ships and misfires hundreds of times across a fleet, you're the one who has to explain it.
The skill moved
The real work is building the check, then learning to live on the other side of it. Automating a skill shifts the effort onto judgment, and there turns out to be more of that than the old job ever made visible.
Brian Concannon puts this bluntly in More Rules, Less Confidence: The cost of producing a rule fell to near zero, and "the hard part of detection engineering was never typing the rule. It was everything that happens after." His answer is a gate: test every rule against known-good and known-bad data before it ships, so junk never reaches production. My approach sits further upstream. The same check is what decides whether I take a second pass at all. A gate protects production from me; a check I can run cheaply is what lets me iterate.
Dan Shipper's After Automation makes a related, more general point: Once anyone (using AI) can produce the same competent output, the scarce thing becomes taste, the judgment of which output actually fits. Anyone can generate a rule now; knowing whether it's the right rule, and being able to prove it fast, is the part that keeps getting more valuable. Some of that is subjective: knowing what to look for, and what a passing result looks like, is its own judgment. But that ranks the options; the check decides which one is correct. That ordering only holds where there's a correct answer to find. In writing, in design, in most of what Shipper's describing, there's no oracle, and taste gets the last word because nothing downstream can overrule it. In the kind of work where being wrong is expensive and checkable, verifiability comes first.
What you keep when you hand over the doing
Then there's the harder kind of holding-on, the one that runs deeper than risk. You guard the work that is your expertise, because handing it over feels like conceding it was never that special. "Good at my job" used to mean doing the thing with my hands and producing the output. Now it means knowing what a good result looks like. Hand the "doing" over and what's left is the judgment. For many crafts, this was the better half all along, the part that made you worth listening to; the keystrokes were just the part that showed.
There's a real risk in the handoff, though. Give away all the "doing" and you can lose the instinct judgment runs on; you get worse at spotting a subtly wrong answer if you never produce answers yourself. So do enough of the work to keep your eye, and let the machine carry the bulk of the load once the check is built.
The question to ask instead
There's probably something you won't hand over to AI yet. Maybe it's the work you're best at; maybe it's the work you can't afford to get wrong. Either way, the question to ask yourself is whether you'd catch it, fast, if the answer came back wrong. If you wouldn't, you're missing a check. You still own the work; it just looks different. Build the check.
