AI is everywhere in detection engineering right now. Models draft rules, translate them between query languages, suggest tuning, and backtest against real logs. With that come the same questions, over and over. If the model has the docs, do you still need to know the query language? Can you trust what it writes? Where does the human actually fit?

This is Part 2 of Becoming a Detection Engineer, and this one is a snapshot of where the field stands on AI. I took the questions I keep hearing to a detection engineer I know, someone who has led incident response and writes detections, and then checked their answers against what teams are publishing.

"If the model has the docs, do I still need to know the query language?"

Their answer was an immediate yes, followed by a question back: how do you investigate somewhere that has no AI?

The deeper point is about trust. If you can't read the query, all you can really do is approve the rule. The same goes for tuning. Every exclusion the model suggests gets taken on faith, and if it quietly suppresses something important, nobody in the loop can tell.

"Can't you just give the model enough context?"

You can, and you should, but it won't be perfect. The model doesn't know the environment the way the people inside it do. Activity that's completely normal at a software company could be a major incident at a bank. Context helps the model make a better guess, and it's still a guess.

And as they put it, if the model really were perfect with context, there'd be no reason to hire a detection engineer at all.

"So what is AI actually good for?"

A lot, honestly. Over the past month it's been the fastest teammate I have. An AI-assisted audit found a batch of our rules that could never fire, by checking which fields actually show up in our data. When two of our log feeds went quiet, it traced the problem with a few targeted queries in minutes. It also walked me through our own deployment pipeline's code until I could explain every stage. All of that would have taken me far longer by hand.

But speed cuts both ways. The phrase we landed on was garbage in, garbage out. AI multiplies whatever judgement you bring to it. Good judgement gets faster. Bad judgement gets faster too.

It's also agreeable. Ask it for a bad detection and it will build one without pushing back. Let it propose a bad tuning decision and it will present it cleanly enough to believe. It will find connections that aren't really there.

I ran into this myself. During a cleanup, the model proposed rewriting a rule, and the new version looked better on paper. Before merging, I stopped and asked what had actually changed, and whether we were just changing the rule for the sake of changing it. That time the rewrite held up, but I only knew that because I checked. Nothing in the model would have asked that question. That's the part that has to come from a person.

Those dead rules from earlier are the same lesson from another angle. In Part 1 I wrote about how they assumed field names our data doesn't have. Our pipeline didn't catch them, because it only checks that a rule is well-formed. More on that in Part 3. Whoever writes a rule, a person or a model, someone has to be able to read it and check it against the data.

"Is that just one engineer's opinion?"

After that conversation, I wanted to know where the field actually is. So I read what teams using AI for detection work have published, and they land in roughly the same place.

Magonia's five-year retrospective on detection-as-code says LLMs are useful for auditing detections, catching errors and tagging ATT&CK techniques, but shouldn't be relied on to write the logic.

Nebulock's Agentic Detection Engineering Framework sums up its model in four words: "agents propose, humans approve." Its first release takes no autonomous actions at all. It never connects to the SIEM and never writes to the rule repo.

LimaCharlie's write-up on detection engineering with Claude Code gives the model the translation work, like converting community rules, and keeps the decisions that need expertise with analysts. When it tunes, it only suppresses alerts the model is highly confident are benign, and leaves the rest for a human.

The pattern is consistent. AI does the toil. Humans own the judgement. And the checking has to come from outside the model's own assumptions.

If you're using AI for detection engineering

  • Learn the query language anyway. You'll still use AI, and knowing the language is how you tell when it's wrong. Try writing the query yourself first and let the model check your work.
  • Treat every AI-written rule as a draft from a confident junior who has never seen your environment. Fast, often right, and still in need of someone who can say no.
  • Give it context, and keep the final call. You know what's normal in your environment far better than the model does.
  • Check against your own data. A backtest, a field check or a test event tells you something the model can't.

Next up, Part 3: detection-as-code and CI/CD, and what the pipeline actually checks.

Readings that helped