Agentic SOC: Which AI, for Which Problem, in Microsoft Sentinel

The agentic SOC is not one technology. It is a stack of very different AI, each with its own economics, failure modes and evidence base. Getting the assignment right is most of the work.

What is an agentic SOC?

An agentic SOC uses AI agents (language models with goals, tools and permission to act) alongside classical machine learning and deterministic playbooks to triage, investigate and respond to alerts. In practice it is a stack, not a product: each job in the SOC belongs to the type of AI whose cost, accuracy and auditability fit it. The rest of this guide is about making that assignment.

The best alert classifier in the test was invented in 1995

In June 2026, Expert Systems with Applications published the head-to-head a lot of people have been arguing about without data. Researchers at the University of Chester took 178 manually labelled security alerts: 104 true positives, 74 false positives, generated against Wazuh and Suricata using Atomic Red Team techniques mapped to MITRE ATT&CK and ran eight large language models against three ordinary machine-learning classifiers on the same task [1].

The winner was a linear support vector machine. F1 score 89.64%, false positive rate 13.51%. The best language model, GPT-4.5 Preview, managed F1 85.84% with a false positive rate of 33.78%; two and a half times as many false alarms. GPT-4o reached an F1 of 77.17%, with a false positive rate of 70.27% [1].

Then the cost column. GPT-4.5 Preview cost $0.1247 per alert to run [1]. A SOC handling 10,000 alerts a day would spend roughly $455,000 a year on that model, to classify alerts worse than a technique that predates the iPhone by twelve years.

Caveats, because they matter: 178 alerts is a small set, they came from a simulated lab rather than a live estate, and the authors used zero-shot prompting without tuning [1]. This is one study, not a law of nature. But it points at something the agentic SOC conversation keeps skating over.

AI is not one thing you switch on. It is six or seven different technologies with wildly different costs, failure modes and evidence behind them. The teams getting value are the ones treating it as an assignment problem, which tool for which job, rather than a purchase. So the question is not whether to use AI in the SOC. It is which one, where, and on whose evidence.

“AI” is six different things in a trench coat

Strip the marketing and AI is just several methods for joining a function to data. They layer like an onion, each ring wrapping the one inside rather than replacing it.

At the core, linear and logistic regression is a handful of coefficients, fully inspectable, still running most of the risk scoring in the world. Around it, splines and additive models: the straight line bent where the data bends, each variable’s contribution still readable. Then classical machine learning, i.e. trees, random forests, gradient boosting, support vector machines. These handle interactions and non-linearity, train in minutes, score in milliseconds, and remain strong on structured, labelled, tabular problems. Which is exactly what a security alert is: forty fields and years of analyst dispositions.

Outside that, neural networks trade interpretability for learning their own features. Then transformers, and with them large language models, which make language the interface. The outer ring is agents: an LLM with a goal, a set of tools, and permission to plan its own route.

Two consequences, and the market keeps skipping both.

First, every ring outward costs more to run and explains itself less. So you reach outward only when the inner rings genuinely cannot do the job; not because the outer ring is newer. The Chester result is that principle showing up in a controlled test [1].

Secondly, an agent is not smarter than a gradient-boosted classifier. It is more general. Generality is what you want when the task is open-ended and a human reads the output. It is the last thing you want when the task is “score this the same way every time and let me measure whether you were right.”

Sentinel already runs several of these rings. Fusion correlates low-fidelity anomalies into multistage detections using machine learning that has nothing to do with language models, and UEBA baselines behaviour the same way [2]. The agentic layer arriving on top is an addition to that stack, not a replacement for it.

The attackers already picked their tool

The Chester result is one data point. Here is the other, and it points the opposite way.

At EACL 2026, researchers from the University of Illinois Urbana-Champaign set a team of coordinated LLM agents against fourteen real zero-day vulnerabilities, all disclosed after the model’s training cut-off, with no description supplied. Forty-two percent fell within five attempts, at $4.39 per run. Open-source scanners on the same targets solved none [3]. Here the outermost ring of the onion wins outright, and the deterministic tools do not score at all.

Two studies, opposite winners. What separates them is not the quality of the models. It is the shape of the problem.

Finding an exploitable path is open-ended, rare and inexpensive to get wrong. The attacker runs many attempts, pays $4.39 for each failure, and needs one to land. Classifying an alert is narrow, repetitive and expensive to get wrong. It runs ten thousand times a day, every result is a verdict someone may have to defend, and the same mistake repeats until somebody finds it. Open-ended and inexpensive favors the outer rings. Narrow and repetitive favors the inner ones.

That is the rule for the rest of this piece. The tool follows the shape of the problem, not the capability of the model.

Defence has one variable the attacker does not. For the attacker the cost of a wrong answer is fixed at $4.39. For a SOC it is not fixed. It depends on which alerts you decide to accept error on, and how much. No SOC gets everything right and none is funded to try, so part of the shape of a defensive problem is something you decide yourself.

Which is why the three choices that follow have no single answer. Each is a dilemma: two opposite courses, both defensible, resolved only once you say what you are willing to weigh.

Dilemma one: a number you can measure, or an output you can argue about

Triage is the narrow, repetitive end of the scale, so the rule says inner rings. But there is a real choice to make first, and both sides of it are taken by serious teams.

A number you can measure. Classical ML is deterministic by construction. Same alert, same model, same probability, today, next Tuesday and in the post-incident review. That is what makes evaluation possible. Hold out a labelled test set, build a confusion matrix, state recall and precision as numbers that mean something. Regression-test a model change the way you regression-test code. When someone asks why an alert was closed, show the features and their weights. What it costs you is time: labelled history, scoped rules, and weeks before it is worth anything.

An output you can argue about. A language model needs none of that. No labels, no training, no scoping. It reads the alert and tells you what it thinks, in English, on Friday. That is a real advantage and it is why so many pilots start there. What it costs you is the number.

The Chester study shows the size of that cost. On binary true-positive and false-positive classification the LLMs were at least in the game. On prioritization they fell apart: false negative rates on high and critical alerts ran from 17% to 63% across the models that did not simply escalate everything, and the one that missed nothing got there at 19.6% precision [1]. A layer that misses between a sixth and two thirds of what needed escalating is not triage. It is a second queue.

The deeper issue is what quality comes to mean. With a classifier, quality is a measurement: F1 was 89.64% on the holdout set [1]. With a language model, quality is an opinion about the output, unless you build an evaluation harness: a graded set, a rubric, a scoring model, a named owner. That harness is real engineering work, and it is the work almost every agentic SOC pilot skips.

So decide it on the question you will be asked at month six. Is this layer getting better or worse? The measured option can answer, which is why its errors shrink over time. The other cannot, which is why its errors are simply whatever they were on day one. If you cannot answer that about a layer of your SOC, you do not have a capability there. You have a subscription.

FIGURE 2 Cost against accuracy on the same 178 labelled alerts. The best result is the one with no per-alert price.

Dilemma two: close it by machine, or close it by hand

Dilemma one was about what the layer produces. This one is about whether its output is allowed to end anything, and it gets decided per alert rule rather than once.

Close it by machine. The verdict stands without a person. What you accept is that some share of what was closed was real, and that you will not find out which until it surfaces somewhere else.

Close it by hand. Every closure has a person behind it. What you accept is that person’s own error rate, and a coverage ceiling set by headcount: detections you cannot staff do not get switched on.

Closing by hand is treated as the safe option. That is the assumption worth testing, because the human error rate is not zero and it is no longer unmeasured. In a field study at a working SOC, operators triaging real alarms from their own environment got 83% of the calls right [4]. One in six wrong. The same study found that only 39% of the explanations they gave reflected the actual root cause of the decision they had just made [4].

Read those two numbers as a risk position rather than as a criticism of analysts. Closing by hand means carrying an error rate near one in six, on a queue that grows every time you add a detection, with a reasoning trail that does not survive inspection in most cases. You cannot sample it, you cannot state it to an auditor, and you cannot tell whether it improved last quarter. It is not a lower risk than the machine. It is an unpriced one.

Closing by machine carries error too, and a machine will be wrong the same way every time until someone notices. The difference is that it is wrong in public. Every closure is a score against a rule, so you can pull a month of them, re-run them against a new model, and put a number on what you missed.

So base the choice on which error you can act on afterwards. That is also why the distinction most vendors blur has to survive here: detection rate is not auto-closure rate. Classifying false positives correctly is not the same as closing them automatically. Auto-closure is set per alert rule and is opt-in, so a team might auto-close six rules and work the rest by hand, with a verdict and a reason already attached. That gap is not a shortfall. It is the control surface, and it is what makes machine closure survivable on the rules where you choose it.

Dilemma three: the playbook decides, or the agent decides

Dilemma two was about ending an alert. This one is about starting a response, and the question is who holds the authority to act.

The playbook decides. Deterministic response. Trigger, condition, action, the same every time, and it fires in seconds. You can read it beforehand and know exactly what it will do. What you accept is that it only covers what somebody anticipated and wrote down. Everything else waits for a human, and the playbook inventory grows and rots.

The agent decides. The model plans the response, picks the tools and adapts to what it finds. It covers the situations nobody encoded, and the EACL results show how capable that is on open-ended work [3]. What you accept is two things: the same incident can produce a different response twice, and every response costs thinking time.

That second cost is the one most people underestimate. A single classification call in the Chester study took between 1.88 and 85.8 seconds [1]. An agent is not one call, it is a loop of them with tool use in between, so a decision lands in minutes rather than seconds. Set that against how fast the threat moves: CrowdStrike puts average breakout time at 29 minutes, with the fastest observed at 27 seconds (vendor figure) [9]. Minutes of deliberation is a meaningful share of the window.

So base it on what the action costs when it is wrong. Revoking a session, forcing reauthentication, quarantining a file: reversible, low impact on the user, inexpensive to undo. For those, thinking about it costs more than getting it wrong. Fire the playbook instantly and review afterwards. Isolating a domain controller or disabling a service account is a different matter, and there the minutes are worth paying, because a person would have taken longer and been no more consistent.

Microsoft builds its own platform on the same split, pairing policy-bound controls with task agents [8]. The rule that follows fits on a wall: an agent choosing which pre-approved action to run is still the playbook holding authority. An agent composing an action of its own is not.

Which means this dilemma is answered per action type, not once for the SOC, and that only works when both mechanisms sit on the same platform. That is what Seculyze provides: responses you design yourself from inputs and conditions, firing in seconds on the reversible actions, and an agent that fires automatically on true positives and can be set up across the rest of your rules.

After the false positives are gone

The three choices above decide what noise is and what to do when it is not. What is left is a small number of real incidents, with a different shape.

Open-ended, because no two are alike. Tool-heavy: intelligence, related signals, what the entity did before and after. And a person reads it before the incident is called, so the machine concludes nothing. That is the shape the EACL agents were scored on [3]. An agent assembling it hands the analyst a case with the intelligence pulled and the timeline built, which shows in time to close and in cost, because that layer runs across tens of cases and not tens of thousands.

Place threat intelligence deliberately. In front of classification, it has to be affordable at full alert volume, so open-source in practice, and everything downstream inherits it: the classifier gains features to score on, the agent gets reputation already attached. Behind classification, commercial feeds priced per lookup become affordable, but reach only investigations already underway.

Writing is the other half, and the larger share of the time. Incident reports, monthly service reports, executive summaries, evidence packs, handovers, all turning facts the platform already holds into prose for a specific reader. In our own customer conversations, these tasks take a quarter to half of SOC time: our observation rather than a published benchmark. The gain is not only the hours. A monthly report that is written properly every month is worth more to whoever receives it than one that slips whenever the queue is bad.

Both sit on the same side of one line: a person reads the output before anything happens, so an error costs an edit rather than an incident.

FIGURE 3 Two universes. The same incident, two routes to an action, and what each costs in time.

What the recovered capacity is actually for

Machine closure and machine drafting give hours back. Most teams spend them on a quieter queue, and that is the wrong instinct.

A quieter queue reduces the risk of mishandling what you already see. It does nothing about what you never see, and the uncovered surface is larger than most estates assume. In the DEEPCASE study, across 10.5 million real security events from twenty organizations, hand-written expert rules covered only 25% to 37% of events, despite being near-perfect on the events they did cover [6]. Precision on a narrow slice, nothing at all on the rest.

An adversary moving across the ATT&CK chain only needs it unobserved where they happen to touch it, so coverage behaves like a product rather than a sum. A gap anywhere is a gap in the whole.

The risk arithmetic here changed when triage stopped being human. Under hand closure, adding detections meant adding decisions at roughly one in six error [4], so cutting rules was rational. Under a measured classifier at a lower rate, you can add rules until the SOC is handling the number of incidents it handles today, with the noise removed, and total error falls rather than rises. Same headcount, same volume of human work, more of the estate watched.

So ask where your incidents are actually arriving from. Through alerts that were seen and mishandled, or through techniques that never raised an alert at all? Most estates running three years or more are the second, and are spending as though they were the first. Our best-performing customer took the wider net: up to +124% more MITRE ATT&CK coverage [5], and the published manufacturing case landed +65% alongside a 24% reduction in Sentinel spend [7]. Neither came from making the queue smaller.

The assignment map

The question is never “should we use AI.” It is which ring of the onion this job belongs to. The answers are stable, because they follow from the shape of the problem rather than the state of the market.