Nobody reads your alerts, and it isn't their fault
Every security operations team I've worked with has a queue of alerts that nobody meaningfully triages. Not because the analysts are lazy, but because at some point the queue stopped being a signal and started being weather — something that happens to you, that you can't do much about, that you learn to work around.
The instinct is to fix the people: more training, stricter SLAs, a dashboard showing who cleared the most tickets. That almost never works, because alert fatigue isn't a discipline problem. It's a design problem, and it has design fixes.
The three questions every alert should answer
Before a detection goes live, it should be able to answer three things. If it can't, it isn't ready.
What happened? Not "suspicious PowerShell detected" — that's a category, not an event. An alert should say which host, which user, which parent process, and what the command line actually was. If an analyst has to run three queries to understand what the alert is about, you've moved the work rather than done it.
Why does it matter here? The same behaviour is routine on a developer workstation and alarming on a domain controller. Context that exists in your CMDB but not in your alert is context your analyst doesn't have at 3am.
What should I do about it? Every detection needs a response path, even if that path is "confirm with the user and close". An alert with no defined next step will accumulate a folklore response instead, and folklore is not repeatable.
Measure the wrong thing and you'll get the wrong thing
The metric most teams track is alerts closed. It is almost useless. It rewards speed of dismissal, and the fastest way to close an alert is to decide it's a false positive.
Two metrics that behave better:
- Actionability rate — what proportion of alerts from a given rule resulted in any action, including a deliberate tuning change. A rule sitting at 2% is not a detection; it's a tax.
- Detections per incident — after a real incident, how many of your existing rules should have caught it and didn't? This is the only measure that tells you about the alerts you aren't getting.
Track them per rule, not per team. The team average hides the two rules generating 80% of the noise.
Tuning is a first-class activity, not cleanup
The most common failure I see is that nobody owns tuning. Detections get written during a project, deployed, and then left, because tuning is unglamorous and never appears on a roadmap.
Some things that help:
- Give every rule an owner and a review date. A detection with no owner is a detection nobody is allowed to change, which means it can only ever get noisier.
- Make "tune this rule" a valid alert disposition. If the only options are true positive and false positive, you've made the fix invisible.
- Review the top five noisiest rules every fortnight. Not all of them — five. It's small enough to actually happen.
- Delete rules. A detection that has never produced a true positive in eighteen months is not insurance. It's noise with a good story attached.
Start with coverage, not volume
When a team is drowning, the temptation is to turn things off until the queue is manageable. That works, but it's blind — you don't know what you've stopped seeing.
A better sequence is to map what you have against MITRE ATT&CK first, so you know which techniques you're covering, then decide what to cut. It's a far more comfortable conversation with leadership when you can say "we removed forty rules and our technique coverage went up" — which happens more often than you'd expect, because noisy rules crowd out the work needed to build good ones.
The goal was never fewer alerts. It was alerts that mean something.