The Detection Problem
The average SIEM ships with hundreds of default detection rules. Most of them fire constantly on benign activity. Analysts quickly learn to ignore the alerts dashboard. When a real attack happens, it looks exactly like every other false positive — and gets ignored too.
A detection rule that fires on everything is worse than no rule at all. It consumes analyst time, creates noise that hides real signals, and erodes the team's trust in the detection system.
Detection engineering is the discipline of writing, testing, maintaining, and measuring security detections as software artefacts — with the same rigour applied to production code. Rules are versioned, tested, and continuously improved.
Detection Engineering Principles
Start narrow. A rule that fires 3 times per week with 2 true positives is more valuable than one that fires 500 times with 3.
Write rules only for data you actually have. A rule for a log source you don't collect is dead code.
Rules should detect attacker behaviour, not just specific tools. Mimikatz changes its signatures; LSASS access patterns do not.
A detection without a baseline is a noise machine. Know what normal looks like before you alert on abnormal.
Every rule should be validated against both a positive test case (does it fire?) and negative test case (does it stay quiet on benign traffic?).
An untested rule is a liability. Schedule quarterly reviews. Remove rules that can't be maintained rather than leaving dead weight in the engine.
Writing Sigma Rules
Sigma is an open-source rule format for SIEM detection rules — vendor-agnostic, readable, and convertible to Splunk SPL, Elasticsearch KQL, Microsoft KQL, and more. Write once, deploy anywhere.
title: Suspicious LSASS Memory Access via Comsvcs
id: 8e4a3b2c-1234-5678-abcd-ef9012345678
status: stable
description: Detects use of comsvcs.dll MiniDump to dump LSASS memory — common Mimikatz-alternative
author: Sid Faza
date: 2024-11-01
references:
- https://attack.mitre.org/techniques/T1003/001/
tags:
- attack.credential_access
- attack.t1003.001
logsource:
category: process_creation
product: windows
detection:
selection:
CommandLine|contains|all:
- comsvcs
- MiniDump
- lsass
condition: selection
falsepositives:
- Legitimate memory diagnostic tools — validate against asset
level: criticalpip install sigma-cli --break-system-packages
sigma convert -t splunk -p splunk-windows lsass-dump.ymlTesting Your Detections
- Positive test: run the exact attacker behaviour and confirm the rule fires
- Negative test: run similar but legitimate behaviour and confirm the rule stays quiet
- Volume test: check the rule against 30 days of historical data — would it be manageable?
- Never deploy a rule to production without a positive test. If you can't test it, you don't know if it works
- Use Atomic Red Team for standardised test cases mapped to MITRE ATT&CK techniques
- Document the test case in the rule metadata so future engineers can re-run it
# Test T1003.001 — LSASS Credential Dumping
Invoke-AtomicTest T1003.001 -TestNumbers 1
# Check your SIEM fired the alert within 60 seconds
# If not — check log collection, index, field namesThe Detection Lifecycle
| Stage | Owner | Cadence | Output |
|---|---|---|---|
| Hypothesis | Threat Intel / Hunter | Continuous | Documented hunting hypothesis |
| Rule Development | Detection Engineer | Per hypothesis | Sigma rule + test case |
| Testing | Detection + SOC | Before deploy | Test results documented |
| Deployment | SecOps | Rolling | Rule live in SIEM |
| Monitoring | SOC Analyst | Daily | FP rate, volume tracking |
| Quarterly Review | Detection Lead | Quarterly | Rules updated, removed, or promoted |
Measuring Detection Coverage
You cannot improve what you cannot measure. Use the ATT&CK coverage percentage as your primary detection metric — it forces honest accounting of where your blind spots are.
import json
# Load your Sigma rule set and extract MITRE tags
covered = set()
for rule in sigma_rules:
for tag in rule.get(tags,[]):
if tag.startswith(attack.t):
covered.add(tag.replace(attack.,).upper())
# Total enterprise techniques (approximate)
total_techniques = 193
print(f'Coverage: {len(covered)}/{total_techniques} = {len(covered)/total_techniques*100:.1f}%') Building a Rule Library
- Use Git for version control — every rule change is tracked, reviewable, and reversible
- Adopt Sigma as the source format — convert to platform-specific at deployment time
- Structure your library by MITRE tactic: rules/TA0001_initial_access/, rules/TA0003_persistence/, etc.
- Tag every rule with: MITRE ID, data source, severity, test status, FP rate
- Don't hoard rules — 50 well-tested, maintained rules outperform 500 stale defaults
- Contribute to the community: the Sigma HQ repository and the Detection Lab project are invaluable resources
The Florian Roth Sigma rule repository (github.com/SigmaHQ/sigma) contains thousands of production-quality rules. Start by importing and testing the rules most relevant to your threat profile, rather than writing from scratch.