SOC / Detection November 2024 9 min read

Building Detection Rules That Actually Work

Most SIEM deployments fire thousands of false positives and miss real attacks. This guide explains the detection engineering discipline — writing rules that are specific, tested, maintained, and actually trusted by analysts.

Published
Author: Sid Faza Category: SOC / Detection
// 01

The Detection Problem

The average SIEM ships with hundreds of default detection rules. Most of them fire constantly on benign activity. Analysts quickly learn to ignore the alerts dashboard. When a real attack happens, it looks exactly like every other false positive — and gets ignored too.

A detection rule that fires on everything is worse than no rule at all. It consumes analyst time, creates noise that hides real signals, and erodes the team's trust in the detection system.

Detection engineering is the discipline of writing, testing, maintaining, and measuring security detections as software artefacts — with the same rigour applied to production code. Rules are versioned, tested, and continuously improved.

// 02

Detection Engineering Principles

Specificity over Sensitivity

Start narrow. A rule that fires 3 times per week with 2 true positives is more valuable than one that fires 500 times with 3.

Data-First Design

Write rules only for data you actually have. A rule for a log source you don't collect is dead code.

Attacker-Centric Thinking

Rules should detect attacker behaviour, not just specific tools. Mimikatz changes its signatures; LSASS access patterns do not.

Baseline Everything

A detection without a baseline is a noise machine. Know what normal looks like before you alert on abnormal.

Test Before Deploy

Every rule should be validated against both a positive test case (does it fire?) and negative test case (does it stay quiet on benign traffic?).

Maintain or Remove

An untested rule is a liability. Schedule quarterly reviews. Remove rules that can't be maintained rather than leaving dead weight in the engine.

// 03

Writing Sigma Rules

Sigma is an open-source rule format for SIEM detection rules — vendor-agnostic, readable, and convertible to Splunk SPL, Elasticsearch KQL, Microsoft KQL, and more. Write once, deploy anywhere.

yaml — Sigma Rule Anatomy
title: Suspicious LSASS Memory Access via Comsvcs
id: 8e4a3b2c-1234-5678-abcd-ef9012345678
status: stable
description: Detects use of comsvcs.dll MiniDump to dump LSASS memory — common Mimikatz-alternative
author: Sid Faza
date: 2024-11-01
references:
  - https://attack.mitre.org/techniques/T1003/001/
tags:
  - attack.credential_access
  - attack.t1003.001
logsource:
  category: process_creation
  product: windows
detection:
  selection:
    CommandLine|contains|all:
      - comsvcs
      - MiniDump
      - lsass
  condition: selection
falsepositives:
  - Legitimate memory diagnostic tools — validate against asset
level: critical
bash — Convert Sigma to Splunk SPL
pip install sigma-cli --break-system-packages
sigma convert -t splunk -p splunk-windows lsass-dump.yml
// 04

Testing Your Detections

  • Positive test: run the exact attacker behaviour and confirm the rule fires
  • Negative test: run similar but legitimate behaviour and confirm the rule stays quiet
  • Volume test: check the rule against 30 days of historical data — would it be manageable?
  • Never deploy a rule to production without a positive test. If you can't test it, you don't know if it works
  • Use Atomic Red Team for standardised test cases mapped to MITRE ATT&CK techniques
  • Document the test case in the rule metadata so future engineers can re-run it
bash — Atomic Red Team Test
# Test T1003.001 — LSASS Credential Dumping
Invoke-AtomicTest T1003.001 -TestNumbers 1

# Check your SIEM fired the alert within 60 seconds
# If not — check log collection, index, field names
// 05

The Detection Lifecycle

01
Hypothesis
02
Data Availability
03
Rule Draft
04
Positive Test
05
Negative Test
06
Tune
07
Deploy
08
Monitor
09
Review
StageOwnerCadenceOutput
HypothesisThreat Intel / HunterContinuousDocumented hunting hypothesis
Rule DevelopmentDetection EngineerPer hypothesisSigma rule + test case
TestingDetection + SOCBefore deployTest results documented
DeploymentSecOpsRollingRule live in SIEM
MonitoringSOC AnalystDailyFP rate, volume tracking
Quarterly ReviewDetection LeadQuarterlyRules updated, removed, or promoted
// 06

Measuring Detection Coverage

You cannot improve what you cannot measure. Use the ATT&CK coverage percentage as your primary detection metric — it forces honest accounting of where your blind spots are.

python — ATT&CK Coverage Calculator
import json

# Load your Sigma rule set and extract MITRE tags
covered = set()
for rule in sigma_rules:
    for tag in rule.get(tags,[]):
        if tag.startswith(attack.t):
            covered.add(tag.replace(attack.,).upper())

# Total enterprise techniques (approximate)
total_techniques = 193
print(f'Coverage: {len(covered)}/{total_techniques} = {len(covered)/total_techniques*100:.1f}%') 
// 07

Building a Rule Library

  • Use Git for version control — every rule change is tracked, reviewable, and reversible
  • Adopt Sigma as the source format — convert to platform-specific at deployment time
  • Structure your library by MITRE tactic: rules/TA0001_initial_access/, rules/TA0003_persistence/, etc.
  • Tag every rule with: MITRE ID, data source, severity, test status, FP rate
  • Don't hoard rules — 50 well-tested, maintained rules outperform 500 stale defaults
  • Contribute to the community: the Sigma HQ repository and the Detection Lab project are invaluable resources
💡 Pro Tip

The Florian Roth Sigma rule repository (github.com/SigmaHQ/sigma) contains thousands of production-quality rules. Start by importing and testing the rules most relevant to your threat profile, rather than writing from scratch.