SOC / Detection March 2025 8 min read

Understanding SIEM: From Logs to Alerts

A clear-eyed explanation of how SIEM platforms actually work — log collection, normalisation, correlation rules, alerting, and why most SIEM deployments fail to deliver on their promise.

Published
Author: Sid Faza Category: SOC / Detection
// 01

What is a SIEM?

A Security Information and Event Management (SIEM) platform collects, normalises, correlates, and analyses security logs from across an environment to detect threats and support incident response. The name is clunky; the function is essential.

A SIEM is not a magic threat detection box. It is a data aggregation and correlation platform that is only as useful as the data you feed it, the rules you write for it, and the analysts who use it.

SIEMs have two core components: SIM (Security Information Management) — log storage, search, and compliance reporting; and SEM (Security Event Management) — real-time correlation, alerting, and dashboards. Most modern platforms do both reasonably well. Getting value from either requires significant ongoing work.

// 02

How SIEM Works — Architecture

01
Log Source
02
Agent/Syslog
03
Collector
04
Parser/Normalise
05
Index/Store
06
Correlate
07
Alert
08
Analyst
Log Collection

Agents deployed on endpoints, syslog forwarding from network devices, API connectors for cloud services. Every uncollected source is a blind spot.

Normalisation

Raw logs come in hundreds of formats. The parser translates them into a common schema — same field names regardless of source device.

Indexing & Storage

Normalised events are indexed for fast search. Retention policy determines how far back you can investigate. 90 days minimum; 12 months preferred.

Correlation Engine

Rules that fire alerts when multiple events match a pattern. E.g., 5 failed logins followed by a success from the same IP triggers a brute-force alert.

// 03

Log Sources That Matter Most

SourceLog TypeKey EventsPriority
Windows EndpointsWindows Event Logs4624/4625 (logon), 4688 (process), 4698 (task)Critical
Domain ControllersAD Event Logs4768/4769 (Kerberos), 4776 (NTLM), 4662 (DC access)Critical
Firewall/NGFTraffic + ThreatAllow/Deny by IP, IPS alerts, C2 communicationCritical
EDR PlatformProcess + NetworkProcess trees, file writes, network connectionsCritical
Email GatewayDelivery LogsBlocked attachments, URL clicks, sender reputationHigh
DNSQuery LogsNXDOMAIN spikes, DGA patterns, tunnelling indicatorsHigh
Web ProxyHTTP LogsUser-agent strings, destination categories, data volumeHigh
Cloud (AWS/Azure)Audit TrailsAPI calls, IAM changes, storage access patternsHigh
Reality

Most SIEM deployments collect far too many low-value logs (verbose application logs, SNMP traps) and miss critical ones (DNS query logs, EDR telemetry). Audit your log coverage before tuning rules.

// 04

Correlation Rules — How Detections Work

A correlation rule defines the pattern of events that constitutes a security alert. Good rules are specific enough to avoid false positives but broad enough to catch real attacks.

pseudo — Brute Force Login Rule
RULE: Brute Force Account Lockout
Trigger when:
  EventID = 4625 (Failed Login)
  AND count >= 5
  AND within 60 seconds
  AND same src_ip
Then:
  ALERT severity=HIGH
  ENRICH src_ip from TI feed
  CREATE ticket in ServiceNow
splunk — SPL Rule Implementation
index=windows EventCode=4625
| bucket _time span=60s
| stats count by _time, src_ip, ComputerName
| where count >= 5
| eval alert="Brute Force - " + ComputerName
| table _time, src_ip, ComputerName, count, alert
// 05

Tuning — The Real, Ongoing Work

A freshly deployed SIEM with default rules will generate thousands of false positives per day. Tuning is not a one-time task — it is the primary ongoing work of a functioning SOC.

  • Suppress alerts from known-good sources: vulnerability scanners, monitoring tools, backup jobs
  • Add asset context — alerts from a domain controller should be treated differently from a workstation
  • Threshold alerts based on baseline: '5 failed logins' may be normal from a helpdesk IP
  • Never disable a rule because it's noisy — suppress specific false positive sources instead. The underlying detection may still catch real attacks
  • Review all suppression rules quarterly — suppressed sources change, and stale suppressions are blind spots
  • Track false positive rate per rule — any rule above 90% FP rate should be rewritten or suppressed selectively
// 06

Platform Comparison

PlatformDeploymentBest ForPricing Model
Splunk EnterpriseOn-prem / CloudLarge enterprises, advanced analyticsPer GB ingested
Microsoft SentinelAzure cloudAzure-heavy environments, SOAR integrationPer GB ingested
Elastic SIEMOn-prem / CloudCost-conscious teams, ELK expertiseCompute-based
WazuhOn-prem / CloudSMBs, open-source, complianceFree / Support
IBM QRadarOn-premEnterprise, compliance-heavy regulated industriesPer EPS
Chronicle SIEMGoogle CloudPetabyte-scale, flat-rate pricingFlat annual
Pick

For most mid-market organisations: Wazuh for cost efficiency, Sentinel if already in Azure, Splunk if you need maximum flexibility and have the budget.

// 07

Deployment Best Practices

  • Collect critical log sources first — DCs, endpoints, firewall, EDR — before anything else
  • Set realistic retention: 90-day hot index minimum, 12-month cold storage for compliance
  • Build a log coverage map: every asset class mapped to whether logs are collected and what fields are available
  • Start with 10–15 high-quality rules rather than enabling all defaults
  • Do not deploy and walk away — budget 2–3 analyst hours per week for ongoing tuning from day one
  • Validate that rules actually fire by running test scenarios in a lab environment before production deployment