SOC-as-Code v2

A self-validating detection engineering pipeline. Every Sigma rule is proven against real, freshly-executed attack telemetry — not synthetic logs — before it is trusted in production.

This is the v2 evolution of an earlier project, SOC-as-Code, which validated Sigma rules in a CI/CD pipeline against synthetic, self-generated log data. This version closes that gap directly: rules here are tested against telemetry produced by actually executing the corresponding Atomic Red Team technique in an isolated lab, then validated against Splunk.

How a rule earns production status

Every rule is evaluated against three corpora, not one:

Corpus	What it proves
Target	Real logs from executing the exact ATT&CK technique (via Atomic Red Team) — does the rule actually fire on the attack it claims to catch?
Cross-domain	Real logs from other, unrelated techniques — does the rule stay silent outside its intended scope?
Benign baseline	Normal background activity — does the rule avoid drowning analysts in nuisance alerts?

A rule only earns production status when it performs acceptably on all three. This validation is automated via a GitHub Actions pipeline (.github/workflows/validate-rule.yml) running on a self-hosted runner with network access into the private lab. Pushing a rule to rules/staging/ is the only manual step — compilation, attack execution, Splunk querying, three-corpus scoring, and report posting happen automatically.

Repo layout

rules/
├── staging/         rules under test, not yet trusted
└── production/      validated rules, organized by ATT&CK tactic
    ├── execution/
    ├── persistence/
    └── credential_access/
scripts/
├── validate_rule.py       orchestration (compile → attack → score → report)
├── promote_rule.py        auto-promote passing rules to production/
├── deploy_saved_search.py deploy rules as live Splunk saved searches
└── file_review_issue.py   auto-file GitHub Issues for failing rules
reports/                   automated validation reports (JSON + Markdown)
.github/
├── workflows/
│   └── validate-rule.yml  CI/CD workflow — the full pipeline
└── CODEOWNERS             enforces review for rules/production/ changes
docs/                      validation log, design notes

Promotion gate

Changes to rules/production/ require review (see .github/CODEOWNERS), enforced via branch protection. Rules that fail automated validation are routed for review with an auto-filed issue documenting the failure, rather than silently dropped or silently merged.

Current rule set

Rule	Technique	Tactic	Status
T1059.001_encoded_powershell.yml	T1059.001	Execution	Production
T1053.005_schtasks_shell_persistence.yml	T1053.005	Persistence	Production
T1547.001_registry_run_key_persistence.yml	T1547.001	Persistence	Production
T1003.001_lsass_comsvcs_minidump.yml	T1003.001	Credential Access	Production
T1110.001_password_guessing_failed_logons.yml	T1110.001	Credential Access	Production
T1218.011_rundll32_proxy_execution.yml	T1218.011	Defense Evasion	Production
T1070.006_timestomp_powershell.yml	T1070.006	Defense Evasion	Production
T1018_remote_system_discovery_net.yml	T1018	Discovery	Production
T1082_system_information_discovery.yml	T1082	Discovery	Production
T1105_certutil_download.yml	T1105	Command & Control	Production

Lab environment

• Windows Server 2025 (VirtualBox, host-only network)
• Sysmon (SwiftOnSecurity baseline config)
• Splunk Enterprise (Universal Forwarder on the lab VM)
• Invoke-AtomicRedTeam / Atomic Red Team atomics library
• Rules authored in Sigma, compiled to SPL via pySigma

Build status

• Lab telemetry pipeline (VM → Sysmon → Forwarder → Splunk) validated end to end
• First Sigma rule manually validated against real attack telemetry
• Second Sigma rule manually validated against real attack telemetry
• Repository structure and promotion gate scaffolding
• Self-hosted GitHub Actions runner
• Automated orchestration (compile → trigger atomic test → score → report)
• Full one-click validation loop (CI workflow wired)
• Production promotion gate (auto-promote passing rules + deploy Splunk saved searches)
• Auto-file GitHub Issues for failing rules
• Expansion to 7 ATT&CK tactics: execution, persistence, credential access, defense evasion, discovery, command & control