Skip to content

Commit 769b3d0

Browse files
authored
feat: add config-change-investigation-demo (#24)
1 parent f1f0973 commit 769b3d0

15 files changed

Lines changed: 1303 additions & 249 deletions

File tree

README.md

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,16 +11,18 @@ Latest milestone: [v0.5.0 — third demo and three-demo structure](https://githu
1111
- [telemetry-window-demo](#telemetry-window-demo)
1212
- [ai-assisted-detection-demo](demos/ai-assisted-detection-demo/README.md)
1313
- [rule-evaluation-and-dedup-demo](demos/rule-evaluation-and-dedup-demo/README.md)
14+
- [config-change-investigation-demo](demos/config-change-investigation-demo/README.md)
1415

1516
| Demo | Input | Deterministic core | LLM role | Main artifacts | Guardrails / non-goals |
1617
| --- | --- | --- | --- | --- | --- |
1718
| [telemetry-window-demo](#telemetry-window-demo) | JSONL / CSV events | Windows<br>Features<br>Alert thresholds | None | `features.csv`<br>`alerts.csv`<br>`summary.json`<br>3 PNG plots | MVP only<br>No realtime<br>No case management |
1819
| [ai-assisted-detection-demo](demos/ai-assisted-detection-demo/README.md) | JSONL auth / web / process | Normalize<br>Rules<br>Grouping<br>ATT&CK mapping | JSON-only case drafting | `rule_hits.json`<br>`case_bundles.json`<br>`case_summaries.json`<br>`case_report.md`<br>`audit_traces.jsonl` | Human verification required<br>No autonomous response<br>No final verdict |
1920
| [rule-evaluation-and-dedup-demo](demos/rule-evaluation-and-dedup-demo/README.md) | JSON raw rule hits | Scope resolution<br>Cooldown grouping<br>Suppression reasoning | None | `rule_hits_before_dedup.json`<br>`rule_hits_after_dedup.json`<br>`dedup_explanations.json`<br>`dedup_report.md` | No realtime<br>No dashboard<br>No AI stage |
21+
| [config-change-investigation-demo](demos/config-change-investigation-demo/README.md) | JSONL config changes<br>Policy denials<br>Follow-on events | Normalize<br>Risky-change rules<br>Bounded correlation | None | `change_events_normalized.json`<br>`investigation_hits.json`<br>`investigation_summary.json`<br>`investigation_report.md` | No realtime<br>No dashboard<br>No AI stage |
2022

2123
## What This Repo Is
2224

23-
`telemetry-lab` is a small portfolio repository for telemetry analytics and constrained detection-oriented workflows. It is organized as three local, file-based demos that are reproducible from committed sample data and intentionally scoped for public review rather than production use.
25+
`telemetry-lab` is a small portfolio repository for telemetry analytics and constrained detection-oriented workflows. It is organized as four local, file-based demos that are reproducible from committed sample data and intentionally scoped for public review rather than production use.
2426

2527
### telemetry-window-demo
2628

@@ -32,7 +34,11 @@ Latest milestone: [v0.5.0 — third demo and three-demo structure](https://githu
3234

3335
### rule-evaluation-and-dedup-demo
3436

35-
`rule-evaluation-and-dedup-demo` starts from raw rule hits and makes cooldown behavior legible. It shows which hits were kept, which were suppressed, how scope was resolved, and why repeated hits collapsed into fewer retained alerts.
37+
`rule-evaluation-and-dedup-demo` starts from raw rule hits and makes cooldown behavior legible. It shows which hits were kept, which were suppressed, how scope was resolved, and why repeated hits collapsed into fewer retained alerts.
38+
39+
### config-change-investigation-demo
40+
41+
`config-change-investigation-demo` follows risky configuration changes into bounded follow-on evidence such as policy denials and service signals. It stays deterministic, file-based, and review-oriented, with no added AI stage.
3642

3743
## Quick Run
3844

@@ -44,7 +50,8 @@ python -m telemetry_window_demo.cli run --config configs/default.yaml
4450
Other demo entrypoints:
4551

4652
- `python -m telemetry_window_demo.cli run-ai-demo`
47-
- `python -m telemetry_window_demo.cli run-rule-dedup-demo`
53+
- `python -m telemetry_window_demo.cli run-rule-dedup-demo`
54+
- `python -m telemetry_window_demo.cli run-config-change-demo`
4855

4956
That command reads `data/raw/sample_events.jsonl` and regenerates:
5057

@@ -109,6 +116,7 @@ Cooldown behavior:
109116
## Repo Guide
110117

111118
- [`demos/rule-evaluation-and-dedup-demo/README.md`](demos/rule-evaluation-and-dedup-demo/README.md) explains the third demo and links its committed before/after dedup artifacts
119+
- [`demos/config-change-investigation-demo/README.md`](demos/config-change-investigation-demo/README.md) explains the config-change investigation demo and its committed artifacts
112120
- [`docs/sample-output.md`](docs/sample-output.md) summarizes the committed sample artifacts
113121
- [`docs/roadmap.md`](docs/roadmap.md) sketches the next demo directions
114122
- [`data/processed/summary.json`](data/processed/summary.json) captures the default run in machine-readable form
Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# Config-Change Investigation Demo
2+
3+
This demo is part of `telemetry-lab` and stays intentionally small, local, and reviewer-friendly.
4+
5+
It focuses on deterministic investigation logic for risky configuration changes and nearby evidence. There is no new AI stage in this demo.
6+
7+
## Purpose
8+
9+
The goal is to make one compact config-change investigation path legible from committed sample data.
10+
11+
The demo starts from configuration changes, policy denials, and follow-on telemetry, then:
12+
13+
- normalizes the inputs into shared internal records
14+
- applies deterministic risky-change rules
15+
- attaches nearby supporting evidence using bounded time and shared-system correlation
16+
- writes machine-readable summaries and a short reviewer-facing report
17+
18+
## Quick Start
19+
20+
From the repository root:
21+
22+
```bash
23+
python -m pip install -e .
24+
python -m telemetry_window_demo.cli run-config-change-demo
25+
```
26+
27+
Generated artifacts are written to `demos/config-change-investigation-demo/artifacts/`.
28+
29+
## Demo Inputs
30+
31+
- config changes: `data/raw/config_changes.jsonl`
32+
- policy denials: `data/raw/policy_denials.jsonl`
33+
- follow-on events: `data/raw/follow_on_events.jsonl`
34+
- investigation config: `config/investigation.yaml`
35+
36+
The bundled sample includes:
37+
38+
- one risky MFA-related change with nearby denials and follow-on signals
39+
- one risky public-bind change with nearby denials and service events
40+
- one benign config change that should not trigger an investigation
41+
- one risky break-glass change with no nearby supporting evidence inside the bounded window
42+
43+
## Deterministic Correlation
44+
45+
This demo uses a bounded correlation window after each triggering config change.
46+
47+
Evidence is attached only when:
48+
49+
1. `target_system` matches the triggering change
50+
2. the evidence timestamp falls within the configured correlation window after the change
51+
52+
## Expected Artifacts
53+
54+
- `artifacts/change_events_normalized.json`
55+
- `artifacts/investigation_hits.json`
56+
- `artifacts/investigation_summary.json`
57+
- `artifacts/investigation_report.md`
58+
59+
## Artifact Semantics
60+
61+
- `change_events_normalized.json`: normalized config changes before any rule match is applied
62+
- `investigation_hits.json`: full investigation records, including the triggering change and attached evidence
63+
- `investigation_summary.json`: reduced machine-readable summaries for each investigation
64+
- `investigation_report.md`: a short reviewer report showing the trigger, evidence counts, and bounded-correlation explanation
65+
66+
## Reviewer Walkthrough
67+
68+
1. Open `change_events_normalized.json` and identify the risky config keys and values.
69+
2. Open `investigation_hits.json` and verify which changes became investigations and which evidence records were attached.
70+
3. Open `investigation_summary.json` and confirm the final summaries stay deterministic and bounded.
71+
4. Open `investigation_report.md` and verify that a risky change with no nearby evidence remains explicit rather than silently discarded.
72+
73+
## Limitations
74+
75+
- synthetic sample data only
76+
- no realtime ingestion or service deployment
77+
- bounded correlation by system and time only
78+
- no model-generated reasoning or autonomous response
Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
[
2+
{
3+
"change_id": "cfg-001",
4+
"timestamp": "2026-03-22T09:00:00Z",
5+
"actor": "ops-admin",
6+
"target_system": "identity-proxy",
7+
"config_key": "disable_admin_mfa",
8+
"old_value": "false",
9+
"new_value": "true",
10+
"change_result": "success",
11+
"change_ticket": "CHG-1001"
12+
},
13+
{
14+
"change_id": "cfg-002",
15+
"timestamp": "2026-03-22T09:20:00Z",
16+
"actor": "deploy-bot",
17+
"target_system": "payments-api",
18+
"config_key": "public_bind_cidr",
19+
"old_value": "10.20.0.0/24",
20+
"new_value": "0.0.0.0/0",
21+
"change_result": "success",
22+
"change_ticket": "CHG-1002"
23+
},
24+
{
25+
"change_id": "cfg-003",
26+
"timestamp": "2026-03-22T09:40:00Z",
27+
"actor": "deploy-bot",
28+
"target_system": "payments-api",
29+
"config_key": "log_level",
30+
"old_value": "info",
31+
"new_value": "debug",
32+
"change_result": "success",
33+
"change_ticket": "CHG-1003"
34+
},
35+
{
36+
"change_id": "cfg-004",
37+
"timestamp": "2026-03-22T10:00:00Z",
38+
"actor": "sre-admin",
39+
"target_system": "vault-gateway",
40+
"config_key": "break_glass_mode",
41+
"old_value": "disabled",
42+
"new_value": "enabled",
43+
"change_result": "success",
44+
"change_ticket": "CHG-1004"
45+
}
46+
]
Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
[
2+
{
3+
"investigation_id": "CCI-001",
4+
"severity": "critical",
5+
"rule_id": "cfg_disable_admin_mfa",
6+
"target_system": "identity-proxy",
7+
"actor": "ops-admin",
8+
"triggering_change": {
9+
"change_id": "cfg-001",
10+
"timestamp": "2026-03-22T09:00:00Z",
11+
"actor": "ops-admin",
12+
"target_system": "identity-proxy",
13+
"config_key": "disable_admin_mfa",
14+
"old_value": "false",
15+
"new_value": "true",
16+
"change_result": "success",
17+
"change_ticket": "CHG-1001"
18+
},
19+
"trigger_reason": "Admin MFA was disabled on a protected system.",
20+
"correlation_window_minutes": 15,
21+
"bounded_correlation_reason": "Attached evidence shares target_system 'identity-proxy' and falls within 15 minutes after the triggering change.",
22+
"attached_policy_denials": [
23+
{
24+
"denial_id": "den-001",
25+
"timestamp": "2026-03-22T09:04:00Z",
26+
"actor": "ops-admin",
27+
"target_system": "identity-proxy",
28+
"policy_name": "admin-login-guard",
29+
"decision": "denied",
30+
"reason": "MFA policy blocked admin login after configuration drift."
31+
},
32+
{
33+
"denial_id": "den-002",
34+
"timestamp": "2026-03-22T09:09:00Z",
35+
"actor": "service-account",
36+
"target_system": "identity-proxy",
37+
"policy_name": "token-exchange-guard",
38+
"decision": "denied",
39+
"reason": "Token exchange blocked after admin-auth policy divergence."
40+
}
41+
],
42+
"attached_follow_on_events": [
43+
{
44+
"event_id": "fo-001",
45+
"timestamp": "2026-03-22T09:05:00Z",
46+
"target_system": "identity-proxy",
47+
"event_type": "auth_fail_burst",
48+
"details": "5 privileged login failures from 203.0.113.24 after the config change."
49+
},
50+
{
51+
"event_id": "fo-002",
52+
"timestamp": "2026-03-22T09:11:00Z",
53+
"target_system": "identity-proxy",
54+
"event_type": "service_restart",
55+
"details": "identity-proxy restarted after an auth-policy reload."
56+
}
57+
],
58+
"evidence_counts": {
59+
"policy_denials": 2,
60+
"follow_on_events": 2
61+
}
62+
},
63+
{
64+
"investigation_id": "CCI-002",
65+
"severity": "high",
66+
"rule_id": "cfg_public_bind_cidr",
67+
"target_system": "payments-api",
68+
"actor": "deploy-bot",
69+
"triggering_change": {
70+
"change_id": "cfg-002",
71+
"timestamp": "2026-03-22T09:20:00Z",
72+
"actor": "deploy-bot",
73+
"target_system": "payments-api",
74+
"config_key": "public_bind_cidr",
75+
"old_value": "10.20.0.0/24",
76+
"new_value": "0.0.0.0/0",
77+
"change_result": "success",
78+
"change_ticket": "CHG-1002"
79+
},
80+
"trigger_reason": "Public bind CIDR was expanded to all addresses.",
81+
"correlation_window_minutes": 15,
82+
"bounded_correlation_reason": "Attached evidence shares target_system 'payments-api' and falls within 15 minutes after the triggering change.",
83+
"attached_policy_denials": [
84+
{
85+
"denial_id": "den-003",
86+
"timestamp": "2026-03-22T09:23:00Z",
87+
"actor": "deploy-bot",
88+
"target_system": "payments-api",
89+
"policy_name": "public-exposure-guard",
90+
"decision": "denied",
91+
"reason": "Public bind CIDR exceeded the approved network range."
92+
}
93+
],
94+
"attached_follow_on_events": [
95+
{
96+
"event_id": "fo-003",
97+
"timestamp": "2026-03-22T09:26:00Z",
98+
"target_system": "payments-api",
99+
"event_type": "service_restart",
100+
"details": "payments-api restarted after listener rebind."
101+
},
102+
{
103+
"event_id": "fo-004",
104+
"timestamp": "2026-03-22T09:31:00Z",
105+
"target_system": "payments-api",
106+
"event_type": "edge_warning",
107+
"details": "Edge listener observed requests from the newly public CIDR."
108+
}
109+
],
110+
"evidence_counts": {
111+
"policy_denials": 1,
112+
"follow_on_events": 2
113+
}
114+
},
115+
{
116+
"investigation_id": "CCI-003",
117+
"severity": "high",
118+
"rule_id": "cfg_break_glass_mode",
119+
"target_system": "vault-gateway",
120+
"actor": "sre-admin",
121+
"triggering_change": {
122+
"change_id": "cfg-004",
123+
"timestamp": "2026-03-22T10:00:00Z",
124+
"actor": "sre-admin",
125+
"target_system": "vault-gateway",
126+
"config_key": "break_glass_mode",
127+
"old_value": "disabled",
128+
"new_value": "enabled",
129+
"change_result": "success",
130+
"change_ticket": "CHG-1004"
131+
},
132+
"trigger_reason": "Break-glass mode was enabled on a sensitive service.",
133+
"correlation_window_minutes": 15,
134+
"bounded_correlation_reason": "Attached evidence shares target_system 'vault-gateway' and falls within 15 minutes after the triggering change.",
135+
"attached_policy_denials": [],
136+
"attached_follow_on_events": [],
137+
"evidence_counts": {
138+
"policy_denials": 0,
139+
"follow_on_events": 0
140+
}
141+
}
142+
]
Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# Config-Change Investigation Demo Report
2+
3+
This deterministic demo correlates risky configuration changes with bounded follow-on evidence.
4+
It does not use an LLM and does not produce autonomous response actions.
5+
6+
## Run Summary
7+
8+
- normalized_change_events: 4
9+
- risky_change_hits: 3
10+
- investigations: 3
11+
- correlation_window_minutes: 15
12+
13+
## CCI-001
14+
15+
- Severity: critical
16+
- Target system: identity-proxy
17+
- Triggering change: cfg-001 (disable_admin_mfa -> true)
18+
- Trigger reason: Admin MFA was disabled on a protected system.
19+
- Attached policy denials: 2
20+
- Attached follow-on events: 2
21+
- Bounded correlation: Attached evidence shares target_system 'identity-proxy' and falls within 15 minutes after the triggering change.
22+
23+
Policy denials:
24+
- den-001: admin-login-guard -> MFA policy blocked admin login after configuration drift.
25+
- den-002: token-exchange-guard -> Token exchange blocked after admin-auth policy divergence.
26+
27+
Follow-on events:
28+
- fo-001: auth_fail_burst -> 5 privileged login failures from 203.0.113.24 after the config change.
29+
- fo-002: service_restart -> identity-proxy restarted after an auth-policy reload.
30+
31+
## CCI-002
32+
33+
- Severity: high
34+
- Target system: payments-api
35+
- Triggering change: cfg-002 (public_bind_cidr -> 0.0.0.0/0)
36+
- Trigger reason: Public bind CIDR was expanded to all addresses.
37+
- Attached policy denials: 1
38+
- Attached follow-on events: 2
39+
- Bounded correlation: Attached evidence shares target_system 'payments-api' and falls within 15 minutes after the triggering change.
40+
41+
Policy denials:
42+
- den-003: public-exposure-guard -> Public bind CIDR exceeded the approved network range.
43+
44+
Follow-on events:
45+
- fo-003: service_restart -> payments-api restarted after listener rebind.
46+
- fo-004: edge_warning -> Edge listener observed requests from the newly public CIDR.
47+
48+
## CCI-003
49+
50+
- Severity: high
51+
- Target system: vault-gateway
52+
- Triggering change: cfg-004 (break_glass_mode -> enabled)
53+
- Trigger reason: Break-glass mode was enabled on a sensitive service.
54+
- Attached policy denials: 0
55+
- Attached follow-on events: 0
56+
- Bounded correlation: Attached evidence shares target_system 'vault-gateway' and falls within 15 minutes after the triggering change.
57+
58+
No nearby supporting evidence fell inside the bounded correlation window.

0 commit comments

Comments
 (0)