CHAODA epicenter probe (negative — refutes WHERE=CHAODA) + IT cross-grid replication of family-basin Weyl

.claude/board/EPIPHANIES.md

@@ -1,3 +1,32 @@
+## 2026-06-19 — E-CHAODA-IS-NOT-THE-SEAM-EPICENTER — REFUTES this session's own "WHERE = CHAODA anomaly" conjecture: on the real ES grid with a Fiedler-sign fingerprint, the brittle 2-line seam is NOT a CHAODA/LFD anomaly (it is LESS anomalous than average); the geometric outlier and the spectral-cut bottleneck do not coincide
+
+**Status:** FINDING (measured NEGATIVE; probe `perturbation-sim/examples/chaoda_surge_epicenter.rs`, gated `--features ndarray-simd`, real `ndarray::hpc::clam::{ClamTree, anomaly_scores}`). **Refutes** the conjecture I posted earlier this session ("a surge's epicenter CAN be modeled as a CHAODA anomaly — the WHERE axis"). The user asked to model CLAM/CHAODA with the real ndarray engine; I did, and it killed the conjecture — measured, not asserted.
+
+**What the probe measured (ES largest component, 261 buses):** each node → a 64-bit Fiedler-sign fingerprint (the spectral embedding) → real ndarray `ClamTree::build` (Hamming) → `anomaly_scores` (leaf-cluster LFD, normalized). The HHTL seam = the inter-HEEL-basin cut (4 nodes — exactly the 2-line bottleneck).
+- mean anomaly, **SEAM** nodes: **0.302** — mean anomaly, ALL nodes: **0.339** → ratio **0.89** (seam is *less* anomalous);
+- seam share of the top-quartile anomalies: 1/4, **lift 1.00** (pure chance).
+**VERDICT: NOT SUPPORTED** (gate was ratio ≥ 1.30 AND lift ≥ 1.30).
+
+**Why (clear in hindsight):** Fiedler-sign fingerprints cluster nodes *by basin*, so boundary/cut nodes sit in a **dense, low-LFD** region → low anomaly. CHAODA flags **high-LFD** geometric complexity, which lives elsewhere (not at the spectral cut). The brittle cut is not a geometric outlier on this embedding. So **CHAODA ≠ epicenter detector** as I'd claimed; the three-axis "surge" decomposition's WHERE leg is **not** simply a CHAODA anomaly — that synthesis claim was over-reach, now corrected.
+
+**Honest scope / non-tuning:** this is ONE encoding (Fiedler-sign) on ONE grid; I did NOT keep swapping encodings until it went green (that's the confirmation-bias trap in reverse). The defensible statement: *on the natural spectral-embedding encoding, CHAODA's LFD anomaly does not identify the seam.* A different fingerprint (e.g. raw bus features, or distance-to-cut) might behave differently — but the clean "epicenter = CHAODA anomaly" claim is **withdrawn** pending such a probe. CHAODA remains a real manifold-anomaly detector; it just doesn't detect *this* (the spectral bottleneck). Cross-refs: `E-CLAM-IS-THE-MANIFOLD-ENGINE` (CHAODA = LFD repulsion — still true, just not seam-aligned), `E-FAMILY-BASIN-WEYL-HOP-LOCAL-AT-CRISP-TIER` (the HOW-it-spreads axis, which DID hold), `ndarray::hpc::clam`.
+
+---
+
+## 2026-06-19 — E-FAMILY-BASIN-WEYL-IT-REPLICATION — the crisp-tier hop-locality reinstatement replicates on the IT grid (independent C-solid grid), strengthening `E-FAMILY-BASIN-WEYL-HOP-LOCAL-AT-CRISP-TIER` from one-grid to cross-grid
+
+**Status:** FINDING (confirmation; same probe `family_basin_weyl_multihop`, country=IT). Run on the IT largest component (192 buses / 259 lines) — the other C-solid grid (probe C `(λ₃−λ₂)/λ₂ = 2.38`):
+
+| tier | basins | within | seam | ratio | verdict |
+|---|---|---|---|---|---|
+| HEEL | 2 | 0.730 | 0.561 | 1.30 | **HOP-LOCAL** |
+| HIP | 4 | 0.791 | 0.587 | 1.35 | **HOP-LOCAL** |
+| LEAF | 8 | 0.592 | 0.512 | 1.16 | leaks |
+
+IT **replicates and slightly extends** the ES result: the crisp band is hop-local at HEEL **and HIP** (ES was HEEL only), LEAF leaks — same shape (crisp tiers localize, finest tier leaks), on an independent grid. So the family-basin Weyl-multi-hop hop-locality is **not an ES artifact** — it holds where the bisection is stable, across grids. The tier at which leakage sets in is grid-specific (the marginal-DK / out-of-family-tie-growth boundary), as expected. Cross-ref: `E-FAMILY-BASIN-WEYL-HOP-LOCAL-AT-CRISP-TIER` (the ES finding this confirms).
+
+---
+
 ## 2026-06-19 — E-FAMILY-BASIN-WEYL-HOP-LOCAL-AT-CRISP-TIER — REINSTATEMENT (tier-scoped) of `E-OUTAGE-CASCADE-IS-NON-LOCAL`: WITH the family-basin partition, a within-basin perturbation is block-localized (95.7 % containment, within/seam ratio 1.54) at the **crisp top split**, so multi-hop Weyl IS hop-local there; finer tiers leak. The earlier non-locality was the *un-partitioned* global solve. (Davis-Kahan is the *mechanism* — see body — NOT the numeric evidence; the gate is the containment ratio.)
 
 **Status:** FINDING (measured, real ES PyPSA data; probe `perturbation-sim/examples/family_basin_weyl_multihop.rs`). **Tier-scoped reinstatement** of the reach claim that `E-OUTAGE-CASCADE-IS-NON-LOCAL` had corrected. Operator hypothesis: "with family basin it works now with weyl multihop." Confirmed — at the crisp tier only.

crates/perturbation-sim/Cargo.toml

@@ -124,3 +124,7 @@ path = "examples/outage_over_hhtl_hops.rs"
 [[example]]
 name = "family_basin_weyl_multihop"
 path = "examples/family_basin_weyl_multihop.rs"
+
+[[example]]
+name = "chaoda_surge_epicenter"
+path = "examples/chaoda_surge_epicenter.rs"

crates/perturbation-sim/examples/chaoda_surge_epicenter.rs

@@ -0,0 +1,159 @@
+//! PROBE — CHAODA epicenter (`ndarray::hpc::clam`) on the real ES grid.
+//! **Gated behind `ndarray-simd`** (uses ndarray's production `ClamTree` +
+//! `anomaly_scores`, the real CLAM/CHAODA engine — not perturbation-sim's lite).
+//!
+//! The WHERE axis of the three-axis surge decomposition
+//! (`E-FAMILY-BASIN-WEYL-HOP-LOCAL-AT-CRISP-TIER`): a surge's fail-first
+//! compartment is the brittle seam — geometrically anomalous on the node
+//! manifold. CHAODA's LFD-based anomaly should flag it **statically, with no
+//! cascade simulation**. Hypothesis: the top-anomaly nodes concentrate on the
+//! HHTL seam — the inter-HEEL-basin cut, the crisp 2-split bottleneck (the
+//! "without family nodes" table's HIP 2-line seam, lines 46/150).
+//!
+//! Encoding: each node → a binary fingerprint = the **sign of its top-K Fiedler
+//! eigenvector coordinates**, packed to bytes (the spectral embedding, the same
+//! Laplacian spectrum the HHTL tiers come from). `ClamTree::build` (Hamming) +
+//! `anomaly_scores` → per-node LFD anomaly. We then compare the seam-cut
+//! endpoints' anomaly against the global mean and their share of the top anomalies.
+//!
+//! Honest gate: the epicenter↔anomaly claim is SUPPORTED only if the seam nodes'
+//! mean anomaly ≥ 1.30× the global mean AND seam nodes are over-represented in
+//! the top-quartile anomalies (lift ≥ 1.30). Reported as-is otherwise.
+//!
+//! Run: cargo run --manifest-path crates/perturbation-sim/Cargo.toml \
+//!        --features ndarray-simd --example chaoda_surge_epicenter \
+//!        -- /tmp/pypsa/buses.csv /tmp/pypsa/lines.csv ES
+
+#[cfg(not(feature = "ndarray-simd"))]
+fn main() {
+    eprintln!(
+        "chaoda_surge_epicenter requires `--features ndarray-simd` \
+         (it uses ndarray::hpc::clam — the real CLAM/CHAODA engine)."
+    );
+}
+
+#[cfg(feature = "ndarray-simd")]
+fn main() {
+    use ndarray::hpc::clam::ClamTree;
+    use perturbation_sim::{from_pypsa_csv, hhtl_keys, symmetric_eigen, Grid};
+    use std::collections::HashSet;
+    use std::fs;
+
+    const K: usize = 64; // spectral-embedding bits per node (Fiedler coords 1..=K)
+
+    let args: Vec<String> = std::env::args().collect();
+    let (bpath, lpath, country) = (
+        args.get(1)
+            .map(String::as_str)
+            .unwrap_or("/tmp/pypsa/buses.csv"),
+        args.get(2)
+            .map(String::as_str)
+            .unwrap_or("/tmp/pypsa/lines.csv"),
+        args.get(3).map(String::as_str).unwrap_or("ES"),
+    );
+    let buses = fs::read_to_string(bpath).expect("read buses.csv");
+    let lines = fs::read_to_string(lpath).expect("read lines.csv");
+    let import = from_pypsa_csv(&buses, &lines, Some(country))
+        .expect("parse pypsa")
+        .largest_component();
+    let grid: &Grid = &import.grid;
+    let (n, m) = (grid.n, grid.edges.len());
+
+    // Spectral embedding: top-K Fiedler eigenvectors (skip j=0, the constant).
+    let eig = symmetric_eigen(&grid.laplacian_of(&vec![true; m]), n);
+    let k = K.min(n.saturating_sub(1));
+    let vecs: Vec<Vec<f64>> = (1..=k).map(|j| eig.eigenvector(j)).collect();
+
+    // Node fingerprint = sign bits of its K Fiedler coords, packed to bytes.
+    let vec_len = k.div_ceil(8);
+    let mut data = vec![0u8; n * vec_len];
+    for (node, fp) in data.chunks_mut(vec_len).enumerate() {
+        for (j, v) in vecs.iter().enumerate() {
+            if v[node] >= 0.0 {
+                fp[j / 8] |= 1 << (j % 8);
+            }
+        }
+    }
+
+    // The real ndarray CLAM/CHAODA engine.
+    let tree = ClamTree::build(&data, vec_len, 4);
+    let scores = tree.anomaly_scores(&data, vec_len);
+
+    // HHTL seam = the inter-HEEL-basin cut (the crisp 2-split bottleneck).
+    let keys = hhtl_keys(grid);
+    let mut seam_nodes: HashSet<usize> = HashSet::new();
+    for e in &grid.edges {
+        if keys[e.from].heel != keys[e.to].heel {
+            seam_nodes.insert(e.from);
+            seam_nodes.insert(e.to);
+        }
+    }
+
+    let mean = |idxs: &[usize]| -> f64 {
+        if idxs.is_empty() {
+            0.0
+        } else {
+            idxs.iter().map(|&i| scores[i].score).sum::<f64>() / idxs.len() as f64
+        }
+    };
+    let all: Vec<usize> = (0..n).collect();
+    let seam: Vec<usize> = seam_nodes.iter().copied().collect();
+    let (anom_seam, anom_all) = (mean(&seam), mean(&all));
+    let anom_ratio = if anom_all > 0.0 { anom_seam / anom_all } else { 0.0 };
+
+    // Top-quartile anomalies: are seam nodes over-represented?
+    let mut ranked: Vec<usize> = (0..n).collect();
+    ranked.sort_by(|&a, &b| {
+        scores[b]
+            .score
+            .partial_cmp(&scores[a].score)
+            .unwrap_or(std::cmp::Ordering::Equal)
+    });
+    let top_n = (n / 4).max(1);
+    let top: HashSet<usize> = ranked[..top_n].iter().copied().collect();
+    let seam_in_top = seam.iter().filter(|i| top.contains(i)).count();
+    let expected = top_n as f64 * seam.len() as f64 / n as f64; // null expectation
+    let lift = if expected > 0.0 {
+        seam_in_top as f64 / expected
+    } else {
+        0.0
+    };
+
+    println!("PROBE — CHAODA surge epicenter (ndarray::hpc::clam, real {country} grid)");
+    println!("  grid: {n} buses, {m} lines; {k}-bit Fiedler fingerprints ({vec_len} B each)");
+    println!(
+        "  seam (inter-HEEL-basin cut): {} nodes; CLAM tree built, anomaly_scores computed",
+        seam.len()
+    );
+    println!("\n  anomaly (LFD-derived, higher = more anomalous):");
+    println!("    mean anomaly, SEAM nodes : {anom_seam:.3}");
+    println!("    mean anomaly, ALL nodes  : {anom_all:.3}");
+    println!("    ratio (seam / all)       : {anom_ratio:.2}");
+    println!(
+        "    seam share of top-{top_n} anomalies: {seam_in_top}/{} (expected {expected:.1}, lift {lift:.2})",
+        seam.len()
+    );
+
+    let supported = !seam.is_empty() && anom_ratio >= 1.30 && lift >= 1.30;
+    println!("\n  VERDICT:");
+    if supported {
+        println!(
+            "    [SUPPORTED] the seam (the brittle fail-first cut) IS a CHAODA anomaly — its mean \
+             LFD anomaly is {anom_ratio:.2}× the global mean and it is {lift:.2}× over-represented \
+             in the top-quartile anomalies. The surge EPICENTER is detectable statically from the \
+             manifold geometry, no cascade simulation — the WHERE axis closes on real data."
+        );
+    } else {
+        println!(
+            "    [NOT SUPPORTED] seam anomaly ratio {anom_ratio:.2} / top-quartile lift {lift:.2} \
+             do not clear 1.30 — on this grid the LFD anomaly does not single out the seam. Report \
+             honestly: CHAODA flags geometric outliers, which need not coincide with the \
+             spectral-cut bottleneck here. Do not promote."
+        );
+    }
+    println!(
+        "    NOTE: CHAODA anomaly = leaf-cluster LFD normalized over the tree; the seam is the \
+         inter-HEEL-basin cut (the crisp 2-split). This is the WHERE axis (epicenter), distinct \
+         from the Weyl HOW-MUCH and the family-basin HOW-it-spreads axes."
+    );
+}