Skip to content

Commit 2a1a1e3

Browse files
committed
impl(sprint-12/wave-F): D-CSV-11 vertical streaming scaffolds (QualiaStream + InferenceStream + SplatFieldStream)
Cross-repo companion to lance-graph PR (sprint-12 Wave F fleet). Three new forward-iterator scaffolds per cognitive-substrate- convergence-v1.md §5 L-20: vertical streaming over the SoA columns to enable the i4 hot-path sweep introduced by lance-graph PR #383 (causal-edge v2) + #384 (QualiaI4_16D) + #387 (MUL i4 evaluation). W-F4 — QualiaStream - NEW `src/hpc/stream/qualia.rs` (~185 LOC + 6 tests) - `QualiaI4Row(pub u64)` — bit-compatible local mirror of `lance_graph_contract::qualia::QualiaI4_16D` (no cross-crate import to avoid producer↔consumer circular dep; documented as intentional in lance-graph TYPE_DUPLICATION_MAP) - `QualiaStream<'a>` forward-iterator over `&[QualiaI4Row]`, yielding `(usize, &QualiaI4Row)` tuples - Iterator + ExactSizeIterator impls; `new`/`len`/`is_empty`/ `remaining`/`reset` W-F5 — InferenceStream - NEW `src/hpc/stream/inference.rs` (~223 LOC + 6 tests) - `InferenceRow(pub u64)` — bit-compatible with `causal_edge::CausalEdge64` v2 layout (bits 46-49 signed mantissa, bits 53-58 W-slot 6b) - `inference_mantissa() -> i8` via `(raw << 4) >> 4` arithmetic- shift sign-extension (matches causal-edge's accessor) - `w_slot() -> u8` via bits-53..58 mask - `InferenceStream<'a>` Iterator + ExactSizeIterator W-F6 — SplatFieldStream - NEW `src/hpc/stream/splat_field.rs` (~240 LOC + 6 tests) - `SplatField { mean: u32, variance: f32, energy: f32, generation: u32 }` `#[repr(C, align(16))]` — bit-compatible with `thinking_engine::splat_ops::SplatField` from W-F7 - `SplatFieldStream<'a>` Iterator + `filter_energy_above(threshold)` combinator returning `impl Iterator` (no allocation) `src/hpc/mod.rs` — added `pub mod stream;` declaration. `src/hpc/stream/mod.rs` — registers all three submodules + re-exports: `InferenceRow / InferenceStream`, `QualiaI4Row / QualiaStream`, `SplatField / SplatFieldStream`. This commit resolves CSI-9 from the W-Meta-Opus honest review: the original W-F4 and W-F6 worker outputs were file-creating but mod.rs-orphan; only W-F5 had registered itself. Companion lance- graph commit pushes the parent meta-review + W-F2/W-F3/W-F1 registration fixes (CSI-7/8). Test status: `cargo test --lib hpc::stream` — **18/18 pass** (6 each × 3 streams). No regressions on the rest of the hpc suite. Sprint-13+: `par_*` rayon variants once rayon is wired into the ndarray feature gate (existing TECH_DEBT note).
1 parent 530ffaa commit 2a1a1e3

5 files changed

Lines changed: 689 additions & 0 deletions

File tree

src/hpc/mod.rs

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -236,6 +236,11 @@ pub mod framebuffer;
236236
/// Transcoded from Opus CELT for the HHTL cascade → waveform pipeline.
237237
pub mod audio;
238238

239+
/// Vertical streaming structs for the EdgeColumn SoA (D-CSV-11b, sprint-12).
240+
/// Per cognitive-substrate-convergence-v1.md §5 L-20.
241+
#[allow(missing_docs)]
242+
pub mod stream;
243+
239244
#[cfg(all(test, feature = "hpc-extras"))]
240245
mod e2e_tests {
241246
//! End-to-end pipeline test: Fingerprint → Node → Seal → Cascade → CLAM → Causality → BNN

src/hpc/stream/inference.rs

Lines changed: 223 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,223 @@
1+
//! InferenceStream — forward-iterator over a borrowed `&[InferenceRow]` slice.
2+
//! Per cognitive-substrate-convergence-v1.md §5 L-20: vertical streaming
3+
//! over the inference-mantissa lane of the EdgeColumn SoA. Used by the
4+
//! integer-SIMD MUL evaluation hot path (D-CSV-8 sprint-12 SIMD vec).
5+
//!
6+
//! Pure iterator scaffold; `par_inference_stream` rayon variant is sprint-13+.
7+
8+
// Local mirror of CausalEdge64 shape (bit-compatible with causal_edge::CausalEdge64).
9+
// No cross-crate import: ndarray is the producer; causal-edge is the consumer.
10+
11+
/// A single row of the EdgeColumn SoA, bit-compatible with
12+
/// `causal_edge::CausalEdge64` v2 layout.
13+
///
14+
/// Fields of interest for the inference-mantissa lane:
15+
/// - bits 46-49: signed 4-bit inference mantissa (−8..+7)
16+
/// - bits 53-58: W-slot corpus root handle (0..=63)
17+
#[repr(C, align(8))]
18+
#[derive(Clone, Copy, PartialEq, Eq, Hash, Debug, Default)]
19+
pub struct InferenceRow(pub u64);
20+
21+
impl InferenceRow {
22+
/// Read the 4-bit signed mantissa at bits 46-49 (matches causal-edge v2
23+
/// `inference_mantissa()` exactly — see `causal-edge/src/layout.rs`).
24+
///
25+
/// Sign-extension: extract 4-bit unsigned value, then sign-extend to i8
26+
/// via arithmetic left-shift trick: `(raw << 4) >> 4`.
27+
#[inline]
28+
pub fn inference_mantissa(&self) -> i8 {
29+
let raw = ((self.0 >> 46) & 0xF) as i8;
30+
(raw << 4) >> 4 // sign-extend 4 → 8 bits
31+
}
32+
33+
/// Read the W-slot at bits 53-58 (6 bits, 0..=63).
34+
///
35+
/// The W-slot is the witness corpus root handle per CausalEdge64 v2 L-6.
36+
/// Returns 0 for zero-initialized rows.
37+
#[inline]
38+
pub fn w_slot(&self) -> u8 {
39+
((self.0 >> 53) & 0x3F) as u8
40+
}
41+
}
42+
43+
/// Forward-iterator over a borrowed slice of [`InferenceRow`] values.
44+
///
45+
/// Provides vertical streaming access to the inference-mantissa lane of the
46+
/// EdgeColumn SoA. Yields `(index, &InferenceRow)` tuples so callers can
47+
/// correlate back to the originating row without maintaining external counters.
48+
///
49+
/// # Example
50+
/// ```rust
51+
/// use ndarray::hpc::stream::inference::{InferenceRow, InferenceStream};
52+
///
53+
/// let rows = vec![InferenceRow(0), InferenceRow(1 << 46)];
54+
/// let mut stream = InferenceStream::new(&rows);
55+
/// assert_eq!(stream.len(), 2);
56+
/// let (idx, row) = stream.next().unwrap();
57+
/// assert_eq!(idx, 0);
58+
/// ```
59+
pub struct InferenceStream<'a> {
60+
rows: &'a [InferenceRow],
61+
cursor: usize,
62+
}
63+
64+
impl<'a> InferenceStream<'a> {
65+
/// Construct a new stream over the given slice. The cursor starts at 0.
66+
pub fn new(rows: &'a [InferenceRow]) -> Self {
67+
Self { rows, cursor: 0 }
68+
}
69+
70+
/// Total number of rows in the underlying slice (not remaining).
71+
pub fn len(&self) -> usize {
72+
self.rows.len()
73+
}
74+
75+
/// Returns `true` if the underlying slice is empty.
76+
pub fn is_empty(&self) -> bool {
77+
self.rows.is_empty()
78+
}
79+
80+
/// Number of rows not yet yielded by the iterator.
81+
pub fn remaining(&self) -> usize {
82+
self.rows.len().saturating_sub(self.cursor)
83+
}
84+
85+
/// Reset the cursor to the beginning so the stream can be iterated again.
86+
pub fn reset(&mut self) {
87+
self.cursor = 0;
88+
}
89+
}
90+
91+
impl<'a> Iterator for InferenceStream<'a> {
92+
type Item = (usize, &'a InferenceRow);
93+
94+
fn next(&mut self) -> Option<Self::Item> {
95+
if self.cursor < self.rows.len() {
96+
let i = self.cursor;
97+
self.cursor += 1;
98+
Some((i, &self.rows[i]))
99+
} else {
100+
None
101+
}
102+
}
103+
104+
fn size_hint(&self) -> (usize, Option<usize>) {
105+
let rem = self.remaining();
106+
(rem, Some(rem))
107+
}
108+
}
109+
110+
impl<'a> ExactSizeIterator for InferenceStream<'a> {
111+
fn len(&self) -> usize {
112+
self.remaining()
113+
}
114+
}
115+
116+
#[cfg(test)]
117+
mod tests {
118+
use super::*;
119+
120+
#[test]
121+
fn test_inference_stream_empty() {
122+
let rows: &[InferenceRow] = &[];
123+
let mut stream = InferenceStream::new(rows);
124+
assert!(stream.is_empty());
125+
assert_eq!(stream.len(), 0);
126+
assert_eq!(stream.remaining(), 0);
127+
assert!(stream.next().is_none());
128+
}
129+
130+
#[test]
131+
fn test_inference_stream_yields_all() {
132+
let rows = vec![
133+
InferenceRow(0),
134+
InferenceRow(1),
135+
InferenceRow(2),
136+
];
137+
let stream = InferenceStream::new(&rows);
138+
let collected: Vec<_> = stream.collect();
139+
assert_eq!(collected.len(), 3);
140+
assert_eq!(collected[0].0, 0);
141+
assert_eq!(collected[1].0, 1);
142+
assert_eq!(collected[2].0, 2);
143+
assert_eq!(collected[0].1 as *const _, &rows[0] as *const _);
144+
assert_eq!(collected[2].1 as *const _, &rows[2] as *const _);
145+
}
146+
147+
#[test]
148+
fn test_mantissa_signed_extraction() {
149+
// Pack bits 46-49 = 0b1111 = 15 (raw), which is -1 in 4-bit two's complement.
150+
let raw_bits: u64 = 0b1111u64 << 46;
151+
let row = InferenceRow(raw_bits);
152+
assert_eq!(row.inference_mantissa(), -1);
153+
154+
// Pack bits 46-49 = 0b0111 = 7 (raw), positive maximum.
155+
let row_pos = InferenceRow(0b0111u64 << 46);
156+
assert_eq!(row_pos.inference_mantissa(), 7);
157+
158+
// Pack bits 46-49 = 0b1000 = 8 (raw), which is -8 in 4-bit two's complement.
159+
let row_min = InferenceRow(0b1000u64 << 46);
160+
assert_eq!(row_min.inference_mantissa(), -8);
161+
162+
// Zero mantissa.
163+
let row_zero = InferenceRow(0);
164+
assert_eq!(row_zero.inference_mantissa(), 0);
165+
}
166+
167+
#[test]
168+
fn test_w_slot_extraction() {
169+
// Pack bits 53-58 = 0b111111 = 63 (maximum W-slot value).
170+
let raw_bits: u64 = 0b111111u64 << 53;
171+
let row = InferenceRow(raw_bits);
172+
assert_eq!(row.w_slot(), 63);
173+
174+
// W-slot = 0 (zero row).
175+
let row_zero = InferenceRow(0);
176+
assert_eq!(row_zero.w_slot(), 0);
177+
178+
// W-slot = 1.
179+
let row_one = InferenceRow(1u64 << 53);
180+
assert_eq!(row_one.w_slot(), 1);
181+
182+
// W-slot = 32 (bit 58 set, bit 53 clear).
183+
let row_32 = InferenceRow(32u64 << 53);
184+
assert_eq!(row_32.w_slot(), 32);
185+
}
186+
187+
#[test]
188+
fn test_remaining_decrements() {
189+
let rows = vec![InferenceRow(0); 4];
190+
let mut stream = InferenceStream::new(&rows);
191+
assert_eq!(stream.remaining(), 4);
192+
stream.next();
193+
assert_eq!(stream.remaining(), 3);
194+
stream.next();
195+
assert_eq!(stream.remaining(), 2);
196+
stream.next();
197+
assert_eq!(stream.remaining(), 1);
198+
stream.next();
199+
assert_eq!(stream.remaining(), 0);
200+
// Exhausted: remaining stays 0.
201+
stream.next();
202+
assert_eq!(stream.remaining(), 0);
203+
}
204+
205+
#[test]
206+
fn test_reset_restarts() {
207+
let rows = vec![InferenceRow(10), InferenceRow(20)];
208+
let mut stream = InferenceStream::new(&rows);
209+
210+
// Exhaust the stream.
211+
assert!(stream.next().is_some());
212+
assert!(stream.next().is_some());
213+
assert!(stream.next().is_none());
214+
assert_eq!(stream.remaining(), 0);
215+
216+
// After reset, the stream yields from the beginning again.
217+
stream.reset();
218+
assert_eq!(stream.remaining(), 2);
219+
let first = stream.next().unwrap();
220+
assert_eq!(first.0, 0);
221+
assert_eq!(first.1.0, 10);
222+
}
223+
}

src/hpc/stream/mod.rs

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
//! Vertical streaming structs for the SoA columns.
2+
//! Per cognitive-substrate-convergence-v1.md §5 L-20.
3+
//!
4+
//! Sprint-12 scope (W-F4/5/6): `QualiaStream` + `InferenceStream` +
5+
//! `SplatFieldStream` forward-iterator scaffolds. Sprint-13+:
6+
//! `par_*` rayon variants once rayon is wired into the ndarray
7+
//! feature gate.
8+
9+
pub mod inference;
10+
pub mod qualia;
11+
pub mod splat_field;
12+
13+
pub use inference::{InferenceRow, InferenceStream};
14+
pub use qualia::{QualiaI4Row, QualiaStream};
15+
pub use splat_field::{SplatField, SplatFieldStream};

0 commit comments

Comments
 (0)