Skip to content

Commit 14bf47a

Browse files
subtleGradientsubtleGradient
committed
triage: add 8 hypothesis invalidation tasks
Ways to experimentally invalidate 'Zig parity is complete': - TASK-190: Extended fuzz testing with stress patterns - TASK-191: Port Python hypothesis property tests - TASK-192: Test against prior Rust/C database files - TASK-193: Audit Rust integration tests for gaps - TASK-194: Real-world app simulation (todo, chat, inventory) - TASK-195: Adversarial/malformed input fuzzing - TASK-196: Deep clock table internals inspection - TASK-197: Performance regression analysis Each task targets a different attack surface for finding divergences.
1 parent b25058a commit 14bf47a

8 files changed

Lines changed: 457 additions & 0 deletions

‎.tasks/triage/TASK-190-fuzz-invalidation-round2.md‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# TASK-190 — Fuzz Invalidation Round 2: Stress the sync protocol
2+
3+
## Goal
4+
Invalidate "Zig parity is complete" hypothesis via extended fuzzing with focus on sync edge cases.
5+
6+
## Status
7+
- State: triage
8+
- Priority: HIGH (hypothesis validation)
9+
- Discovered: 2025-12-23 (Round 69 follow-up)
10+
11+
## Hypothesis to Invalidate
12+
"Zig CR-SQLite is functionally identical to Rust/C CR-SQLite for all sync scenarios."
13+
14+
## Test Approach
15+
Extend `test-fuzz-parity.sh` with:
16+
17+
1. **Higher iteration count** (1000+ instead of 100)
18+
2. **More aggressive schema generation**:
19+
- Tables with 10+ columns
20+
- Deep compound PKs (3-4 columns)
21+
- Mixed type PKs (int + text + blob)
22+
3. **Chaotic operation sequences**:
23+
- Rapid insert/delete/resurrect cycles
24+
- Concurrent column updates on same row
25+
- Interleaved multi-table operations
26+
4. **Sync stress patterns**:
27+
- 3+ node sync topologies
28+
- Out-of-order change application
29+
- Partial sync followed by full sync
30+
5. **Value edge cases**:
31+
- Very long strings (>64KB)
32+
- Binary data with all byte values
33+
- Unicode normalization forms
34+
35+
## Files to Modify
36+
- `zig/harness/test-fuzz-parity.sh` (extend)
37+
- Or create new `zig/harness/test-fuzz-stress.sh`
38+
39+
## Acceptance Criteria
40+
1. Either find a divergence (invalidate hypothesis) OR
41+
2. Complete 10,000 operations without divergence (increase confidence)
42+
43+
## Parent Docs / Cross-links
44+
- Prior fuzz work: `.tasks/done/TASK-127-experimental-parity-invalidation.md`
45+
- Gap backlog: `research/zig-cr/92-gap-backlog.md`
46+
47+
## Progress Log
48+
- 2025-12-23: Created from hypothesis invalidation request.
49+
50+
## Completion Notes
51+
(Empty until done.)

‎.tasks/triage/TASK-191-python-hypothesis-suite.md‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# TASK-191 — Port Python Hypothesis Tests to Zig Parity Suite
2+
3+
## Goal
4+
Port the Python property-based tests (`py/correctness/`) to the bash parity harness to invalidate "Zig parity is complete".
5+
6+
## Status
7+
- State: triage
8+
- Priority: HIGH (these tests were designed to find edge cases)
9+
- Discovered: 2025-12-23 (hypothesis invalidation request)
10+
11+
## Hypothesis to Invalidate
12+
The Python tests use `hypothesis` library for property-based testing. They may cover scenarios our bash tests miss.
13+
14+
## Existing Python Tests
15+
Located in `py/correctness/tests/`:
16+
- `test_cl_merging.py` — Causal length merge logic (~1000 lines)
17+
- `test_sentinel_omission.py` — Sentinel emission rules
18+
- `test_sync.py` — Sync protocol edge cases
19+
20+
## Test Approach
21+
1. **Analyze Python tests** for scenarios not covered by bash harness
22+
2. **Identify key properties** being tested:
23+
- CL merge resolution rules
24+
- Sentinel creation/omission conditions
25+
- Multi-peer sync convergence
26+
3. **Translate to bash tests** that compare Zig vs Rust/C oracle
27+
28+
## Files to Create/Modify
29+
- `zig/harness/test-cl-merge-properties.sh` (new)
30+
- `zig/harness/test-sentinel-properties.sh` (new)
31+
32+
## Acceptance Criteria
33+
1. Port at least 3 key property tests from each Python file
34+
2. Run against both Zig and Rust/C oracle
35+
3. Either find divergence OR increase confidence
36+
37+
## Parent Docs / Cross-links
38+
- Python tests: `py/correctness/tests/`
39+
- Gap backlog: `research/zig-cr/92-gap-backlog.md`
40+
41+
## Progress Log
42+
- 2025-12-23: Created from hypothesis invalidation request.
43+
44+
## Completion Notes
45+
(Empty until done.)

‎.tasks/triage/TASK-192-prior-db-oracle-parity.md‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# TASK-192 — Test Against Prior Database Files (Golden Snapshots)
2+
3+
## Goal
4+
Test Zig extension against real database files created by prior Rust/C versions to invalidate "Zig parity is complete".
5+
6+
## Status
7+
- State: triage
8+
- Priority: HIGH (tests real-world compatibility)
9+
- Discovered: 2025-12-23 (hypothesis invalidation request)
10+
11+
## Hypothesis to Invalidate
12+
"Zig can correctly read/write databases created by Rust/C CR-SQLite."
13+
14+
The prior DB files exist at `py/correctness/prior-dbs/`.
15+
16+
## Test Approach
17+
1. **Load prior DB with Zig** extension
18+
2. **Verify can read**:
19+
- `crsql_db_version()` returns expected value
20+
- `crsql_site_id()` returns stored ID
21+
- `SELECT * FROM crsql_changes` returns expected rows
22+
- Clock tables have expected structure
23+
3. **Verify can write**:
24+
- INSERT new row → clock entries created
25+
- Sync changes to another DB → converges correctly
26+
4. **Compare against Rust/C** doing same operations
27+
28+
## Prior DB Files
29+
- `py/correctness/prior-dbs/` — examine for available versions
30+
31+
## Files to Create
32+
- `zig/harness/test-prior-db-compat.sh` (new)
33+
34+
## Acceptance Criteria
35+
1. Load all prior DB files without error
36+
2. Read operations produce identical results to Rust/C
37+
3. Write operations produce compatible changes
38+
4. Either find divergence OR confirm backward compat
39+
40+
## Parent Docs / Cross-links
41+
- Prior DBs: `py/correctness/prior-dbs/`
42+
- Gap backlog: `research/zig-cr/92-gap-backlog.md`
43+
44+
## Progress Log
45+
- 2025-12-23: Created from hypothesis invalidation request.
46+
47+
## Completion Notes
48+
(Empty until done.)

‎.tasks/triage/TASK-193-rust-integration-check-port.md‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# TASK-193 — Port Rust Integration Check Tests
2+
3+
## Goal
4+
Port tests from `core/rs/integration_check/` to bash parity harness to invalidate "Zig parity is complete".
5+
6+
## Status
7+
- State: triage
8+
- Priority: MEDIUM (many already covered, but check for gaps)
9+
- Discovered: 2025-12-23 (hypothesis invalidation request)
10+
11+
## Hypothesis to Invalidate
12+
"All Rust integration tests have equivalent coverage in the Zig harness."
13+
14+
## Rust Test Files
15+
Located in `core/rs/integration_check/src/t/`:
16+
- `automigrate.rs` — Covered by `test-automigrate.sh`
17+
- `backfill.rs` — Covered by `test-backfill.sh`
18+
- `fract.rs` — Covered by `test-fract*.sh`
19+
- `pack_columns.rs` — Covered by `test-unpack-columns-vtab.sh`
20+
- `pk_only_tables.rs` — Partially covered
21+
- `pk_update.rs` — Covered by `test-pk-update.sh`
22+
- `sync_bit_honored.rs` — Covered by `test-sync-bit-isolation.sh`
23+
- `tableinfo.rs` — Covered by `test-extdata.sh`
24+
- `teardown.rs` — Covered by `test-is-crr.sh`
25+
- `test_cl_set_vtab.rs` — Covered by `test-clset-vtab.sh`
26+
- `test_db_version.rs` — Covered by `test-db-version-parity.sh`
27+
28+
## Test Approach
29+
1. **Audit each Rust test file** for specific assertions
30+
2. **Compare against bash test** to identify gaps
31+
3. **Port missing scenarios** to bash harness
32+
33+
## Files to Create/Modify
34+
- Compare `core/rs/integration_check/src/t/*.rs` vs `zig/harness/test-*.sh`
35+
36+
## Acceptance Criteria
37+
1. Document which Rust tests have bash equivalents
38+
2. Port any missing scenarios
39+
3. Either find divergence OR confirm coverage
40+
41+
## Parent Docs / Cross-links
42+
- Rust tests: `core/rs/integration_check/src/t/`
43+
- Coverage map: `research/zig-cr/92-gap-backlog.md` (Coverage Map Summary section)
44+
45+
## Progress Log
46+
- 2025-12-23: Created from hypothesis invalidation request.
47+
48+
## Completion Notes
49+
(Empty until done.)

‎.tasks/triage/TASK-194-real-world-app-simulation.md‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
# TASK-194 — Real-World Application Simulation Tests
2+
3+
## Goal
4+
Simulate realistic application patterns to invalidate "Zig parity is complete".
5+
6+
## Status
7+
- State: triage
8+
- Priority: HIGH (tests real usage, not contrived scenarios)
9+
- Discovered: 2025-12-23 (hypothesis invalidation request)
10+
11+
## Hypothesis to Invalidate
12+
"Zig behaves correctly under realistic application workloads."
13+
14+
## Test Scenarios
15+
16+
### 1. Todo App Sync
17+
- Create tasks with nested subtasks
18+
- Mark complete/incomplete in different order on two devices
19+
- Sync and verify convergence
20+
21+
### 2. Chat/Notes App
22+
- Long-running conversation with edits
23+
- Offline edits on multiple devices
24+
- Reconnect and merge
25+
26+
### 3. Shopping Cart
27+
- Add/remove items rapidly
28+
- Update quantities concurrently
29+
- Apply discount codes (triggers)
30+
31+
### 4. Collaborative Document
32+
- Multiple users editing same "document" (row with text blob)
33+
- Concurrent field updates
34+
- History/versioning queries
35+
36+
### 5. Inventory Management
37+
- Stock count adjustments
38+
- Transfer between locations
39+
- Audit trail preservation
40+
41+
## Test Approach
42+
1. **Script realistic operation sequences**
43+
2. **Simulate multi-device with separate DBs**
44+
3. **Sync via crsql_changes protocol**
45+
4. **Verify final state matches on all "devices"**
46+
5. **Compare Zig vs Rust/C behavior**
47+
48+
## Files to Create
49+
- `zig/harness/test-app-todo.sh` (new)
50+
- `zig/harness/test-app-chat.sh` (new)
51+
- `zig/harness/test-app-inventory.sh` (new)
52+
53+
## Acceptance Criteria
54+
1. Each app simulation runs without error
55+
2. All "devices" converge to same state
56+
3. Zig and Rust/C produce identical final state
57+
4. Either find divergence OR confirm real-world readiness
58+
59+
## Parent Docs / Cross-links
60+
- Existing realistic tests: `test-realistic-sync.sh`, `test-realistic-offline.sh`, `test-realistic-collab.sh`
61+
- Gap backlog: `research/zig-cr/92-gap-backlog.md`
62+
63+
## Progress Log
64+
- 2025-12-23: Created from hypothesis invalidation request.
65+
66+
## Completion Notes
67+
(Empty until done.)

‎.tasks/triage/TASK-195-adversarial-input-fuzzing.md‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# TASK-195 — Adversarial Input Fuzzing (Malformed crsql_changes)
2+
3+
## Goal
4+
Feed malformed/adversarial inputs to crsql_changes to find divergent error handling.
5+
6+
## Status
7+
- State: triage
8+
- Priority: HIGH (security + robustness)
9+
- Discovered: 2025-12-23 (hypothesis invalidation request)
10+
11+
## Hypothesis to Invalidate
12+
"Zig and Rust/C handle all malformed inputs identically."
13+
14+
## Test Approach
15+
16+
### Malformed Inputs to Generate
17+
1. **Invalid pk blobs**:
18+
- Truncated encoding
19+
- Wrong column count prefix
20+
- Invalid type tags
21+
- Zero-length
22+
- Extremely long
23+
24+
2. **Invalid column values**:
25+
- Wrong type for column
26+
- Oversized blobs
27+
- Invalid UTF-8 in text
28+
- NaN/Inf floats
29+
30+
3. **Invalid metadata**:
31+
- Negative col_version
32+
- Negative db_version
33+
- Negative cl (causal length)
34+
- Invalid site_id (wrong length)
35+
- site_id = all zeros
36+
- site_id = all 0xFF
37+
38+
4. **Invalid cid (column identifier)**:
39+
- Non-existent column name
40+
- Empty string
41+
- Very long column name
42+
- Column name with special chars
43+
44+
5. **Invalid table names**:
45+
- Non-existent table
46+
- System table name
47+
- SQL injection attempts
48+
49+
6. **Sequence attacks**:
50+
- Same pk, different site_id, same col_version
51+
- Duplicate inserts
52+
- Out-of-sequence db_version
53+
54+
## Files to Create
55+
- `zig/harness/test-adversarial-input.sh` (new)
56+
57+
## Acceptance Criteria
58+
1. Both implementations handle malformed input gracefully (error, not crash)
59+
2. Error messages/codes match OR divergence is documented
60+
3. No data corruption from malformed input
61+
4. Either find handling divergence OR confirm robustness parity
62+
63+
## Parent Docs / Cross-links
64+
- Existing error handling: `test-error-handling.sh`
65+
- Gap backlog: `research/zig-cr/92-gap-backlog.md`
66+
67+
## Progress Log
68+
- 2025-12-23: Created from hypothesis invalidation request.
69+
70+
## Completion Notes
71+
(Empty until done.)

0 commit comments

Comments
 (0)