Skip to content

refactor(engine): inline policies on DetectionInput; drop /policies resource#276

Merged
martsokha merged 2 commits into
mainfrom
refactor/policies-inline
Jun 13, 2026
Merged

refactor(engine): inline policies on DetectionInput; drop /policies resource#276
martsokha merged 2 commits into
mainfrom
refactor/policies-inline

Conversation

@martsokha

Copy link
Copy Markdown
Member

Summary

  • Policies stop being a persisted resource. The caller submits the full inline policy bodies on every `POST /api/v1/detections`; the engine snapshots digests onto the persisted detection record and the audit references rules by `(policy_name, rule_name)` instead of `(policy_id, rank)`.
  • `/api/v1/policies[/{id}]` removed (4 endpoints, request/response types, path helper).
  • `Policy::id: Uuid` replaced with `Policy::name: HipStr<'static>`; `PolicyRule` gains a required `name: HipStr<'static>`. Audit-heavy passes share refcounts rather than allocating per-entity.
  • `RuleRank` deleted entirely — no production code ever compared ranks; positions in the chain are now expressed by name.

Approach

Two commits, no functional drift between them.

1. Inline policies + audit-by-name (commit `e8cd7d28`). New types in `nvisy_engine::policy`:

  • `AnyPolicy` — modality-erased sum (Text/Tabular/Image/Audio) mirroring `AnyAudit`. Submitted inline on every detect call.
  • `PolicyDigest { name, version }` — header card stored on the persisted detection record so a SOC-2 auditor can render `" v"` without re-uploading the policy bytes.
  • `PolicyDecisionRef { policy_name, rule_name: Option<...> }` — stamped on every policy-driven `AuditEntry`. `rule_name` is `None` when the policy's `default_action` fallback fired.
  • `validate_policy_namespace` — runs at the top of `Engine::detect` and rejects duplicate policy names within a submission or duplicate rule names within a policy. The audit's identity-by-name guarantee depends on these invariants.

Server-side: `/api/v1/policies[/{id}]` removed; `POST /api/v1/detections` body's `policies: Vec` becomes `Vec`. Registry-side: `register_policy` / `read_policy` / `unregister_policy` / `unregister_all_policies` / `list_policies` / `list_policies_with_summary` / `policy_cache` / `policies_ks` / `ResourceCache` / `ResourceGuard` deleted.

The detect→redact handoff carries the prepared store through an internal `DetectionHandoff` so the redact pipeline can re-evaluate operator specs without the caller re-submitting policies. After a process restart the in-memory record is gone and the redaction can't proceed — that was already the existing semantic; this PR just makes it explicit.

2. Cut clones; tighten the store surface (commit `85c6eaa7`). Follow-up sweep that addresses an audit pass on the first commit:

  • `PolicyStore::from_any_policies(Vec)` consumes by value and moves each `Policy` straight into `Arc::new(p)` — no deep policy clones.
  • `DetectionRecord` holds `Arc` + `Vec` built once at `register_pending` time. Redact handoff is an `Arc::clone` of the store.
  • `DetectionPipeline::register_pending` now consumes `DetectionInput` and returns a `PreparedDetection` that `execute` consumes — the `AnyPolicy` bodies are touched exactly once.
  • `SharedData::policies` typed as `Arc`; dead `with_policy` / `with_policies` builders dropped.
  • `PolicyStore` public surface reduced to construction + `resolve`; `insert` / `set` / `get` / `len` / `is_empty` / `new` dropped. `TypeMap`-backed dispatch retained so adding a new modality is purely a new `AnyPolicy` arm — no hardcoded fields or per-modality methods.
  • `validate_policy_namespace` stops allocating an intermediate `Vec<&str>` per policy.
  • `AnyPolicy::name` returns `&str`. `PolicyDigest` / `PolicyDecisionRef` drop unused `Eq` / `PartialEq` derives.
  • `nvisy-server` README rewritten to match `nvisy-engine` / `nvisy-codec` style (tagline + Overview prose; no endpoint or feature tables). OpenAPI `policies` tag and the policies mention in `service/mod.rs` docs cleaned up.

Net diff

`-1110 / +519` across 31 files. Six files deleted entirely: `registry/resource_cache.rs`, `handler/policies.rs`, `handler/request/policies.rs`, `handler/response/policies.rs`, `build_policy_store` (was internal). `RuleRank` and the `set` / `insert` / `get` / `len` PolicyStore surface gone.

Test plan

  • Four-gate verify clean on `--all-features` (build, clippy `-D warnings`, full test suite ~600 tests, rustdoc `-D warnings`).
  • cargo-deny clean.
  • No-default-features build still works (`-p nvisy-codec --no-default-features --features txt`).
  • One TOML fixture in `crates/nvisy-engine/tests/redaction_policy.rs` updated to add `name` fields on rules — every other test passes unchanged because they never constructed `DetectionInput::policies` directly.
  • 4 new unit tests on `validate_policy_namespace` cover empty input, duplicate-policy rejection, duplicate-rule rejection, and the happy path.

martsokha and others added 2 commits June 13, 2026 12:17
…esource

Policies stop being a persisted resource: the caller submits the
full inline policy bodies on every detect call, the engine snapshots
digests onto the persisted detection record, and the audit references
rules by `(policy_name, rule_name)` instead of `(policy_id, rank)`.

New types in `nvisy_engine::policy`:
- `AnyPolicy` — modality-erased sum (Text/Tabular/Image/Audio)
  mirroring `AnyAudit`. Submitted inline on every detect call.
- `PolicyDigest { name, version }` — header card stored on
  `DetectionResult`. No rule bodies.
- `PolicyDecisionRef { policy_name, rule_name: Option }` — stamped
  on every policy-driven `AuditEntry`. `rule_name` is `None` when the
  policy's `default_action` fallback fired.
- `validate_policy_namespace` — runs at the top of `Engine::detect`,
  rejects duplicate policy names within a submission and duplicate
  rule names within a policy. The audit's identity-by-name guarantee
  depends on these invariants.

`Policy::id: Uuid` replaced with `Policy::name: HipStr<'static>`.
`PolicyRule` gains a required `name: HipStr<'static>` field. `RuleRank`
deleted. `HipStr` chosen for the names so audit-heavy passes share
refcounts rather than allocating per-entity.

Server-side:
- `/api/v1/policies[/{id}]` removed (4 endpoints + request/response
  types + path helper + `service/mod.rs` doc trail).
- `POST /api/v1/detections` body's `policies: Vec<Uuid>` becomes
  `policies: Vec<AnyPolicy>` — inline submission only.

Registry-side:
- `register_policy`/`read_policy`/`unregister_policy`/
  `unregister_all_policies`/`list_policies`/
  `list_policies_with_summary` deleted.
- `policies_ks` Fjall keyspace + `policy_cache` removed.
- `ResourceCache`/`ResourceGuard` deleted (policy persistence was
  its only consumer).

The detect→redact handoff carries the full `Vec<AnyPolicy>` through
an internal `DetectionHandoff` shape so the redact pipeline can
re-evaluate operator specs without the caller re-submitting them.
After a process restart the in-memory record is gone and the
redaction can't proceed — that was already the existing semantic;
this PR just makes it explicit.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Follows up the inline-policies refactor with a sweep that addresses
the audit findings (no functional change).

- PolicyStore::from_any_policies consumes Vec<AnyPolicy> by value
  and moves Policy<M> straight into Arc::new(p) — no deep clones.
- Detection record holds Arc<PolicyStore> + Vec<PolicyDigest>
  built once at register_pending time. Redact handoff is an
  Arc::clone of the store; build_policy_store is gone.
- DetectionPipeline::register_pending now consumes DetectionInput
  and returns a PreparedDetection that execute consumes — the
  AnyPolicy bodies are touched exactly once on the detect side.
- SharedData::policies typed as Arc<PolicyStore>; dead with_policy /
  with_policies builder methods removed.
- PolicyStore public surface reduced to construction + resolve:
  insert / set / get / len / is_empty / new dropped. tests rewritten
  around from_any_policies + resolve.
- validate_policy_namespace stops allocating an intermediate
  Vec<&str> per policy; uses a shared check_rule_names helper.
- AnyPolicy::name returns &str. PolicyDigest and PolicyDecisionRef
  drop unused Eq / PartialEq derives.
- nvisy-server README rewritten to match nvisy-engine / nvisy-codec
  style (tagline + Overview prose + standard footer; no endpoint or
  feature tables).
- OpenAPI 'policies' tag and the policies mention in service/mod.rs
  docs removed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@martsokha martsokha self-assigned this Jun 13, 2026
@martsokha martsokha added engine redaction engine, pipeline runtime, orchestration, configuration refactor code restructuring without behavior change labels Jun 13, 2026
@martsokha martsokha merged commit 8426879 into main Jun 13, 2026
5 checks passed
@martsokha martsokha deleted the refactor/policies-inline branch June 13, 2026 10:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

engine redaction engine, pipeline runtime, orchestration, configuration refactor code restructuring without behavior change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant