From 175c024159c777ef39d56d3d82fca5632b7b8e5e Mon Sep 17 00:00:00 2001 From: James Sadler Date: Mon, 20 Jul 2026 11:27:20 +1000 Subject: [PATCH 1/4] chore: configure agent skill conventions for issue tracking and domain docs Record where issues live and how domain documentation is laid out, so engineering skills read from the right places instead of assuming defaults. - Issue tracker is Linear (Product Engineering, CIP-), not GitHub Issues. GitHub holds code and PRs only. Prefers linear-cli for writes since the Linear MCP tools are unavailable in headless runs. - Domain docs are multi-context: CONTEXT-MAP.md points at per-package CONTEXT.md files under packages/*/. Per-package glossaries and ADRs are created lazily, so their absence is expected. No triage labels recorded; the triage skill is not installed. Stable-Commit-Id: q-5q3yy9rhgw8n99g19fbvgkytvr --- CLAUDE.md | 12 +++++- CONTEXT-MAP.md | 39 +++++++++++++++++++ docs/agents/domain.md | 61 ++++++++++++++++++++++++++++++ docs/agents/issue-tracker.md | 73 ++++++++++++++++++++++++++++++++++++ 4 files changed, 184 insertions(+), 1 deletion(-) create mode 100644 CONTEXT-MAP.md create mode 100644 docs/agents/domain.md create mode 100644 docs/agents/issue-tracker.md diff --git a/CLAUDE.md b/CLAUDE.md index 61267dcc..c08801ee 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -237,4 +237,14 @@ For significant releases, write an `ANNOUNCEMENT.md` for the GitHub Discussions > [!IMPORTANT] > Critical information users must know. ``` -4. After copying content to GitHub Discussions, delete `ANNOUNCEMENT.md` from the repo \ No newline at end of file +4. After copying content to GitHub Discussions, delete `ANNOUNCEMENT.md` from the repo + +## Agent skills + +### Issue tracker + +Issues are tracked in **Linear** (team: Product Engineering, key `CIP-`), not GitHub Issues. GitHub holds code and PRs only. See `docs/agents/issue-tracker.md`. + +### Domain docs + +**Multi-context** — the root `CONTEXT-MAP.md` points at per-package `CONTEXT.md` files under `packages/*/`. See `docs/agents/domain.md`. \ No newline at end of file diff --git a/CONTEXT-MAP.md b/CONTEXT-MAP.md new file mode 100644 index 00000000..4d37470d --- /dev/null +++ b/CONTEXT-MAP.md @@ -0,0 +1,39 @@ +# Context Map + +CipherStash Proxy is a Cargo workspace. Each context below owns a distinct domain +vocabulary. Read the `CONTEXT.md` for the context you're working in before exploring +its code. + +Per-context `CONTEXT.md` files are created lazily by `/domain-modeling` as terms get +resolved — a missing one is expected, not a gap to fill upfront. + +| Context | Path | Domain | +|---|---|---| +| Proxy | `packages/cipherstash-proxy/` | PostgreSQL wire protocol, session and message handling, client authentication, TLS, ZeroKMS key management, encrypt/decrypt of column values | +| EQL Mapper | `packages/eql-mapper/` | SQL parsing, type inference over statements, schema analysis, transformation rules that rewrite plaintext SQL into EQL v2 operations | +| Integration | `packages/cipherstash-proxy-integration/` | End-to-end test harness — container fixtures, encrypted-scenario coverage across the proxy and mapper together | +| Showcase | `packages/showcase/` | Healthcare example data model demonstrating EQL v2 encryption with realistic relationships | + +`packages/eql-mapper-macros/` is proc-macro support for EQL Mapper, not a context of its +own — treat it as part of the EQL Mapper context. + +## Shared vocabulary + +Terms defined once for the whole system live here rather than in any one context. + +- **EQL v2** — Encrypt Query Language; the SQL-level encoding that makes encrypted + values searchable. +- **`eql_v2_encrypted`** — the PostgreSQL column type holding an encrypted value. +- **ZeroKMS** — CipherStash's key management service, which the proxy calls to encrypt + and decrypt. +- **Keyset** — the ZeroKMS key collection a workspace encrypts against. + +## Cross-context term collisions + +Terms that mean different things depending on where you are. Resolve them against your +own context. + +- **Type** — a PostgreSQL wire-protocol type in Proxy; an inferred SQL expression type + in EQL Mapper. +- **Index** — a PostgreSQL index in Proxy; a searchable-encryption index (ORE, match, + unique) in EQL Mapper and EQL config. diff --git a/docs/agents/domain.md b/docs/agents/domain.md new file mode 100644 index 00000000..8623e9f5 --- /dev/null +++ b/docs/agents/domain.md @@ -0,0 +1,61 @@ +# Domain Docs + +How the engineering skills should consume this repo's domain documentation when +exploring the codebase. + +This is a **multi-context** repo — a Cargo workspace whose packages carry distinct +domain vocabularies. + +## Before exploring, read these + +- **`CONTEXT-MAP.md`** at the repo root — it points at one `CONTEXT.md` per context. + Read each one relevant to the topic. +- **`docs/adr/`** — system-wide architectural decisions. +- **`packages//docs/adr/`** — context-scoped decisions for that package. + +If any of these files don't exist, **proceed silently**. Don't flag their absence; +don't suggest creating them upfront. The `/domain-modeling` skill (reached via +`/grill-with-docs` and `/improve-codebase-architecture`) creates them lazily when terms +or decisions actually get resolved. + +Note that `ARCHITECTURE.md` at the repo root already describes system structure. It is +not a glossary — read it for orientation, but domain *terms* belong in `CONTEXT.md`. + +## File structure + +This repo is a Cargo workspace, so contexts live under `packages/`, not `src/`: + +``` +/ +├── CONTEXT-MAP.md +├── docs/adr/ ← system-wide decisions +└── packages/ + ├── cipherstash-proxy/ + │ ├── CONTEXT.md + │ └── docs/adr/ ← context-specific decisions + └── eql-mapper/ + ├── CONTEXT.md + └── docs/adr/ +``` + +## Use the glossary's vocabulary + +When your output names a domain concept (in an issue title, a refactor proposal, a +hypothesis, a test name), use the term as defined in the relevant `CONTEXT.md`. Don't +drift to synonyms the glossary explicitly avoids. + +If the concept you need isn't in the glossary yet, that's a signal — either you're +inventing language the project doesn't use (reconsider) or there's a real gap (note it +for `/domain-modeling`). + +Watch for terms that mean different things in different contexts. "Type", for example, +means a PostgreSQL wire type in `cipherstash-proxy` and an inferred SQL type in +`eql-mapper`. That divergence is exactly why this repo is multi-context — resolve the +term against the context you're working in, not the other one. + +## Flag ADR conflicts + +If your output contradicts an existing ADR, surface it explicitly rather than silently +overriding: + +> _Contradicts ADR-0007 (event-sourced orders) — but worth reopening because…_ diff --git a/docs/agents/issue-tracker.md b/docs/agents/issue-tracker.md new file mode 100644 index 00000000..ed19719a --- /dev/null +++ b/docs/agents/issue-tracker.md @@ -0,0 +1,73 @@ +# Issue Tracker + +Issues for this repo live in **Linear**, not GitHub Issues. + +- **Workspace:** CipherStash +- **Team:** Product Engineering +- **Issue key prefix:** `CIP-` (e.g. `CIP-3233`) + +GitHub (`cipherstash/proxy`) holds code, branches, and pull requests. It is not the +issue tracker. Do not run `gh issue create` for this repo. + +## How to read and write issues + +Two access paths exist. **Prefer the CLI for writes.** + +### `linear-cli` (preferred) + +The `linear` command (`@schpet/linear-cli`, or `npx @schpet/linear-cli`) — see the +`linear-cli` skill for full usage. + +```bash +linear issue list +linear issue view CIP-3233 +linear issue create --title "..." --description-file /path/to/desc.md +linear issue update CIP-3233 --status "In Progress" +``` + +Always pass markdown via `--description-file` / `--body-file` rather than inline +arguments — inline content produces literal `\n` sequences in the Linear UI. + +### Linear MCP tools (fallback, and for querying) + +`mcp__claude_ai_Linear__*` — good for search and read-heavy work +(`list_issues`, `get_issue`, `save_issue`). Note these are +interactively authenticated and **may be unavailable in headless or cron runs**, so +anything scripted should prefer the CLI. + +## Statuses + +The Product Engineering workflow: + +| Status | Type | +|---|---| +| `Backlog` | backlog | +| `Todo` | unstarted | +| `In Progress` | started | +| `In Review` | started | +| `Done` | completed | +| `Canceled` | canceled | +| `Duplicate` | duplicate | + +## How the skills should use this + +- **`to-tickets`** — create one Linear issue per tracer-bullet ticket in team + Product Engineering, starting in `Todo`. Express blocking edges as **native Linear + blocked-by relations**, not as prose in the description and not as files under + `.scratch/`. Any issue whose blockers are `Done` is grabbable. +- **`implement`** — take an issue by its `CIP-####` key. Move it to `In Progress` when + work starts, and `In Review` when the PR opens. Leave the move to `Done` to the + human merging. +- **`to-spec`** — link the resulting spec back to the originating issue as a comment or + attachment on the Linear issue. +- **`code-review`** — the "Spec" axis reads the originating `CIP-####` issue for intent. + +## Linking PRs to issues + +Reference the issue key in the PR title or body (e.g. `CIP-3233`) so Linear's GitHub +integration attaches the PR to the issue automatically. + +## PRs as a request surface + +**Off.** External pull requests are not treated as incoming triage requests for this +repo. Flip this flag if that changes. From 541012b6f6616447c1e36bd917faf43370f6b36c Mon Sep 17 00:00:00 2001 From: James Sadler Date: Mon, 20 Jul 2026 15:36:01 +1000 Subject: [PATCH 2/4] chore: record triage label vocabulary for agent skills Stable-Commit-Id: q-56zbh5gc0z7wg9g0q81b58csv7 --- CLAUDE.md | 4 ++++ docs/agents/triage-labels.md | 25 +++++++++++++++++++++++++ 2 files changed, 29 insertions(+) create mode 100644 docs/agents/triage-labels.md diff --git a/CLAUDE.md b/CLAUDE.md index c08801ee..2ce0ca3e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -245,6 +245,10 @@ For significant releases, write an `ANNOUNCEMENT.md` for the GitHub Discussions Issues are tracked in **Linear** (team: Product Engineering, key `CIP-`), not GitHub Issues. GitHub holds code and PRs only. See `docs/agents/issue-tracker.md`. +### Triage labels + +**Defaults** — the five canonical roles, each label string equal to its name. Only `wontfix` exists in Linear today; the rest are created on demand. See `docs/agents/triage-labels.md`. + ### Domain docs **Multi-context** — the root `CONTEXT-MAP.md` points at per-package `CONTEXT.md` files under `packages/*/`. See `docs/agents/domain.md`. \ No newline at end of file diff --git a/docs/agents/triage-labels.md b/docs/agents/triage-labels.md new file mode 100644 index 00000000..5d5746cf --- /dev/null +++ b/docs/agents/triage-labels.md @@ -0,0 +1,25 @@ +# Triage Labels + +The skills speak in terms of five canonical triage roles. This file maps those roles to +the actual label strings used in this repo's issue tracker (Linear, team Product +Engineering — see [issue-tracker.md](./issue-tracker.md)). + +| Label in mattpocock/skills | Label in our tracker | Meaning | +| -------------------------- | -------------------- | ---------------------------------------- | +| `needs-triage` | `needs-triage` | Maintainer needs to evaluate this issue | +| `needs-info` | `needs-info` | Waiting on reporter for more information | +| `ready-for-agent` | `ready-for-agent` | Fully specified, ready for an AFK agent | +| `ready-for-human` | `ready-for-human` | Requires human implementation | +| `wontfix` | `wontfix` | Will not be actioned | + +When a skill mentions a role (e.g. "apply the AFK-ready triage label"), use the +corresponding label string from this table. + +Edit the right-hand column to match whatever vocabulary you actually use. + +## Creating the labels + +Only `wontfix` currently exists on the Product Engineering team. The other four are +created lazily — the first time `triage` needs one, create it on the team rather than +failing. These are workspace-level Linear labels, unparented; do not nest them under the +existing `Product` / `R&D` / `Issue type` groups. From ac5c4340d6466aea8e2dae57a11e034347abad6c Mon Sep 17 00:00:00 2001 From: James Sadler Date: Thu, 23 Jul 2026 16:05:48 +1000 Subject: [PATCH 3/4] docs(agents): address setup-skills review nits Minor documentation cleanups from the #422 review: - domain.md: language the directory-tree fence (`text`) so it passes markdownlint MD040. - domain.md: replace the copied-in `ADR-0007 (event-sourced orders)` example (no such ADR, wrong domain) with a neutral placeholder. - triage-labels.md: drop the leftover reader-instruction line. - CLAUDE.md: add the missing trailing newline. The review's manager.rs:146 line-ref nit in CONTEXT-MAP.md and eql-mapper's CONTEXT.md is intentionally left alone: downstream commits in this stack rewrite those exact passages, so editing them here only conflicts. Stable-Commit-Id: q-7jzthb123a03fjybs06t5ptvbr --- CLAUDE.md | 2 +- docs/agents/domain.md | 4 ++-- docs/agents/triage-labels.md | 2 -- 3 files changed, 3 insertions(+), 5 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 2ce0ca3e..6030345e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -251,4 +251,4 @@ Issues are tracked in **Linear** (team: Product Engineering, key `CIP-`), not Gi ### Domain docs -**Multi-context** — the root `CONTEXT-MAP.md` points at per-package `CONTEXT.md` files under `packages/*/`. See `docs/agents/domain.md`. \ No newline at end of file +**Multi-context** — the root `CONTEXT-MAP.md` points at per-package `CONTEXT.md` files under `packages/*/`. See `docs/agents/domain.md`. diff --git a/docs/agents/domain.md b/docs/agents/domain.md index 8623e9f5..abe7c818 100644 --- a/docs/agents/domain.md +++ b/docs/agents/domain.md @@ -25,7 +25,7 @@ not a glossary — read it for orientation, but domain *terms* belong in `CONTEX This repo is a Cargo workspace, so contexts live under `packages/`, not `src/`: -``` +```text / ├── CONTEXT-MAP.md ├── docs/adr/ ← system-wide decisions @@ -58,4 +58,4 @@ term against the context you're working in, not the other one. If your output contradicts an existing ADR, surface it explicitly rather than silently overriding: -> _Contradicts ADR-0007 (event-sourced orders) — but worth reopening because…_ +> _Contradicts ADR-000N (its title) — but worth reopening because…_ diff --git a/docs/agents/triage-labels.md b/docs/agents/triage-labels.md index 5d5746cf..dccc2505 100644 --- a/docs/agents/triage-labels.md +++ b/docs/agents/triage-labels.md @@ -15,8 +15,6 @@ Engineering — see [issue-tracker.md](./issue-tracker.md)). When a skill mentions a role (e.g. "apply the AFK-ready triage label"), use the corresponding label string from this table. -Edit the right-hand column to match whatever vocabulary you actually use. - ## Creating the labels Only `wontfix` currently exists on the Product Engineering team. The other four are From 2f907354521eb112f6ce64b6f1bb2bc7830eb0e2 Mon Sep 17 00:00:00 2001 From: James Sadler Date: Mon, 20 Jul 2026 15:52:14 +1000 Subject: [PATCH 4/4] docs: add domain glossaries for Proxy and EQL Mapper Resolve the terms that collide hardest across the two contexts, and correct the Index entry in CONTEXT-MAP.md, which had the collision backwards. - EqlTrait is a capability; the storage that satisfies it is a SEM term (searchable encrypted metadata), many-to-one. Neither is an index. - Ban 'session' in Proxy: the code used it for both a connection and a single statement, and PostgreSQL owns the word for a third thing. - Split EncryptConfig into encrypt credentials vs column encrypt config; the single name currently covers two unrelated public structs. - Record that Proxy assigns EqlTraits::all() to every encrypted column, so EQL Mapper's bound checking cannot fire in production. Stable-Commit-Id: q-0tpfq3xh26grdk0fwpcxr1v6hg --- CONTEXT-MAP.md | 56 ++++++++++++- packages/cipherstash-proxy/CONTEXT.md | 115 ++++++++++++++++++++++++++ packages/eql-mapper/CONTEXT.md | 102 +++++++++++++++++++++++ 3 files changed, 269 insertions(+), 4 deletions(-) create mode 100644 packages/cipherstash-proxy/CONTEXT.md create mode 100644 packages/eql-mapper/CONTEXT.md diff --git a/CONTEXT-MAP.md b/CONTEXT-MAP.md index 4d37470d..31c69ffa 100644 --- a/CONTEXT-MAP.md +++ b/CONTEXT-MAP.md @@ -9,14 +9,32 @@ resolved — a missing one is expected, not a gap to fill upfront. | Context | Path | Domain | |---|---|---| -| Proxy | `packages/cipherstash-proxy/` | PostgreSQL wire protocol, session and message handling, client authentication, TLS, ZeroKMS key management, encrypt/decrypt of column values | -| EQL Mapper | `packages/eql-mapper/` | SQL parsing, type inference over statements, schema analysis, transformation rules that rewrite plaintext SQL into EQL v2 operations | +| Proxy | [`packages/cipherstash-proxy/`](./packages/cipherstash-proxy/CONTEXT.md) | PostgreSQL wire protocol, connection and message handling, client authentication, TLS, ZeroKMS key management, encrypt/decrypt of column values | +| EQL Mapper | [`packages/eql-mapper/`](./packages/eql-mapper/CONTEXT.md) | SQL parsing, type inference over statements, schema analysis, transformation rules that rewrite plaintext SQL into EQL v2 operations | | Integration | `packages/cipherstash-proxy-integration/` | End-to-end test harness — container fixtures, encrypted-scenario coverage across the proxy and mapper together | | Showcase | `packages/showcase/` | Healthcare example data model demonstrating EQL v2 encryption with realistic relationships | `packages/eql-mapper-macros/` is proc-macro support for EQL Mapper, not a context of its own — treat it as part of the EQL Mapper context. +## Relationships + +- **Proxy → EQL Mapper**: Proxy loads the database schema and hands it over with each + column marked native or encrypted; EQL Mapper returns a type-checked, rewritten + statement. Proxy then reads the per-node EQL term shapes back out to decide how to + encrypt each value. +- **Identity across the seam**: EQL Mapper's `TableColumn` and Proxy's `Identifier` are + the same `table.column` pair under two names. That pair is the only key joining a typed + AST node to its encryption config. +- **Capability across the seam — currently broken.** Proxy marks every encrypted column + with *all* `EqlTrait`s (`packages/cipherstash-proxy/src/proxy/schema/manager.rs:146`) + because it derives them from the PostgreSQL column type alone. The column encrypt + config, which knows the SEM terms actually configured, is loaded by a separate manager + that never meets the schema loader. EQL Mapper's bound checking is therefore + unreachable in production: a query needing `Ord` on a column with no ordering SEM term + type-checks cleanly and fails later. Read `EqlTraits` on a column as *intended* + capability, not observed. + ## Shared vocabulary Terms defined once for the whole system live here rather than in any one context. @@ -27,6 +45,22 @@ Terms defined once for the whole system live here rather than in any one context - **ZeroKMS** — CipherStash's key management service, which the proxy calls to encrypt and decrypt. - **Keyset** — the ZeroKMS key collection a workspace encrypts against. +- **SEM (searchable encrypted metadata)** — the encrypted material stored alongside a + value that makes some operation on it possible without decryption. A **SEM term** is + one such piece of metadata; ORE, unique, match and SteVec are SEM **types**. +- **EqlTrait** — a *capability* an encrypted column has: equality, ordering, token + match, JSON traversal, containment. Several SEM types can satisfy the same trait, so + the relationship is many-to-one: SEM terms are the storage, traits are what the storage + buys you. + + Say **capability/trait** when you mean what a query may do, and **SEM term** when you + mean what is written to the column. Do not call either an **index** — that word is + reserved for a PostgreSQL index. + + > The EQL config JSON and `cipherstash_client`'s `ColumnConfig` still spell SEM terms + > `indexes` / `IndexType`. That spelling is a wire and storage format shared with EQL + > and customer-authored config, so it is not ours to rename unilaterally — prefer SEM + > in prose and new code, and read `indexes` in those payloads as "SEM terms". ## Cross-context term collisions @@ -35,5 +69,19 @@ own context. - **Type** — a PostgreSQL wire-protocol type in Proxy; an inferred SQL expression type in EQL Mapper. -- **Index** — a PostgreSQL index in Proxy; a searchable-encryption index (ORE, match, - unique) in EQL Mapper and EQL config. +- **Index** — means a PostgreSQL index in *both* contexts (EQL Mapper's only use of the + word is GIN indexes). Where it appears meaning searchable encryption — the `indexes` + key in EQL config, `IndexType`, "Unknown Index Term" — read it as **SEM term** and see + the shared vocabulary above. +- **Column** — an encrypted column's runtime config in Proxy; a schema column or a + projection column in EQL Mapper. Also `DataColumn` (a wire value) and + `RowDescriptionField` (a result descriptor) in Proxy. +- **Statement** — a type-analysed statement in Proxy; the parsed AST or a + `TypeCheckedStatement` in EQL Mapper. Four senses are live in Proxy alone. +- **Term** — the *shape* of an encrypted payload (`EqlTerm`) in EQL Mapper; a piece of + searchable encrypted metadata in Proxy and EQL config. Qualify as *EQL term* or + *SEM term*. +- **Projection** — a type in EQL Mapper's lattice; in Proxy, a positional list where a + missing entry means "native, do not encrypt". +- **Session** — banned in Proxy (see its `CONTEXT.md`); PostgreSQL owns the word, and the + code has used it for both a connection and a single statement. diff --git a/packages/cipherstash-proxy/CONTEXT.md b/packages/cipherstash-proxy/CONTEXT.md new file mode 100644 index 00000000..9289b0a0 --- /dev/null +++ b/packages/cipherstash-proxy/CONTEXT.md @@ -0,0 +1,115 @@ +# Proxy + +Speaks the PostgreSQL wire protocol to client applications, hands their SQL to EQL Mapper +for type checking and rewriting, encrypts and decrypts column values through ZeroKMS, and +forwards everything to the real database. It owns everything about *connections* and +*ciphertext*; it owns nothing about type inference. + +## Language + +### Connections and statements + +**Connection**: +One client application's TCP connection to Proxy, from startup to termination. All +per-client state hangs off it. +_Avoid_: session — PostgreSQL already owns that word for something else, and this +codebase has used it for both a connection and a single statement. + +**Context**: +The mutable state belonging to one connection — its prepared statements, portals, +pending describes and executes, and keyset. + +**Frontend** / **Backend**: +The two halves of the proxy pipeline by direction of travel: Frontend carries client to +database, Backend carries database to client. These are *not* the PostgreSQL protocol's +frontend/backend roles, and Backend is not the database. + +**Statement**: +A type-analysed SQL statement — its param columns, projection columns, literal columns +and PostgreSQL param type OIDs. The parsed AST and EQL Mapper's `TypeCheckedStatement` +are different things; do not call either of them a Statement here. + +**Portal**: +A prepared statement bound to parameter values, ready to execute. Either `Encrypted`, +carrying the analysed statement, or `Passthrough` when nothing in it is encrypted. + +**Statement metrics scope**: +The measurement window around a single statement — opened at Parse or Query, closed when +the statement completes. Many of these occur per connection. +_Avoid_: session (`start_session`, `SessionId` and the +`..._statements_session_duration_seconds` metric all use this sense and are misnamed). + +**Passthrough**: +Traffic forwarded without encryption or rewriting. Three independent conditions produce +it and they are not interchangeable: no encrypt config is loaded, mapping is disabled by +configuration, or `SET CIPHERSTASH.UNSAFE_DISABLE_MAPPING` was issued on this connection. +Name the condition rather than saying "passthrough" alone. + +### Encryption + +**Column**: +An encrypted column as this crate sees it — its `Identifier`, its column config, its +resolved PostgreSQL type, and the EQL term shape it takes in the current statement. +_Avoid_: confusing with EQL Mapper's schema/projection columns, or with `DataColumn` +(a value on the wire) and `RowDescriptionField` (a result descriptor). + +**Identifier**: +The `table.column` pair that keys the encrypt config. EQL Mapper calls the same thing a +`TableColumn`. + +**Encrypt credentials**: +The CipherStash client credentials Proxy authenticates to ZeroKMS with — client id, +client key, default keyset. Read from `[encrypt]` / `CS_ENCRYPT__*`. +_Avoid_: `EncryptConfig`, which currently names this *and* an unrelated struct. + +**Column encrypt config**: +The per-column encryption configuration — which columns are encrypted and with which SEM +terms — read from the EQL config table in the *database* at runtime, and reloaded when +DDL is observed. +_Avoid_: `EncryptConfig`; also "encrypt schema", despite `ReloadCommand::EncryptSchema` +naming the reload of this thing. + +**Client id**: +Ambiguous — always qualify. The **connection number** is the incrementing counter stamped +on log lines to correlate one connection's traffic. The **CipherStash client id** is the +credential UUID from `CS_ENCRYPT__CLIENT_ID`. They share a name and a field spelling and +are otherwise unrelated. + +**Keyset**: +The ZeroKMS collection of keys a workspace encrypts against. Addressed by UUID or by +name, and selectable per connection via `SET CIPHERSTASH.KEYSET_ID` / `KEYSET_NAME`. + +**Scoped cipher**: +A cipher bound to exactly one keyset. Cached per keyset, so switching keyset switches +cipher. + +**Plaintext**: +A value in its typed, pre-encryption form. The boundary type between PostgreSQL wire +values and ZeroKMS. + +**EQL ciphertext**: +An encrypted value in the shape stored in the database. What encryption produces and +decryption consumes. + +**Store** / **Query** operation: +The two directions of encryption. Storing writes a value with all its SEM terms; querying +produces only the term needed to match. The statement's EQL term shape decides which. + +### Control plane + +**`SET CIPHERSTASH.*`**: +Proxy's in-band control API, intercepted rather than forwarded — `KEYSET_ID`, +`KEYSET_NAME`, `UNSAFE_DISABLE_MAPPING`. These are the supported spellings; log messages +that print `CIPHERSTASH.DISABLE_MAPPING` are wrong. + +**Reload**: +Re-reading state from the database after observed DDL. Two independent things reload: the +database schema, and the column encrypt config. + +## Note on `session` + +This glossary bans `session` because the code uses it for two different things and +PostgreSQL uses it for a third. The identifiers `SessionId`, `start_session`, +`finish_session` and the Prometheus metric name all still carry it. Read them as +*statement metrics scope*; the doc comments on `Frontend` and `Backend` that say "session +context" mean *connection*. diff --git a/packages/eql-mapper/CONTEXT.md b/packages/eql-mapper/CONTEXT.md new file mode 100644 index 00000000..31e532b2 --- /dev/null +++ b/packages/eql-mapper/CONTEXT.md @@ -0,0 +1,102 @@ +# EQL Mapper + +Parses SQL, infers a type for every node, and rewrites statements that touch encrypted +columns into their EQL v2 equivalents. It knows nothing about the PostgreSQL wire +protocol, ZeroKMS, or ciphertext — it reasons about types and rewrites syntax. + +## Language + +### The type system + +**Type**: +A node's inferred type in this crate's own lattice — either a resolved `Value`, a +unification `Var`, or an `Associated` type. Never a PostgreSQL wire type. +_Avoid_: SQL type, Postgres type (those belong to Proxy). + +**Value**: +A *resolved* type — one of `Eql`, `Native`, `Array`, `Projection`, `SetOf`. It means +"value type", not a datum. The SQL AST's literal node is also called `Value`; when both +are in scope, alias the AST one. +_Avoid_: using bare `Value` for a SQL literal. + +**Native**: +A column or expression that is not encrypted. Native types satisfy *every* `EqlTrait` +bound — that is a deliberate escape hatch for the type checker, not a claim about the +database. + +**EqlValue**: +The identity of an encrypted column paired with the capabilities configured for it — +a `TableColumn` plus `EqlTraits`. + +**EqlTrait**: +A class of operation an encrypted value supports: `Eq`, `Ord`, `TokenMatch`, `JsonLike`, +`Contain`. A **capability**, not a storage structure — several SEM terms can satisfy one +trait. See the shared vocabulary in `CONTEXT-MAP.md`. +_Avoid_: index, index type (those name the storage, not the capability). + +**EqlTraits**: +A set of `EqlTrait`s. Read in two opposite directions depending on position: as +*required bounds* on a `Var`, and as *implemented capabilities* on an `EqlValue`. +_Avoid_: features, bounds, trait_impls as distinct concepts — they are all this one type. + +**EqlTerm**: +The shape of encrypted payload a node needs — `Full`, `Partial`, `JsonAccessor`, +`JsonPath`, `Tokenized`. `Partial` is a subset of `Full`, not an unresolved type. +_Avoid_: "term" meaning an encrypted search term; that sense belongs to Proxy. + +**Projection**: +The resultset shape of a statement or subquery — an ordered list of optionally-named +column types. + +**alias**: +The *effective name* of a projection column, which is the user's alias when one was +written and the underlying schema column name otherwise. Identifier resolution matches +against it. +_Avoid_: reading `alias` as "user-supplied alias only". + +### Naming and resolution + +**Relation**: +Anything you can select from — a table, view, CTE, or derived subquery — reduced to a +name plus a projection type. + +**Scope**: +The lexical frame holding the relations visible at a point in the statement. + +**TableColumn**: +The `table.column` identity of an encrypted column. It is the join key to Proxy's +encrypt config. +_Avoid_: identifier (Proxy uses that for its own type). + +**Schema**: +The database's tables and columns as this crate sees them, each column marked `Native` +or `Eql`. Loaded from the live database, not from a migration file. + +**Overlay**: +A per-transaction mask over the loaded `Schema` recording DDL that has run but not +committed. A table is either shadowed by a new definition or hidden as dropped. + +### Output + +**TypeCheckedStatement**: +The result of type checking — the projection, params, literals and node types, plus the +entry point that applies transformations. + +**TransformationRule**: +A composable mutation of the typed AST that rewrites plaintext SQL into EQL v2 +operations. Tuples of rules are themselves rules. +_Avoid_: using "rule" for a typing rule; those are operator/function signature +declarations. + +**Param**: +A `$N` placeholder position in the statement. Its PostgreSQL type OID is Proxy's +concern, not this crate's. + +## Known model gap + +The trait-bounds machinery here is complete — `satisfy_bounds` and `UnsatisfiedBounds` +exist and work — but Proxy currently feeds every encrypted column `EqlTraits::all()` +(`packages/cipherstash-proxy/src/proxy/schema/manager.rs:146`) rather than the traits its +configured SEM terms actually provide. So bound violations cannot be caught at type-check +time in production today. Treat `EqlTraits` on an `EqlValue` as *intended* capability, +not observed capability, until that join exists.