Release v1.3.0 — LOOKUP columns

CHANGELOG.md

@@ -7,6 +7,61 @@ Versions follow [Semantic Versioning](https://semver.org/) — minor bump per su
 
 ---
 
+## [1.3.0] — 2026-07-13
+
+### Added
+- `LOOKUP` column qualifier — language grammar and storage layer (#58). Any column can
+  declare a constraint on its legal values: `LOOKUP <table>.<column> [DISPLAY <column>]`
+  for a live table-driven lookup, or `LOOKUP ("a","b",...)` for a literal list. Parsed by
+  `CREATE TABLE`/`ALTER TABLE ADD`/`ALTER TABLE ALTER`, persisted per-column in
+  `ColumnMetaStore` via an additive migration (existing declared types are never touched
+  or dropped), and resolvable to concrete `{value,label}` options via the new
+  `src/interpreter/LookupResolver.ts` (degrades to free entry — never truncates — when the
+  source is missing, empty, or exceeds 1000 distinct values). This is a WebBase-III
+  extension with no dBASE III ancestor.
+- `LOOKUP` enforcement + BROWSE dropdown (#60). `REPLACE` and the BROWSE grid's `grid-edit`
+  now reject a value outside a column's declared `LOOKUP`, re-resolving the constraint fresh
+  against the live database on every write (so a value that only just became legal, or that
+  just stopped being legal, is judged correctly — never a stale cached list). An unresolvable
+  lookup (source table dropped, empty, or over 1000 values) degrades to free entry with a
+  warning rather than locking the column. BROWSE renders a lookup column as a dropdown —
+  `DISPLAY` labels shown while editing, the stored code shown once committed, matching
+  `LIST`/report output.
+- **Field-bound `@ SAY GET`** (#59). `@ r,c SAY "…" GET <name>` now binds directly to the
+  active table's column when one matches — dBASE III's actual behavior — instead of only
+  ever collecting into a memory variable. Fields take precedence over a memory variable of
+  the same name (why the `m_` prefix convention exists). A field-bound `GET` needs a current
+  record and prefills from it; a lookup column renders the same picker BROWSE does. `READ`'s
+  submit validates every field-bound value (declared type + lookup membership) before
+  writing any of them — a rejection sends a new `form-error` message that keeps the form
+  open with the bad fields outlined, rather than silently discarding the valid ones.
+  Writes target the row captured at `GET` time, mirroring how `grid-edit` already writes by
+  rowid. This is the PR that promotes `LOOKUP` to the README command reference in full —
+  both BROWSE and forms now declare, enforce, and render it end to end.
+- **Demos adopt `LOOKUP`** (#61). `demos/overtime.prg` gains a `SCHEDULES` catalog table
+  (`SCHEDID`, `DESCR`) as the lookup source for `EMPLOYEES.SCHEDID` — Add Employee is now a
+  check-first, two-form flow where the schedule is picked from a dropdown showing the
+  description ("Standard 40h (08:00-16:30)") instead of typed from memory. `demos/crm.prg`'s
+  `DEALS.STAGE` is constrained to a literal `LOOKUP` list matching its own seeded vocabulary
+  exactly, exercising the other lookup kind in a real, working demo.
+- **Assistant wizard support** (#62). The New-table and Modify-structure wizards gain a
+  per-column "lookup (optional)" field accepting `TABLE.COLUMN [DISPLAY COLUMN]` or a quoted
+  list, so declaring a `LOOKUP` no longer requires dropping into raw W3Script syntax. This
+  closes out the v1.3.0 lookup-columns milestone (#58–#62): the language, storage, resolver,
+  BROWSE/REPLACE enforcement, field-bound forms, two demo apps, and the GUI wizards all now
+  agree on one constraint, declared once, on the column.
+
+### Fixed
+- A memory-variable `@ SAY GET` no longer shows blank when the variable already holds a
+  value. Field-binding (#59) rewrote the non-field fallback path to hardcode an empty
+  prefill instead of reading the variable's current value, silently breaking every demo
+  form that pre-fills a default via `STORE`: `overtime.prg`'s Week Monday date (Open/Prep
+  Week, Recalculate Week) and leave date (Register Leave Taken), `INVENTORY.prg`'s stock/
+  reorder/price/quantity defaults (Add Product, Stock In, Stock Out), and `crm.prg`'s deal
+  value default (Add Deal). Found by exercising the running app, not by the test suite —
+  every existing test for this path used a variable with no prior value, where an empty
+  prefill happens to be correct either way.
+
 ## [1.2.0] — 2026-07-09 — TIME columns, WEEK()/DATEADD(), BROWSE cell validation, Overtime demo
 
 ### Added

CLAUDE.md

@@ -63,6 +63,9 @@ src/
     Executor.ts         Async AST runner; manages state (db/table/filter/vars/rowPtr/activeIndex). Emits fire-and-forget client side-effects (CSV download, report preview, CSV upload picker) via onSideEffect so they work inside program blocks
     IndexCommands.ts    Index command handlers (extracted from Executor)
     ReportCommands.ts   Report command handlers delegating to ReportRunner
+    LookupResolver.ts   Resolves a column's LOOKUP constraint (literal list, or live table+column+DISPLAY)
+                        to concrete {value,label} options against IDatabaseBridge; degrades to null
+                        (never truncates) on a missing source, empty result, or >1000 distinct values
 
   terminal/
     Terminal.ts         REPL UI — command history, multi-line block accumulation
@@ -194,6 +197,42 @@ WebBase-III supports **unlimited work areas** (no DOS 10-area limit). Cross-area
 
 Declared types are recorded per `(database, table, column)` in `server/ColumnMetaStore.ts`, because SQLite only keeps a storage affinity: `TIME`/`DATE`/`CHAR` are all `TEXT`, `LOGICAL`/`INT` are both `INTEGER`, and a `NUM(p,s)` qualifier is lost entirely.
 
+#### `LOOKUP` column qualifier (v1.3.0)
+
+Any column may add a `LOOKUP` clause after its type, constraining it to a set of legal
+values — a WebBase-III extension with no dBASE III ancestor (dBASE III+ only had
+`PICTURE "@M a,b,c"`, a literal spacebar-cycled list with no table-driven form):
+
+```
+SCHEDID CHAR(4) LOOKUP SCHEDULES.SCHEDID DISPLAY DESCR   -- live table lookup
+STAGE   CHAR(12) LOOKUP ("Lead","Won","Lost")             -- literal list
+```
+
+`CREATE TABLE`/`ALTER TABLE ADD`/`ALTER TABLE ALTER` parse and store it (`ColumnMetaStore`,
+same additive-migration discipline as the type columns above). `src/interpreter/LookupResolver.ts`
+turns a stored `LOOKUP` into concrete `{value,label}` options against the live database,
+degrading to `null` (never truncating) when the source is missing, empty, or exceeds 1000
+distinct values.
+
+BROWSE renders a lookup column as a dropdown (`Grid.ts`'s `startEdit`) fed by the options
+`Session.sendGridData` resolves at `grid-open`; the stored value shows once committed,
+matching `LIST`/report output — only the edit-mode dropdown shows `DISPLAY` labels.
+`REPLACE` and `grid-edit` both enforce membership, re-resolving fresh at write time (not
+trusting a client-held list) so a value that just became legal or just stopped being legal
+is judged correctly; an unresolvable lookup degrades to free entry with a warning rather
+than locking the column.
+
+`@ r,c SAY "…" GET <name>` binds to the active table's column when one matches — dBASE III's
+actual behavior, and why the field takes precedence over a memory variable of the same name
+(the `m_` prefix convention exists for this reason). A field-bound `GET` requires a current
+record (`APPEND RECORD` first — `** Error: GET <field>: no current record` otherwise),
+prefills from the record, and renders a picker when the column has a resolvable lookup.
+`READ`'s submit is all-or-nothing across every field-bound `GET` in the form: every value is
+validated (declared type + lookup membership) before any is written, and a rejection sends a
+`form-error` message that keeps the form open with the offending fields outlined — Escape
+still writes nothing. Writes target the rowid captured at `GET` time (`fetchCurrentRow`),
+mirroring `grid-edit`, so pointer motion between `GET` and submit can't retarget the write.
+
 #### Cell validation (`BROWSE`)
 
 `src/shared/cellValidation.ts` holds the rules and runs on **both** sides: `Grid.ts` checks before commit (an invalid edit keeps the cell in edit mode, outlined red, with the reason shown; the error clears as the value becomes valid), and `Session`'s `grid-edit` handler re-checks authoritatively before writing — a WS message can reach the server without passing through the grid.
@@ -207,7 +246,7 @@ Declared types are recorded per `(database, table, column)` in `server/ColumnMet
 | `LOGICAL` | `.T.`/`.F.`/`.TRUE.`/`.FALSE.`/`T`/`F`/`TRUE`/`FALSE`/`1`/`0` |
 | `CHAR` / `MEMO` | anything (length is not enforced) |
 
-An empty value is always allowed (clears the cell). Columns with no recorded declared type are unconstrained. `REPLACE` enforces only `TIME` — widening it would change the semantics of existing programs.
+An empty value is always allowed (clears the cell). Columns with no recorded declared type are unconstrained. `REPLACE` enforces `TIME` and, since v1.3.0, lookup membership on any column that declares one (additive — no pre-v1.3.0 column declares a `LOOKUP`, so no existing program's behavior changes); widening validation beyond that would change the semantics of existing programs.
 
 ### Indexing & search
 | Command | What it does |
@@ -339,7 +378,7 @@ npm run coverage        # Vitest + v8 coverage report (reporting only, no thresh
 npx playwright test     # E2E browser tests — requires dev server on :5173/:3000
 ```
 
-Playwright suites (94 tests): `tests/assistant.spec.ts` (23 tests — sidebar, wizards, report designer, MODIFY STRUCTURE round-trip, `TIME(15)` column + REPLACE validation, `NUM(p,s)` wizard, Browse-action grid validation, program run, CSV/SORT/SUM-AVERAGE/REINDEX/PACK actions, demo launchers), `tests/integration.spec.ts` (20 tests — full REPL scenario), `tests/overtime.spec.ts` (9 tests — overtime.prg: menu, seeding, DATEADD week prep, TIME(15) grid rejection, recalculation, live balance, quarter-hour leave check, report, CSV), `tests/inventory.spec.ts` (8 tests — INVENTORY.prg menu + valuation/low-stock report/sort/CSV/JOIN), `tests/crm.spec.ts` (6 tests — CRM demo menu, pipeline summary, sort, report, CSV, JOIN), `tests/parity-commands.spec.ts` (6 tests — `?`/`??`, built-in functions, `WEEK()`, `DATEADD()`, `SUM`/`AVERAGE`, `SORT ON … TO`), `tests/multiarea.spec.ts` (4 tests — multi-work-area, relations, alias.field), `tests/demos.spec.ts` (4 tests — demo program + report seeding), `tests/grid-validation.spec.ts` (3 tests — BROWSE per-cell validation: TIME(15), NUM(p,s)/DATE, Esc abandons), `tests/schema-errors.spec.ts` (3 tests — malformed CREATE TABLE errors, NUM(p,s) column count, bare INPUT stores its value), `tests/copycsv.spec.ts` (2 tests — COPY TO download + APPEND FROM upload), `tests/splash.spec.ts` (2 tests — version banner + demo discoverability), `tests/join.spec.ts` (1 test — JOIN materialization), `tests/propagation.spec.ts` (1 test — live multiuser refresh), `tests/program-side-effects.spec.ts` (1 test — CSV/report side-effects fire from inside a program block).
+Playwright suites (95 tests): `tests/assistant.spec.ts` (23 tests — sidebar, wizards, report designer, MODIFY STRUCTURE round-trip, `TIME(15)` column + REPLACE validation, `NUM(p,s)` wizard, Browse-action grid validation, program run, CSV/SORT/SUM-AVERAGE/REINDEX/PACK actions, demo launchers), `tests/integration.spec.ts` (20 tests — full REPL scenario), `tests/overtime.spec.ts` (10 tests — overtime.prg: menu, seeding, DATEADD week prep, TIME(15) grid rejection, recalculation, live balance, quarter-hour leave check, report, CSV, Add-Employee lookup dropdown), `tests/inventory.spec.ts` (8 tests — INVENTORY.prg menu + valuation/low-stock report/sort/CSV/JOIN), `tests/crm.spec.ts` (6 tests — CRM demo menu, pipeline summary, sort, report, CSV, JOIN), `tests/parity-commands.spec.ts` (6 tests — `?`/`??`, built-in functions, `WEEK()`, `DATEADD()`, `SUM`/`AVERAGE`, `SORT ON … TO`), `tests/multiarea.spec.ts` (4 tests — multi-work-area, relations, alias.field), `tests/demos.spec.ts` (4 tests — demo program + report seeding), `tests/grid-validation.spec.ts` (3 tests — BROWSE per-cell validation: TIME(15), NUM(p,s)/DATE, Esc abandons), `tests/schema-errors.spec.ts` (3 tests — malformed CREATE TABLE errors, NUM(p,s) column count, bare INPUT stores its value), `tests/copycsv.spec.ts` (2 tests — COPY TO download + APPEND FROM upload), `tests/splash.spec.ts` (2 tests — version banner + demo discoverability), `tests/join.spec.ts` (1 test — JOIN materialization), `tests/propagation.spec.ts` (1 test — live multiuser refresh), `tests/program-side-effects.spec.ts` (1 test — CSV/report side-effects fire from inside a program block).
 
 ## Test discipline
 

README.md

@@ -2,7 +2,7 @@
 
 **dBASE III is back. In your browser. `USE customers` like it's 1984.**
 
-![WebBase-III demo — USE, LIST, SEEK, BROWSE](docs/screenshots/demo.gif)
+![WebBase-III demo — USE, LIST, SEEK, BROWSE, and a LOOKUP column dropdown](docs/screenshots/demo.gif)
 
 Remember the dot prompt? Before SQL won, before ORMs, before anyone said "full-stack" — there was dBASE III. You typed `USE customers`, then `LIST`, and your data was just *there*. WebBase-III brings that whole world back: the terminal, the language, `BROWSE`, `@ SAY GET` forms, `.prg` programs, indexes, reports — rebuilt from scratch as a modern web app with its own interpreter in TypeScript, backed by Node.js, WebSockets, and SQLite.
 
@@ -98,6 +98,20 @@ as you fix it, and `Esc` abandons the edit. Here a `TIME(15)` column rejects `08
 
 ---
 
+### `LOOKUP` columns — pick a value instead of typing it
+
+A column declared `LOOKUP <table>.<column> DISPLAY <column>` edits through a dropdown in
+both BROWSE and forms — showing the friendlier label, storing the code. Here `SCHEDID`
+is constrained to `SCHEDULES.SCHEDID`, displaying `DESCR`.
+
+![BROWSE editing a LOOKUP column via dropdown](docs/screenshots/screenshot-lookup-browse.png)
+
+A field-bound `@ SAY GET` (one whose name matches a table column) renders the same picker:
+
+![A form field bound to a LOOKUP column](docs/screenshots/screenshot-lookup-form.png)
+
+---
+
 ### Aggregate commands & dBASE III parity
 
 `SUM`, `AVERAGE`, `? ROUND(…)`, `? MAX(…)`, and `SORT ON … TO` — numeric aggregates and sorted copies, honouring the active filter.
@@ -140,6 +154,8 @@ build indexes, search, reindex, pack the database, design and run reports, run p
 table structure. Every click generates a real W3Script command that echoes into the terminal —
 watch it to learn the language. Wizards (New table, Filter, Sort, Sum/Average, Modify structure,
 report designer, …) open in the main area and show a live preview of the command they will run.
+New table and Modify structure both offer an optional per-column "lookup" field — `LOOKUP`
+without typing raw syntax.
 
 ---
 
@@ -244,6 +260,43 @@ WebBase-III supports **unlimited work areas** — each independently holding a t
 
 **Column types**: `CHAR(n)` (aliases `CHARACTER`/`VARCHAR`/`STRING`/`MEMO`), `NUM`/`NUM(p,s)` (`NUMERIC`/`FLOAT`/`DOUBLE`/`DECIMAL`), `INT`/`INTEGER`, `LOGICAL`/`BOOLEAN`, `DATE`, and `TIME`/`TIME(n)`. `TIME` stores `HH:MM` (24-hour); the optional `TIME(n)` qualifier (e.g. `TIME(15)`) requires minutes to be a multiple of `n`. `REPLACE ... WITH` rejects a malformed or off-granularity `TIME` value instead of silently coercing it, and `LIST STRUCTURE` prints the declared type (`NUM(8,2)`, `TIME(15)`) rather than SQLite's storage class.
 
+### `LOOKUP` columns — constrain a column to legal values
+
+Any column may add a `LOOKUP` clause after its type, declaring the values it's allowed to hold:
+
+```
+CREATE TABLE EMPLOYEES (EMPID CHAR(4), NAME CHAR(30),
+                        SCHEDID CHAR(4) LOOKUP SCHEDULES.SCHEDID DISPLAY DESCR)
+CREATE TABLE DEALS (STAGE CHAR(12) LOOKUP ("Lead","Qualified","Proposal","Won","Lost"))
+```
+
+Two forms: `LOOKUP <table>.<column> [DISPLAY <column>]` looks up live values from another
+table (optionally showing a friendlier label while storing the code), or
+`LOOKUP ("a","b",...)` is a fixed list. `CREATE TABLE`/`ALTER TABLE ADD`/`ALTER TABLE ALTER`
+all accept it.
+
+BROWSE renders a lookup column as a dropdown — `DISPLAY` labels while editing, the stored
+code once committed, matching `LIST`/report output. `REPLACE` and BROWSE edits both reject
+a value outside the list, re-checked fresh against the live database on every write (a
+value that just became legal, or just stopped being legal, is judged correctly — never a
+stale cached list). A lookup that can't be resolved (source table dropped, empty, or over
+1000 distinct values) degrades to free entry with a warning instead of locking the column.
+
+Forms pick it up too: `@ r,c SAY "…" GET <name>` binds to the active table's column when
+one matches — a field takes precedence over a memory variable of the same name, which is
+why programs use an `m_` prefix for scratch variables. A field-bound `GET` needs a current
+record (`APPEND RECORD` first) and prefills from it; if the column has a lookup, the form
+renders the same picker BROWSE does. `READ` validates every field-bound value before
+writing any of them — a rejection keeps the form open with the bad field outlined instead
+of silently dropping the others.
+
+> **Deviation from dBASE III:** `LOOKUP` has no dBASE III ancestor — dBASE III+ only offered
+> `PICTURE "@M a,b,c"`, a literal value list cycled with the spacebar, with no table-driven
+> form and no display label. This is a WebBase-III extension, in the same spirit as
+> unlimited work areas and `alias.field` dot notation. Field-bound `GET` *is* authentic
+> dBASE III behavior — WebBase-III's own memory-variable-only forms were the deviation,
+> now corrected.
+
 > **CSV format (`COPY TO` / `APPEND FROM`):** Unlike dBASE III's headerless,
 > positional `DELIMITED`/`SDF` formats, WebBase-III uses modern **header-based CSV**
 > (RFC-4180, mapped by column name). Export downloads through the browser and
@@ -318,7 +371,7 @@ WebBase-III supports **unlimited work areas** — each independently holding a t
 | `? <expr>[, <expr>...]` | Evaluate expression(s) and print the result (numbers right-justified; bare `?` prints a blank line; `??` also accepted) |
 | `STORE <val> TO <var>` | Assign a variable |
 | `INPUT "prompt" TO <var>` | Collect keyboard input |
-| `@ r,c SAY "text" GET <var>` | Define a form field |
+| `@ r,c SAY "text" GET <var>` | Define a form field; a name matching a column of the active table binds that column (lookup columns render a picker) |
 | `READ` | Display the form and wait for submit |
 
 ### Control flow

demos/crm.prg

@@ -45,7 +45,7 @@ IF RECCOUNT() == 0
 
   SELECT DEAL
   DROP TABLE DEALS
-  CREATE TABLE DEALS (DEALID CHAR(6), COMPID CHAR(5), TITLE CHAR(40), STAGE CHAR(12), VALUE NUM(12,2), CLOSEMONTH NUM(6))
+  CREATE TABLE DEALS (DEALID CHAR(6), COMPID CHAR(5), TITLE CHAR(40), STAGE CHAR(12) LOOKUP ("Lead","Qualified","Proposal","Won","Lost"), VALUE NUM(12,2), CLOSEMONTH NUM(6))
   INDEX ON DEALID TO BYDEAL
   INDEX ON COMPID TO DEALCOMP
   APPEND RECORD