[Task] Phase 1 — MVP formula engine

[YousefHadder]

Scope

Ship the minimum useful formula engine end-to-end. Goal: a user can write a TBLFM comment under a table, hit <localleader>tF=, and see values populate correctly. Opt-in via table.formulas.enabled = true.

This scope was tightened after a design-review pass that surfaced a latent parser bug, ambiguity in the formula clause separator, and unsound fixed-point semantics. See "Design review revisions" below.

In scope

• Parser fix (latent bug): tighten lua/markdown-plus/table/parser.lua is_table_row to exclude HTML comments. Without this, the new TBLFM comment line would be slurped into the table and corrupted by format.format_table().
• TBLFM annotation parser/writer: read <!-- TBLFM: ... --> immediately after a table (first non-empty line at matching indentation; multi-line comments rejected in Phase 1). Smart clause splitter on top-level | only (respects parens, strings if we ever add them).
• Lexer + Pratt parser for expression grammar.
• Reference resolver:
  • $N positional (1-indexed)
  • $Name header-name, normalized via lower() + non-alphanumerics collapsed to _. Duplicate normalized headers produce #REF! for that reference with a warning listing the colliding headers. Empty/punctuation-only headers are positionally addressable only.
  • @N row (org-mode: @1 header, @2 first data row; separator unaddressable)
  • @N$M cell
  • Vertical ranges @a$c..@b$c and horizontal ranges $colA..$colB. Ranges that include @1 return #RANGE!. Reversed ranges normalized when axis is obvious, else #RANGE!. Single-cell ranges valid. Empty ranges: vsum/vcount/vcounta return 0; vmean/vmin/vmax return #ERROR.
• Conservative numeric coercion: trim whitespace; accept 123, 123.45, .5, 1.2e5; accept comma thousands when grouped correctly (1,200, 12,345); reject malformed (12,34). No currency, no percent. Document the rule.
• Function library (numeric-only MVP): vsum, vmean, vmin, vmax, vcount (numeric only), vcounta (non-empty), round, floor, ceil, abs, if(cond, a, b). Constants pi, e.
• Operators: arithmetic + - * / % ^, comparison < <= > >= == != (return 1/0).
• Target-cell precomputation: before evaluation, compile annotations into a target_cell -> formula map. Column formulas expand to per-row targets; specific-cell formulas overwrite that map. Duplicate ownerships emit a warning.
• Dependency-graph evaluation: build dep graph from compiled targets, topologically sort, evaluate in order. Strongly-connected components (true cycles) produce #CYCLE! on participating cells. No max_iterations knob.
• Error sentinels (in-cell, round-trip safe): #REF! (unknown ref / header collision), #DIV0!, #CYCLE!, #RANGE! (range crosses header / reversed / invalid), #ERROR (generic).
• Operations guards (deferred-to-Phase-2 work pulled forward): when the table has a TBLFM annotation, sort/transpose/insert/delete row/insert/delete column/move row/move column prompt the user (default No) before proceeding. Without this, those ops would silently corrupt formula references.
• On-demand recalc command + <Plug>(MarkdownPlusTableFormulaRecalc) + default <localleader>tF=. All formula features sit under the <localleader>tF (capital F) sub-prefix to avoid colliding with the existing <localleader>tf (TableFormat) default.
• Config wiring: table.formulas.{enabled}. Default enabled = false. Update types, validator, README, vimdoc, tests (the six-file rule).
• Tests: separate spec files per concern. Mirror the footnotes/ test split:
  • table_formula_lexer_spec.lua
  • table_formula_parser_spec.lua
  • table_formula_refs_spec.lua
  • table_formula_functions_spec.lua
  • table_formula_evaluator_spec.lua
  • table_formula_annotation_spec.lua
  • table_formula_coercion_spec.lua
  • table_formula_dependency_spec.lua
  • table_formula_operations_guard_spec.lua
  • table_formula_spec.lua — end-to-end integration
• Also extend table_spec.lua to verify the new is_table_row HTML-comment exclusion doesn't regress existing tables.

Out of scope (deferred to Phase 2+)

• String values + string comparison (== on text cells)
• Logical operators &&, ||, ! (would conflict with | clause separator)
• nan constant
• Format directives (;%.2f)
• Extmark indicators on formula cells
• auto_recalc = "save"
• Numeric-ref auto-rewrite on row/column ops (Phase 1 only guards, doesn't rewrite)
• Quickfix list population
• Float-window formula editor
• Interactive formula authoring command
• Currency / percent / locale-aware coercion

Design review revisions (locked)

These resolve specific risks identified in the design-review pass. They supersede the looser language in the original design plan.

1. Parser fix is a prerequisite, not optional. Without HTML-comment exclusion in is_table_row, TBLFM comments would be parsed as table data.
2. Clause separator is |. Logical OR/AND/NOT are deferred to avoid a smart-splitter requirement in Phase 1.
3. Evaluation is dependency-graph with topo sort + SCC detection, not fixed-point iteration. No max_iterations.
4. Target-cell precomputation before evaluation. Specific-cell formulas don't compete with column formulas at evaluation time.
5. Duplicate normalized headers → #REF! with warning, not last-wins.
6. Annotation locality: first non-empty line after the table at matching indentation. Single occurrence. Multi-line rejected.
7. Existing destructive table operations must guard when TBLFM is present. Pulled into Phase 1 from Phase 2.

Acceptance criteria

• Existing table behavior unchanged when table.formulas.enabled = false (the default).
• HTML comments inside or immediately after tables no longer corrupt parsing (verified by added test).
• Scenarios 1, 2, 3, 5, 6 from the design doc work end-to-end (invoice, sprint, grades, A/B matrix, habit tracker).
• Sorting/transposing/inserting/deleting on a formula-bearing table prompts before proceeding.
• All listed spec files exist and pass.
• make check passes.
• README and vimdoc include a "Table Formulas" section with at least one worked example and the locked semantics (numeric-only, header normalization rule, error sentinels).