Skip to content

parser/lexer: 5.1 error-token quoting + pin break-outside-loop wording (#95, #105)#112

Merged
ianm199 merged 17 commits into
mainfrom
fix/95-105-lua51-error-token-quoting
Jun 2, 2026
Merged

parser/lexer: 5.1 error-token quoting + pin break-outside-loop wording (#95, #105)#112
ianm199 merged 17 commits into
mainfrom
fix/95-105-lua51-error-token-quoting

Conversation

@ianm199
Copy link
Copy Markdown
Owner

@ianm199 ianm199 commented Jun 2, 2026

Closes #105. Closes #95.

What

Two localized, version-gated parser/lexer wording fixes landed together (they overlap on Lua 5.1's break message).

#105 — 5.1 quotes special near/expected tokens

Lua 5.1's luaX_lexerror / error_expected wrap the whole near/expected token in LUA_QS ('%s') unconditionally, so the special multi-char labels (<eof>, <name>, <number>, <string>) come out quoted. 5.2 rewrote txtToken/luaX_token2str to leave those bare and quote only symbols/reserved/literals. lua-rs implemented the 5.2+ rule on every version, so all 5.1 syntax errors involving a special token diverged by exactly the surrounding quotes.

input (-e) before (5.1) after / ref 5.1.5
if unexpected symbol near <eof> unexpected symbol near '<eof>'
local <name> expected near <eof> '<name>' expected near '<eof>'
return 1 2 <eof> expected near '2' '<eof>' expected near '2'
break no loop to break near <eof> no loop to break near '<eof>'

Seam: snapshot the active LuaVersion onto LexState at lexer setup (the error formatters take only &LexState, so this avoids threading &LuaState through every syntax-error callsite), and quote the token >= TK_EOS arm of token2str_raw for 5.1. Covers both the near-token (lex_error/txt_token) and expected-token (error_expected/token2str) paths. The 5.2+ path is provably byte-identical (token < TK_EOS || version == V51 reduces to the old condition for non-5.1).

#95 — break-outside-loop wording

The wording was already correctly version-gated in breakstat/undef_goto. This PR adds regression tests pinning all five arms so a future refactor can't collapse them, and 5.1's arm picks up its quote from the #105 fix:

  • 5.1 (eager, post-consume): no loop to break near '<eof>'
  • 5.2 / 5.3 (deferred): <break> at line N not inside a loop
  • 5.4 (deferred): break outside loop at line N
  • 5.5 (eager): break outside loop near 'break'

Tests

  • 6 new cases in crates/lua-rs-runtime/tests/multiversion_oracle.rs covering all 5 versions (the differential oracle); the parser/lexer: 5.1 doesn't quote special near/expected tokens (<eof>, <name>, ...) #105 cases failed before the fix and pass after.
  • Full lua-rs-runtime + lua-lex + lua-parse crate suites green.
  • Official 5.4 errors / constructs / attrib / goto pass; CLI verified end-to-end across all five versions via LUA_RS_VERSION.

🤖 Generated with Claude Code

ianm199 and others added 17 commits June 1, 2026 20:24
#95, #105)

Lua 5.1's luaX_lexerror/error_expected wrap the whole near/expected token
in LUA_QS ('%s') unconditionally, so the special multi-char labels (<eof>,
<name>, <number>, <string>) come out quoted. 5.2 rewrote txtToken/
luaX_token2str to leave those bare and quote only symbols/reserved/literals.
lua-rs implemented the 5.2+ rule on every version, so all 5.1 syntax errors
involving a special token diverged by exactly the surrounding quotes.

Fix #105: snapshot the active LuaVersion onto LexState at lexer setup (the
error formatters take only &LexState, so this avoids threading &LuaState
through every syntax-error callsite) and quote the token >= TK_EOS arm of
token2str_raw for 5.1. This covers both the near-token (lex_error/txt_token)
and the expected-token (error_expected/token2str) paths. The 5.2+ path is
byte-identical (`token < TK_EOS || version == V51` reduces to the old
condition for non-5.1).

#95: the break-outside-loop wording is already version-gated in breakstat/
undef_goto; this adds regression tests pinning all five arms (5.1 eager
"no loop to break near '<eof>'", 5.2/5.3 "<break> at line N not inside a
loop", 5.4 "break outside loop at line N", 5.5 "break outside loop near
'break'") so a refactor can't collapse them. 5.1's arm also gains its quote
from the #105 fix.

Tests: 6 new multiversion_oracle cases (all 5 versions); full crate suite
green; official errors/constructs/attrib/goto pass on 5.4.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ianm199 ianm199 merged commit 71c2e97 into main Jun 2, 2026
3 checks passed
@ianm199 ianm199 deleted the fix/95-105-lua51-error-token-quoting branch June 2, 2026 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

parser/lexer: 5.1 doesn't quote special near/expected tokens (<eof>, <name>, ...) parser: version-gate 'break' outside loop error wording

1 participant