Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- **AccuracySNES: the Mesen2 oracle diagnosed, and a stale frame budget aligned.** Three `v1.28.0`
items ended at "no oracle can arbitrate", so the headless runner was investigated rather than
accepted as environmental. Established: the hang is genuinely pre-existing (the `v1.25.0`-era image
times out identically); it is **not** the frame budget (the battery needs ~431 frames and 4000
still times out); the Lua never sees `DONE = 0xA5` and the results magic never reads as `ACSN`
under either `snesWorkRam`+offset (`$A1`) or `snesMemory`+full address (`$52`), so the failure is
in the **Lua-to-emulator memory bridge or the battery not executing**, not a timeout — exit code
254 has been misreporting the cause. `emu.log` does not surface in `--testrunner` mode, so
diagnosis has to go out through `emu.stop(value)`.

Not fixed. `docs/accuracysnes-plan.md` records the next step and says plainly not to spend further
effort on rows needing an arbiter until it resolves.

Also settled there: the instrument's fixed cost, **measured at 175 dots** (`measure_begin`
immediately followed by `measure_end`), with 8 `NOP`s costing exactly 28 dots marginal — so its
marginal accuracy is exact and only the intercept is large. `A5.19`'s 189-dot intercept *is* the
instrument, and the `v1.28.0` assessment was right. The corrected span budget is `341 - 175 = 166`
dots, which moves a `BRK` round trip from 3 iterations to 4; `A5.18` stays parked on that
arithmetic rather than on a disputed figure.

Separately real: `mesen_crossval.lua`'s `MAX_FRAMES` was **900** against the harness's 1500,
`mesen_scenes.lua`'s 4000 and `libretro_crossval.c`'s 2000. The harness comment naming the budgets
that must move together omitted this file, which is how it was missed. Aligned to 4000 and the
comment corrected — a latent inconsistency, not the cause of the hang.

- **AccuracySNES: `C7.06` — Time Over reads set by `V = OBJ.YLOC + 1, H = 0`.** `C7` on-cart
coverage 7 → 8 of 16.

Expand Down
7 changes: 4 additions & 3 deletions crates/rustysnes-test-harness/tests/accuracysnes.rs
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,10 @@ const FORMAT_VERSION: u16 = 1;
/// Raised from 600 when the cartridge image grew to 256 KiB. Most of the battery's frames are one
/// test: `G1.11` walks the entire cartridge byte by byte to check the header checksum, so doubling
/// the image doubled it — about 320 of the current 431 frames. The margin at 600 was 169 frames and
/// is now comfortable again. **When this needs raising, check `mesen_scenes.lua`'s `MAX_FRAMES` and
/// `libretro_crossval.c`'s `max_frames` in the same change**: all three bound the same run, and the
/// Mesen2 one silently reports "no scenes" rather than "timed out" from the gate's point of view.
/// is now comfortable again. **When this needs raising, check `MAX_FRAMES` in `mesen_scenes.lua`
/// and `mesen_crossval.lua`, and `libretro_crossval.c`'s `max_frames`, in the same change**: all
/// four bound the same run, and the Mesen2 ones silently report "no scenes" rather than "timed
/// out" from the gate's point of view.
const MAX_FRAMES: u32 = 1500;

/// Minimum share of scoring tests that must pass.
Expand Down
51 changes: 51 additions & 0 deletions docs/accuracysnes-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -455,6 +455,57 @@ So nothing is asserted and nothing is recorded. What is known:
The next attempt should start from a scratch build that dumps `$213F` before and after each step
into slots verified unused, rather than from a folded variant that hides where the discrepancy is.

### The instrument's fixed cost is ~175 dots — measured (`v1.29.0`)

**Measured, after an earlier revision of this section wrongly called the figure into doubt.** The
empty span — `measure_begin` immediately followed by `measure_end`, nothing between — is **175
dots**. A control in the same probe put 8 `NOP`s at exactly **28 dots** marginal (8 x 14 clocks / 4),
so the instrument's *marginal* accuracy is exact; only its fixed cost is large.

That settles it: `A5.19`'s 189-dot intercept **is** the instrument (175, plus ~14 for the two no-op
`clc`/`xce` pairs inside its span). The `v1.28.0` assessment was right the first time.

The doubt came from reading `hv_begin`/`hv_end` and estimating "only ~30 dots should be inside the
delta". That estimate was simply bad arithmetic: the pair runs ~70-90 CPU *cycles* between the two
latches (`sta f:`, `php`/`plp`, `pha`/`pla`, two `jsr`/`rts` pairs, and `hv_read_raw`'s eleven
instructions), and a cycle is 6-8 master clocks — **1.5-2 dots each**, not the ~0.4 the estimate
implied. Counting instructions instead of cycles is what produced the discrepancy.

**The corrected span budget is `341 - 175 = 166` dots**, slightly more generous than the 152 the
`A5.18` note assumed. That moves a `BRK` round trip (39 dots) from 3 iterations to **4**, giving 8
dots of signal against `TOL` 2 — better than the 6 that parked it, still a difference-of-differences
accumulating quantisation from four measurements, and still well short of `A5.19`'s 16. `A5.18`
stays parked, on corrected arithmetic rather than a wrong figure; the honest verdict is *borderline*,
not *impossible*.
Comment on lines +474 to +479

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Update the remaining A5.18 arithmetic.

This section changes the budget to 166 dots and supports four iterations. The same plan still states:

  • Line 513-515: approximately 189 dots of overhead and 152 usable dots.
  • Line 520: only three iterations and six dots of signal.
  • Line 541-544: the same obsolete rationale.

Replace those values with the corrected assessment, or remove the obsolete explanation. Otherwise, the plan gives two incompatible A5.18 designs.

As per path instructions, docs are the spec, not a history log.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/accuracysnes-plan.md` around lines 474 - 479, Update the remaining A5.18
references to consistently use the corrected 166-dot budget: replace the
obsolete 189-dot overhead, 152 usable dots, three iterations, and six-dot signal
in the surrounding assessment, and revise or remove the outdated rationale near
the final A5.18 conclusion. Ensure docs/accuracysnes-plan.md presents only the
corrected four-iteration, eight-dot assessment.

Source: Path instructions


### The Mesen2 oracle — diagnosed, not yet fixed (`v1.29.0`)

Three `v1.28.0` items ended at "no oracle can arbitrate": `C7.07`'s errata, the scene goldens, and
the *dot* half of `C7.05`. So the Mesen2 headless runner was investigated rather than accepted as
"environmental". What is now established:

- **The hang is genuinely pre-existing.** The `v1.25.0`-era cartridge image, restored from git and
run unmodified, times out identically. Nothing in the `v1.28.0` rows caused it.
- **It is not the frame budget.** The battery needs ~431 frames on RustySNES (`MAX_FRAMES` 1500) and
snes9x is given 2000. Mesen2 still times out at **4000**.
- **`mesen_crossval.lua`'s `MAX_FRAMES = 900` is nonetheless stale** — the harness comment naming the
three budgets that must move together lists `mesen_scenes.lua` (4000) and `libretro_crossval.c`,
but not `mesen_crossval.lua`, which is why it was missed. Worth fixing regardless; it is not the
cause.
- **The Lua never observes `DONE = 0xA5`, and the results magic never reads as `ACSN`** — under
either `emu.memType.snesWorkRam` with a `$F000` offset (reads `$A1`) or `emu.memType.snesMemory`
with the full `$7EF000` address (reads `$52`). Neither is the expected `$41`. So the failure is in
the **Lua-to-emulator memory bridge, or the battery not executing**, and *not* a timeout — the
exit code 254 has been misreporting the cause all along.
- **`emu.log` does not surface in `--testrunner` mode.** Diagnosis has to be smuggled out through the
process exit code (`emu.stop(value)`), which is how the two byte readings above were obtained.

**Next step:** determine which of the two remaining explanations holds, by exiting with a byte the
runtime writes *early* and whose value is known, then walking the address until it reads back. If the
bridge is the problem the fix is a one-line `memType`/offset change; if the battery genuinely does
not execute, that is a Mesen2 configuration question and a much larger one. Do not spend further
effort on rows that need an arbiter until this resolves.

### v1.28.0 — what each remaining row actually needs, measured rather than assumed

`A5.19` landed (below). Working the rest turned up that several rows filed as "needs no new
Expand Down
7 changes: 6 additions & 1 deletion scripts/accuracysnes/mesen_crossval.lua
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,12 @@ local COUNT = BASE + 0x06
local DONE = BASE + 0x08
local STATUS = BASE + 0x20

local MAX_FRAMES = 900
-- Bounds the same run as the in-repo harness's MAX_FRAMES (1500), mesen_scenes.lua's (4000) and
-- libretro_crossval.c's max_frames (2000). This one was left at 900 when the others grew -- the
-- harness comment naming the budgets that must move together does not list this file, which is how
-- it was missed. It is NOT why this runner times out (4000 fails identically); see
-- docs/accuracysnes-plan.md on the Mesen2 oracle.
Comment on lines +24 to +28

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail
rg -n -C 5 'libretro_crossval\.c|libretro_crossval|max_frames|MAX_FRAMES|mesen_crossval\.lua' .

Repository: doublegate/RustySNES

Length of output: 50378


🏁 Script executed:

#!/bin/bash
set -euo pipefail
printf '%s\n' '--- mesen_crossval.lua ---'
sed -n '20,32p' scripts/accuracysnes/mesen_crossval.lua
printf '%s\n' '--- accuracysnes-plan.md ---'
sed -n '487,495p' docs/accuracysnes-plan.md
printf '%s\n' '--- CHANGELOG.md ---'
sed -n '32,39p' CHANGELOG.md
printf '%s\n' '--- crossval.sh ---'
sed -n '168,176p' scripts/accuracysnes/crossval.sh
printf '%s\n' '--- libretro_crossval.c ---'
sed -n '217,241p' scripts/accuracysnes/libretro_crossval.c

Repository: doublegate/RustySNES

Length of output: 3446


Describe 2000 as the explicit invocation budget.

scripts/accuracysnes/crossval.sh passes 2000; libretro_crossval.c defaults to 2400 when no argument is provided. Update the mesen_crossval.lua and CHANGELOG.md descriptions to distinguish these values.

📍 Affects 3 files
  • scripts/accuracysnes/mesen_crossval.lua#L24-L28 (this comment)
  • docs/accuracysnes-plan.md#L487-L494
  • CHANGELOG.md#L34-L37
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/accuracysnes/mesen_crossval.lua` around lines 24 - 28, Update the
descriptions in scripts/accuracysnes/mesen_crossval.lua (lines 24-28),
docs/accuracysnes-plan.md (lines 487-494), and CHANGELOG.md (lines 34-37) to
distinguish the explicit 2000-frame invocation budget passed by
scripts/accuracysnes/crossval.sh from libretro_crossval.c’s 2400-frame default
when no argument is provided; preserve the existing budget relationships and do
not describe 2000 as the default.

local MAX_FRAMES = 4000
Comment on lines +24 to +29

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🔵 Trivial

Raising MAX_FRAMES to 4000 quadruples the wait time for a run already known to hang for the full duration.

docs/accuracysnes-plan.md documents that Mesen2 still times out at 4000 frames. Until the underlying hang is fixed, every invocation of this runner now waits over 4x longer than before (900 → 4000 frames) before failing, with no additional diagnostic value from the extra frames.

Consider gating or skipping this Mesen2 cross-validation step in CI until the root cause is resolved, rather than only raising the budget it hangs at. If it already runs unattended in CI, this is worth confirming so the wall-clock cost is intentional.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/accuracysnes/mesen_crossval.lua` around lines 24 - 29, Revert the
MAX_FRAMES increase in the Mesen2 cross-validation runner and prevent this step
from running unattended in CI while the documented hang remains unresolved.
Update the CI gating or skip logic associated with mesen_crossval.lua,
preserving an explicit way to run it manually for investigation.

Source: Path instructions

local frames = 0

local function rd(a)
Expand Down
Loading