Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -430,20 +430,20 @@ All extracted Player modules follow a standardized naming pattern where the modu
| player_core | `PlayerCore_` | `PlayerCore_buildGameInfo()`, `PlayerCore_processAVInfo()` |
| player_menu | `PlayerMenu_` | `PlayerMenu_init()`, `PlayerMenuNav_navigate()` |
| player_env | `PlayerEnv_` | `PlayerEnv_setRotation()`, `PlayerEnv_handleGeometry()` |
| player_cpu | `PlayerCPU_` | `PlayerCPU_update()`, `PlayerCPU_detectFrequencies()` |
| cpu | `CPU_` | `CPU_update()`, `CPU_detectFrequencies()` |
| player_game | `PlayerGame_` | `PlayerGame_parseExtensions()`, `PlayerGame_detectM3uPath()` |
| player_scaler | `PlayerScaler_` | `PlayerScaler_calculate()` |

**Type naming:** Types follow the same pattern with `Player[Module]TypeName`:

- `PlayerCPUState`, `PlayerCPUConfig`, `PlayerCPUDecision`
- `CPUState`, `CPUConfig`, `CPUDecision`
- `PlayerOption`, `PlayerOptionList`
- `PlayerMemoryResult`, `PlayerStateResult`

**Constants:** Module-specific constants use `PLAYER_MODULE_` prefix:

- `PLAYER_CPU_MAX_FREQUENCIES`
- `PLAYER_CPU_DEFAULT_WINDOW_FRAMES`
- `CPU_MAX_FREQUENCIES`
- `CPU_DEFAULT_WINDOW_FRAMES`
- `PLAYER_MEM_OK`, `PLAYER_STATE_OK`

This standardization makes it immediately clear which module owns each function and prevents naming collisions as the codebase grows.
Expand Down Expand Up @@ -517,7 +517,7 @@ See `.clang-format` for complete style definition.
| Player core AV processing | `workspace/all/player/player_core.c` |
| Player memory persistence | `workspace/all/player/player_memory.c` |
| Player save states | `workspace/all/player/player_state.c` |
| Player CPU scaling | `workspace/all/player/player_cpu.c` |
| CPU scaling | `workspace/all/common/cpu.c` |
| Player input handling | `workspace/all/player/player_input.c` |
| Player save paths | `workspace/all/player/player_paths.c` |
| Launcher Entry type | `workspace/all/launcher/launcher_entry.c` |
Expand Down Expand Up @@ -569,7 +569,7 @@ To enable comprehensive testing, complex logic has been extracted from large fil
| player_scaler.c | 26 | player.c | Video scaling geometry calculations |
| player_core.c | 23 | player.c | Core AV info processing, aspect ratio calculation |
| effect_system.c | 43 | platform files | Visual effect state management |
| player_cpu.c | 42 | player.c | Auto CPU scaling algorithm |
| cpu.c | 42 | player.c | CPU topology + auto scaling algorithm |
| player_utils.c | 41 | player.c | Core name extraction, string utilities |
| player_menu.c | 41 | player.c | In-game menu, context pattern validation |
| nointro_parser.c | 39 | (original) | No-Intro ROM naming conventions |
Expand Down
15 changes: 8 additions & 7 deletions Makefile.qa
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,7 @@ TEST_UNITY = tests/vendor/unity/unity.c
PATHS_STUB = tests/support/paths_stub.c

# All test executables (built from tests/unit/ and tests/integration/)
TEST_EXECUTABLES = tests/utils_test tests/nointro_parser_test tests/pad_test tests/gfx_text_test tests/audio_resampler_test tests/player_paths_test tests/launcher_utils_test tests/m3u_parser_test tests/launcher_file_utils_test tests/map_parser_test tests/collection_parser_test tests/recent_parser_test tests/recent_writer_test tests/recent_runtime_test tests/directory_utils_test tests/binary_file_utils_test tests/ui_layout_test tests/str_compare_test tests/effect_system_test tests/effect_generate_test tests/player_utils_test tests/player_config_test tests/player_options_test tests/platform_variant_test tests/launcher_entry_test tests/directory_index_test tests/player_archive_test tests/player_memory_test tests/player_state_test tests/launcher_launcher_test tests/player_cpu_test tests/player_input_test tests/launcher_state_test tests/player_menu_test tests/player_env_test tests/player_game_test tests/player_scaler_test tests/player_core_test tests/launcher_directory_test tests/launcher_navigation_test tests/launcher_thumbnail_test tests/launcher_context_test tests/emu_cache_test tests/res_cache_test tests/render_common_test tests/integration_workflows_test tests/log_test tests/frame_pacer_test
TEST_EXECUTABLES = tests/utils_test tests/nointro_parser_test tests/pad_test tests/gfx_text_test tests/audio_resampler_test tests/player_paths_test tests/launcher_utils_test tests/m3u_parser_test tests/launcher_file_utils_test tests/map_parser_test tests/collection_parser_test tests/recent_parser_test tests/recent_writer_test tests/recent_runtime_test tests/directory_utils_test tests/binary_file_utils_test tests/ui_layout_test tests/str_compare_test tests/effect_system_test tests/effect_generate_test tests/player_utils_test tests/player_config_test tests/player_options_test tests/platform_variant_test tests/launcher_entry_test tests/directory_index_test tests/player_archive_test tests/player_memory_test tests/player_state_test tests/launcher_launcher_test tests/cpu_test tests/player_input_test tests/launcher_state_test tests/player_menu_test tests/player_env_test tests/player_game_test tests/player_scaler_test tests/player_core_test tests/launcher_directory_test tests/launcher_navigation_test tests/launcher_thumbnail_test tests/launcher_context_test tests/emu_cache_test tests/res_cache_test tests/render_common_test tests/integration_workflows_test tests/log_test tests/sync_manager_test

# Default targets: use Docker for consistency
test: docker-test
Expand Down Expand Up @@ -408,14 +408,15 @@ tests/launcher_launcher_test: tests/unit/all/launcher/test_launcher_launcher.c w
@echo "Building launcher command tests..."
@$(CC) -o $@ $^ $(TEST_INCLUDES) $(TEST_CFLAGS)

# Build auto CPU scaling tests (pure algorithm, no external dependencies)
tests/player_cpu_test: tests/unit/all/player/test_player_cpu.c workspace/all/player/player_cpu.c $(TEST_UNITY)
@echo "Building auto CPU scaling tests..."
# Build CPU scaling tests (pure algorithm, no external dependencies)
tests/cpu_test: tests/unit/all/common/test_cpu.c workspace/all/common/cpu.c $(TEST_UNITY)
@echo "Building CPU scaling tests..."
@$(CC) -o $@ $^ $(TEST_INCLUDES) $(TEST_CFLAGS)

# Build frame pacing tests (pure algorithm, no external dependencies)
tests/frame_pacer_test: tests/unit/all/player/test_frame_pacer.c workspace/all/player/frame_pacer.c $(TEST_UNITY)
@echo "Building frame pacer tests..."
# Build sync manager tests (vsync measurement and mode switching)
# Note: Uses test stub for getMicroseconds, not utils.c version
tests/sync_manager_test: tests/unit/all/player/test_sync_manager.c workspace/all/player/sync_manager.c workspace/all/common/log.c $(TEST_UNITY)
@echo "Building sync manager tests..."
@$(CC) -o $@ $^ $(TEST_INCLUDES) $(TEST_CFLAGS) -lm

# Build input handling tests (pure state queries and mapping lookups)
Expand Down
153 changes: 71 additions & 82 deletions docs/audio-rate-control.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,55 @@ Retro game consoles are highly synchronous - audio generation is locked to video

**The fundamental challenge**: Synchronize to vsync (smooth video) while never underrunning or blocking on audio.

## The Algorithm
## Runtime-Adaptive Sync System

### Arntzen's Core Formula
LessUI uses a runtime-adaptive approach that measures the actual display refresh rate and selects the appropriate sync mode automatically.

The paper's pure proportional control adjusts resampling ratio based on buffer fill:
### Two Sync Modes

| Mode | Timing Source | Audio Handling | When Used |
| --------------- | --------------------- | --------------------------------- | -------------------------------- |
| **Audio Clock** | Blocking audio writes | Fixed ratio (no rate control) | Startup default, Hz mismatch >1% |
| **Vsync** | Display vsync | P rate control (±1.2% max adjust) | Hz mismatch <1% from game fps |

### Mode Selection Algorithm

```
error = 1 - 2×fill
adjustment = error × d
ratio = 1 - adjustment
1. Start in Audio Clock mode (safe default, works on all hardware)
2. Measure actual display Hz via vsync timing (~2 seconds warmup)
3. If measured Hz within 1% of game fps → switch to Vsync mode
4. Monitor for drift; fall back to Audio Clock if Hz becomes unstable
```

This eliminates compile-time mode selection and handles hardware variance automatically.

## Audio Clock Mode

When display Hz differs significantly from game fps (>1%), rate control cannot compensate without audible pitch changes. Instead:

- Audio writes **block** when the buffer is full
- Audio hardware clock drives emulation timing
- Frame duplication occurs naturally (less visible than frame skipping)
- No rate control needed - the blocking provides natural backpressure

**Benefits:**

- Works with any display refresh rate
- Audio buffer stays naturally stable
- No controller oscillation or windup

## Vsync Mode (Rate Control Active)

When display Hz closely matches game fps (<1%), vsync provides timing and rate control keeps the audio buffer stable.

### Arntzen's Proportional Control

The paper's proportional control adjusts resampling ratio based on buffer fill:

```c
error = 1 - 2 * fill; // +1 when empty, 0 at half, -1 when full
adjustment = error * d; // Bounded by ±d
ratio = 1 - adjustment; // Resampling ratio
```

**Behavior:**
Expand All @@ -33,64 +72,41 @@ ratio = 1 - adjustment

The paper proves this converges exponentially to a stable equilibrium.

### Our Extension: Dual-Timescale PI Controller
### Why Pure P Works

Pure proportional control works when the host display/audio clocks match the emulated system. On cheap handheld hardware, persistent clock mismatches cause the buffer to settle away from 50%.
Our 1% Hz tolerance for vsync mode ensures we're within the paper's "reasonably close" bounds:

We extend Arntzen with an integral term on a **separate, slower timescale**:
- **Arntzen tested with:** 0.36% Hz mismatch, d=0.5% → 1.4x headroom
- **Our parameters:** up to 1% Hz mismatch, d=0.8% → 1.25x headroom better than Arntzen's ratio

```c
// Fast timescale (proportional): immediate response to buffer jitter
float error = 1.0f - 2.0f * fill;
float p_term = error * d;

// Slow timescale (integral): learns persistent clock offset over ~5 seconds
error_avg = α * error + (1-α) * error_avg; // Smooth error first
integral += error_avg * ki; // Then integrate
integral = clamp(integral, -0.02, +0.02); // Limit to ±2%

// Combined adjustment
float adjustment = p_term + integral;
```

**Key insight**: Original PI failed because both terms operated on the same timescale, causing them to fight. By smoothing error before integrating (~5 seconds), the integral only sees persistent trends, not per-frame noise.
The 1% gate ensures devices in vsync mode have mismatch bounded within what proportional control can handle. Devices outside this range fall back to audio-clock mode where rate control isn't needed.

### Parameters

| Parameter | Value | Purpose |
| ---------- | -------- | ------------------------------------------------------ |
| **d** | 1.0% | Proportional gain. Handles frame-to-frame jitter. |
| **ki** | 0.00005 | Integral gain. Learns persistent clock offset. |
| **α** | 0.003 | Error smoothing (~333 frames / 5.5 seconds at 60fps). |
| **clamp** | ±2% | Max integral correction. Handles hardware clock drift. |
| **buffer** | 5 frames | ~83ms latency. Headroom for timing variance. |
| Parameter | Value | Purpose |
| ---------- | -------- | ----------------------------------------------------------- |
| **d** | 0.8% | Proportional gain. Handles frame-to-frame jitter. |
| **buffer** | 8 frames | ~133ms latency. Matches RetroArch handheld default (128ms). |

## Implementation Details

### Per-Frame Integral Update
### Sync Mode Callbacks

The integral must update **once per frame**, not once per audio batch. Some cores (e.g., 64-bit snes9x) use per-sample audio callbacks, calling `SND_batchSamples()` ~535 times per frame. Without this fix, effective ki = 535× intended, causing wild oscillation.
The audio system queries the sync manager to determine behavior:

```c
// Called once per frame from main loop, before core.run()
void SND_newFrame(void) {
SDL_LockAudio();

float fill = SND_getBufferFillLevel();
float error = 1.0f - 2.0f * fill;

// Update smoothed error and integral (once per frame)
error_avg = α * error + (1-α) * error_avg;
integral += error_avg * ki;
integral = clamp(integral, -0.02, +0.02);

SDL_UnlockAudio();
}
// Set by player at init
SND_setSyncCallbacks(
SyncManager_shouldUseRateControl, // true in Vsync mode
SyncManager_shouldBlockAudio // true in Audio Clock mode
);

// In SND_batchSamples()
bool should_block = snd.should_block_audio();
bool should_use_rate_control = !should_block && snd.should_use_rate_control();
```

### Thread Safety

Rate control state is shared between the main thread (integral updates) and audio thread (buffer reads). All shared state access requires `SDL_LockAudio()` to prevent torn reads on 64-bit ARM where float operations aren't atomic.
This decouples the audio system from sync mode decisions.

### Sample Rate Policy

Expand All @@ -104,41 +120,14 @@ int PLAT_pickSampleRate(int requested, int max) {

Forcing a different rate (e.g., always 48kHz when core wants 32kHz) causes unnecessary resampling and wider buffer swings.

### Vsync Cadence

When a libretro core skips rendering (passes NULL to video_refresh), we still flip to maintain vsync timing:

```c
if (!data) {
frame_ready_for_flip = 1; // Still flip to maintain vsync cadence
return;
}
```

Without this, skipped frames cause: no vsync wait → 4ms frame → next frame waits 2 vblanks → 30ms frame. This creates 20% buffer oscillation even with perfect rate control.

## Tuning Results

Tested across three platforms with different timing characteristics:

| Device | Fill | Variance | Integral | Underruns | Notes |
| ---------- | ---- | -------- | -------- | --------- | ---------------------------- |
| rg35xxplus | 59% | ±8% | +0.15% | 0 | Rock solid |
| tg5040 | 61% | ±16% | -0.71% | 0 | Integral learns clock offset |
| miyoomini | 64% | ±14% | +0.42% | 0 | Fixed by sample rate policy |

**Key findings:**

- d=0.010 (1.0%) is optimal for handheld timing variance (paper's 0.2-0.5% is for desktop)
- Integral converges in ~15-20 seconds to steady-state offset
- Each device has different clock characteristics that the integral learns

## Code References

- PI controller: `workspace/all/common/api.c` (SND_calculateRateAdjust, SND_newFrame)
- Parameters: `workspace/all/common/api.c` (lines 1640-1652)
- Sync manager: `workspace/all/player/sync_manager.c` (mode selection, Hz measurement)
- Rate control: `workspace/all/common/api.c` (`SND_calculateRateAdjust`)
- Sync callbacks: `workspace/all/common/api.c` (`SND_setSyncCallbacks`)
- Parameters: `workspace/all/common/defines.h` (`SND_RATE_CONTROL_D`)
- Resampler: `workspace/all/common/audio_resampler.c`
- Sample rate policy: `workspace/<platform>/platform/platform.c` (PLAT_pickSampleRate)
- Sample rate policy: `workspace/<platform>/platform/platform.c` (`PLAT_pickSampleRate`)

## References

Expand Down
Loading