Software-decoded video renderer silently stops accepting frames (`ready=false`) on a packed-bitstream XviD/MPEG-4 AVI — audio keeps playing, video freezes on the loading spinner

[akacores]

What happened?

Playing an .avi file with codec=mpeg4 (XviD, "packed bitstream" variant) starts audio playback normally, but no video frame after the first one ever reaches the screen. The host app is left showing its loading spinner indefinitely while audio continues. No error is ever surfaced — AetherEngine's $state publisher never transitions to .error, so the host app has no way to detect or recover from this.

• AetherEngine version: 6.6.0 (Swift Package, https://github.com/superuser404notfound/AetherEngine)
• tvOS 26.0, Apple TV 4K (3rd generation) Simulator, 1080p
• Swift 6.2, Xcode (tvOS 26.5 SDK)
• Host app integration: AetherEngine() created per-playback, load(url:options:), $state observed via Combine — see reproduction context below.

Source file

Demuxer opened: 2 streams, duration=1292482883 us
  stream[0] type=video codec=mpeg4 544x400
  stream[1] type=audio codec=mp2

Delivered over HTTP (range requests, AVIOReader pump), .avi container, video track is MPEG-4 Part 2 (XviD) at 544×400, audio is MP2 at 44.1kHz/2ch. Filename: Sabrina, cosas de brujas 1x01 - El cumpleaños de Sabrina.avi.

Steps to reproduce

1. Load and play an AVI file whose video track is a "packed" XviD/MPEG-4 bitstream (B-frames stored in the older "packed" layout).
2. Playback starts: audio is audible immediately.
3. Exactly one video frame is decoded, enqueued, and rendered.
4. Within ~1.3s, the renderer's ready flag flips to false and never recovers — no further frames are displayed, though decode/enqueue calls keep being logged.
5. The host app (which shows a loading spinner until it receives a non-nil player/engine reference and an actual rendered frame) is stuck on that spinner for the rest of the file's duration, while audio plays normally.

Expected behavior

Either:

• Frames continue to be decoded and rendered normally (packed XviD is an old but well-documented format — FFmpeg's own mpeg4 decoder has explicit (if imperfect) handling for it), or
• If the stream is genuinely unrecoverable, AetherEngine's state publisher should transition to .error so the host app can show an error instead of hanging forever on "loading."

Actual behavior

Video rendering silently stalls with no error signaled anywhere. Audio pipeline is completely unaffected and keeps playing to completion.

Relevant logs (verbatim, de.superuser404.AetherEngine + related CoreMedia/AVKit subsystems, log stream --predicate 'process == "coreshub"' --style compact --level debug)

Session start — dispatched to the software decode path (expected, since AVFoundation has no native XviD/MPEG-4-in-AVI decoder):

[AetherEngine] load url=https://.../Sabrina%2C%20cosas%20de%20brujas%201x01%20-%20El%20cumplea%C3%B1os%20de%20Sabrina.avi?api_key=... source-format=sdr effective-format=sdr rate=23.976
[AetherEngine] dispatch: codec=12 → software
[AetherEngine] renderer audio session active: sourceCh=2 maxChannels=2 preferred=2 output=2
[SWHost] session start: videoCodecID=12 audioCodecID=86016 duration=1292.5s
[SWDecoder] Opened: 544x400, codec=mpeg4, threads=8, 8-bit
[AudioDecoder] Opened: 44100Hz, 2ch, codec=mp2
[AudioDecoder] Resampler ready: 44100Hz, 2ch, inFmt=6

Audio starts fine and keeps going (no further issues logged for this path for the rest of the session):

[AudioOutput] first enqueue: 44100.0Hz 2ch, 1152 samples, renderer.error=nil
[AudioOutput] seekClock to=0.000 rate=1.0

First video frame is decoded and rendered successfully:

[SWHost] first video frame enqueued: pixfmt=0x34323076 size=544x400 pts=0.042s
<<<< AVSampleBufferVideoRenderer >>>> -[AVSampleBufferVideoRenderer _setStatus:error:]_block_invoke: <VR/PEK|0x11670d2a0> Change status from Unknown to Rendering with success
[Renderer] enqueue #1: status=rendering ready=true error=nil

~1.3s later, ready flips to false and never recovers — this is the actual bug, reproduced identically at two later checkpoints ~3.5s apart, with no error ever attached:

[Renderer] enqueue #30: status=rendering ready=false error=nil       (t≈+1.37s)
[Renderer] enqueue #100: status=rendering ready=false error=nil      (t≈+4.74s)

FFmpeg's own decoder logs three warnings during this window, strongly suggesting the packed-bitstream handling is where this goes wrong:

[de.superuser404.AetherEngine:ffmpeg] [mpeg4 @ 0x11c712710] [warning] Discarding excessive bitstream in packed xvid
[de.superuser404.AetherEngine:ffmpeg] [mpeg4 @ 0x122006270] [warning] Discarding excessive bitstream in packed xvid
[de.superuser404.AetherEngine:ffmpeg] [mpeg4 @ 0x12200c380] [warning] Discarding excessive bitstream in packed xvid

For context, AVKit's own now-playing controls logic confirms the host is stuck in a perpetual "loading" state from the very start of this session (playerItem.status=0 = unknown, never progresses):

-[AVNowPlayingPlaybackControlsViewController _updateLoadingAndErrorIndicators_NoTimeline] spinner: playerItem.status=0, timeControlStatus=0, loading=YES, isSeeking=NO, isEffectiveRateNonZero=false

(There are also several -[AVPlayerViewController _becomeNowPlaying] denied: Error Domain=AVFoundationErrorDomain Code=-11862 "no player item" lines in the same window — these look like benign timing noise during the load transition, not the root cause, since they all cluster within milliseconds of session start and audio playback proceeds normally afterward. Flagging them only for completeness.)

Hypothesis

The software decode path's handling of packed XviD bitstreams (mpeg4 codec, B-frames stored in the pre-DivX-5/legacy "packed" layout) appears to desynchronize the renderer after the first frame — the FFmpeg decoder itself logs Discarding excessive bitstream in packed xvid, and immediately after, the Renderer's ready flag gets stuck at false for the rest of playback, even though decode/enqueue calls keep happening (audio is unaffected, confirming the demux/read/audio-decode pipeline is healthy — this is isolated to the video renderer's frame acceptance after the packed-bitstream discard).

Suggested improvements regardless of root cause

1. Surface an .error state (or at least a distinct "stalled"/"no video" state) when the renderer's ready flag stays false for an extended period after having previously been true — right now the host app has no signal at all and can only detect this via a user-visible symptom (audio with no video).
2. Consider routing AetherEngine's internal logging through os_log/Logger with a stable subsystem name if not already fully doing so — the de.superuser404.AetherEngine:* subsystems captured here were the only reason this was diagnosable at all; a plain print()-only path would leave zero trace for a downstream integrator.

Additional info available on request

• Full log stream capture (33k+ lines, this excerpt is a small slice) for this exact repro if useful.
• Happy to test a patched build/branch against this same file.

AetherEngine version or commit SHA

6.6.0

Host app

Custom / my own integration

Platform

tvOS

OS version

tvOS 26.0

Device / chip

Apple TV 4K (3rd generation)

Playback path

Software (dav1d) host

Source media (for playback bugs)

AVI, MPEG-4 Part 2 (XviD, packed bitstream), 544x400, SDR, MP2 audio 44.1kHz 2ch

Error codes / log lines

[Renderer] enqueue #1: status=rendering ready=true error=nil
[Renderer] enqueue #30: status=rendering ready=false error=nil
[Renderer] enqueue #100: status=rendering ready=false error=nil

[mpeg4 @ 0x11c712710] [warning] Discarding excessive bitstream in packed xvid
[mpeg4 @ 0x122006270] [warning] Discarding excessive bitstream in packed xvid
[mpeg4 @ 0x12200c380] [warning] Discarding excessive bitstream in packed xvid

-[AVNowPlayingPlaybackControlsViewController _updateLoadingAndErrorIndicators_NoTimeline] spinner: playerItem.status=0, timeControlStatus=0, loading=YES, isSeeking=NO, isEffectiveRateNonZero=false

Anything else

No response

[superuser404notfound]

Thanks for the report, and for capturing the [Renderer] counters. They are what made this answerable without the file.

The three quoted lines are what a healthy software session prints, though, and that is worth saying plainly before fixing anything.

isReadyForMoreMediaData is the demux loop's back-pressure gate: SoftwarePlaybackHost parks on it before feeding a video packet, so a full queue immediately after an enqueue is the gate working, not a queue that stopped accepting. Two things in that capture rule out a stall:

• the counter reaches PGS cues arriving after a reader stall flash through the overlay one by one — each inserts with an open-ended endTime (start + 2^32 ms) and only its successor's trim removes it #100 at t+4.74s, about 21 frames per second against a 23.976 fps source, which is real-time pacing;
• audio and video are fed by the same loop. Had the video queue stopped draining, that loop would have parked on the gate within one queue depth and taken the audio with it. Audio played to completion, so the queue kept draining.

Reproduced here to be sure, on ffmpeg -f lavfi -i testsrc=size=544x400:rate=24:duration=20 -f lavfi -i sine=frequency=440:duration=20 -c:v mpeg4 -vtag XVID -bf 2 -c:a mp2 out.avi through aetherctl play --sw: the same three lines, the same pts=0.042s first frame, clock advancing, verdict OK. aetherctl binds no render surface at all, and therefore also shows no picture.

That is the actual gap, and it is mine: nothing named it. A software session renders into an AVSampleBufferDisplayLayer the engine owns, and that layer reaches the screen only once the host binds a surface, either AetherEngine.bind(view:) with an AetherPlayerView or AetherPlayerSurface in SwiftUI. It does not render through AVPlayerViewController, which has no AVPlayerItem on this path. The AVKit lines in the capture say the same thing from the other side: playerItem.status=0, loading=YES and _becomeNowPlaying denied: no player item, from the first second onward and never recovering. A host presenting an AVPlayerViewController for a software-routed source gets audio, a clean engine log, and AVKit's own spinner over an empty video plane.

So the question that settles it: is a surface bound for this source, and if so, is the bound view laid out at a non-zero size? The swdecode verdict was already pointing at the same place ("the failure is downstream in SoftwarePlaybackHost frame-enqueue, display-layer attach, or audio-clock sync"), which is as far as it can get without knowing anything about the host's view tree.

Both conditions are now named by the session itself, once, about 2 s after frames start flowing (#300, d9d7868e, not in a release yet):

[SWHost] 50 frames decoded into a display layer that is in no view hierarchy: the host never
bound a render surface (AetherEngine.bind(view:) / AetherPlayerSurface). The software path
renders into that layer, not through AVPlayerViewController, so audio plays and no picture
can appear
[SWHost] display layer is bound but sized 0x0 after 50 frames: the bound view has no layout,
so no frame can be visible

On the packed bitstream: I could not reproduce the timestamp failure that hypothesis needs. The demuxer opens with fflags +genpts, so AVI's missing anchor-frame timestamps are regenerated before the decoder sees them (pktdump on the fixture: NOPTS_pts=0, presentation order I(1) B(2) B(3) P(4)). And pts=0.042s in the capture is itself evidence that at least the first frame carried a real timestamp, since an untimed one prints pts=nan there. Discarding excessive bitstream in packed xvid is libavcodec's own note about the packed layout; it did not stop the pipeline in any run here.

The mechanism is closed regardless, because it was reachable in principle: the renderer now refuses a frame whose PTS is not numeric, and counts what it drops. AV_NOPTS_VALUE arrives as CMTime.invalid, and CoreMedia builds a sample buffer from it without complaint (measured: CMSampleBufferCreateReadyWithImageBuffer returns noErr and the sample's PTS reads back as NaN seconds), so the display queue was the first place it could do damage. The deinterlace path already dropped its untimestamped output for that reason; this is the same rule one layer lower, before the reorder buffer, where a NaN would also break ordering for its neighbours. If that particular file does produce untimed frames, the next run names them:

[Renderer] dropped N frame(s) with no usable timestamp (unschedulable)

On the two suggestions:

1. No stalled or error state for now. A watchdog on "ready has been false for a while" would fire on healthy back-pressure, which is the reading this issue started from, and with the queue draining normally there is nothing to time out. A genuine wedge has a different signature: the audio stops alongside the video, because of the shared loop above. That one is worth its own issue if it ever shows up.
2. Logging is already os_log end to end, subsystem de.superuser404.AetherEngine, one category per subsystem (sw.playback for this path). log stream --predicate 'subsystem == "de.superuser404.AetherEngine"' --style compact gets all of it; add --level debug for the per-segment and per-request trace, which is emitted at .debug only.

The branch is fix/298-sw-surface-visibility if you want to run it against that file before it lands. The offer of the full capture is appreciated; the answer on the surface binding, or the first [SWHost] line the patched build prints, would settle it faster.

[superuser404notfound]

Shipped in 6.6.3 (d9d7868e).

Both lines are in: the session names an unbound or unlaid-out display layer about two seconds after frames start flowing, and the renderer counts any frame it has to drop for an unschedulable timestamp. Running that file on 6.6.3 should say which of the two this is, or say neither, which would be its own information.

Leaving this open for that answer rather than closing it on my own reproduction, since the file is yours and mine only matches its shape.