Skip to content

feat: model a machine that fails to reach started (image-pull failure injection) (#61)#62

Merged
lex00 merged 1 commit into
mainfrom
feat/61-unpullable-image-failure
Jul 16, 2026
Merged

feat: model a machine that fails to reach started (image-pull failure injection) (#61)#62
lex00 merged 1 commit into
mainfrom
feat/61-unpullable-image-failure

Conversation

@lex00

@lex00 lex00 commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Closes #61.

What

mudflaps' advancer only ever drove a machine to started (StateFailed was defined but never set), so a Machines-API client's deploy-failure path was untestable. This adds a sentinel that makes a machine settle into failed instead of started, modeling a boot-time image-pull failure — the shape of this Fly forum report.

Mechanism (sentinel image)

A machine whose config.image is mudflaps/unpullable (bare, or with a :tag/@digest) settles into failed on its next boot transition — create, start, restart, or an update to that image. Chosen over a control endpoint because the failure rides in the config the client already sends, so no new wire surface.

  • The boot transition is now settle, which reads the machine's latest config when it fires — so an update back to a real image before the transition still boots (recovery).
  • .../wait?state=started never falsely reports started; it times out (408).
  • GET .../machines/{id} reports state: failed, and the version history tracks it.
  • Fully deterministic on the injected clock.

Real flaps has no such magic image — mudflaps models the Machines-API state layer, not an image registry, so this is the single injection point that lets an applier (e.g. chant's flyApply, which polls .../wait for started) exercise its failure/cancel/rollback path offline.

Scope note

The failure reason as a machine event (issue #61 acceptance item 3, marked "if modeled") is intentionally deferred: mudflaps models no events today, and adding an events array means committing to fly-go's MachineEvent wire shape, which deserves its own change rather than a best-guess here. state: failed + the sentinel is a faithful, verifiable failure; the reason-event can layer on later without changing this.

Tests

  • Advancer: create reaches failed for an unpullable image (step-by-step); sentinel matching across bare/:tag/@digest, non-matches, and nil config; update-to-good-image recovery.
  • Server: create with the sentinel → failed, GET observes it, wait?state=started408 (goroutine + clock-advance, mirroring the existing wait-timeout test).
  • go build / go vet / gofmt / golangci-lint (0 issues) / go test -race all clean.

Docs

README feature bullet + a "Deploy-failure injection" section in docs/fidelity.md (state diagram gains starting --> failed, failed added to the resting-states list). CHANGELOG staged as 0.5.0 (new behavior → minor, pre-1.0).

Acceptance

  • A machine can be driven to failed deterministically (sentinel image), on the injected clock.
  • wait?state=started against such a machine does not falsely report started.
  • Machine event with a reason — deferred (see Scope note).
  • Tests cover the failed-deploy path; docs note failure injection.

…re injection) (#61)

The lifecycle advancer only ever drove a machine to `started`; `StateFailed`
was defined but never set, so a Machines-API client's deploy-failure path
couldn't be tested. A machine whose `config.image` is the sentinel
`mudflaps/unpullable` (bare, or with a `:tag`/`@digest`) now settles into
`failed` instead of `started` on its next boot transition — modeling a
boot-time image-pull failure ("unable to pull image, canceling deploy").

The boot transition (`settle`) reads the machine's latest config when it
fires, so an update to a real image beforehand still boots. A client that
polls `.../wait?state=started` sees a deploy that never comes up: the wait
times out (`408`, never a false `started`) and `GET` shows `state: failed`.
Everything stays deterministic on the injected clock. Real flaps has no such
magic image; the sentinel is the single injection point.

Tests cover the advancer failed-boot path, sentinel matching (tag/digest,
nil config, non-matches), update-to-good-image recovery, and the server-level
deploy-failure path (create → failed → wait 408). README + fidelity docs note
the injection. Staged as 0.5.0.
@lex00
lex00 merged commit a3c4891 into main Jul 16, 2026
2 checks passed
@lex00
lex00 deleted the feat/61-unpullable-image-failure branch July 16, 2026 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fidelity: model a machine that fails to reach started (image-pull failure injection)

1 participant