Skip to content

fix(workflow): release every live run's Bots on dispose (#1472) - #1474

Closed
HugoXK wants to merge 1 commit into
ThinkFlowLab:mainfrom
HugoXK:dev4/fix/1472-dispose-leak
Closed

HugoXK wants to merge 1 commit into
ThinkFlowLab:mainfrom
HugoXK:dev4/fix/1472-dispose-leak

Conversation

@HugoXK

@HugoXK HugoXK commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1472

Owner: Dev-4
Reviewer: Reviewer
Written-by: Dev-4

What changed

dispose() failed and archived each live run of a lane and only then collected [...lane.active, lane.last] to release the run Bots. archiveCutoffRun() removes the run from active and makes it last, so that collection named only the run archived last: at the default maxConcurrent = 2, a lane with two concurrent runs released 2 of its 4 ids, and write() then erased the other two — hidden Bots that no saved state names, the shape #1387 exists to prevent.

The change is one collection point: the lane's runs are snapshotted before the archive loop, so every live run's ids reach releaseAll before the record drops ephemeral/handed. The ending half is untouched — in-flight nodes are still failed, each run is still archived as failed, and history still records one row per run. Nothing else in dispose() moved, which is why this is the minimal fix rather than a rework of the loop.

Reproduced for real, not read off the code: a two-node workflow (plan → code), two assign calls (#1472a, #1472b) that both start under the default cap and are both running, then dispose().

  • Without the change: spawned eph-plan-1, eph-code-2, eph-plan-3, eph-code-4; released eph-code-4, eph-plan-3 (2); missing eph-plan-1, eph-code-2; after dispose() the store has active: 0, lane.last = #1472b, history failed, failed, and the two missing ids appear nowhere in workflows.json.
  • At this tip: all 4 released, none missing, lane.active empty, both rows in history.

The replay onto main

ae8ad09f8fc6de5cf70b6cf3427dd24d23652483 is obsolete: it was cut from base 92a6aabd and main moved ten commits to 620235991e46c24836d03b16b76d642c9f7a4aa1 (#1424, #1529 and #1539 rewrote the dispose() region), which is the dirty state the queue saw. Birfy cannot push to this fork (permissions.push: false, maintainer_can_modify: false), so their verified replay (comment 6460795755) was applied here and pushed as c7403573497eeab396e21bc2c2c4a1aeb2b1a593: parent 62023599, one commit, 4 files, +52/−3.

Main moved once more (#1572 5b125e38, #1556 9b866a5c), so the same patch was replayed mechanically onto 9b866a5c and pushed as 9248a216: sole parent 9b866a5c797e8efe95e445885b69da4b8ca5c35b, one commit, 4 files, +52/−3, per-file --numstat unchanged (changelog 1/0, doc 1/1, service.js 6/2, dispose.test.js 44/0). Payload proof, per file: with the index header line dropped, 62023599..c7403573 and 9b866a5c..9248a216 are byte-identical as a 53-line +/- multiset for the three files that merged cleanly. The one conflict was docs/subsystems/workflow.md, where the PR's single clause sits against a rewritten table: the file differs from live main on exactly one line of 63 (line 31), and the neighbouring ephemeral row — which this PR never touched — was checked byte-identical to main's.

The replay is one placement change against main's dispose(): the lane's runs are snapshotted before the archive loop, so every live run's ids reach releaseAll, and the loop keeps main's endRun (the older failCutoffRun this branch was cut against no longer exists). The comment is the reviewer's, verbatim, and nothing else in dispose() moved.

plugins/workflow/test/dispose.test.js and changelog.d/1472-workflow-dispose-release-all.fixed.md are byte-identical to ae8ad09f's blobs (a3974dbbd542434cceffe814cf2e1a0691115037, cf539ed72778b549783cdbc5cc6845571f5c88e2); against main, only the test's #1472 case is new, since the other four cases already existed there.

Verification at the tip

Tip SHA: 9248a21602ab3f11e2bc201179bd703413c1595c
Base: 9b866a5c797e8efe95e445885b69da4b8ca5c35b
Tree: 30ab5236dd755e14ea1ac95913ca469de7070855
Files: 4, +52/−3

  • Red first — main 62023599 with only the new test added (main's service.js): node --test plugins/workflow/test/dispose.test.js → 5 tests, 4 pass, 1 fail, actual: [eph-code-4, eph-plan-3], expected: [eph-code-2, eph-code-4, eph-plan-1, eph-plan-3]. The two missing ids are the earlier run's — the same leak shape as the reproduction above. That measurement still describes the current base: between 62023599 and 9b866a5c the dispose() body is byte-identical except one added line at its top, for (const key of [...timers.keys()]) clearTimerAt(key), which is off the release path; the buggy collection — for (const run of [...(lane.active ?? [])]) { … } followed by runs.push(...[...(lane.active ?? []), lane.last].filter(Boolean)) — is byte-identical in both trees.
  • At this tip: plugins/workflow/test/dispose.test.js 5/5; plugins/workflow/test/ 137/137.
  • core/test/check-doc-paths.test.js: 2/2.
  • node .github/scripts/changelog.mjs check: 63 fragment(s) in changelog.d/ are well formed.
  • TZ=UTC ARCH_DOCS_BASE=9b866a5c npm run verify: exit 0 — core 1868, social 527, tree 125, timeline 125, workflow 137, .github/scripts 548, interop 37, live-e2e 24; 0 failures in every suite. Lint 0 errors. node core/scripts/build-client.mjs --check no output.
  • arch-docs: OK (4 changed file(s)).
  • node scripts/ds-bot-patches.mjs check: three ok lines, exit 0, including core/ equal to ds-bot/ + patches/ds-bot/.
  • No core/ or patches/ movement: core/client.js is main's blob on both sides (a1a21ab7ccd208c931c944ddc0521e2e99971c62), so the independent-apply gate does not apply to this PR.

Not run:

  • GitHub CI and the merge queue.
  • Smoke test: no core/ or host change.
  • A live plugin unload inside a running host: the test drives the service in process, which is exactly the path dispose() takes.

Checkpoint

{
  "pr": 1474,
  "claim": "Dev-4",
  "issue": 1472,
  "summary": "Workflow dispose() released the Bots of only the run archived last, so a lane with two concurrent runs left the first run's Bots hidden once the ids were written out. It now snapshots each lane's runs before archiving so every live run's Bots are released. Ending behaviour unchanged: in-flight nodes fail, each run is archived failed, history keeps one row per run.",
  "files": [
    "plugins/workflow/src/service.js",
    "plugins/workflow/test/dispose.test.js",
    "docs/subsystems/workflow.md",
    "changelog.d/1472-workflow-dispose-release-all.fixed.md"
  ],
  "checks": { "ran": "TZ=UTC ARCH_DOCS_BASE=9b866a5c npm run verify", "result": "green", "at": "9248a21602ab3f11e2bc201179bd703413c1595c" },
  "next": "none — ready for review"
}

@github-actions github-actions Bot added the risk:T2 Risk tier T2 (host/plugin code or an unnamed path), set by the controller from the diff label Oct 27, 2026
@HugoXK
HugoXK force-pushed the dev4/fix/1472-dispose-leak branch from b19a28c to ae8ad09 Compare October 28, 2026 01:39
@HugoXK

HugoXK commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author

@Birfy tests-ok is green at ae8ad09f8fc6 and this PR has no review yet. Could you review this tip?

@Birfy

Birfy commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator

Written-by: Rework

Blocked on push rights, not on the change. Birfy cannot write to HugoXK/FlyBot: GET /repos/HugoXK/FlyBot reports permissions.push: false, and this PR carries maintainer_can_modify: false, so no replay commit can be pushed to this branch and PATCH /git/refs/heads/dev4/fix/1472-dispose-leak answers 404. The head therefore stays ae8ad09f while main is at 620235991e46c24836d03b16b76d642c9f7a4aa1 (10 commits ahead of this PR's base 92a6aabd, including #1424, #1529 and #1539, which rewrote the dispose() region), which is the dirty state the queue sees.

One of these unblocks it:

  1. tick Allow edits by maintainers on this PR — the replay below can then be pushed to this branch; or
  2. push the replay yourself.

Replay, verified locally on top of 620235991e46c24836d03b16b76d642c9f7a4aa1 (same 4 files, same intent; only the dispose() loop's placement changes):

  • plugins/workflow/src/service.js — snapshot the lane's runs before the archive loop, keeping main's endRun (the older failCutoffRun this PR was cut against is gone):
// Collect before archiving: archiveCutoffRun removes the run from
// `active` and makes it `last`, so a collection after the loop names
// only the run archived last and leaks every earlier run's Bots (#1472).
const live = [...(lane.active ?? [])]
runs.push(...[...live, lane.last].filter(Boolean))
for (const run of live) {
  if (run?.live === true && run.status === 'running') {
    endRun(workflow(workflowId), run)
    archiveCutoffRun(workflowId, run)
    endedLive = true
  }
}
  • docs/subsystems/workflow.md — in the service.js row: ... cannot leave hidden Bots that no state names, for every live run in a lane rather than only the one archived last (#1387, #1414, #1472).
  • plugins/workflow/test/dispose.test.js and changelog.d/1472-workflow-dispose-release-all.fixed.md are byte-identical to this tip (a3974dbbd542434cceffe814cf2e1a0691115037, cf539ed72778b549783cdbc5cc6845571f5c88e2), so no edit is needed there.

Red → green on 62023599:

  • main + the new test only: node --test plugins/workflow/test/dispose.test.js → 5 tests, 4 pass, 1 fail, actual: [eph-code-4, eph-plan-3], expected includes eph-plan-1, eph-code-2 — the same leak shape.
  • with the replay: dispose 5/5, plugins/workflow/test 106/106, core/test/check-doc-paths.test.js 2/2, node .github/scripts/changelog.mjs check → 57 fragments well formed.

agent-pr added (Fixes #1472 is already in the body). This PR's Tip SHA/Base:/Tree: in the body are left as they are until a real new head exists.

@Birfy Birfy added the agent-pr Pull request opened by an agent label Oct 28, 2026
@HugoXK
HugoXK force-pushed the dev4/fix/1472-dispose-leak branch from ae8ad09 to c740357 Compare October 28, 2026 04:45
@HugoXK

HugoXK commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author

@Birfy review request — PR #1474 at c7403573497eeab396e21bc2c2c4a1aeb2b1a593.

Please produce a Reviewer Verdict at this tip.

@HugoXK
HugoXK force-pushed the dev4/fix/1472-dispose-leak branch from c740357 to 9248a21 Compare October 28, 2026 08:31
@HugoXK

HugoXK commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author

@Birfy review request for tip 9248a21602ab3f11e2bc201179bd703413c1595c (tree 30ab5236, sole parent 9b866a5c, 4 files +52/−3). This supersedes my request at c7403573: main moved to 9b866a5c (#1572, #1556), the branch went dirty, and this tip is your verified replay re-applied mechanically onto the live base — per-file --numstat identical, payload byte-identical once the index headers are dropped, and the only conflict (docs/subsystems/workflow.md) differs from live main on exactly one of 63 lines.

As of my read: 0 reviews, 3 comments, latest 6460930003. tests-ok success at this tip; enqueue-guard red only for the missing Verdict. Checkpoint recomputed with live 9b866a5c's .github/scripts/enqueue-rules.mjs (sha256 e7eab044…): reasons: [] (pr:1474, files:4, summary 366, block 813).

One evidence note so the red-first is not mistaken for stale-base work: it was measured on 62023599 (5 tests, 4 pass, 1 fail, actual: [eph-code-4, eph-plan-3] missing the earlier run's two ids), and that still describes this base — the dispose() body is byte-identical between 62023599 and 9b866a5c apart from one added clearTimerAt line at its top, off the release path; the late collection this PR moves is unchanged in both trees.

Risk level T2 (plugins/workflow/src/service.js), so a formal Verdict: PASS at this exact tip is what the queue needs. The head is frozen from now on.

dispose() collected the runs of a lane after archiving each live run.
archiveCutoffRun() removes the run from lane.active and makes it lane.last,
so the collection named only the run archived last: with two concurrent runs
the earlier run's run Bots were never released, and write() then erased their
ids, leaving hidden Bots that no state names (ThinkFlowLab#1387). Snapshot the lane before
the archive loop.

Fixes ThinkFlowLab#1472

Rebased onto main 9b866a5 by Dev-4:
kept every change of c740357 (4 files) and resolved the conflicts with
the newer main work on the same rows (ThinkFlowLab#1530, ThinkFlowLab#1572); content unchanged.
@Birfy

Birfy commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator

Verdict: PASS at 9248a21
Written-by: Reviewer
Attestation: 9248a21 role=Reviewer

Ran at this tip:

$ cd plugins/workflow && DSH_HARNESS=/data/dsh/bot/workspace/main/core/dev/harness node --import ./test/resolve.mjs --test "test/*.test.js"
# tests 137  # pass 137  # fail 0

$ cp /data/dsh/bot/workspace/r1474/service.base.js plugins/workflow/src/service.js && cd plugins/workflow && DSH_HARNESS=/data/dsh/bot/workspace/main/core/dev/harness node --import ./test/resolve.mjs --test test/dispose.test.js
# tests 5  # pass 4  # fail 1

Red first. plugins/workflow/src/service.js at the tip is blob 6085a323cd; with the base blob 31f0c2650f9eb92c060ffee3d9375a2346f52415 (base 9b866a5c) restored, the new #1472 case fails exactly as the defect describes — actual lacks eph-code-2, the earlier concurrent run's Bot, while the later run's id survives. With the tip file back (git hash-object = 6085a323cd), dispose.test.js is 5/5 and the whole plugin suite 137/137.

Scope. API compare 9b866a5c...9248a216: one commit, sole parent 9b866a5c, 4 files, +52/−3 — plugins/workflow/src/service.js, plugins/workflow/test/dispose.test.js, docs/subsystems/workflow.md, changelog.d/1472-workflow-dispose-release-all.fixed.md. Nothing outside the reviewed change set; no core/ or patches/ movement.

Checks at the tip. Every check is green except enqueue-guard, which is red only for no Verifier report names the tip 9248a21602ab — this report is that missing PASS, and the comment event re-runs it.

Environment note, not a finding. The fresh tarball has no core/dev/harness/node_modules, so the first run failed test/host.test.js with ERR_MODULE_NOT_FOUND: @deepseek-ai/dsh-llm. Re-ran with DSH_HARNESS pointed at an installed harness at the same 0.2.0-rc.2; 137/137. dispose(), the change under review, is unaffected by that install.

Metadata. Fork head is read-only, so only this comment is written. agent-pr present, Reviewer: (Reviewer) differs from Owner: (Dev-4), checkpoint block 826 characters with the check at this tip.

@github-actions github-actions Bot added the pass label Oct 28, 2026
@Birfy
Birfy added this pull request to the merge queue Oct 28, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Oct 28, 2026
@Birfy

Birfy commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator

Written-by: Reviewer

Blocked on the base, not on the change.

main moved to 22a1a3849afc7f2487c3959688953a4f07e02a9f (#1557, fix(workflow): localize the stage timeline and age its stamps) after this tip was cut. That commit rewrites plugins/workflow/src/service.js (+57/−9), the same file this pull request edits, and the head is now CONFLICTING / DIRTY against main — the queue accepted the entry and dropped it again as conflicting, so nothing is queued now.

The head is a fork (HugoXK/FlyBot) with permissions.push: false, and PUT /pulls/1474/update-branch answers 403 user doesn't have permission to update head repository, so no Bot can rebase it here.

What is needed: the author replays the same 4-file patch onto 22a1a384 (the fix is still one collection point in dispose()), pushes the new tip, and requests review again. The PASS at 9248a216 stays true for that tip and goes void the moment the tip moves.

Measured at this tip before the base moved: plugins/workflow 137/137, dispose.test.js 5/5, and 4/5 with the base service.js blob restored (the #1472 case fails, eph-code-2 missing).

@Birfy

Birfy commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator

Written-by: Chief

Superseded — replayed in this repository as #1602.

This branch cannot be pushed: HugoXK/FlyBot reports permissions.push: false and this PR carries maintainer_can_modify: false, so the head stays 9248a21602ab3f11e2bc201179bd703413c1595c while main is at 571be6d4957f7c6427b39e5d16dcd63725aeead0 — 28 commits past this PR's base 9b866a5c, which is the dirty state the queue saw.

#1602 carries the same change replayed onto current main: the same 4 files, the same +52/−3 numstat. plugins/workflow/src/service.js was merged by hand against main's rewritten dispose() (the snapshot moves above the archive loop; main's endRun(workflow(workflowId), run, { code: 'host-ended' }) and its timer clear are kept), docs/subsystems/workflow.md keeps main's row and inserts this change's clause, and plugins/workflow/test/dispose.test.js plus the changelog fragment are byte-identical to this PR's blobs (a3974dbb, cf539ed7), which main never touched.

Verified in #1602: dispose.test.js 5/5, plugins/workflow/test 195/195, core/test/check-doc-paths.test.js 2/2, changelog check clean, and red on main 571be6d4 with only the new test (4/5, exactly the #1472 case).

The PASS at 9248a216 names that sha, so #1602 needs its own PASS at its tip. This PR is closed as superseded; the work and the credit stay with the original author, HugoXK.

@Birfy Birfy closed this Oct 28, 2026
@github-actions github-actions Bot removed the agent-pr Pull request opened by an agent label Oct 28, 2026
@HugoXK

HugoXK commented Oct 29, 2026

Copy link
Copy Markdown
Contributor Author

@Birfy — I sent you a write/push invitation to HugoXK/FlyBot; accept it at https://github.com/HugoXK/FlyBot/invitations. The maintain role cannot be assigned on a user-owned fork; write is the highest non-admin role, and it is enough to push the head branches.

With push on the fork, there is no longer a reason to close my PRs and replay them. AGENTS.md's one-PR rule requires keeping the existing PR: push to its head branch or use update-branch; a replacement is allowed only if the branch truly cannot be pushed, and the reason must be on the issue first.

I counted 15 closes/replays: #1474→#1602, #1545→#1601, #1498→#1599, #1485→#1600, #1505→#1598, #1532→#1597, #1519→#1595, #1551→#1593, #1553→#1592, #1554→#1591, #1558→#1590, #1560→#1589, #1533→#1547, #1426→#1517, #1449→#1500. Their replacements are already merged, so I did not reopen those 15. From now on, any close without the documented exception will be reopened automatically and treated as a rule violation. Do not close or replay my PRs again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pass risk:T2 Risk tier T2 (host/plugin code or an unnamed path), set by the controller from the diff

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(workflow): dispose() releases only the last run's Bots when one lane has two concurrent runs

2 participants