shipyard: ship TaskScheduler 0.4.0 milestone

Milestone shipped via PR #115 (merged to master as commit 190f122).
All 4 phases complete across 2 repositories:

- Phase 1 (sibling TaskScheduler repo): Lock contention fix — NetMQPoller
  owns _actor, outbound SetCount routed through NetMQQueue, _lockSocket
  eliminated. 9/9 unit tests green, 5/5 concurrency loop green.
- Phase 2 (sibling TaskScheduler repo): NuGet 0.4.0 published to nuget.org
  via tag-triggered GH Actions. Symbols + deterministic Source Link green.
- Phase 3 (this repo): New net10.0 integration test project consuming the
  0.4.0 NuGet via CPM. 3 test classes: ConcurrencyRegressionTests (real
  cross-repo regression guard), NodeDiscoveryTests, EndToEndSchedulingTests
  (smoke). 4/4 green in 5/5 flakiness loop. 896/896 core + 57/57 Memory
  integration tests still green.
- Phase 4 (this repo): CI wiring — 1 new GH Actions step + 14th Jenkins
  parallel stage (sleep 65s, (n-1)*5 stagger preserved). 15-line additive
  diff. Optimistic UDP multicast on Docker bridge worked on first Jenkins
  run, no beacon-skip fallback needed.

Ship-time artifacts:
- MILESTONE-REPORT.md: aggregated phase summaries, key decisions, known
  issues, metrics
- LESSONS.md: 15-entry milestone section covering process lessons
  (Jenkins PR-triggered, git fetch discipline, agent turn budgets),
  code/API discoveries (SchedulerContainer closure pattern, Start()
  invariant, queue name format, Memory per-container storage,
  cross-namespace walk-up), and CI config clarifications
- CLAUDE.md: 9 new Lessons Learned entries, plus the "13 → 14 parallel
  integration test stages" update and the Jenkins PR-trigger note added
  to the Conventions CI section
- STATE.json: status=shipped, position="TaskScheduler 0.4.0 milestone
  shipped via PR #115"

Worktree .worktrees/phase-4-ci-wiring removed; local phase-4-ci-wiring
branch deleted; remote branch auto-deleted by GitHub on merge.

30 issues remain open in ISSUES.md, preserved for the next milestone.
Notable: ISSUE-030 upstream README positional-args workaround still
applies; pre-existing SYSLIB0012 warnings in LiteDB/SQLite
ConnectionString.cs tracked for a future cleanup phase.

Author: blehnen

.shipyard/HISTORY.md

@@ -793,3 +793,4 @@
 - [2026-04-15T16:40:31Z] Phase 4: Building phase 4 (worktree phase-4-ci-wiring) (building)
 - [2026-04-15T16:52:50Z] Phase 4: Phase 4 build complete in worktree; pending CI validation on push (success criteria #1, #2) (complete)
 - [2026-04-15T16:53:01Z] Phase 4: Phase 4 build complete in worktree; pending CI validation on push (complete)
+- [2026-04-15T19:26:38Z] Phase 4: TaskScheduler 0.4.0 milestone shipped via PR #115 (shipped)

.shipyard/LESSONS.md

@@ -316,3 +316,45 @@
 - The retroactive reviewer pattern (dispatch N parallel reviewers for already-committed plans, then proceed to verifier) is safe and efficient — document it as a supported resume path if a build session interrupts between waves and review gates
 
 ---
+
+## [2026-04-15] Milestone: TaskScheduler 0.4.0 + DNQ Integration Tests + CI Wiring (issue #6)
+
+**Shipped via PR #115. Cross-repo: Phases 1-2 in sibling `DotNetWorkQueue.TaskScheduling.Distributed.TaskScheduler`, Phases 3-4 in DNQ.**
+
+### What Went Well
+
+- **Optimistic UDP multicast on Jenkins Docker paid off.** CONTEXT-4 decision #1 (ship without pre-emptive `[TestCategory("BeaconRequired")]` skip) turned out correct — NetMQ beacon discovery worked on the Docker bridge network on the first Jenkins run of PR #115, and no follow-up skip mechanism was needed.
+- **Phase 1's NetMQ poller refactor held under real concurrency load.** ConcurrencyRegressionTests (12 threads × 5000 iters, 30s deadlock detector) passed in 5/5 consecutive runs with ~2s wall clock per run. The fix is real and the regression guard catches it.
+- **Phase 2's tag-triggered GH Actions publish path mirrored 0.3.0 cleanly.** No local `dotnet nuget push` needed; both `.nupkg` and `.snupkg` published with green badges on nuget.org via run 24423676631.
+- **Phase 4's additive-only CI change** (15 lines across 2 files, 0 deletions, 0 modifications) made both Jenkins and GitHub Actions integration boringly predictable once the merge conflict was resolved.
+
+### Surprises / Discoveries
+
+- **Jenkins is PR-triggered, not branch-triggered.** Any "feature-branch-first validation" workflow MUST open a (draft) PR before Jenkins will pick it up. Pushing a feature branch alone is insufficient. **Implication:** the Shipyard worktree workflow's "push for validation" step for CI-sensitive phases must include `gh pr create --draft` as the actual trigger, not just `git push`.
+- **Agent turn budget exhaustion is the dominant failure mode** for multi-task builder dispatches. 4/5 Phase 3 builder agents dropped their commits or SUMMARY files mid-response, forcing the main driver to take over. **For 1-task plans with exact-match anchors, direct edits from the main thread are faster and more reliable than dispatching an agent.** This became Phase 4's default execution strategy and worked well.
+- **`git fetch origin` silently skipped at milestone start caused the ship-time merge conflict.** Local master worked the TaskScheduler milestone in isolation (9 commits) while origin was shipping Phase 5 dashboard-coverage via PR #114 (5 commits) in parallel. The two diverged until PR #115 couldn't merge. Recovery: merge origin/master into local master, resolve conflicts in `.shipyard/ROADMAP.md` (ours) + `.shipyard/HISTORY.md` (concatenate) + `.shipyard/STATE.json` (ours) + delete a stray archived phase file, then rebase `phase-4-ci-wiring` onto the unified master. **Always `git fetch origin master` before `/shipyard:plan 1` on a new milestone.**
+- **Plan "mirror project X" directives can contradict literal specs** when project X has drifted from the plan author's mental model. PLAN-1.1 told the Phase 3 builder to mirror `Memory.Integration.Tests` AND spec'd `net10.0;net8.0`, but Memory is `net10.0`-only — the builder had to choose, and the plan's literal spec was wrong. Spot-check inheritance targets at plan time with an actual `cat` / `grep`, not from memory.
+- **Shared test runners may not expose the seam the plan assumes.** PLAN-2.1 told the Phase 3 builder to "use `DotNetWorkQueue.IntegrationTests.Shared.Consumer.Implementation.SimpleConsumer.Run<>()`" to inject the distributed task scheduler — but that runner's signature exposes `Action<TTransportCreate> setOptions` (transport options), not `Action<IContainer> registerService` (container registration). Research must verify the exact seam exists before the architect commits to a plan pattern.
+- **Memory transport storage is per-container.** Two `QueueContainer<MemoryMessageQueueInit>` instances don't share the underlying `IDataStorage` via `RegisterNonScopedSingleton(scope)` alone. A naive producer/consumer split across two containers never sees the produced messages. Phase 3 EndToEndSchedulingTests was scope-reduced to a SimpleInjector `Verify()` smoke test because of this constraint.
+- **PLAN-1.2 had a plan data error** (expected "unstash count = 13" but pre-edit actual was 14). Research should spot-check grep counts with exact commands, not estimated counts.
+- **The auditor agent wrote `AUDIT-4.md` to the WRONG directory** (`.shipyard/phases-archive-code-coverage/4/results/` instead of `.shipyard/phases/4/results/`). The agent's path resolution found an existing archive directory and wrote there by default. Had to move the file to the correct location before commit. **Watch for this in future ship flows where an archive directory with the same shape exists.**
+
+### Pitfalls to Avoid
+
+- **`SchedulerContainer.GetInstance<T>()` does not exist in `TaskScheduler 0.4.0`.** The only way to resolve `ITaskSchedulerJobCountSync` is the **IContainer closure pattern**: capture `IContainer` during the `SchedulerContainer(registerService)` callback, trigger build via `CreateTaskScheduler()`, then resolve from the captured container. An earlier NodeDiscoveryTests draft used the nonexistent API and produced 10 compile errors; rewritten during build recovery using the pattern discovered by the PLAN-2.2 builder.
+- **`Start()` before spawning hammering threads is non-negotiable for `ConcurrencyRegressionTests`.** Without `Start()`, `_outbound` is null and the null-safe guard from Phase 1 short-circuits every `IncreaseCurrentTaskCount` / `DecreaseCurrentTaskCount` call — the test becomes a false positive that would pass even if Phase 1's lock fix were reverted. A comment in the test documents this invariant explicitly.
+- **DNQ queue names must be alphanumeric/underscore/dot.** `Guid.NewGuid().ToString()` produces hyphenated strings that DNQ validation rejects with `Queue name contains invalid characters`. Use `Guid.NewGuid().ToString("N")` (no hyphens) or a sanitized format.
+- **Cross-namespace walk-up gotcha for `IDataStorage`.** Cloning `SharedClasses.cs` from the Memory test project into a differently-named namespace breaks walk-up resolution of `DotNetWorkQueue.Transport.Memory.IDataStorage`. Same root cause as the existing `IConfiguration` and `Metrics.Metrics` shadowing lessons. **Always add `using DotNetWorkQueue.Transport.Memory;` explicitly when cloning from that project.**
+- **`-p:CI=true` is a NuGet packaging flag, not a test-run flag.** It enables `ContinuousIntegrationBuild` in `Directory.Build.props` for deterministic Source Link during `dotnet build -c Release`. It has no effect on `dotnet test`. Don't copy-paste it into test invocations — the Phase 4 CONTEXT-4 draft made this mistake and had to be corrected mid-research.
+- **Jenkinsfile stagger formula `(n-1) * 5` is at its ceiling with 14 stages** (worst-case startup delay 65s). A future 15th stage will push to 70s and may need the formula revisited or a different approach. Captured as a known issue for a future cleanup phase.
+
+### Process Improvements
+
+- **Before `/shipyard:plan 1` on any new milestone: `git fetch origin master && git log HEAD..origin/master --oneline`** to confirm local master is current. If origin has unpulled work, decide whether to pull or branch from origin/master BEFORE the milestone starts, not at ship time.
+- **For 1-task plans with exact-match anchors, execute inline from the main thread** rather than dispatching a builder agent. Agent reliability for tiny plans is worse than direct edits because of turn budget exhaustion. Reserve builder agents for multi-task plans or plans that require iterative API discovery.
+- **Always `gh pr create --draft` for CI-sensitive feature branches.** The Shipyard worktree protocol's "feature branch first" validation step needs a draft PR to trigger Jenkins, not just a push. Update `/shipyard:worktree create` docs / the CONTEXT capture template to mention this.
+- **When cloning a file from one test project into another, do a namespace walk-up sanity check.** For every `using` the new file relies on implicitly, verify the new project's namespace hierarchy can reach it. When in doubt, add explicit `using` directives up front rather than debugging walk-up resolution later.
+- **Auditor agent path-resolution hardening:** the auditor should verify the path it's writing to is under `.shipyard/phases/{N}/results/` and NOT under `.shipyard/phases-archive*/`. Captured as a Shipyard framework feedback item.
+
+---
+

.shipyard/MILESTONE-REPORT.md

@@ -1,54 +1,100 @@
-# Milestone Report: Replace Schyntax with Cronos (issue #100)
+# Milestone Report: TaskScheduler 0.4.0 + DNQ Integration Tests + CI Wiring
 
-**Completed:** 2026-04-08
-**Version:** 0.9.30
-**Phases:** 5/5 complete
-**GitHub Issue:** #100
+**Completed:** 2026-04-15
+**Shipped via:** PR #115 merged to master (commit `190f1226`)
+**Phases:** 4/4 complete
+**Cross-repo scope:** Phases 1-2 shipped from `DotNetWorkQueue.TaskScheduling.Distributed.TaskScheduler`, Phases 3-4 shipped from `DotNetWorkQueue` (this repo)
 
-## Summary
+## Overview
 
-Replaced the vendored Schyntax DLL (custom DSL, no NuGet package) with Cronos (MIT, standard cron expressions) and CronExpressionDescriptor (human-readable descriptions). Breaking change.
+Fixed NetMQ lock contention in `TaskSchedulerJobCountSync` (issue #6), released the fix as NuGet `0.4.0` on nuget.org, added a new integration test project in DNQ that consumes the published package, and wired the new tests into both CI surfaces (Jenkins + GitHub Actions).
 
 ## Phase Summaries
 
-### Phase 1: Core library
-- Rewrote JobSchedule.cs from Schyntax.Schedule to Cronos.CronExpression
-- IJobSchedule.Previous() now returns DateTimeOffset? (nullable)
-- Added IJobSchedule.Description property
-- Auto-detects 5-field vs 6-field cron by field count
+### Phase 1: TaskScheduler Lock Contention Fix + Unit Tests — SHIPPED
+**Repo:** `DotNetWorkQueue.TaskScheduling.Distributed.TaskScheduler` (branch `phase-1-lock-fix`, 12 commits)
 
-### Phase 2: Transport heartbeat defaults
-- 3 Schyntax strings converted to cron equivalents
+Eliminated `_lockSocket` contention in `TaskSchedulerJobCountSync` so `IncreaseCurrentTaskCount` / `DecreaseCurrentTaskCount` no longer stall up to 10ms behind the receive loop. Replaced the `TryReceiveFrameString(10ms)` polling pattern with a `NetMQPoller` that owns `_actor` on a single dedicated thread, and routed outbound `SetCount` messages through a `NetMQQueue<T>`.
 
-### Phase 3: Test schedule strings
-- 7 Schyntax strings converted across 2 test files, 878 tests pass
+**Outcome:** `_lockSocket = 0`, 9/9 unit tests green (concurrency 5/5 in loop), Release build clean. Audit CLEAN, Simplification production-clean (2 deferred), Documenter CHANGELOG draft ready for Phase 2.
 
-### Phase 4: CronExpressionDescriptor logging
-- Structured log statements in JobScheduler.cs when jobs are added
+### Phase 2: TaskScheduler NuGet 0.4.0 Release — SHIPPED
+**Repo:** `DotNetWorkQueue.TaskScheduling.Distributed.TaskScheduler`
 
-### Phase 5: Cleanup, docs, version bump
-- Deleted Lib/ directory, updated README/CLAUDE.md/CHANGELOG, version 0.9.30
+0.4.0 is live on nuget.org, published via GH Actions tag-triggered workflow mirroring 0.3.0. Release commit `b904ac3` (5 files, +23/-2), tag `v0.4.0` (annotated, unsigned). Both `.nupkg` and `.snupkg` published cleanly via run 24423676631. Symbols + deterministic Source Link confirmed green on nuget.org.
+
+**Deferred gates by explicit user decision:** audit / simplifier / documenter — Phase 2 diff was 5-file version/text/doc, no code logic changes, no new deps.
+
+**Closed:** ISSUE-028 (Start() remarks XML doc landed in release commit).
+**Opened:** ISSUE-029 (GH Actions Node.js 20 deprecation advisory).
+
+### Phase 3: DotNetWorkQueue Integration Test Project — COMPLETE
+**Repo:** `DotNetWorkQueue` (this repo)
+
+Created `Source/DotNetWorkQueue.TaskScheduling.Distributed.TaskScheduler.Integration.Tests/` as a net10.0-only test project that `PackageReference`s `DotNetWorkQueue.TaskScheduling.Distributed.TaskScheduler 0.4.0` via Central Package Management.
+
+**Three test classes:**
+- **ConcurrencyRegressionTests** (PLAN-2.2) — **critical cross-repo regression guard for Phase 1's lock fix.** 12 threads × 5000 iterations hammering `Increase/DecreaseCurrentTaskCount`, 30s deadlock detector via `Task.WaitAll`, final count asserted == 0 via FluentAssertions. `Start()` called before spawning threads so `_outbound` is initialized and the real concurrency path is exercised (without `Start()`, the null-safe guard from Phase 1 makes the test a false positive).
+- **NodeDiscoveryTests** (PLAN-2.3) — UDP beacon discovery convergence + disposal decay. Two `SchedulerContainer` instances share one port, assert `RemoteCountChanged` convergence and post-dispose decay.
+- **EndToEndSchedulingTests** (PLAN-2.1) — **scope-reduced during build** to a SimpleInjector `Verify()` smoke test. The shared `SimpleConsumer.Run<>()` runner has no `Action<IContainer>` seam for the distributed scheduler injection, Memory transport storage is per-container, and `SharedSetup` / `VerifyMetrics` / `Metrics.Metrics` are internal to the shared test project. Smoke test proves `InjectDistributedTaskScheduler` passes SimpleInjector `Verify()` in a real DNQ consumer container.
+
+**Spec correction during build:** `TargetFrameworks` changed from `net10.0;net8.0` → `net10.0` single-target. The plan's claim "matches the rest of DNQ's test projects" was factually wrong — every other DNQ integration test project is `net10.0`-only, and Jenkins CI runs `net10.0` only on ubuntu-latest.
+
+**5/5 flakiness loop:** full suite runs green consecutively, ~26s per run, zero flakes. **896/896 core unit tests** and **57/57 Memory integration tests** continue to pass (regression check).
+
+### Phase 4: CI Wiring (Jenkins + GitHub Actions) — SHIPPED (via PR #115)
+**Repo:** `DotNetWorkQueue`
+
+- **`.github/workflows/ci.yml`** +3 lines: appended `Integration Tests - TaskScheduler Distributed` step after the last unit-test step (first integration test in `ci.yml`).
+- **`Jenkinsfile`** +12 lines: appended `stage('TaskScheduler Distributed')` as the 14th parallel stage after `stage('Dashboard')` with `sleep(time: 65, unit: 'SECONDS')` following the `(n-1)*5` stagger formula.
+
+**Total Phase 4 diff:** 15 insertions, 0 deletions, 2 files. Strictly additive.
+
+**Optimistic UDP decision paid off:** NetMQ beacon discovery on Docker bridge network worked on the first Jenkins run of PR #115. No `[TestCategory("BeaconRequired")]` skip mechanism needed.
+
+**All 14 Jenkins parallel stages + GitHub Actions build-and-test job green** on the PR before merge.
 
 ## Key Decisions
 
-1. Reuse ScheduledJob.Window for Previous() lookback (no new config)
-2. Keep Func<DateTimeOffset> constructor param on JobSchedule
-3. Auto-detect cron format by field count
-4. Pin Cronos to 0.11.1 (0.12.0 had 0 downloads at time of work)
-5. Dashboard API scoped to logging only (DashboardJob lacks schedule expression field)
-6. Version 0.9.30 (0.9.3 < 0.9.19 in NuGet versioning)
+1. **Phase 2 — audit/simplifier/documenter deferred** (Phase 2 was a 5-file version/text/doc change with no code logic changes).
+2. **Phase 3 — target framework corrected to net10.0 single-target** mid-build after discovering the plan's "matches DNQ test projects" rationale was incorrect.
+3. **Phase 3 — EndToEndSchedulingTests scope reduced** to a SimpleInjector `Verify()` smoke test. The shared runner's seam limitation and Memory transport's per-container storage made the original "produce 50 messages, consume them" pattern infeasible without modifying shared test infrastructure. ConcurrencyRegressionTests is Phase 3's real regression guard.
+4. **Phase 4 — optimistic UDP multicast** (no pre-emptive `[TestCategory]` skip, no `--network=host`). Risk accepted; outcome: beacon worked on the first Jenkins run.
+5. **Phase 4 — new stage excluded from Codecov** (no `--collect`, no `--settings`, no `--results-directory`, no `stash`). The Phase 3 project tests an external NuGet; its ProjectReference DLLs are already covered by the other 13 Jenkins stages.
+6. **Phase 4 — stage appended at end** of Jenkins parallel block (stagger 65s), preserving the existing 13 stages' stagger offsets one-for-one.
+7. **Pre-ship — local master merged origin/master**. Local master had been working the TaskScheduler milestone in isolation while origin shipped Phase 5 of the dashboard-coverage milestone (PR #114). The two diverged; a merge commit unified them, then `phase-4-ci-wiring` was rebased onto the unified master so PR #115 could ship cleanly.
+
+## Documentation Status
 
-## Quality Gates
+- **Per-phase SUMMARY files:** complete for all plans across Phases 1-4 in `.shipyard/phases/{N}/results/`
+- **Per-phase REVIEW files:** complete, all PASS verdicts
+- **Per-phase VERIFICATION:** complete, all PASS
+- **Per-phase AUDIT:** Phase 1 (part of Phase 1 ship), Phase 3 (PASS, 0 blocking), Phase 4 (PASS, 0 blocking, 2 informational). Phase 2 audit deferred.
+- **Per-phase SIMPLIFICATION:** Phase 3 PASS_NO_ACTION, Phase 4 PASS_NO_ACTION
+- **Per-phase DOCUMENTATION:** Phase 3 + Phase 4 both PASS_NO_ACTION, deferred to ship time
+- **CLAUDE.md updates** (pending ship cleanup): 13 → 14 parallel Jenkins stages; new lessons from this milestone
+- **CHANGELOG:** Phase 2 shipped a release commit in the sibling repo with CHANGELOG + README updates for `v0.4.0`
 
-| Gate | Result |
-|------|--------|
-| Build (Debug + Release) | PASS |
-| Unit tests | 878/878 |
-| Security Audit | PASS |
-| Schyntax references | 0 in Source/ |
+## Known Issues
+
+**30 open issues** in `.shipyard/ISSUES.md` — accumulated across multiple milestones, preserved for the next cleanup pass. Notable Phase 3 / Phase 4 entries:
+- **ISSUE-030** — README usage example in sibling `TaskScheduler` repo uses the wrong named argument (`udpBroadcastPort:` instead of `broadCastPort:`). Workaround applied in Phase 3 tests: positional args only.
+- **Pre-existing `SYSLIB0012` warnings** in `LiteDB/SQLite ConnectionString.cs` — `Assembly.CodeBase` is obsolete in net10.0. Trivial cleanup, deferred.
+- **Jenkinsfile stagger at ceiling** — 14 stages × 5s = 65s worst-case startup delay. A future 15th stage will need formula revisit.
 
 ## Metrics
 
-- Commits: 15
-- Files modified: ~20
-- Files deleted: 7 (Lib/ directory)
+**Commits:** 20 on the TaskScheduler milestone branches (4 phases), plus 1 GitHub merge commit on master (`190f1226`).
+
+**Files created:**
+- `Source/DotNetWorkQueue.TaskScheduling.Distributed.TaskScheduler.Integration.Tests/` — 7 `.cs` files (AssemblyInit, TestHelpers, SharedClasses, EndToEndSchedulingTests, ConcurrencyRegressionTests, NodeDiscoveryTests, csproj)
+- `.shipyard/phases/1/` through `.shipyard/phases/4/` — plan + result artifacts
+
+**Files modified:**
+- `Source/Directory.Packages.props` — added CPM entry for `DotNetWorkQueue.TaskScheduling.Distributed.TaskScheduler 0.4.0`
+- `Source/DotNetWorkQueue.sln` — added new test project entry
+- `.github/workflows/ci.yml` — added integration test step
+- `Jenkinsfile` — added 14th parallel stage
+- `.gitignore` — added `.worktrees/` pattern
+
+**Regression check:** 896/896 core unit tests + 57/57 Memory integration tests + 4/4 new TaskScheduler Distributed integration tests — all green on the merged master head.

.shipyard/STATE.json

@@ -1,8 +1,8 @@
 {
   "schema": 3,
   "phase": 4,
-  "position": "Phase 4 build complete in worktree; pending CI validation on push",
-  "status": "complete",
-  "updated_at": "2026-04-15T16:53:01Z",
+  "position": "TaskScheduler 0.4.0 milestone shipped via PR #115",
+  "status": "shipped",
+  "updated_at": "2026-04-15T19:26:38Z",
   "blocker": null
 }