diff --git a/shift/README.md b/shift/README.md index 83f01f4..1186812 100644 --- a/shift/README.md +++ b/shift/README.md @@ -45,9 +45,10 @@ It merges the entry below (safe globally — the hook no-ops in any repo without ```bash cd your-repo -mkdir queue && $EDITOR queue/01-first-task.md # one brief per file -shift start --dry-run # preview the queue, branch, bounds -shift start # init run + create shift/ branch +shift init # scaffold queue/ + a template brief, and gitignore .shift/ +$EDITOR queue/01-example.md # write your first brief (one bin per file) — see "Writing bins" +shift start --dry-run # preview the queue, branch, bounds +shift start # init the run + create the shift/ branch ``` Then either: @@ -62,6 +63,31 @@ shift stop # stop cleanly after the current bin When it ends, read `.shift/summary.md` (bins done/blocked/skipped + a "Needs you" section) and review the `shift/` branch. +## Writing bins + +A **bin** is one Markdown file = one unit of work. The shape that works well unattended: + +```markdown +# Short title for the task + +What to do, in plain language. Be specific about scope and any constraints — +the agent runs with no chance to ask follow-up questions. + +Definition of done: how to tell this bin is complete (e.g. "the endpoint returns +200 with the new field and `npm test` passes; change committed on the run branch"). +``` + +- **One bin per file**, discovered in order (source folder, then filename) — so `queue/01-…`, `queue/02-…` set the sequence. +- **The `Definition of done:` line is the load-bearing part.** Unattended, the agent can't ask "is this what you meant?" — a crisp, checkable done-condition is what keeps a bin on-target. (For a hard gate, set `verify.command` so a bin only counts as done when e.g. `npm test` passes.) +- **Scope each bin to one reviewable change.** Smaller bins → cleaner per-bin commits and a tidier `shift/` diff. +- **Multiple sources**: point `sources` at more than one folder — e.g. hand-written `queue/` plus a plugin's `docs/superpowers/plans/`. They're treated identically; `kind` only frames defaults. + +### Self-generated / dynamic work + +shift re-discovers its source folders on **every cycle**, so a bin can grow the backlog: any new `queue/NN-*.md` an agent writes mid-run is picked up as a fresh pending bin and worked in turn. This is always true (it's how new files added between `shift start` and runtime get in), but by default the agent isn't *told* it may do this. + +Set **`"allowSelfQueue": true`** in `.shift/config.json` to invite it — the brief then tells the agent it may queue genuine follow-ups as `queue/NN-.md`. It's bounded by `maxIterations`, branch isolation, and your end-of-run review, so a run can't recurse forever. Leave it off (the default) for a fixed, predictable queue. + ## Watch it live + steer it (`shift watch`) An unattended run is the *least* transparent mode there is — so `shift` gives you a live window into it. In a second terminal: @@ -122,6 +148,7 @@ For an at-a-glance signal in the [Code Status Bar](../code-status-bar), `shift s "definitionOfDone": "Builds and tests pass; work committed on the run branch.", "verify": { "command": "npm test", "maxAttempts": 2 }, "permissionMode": "acceptEdits", + "allowSelfQueue": false, "git": { "branch": "shift/{date}", "allowPush": false, "allowOutwardActions": false } } ``` @@ -131,6 +158,7 @@ For an at-a-glance signal in the [Code Status Bar](../code-status-bar), `shift s - **`maxResumes`** — the runner's own backstop on the number of `claude` spawns (independent of the hook-maintained `maxIterations`/`maxHours`). - **`spawnTimeoutMinutes`** — hard per-spawn wall: a wedged `claude` is killed (SIGTERM) so it can't hang the runner. Default 30. - **`verify.command`** — per-bin acceptance gate; `null` disables it. +- **`allowSelfQueue`** — when `true`, the brief invites the agent to queue follow-up bins (`queue/NN-*.md`); see "Self-generated / dynamic work." Default `false`. > A headless `shift run` grades success on `.shift/summary.md` (written only when the engine finalizes), not on the exit line: a `claude -p` that exits without finalizing is reported as *"no summary written — did NOT finalize"* with a hint to check the hook wiring, never as a false success. @@ -143,6 +171,34 @@ For an at-a-glance signal in the [Code Status Bar](../code-status-bar), `shift s Pick the narrowest mode that lets the work actually proceed. +## Using shift with your `CLAUDE.md` + +When shift drives a session, the agent reads your repo's `CLAUDE.md` on top of shift's injected brief — so `CLAUDE.md` is where you set the *house style* for unattended runs. The brief sets the non-negotiable rules; `CLAUDE.md` tunes how the work gets done. A block like this earns its keep: + +````markdown +## Working under shift (unattended runs) + +- Keep each bin to one focused, reviewable change; commit it on the run branch with a + clear message. Don't fold unrelated work into one commit. +- A bin is done only when its "Definition of done" is met and `npm test` passes — if it + doesn't, fix it; don't mark it done. +- Never push, open PRs, or touch anything outside this repo. If a step needs that, append + `Needs you: ` to `.shift/log.md` and move on. +- Prefer the smallest change that satisfies the bin; record loose ends as `Needs you:` + notes rather than sprawling. +- (If `allowSelfQueue` is on) when you find genuine follow-up work, add it as + `queue/NN-.md` rather than expanding the current bin. +```` + +Two things worth knowing: + +- **Don't restate the safety rules.** shift's brief already forbids push/outward actions and protects its bookkeeping; `CLAUDE.md` is for *preferences* (commit style, test discipline, scope) that the brief deliberately leaves to you. +- **`CLAUDE.md` can also make the agent reach for shift** — a line like *"when the user has a batch of independent tasks, offer to set them up as a shift queue"* turns it into a tool your sessions suggest, not just one you remember. + +### Optional: a nudge hook (encourage usage) + +The only hook shift needs is the `Stop` engine. If you want active encouragement, a small `SessionStart` hook can surface an idle queue — *"this repo has N pending shift bins; `begin the shift` or `shift run`."* Genuinely helpful, but it can nag, so it's intentionally **not** part of `install.sh` — add it yourself if you want it. Easing the *start* is better handled by `shift init` than by a hook. + ## Develop ```bash diff --git a/shift/SPEC.md b/shift/SPEC.md index 132ec97..8c24761 100644 --- a/shift/SPEC.md +++ b/shift/SPEC.md @@ -296,4 +296,12 @@ Per the candor goal of making consumption legible: the dashboard header and `sta ### Engine state moved out of the repo — per-bin attribution made robust (2026-06-16) -The first cut of per-bin attribution was unreliable headless, and the investigation found the true cause: an autonomous agent **rewrites `.shift/state.json` to mark bins done itself** (and rewrites `log.md`, deletes `config.json`/`timeline.jsonl`) — usurping the keep-going engine so the hook never drives the queue and records no boundaries. A probe hook (one real `claude -p` run) then **disproved a sandbox**: a Stop hook can write anywhere, including `~/.local/state` and `/tmp`. So the fix is an **engine-owned store outside the working repo**, in `lib/store.cjs`: `engineDir(cwd)` = `$XDG_STATE_HOME/shift/` (canonicalized so `/tmp` and `/private/tmp` agree; full-path hash so siblings don't collide; `SHIFT_STATE_DIR` overrides). `state.json`, `usage.json`, `history.jsonl`, and the timeline now live there — the hook owns them and the agent (which only operates inside the repo) can't see or touch them. `.shift/` keeps only what the user/agent legitimately use: `config.json` (user-edited, also snapshotted into the engine dir so a deletion can't break a run), `summary.md` (user-read), `log.md`/`blocked.jsonl` (agent-appended), and `STOP`/`PAUSE`/`SKIP` (control). The engine is also robust if the agent *does* still write a stray `.shift/state.json` — that file is simply ignored. **Validated:** a real fully-headless `bypassPermissions` run now records per-bin runtime + tokens for every bin (e.g. `35s · 7k`, `13s · 2k`) and a complete history row. **Tests:** 99 in `shift`, all green. +The first cut of per-bin attribution was unreliable headless, and the investigation found the true cause: an autonomous agent **rewrites `.shift/state.json` to mark bins done itself** (and rewrites `log.md`, deletes `config.json`/`timeline.jsonl`) — usurping the keep-going engine so the hook never drives the queue and records no boundaries. A probe hook (one real `claude -p` run) then **disproved a sandbox**: a Stop hook can write anywhere, including `~/.local/state` and `/tmp`. So the fix is an **engine-owned store outside the working repo**, in `lib/store.cjs`: `engineDir(cwd)` = `$XDG_STATE_HOME/shift/` (canonicalized so `/tmp` and `/private/tmp` agree; full-path hash so siblings don't collide; `SHIFT_STATE_DIR` overrides). `state.json`, `usage.json`, `history.jsonl`, and the timeline now live there — the hook owns them and the agent (which only operates inside the repo) can't see or touch them. `.shift/` keeps only what the user/agent legitimately use: `config.json` (user-edited, also snapshotted into the engine dir so a deletion can't break a run), `summary.md` (user-read), `log.md`/`blocked.jsonl` (agent-appended), and `STOP`/`PAUSE`/`SKIP` (control). The engine is also robust if the agent *does* still write a stray `.shift/state.json` — that file is simply ignored. **Validated:** a real fully-headless `bypassPermissions` run now records per-bin runtime + tokens for every bin (e.g. `35s · 7k`, `13s · 2k`) and a complete history row. + +### Onboarding + self-generated work (2026-06-17) + +- **`shift init`** scaffolds a repo for shift: `queue/` + a template brief (the bin anatomy) and `.shift/` added to `.gitignore`. Removes the activation friction (the highest-leverage "easier to get going" lever — more than any doc or extra hook). +- **Self-generated / dynamic work.** Re-discovery of source folders on every cycle already means any new `queue/NN-*.md` an agent writes mid-run becomes a pending bin — this is inherent and always on. New config flag **`allowSelfQueue`** (default `false`) makes `renderBrief` *invite* the agent to queue genuine follow-ups; bounded by `maxIterations` + branch isolation + review. Default-off keeps a fixed, predictable queue; opt in for "grow your own backlog." +- **Docs.** README gains a "Writing bins" section (brief anatomy + the `Definition of done` discipline), the self-generated-work explanation, and a "Using shift with your `CLAUDE.md`" section (a paste-in *house-style* block for unattended runs + the deliberate non-duplication of the brief's safety rules). The only required hook remains `Stop`; a `SessionStart` nudge is documented as an optional, opt-in recipe rather than shipped (nag risk). + +**Tests:** 120 in `shift`, all green. diff --git a/shift/bin/shift b/shift/bin/shift index 40d8b83..c98af6a 100755 --- a/shift/bin/shift +++ b/shift/bin/shift @@ -23,6 +23,7 @@ const DEFAULT_CONFIG = { definitionOfDone: 'Builds and tests pass; work committed on the run branch.', verify: { command: null, maxAttempts: 2 }, permissionMode: 'acceptEdits', + allowSelfQueue: false, // when true, the brief invites the agent to queue follow-up bins (bounded by maxIterations) git: { branch: 'shift/{date}', allowPush: false, allowOutwardActions: false } }; @@ -38,6 +39,35 @@ function ensureBranch(cwd, branch) { return false; } +// Scaffold a repo for shift: a queue/ with a template brief, and .shift/ gitignored. +// Lowers the "how do I even start a queue" friction; safe + idempotent (never clobbers). +function cmdInit() { + const cwd = process.cwd(); + fs.mkdirSync(path.join(cwd, 'queue'), { recursive: true }); + const example = path.join(cwd, 'queue', '01-example.md'); + if (!fs.existsSync(example)) { + fs.writeFileSync(example, [ + '# ', + '', + '', + '', + 'Definition of done: ', + '' + ].join('\n')); + console.log('created queue/01-example.md (edit it; add more as queue/NN-*.md)'); + } else { + console.log('queue/01-example.md already exists — left as-is'); + } + const gi = path.join(cwd, '.gitignore'); + let body = ''; + try { body = fs.readFileSync(gi, 'utf8'); } catch { /* none */ } + if (!body.split('\n').some(l => l.trim() === '.shift/' || l.trim() === '.shift')) { + fs.writeFileSync(gi, body + (body && !body.endsWith('\n') ? '\n' : '') + '.shift/\n'); + console.log('added .shift/ to .gitignore'); + } + console.log('next: edit your briefs, then `shift start --dry-run` to preview, then `shift start`.'); +} + function cmdStart(args) { const cwd = process.cwd(); const dir = path.join(cwd, '.shift'); @@ -276,14 +306,16 @@ async function cmdRun() { } const [, , sub, ...rest] = process.argv; -if (sub === 'start') cmdStart(rest); +if (sub === 'init') cmdInit(); +else if (sub === 'start') cmdStart(rest); else if (sub === 'status') cmdStatus(rest); else if (sub === 'watch') cmdWatch(); else if (sub === 'history') cmdHistory(rest); else if (sub === 'stop') cmdStop(); else if (sub === 'run') cmdRun().catch(e => { console.error(e); process.exit(1); }); else { - console.log('usage: shift [--dry-run]'); + console.log('usage: shift [--dry-run]'); + console.log(' init scaffold queue/ + a template brief + gitignore .shift/'); console.log(' watch live dashboard + control: ↑/↓ select · ⏎ details · [p]ause [k]skip [q]stop [x]exit'); console.log(' history [run] the work record: per-run runtime/tokens + totals (or one run\'s detail)'); console.log(' status --line one-line summary for a status bar'); diff --git a/shift/lib/brief.cjs b/shift/lib/brief.cjs index 02584c3..36b6463 100644 --- a/shift/lib/brief.cjs +++ b/shift/lib/brief.cjs @@ -10,11 +10,18 @@ function renderBrief(bin, config) { const guard = forbidden.length ? `Do NOT ${forbidden.join(', or ')}; if the work needs one, treat it as a "Needs you" item (below) and continue with the rest.` : ''; + // Opt-in (config.allowSelfQueue): invite the agent to grow its own backlog. New .md files + // in a source folder are discovered on the next cycle regardless; this just tells the agent + // it MAY do so. Bounded by maxIterations + branch isolation + your end-of-run review. + const selfQueue = (config && config.allowSelfQueue) + ? 'If finishing this bin surfaces genuine follow-up work, you MAY queue it: write a new brief to a source folder as `queue/NN-.md` (NN after the current files) — shift will pick it up as a new bin. Use this sparingly for real follow-ups, not to defer the current bin.' + : ''; return [ 'You are running unattended under `shift`. Complete the brief below end-to-end using your best judgment.', 'Do NOT ask questions — if you would normally ask, decide and APPEND the decision as a line to .shift/log.md.', `Definition of done: ${dod}`, 'When finished, commit your work on the current branch.', + selfQueue, '`.shift/` is shift\'s own run bookkeeping. The ONLY writes you may make under it are APPENDING a line to .shift/log.md or .shift/blocked.jsonl. Never edit, overwrite, or "tidy" .shift/config.json or .shift/summary.md, and never rewrite .shift/log.md — shift maintains those itself (run progress, per-bin runtime + tokens), and changing them corrupts the run record. (Authoritative engine state — run progress, usage, timeline, history — lives outside the repo and is maintained by shift; you do not need to touch it.)', 'Flag anything that needs the human (a deferred decision, an action you could not take) by appending a line to .shift/log.md as: "Needs you: " — these surface in the run summary.', 'If a true blocker stops you from finishing this bin, append one line to .shift/blocked.jsonl: {"id":"","note":""} then stop.', diff --git a/shift/test/brief.test.cjs b/shift/test/brief.test.cjs index 47ef892..9f91ea3 100644 --- a/shift/test/brief.test.cjs +++ b/shift/test/brief.test.cjs @@ -30,6 +30,13 @@ test('always explains decision logging, the Needs-you convention, and blocker fl assert.match(out, /blocked\.jsonl/); }); +test('allowSelfQueue invites follow-up bins only when enabled', () => { + assert.doesNotMatch(renderBrief(bin, { git: {} }), /queue it|queue\/NN/); + const out = renderBrief(bin, { git: {}, allowSelfQueue: true }); + assert.match(out, /queue\/NN-\.md/); + assert.match(out, /MAY queue/); +}); + test('the forbid-guard reflects each git flag combination independently', () => { const pushOnly = renderBrief(bin, { git: { allowPush: false, allowOutwardActions: true } }); assert.match(pushOnly, /Do NOT push to any remote/); diff --git a/shift/test/cli.test.cjs b/shift/test/cli.test.cjs index 04acb79..a378b1f 100644 --- a/shift/test/cli.test.cjs +++ b/shift/test/cli.test.cjs @@ -69,6 +69,19 @@ test('a second `shift start` scrubs stale control/blocker signals from the prior } }); +test('init scaffolds queue/ + a template brief + gitignores .shift/ (idempotent, no clobber)', () => { + const cwd = fs.mkdtempSync(path.join(os.tmpdir(), 'shift-init-')); + run(cwd, ['init']); + assert.ok(fs.existsSync(path.join(cwd, 'queue', '01-example.md'))); + assert.match(fs.readFileSync(path.join(cwd, 'queue', '01-example.md'), 'utf8'), /Definition of done:/); + assert.match(fs.readFileSync(path.join(cwd, '.gitignore'), 'utf8'), /^\.shift\/$/m); + // re-run: must not clobber an edited brief nor duplicate the gitignore line + fs.writeFileSync(path.join(cwd, 'queue', '01-example.md'), '# edited\n'); + run(cwd, ['init']); + assert.equal(fs.readFileSync(path.join(cwd, 'queue', '01-example.md'), 'utf8'), '# edited\n'); + assert.equal((fs.readFileSync(path.join(cwd, '.gitignore'), 'utf8').match(/\.shift\//g) || []).length, 1); +}); + const { appendRecord } = require('../lib/history.cjs'); function runSafe(cwd, args) { // capture output + exit code even on non-zero exit @@ -130,7 +143,7 @@ test('history drills into one run; a branch suffix resolves; unknown -> test('unknown subcommand prints usage and exits non-zero', () => { const r = runSafe(repoWithQueue(), ['bogus']); assert.equal(r.code, 1); - assert.match(r.out, /usage: shift /); + assert.match(r.out, /usage: shift /); }); test('start shallow-merges a partial .shift/config.json over the defaults', () => {