Skip to content

prompt(skill+memory): enforce sequential destructive pair + final-text honesty - #12

Merged
RowitZou merged 1 commit into
mainfrom
fix/skill-memory-curator-prompts
May 26, 2026
Merged

prompt(skill+memory): enforce sequential destructive pair + final-text honesty#12
RowitZou merged 1 commit into
mainfrom
fix/skill-memory-curator-prompts

Conversation

@RowitZou

Copy link
Copy Markdown
Owner

Summary

Closes the prompt-side root causes (#2 + #6) from the 2026-05-26 dogfood Bug 1 — companion to #11 which physically blocked the silent data loss with a same-name destructive guard.

The dogfood signal: skillConsolidator emitted SkillWrite(name:X, overwrite:true) + SkillDelete(name:X) in one batch, SkillWrite failed validation, SkillDelete ran anyway, user's prior on-disk skill silently gone. Model final text: "nothing to merge this pass" — it didn't even notice it had destroyed user data.

Prompt changes

skillConsolidator.ts

Step 2 — explicit "single skill is never a merge candidate" guard. Today's degenerate case was a size-1 group treated as X merges into X.

Step 3 split into 3.a / 3.b / 3.c with hard sequencing:

  • 3.a SkillWrite is a standalone tool_use; wait for is_error:false
  • 3.b deletes only run after that confirmation, and never delete the surviving name
  • 3.c on is_error:true, abort the group; no blind retry
  • "Do not bundle 3.a and 3.b in the same assistant turn"

Output discipline — final text must enumerate SkillWrite + SkillDelete success counts plus any aborted groups. Explicit Never say "nothing to merge" if you called any SkillWrite or SkillDelete — that exact phrase is what the model used while it had just deleted the user's only skill.

memoryCurator.ts

Same sequencing applied to the MemoryWriteAt + MemoryDelete pair:

  • Cross-role promotion: "first MemoryWriteAt ... as a standalone tool_use, wait for is_error:false, then MemoryDelete the originals. If the MemoryWriteAt returns is_error:true, abort the promotion — do not delete any originals."
  • Within-directory cleanup: same chain
  • Output discipline: enumerate destructive op counts; never claim "nothing changed" when destructive tools fired

Test plan

  • Snapshot test hashes (role-prompt.snapshot.test.ts + prompt/prompt-for-role.test.ts) updated for the 2 roles — deliberate semantic change, not a structural refactor (per [feedback_prompt_changes]).
  • New curator-prompt-contract.test.ts (13 cases) anchors the key sequencing phrases as intent guards on top of the snapshot's byte pin. If a future rewrite forgets the sequence-after-success rule, contract tests fail loudly with which phrase is missing.
  • pnpm test 1724/1724 PASS (1711 baseline + 13 new contract cases)
  • pnpm typecheck clean
  • pnpm build clean
  • Manual: dogfood — let the curators run a dream pass; confirm skillConsolidator no longer emits same-name SkillDelete, and confirm final text now enumerates op counts honestly

Relationship to #11

#11 (merged-ready) This PR
Layer tool / parser / observability prompt
Mechanism (userId, name) TTL refuses same-name delete after recent write failure; flow-style YAML now parses; actionable error msgs; stderr Tells the model to sequence the pair and own up to destructive ops
Why both Defense-in-depth: prompt makes the right thing easy, guard catches the wrong thing

After both ship, the dogfood failure cannot recur even if any one layer regresses.

🤖 Generated with Claude Code

…t honesty

PR-A (#11) physically blocked the silent data loss with a same-name
destructive guard; this PR closes the prompt-side root causes from the
2026-05-26 dogfood so the model stops emitting the destructive pair in
the first place.

skillConsolidator workflow (`src/agents/bundled/skillConsolidator.ts`):
- Step 2: "A single skill is never a merge candidate" — explicit guard
  against today's degenerate case (model treated size=1 group as
  X-merges-into-X and SkillDelete'd the same name).
- Step 3 split:
  3.a SkillWrite is a standalone tool_use; wait for `is_error:false`.
  3.b deletes only run after that confirmation, and **never** delete
      the surviving name.
  3.c on `is_error:true`, abort the group; do not retry blindly.
  Bundling 3.a and 3.b in the same assistant turn is forbidden.
- Output discipline: final text must enumerate SkillWrite and
  SkillDelete success counts plus any aborted groups. Forbids
  "nothing to merge" if any destructive tool fired — that exact phrase
  is what the model used in the dogfood while it had just deleted the
  user's only skill.

memoryCurator (`src/agents/bundled/memoryCurator.ts`): mirrors the same
sequencing for MemoryWriteAt + MemoryDelete (both in cross-role
promotion and within-directory cleanup), plus the same final-text
honesty rule.

Snapshot test hashes (`role-prompt.snapshot.test.ts` +
`prompt/prompt-for-role.test.ts`) updated for memoryCurator
(22c073df → c37344ca) and skillConsolidator (3e45d127 → c2700894);
deliberate semantic change, not a structural refactor — see
[feedback_prompt_changes].

New semantic contract test (`curator-prompt-contract.test.ts`, 13
cases) anchors the key sequencing phrases so a future rewrite that
forgets the ordering rules fails loudly. These are intent guards on
top of the snapshot's byte-level pin.

Test plan:
- pnpm test: 1724/1724 PASS (1711 from main + 13 new contract cases)
- pnpm typecheck clean
- pnpm build clean
- Re-confirmed the cleanup-line regex matches by running just that one
  test before adding the rest

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 26, 2026 03:55

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the bundled skillConsolidator and memoryCurator role prompts to prevent destructive “write+delete” pairs from executing out of order (or after a failed write), and tightens final-text reporting so the agent acknowledges destructive operations.

Changes:

  • Update skillConsolidator prompt to require sequential SkillWrite → (only on success) SkillDelete, forbid same-name delete, and improve “nothing to merge” honesty rules.
  • Update memoryCurator prompt to require sequential MemoryWriteAt → (only on success) MemoryDelete, and add honest final-text reporting rules.
  • Update snapshot hashes and add a new contract test suite to pin key sequencing/intent phrases beyond byte-level snapshots.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/agents/bundled/skillConsolidator.ts Adds explicit sequencing + safety rules for SkillWrite/SkillDelete and more honest final output requirements.
src/agents/bundled/memoryCurator.ts Adds explicit sequencing rules for MemoryWriteAt/MemoryDelete and honest final output requirements.
src/agents/bundled/curator-prompt-contract.test.ts New intent/contract tests to ensure critical sequencing phrases remain present in prompts.
src/agents/role-prompt.snapshot.test.ts Updates expected snapshot hash for prompt semantic changes.
src/prompt/prompt-for-role.test.ts Updates expected snapshot hash for prompt semantic changes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +58 to +59
' a. Call `SkillWrite(name:<surviving>, overwrite:true, ...)` with the',
" merged markdown. Frontmatter `roles` is the union of the group's",
@RowitZou
RowitZou merged commit 4ebe813 into main May 26, 2026
1 check passed
@RowitZou
RowitZou deleted the fix/skill-memory-curator-prompts branch May 26, 2026 12:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants