Merge pull request #670 from MachineWisdomAI/feature-hermes-consult-backend

feat(consult): add Hermes backend support + ignore .hermes artifacts

Author: waleedkadous

.gitignore

@@ -3,6 +3,7 @@ tmp/
 .builders/
 worktrees/
 .consult/
+.hermes/
 .agent-farm/
 .env
 .update-hashes.json

codev-skeleton/resources/commands/consult.md

@@ -1,6 +1,6 @@
 # consult - AI Consultation CLI
 
-The `consult` command provides a unified interface for AI consultation with external models (Gemini, Codex, Claude). It operates in three modes: general (ad-hoc prompts), protocol-based (structured reviews), and stats.
+The `consult` command provides a unified interface for AI consultation with external models (Gemini, Codex, Claude, Hermes). It operates in three modes: general (ad-hoc prompts), protocol-based (structured reviews), and stats.
 
 ## Synopsis
 
@@ -22,6 +22,7 @@ consult stats [options]
 | `gemini` | `pro` | gemini-cli | File access via --yolo, fast |
 | `codex` | `gpt` | @openai/codex | Read-only sandbox, thorough |
 | `claude` | `opus` | Claude Agent SDK | Balanced analysis with tool use |
+| `hermes` | - | hermes CLI (`hermes chat -q`) | Uses Hermes agent as consult backend |
 
 ## Modes
 
@@ -108,16 +109,21 @@ These flags are used by porch (the protocol orchestrator) when generating consul
 --project-id <id>       Project ID for metrics
 ```
 
-## Parallel Consultation (3-Way Reviews)
+## Parallel Consultation (Multi-Model Reviews)
+
+Default project configuration uses a 3-model set (`gemini`, `codex`, `claude`).
 
 For thorough reviews, run multiple models in parallel:
 
 ```bash
-# 3-way spec review
+# Default 3-way spec review
 consult -m gemini --protocol spir --type spec &
 consult -m codex --protocol spir --type spec &
 consult -m claude --protocol spir --type spec &
 wait
+
+# Optional: include Hermes as a 4th reviewer
+consult -m hermes --protocol spir --type spec
 ```
 
 ## Performance
@@ -183,12 +189,15 @@ consult -m codex --protocol spir --type pr --issue 42
 # Protocol: implementation review with bugfix protocol
 consult -m claude --protocol bugfix --type impl
 
-# 3-way parallel review
+# Default 3-way parallel review
 consult -m gemini --protocol spir --type spec &
 consult -m codex --protocol spir --type spec &
 consult -m claude --protocol spir --type spec &
 wait
 
+# Optional: include Hermes as an additional reviewer
+consult -m hermes --protocol spir --type spec
+
 # Stats
 consult stats --days 7 --json
 ```

codev/plans/671-hermes-consult-optional-backend.md

@@ -0,0 +1,64 @@
+# Plan 671: Hermes Consult Backend (Optional, Not Default)
+
+## Metadata
+- Specification: `codev/specs/671-hermes-consult-optional-backend.md`
+- Issue: https://github.com/cluesmith/codev/issues/671
+- PR: https://github.com/cluesmith/codev/pull/670
+
+## Executive Summary
+Implement Hermes as an additional consult backend with explicit model selection, preserve default 3-way consultation models, and document Hermes as optional. Ensure large prompts avoid ARG_MAX/E2BIG via temp-file indirection.
+
+## Phases (Machine Readable)
+
+```json
+{
+  "phases": [
+    {"id": "phase_1", "title": "Backend wiring and validation"},
+    {"id": "phase_2", "title": "Docs and tests alignment"}
+  ]
+}
+```
+
+## Phase Breakdown
+
+### Phase 1: Backend wiring and validation
+Objectives:
+- Add Hermes model config and routing in consult command.
+- Add Hermes to porch model validation.
+- Preserve default consultation model set.
+- Add large-prompt guard for CLI argument limits.
+
+Deliverables:
+- `packages/codev/src/commands/consult/index.ts`
+- `packages/codev/src/commands/porch/next.ts`
+- No default change in `packages/codev/src/lib/config.ts`
+
+Acceptance criteria:
+- Hermes path is reachable via `-m hermes`.
+- Large prompt path uses temp-file indirection.
+- Default consultation models remain `['gemini', 'codex', 'claude']`.
+
+### Phase 2: Docs and tests alignment
+Objectives:
+- Update CLI help and docs to include Hermes.
+- Ensure docs frame Hermes as optional.
+- Add/adjust tests for Hermes support and large prompt behavior.
+
+Deliverables:
+- `packages/codev/src/cli.ts`
+- `codev/resources/commands/consult.md`
+- `codev-skeleton/resources/commands/consult.md`
+- `packages/codev/src/__tests__/consult.test.ts`
+- `packages/codev/src/__tests__/cli/consult.e2e.test.ts`
+- `packages/codev/src/commands/consult/__tests__/persistent-output.test.ts`
+
+Acceptance criteria:
+- Source and skeleton docs are synchronized.
+- Examples show default 3-way and optional Hermes usage.
+- Test commands pass from `packages/codev/`.
+
+## Validation Commands
+Run from `packages/codev/`:
+- `pnpm build`
+- `pnpm exec vitest run src/commands/porch/__tests__/consultation-models.test.ts src/commands/consult/__tests__/persistent-output.test.ts src/__tests__/consult.test.ts`
+- `pnpm exec vitest run --config vitest.cli.config.ts src/__tests__/cli/consult.e2e.test.ts`

codev/projects/671-hermes-consult-optional-backend/671-plan-iter1-rebuttals.md

@@ -0,0 +1,29 @@
+id: '671'
+title: hermes-consult-optional-backend
+protocol: spir
+phase: review
+plan_phases:
+  - id: phase_1
+    title: Backend wiring and validation
+    status: complete
+  - id: phase_2
+    title: Docs and tests alignment
+    status: complete
+current_plan_phase: null
+gates:
+  spec-approval:
+    status: approved
+    requested_at: '2026-04-15T20:13:26.446Z'
+    approved_at: '2026-04-15T20:13:36.692Z'
+  plan-approval:
+    status: approved
+    requested_at: '2026-04-15T20:16:18.381Z'
+    approved_at: '2026-04-15T20:16:25.454Z'
+  pr:
+    status: pending
+    requested_at: '2026-04-15T21:49:59.445Z'
+iteration: 1
+build_complete: false
+history: []
+started_at: '2026-04-15T16:11:54.376Z'
+updated_at: '2026-04-15T21:49:59.446Z'

codev/projects/671-hermes-consult-optional-backend/671-specify-iter1-rebuttals.md

@@ -1,6 +1,6 @@
 # consult - AI Consultation CLI
 
-The `consult` command provides a unified interface for AI consultation with external models (Gemini, Codex, Claude). It operates in three modes: general (ad-hoc prompts), protocol-based (structured reviews), and stats.
+The `consult` command provides a unified interface for AI consultation with external models (Gemini, Codex, Claude, Hermes). It operates in three modes: general (ad-hoc prompts), protocol-based (structured reviews), and stats.
 
 ## Synopsis
 
@@ -22,6 +22,7 @@ consult stats [options]
 | `gemini` | `pro` | gemini-cli | File access via --yolo, fast |
 | `codex` | `gpt` | @openai/codex | Read-only sandbox, thorough |
 | `claude` | `opus` | Claude Agent SDK | Balanced analysis with tool use |
+| `hermes` | - | hermes CLI (`hermes chat -q`) | Uses Hermes agent as consult backend |
 
 ## Modes
 
@@ -108,16 +109,21 @@ These flags are used by porch (the protocol orchestrator) when generating consul
 --project-id <id>       Project ID for metrics
 ```
 
-## Parallel Consultation (3-Way Reviews)
+## Parallel Consultation (Multi-Model Reviews)
+
+Default project configuration uses a 3-model set (`gemini`, `codex`, `claude`).
 
 For thorough reviews, run multiple models in parallel:
 
 ```bash
-# 3-way spec review
+# Default 3-way spec review
 consult -m gemini --protocol spir --type spec &
 consult -m codex --protocol spir --type spec &
 consult -m claude --protocol spir --type spec &
 wait
+
+# Optional: include Hermes as a 4th reviewer
+consult -m hermes --protocol spir --type spec
 ```
 
 ## Performance
@@ -183,12 +189,15 @@ consult -m codex --protocol spir --type pr --issue 42
 # Protocol: implementation review with bugfix protocol
 consult -m claude --protocol bugfix --type impl
 
-# 3-way parallel review
+# Default 3-way parallel review
 consult -m gemini --protocol spir --type spec &
 consult -m codex --protocol spir --type spec &
 consult -m claude --protocol spir --type spec &
 wait
 
+# Optional: include Hermes as an additional reviewer
+consult -m hermes --protocol spir --type spec
+
 # Stats
 consult stats --days 7 --json
 ```

codev/projects/671-hermes-consult-optional-backend/status.yaml

@@ -0,0 +1,109 @@
+# Review 671: Hermes Consult Backend (Optional, Not Default)
+
+## Summary
+Implemented Hermes as an additional consult backend and kept default consultation fan-out unchanged. Added large-prompt protection for Hermes to avoid CLI argument length failures. Updated docs and tests accordingly.
+
+## Spec Compliance
+- [x] Hermes backend available via `consult -m hermes`.
+- [x] Porch validation accepts `hermes`.
+- [x] Default consultation models remain `gemini`, `codex`, `claude`.
+- [x] Large Hermes prompts use temp-file indirection (ARG_MAX/E2BIG mitigation).
+- [x] Source/skeleton docs synchronized and clarified as optional Hermes.
+- [x] Tests updated for acceptance and large-prompt behavior.
+
+## Implementation Notes
+Key changes landed in:
+- `packages/codev/src/commands/consult/index.ts`
+- `packages/codev/src/commands/porch/next.ts`
+- `packages/codev/src/cli.ts`
+- `codev/resources/commands/consult.md`
+- `codev-skeleton/resources/commands/consult.md`
+- `.gitignore`
+- test files under `packages/codev/src/__tests__/...`
+
+## Validation
+Executed targeted build/test flow from `packages/codev/`:
+- `pnpm build`
+- `pnpm exec vitest run src/commands/porch/__tests__/consultation-models.test.ts src/commands/consult/__tests__/persistent-output.test.ts src/__tests__/consult.test.ts`
+- `pnpm exec vitest run --config vitest.cli.config.ts src/__tests__/cli/consult.e2e.test.ts`
+
+All passed at execution time.
+
+## Deviations and Corrections
+- Initial docs examples incorrectly implied 4-way default review.
+- Corrected docs to explicitly show 3-way default and Hermes as optional.
+- Initial implementation drifted from the spec by treating Hermes as a default consultation backend. Review feedback corrected this before merge by restoring 3-way defaults and moving Hermes coverage to explicit opt-in paths.
+
+## Consultation Feedback
+
+### Specify Phase (Round 1)
+
+#### Gemini
+- No concerns raised — approved.
+
+#### Codex
+- No concerns raised — approved.
+
+#### Claude
+- No concerns raised — approved.
+
+#### Hermes
+- No concerns raised — approved.
+
+### Plan Phase (Round 1)
+
+#### Gemini
+- No concerns raised — approved.
+
+#### Codex
+- No concerns raised — approved.
+
+#### Claude
+- No concerns raised — approved.
+
+#### Hermes
+- No concerns raised — approved.
+
+### Implement Phase 1 (Round 1)
+
+#### Gemini
+- No concerns raised — approved.
+
+#### Codex
+- No concerns raised — approved.
+
+#### Claude
+- No concerns raised — approved.
+
+#### Hermes
+- No concerns raised — approved.
+
+### Implement Phase 2 (Round 1)
+
+#### Gemini
+- **Concern**: Hermes must remain optional rather than becoming part of the default fan-out.
+  - **Addressed**: Restored the default consultation models to `gemini`, `codex`, `claude` and reverted default-facing fixtures to 3-way behavior.
+
+#### Codex
+- **Concern**: `afx bench` must not require Hermes by default.
+  - **Addressed**: Reverted `ENGINES` to the 3-engine default set and updated the bench tests accordingly.
+
+#### Claude
+- **Concern**: Tests should distinguish between allowed backends and default backends.
+  - **Addressed**: Kept Hermes in `VALID_MODELS`, restored 3-way default assertions, and added explicit opt-in Hermes test coverage.
+
+## Architecture Updates
+No architecture updates needed. The final change preserves the existing consultation architecture: Hermes remains an allowed backend, but the default orchestration and benchmark topology stay unchanged.
+
+## Lessons Learned Updates
+No lessons learned updates needed. This correction reinforces existing lessons already recorded in `codev/resources/lessons-learned.md` about preserving defaults unless product intent explicitly changes and keeping allowlists distinct from defaults.
+
+## Flaky Tests
+- No flaky tests encountered in the scoped validation flow used for this project.
+
+## Pull Request
+- Existing PR used for this work: `#670`
+
+## Follow-ups
+- If additional CLI backends are added later, keep default fan-out stable unless explicitly changed by product decision.
+- Preserve large-prompt transport tests for each CLI backend to prevent ARG_MAX regressions.

codev/resources/commands/consult.md

@@ -0,0 +1,40 @@
+# Spec 671: Hermes Consult Backend (Optional, Not Default)
+
+## Problem
+Codev consult supported Gemini, Codex, and Claude, but did not support Hermes. Teams that use Hermes (including pay-as-you-go workflows) could not use `consult -m hermes`.
+
+A second problem appeared during rollout: documentation examples drifted toward 4-way default review, which conflicts with Codev defaults and intended model positioning.
+
+## Desired State
+Add Hermes as an additional consult backend while preserving current default behavior.
+
+## Requirements
+1. `consult -m hermes` must be supported in CLI routing.
+2. Porch consultation model validation must accept `hermes` when explicitly configured.
+3. Default consultation fan-out must remain unchanged (`gemini`, `codex`, `claude`).
+4. Hermes prompt transport must avoid ARG_MAX/E2BIG failures for large prompts.
+5. Docs must present Hermes as optional and keep source/skeleton docs synchronized.
+6. Tests must cover model acceptance and large-prompt behavior.
+
+## Non-Goals
+- Changing default consultation models to 4-way.
+- Adding shorthand alias for Hermes in this iteration.
+
+## Success Criteria
+- [ ] Hermes model works in consult command.
+- [ ] Large prompts are handled without inline-arg overflow failures.
+- [ ] Default model configuration remains 3-way.
+- [ ] Docs in `codev/` and `codev-skeleton/` match and state Hermes is optional.
+- [ ] Relevant tests pass.
+
+## Risks
+- Documentation can drift from runtime defaults.
+- Backend-specific prompt transport can regress and reintroduce E2BIG.
+
+## Mitigations
+- Add explicit tests for Hermes large-prompt path.
+- Keep docs examples aligned with `packages/codev/src/lib/config.ts` defaults.
+
+## References
+- Issue: https://github.com/cluesmith/codev/issues/671
+- PR: https://github.com/cluesmith/codev/pull/670

codev/reviews/671-hermes-consult-optional-backend.md

@@ -125,6 +125,11 @@ describe('consult command (CLI)', () => {
     expect(result.status).toBe(0);
   });
 
+  it('accepts --model hermes option', () => {
+    const result = runConsult(['--model', 'hermes', '--help'], env.dir, env.env);
+    expect(result.status).toBe(0);
+  });
+
   // === Input Validation ===
 
   it('--type without --model fails', () => {