Skip to content

M2: MongoDB Search-native engine, no fallback - #123

Merged
romiluz13 merged 124 commits into
mainfrom
integration/m2-20261019
Oct 19, 2026
Merged

romiluz13 merged 124 commits into
mainfrom
integration/m2-20261019

Conversation

@romiluz13

Copy link
Copy Markdown
Owner

M2: MongoDB Search-native engine, no fallback. Spec #109; tickets #110-#122.

The engine requires MongoDB 8.3+ with Search and autoEmbed and refuses to start otherwise. Content matching runs only in $search, $vectorSearch, $rankFusion and $scoreFusion, and every failure is typed instead of degrading. See BEHAVIOR-CHANGES.md.

Draft while tickets land. Merged so far: T1 #110 startup contract, T4 #113 KB search native.

🤖 Generated with Claude Code

romiluz13 and others added 30 commits October 19, 2026 09:40
…ailed call

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…d novelty scan

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…write instead of scoring them

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…declare best-effort bookkeeping

Removes the 'transactions unsupported -> direct writes' fallbacks in
structured memory, procedures, graph relations, KB and sync; the error is
now TransactionUnsupportedError. Audit and projection-run bookkeeping goes
through one shared bestEffort helper (log line + per-label counter).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… of empty results

readFile propagates ErasureGateConflictError instead of an erasure-suppressed
miss shape; a non-allowlisted trace collection throws ReasoningChainCollectionError.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…P errors carry code and outcomes

/read-file maps an erasure conflict to 409 ERASURE_IN_PROGRESS (contract, OpenAPI);
/search-detailed searchConfig, the MCP tools and the client types reject js-merge
and unknown fusion modes; MCP tool errors keep code, status and outcomes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Remove the zero-hit cascade, capability downgrades, KB js-merge with survivor-stands, the $text tier and admission-denial-to-text. A healthy zero is final, a denial is a typed budget-denied failure, a fusion error is a typed lane failure, and a filter that cannot be resolved fails typed instead of being dropped.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ped 400

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Required-index manifest derived from ensureSearchIndexes. One 180 s deadline covers version 8.3+, Search management privileges, autoEmbed and every manifest index READY, queryable and serving its definition per host. Repair creates or updates, never drops. Failures are typed MemongoBootError with redacted diagnostics; swallowed index and capability errors now rethrow. Removes the paid storedSource probe, $text index creation and the STRICT/REQUIRE_VECTOR envs; invalid config fails boot. Adds a no-write preflight.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
)

The server exits 1 before binding on any MemongoBootError (X-06); /ready adds a read-only manifest lane (X-08); memongoBridgeGetManager keeps the boot failure type (X-04). Adds the preflight script.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#110)

publish.yml is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The probe embedded a text query on every call (four per /stats request, outside
the cost ledger). Automated coverage under a tenant filter now reports unknown.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Out-of-ownership follow-ons, named in the 1222 Return: e2e fixtures create their own $text indexes; the memory-jobs factory test runs keyed only; the benchmark factory test and proof-pack script drop the removed STRICT env; G4-77 propagation test; self-host doc pin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… bridge README envelope

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
# Conflicts:
#	BEHAVIOR-CHANGES.md
#	packages/memory-engine/src/kb-scope-community.e2e.test.ts
…gate (#112)

mongoSearch runs the configured scoreFusion or rankFusion as one aggregate. The tier ladder, JS merge, vector-only and keyword-only downgrades, $text and denial-to-keyword are removed; a failure or denial is a typed lane error. The planner's hybrid:bridge sub-lane rides in the same aggregate as a sibling input pair with its own filter, and the ledger charges one aggregation plus each embedded vector input. A throttled search returns a typed budget-denied row instead of a bare empty list.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…old-import timeout

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…er (#112)

Delete legacySearch, the legacySearchFallback re-run in search and searchDetailed, the method-detection helpers and the lexicalPrefilter option. A healthy zero-hit planner answer is final. A request that carries searchConfig.lexicalPrefilter now throws a typed request-stage error. An early admission denial in searchDetailed returns a typed budget-denied row and failed outcome status.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…nd bridge (#112)

The API search config schema is strict, so a request that carries lexicalPrefilter gets a typed 400 VALIDATION_ERROR. The key is removed from the client types, the MCP search tool schema, the bridge options and the OpenAPI search paths.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…back e2e cases (#112)

The suite covered legacySearch, which no longer exists. Drop it from the tier-a script and the required-suite lists in ci.yml and publish.yml. Remove the e2e cases for the keyword-only and $text paths from the real-cluster suites.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
# Conflicts:
#	BEHAVIOR-CHANGES.md
#	packages/memory-bridge/src/memongo-bridge.ts
#	packages/memory-engine/src/mongodb-schema-search-indexes.ts
…xplicit timeout

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… option (#112)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…entity batches with Search, run consolidation on one ledger

Consolidator, contradiction, adjudication, novelty, reasoning, temporal,
relation, entity and derived-memory failures now throw typed errors and
fail the affected event or run instead of defaulting to ADD, no conflict,
NO_MERGE, 0.5 or regex-only. Provider resolution failure throws
EnrichmentConfigError. The entity filter is a $search text query with the
batch window in compound.filter. A consolidation run charges one search
ledger.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…112)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…pings to the manifest (#116)

findExactProcedureMatches now emits one $search with equals on name and
triggerQueries (token, lowercase normalizer) and revalidates scope, state
and validity on the hydrated document before the limit. A Search failure is
a typed exact-match error. The manifest gains the token mappings for
procedures and entities and the string mappings for episodes that the
other store lanes in this ticket query. Adds a Search-stage test double
that evaluates a lane's pipeline against indexed and current documents.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
romiluz13 and others added 27 commits October 19, 2026 14:51
…ded 10 (#121)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e episode gate, consolidator trace tenant (#121)

Lead edits in the merge: the behavior-log heading is named properly, and
the index-health test drops its dead mock of the removed reranker module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e clock (R9)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e finite generations (R4)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- check-no-fallbacks follows RegExp aliases, assignments and picks to a fixed point (R10)
- waitForSearchIndexesQueryable moves to an e2e test helper; the engine stops exporting it (R11)
- e2e drops the retired idx_chunks_text and $text expectations (R12)
- /v1/search and /v1/search-detailed send the admission Retry-After on the denied 503 (ruling 4)
- matrix Decision 8 and BEHAVIOR-CHANGES state the real typed 503 path

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… manager scrubs them (R6)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The read now carries the driver timeoutMS and a local deadline, so a
stalled response or a read that never settles is a failed read, which is
index-unavailable, instead of holding the request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… at the return boundary (R3)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… advances the marker (R7)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…d a failed KB refresh is KB_REFRESH_FAILED (R8)

/consolidate and /probes/vector run engine Search underneath. A recognized
typed Search failure (lane failure, index-unavailable, index-stale, budget
denial) now goes through the shared outcome and status mapper with its
outcomes row and Retry-After; an unexpected error keeps the route's generic
500. /sync answers a failed KB refresh as a typed 500 naming the reason and
the import's commit counts. OpenAPI, the failure matrix rows and the behavior
changes are updated.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ailure matrix

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…trix row (R9)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The graph lane's entity lookups and the episodic lane's episode Search run
through the charged executor, and both lanes reserve their real cost (up to
4 lookups, 1 Search) instead of none. The retrieval key carries the graph
candidates and depth the lane reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…quest and job (#122 R1 R2)

Discovery, the context bundle and novelty run on one request ledger with the
configured limits, shared by sibling sections and the nested search. Novelty
reserves every k-NN probe before the first one runs, so a scan the ledger
cannot pay for is a typed reservation denial with no probe emitted.

Public novelty and consolidate, each extraction job and each consolidation
job take one index-health snapshot. Novelty's probes, the consolidator's
similarity, dedup, prune and entity Search, and the supporting-event Search
of a promotion decision check their planned index against it first, so a
STALE or missing index fails typed before the native operation runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Also pins the novelty route's 503 SEARCH_BUDGET_EXHAUSTED answer to the new
reservation denial.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t index assessment, scrubbed config refusals, typed KB refresh, typed Search failures on non-search routes, chain leaf lifecycle

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lth snapshot per retrieval request and job, bounded health metadata read

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n errors are scrubbed

- a public novelty scan opens its own 30/30 ledger, as consolidation does, so the 12/5 default no longer refuses scans over five candidates (1240 decision 1)
- getMemorySearchManager scrubs URI, host and database from creation and close failures (R6 follow-through)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…42 D1)

Reverts the 30/30 novelty ledger: spec decision 14 keeps 12/5 per retrieval request, and a lower configured limit is honored (new test).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rors (D2)

ingestFilesToKB logged and dropped a configured path it could not scan or a
file it could not read, so the refresh counted as clean and advanced its
marker. The failure now joins the import's errors beside the real counts of
the documents that succeeded; the existing KBRefreshError path fails the
refresh. Unsupported extensions, symlinks and unchanged content stay skips.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A consolidation run that fails after committing work, and a sync whose file
sync completed before its KB refresh failed, now attach a bounded commit
report to the typed error they already throw: commitState none, committed or
unknown, plus the run identity and known committed counts, or the completed
file-sync phase and the import counts only when the import produced them.
An uncertain write makes the report unknown, never zero.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ror body (D3)

A typed Search failure from /v1/consolidate and a KB_REFRESH_FAILED or other
failure from /v1/sync keep their status, code and outcome rows and add the
engine's commitState (none, committed, unknown) and a bounded, value-free
commit detail rebuilt from the documented fields. The SDK parses both onto
MemongoClientError, MCP tool errors pass them through, and the shared ApiError
OpenAPI schema and the route descriptions document them.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… fixes D2 and D3

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…capability matrix

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…post-commit failures carry their commit state

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…er-write-failed says its outcome is not known

Lead rulings on builder 1243 decisions 2 and 3 (review 1242 D3 residues).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… sync (review 1244 E1, S1)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@romiluz13
romiluz13 marked this pull request as ready for review October 19, 2026 14:11
@romiluz13
romiluz13 merged commit 6b3fe0c into main Oct 19, 2026
4 checks passed
@romiluz13
romiluz13 deleted the integration/m2-20261019 branch October 19, 2026 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant