Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/knowledge-base.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ separate work, not an implemented capability of this knowledge base.
```text
src/inferrsi/data/skill_library/
README.md Library navigation
skill-index.json Stable IDs → categorized paths/backend (60 skills)
skill-index.json Stable IDs → categorized paths/backend (61 skills)
hardware/{ascend,cuda,host}/<id>/ Hardware/platform-specific complete skills
frameworks/<framework>/<id>/ Framework-owned complete skills
repositories/<repo>/ Engine entry, sub-skills and focused complete skills
Expand Down Expand Up @@ -168,18 +168,18 @@ Complete procedures, sidecars, references and cases are packaged, with 13 Ascend
repositories, methods or workflows. The 21 engine and research-router skills
use the same ownership layout under `repositories/<engine>/` and
`workflows/inference-engine-research/`, including their sub-skills and references.
The library contains 81 `SKILL.md` files, all in English.
The library contains 82 `SKILL.md` files, all in English.
The collection overview, import notes and source manifest belong to the research
workflow's references; Strata design-space analysis belongs to Strata. The library
root contains only navigation and the registry.

The [generated catalog](../src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md)
projects the 60 registered skills into the contribution catalog. Its records
projects the 61 registered skills into the contribution catalog. Its records
use `inferrsi.<backend>.skills.<name_with_underscores>` IDs, with `ascend` and
`cuda` for scopes that depend on those backends and `common` for the remaining skills.
Each `skill_documents.original` points to the full packaged procedure. The
`skills` CLI retains the original hyphenated skill IDs and defaults to this
60-ID registry; engine procedures remain available through library navigation
61-ID registry; engine procedures remain available through library navigation
and catalog document pointers.

The registered [h3.c runtime audit](../src/inferrsi/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/SKILL.md)
Expand Down
2 changes: 1 addition & 1 deletion docs/reference-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ Reading rules:
| **C4 / NVIDIA Resiliency Extension** @ `49af0b03f14550d36a60dcfebd573848b5fd8435` (Apache-2.0 source headers; Megatron-Core lineage) | local `base_manager.py` complete-iteration selection/save/finalizer and `local_manager.py` dirty/final file lifecycle | mined (contract only) — CUDA D2H synchronized before async request returns; no NPU qualification | mined — snapshot/replica lifetime constraints → `checkpoint-recovery-consistency` | mined — clean/dirty publication, complete-rank generation selection and incomplete-save rejection; no power-loss proof | mined — all-rank availability/finalization and replica failure domains, not transport throughput | out-of-scope |
| **CR1 / DeepSpeed ZeRO-Infinity** @ `5f266e26d216a705d9c64cac88fb813cefdbac4b` (Apache-2.0) | complete short `offload_config.py`, official ZeRO-3 live offload-field docs; no runtime/checkpoint audit | mined — parameter stage-3 and optimizer stage-1/2/3 boundaries → `training-resource-budget` | mined — CPU optimizer budget, pinned RAM, element-count residency and gated core share | mined — NVMe buffer pools and tiled read/compute/write overlap intent; buffer-size units unresolved in approved excerpt | out-of-scope — CPU/device transfer is not network coverage | mined — CPU resource-budget qualification; no privileged changes |
| **CR5 / Ascend MindSpeed adaptive memory** — `core_r0.8.0`, page history `53193db8` (full pin and license unresolved) | original GitCode historical `docs/features/adaptive-memory.md` body directly read; implementation unavailable | mined (docs-only case) — spare-NPU-memory versus recompute/swap decision → `training-resource-budget` historical case; changed-layer comparison rejected as controlled gain | mined (docs-only) — CPU swap and allocator/affinity prerequisites | out-of-scope | out-of-scope — NPU/CPU copies not network evidence | mined (historical conditions) — incompatible allocator features and CPU-affinity prerequisites; no current compatibility claim |
| InferMatrixLab/vllm-rlt (legacy ThinkFlowLab/vllm-rlt) @ `4a350344027960a48384626aa646f1765fda6d4f` (root Apache-2.0; model attribution in NOTICE) | Fixed-pin bounded depth-aware KV manager/runner/docs/test reads; [intake](reviews/issue91-vllm-rlt-intake.md); descriptor-only / intake-plan | located-not-mined — synchronous early-exit LAST_EXITED propagation versus SHARED and bookkeeping-only marking; no operating skill or runtime qualification | out-of-scope — device KV does not establish host-RAM coverage | out-of-scope — no storage mechanism extracted | out-of-scope — inspected KV copies are local tensor copies; no transport mechanism extracted | out-of-scope — no OS mechanism extracted |
| InferMatrixLab/vllm-rlt (legacy ThinkFlowLab/vllm-rlt) @ `4a350344027960a48384626aa646f1765fda6d4f` (root Apache-2.0; model attribution in NOTICE) | Bounded manager/runner/scheduler/Ouro/host-metadata/Torch-consumer chain; [candidate](../src/inferrsi/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md), [source map](../src/inferrsi/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/source-notes.md), [construction/cases](reviews/issue91-vllm-rlt-finalization.md); historical [intake](reviews/issue91-vllm-rlt-intake.md) retained | source-guidance — synchronous LAST_EXITED copying versus SHARED, written-prefix validity, later deeper attention and normal release/wait/free; candidate, native runtime and efficacy not_run | out-of-scope — host bookkeeping does not establish host-RAM coverage | out-of-scope — no storage mechanism extracted | out-of-scope — local tensor copies; no transport mechanism extracted | out-of-scope — no OS mechanism extracted |
| vllm-project/afd-plugin @ `df8d6a0c403c3ca1b4ed9cf1687e7204908bebec` (Apache-2.0 selected Python readset; retained MIT CAM subset notice) | Fixed-pin topology/factory/runner/metadata/DeepSeek dense proxy and P2pNccl reads; [full ledger](../src/inferrsi/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/references/source-notes.md); [historical intake](reviews/issue91-afd-plugin-intake.md) retained | mined (bounded source-only candidate) — role/world/subgroup rank ledger and eager dense-FFN per-stage/per-peer payload ownership → [audit-afd-p2p-exchanges](../src/inferrsi/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md); native/backend/model validation not_run | out-of-scope — no host-memory mechanism extracted | out-of-scope — no storage mechanism extracted | source call chain mined — actual NCCL-backed send/recv consumer identified; underlying completion, graph and multi-host/physical transport unqualified; no Network coverage credit | out-of-scope — no OS mechanism extracted |

Wave-8 implementation provenance (Dot bundles integrated 2026-10-15):
Expand Down
47 changes: 47 additions & 0 deletions docs/reviews/issue91-vllm-rlt-finalization.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# vLLM-RLT synchronous KV finalization

Related to #91. This is one source-derived candidate capability, extending the descriptor-only [intake](issue91-vllm-rlt-intake.md) without replacing its historical evidence.

## Construction record

The source anchor is `InferMatrixLab/vllm-rlt@4a350344027960a48384626aa646f1765fda6d4f`, tree `4c2cab1f497513ea0590060eb53ee0b179a5bf70`. The approved scope is synchronous native Ouro with host `KVMetadata`: LAST_EXITED propagation versus SHARED, later attention consumption, and normal release/wait/free order. Asynchronous preemption/speculation, device-owned metadata, transfer/prefix ownership, capacity estimation and whole-engine extraction are excluded.

Portable AREX `distill-ml-knowledge` scope → ground → construct → verify informs the direct, task-agnostic source construction. The existing `extend-repo-skill` ownership check found no RLT operating owner; the separately approved `audit-recurrent-kv-finalization` capability is the only new node. InferRSI's existing typed metadata, canonical library, registry and readers are the packaging contract. No DisCo managed import, extra router, dependency installation or evaluation framework is introduced.

The grounded evidence is the intake's bounded manager/runner/test/license material plus the fixed caller, request, metadata, Ouro consumer and Torch dispatch tuple. The [source notes](../../src/inferrsi/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/source-notes.md) explain the call chain and exclusions; the [file ledger](../../src/inferrsi/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/source-identity.json) records exact Git blobs, SHA256, ranges and carried versus added acquisition. The original Apache-2.0 LICENSE and model/configuration NOTICE bytes are retained. No implementation or model weights are copied.

```mermaid
flowchart LR
A["Pinned manager, scheduler, runner, Ouro and metadata sources"] --> B["One candidate procedure + source ledger + notices"]
B --> C["Existing registry and library navigation"]
C --> D["skills / checkout reader: complete local resources"]
C --> E["Stock projection from committed canonical source"]
E --> F["catalog / package reader"]
```

The two rules are narrow: copy the final executed recurrent-layer K/V into skipped LAST_EXITED planes before a later deeper history read; retain the last completion dependency until runner release precedes cache-reference reclamation. The procedure preserves allocated-capacity, written-validity, copied-data and completed-work distinctions. SHARED is a single-plane control; bookkeeping-only `mark_finalized` is not a copying substitute. Backend placement is navigation, not qualification.

## Proposed representative and regression cases

These are candidate inputs and expected discriminators, not results. Native tests use the original package and real Torch tensors. An absent/new skill comparison can use the same questions, without changing the engine, inputs or expected source contract.

| Case | Positive / negative discriminator | Grounded input |
|---|---|---|
| Executed-plane regression | Preserve shallower/executed K/V; deeper propagation is a physical copy, not an alias | Upstream `test_last_exited_copies_every_layer_and_preserves_executed_depths`, 38–58 |
| Later deeper attention | Finalized earlier tokens remain readable as later tokens reach deeper planes across a block boundary | `test_early_exit_then_later_token_loops_deeper_across_block_boundary`, 61–77 |
| Premature finalize / prefix gap | Allocated capacity or a sparse later write must not make missing history valid | `test_validation_happens_before_mutation`, 97–107; `test_out_of_order_writes_only_expose_complete_prefix`, 116–127 |
| Bookkeeping sentinel | `mark_finalized` preserves deeper sentinels; `finalize_token` performs the copy | `test_mark_finalized_records_deeper_planes_without_copying`, 216–233 |
| SHARED control | One plane serves every depth; do not expect LAST_EXITED isolation or extra propagation | `test_shared_layout_exposes_one_plane`, 168–174; `test_mark_finalized_is_a_no_op_for_shared_layout`, 245–258 |
| Reused-page regression | A new request cannot consume stale old-request bytes merely because pages are reused | `test_block_reuse_never_exposes_stale_request_history`, 80–94 |
| Retirement investigation | Identify the latest event and release wait before scheduler finish/free; recording alone is insufficient | Runner 1331–1375; engine 626–640; scheduler 181–193 |
| Unsupported-path counterexample | Refuse to qualify async/device-owned metadata from host-prefix checks or synchronous call order | README 70–75; metadata 45–54; manager 716–727 |

All test names above are in the pinned upstream `tests/test_kv_cache.py`. Its manager tuple is CPU/float32/Torch attention, 2 layers, 1 KV head, head dimension 4, 18 blocks, block size 2 and 3 loops. The package initializer and test conftest require real Torch. The available construction environment lacks Torch and other declared CPU dependencies, so upstream import and native CPU tests are **not_run**; no classes, tensors or formulas were substituted. This is a concrete execution gap, not a correctness failure.

## Verification boundaries and handoff

Schema/frontmatter, registry navigation, source identity/ranges, local resource links, retained notices, checkout reads and deterministic catalog generation are content/package checks. Matching package reads and source hashes must be checked on the real generated child of the committed canonical source. They establish access and provenance, not RLT runtime correctness.

Native CPU/CUDA, model output equivalence, quality, performance and agent efficacy remain **not_run / not evaluated**. CPU cache tests cannot establish CUDA completion. The existing asynchronous-output/cleanup warnings remain unresolved. Candidate maturity and `empirical_validation=not_run` are preserved independently of software checks or merge status.

Author: Codex / gpt-6.1-sol / xhigh. Independent review, final source/generated commit identities and actual check receipts belong to the revision-scoped PR. Governing tracking Issue closure is not claimed.
2 changes: 1 addition & 1 deletion docs/searchspace-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,7 @@ System-level: scheduler, DVFS, process isolation, environment variables, driver
| Distributed recovery qualification | RAM / SSD / Network / Verify | Low | C3/C4 contracts extracted; replica transport, whole-state completeness, world-size changes and actual failure recovery remain unvalidated |
| Cross-resource applicability and historical evidence | Accelerator / RAM / OS | Low | DeepSpeed buffer_size units/dispatch outside inspected config; MindSpeed full pin/license/implementation/current compatibility unresolved; CR6 is expert methodology, not measured gains |
| OS/System isolcpus and scheduler priorities (nice / ionice / SCHED_FIFO / RR) | OS / Optimize | Low | cgroup v2 controls are now sourced (`cgroup-resource-controls`); the scheduler-priority class remains without a source and deferred |
| vLLM-RLT recurrent-depth KV finalization | Accelerator / KV correctness / Verify | Intake only; not coverage | `vllm_rlt` remains descriptor-only; canonical owner corrected. Follow the [bounded early-exit intake](reviews/issue91-vllm-rlt-intake.md) before extraction; tensor copying, written-position bookkeeping and device completion remain distinct, with no runtime qualification |
| vLLM-RLT recurrent-depth KV finalization | Accelerator / KV correctness / Verify | Source-guidance candidate; native/efficacy not_run | [One operating candidate](../src/inferrsi/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md) traces synchronous host-KVMetadata exit/copy/validity → later depth-specific attention → normal release/wait/free. LAST_EXITED planes, SHARED single plane, bookkeeping and CUDA completion remain distinct; DeviceKVMetadata/async/speculation/capacity/transport excluded. [Intake](reviews/issue91-vllm-rlt-intake.md) retained |
| AFD P2pNccl completion, graph and transport qualification | Accelerator / Communication / Verify | Candidate audit delivered; native qualification gap remains | [audit-afd-p2p-exchanges](../src/inferrsi/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md) covers rank and peer/stage ownership at its pin. The `afd_plugin` runtime descriptor remains unverified; underlying completion, DBO/graph and physical/multi-host transport are unqualified. Historical [intake](reviews/issue91-afd-plugin-intake.md) retained; no new Network coverage |

**Resolved gaps since initial matrix (all high/medium priority cleared):**
Expand Down
7 changes: 4 additions & 3 deletions src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# InferRSI skill discovery index

Generated abstracts for all 60 registered skills in `data/skill_library/`.
Generated abstracts for all 61 registered skills in `data/skill_library/`.
Their complete procedures, references and cases ship in the same package.
`skill-index.json` maps stable skill IDs to categorized directories. The
reference and searchspace matrices
Expand All @@ -15,7 +15,7 @@ These are knowledge abstracts (`--kind knowledge`), with namespaced package
IDs distinct from explicitly selected `checkout_ref` skills. Ascend/CUDA
procedures use their corresponding backend; shared methods use `common`.

Source snapshot: [933c3e31f2b6f4f7957216ac641bf5dbb63ba002](https://github.com/InferMatrixLab/InferRSI/tree/933c3e31f2b6f4f7957216ac641bf5dbb63ba002/src/inferrsi/data/skill_library).
Source snapshot: [0115f1be0539b704b393cffd97598d8bb076bd0c](https://github.com/InferMatrixLab/InferRSI/tree/0115f1be0539b704b393cffd97598d8bb076bd0c/src/inferrsi/data/skill_library).

```bash
inferrsi knowledge search 'profiling' --repo-type inferrsi --limit 50
Expand All @@ -29,7 +29,7 @@ contain the current input paths and bytes (Git checkout filters are respected).

```bash
python tools/build_skill_index.py --source-revision <committed-source-revision>
python tools/build_skill_index.py --source-revision 933c3e31f2b6f4f7957216ac641bf5dbb63ba002 --check
python tools/build_skill_index.py --source-revision 0115f1be0539b704b393cffd97598d8bb076bd0c --check
```

`--check` detects missing, stale and unexpected generated records without writing.
Expand All @@ -52,6 +52,7 @@ References and cases ship with their owning skill, not as duplicate knowledge ab
| [audit-afd-p2p-exchanges](cuda/skills/audit-afd-p2p-exchanges.json) | cuda | candidate |
| [audit-h3-runtime-residency](common/skills/audit-h3-runtime-residency.json) | common | candidate |
| [audit-llama-runtime-state](common/skills/audit-llama-runtime-state.json) | common | candidate |
| [audit-recurrent-kv-finalization](common/skills/audit-recurrent-kv-finalization.json) | common | candidate |
| [audit-sd-cpp-residency](common/skills/audit-sd-cpp-residency.json) | common | candidate |
| [audit-sglang-diffusion-lifetimes](common/skills/audit-sglang-diffusion-lifetimes.json) | common | candidate |
| [audit-vllm-kv-lifetimes](common/skills/audit-vllm-kv-lifetimes.json) | common | candidate |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/933c3e31f2b6f4f7957216ac641bf5dbb63ba002/src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md",
"commit": "933c3e31f2b6f4f7957216ac641bf5dbb63ba002",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/0115f1be0539b704b393cffd97598d8bb076bd0c/src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md",
"commit": "0115f1be0539b704b393cffd97598d8bb076bd0c",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/metadata.json",
Expand All @@ -41,7 +41,7 @@
"src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md": "d494fc7e539cbb7388189bd7b3f04bb5c4983fdd7a82484348aad1cee7f741f2",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/metadata.json": "e4c02d4ae85e473947e024c38aed121ca8e5cfbfe1091c5792ce58a349248187",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/references/source-notes.md": "6ffbfb68cc7e04923b4fbca2a73a7555b7c3b45a6f40819bb5f3213ef05cf88d",
"src/inferrsi/data/skill_library/skill-index.json": "8ce35e18f7629260d67a95617b9918eecff73ed8d9370dd3d4595c725ada9aa1"
"src/inferrsi/data/skill_library/skill-index.json": "5d5e722125a2997b41db5befd1fe52b87e20e88597ffe3b142f0cca3c65032ad"
},
"skill_metadata": {
"name": "ascend-aclgraph-capture-replay",
Expand Down
Loading
Loading