diff --git a/docs/reference-matrix.md b/docs/reference-matrix.md index ee72605f..6b92f376 100644 --- a/docs/reference-matrix.md +++ b/docs/reference-matrix.md @@ -151,6 +151,25 @@ the supplied local directory. Its six-paper extraction and standalone candidates are not imported or counted here. Existing DeepSpeed configuration evidence and Mooncake transport discovery are not substitutes for those paper readsets. +Issue #131 independently adds the scoped sources below; it does not restore that +archive or claim coverage of its uninspected ZeRO-Infinity, FlexGen or Mooncake +papers. The existing [search-policy owner](../src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md) +retains the [readset, attribution and notices](../src/inferrsi/data/skill_library/methods/optimization-search-policy/references/foundational-sources.md). + +| Source edition | Added case / inspected scope | Evidence limit | +| --- | --- | --- | +| Sarathi-Serve, OSDI 2024 | `case.sarathi-chunk-budget`; §§4.1–4.3, 5.3–5.4 | Chunking overhead and component ablation; no local replication | +| PagedAttention, 2309.06180v1 | `case.paged-kv-allocation`; §§4.1–4.5, 7.1–7.3 | Allocation efficiency versus kernel/block/transfer overhead | +| SGLang, 2312.07104v2; vLLM prefix design v0.10.2 | `case.prefix-reuse-locality`; §3, §6.3, A.4; identity/isolation/operations | Separate source versions; hot-replica counterexample is hypothetical | +| DistServe, OSDI 2024 | `case.distserve-phase-placement`; §§3–4, 6.1, 6.3–6.4 | Real system and simulated ablation distinguished | +| CUDA Best Practices 11.8.0 | `case.cuda-copy-overlap`; §§9.1–9.1.2 | Documented CUDA prerequisites; no native or other-backend validation | + +The sixth decision case reuses `case.admission-drain-occupancy` unchanged, retaining +its original INCONCLUSIVE campaign. Orca and Alpa remain with their existing +owners; Alpa supplies training applicability only. All new cases are candidates, +empirical validation not_run; CPU fixtures establish packaging and deterministic +retrieval contracts, not optimization efficacy. + ### Negative findings (scoped to inspected paths and revisions) - **Disk-level storage I/O mechanics** (mmap/pinned staging buffers, diff --git a/docs/searchspace-matrix.md b/docs/searchspace-matrix.md index d18f9812..c882c64c 100644 --- a/docs/searchspace-matrix.md +++ b/docs/searchspace-matrix.md @@ -10,6 +10,17 @@ the source-inspection or maturity claims below. Top-level rows are hardware resource layers. Second-level optimization targets (Kernel, Operator, KV Cache, Attention, Runtime, Distributed, Serving, VAE/Codec) live **inside** the row that owns their bottleneck. +When optimization stalls, use the existing search policy's +[foundational decision cards](../src/inferrsi/data/skill_library/methods/optimization-search-policy/references/foundational-mechanism-decisions.md). +Scheduling, caching/reuse and parallelism/pipelining are intervention axes across +these resource rows, not replacement layers. The six cases distinguish queueing +and admission, HBM allocation/reuse, host-device transfer and topology-dependent +phase placement. Network applies only to actual inter-node traffic; CUDA copy +overlap does not establish another backend's support. Existing RAM/SSD tiering +guidance remains with `kv-tier-management`. Source scope is in the Reference +Matrix; all new cases are locally unverified and missing diagnosis calls for +measurement, not an inferred recommendation or a new hardware coverage claim. + **Maturity key:** 🟢 covered (candidate skill + source evidence — candidate intelligence with source support, **not proven gains**) · 🟡 partial (source read, no dedicated skill) · 🔴 gap (no source, no skill) Per decision D05 (2026-10-15): approval, extraction, coverage, and empirical diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md b/src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md index 5113cc5e..a1b93782 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md @@ -15,7 +15,8 @@ These are knowledge abstracts (`--kind knowledge`), with namespaced package IDs distinct from explicitly selected `checkout_ref` skills. Ascend/CUDA procedures use their corresponding backend; shared methods use `common`. -Source snapshot: [933c3e31f2b6f4f7957216ac641bf5dbb63ba002](https://github.com/InferMatrixLab/InferRSI/tree/933c3e31f2b6f4f7957216ac641bf5dbb63ba002/src/inferrsi/data/skill_library). +Each record retains its own source snapshot in `sources`; unchanged skills +can keep their existing provenance when another skill is updated. ```bash inferrsi knowledge search 'profiling' --repo-type inferrsi --limit 50 @@ -29,10 +30,13 @@ contain the current input paths and bytes (Git checkout filters are respected). ```bash python tools/build_skill_index.py --source-revision -python tools/build_skill_index.py --source-revision 933c3e31f2b6f4f7957216ac641bf5dbb63ba002 --check +# Update or check one skill without repinning unrelated records: +python tools/build_skill_index.py --source-revision --skill +python tools/build_skill_index.py --source-revision --skill --check ``` `--check` detects missing, stale and unexpected generated records without writing. +With `--skill`, only that record and the shared navigation are updated/checked. References and cases ship with their owning skill, not as duplicate knowledge abstracts. | Skill | Backend | Original declared status | diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/optimization-search-policy.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/optimization-search-policy.json index f4490072..17326bb0 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/optimization-search-policy.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/optimization-search-policy.json @@ -8,12 +8,19 @@ "component": "skills", "scenario": "skills.discovery", "status": "imported", - "summary": "Policy for how an optimization agent should select, order, and stop optimization experiments. Covers candidate generation, exploration vs exploitation, diversity, negative experiment memory, budget allocation, and stopping criteria. Drawn from vllm-evolve (phase-locked optimization round), TRT-LLM casebook (prior-art matching), and SkillOpt (bounded edit + gate). Use when deciding what to try next and when to stop.\n", + "summary": "Policy for how an optimization agent should select, order, and stop optimization experiments. Use when optimization stalls: re-derive the whole-system bottleneck and critical path from first principles before choosing scheduling, caching/reuse, or parallelism/pipelining. Covers candidate generation, exploration vs exploitation, diversity, negative experiment memory, budget allocation, and stopping criteria. Drawn from vllm-evolve (phase-locked optimization round), TRT-LLM casebook (prior-art matching), and SkillOpt (bounded edit + gate). Use when deciding what to try next and when to stop.\n", "tags": [ "optimization", "search", "policy", - "experiment-design" + "experiment-design", + "whole-system", + "bottleneck", + "critical-path", + "scheduling", + "caching", + "parallelism", + "pipelining" ], "origin": [ "external" @@ -21,11 +28,23 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/933c3e31f2b6f4f7957216ac641bf5dbb63ba002/src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md", - "commit": "933c3e31f2b6f4f7957216ac641bf5dbb63ba002", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/d0338d71010088a9382a03f10b087a4f5330c7d3/src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md", + "commit": "d0338d71010088a9382a03f10b087a4f5330c7d3", "paths": [ "src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/cuda-copy-overlap.json", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/cuda-copy-overlap.md", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/distserve-phase-placement.json", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/distserve-phase-placement.md", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/paged-kv-allocation.json", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/paged-kv-allocation.md", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/prefix-reuse-locality.json", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/prefix-reuse-locality.md", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.json", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.md", "src/inferrsi/data/skill_library/methods/optimization-search-policy/metadata.json", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/references/foundational-mechanism-decisions.md", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/references/foundational-sources.md", "src/inferrsi/data/skill_library/methods/optimization-search-policy/references/papers-20261015/INTEGRATION_NOTES.md", "src/inferrsi/data/skill_library/methods/optimization-search-policy/references/papers-20261015/MECHANISMS.md", "src/inferrsi/data/skill_library/methods/optimization-search-policy/references/papers-20261015/NEXT_GATE_LEADS.md", @@ -34,7 +53,7 @@ "src/inferrsi/data/skill_library/methods/optimization-search-policy/references/papers-20261015/updates/optimization-search-policy.md", "src/inferrsi/data/skill_library/skill-index.json" ], - "content_sha256": "882e7850b9b49ad692a2d11618701afe0f4e83727ea78df114be2029923c725a" + "content_sha256": "e1a18eda78e392acb0bc9d6f38e1b8fc651ce31901426352230407f51c4f040c" } ], "index_type": "packaged-skill", @@ -43,8 +62,20 @@ }, "checkout_ref": "checkout:optimization-search-policy", "source_files": { - "src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md": "882e7850b9b49ad692a2d11618701afe0f4e83727ea78df114be2029923c725a", - "src/inferrsi/data/skill_library/methods/optimization-search-policy/metadata.json": "113c761ae488055da7e59c77df2db4247a545fdaf558188f01bde4b0960123c6", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md": "e1a18eda78e392acb0bc9d6f38e1b8fc651ce31901426352230407f51c4f040c", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/cuda-copy-overlap.json": "ec472906895b7e197402447e706c6e92f620503f33861b017aff1567bab9e4a7", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/cuda-copy-overlap.md": "9a6d9bba0b42748960625b6d4b1d96de281d73be4f16ed4c9068c30616810549", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/distserve-phase-placement.json": "5afb296e26bfb43be54c4ca1eb53f214add6e83e0c45d86ca9c9c8010c045379", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/distserve-phase-placement.md": "21f69e2415f757b7e7e699744722d8e979bfd924286e3e941db0be2099e0351a", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/paged-kv-allocation.json": "e4579d78dee6c2fcfc33799e832ea1a23c513268dfc9a53054ee4447ccda8bb6", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/paged-kv-allocation.md": "1c2ab146f9c55f74c5ff8aee803c04830961b3a1e99c45c2a842d62806479d9f", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/prefix-reuse-locality.json": "9d027427bc8ab5c3c3fac1c64be863200128a2d8fd29642492820bae6d57ff15", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/prefix-reuse-locality.md": "bba1812d51bc46427ce9e7933e1d25de07308bd7827e44426e0138019a9a5c2e", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.json": "7745caa598884af720679b8d095c1980227fd4f0ecdb1ae901dbd200b02710c3", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.md": "e012ff1e07835e19691140717e0bad5022535b5802b23dbf8f15c8cc5d7e87f4", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/metadata.json": "99a57a232b3a89cb9d8f3315b30c2b675a1bb9ec7696e2468e57341edbe6192c", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/references/foundational-mechanism-decisions.md": "e9b0f39c47d07c93c381c83ded449a3e908c1897f306c21424dbfa3575b8c7d0", + "src/inferrsi/data/skill_library/methods/optimization-search-policy/references/foundational-sources.md": "3a76af59b18872f8e5c5b5a81ecaa45d06b9ff21b4a028f3c96102cf60d17c6f", "src/inferrsi/data/skill_library/methods/optimization-search-policy/references/papers-20261015/INTEGRATION_NOTES.md": "157e74ba180426b279b548a729769f152bf18f9a78ffca4ebfcaf26b7d0c5292", "src/inferrsi/data/skill_library/methods/optimization-search-policy/references/papers-20261015/MECHANISMS.md": "18bb2f5d7e52773c516420a142edfb531d0b9c822a2934d60f3234ac1cb9d40e", "src/inferrsi/data/skill_library/methods/optimization-search-policy/references/papers-20261015/NEXT_GATE_LEADS.md": "16a264c7159f9c2c700ce8ba46e63f3e636b277c987da4970f6d2fa085fcb053", @@ -55,12 +86,19 @@ }, "skill_metadata": { "name": "optimization-search-policy", - "description": "Policy for how an optimization agent should select, order, and stop optimization experiments. Covers candidate generation, exploration vs exploitation, diversity, negative experiment memory, budget allocation, and stopping criteria. Drawn from vllm-evolve (phase-locked optimization round), TRT-LLM casebook (prior-art matching), and SkillOpt (bounded edit + gate). Use when deciding what to try next and when to stop.\n", + "description": "Policy for how an optimization agent should select, order, and stop optimization experiments. Use when optimization stalls: re-derive the whole-system bottleneck and critical path from first principles before choosing scheduling, caching/reuse, or parallelism/pipelining. Covers candidate generation, exploration vs exploitation, diversity, negative experiment memory, budget allocation, and stopping criteria. Drawn from vllm-evolve (phase-locked optimization round), TRT-LLM casebook (prior-art matching), and SkillOpt (bounded edit + gate). Use when deciding what to try next and when to stop.\n", "tags": [ "optimization", "search", "policy", - "experiment-design" + "experiment-design", + "whole-system", + "bottleneck", + "critical-path", + "scheduling", + "caching", + "parallelism", + "pipelining" ], "status": "candidate", "profile": "local-skill-v1", @@ -69,14 +107,86 @@ "origin": [ "external" ], - "sources": [], + "sources": [ + { + "type": "paper", + "id": "Sarathi-Serve", + "version": "OSDI 2024 proceedings", + "url": "https://www.usenix.org/system/files/osdi24-agrawal.pdf", + "sections": [ + "4.1-4.3", + "5.3", + "5.4, Figure 14 and Table 4" + ], + "readset": "references/foundational-sources.md" + }, + { + "type": "paper", + "id": "2309.06180", + "version": "v1", + "url": "https://arxiv.org/html/2309.06180v1", + "sections": [ + "4.1-4.5", + "7.1-7.3" + ], + "readset": "references/foundational-sources.md" + }, + { + "type": "paper", + "id": "2312.07104", + "version": "v2", + "url": "https://arxiv.org/html/2312.07104v2", + "sections": [ + "3", + "6.3", + "A.4" + ], + "readset": "references/foundational-sources.md" + }, + { + "type": "document", + "version": "v0.10.2", + "url": "https://docs.vllm.ai/en/v0.10.2/design/prefix_caching.html", + "sections": [ + "Hash identity", + "Cache Isolation for Security", + "Data Structure", + "Operations" + ], + "readset": "references/foundational-sources.md" + }, + { + "type": "paper", + "id": "DistServe", + "version": "OSDI 2024 proceedings", + "url": "https://www.usenix.org/system/files/osdi24-zhong-yinmin.pdf", + "sections": [ + "3-4", + "6.1", + "6.3-6.4" + ], + "readset": "references/foundational-sources.md" + }, + { + "type": "document", + "version": "11.8.0", + "url": "https://docs.nvidia.com/cuda/archive/11.8.0/cuda-c-best-practices-guide/index.html", + "sections": [ + "9.1", + "9.1.1", + "9.1.2" + ], + "readset": "references/foundational-sources.md" + } + ], "empirical_validation": "not_run", "applicability": { "scope": "See procedure prerequisites and counter-signals" }, "limitations": [ "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets.", + "Foundational cases guide diagnosis after a plateau; no automatic stall detector, agent efficacy evaluation or engine speedup is established." ], "provenance_readsets": [ "references/papers-20261015/INTEGRATION_NOTES.md", @@ -84,7 +194,8 @@ "references/papers-20261015/MECHANISMS.md", "references/papers-20261015/NEXT_GATE_LEADS.md", "references/papers-20261015/references/source-notes.md", - "references/papers-20261015/updates/optimization-search-policy.md" + "references/papers-20261015/updates/optimization-search-policy.md", + "references/foundational-sources.md" ], "references": [ "../inference-optimization-casebook/cases/qwen-b300-rejected-directions.md", @@ -92,7 +203,30 @@ "../inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md", "../inference-optimization-casebook/cases/qwen-ascend-rejected-directions.json", "../inference-optimization-casebook/cases/drama3-rejected-directions.md", - "../inference-optimization-casebook/cases/drama3-rejected-directions.json" + "../inference-optimization-casebook/cases/drama3-rejected-directions.json", + "references/foundational-mechanism-decisions.md", + "cases/sarathi-chunk-budget.md", + "cases/sarathi-chunk-budget.json", + "cases/paged-kv-allocation.md", + "cases/paged-kv-allocation.json", + "cases/prefix-reuse-locality.md", + "cases/prefix-reuse-locality.json", + "cases/distserve-phase-placement.md", + "cases/distserve-phase-placement.json", + "cases/cuda-copy-overlap.md", + "cases/cuda-copy-overlap.json", + "../inference-optimization-casebook/cases/admission-drain-occupancy.md", + "../inference-optimization-casebook/cases/admission-drain-occupancy.json", + "../assess-serving-preemption/SKILL.md", + "../assess-serving-preemption/references/source-notes.md", + "../performance-ceiling-analysis/SKILL.md", + "../choose-parallel-training-plan/SKILL.md", + "../inference-benchmark-design/SKILL.md", + "../dense-performance-evidence/SKILL.md", + "../../repositories/vllm/diagnose-vllm-admission-pressure/SKILL.md", + "../../repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md", + "../../repositories/lmcache/kv-tier-management/SKILL.md", + "../../repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md" ], "consumption_mode": "packaged" }, @@ -109,7 +243,8 @@ "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets.", + "Foundational cases guide diagnosis after a plateau; no automatic stall detector, agent efficacy evaluation or engine speedup is established." ], "verification": { "locally_verified": false, diff --git a/src/inferrsi/data/skill_library/README.md b/src/inferrsi/data/skill_library/README.md index 1e41f230..559bc5c5 100644 --- a/src/inferrsi/data/skill_library/README.md +++ b/src/inferrsi/data/skill_library/README.md @@ -20,6 +20,13 @@ existing resource directory. ## Read registered skills +When local optimization stalls, start with the existing +[optimization search policy](methods/optimization-search-policy/SKILL.md) and its +[whole-system decision cards](methods/optimization-search-policy/references/foundational-mechanism-decisions.md). +They distinguish scheduling, caching/reuse and parallelism/pipelining using +measured resource constraints, six reusable cases and explicit abstention when +evidence is missing. These source-derived candidates do not establish speedups. + [`skill-index.json`](skill-index.json) maps 60 stable skill IDs to their canonical paths and backend scopes. Install the optional `skills` extra for these commands: diff --git a/src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md b/src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md index eb00f2e9..c1c5618b 100644 --- a/src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md +++ b/src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md @@ -1,6 +1,6 @@ --- name: optimization-search-policy -description: 'Policy for how an optimization agent should select, order, and stop optimization experiments. Covers candidate generation, exploration vs exploitation, diversity, negative experiment memory, budget allocation, and stopping criteria. Drawn from vllm-evolve (phase-locked optimization round), TRT-LLM casebook (prior-art matching), and SkillOpt (bounded edit + gate). Use when deciding what to try next and when to stop. +description: 'Policy for how an optimization agent should select, order, and stop optimization experiments. Use when optimization stalls: re-derive the whole-system bottleneck and critical path from first principles before choosing scheduling, caching/reuse, or parallelism/pipelining. Covers candidate generation, exploration vs exploitation, diversity, negative experiment memory, budget allocation, and stopping criteria. Drawn from vllm-evolve (phase-locked optimization round), TRT-LLM casebook (prior-art matching), and SkillOpt (bounded edit + gate). Use when deciding what to try next and when to stop. ' --- @@ -21,6 +21,23 @@ An optimization agent without a search policy tends to: This skill addresses all four. +## When local optimization stalls + +If repeated variants cannot improve the frozen objective, local headroom is +exhausted, or the bottleneck moves, return to Phase 1 using the +[foundational mechanism decisions](references/foundational-mechanism-decisions.md). +Reconstruct useful work, dependencies, state ownership and constrained resources +across the whole request path, including queueing, host work and communication. +Distinguish scheduling, work/data reuse and parallel overlap; also consider +algorithm, precision, fusion and layout changes within the authorized contract. + +Return a bounded hypothesis with prerequisites, counter-indications, invariants, +metrics and rollback. **Abstain from recommending an intervention when the +bottleneck or correctness prerequisites are unknown:** request the smallest +discriminating measurement. Stop for exhausted budget or missing authorization; +a plateau is not permission to broaden execution. These are agent instructions, +not an automatic stall detector or runtime scheduler. + ## Phase-locked optimization round An effective optimization round has locked phases that cannot be reordered. @@ -170,6 +187,10 @@ Stop local tuning and switch strategy when any of the following is true: escalate the abstraction level (e.g., change parallelism strategy rather than tuning an individual op). +For these strategy changes, use the three decision cards above before proposing +another variant. A measurement problem calls for better evidence; a capacity +limit or changed critical path calls for a different mechanism. + ## Budget allocation and final-artifact integrity Freeze the useful work, quality contract, allowed approximations, timing boundary diff --git a/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/cuda-copy-overlap.json b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/cuda-copy-overlap.json new file mode 100644 index 00000000..44d80f29 --- /dev/null +++ b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/cuda-copy-overlap.json @@ -0,0 +1,33 @@ +{ + "id": "case.cuda-copy-overlap", + "kind": "case", + "origin": [ + "external" + ], + "state": "candidate", + "content": "cuda-copy-overlap.md", + "sources": [ + { + "type": "document", + "version": "11.8.0", + "url": "https://docs.nvidia.com/cuda/archive/11.8.0/cuda-c-best-practices-guide/index.html", + "sections": [ + "9.1", + "9.1.1", + "9.1.2" + ] + } + ], + "empirical_validation": "not_run", + "applicability": [ + "CUDA host-device copy/compute overlap with independent chunks", + "Pinned-memory, device-capability and stream-ordering prerequisites" + ], + "limitations": [ + "No local campaign baseline/candidate evidence; imported evidence remains locally unverified.", + "Documentation-derived mechanism, not measured performance evidence", + "No Ascend or other backend portability claim", + "Tiny-chunk and contention regressions are hypothetical counterexamples" + ], + "attribution": "NVIDIA Corporation, CUDA C Best Practices Guide, archived release 11.8.0." +} diff --git a/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/cuda-copy-overlap.md b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/cuda-copy-overlap.md new file mode 100644 index 00000000..0533b981 --- /dev/null +++ b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/cuda-copy-overlap.md @@ -0,0 +1,30 @@ +# Asynchronous submission does not establish overlap + +ID: `case.cuda-copy-overlap`. Owner: `optimization-search-policy`. +Axis: pipelining. Resources: host RAM, Accelerator copy/compute and host-device link. +Documentation-derived candidate; local validation not_run. No local campaign evidence. + +## Source conditions + +[NVIDIA CUDA C Best Practices Guide 11.8.0](https://docs.nvidia.com/cuda/archive/11.8.0/cuda-c-best-practices-guide/index.html), +§§9.1–9.1.2: minimize transfers first. Copy/compute overlap requires device +capability, pinned host memory and appropriately separate non-default streams. +Same-stream copy and dependent kernel remain ordered. Excessive pinned memory +can hurt the system. The staged-copy example is a mechanism illustration, not +an observed InferRSI service result or an Ascend portability claim. + +## Proposed decision + +Signal: a timeline shows exposed copies and independent work available to cover them. +Prerequisites: backend support, chunk independence and bounded host/device buffers. +Candidate: pipeline copies and eligible work with double buffering; keep serial +execution as the reference. Invariants: producer/consumer dependencies, completion +events before buffer reuse, ordering, cancellation and backpressure. Metrics: +critical-path elapsed time, exposed/hidden copy time, transfer bandwidth, launch +overhead, pinned/peak device memory and complete-output correctness. +Hypothetical counterexample: tiny chunks add more launches than overlap saves, +or compute and copies contend for memory bandwidth. If independence or completion +is unknown, request a dependency/lifetime trace before recommending more streams. +Rollback: join owned work before freeing buffers and restore serial submission. + +Attribution and document version: [case record](cuda-copy-overlap.json). diff --git a/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/distserve-phase-placement.json b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/distserve-phase-placement.json new file mode 100644 index 00000000..16a46965 --- /dev/null +++ b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/distserve-phase-placement.json @@ -0,0 +1,34 @@ +{ + "id": "case.distserve-phase-placement", + "kind": "case", + "origin": [ + "external" + ], + "state": "candidate", + "content": "distserve-phase-placement.md", + "sources": [ + { + "type": "paper", + "id": "DistServe", + "version": "OSDI 2024 proceedings", + "url": "https://www.usenix.org/system/files/osdi24-zhong-yinmin.pdf", + "sections": [ + "3-4", + "6.1", + "6.3-6.4" + ] + } + ], + "empirical_validation": "not_run", + "applicability": [ + "Phase interference with feasible KV handoff and measured topology", + "Equal-resource inference parallelism and pipelining comparisons" + ], + "limitations": [ + "No local campaign baseline/candidate evidence; imported evidence remains locally unverified.", + "Figure 11 is a simulated ablation, separate from real-system evaluation", + "Transfer-bound reversal is hypothetical, not an observed local regression", + "AFD hidden-state exchanges and training parallelism have different contracts" + ], + "attribution": "Yinmin Zhong et al., DistServe, OSDI 2024, pp. 193-210." +} diff --git a/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/distserve-phase-placement.md b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/distserve-phase-placement.md new file mode 100644 index 00000000..44300356 --- /dev/null +++ b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/distserve-phase-placement.md @@ -0,0 +1,31 @@ +# Phase separation must pay for state transfer + +ID: `case.distserve-phase-placement`. Owner: `optimization-search-policy`. +Axes: parallelism, pipelining. Resources: Accelerator compute/HBM, accelerator +links and Network only for actual inter-node traffic. +Source-derived candidate; local validation not_run. No local campaign evidence. + +## Source observation + +Zhong et al., [DistServe, OSDI 2024](https://www.usenix.org/system/files/osdi24-zhong-yinmin.pdf), +§§3–4, 6.1, 6.3–6.4: separate prefill/decode resource and parallelism choices, +with bandwidth-aware KV placement. Real-system tests use A100 nodes with limited +cross-node bandwidth; corresponding stages can transfer locally. Figure 11's +parallelism/placement ablation is simulated. It reports no improvement from +searching vLLM parallelism in its OPT-66B/ShareGPT setting, where the reference +choice already wins. The simulation is not another measured multi-node result. + +## Proposed decision + +Signal: measured phase interference and distinct phase service demands. +Prerequisites: supported KV handoff, compatible layouts, topology/bandwidth and +per-rank memory feasibility. Candidate: compare a phase split to the incumbent +at equal total accelerator/resource cost. Invariants: handoff completion, token +ordering, cancellation, collective order and bounded destination queues. +Metrics: joint TTFT/TPOT attainment, per-class tails, transfer bytes/time, +stage idle time, peak memory and per-resource goodput. Hypothetical counterexample: +exposed KV transfer and duplicate residency outweigh removed interference on a +constrained link. Low load may not amortize separation. Rollback: drain owned +handoffs, restore colocated placement and verify complete request outputs. + +Attribution and source version: [case record](distserve-phase-placement.json). diff --git a/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/paged-kv-allocation.json b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/paged-kv-allocation.json new file mode 100644 index 00000000..3e7f4dae --- /dev/null +++ b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/paged-kv-allocation.json @@ -0,0 +1,31 @@ +{ + "id": "case.paged-kv-allocation", + "kind": "case", + "origin": [ + "external" + ], + "state": "candidate", + "content": "paged-kv-allocation.md", + "sources": [ + { + "type": "paper", + "id": "2309.06180", + "version": "v1", + "url": "https://arxiv.org/html/2309.06180v1", + "sections": [ + "4.1-4.5", + "7.1-7.3" + ] + } + ], + "empirical_validation": "not_run", + "applicability": [ + "Dynamic KV allocation or duplication constraining HBM capacity and admission" + ], + "limitations": [ + "No local campaign baseline/candidate evidence; imported evidence remains locally unverified.", + "Paging, prefix reuse and tiering are separate interventions", + "Reported block-size and kernel counter-results are workload-specific" + ], + "attribution": "Woosuk Kwon et al., Efficient Memory Management for Large Language Model Serving with PagedAttention, arXiv:2309.06180v1." +} diff --git a/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/paged-kv-allocation.md b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/paged-kv-allocation.md new file mode 100644 index 00000000..f09e7a10 --- /dev/null +++ b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/paged-kv-allocation.md @@ -0,0 +1,30 @@ +# Allocation efficiency is distinct from prefix reuse + +ID: `case.paged-kv-allocation`. Owner: `optimization-search-policy`. +Axis: caching/state management. Resource: Accelerator HBM capacity. +Source-derived candidate; local validation not_run. No local campaign evidence. + +## Source observation + +Kwon et al., [PagedAttention, arXiv:2309.06180v1](https://arxiv.org/html/2309.06180v1), +§§4.1–4.5, 7.1–7.3: logical blocks map to separately allocated physical KV blocks; +sharing and copy-on-write reduce duplication. §7.1 reports attention-kernel +overhead despite system-level gains. §7.2/Figure 18 finds large blocks degrade +Alpaca short-sequence performance. §7.3 reports small transfers reduce swapping +efficiency. These ablations support a capacity/overhead trade-off, not a claim +that paging makes every kernel faster or makes prefix lookup unnecessary. + +## Proposed decision + +Signal: live/reserved KV and allocation traces show wasted capacity limiting admission. +Prerequisites: allocator/block accounting and a supported paged-attention path. +Candidate: change allocation/block size or sharing independently of scheduler policy. +Invariants: reference counts, copy-on-write, cancellation and no reuse before all +consumers finish. Metrics: live/reserved bytes, fragmentation, admitted work, +kernel latency, preemption/recompute and service tails/goodput. Counter-indication: +spare memory with compute saturation may offer no capacity benefit. Additional +batch capacity is not permission to weaken latency gates. Tiering is a separate +candidate with transfer costs. Rollback: restore the original allocator settings +after draining owned requests; never reinterpret live blocks under a new layout. + +Attribution and source version: [case record](paged-kv-allocation.json). diff --git a/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/prefix-reuse-locality.json b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/prefix-reuse-locality.json new file mode 100644 index 00000000..1a9db603 --- /dev/null +++ b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/prefix-reuse-locality.json @@ -0,0 +1,45 @@ +{ + "id": "case.prefix-reuse-locality", + "kind": "case", + "origin": [ + "external" + ], + "state": "candidate", + "content": "prefix-reuse-locality.md", + "sources": [ + { + "type": "paper", + "id": "2312.07104", + "version": "v2", + "url": "https://arxiv.org/html/2312.07104v2", + "sections": [ + "3", + "6.3", + "A.4" + ] + }, + { + "type": "document", + "version": "v0.10.2", + "url": "https://docs.vllm.ai/en/v0.10.2/design/prefix_caching.html", + "sections": [ + "Hash identity", + "Cache Isolation for Security", + "Data Structure", + "Operations" + ] + } + ], + "empirical_validation": "not_run", + "applicability": [ + "Exact prefix reuse with complete identity and authorized trust boundaries", + "Cache-aware scheduling with per-class fairness checks" + ], + "limitations": [ + "No local campaign baseline/candidate evidence; imported evidence remains locally unverified.", + "Hot-replica unfairness is hypothetical, not an observed source regression", + "vLLM V1 identity documentation does not describe the SGLang implementation", + "Approximate or semantic caching requires separate quality evaluation" + ], + "attribution": "Lianmin Zheng et al., SGLang, arXiv:2312.07104v2; vLLM contributors, Automatic Prefix Caching documentation v0.10.2." +} diff --git a/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/prefix-reuse-locality.md b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/prefix-reuse-locality.md new file mode 100644 index 00000000..646f5308 --- /dev/null +++ b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/prefix-reuse-locality.md @@ -0,0 +1,34 @@ +# Prefix reuse must preserve identity and queue fairness + +ID: `case.prefix-reuse-locality`. Owner: `optimization-search-policy`. +Axes: caching, scheduling. Resources: Accelerator HBM and OS/request queues. +Source-derived candidate; local validation not_run. No local campaign evidence. + +## Source observation + +Zheng et al., [SGLang, arXiv:2312.07104v2](https://arxiv.org/html/2312.07104v2), +§3, §6.3 and Appendix A.4: RadixAttention retains prefix KV with reference-counted +eviction and cache-aware ordering. Its offline cache-hit optimality assumptions +do not prove online fairness. The no-reuse benchmark reports small, nonzero +management overhead; distributed locality versus parallel efficiency remains a +trade-off. These results are not evidence of a hot-replica regression. + +[vLLM v0.10.2 prefix-cache design](https://docs.vllm.ai/en/v0.10.2/design/prefix_caching.html) +describes parent/token hashes, extra identities such as adapters/multimodal inputs, +and optional request salts for trust-group isolation in V1. This versioned design +is separate evidence, not a claim about SGLang's implementation. + +## Proposed decision + +Signal: repeated exact prefixes account for material prefill work. +Prerequisites: complete reuse identity, known invalidation and authorized sharing. +Candidate: enable exact reuse first; vary cache-local routing separately. +Invariants: model/config/input/position/adapter identity, isolation, in-flight +ownership and cleanup. Metrics: cold/warm token-weighted hits, avoided work, +lookup/eviction cost, replica load, per-class queue tails and end-to-end goodput. +Hypothetical counterexample: routing every common-prefix request to one replica +increases its queue despite higher reuse. No matching prefix or decode-dominated +work also limits opportunity. Rollback: restore routing and disable/invalidate +the candidate cache safely. Approximate response reuse needs different quality gates. + +Attribution and source versions: [case record](prefix-reuse-locality.json). diff --git a/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.json b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.json new file mode 100644 index 00000000..5801e57f --- /dev/null +++ b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.json @@ -0,0 +1,33 @@ +{ + "id": "case.sarathi-chunk-budget", + "kind": "case", + "origin": [ + "external" + ], + "state": "candidate", + "content": "sarathi-chunk-budget.md", + "sources": [ + { + "type": "paper", + "id": "Sarathi-Serve", + "version": "OSDI 2024 proceedings", + "url": "https://www.usenix.org/system/files/osdi24-agrawal.pdf", + "sections": [ + "4.1-4.3", + "5.3", + "5.4, Figure 14 and Table 4" + ] + } + ], + "empirical_validation": "not_run", + "applicability": [ + "Autoregressive serving with measured prefill/decode interference", + "Scheduling and pipeline balance on the accelerator" + ], + "limitations": [ + "No local campaign baseline/candidate evidence; imported evidence remains locally unverified.", + "Author-reported chunk overhead and component ablation; not a local experiment", + "Token budget and results do not transfer unchanged across workloads" + ], + "attribution": "Amey Agrawal et al., Sarathi-Serve, OSDI 2024, pp. 117-134." +} diff --git a/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.md b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.md new file mode 100644 index 00000000..542968f3 --- /dev/null +++ b/src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.md @@ -0,0 +1,29 @@ +# Chunk budget controls interference and overhead + +ID: `case.sarathi-chunk-budget`. Owner: `optimization-search-policy`. +Axes: scheduling, pipelining. Resource: Accelerator compute/HBM. +Source-derived candidate; local validation not_run. No local campaign evidence. + +## Source observation + +Agrawal et al., [Sarathi-Serve, OSDI 2024](https://www.usenix.org/system/files/osdi24-agrawal.pdf), +§§4.1–4.3, 5.3–5.4: schedule decodes before fitting prefill chunks into a token +budget. More uniform batch work reduces pipeline imbalance. Figure 14 reports +extra prefill time from smaller chunks for Yi-34B TP2. Table 4's two-A100, +128-request ablation separates chunking-only, hybrid-batching-only and combined +scheduling: chunking alone worsens TTFT relative to the combined configuration; +hybrid batching alone worsens P99 TBT. This is author-reported evidence, not a +universal chunk size or an InferRSI measurement. + +## Proposed decision + +Signal: decode stalls coincide with long prefills in a measured timeline. +Prerequisites: supported chunking, known token budget and request-phase accounting. +Candidate: vary only the chunk budget while preserving model, workload and resources. +Invariants: token order, complete prompts, cancellation, KV ownership and fair admission. +Metrics: TTFT, inter-token tails, queue time by class, chunk overhead, bubbles and +goodput. Counter-indication: prefill-only or low-load traffic may pay chunk overhead +without enough interference to remove. Rollback: restore the frozen scheduler and +budget, drain task-owned in-flight work safely and recheck the original workload. + +Attribution and source version are retained in [the case record](sarathi-chunk-budget.json). diff --git a/src/inferrsi/data/skill_library/methods/optimization-search-policy/metadata.json b/src/inferrsi/data/skill_library/methods/optimization-search-policy/metadata.json index 8d598948..7272e0ec 100644 --- a/src/inferrsi/data/skill_library/methods/optimization-search-policy/metadata.json +++ b/src/inferrsi/data/skill_library/methods/optimization-search-policy/metadata.json @@ -3,7 +3,14 @@ "optimization", "search", "policy", - "experiment-design" + "experiment-design", + "whole-system", + "bottleneck", + "critical-path", + "scheduling", + "caching", + "parallelism", + "pipelining" ], "status": "candidate", "profile": "local-skill-v1", @@ -12,14 +19,86 @@ "origin": [ "external" ], - "sources": [], + "sources": [ + { + "type": "paper", + "id": "Sarathi-Serve", + "version": "OSDI 2024 proceedings", + "url": "https://www.usenix.org/system/files/osdi24-agrawal.pdf", + "sections": [ + "4.1-4.3", + "5.3", + "5.4, Figure 14 and Table 4" + ], + "readset": "references/foundational-sources.md" + }, + { + "type": "paper", + "id": "2309.06180", + "version": "v1", + "url": "https://arxiv.org/html/2309.06180v1", + "sections": [ + "4.1-4.5", + "7.1-7.3" + ], + "readset": "references/foundational-sources.md" + }, + { + "type": "paper", + "id": "2312.07104", + "version": "v2", + "url": "https://arxiv.org/html/2312.07104v2", + "sections": [ + "3", + "6.3", + "A.4" + ], + "readset": "references/foundational-sources.md" + }, + { + "type": "document", + "version": "v0.10.2", + "url": "https://docs.vllm.ai/en/v0.10.2/design/prefix_caching.html", + "sections": [ + "Hash identity", + "Cache Isolation for Security", + "Data Structure", + "Operations" + ], + "readset": "references/foundational-sources.md" + }, + { + "type": "paper", + "id": "DistServe", + "version": "OSDI 2024 proceedings", + "url": "https://www.usenix.org/system/files/osdi24-zhong-yinmin.pdf", + "sections": [ + "3-4", + "6.1", + "6.3-6.4" + ], + "readset": "references/foundational-sources.md" + }, + { + "type": "document", + "version": "11.8.0", + "url": "https://docs.nvidia.com/cuda/archive/11.8.0/cuda-c-best-practices-guide/index.html", + "sections": [ + "9.1", + "9.1.1", + "9.1.2" + ], + "readset": "references/foundational-sources.md" + } + ], "empirical_validation": "not_run", "applicability": { "scope": "See procedure prerequisites and counter-signals" }, "limitations": [ "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets.", + "Foundational cases guide diagnosis after a plateau; no automatic stall detector, agent efficacy evaluation or engine speedup is established." ], "provenance_readsets": [ "references/papers-20261015/INTEGRATION_NOTES.md", @@ -27,7 +106,8 @@ "references/papers-20261015/MECHANISMS.md", "references/papers-20261015/NEXT_GATE_LEADS.md", "references/papers-20261015/references/source-notes.md", - "references/papers-20261015/updates/optimization-search-policy.md" + "references/papers-20261015/updates/optimization-search-policy.md", + "references/foundational-sources.md" ], "references": [ "../inference-optimization-casebook/cases/qwen-b300-rejected-directions.md", @@ -35,7 +115,30 @@ "../inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md", "../inference-optimization-casebook/cases/qwen-ascend-rejected-directions.json", "../inference-optimization-casebook/cases/drama3-rejected-directions.md", - "../inference-optimization-casebook/cases/drama3-rejected-directions.json" + "../inference-optimization-casebook/cases/drama3-rejected-directions.json", + "references/foundational-mechanism-decisions.md", + "cases/sarathi-chunk-budget.md", + "cases/sarathi-chunk-budget.json", + "cases/paged-kv-allocation.md", + "cases/paged-kv-allocation.json", + "cases/prefix-reuse-locality.md", + "cases/prefix-reuse-locality.json", + "cases/distserve-phase-placement.md", + "cases/distserve-phase-placement.json", + "cases/cuda-copy-overlap.md", + "cases/cuda-copy-overlap.json", + "../inference-optimization-casebook/cases/admission-drain-occupancy.md", + "../inference-optimization-casebook/cases/admission-drain-occupancy.json", + "../assess-serving-preemption/SKILL.md", + "../assess-serving-preemption/references/source-notes.md", + "../performance-ceiling-analysis/SKILL.md", + "../choose-parallel-training-plan/SKILL.md", + "../inference-benchmark-design/SKILL.md", + "../dense-performance-evidence/SKILL.md", + "../../repositories/vllm/diagnose-vllm-admission-pressure/SKILL.md", + "../../repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md", + "../../repositories/lmcache/kv-tier-management/SKILL.md", + "../../repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md" ], "consumption_mode": "packaged" } diff --git a/src/inferrsi/data/skill_library/methods/optimization-search-policy/references/foundational-mechanism-decisions.md b/src/inferrsi/data/skill_library/methods/optimization-search-policy/references/foundational-mechanism-decisions.md new file mode 100644 index 00000000..e533dcec --- /dev/null +++ b/src/inferrsi/data/skill_library/methods/optimization-search-policy/references/foundational-mechanism-decisions.md @@ -0,0 +1,135 @@ +# Foundational mechanism decisions + +Owner: `optimization-search-policy`. Candidate guidance; local efficacy not_run. +Use during Phase 1 when optimization stalls or a successful change moves the +bottleneck. This supplements the existing search/evidence loop. Resource layers +answer **where** time or capacity is constrained; intervention axes answer **how** +to change it. They are neither new SearchSpace rows nor an exhaustive taxonomy. + +## Re-derive the system before selecting a technique + +1. Freeze useful work, latency/throughput or goodput objective, quality and + fairness gates, timing boundary, topology, versions, resource cost and budget. + Keep preparation, cold start, warmup, steady admission and drain distinct. +2. Map arrival → host preparation → admission → prefill → decode → output, + including transfers, codecs and cleanup where applicable. Record dependencies, + queue waits, stage times, bytes/FLOPs and live/reserved/in-flight state owners. + Check utilization, saturation and errors per resource; utilization alone does + not prove a bottleneck. Separate HBM, host RAM, storage, accelerator links and + inter-node network. Missing measurements stay unknown. +3. Bound the opportunity with [performance-ceiling-analysis](../../performance-ceiling-analysis/SKILL.md). + For a fixed work graph, elapsed time cannot beat its dependency critical path + or work/resource-capacity bounds. Overlapping stage durations must not be + added as if serial. Amdahl bounds apply to the measured removable fraction; + changing the algorithm or removing work changes those premises. Ask what + must execute, what can be reused, what must wait and what can overlap. +4. Compare a small set of mechanisms against the same constrained resource. + For each, state the signal, expected removable cost, new work/state/traffic, + prerequisites, counter-signal, correctness invariant and cheapest falsifying + check. Include keeping the incumbent. Prefer a simpler intervention when it + addresses the same cost; extra workers or cache tiers need their own rationale. +5. If no measured signal discriminates candidates, **abstain and request + measurement**. For an unexplained plateau, first request a scoped timeline, + workload/phase breakdown and uncertainty estimate, not a larger experiment. + Never convert an unrelated keyword, unsupported backend or low utilization + into a recommendation. No remaining budget means stop and report the gap. + +## Card 1: admission versus preemption + +**Question:** is waiting caused by eligibility/order, insufficient future KV +capacity, long prefill interference, or a finite cohort's drain? + +| Evidence | Bounded candidate / owner | Counter-indication and invariant | +| --- | --- | --- | +| Whole-batch join/retire delay | Iteration eligibility; existing [Orca/preemption procedure](../../assess-serving-preemption/SKILL.md) | Does not fix a long individual prefill; preserve ordering, cancellation and cleanup | +| KV allocation failures or suspended requests | [vLLM admission pressure](../../../repositories/vllm/diagnose-vllm-admission-pressure/SKILL.md); distinguish admission from service-order preemption | Do not admit without forward-progress/recovery capacity; price recompute and restore costs | +| Prefill coincides with decode stalls | [Sarathi chunk budget](../cases/sarathi-chunk-budget.md) | Chunk overhead can hurt TTFT; check decode tails and per-class waiting as well as throughput | +| Low mean occupancy mostly during drain | Reuse [case.admission-drain-occupancy](../../inference-optimization-casebook/cases/admission-drain-occupancy.md) | Selected-interval occupancy is not whole-service goodput; retain every request in acceptance | + +Measure queue time, TTFT, inter-token latency, preemptions/recomputation and +completed requests meeting the declared SLOs per second. Check starvation and +per-class tails; a cache-friendly or short-job priority is not a fairness proof. + +## Card 2: allocation/reuse versus tiering + +**Question:** is the cost wasted allocation, repeat computation or actual resident +capacity? Autoregressive KV state, cross-request prefix reuse and paged allocation +are distinct. Compiled artifacts and staging buffers need different validity rules. + +| Evidence | Bounded candidate / owner | Counter-indication and invariant | +| --- | --- | --- | +| Reserved capacity exceeds live KV; duplication blocks admission | [Paged allocation/sharing](../cases/paged-kv-allocation.md) | Spare capacity with compute saturation may not help; retain block ownership and copy-on-write | +| Repeated exact prefixes consume prefill work | [Prefix reuse and locality](../cases/prefix-reuse-locality.md) | No reusable work or overloaded cache-local replicas; validate complete reuse identity | +| Useful state cannot remain resident; reload/recompute dominates | Existing [KV tier management](../../../repositories/lmcache/kv-tier-management/SKILL.md) | Compare retain/move/recompute including lookup, transfer, staging and eviction; no generic offload win | + +Reuse identity must include all relevant model/weight, configuration, token/input, +position and adapter identities and trust boundaries; approximate/semantic reuse +changes the quality contract. Use [KV lifetime audit](../../../repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md) +for in-flight ownership, invalidation and release. Measure cold and realistically +warm conditions, token-weighted reuse, avoided work, transfer/lookup time, +live/reserved memory and eviction churn. A hit rate is not an end-to-end benefit. + +## Card 3: pipeline bubbles versus communication/memory pressure + +**Question:** which independent work can run concurrently, and which dependent +stages can overlap across requests or chunks within the same resource budget? + +| Evidence | Bounded candidate / owner | Counter-indication and invariant | +| --- | --- | --- | +| Unbalanced batches leave stages idle | Inspect stage balance and chunk granularity using the Sarathi case | More microbatches add overhead and live buffers; distinguish drain from steady bubbles | +| Prefill/decode interference and different phase needs | [DistServe phase placement](../cases/distserve-phase-placement.md) | KV transfer or duplicate model residency can erase benefit; require measured topology and memory feasibility | +| Exposed H2D/D2H transfer with independent chunks | [CUDA copy/compute overlap](../cases/cuda-copy-overlap.md) | An asynchronous API is not overlap; backend/hardware conditions must hold | +| Attention/FFN P2P exchange on the selected CUDA path | Existing [AFD exchange audit](../../../repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md) | AFD exchanges hidden states, not DistServe's KV; preserve rank mapping and buffer ownership | + +Preserve dependencies, collective order, buffer lifetimes, cancellation and +bounded queues/backpressure. Measure the critical path, stage utilization, +bubbles, exposed communication, peak memory and resource-normalized goodput. +Extra ranks may increase traffic/contention. Single-request latency and throughput +are separate objectives. [Parallel training plans](../../choose-parallel-training-plan/SKILL.md) +apply only to training; their optimizer/activation assumptions are not inference recipes. + +## Reusable cases and existing inventory + +The six case IDs below use the existing Case contract, not a new mechanism +catalog. New interpretations are proposals; imported observations remain locally +unverified. [Source inventory and notices](foundational-sources.md) identifies +the inspected versions and the prior extraction that is reused. + +| Stable case ID | Axes / resource layer | Evidence and counterexample | +| --- | --- | --- | +| `case.admission-drain-occupancy` (existing owner) | scheduling / Accelerator, OS | Historical diagnostic; campaign INCONCLUSIVE, drain confounds mean occupancy | +| `case.sarathi-chunk-budget` | scheduling, pipelining / Accelerator | Paper ablation; smaller chunks add prefill overhead | +| `case.paged-kv-allocation` | caching / Accelerator HBM | Paper ablation; oversized blocks hurt short-sequence workloads | +| `case.prefix-reuse-locality` | caching, scheduling / Accelerator HBM, OS | Paper plus versioned identity rules; hot-replica unfairness is a hypothetical counterexample | +| `case.distserve-phase-placement` | parallelism, pipelining / Accelerator, Network when inter-node | System evaluation separate from simulated parallelism ablation; transfer-bound reversal is hypothetical | +| `case.cuda-copy-overlap` | pipelining / RAM, Accelerator link | Versioned documentation; overlap prerequisites, not an observed local speedup | + +Orca's O1–O4 and FastServe's low-load no-advantage result already live in +[assess-serving-preemption source notes](../../assess-serving-preemption/references/source-notes.md). +Alpa and split-backward training material retain their own owners. No duplicate +Orca/Alpa records or new stable skill IDs are needed. + +## Return to the existing evidence loop + +Record one hypothesis and SIAS plane, candidate change, prerequisites, invariants, +counter-indications, metrics, rollback and scope. Point to issue-scoped frozen +baseline/candidate evidence in `plan/`, `results/`, `patch/`, `validator/`; never +put campaign pins or numeric gates in reusable cases. For these imported cases, +local experiment pointers are absent because no local experiment was run. +Use [benchmark design](../../inference-benchmark-design/SKILL.md) and +[performance evidence](../../dense-performance-evidence/SKILL.md) for verification. +Preserve negative/inconclusive results and re-profile after any accepted change. + +If a later experiment is separately authorized, freeze repeated controlled runs, +low-load and saturation regimes, representative held-out traces, cold/warm state, +per-class tails, quality gates, resource budget and stop condition. Compare isolated +factors before selected interactions: locality versus fairness, KV capacity versus +batch latency, prefix reuse versus phase split, chunk size versus bubbles, and +replicas/disaggregation versus cache locality and transfer. Measure the original +baseline against the final combination directly; never add percentage gains. + +CPU fixtures check reader/selection and package contracts only. The existing Skill +Eval performance-report-audit task does not grade this diagnostic task shape; +an adapter/grader change needs its own review. Neither fixture success nor Skill +Eval scores establish agent efficacy or engine performance. No controller, +experiment, automatic promotion or canonical-store mutation is introduced. diff --git a/src/inferrsi/data/skill_library/methods/optimization-search-policy/references/foundational-sources.md b/src/inferrsi/data/skill_library/methods/optimization-search-policy/references/foundational-sources.md new file mode 100644 index 00000000..92aba33f --- /dev/null +++ b/src/inferrsi/data/skill_library/methods/optimization-search-policy/references/foundational-sources.md @@ -0,0 +1,43 @@ +# Foundational source inventory and notices + +Scope: issue #131 knowledge consolidation. Inspected 2026-10-26; no engines, +models or GPU experiments executed. Authored summaries retain attribution and +source versions; no upstream paper, figure, code or documentation is redistributed. +Upstream rights remain with their authors/publishers. Public availability does +not grant a software license. Unknown reuse rights remain UNKNOWN; the repository +license for this authored synthesis does not relicense cited sources. + +## Inventory before extraction + +- `assess-serving-preemption` already owns Orca O1–O4 and FastServe, including + source-reported low-load and kernel counter-results. Reuse its + [readset](../../assess-serving-preemption/references/source-notes.md), without + claiming a fresh paper reread or a new mechanism. +- `case.admission-drain-occupancy` already owns the finite-cohort diagnostic and + frozen evidence pointers. Reuse that ID and its INCONCLUSIVE boundary. +- `diagnose-vllm-admission-pressure`, `audit-vllm-kv-lifetimes`, + `kv-tier-management` and `audit-afd-p2p-exchanges` own engine-specific procedures. + Their retained source revisions are independent of these paper/document versions. +- `choose-parallel-training-plan` already owns Alpa stage/mesh/sharding guidance; + it remains training-only. No training procedure is relabeled as inference. +- The earlier foundation archive was absent from the packaged library. Historical + Sarathi/DistServe/PagedAttention mentions are leads, not retained case bodies. + The five new cases below fill missing decision examples; they do not reconstruct + the unavailable archive or claim its remaining papers were read. + +## Newly inspected sources + +| Source and immutable edition | Actual readset | Case / evidence boundary | Rights | +| --- | --- | --- | --- | +| Agrawal et al., [Sarathi-Serve](https://www.usenix.org/system/files/osdi24-agrawal.pdf), OSDI 2024 proceedings, pp. 117–134 | §§4.1–4.3; §5.3; §5.4/Fig. 14/Table 4, text and table | `case.sarathi-chunk-budget`; chunk/scheduler interaction and reported prefill overhead, no digitized curves | Authors/USENIX; reuse license UNKNOWN | +| Kwon et al., [PagedAttention](https://arxiv.org/html/2309.06180v1), arXiv 2309.06180v1 | §§4.1–4.5; §§7.1–7.3, text and captions | `case.paged-kv-allocation`; allocation versus kernel/block/transfer costs | See source license; no source material copied | +| Zheng et al., [SGLang](https://arxiv.org/html/2312.07104v2), arXiv 2312.07104v2 | §3; §6.3; Appendix A.4 | `case.prefix-reuse-locality`; reuse/ordering, no-reuse overhead and distributed trade-off | arXiv perpetual non-exclusive license; no redistribution grant inferred | +| [vLLM prefix caching](https://docs.vllm.ai/en/v0.10.2/design/prefix_caching.html), v0.10.2 | Hash identity, cache isolation, data structure, allocation/free/eviction | Same prefix case; V1 identity/isolation supplement, separate from SGLang | Documentation license not independently resolved: UNKNOWN | +| Zhong et al., [DistServe](https://www.usenix.org/system/files/osdi24-zhong-yinmin.pdf), OSDI 2024 proceedings, pp. 193–210 | §§3–4; §6.1; §§6.3–6.4, text/table and ablation description | `case.distserve-phase-placement`; real-system and simulated evidence distinguished | Authors/USENIX; reuse license UNKNOWN | +| NVIDIA, [CUDA C Best Practices Guide](https://docs.nvidia.com/cuda/archive/11.8.0/cuda-c-best-practices-guide/index.html), archived 11.8.0 | §§9.1–9.1.2 | `case.cuda-copy-overlap`; documented prerequisites, no experiment | NVIDIA documentation terms; no code or diagrams copied | + +Case JSON uses existing `sources`, `applicability`, `limitations` and validation +fields. Axes, resource layers, signals, invariants, metrics and rollback remain +in the case body rather than a new schema. Local baseline/candidate artifacts +are absent (not_run); imported author reports never become locally verified +through packaging, deterministic retrieval tests or matrix placement. diff --git a/tests/check_installed.py b/tests/check_installed.py index cb04b230..b2124239 100644 --- a/tests/check_installed.py +++ b/tests/check_installed.py @@ -53,6 +53,22 @@ ) assert case_path in indexed.content assert indexed.dependencies +foundational = KnowledgeReader().read("inferrsi.common.skills.optimization_search_policy") +foundational_prefix = "data/skill_library/methods/optimization-search-policy/" +for relative in ( + "references/foundational-mechanism-decisions.md", "references/foundational-sources.md", + "cases/sarathi-chunk-budget.md", "cases/sarathi-chunk-budget.json", + "cases/paged-kv-allocation.md", "cases/paged-kv-allocation.json", + "cases/prefix-reuse-locality.md", "cases/prefix-reuse-locality.json", + "cases/distserve-phase-placement.md", "cases/distserve-phase-placement.json", + "cases/cuda-copy-overlap.md", "cases/cuda-copy-overlap.json", +): + assert foundational_prefix + relative in foundational.content +foundational_record = json.loads(foundational.content["record.json"]) +assert not foundational_record["verification"]["locally_verified"] +assert foundational_record["skill_metadata"]["empirical_validation"] == "not_run" +assert case_path in foundational.content +assert "yaml" not in sys.modules, "Foundational package reads must not require the skills extra" assert json.loads(indexed.content["record.json"])["skill_metadata"]["license"] == "UNKNOWN" h3_identity = "inferrsi.common.skills.audit_h3_runtime_residency" assert any(item["id"] == h3_identity for item in search_knowledge( diff --git a/tests/fixtures/foundational_selection.json b/tests/fixtures/foundational_selection.json new file mode 100644 index 00000000..e26397ab --- /dev/null +++ b/tests/fixtures/foundational_selection.json @@ -0,0 +1,40 @@ +[ + { + "name": "stalled-whole-system-search", + "cue": "optimization stalls whole-system bottleneck critical path first principles", + "expected": "inferrsi.common.skills.optimization_search_policy", + "excluded": [] + }, + { + "name": "admission-pressure-before-preemption", + "cue": "vllm admission pressure KV allocation", + "expected": "inferrsi.common.skills.diagnose_vllm_admission_pressure", + "excluded": ["inferrsi.common.skills.choose_parallel_training_plan"] + }, + { + "name": "tiering-owner", + "cue": "LMCache L1 L2 prefetch eviction", + "expected": "inferrsi.common.skills.kv_tier_management", + "excluded": ["inferrsi.common.skills.choose_parallel_training_plan"] + }, + { + "name": "exchange-ownership", + "cue": "AFD P2pNccl rank mapping hidden states", + "backend": "cuda", + "expected": "inferrsi.cuda.skills.audit_afd_p2p_exchanges", + "excluded": ["inferrsi.common.skills.choose_parallel_training_plan"] + }, + { + "name": "unsupported-backend-abstains", + "cue": "P2pNccl", + "backend": "ascend", + "expected": null, + "excluded": ["inferrsi.cuda.skills.audit_afd_p2p_exchanges"] + }, + { + "name": "unrelated-request-abstains", + "cue": "floristry orchid", + "expected": null, + "excluded": [] + } +] diff --git a/tests/test_foundational_mechanisms.py b/tests/test_foundational_mechanisms.py new file mode 100644 index 00000000..882609b7 --- /dev/null +++ b/tests/test_foundational_mechanisms.py @@ -0,0 +1,126 @@ +"""Knowledge contracts only: no agent/model efficacy or accelerator execution.""" + +import json +import shutil +from pathlib import Path + +import pytest + +from inferrsi.catalog import get_knowledge, search_knowledge +from inferrsi.knowledge_operations import KnowledgeReader, RevisionConflict +from inferrsi.skill_catalog import skill_reference + +PROJECT = Path(__file__).resolve().parents[1] +ROOT = PROJECT / "src/inferrsi/data/skill_library" +OWNER = "methods/optimization-search-policy" +IDENTITY = "inferrsi.common.skills.optimization_search_policy" +NEW_CASES = ( + "sarathi-chunk-budget", + "paged-kv-allocation", + "prefix-reuse-locality", + "distserve-phase-placement", + "cuda-copy-overlap", +) + + +def test_case_identity_sources_and_existing_observation_are_preserved(): + reader = KnowledgeReader(skill_root=ROOT) + for slug in NEW_CASES: + bundle = reader.read(f"case.{slug}") + assert bundle.metadata["state"] == "candidate" + assert bundle.metadata["empirical_validation"] == "not_run" + assert bundle.metadata["origin"] == ["external"] + assert bundle.metadata["applicability"] and bundle.metadata["limitations"] + assert bundle.metadata["attribution"] + assert all(source["version"] for source in bundle.metadata["sources"]) + assert all(source["type"] in {"paper", "document"} + for source in bundle.metadata["sources"]) + assert bundle.content[f"{slug}.md"] == (ROOT / OWNER / "cases" / f"{slug}.md").read_bytes() + assert bundle.view()["revision"] == bundle.revision + + # One historical ID and its original evidence remain with the original owner. + existing = reader.read("case.admission-drain-occupancy") + assert existing.metadata["state"] == "historical_observation" + assert "Campaign INCONCLUSIVE" in existing.metadata["limitations"] + reference = "../inference-optimization-casebook/cases/admission-drain-occupancy.md" + assert skill_reference(ROOT, "optimization-search-policy", reference) == skill_reference( + ROOT, "inference-benchmark-design", reference, + ) + + +def test_checkout_reference_closure_and_source_drift(tmp_path): + library = tmp_path / "library" + shutil.copytree(ROOT, library) + reader = KnowledgeReader(skill_root=library) + bundle = reader.read("optimization-search-policy") + expected = { + f"{OWNER}/references/foundational-mechanism-decisions.md", + f"{OWNER}/references/foundational-sources.md", + "methods/assess-serving-preemption/references/source-notes.md", + "methods/inference-optimization-casebook/cases/admission-drain-occupancy.json", + "repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md", + "repositories/lmcache/kv-tier-management/SKILL.md", + "repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md", + } + assert expected <= bundle.content.keys() + case = reader.read("case.prefix-reuse-locality") + path = library / OWNER / "cases/prefix-reuse-locality.json" + record = json.loads(path.read_text(encoding="utf-8")) + record["sources"][0]["version"] = "fixture-changed-version" + path.write_text(json.dumps(record), encoding="utf-8") + with pytest.raises(RevisionConflict): + reader.read("case.prefix-reuse-locality", revision=case.revision) + with pytest.raises(RevisionConflict): + reader.read("optimization-search-policy", revision=bundle.revision) + + +def test_packaged_discovery_and_complete_bounded_context(): + assert IDENTITY in {r["id"] for r in search_knowledge("whole-system", repo="inferrsi")} + record = get_knowledge(IDENTITY) + assert not record["verification"]["locally_verified"] + assert record["skill_metadata"]["empirical_validation"] == "not_run" + reader = KnowledgeReader() + # A context budget must not silently drop prerequisites or counterexamples. + small = reader.collect(ids=[IDENTITY], context_bytes=1) + assert not small["contexts"] + omission, = small["context_omissions"] + assert omission["reason"] == "context_limit" + result = reader.collect(ids=[IDENTITY], context_bytes=omission["required_bytes"]) + assert not result["context_omissions"] + context, = result["contexts"] + resources = {r["path"]: r["content"] for r in context["resources"]} + prefix = f"data/skill_library/{OWNER}/" + assert prefix + "references/foundational-mechanism-decisions.md" in resources + for slug in NEW_CASES: + for suffix in ("md", "json"): + path = prefix + f"cases/{slug}.{suffix}" + assert resources[path].encode() == reader.resource( + IDENTITY, path, revision=context["revision"], + ) + + +@pytest.mark.parametrize("fixture", json.loads( + (Path(__file__).parent / "fixtures/foundational_selection.json").read_text(encoding="utf-8") +), ids=lambda fixture: fixture["name"]) +def test_frozen_method_selection_or_abstention(fixture): + # Actual lexical Collect, not a fixture-only router or an LLM grader. + result = KnowledgeReader().collect( + relevance=fixture["cue"], backend=fixture.get("backend"), + repo="inferrsi", limit=3, + ) + identities = {item["id"] for item in result["items"]} + if fixture["expected"] is None: + assert result["no_match"] and not identities + else: + assert fixture["expected"] in identities + assert not (set(fixture["excluded"]) & identities) + + +def test_missing_measurement_and_training_boundaries_reach_the_reader(): + # Text availability is the contract; this does not prove an agent obeys it. + bundle = KnowledgeReader(skill_root=ROOT).read("optimization-search-policy") + guide = bundle.content[f"{OWNER}/references/foundational-mechanism-decisions.md"].decode() + core = bundle.content[f"{OWNER}/SKILL.md"].decode() + assert "abstain and request\n measurement" in guide + assert "apply only to training" in guide + assert "not an automatic stall detector" in core diff --git a/tests/test_skill_index.py b/tests/test_skill_index.py index 01985be3..8653cb44 100644 --- a/tests/test_skill_index.py +++ b/tests/test_skill_index.py @@ -52,8 +52,10 @@ def test_complete_index_retains_metadata_states_and_unknown_license(): def test_generated_index_matches_current_sources_without_historical_git_objects(): records = search_knowledge(repo="inferrsi", limit=100) - revisions = {record["sources"][0]["commit"] for record in records} - assert len(revisions) == 1 + revisions = { + record["skill_metadata"]["name"]: record["sources"][0]["commit"] + for record in records + } # CI uses shallow checkouts. Compare the whole generated projection and its # text hashes without requiring the old source commit in the local object DB. source_files = { @@ -62,7 +64,7 @@ def test_generated_index_matches_current_sources_without_historical_git_objects( ).hexdigest() for relative in source_inputs(ROOT) } - generated = render_index(ROOT, revisions.pop(), source_files) + generated = render_index(ROOT, revisions, source_files) destination = PROJECT / DESTINATION actual = { path.relative_to(destination).as_posix(): path.read_text(encoding="utf-8") @@ -148,6 +150,82 @@ def git(*args): build_index(tmp_path, "HEAD") +def test_selected_generation_preserves_unrelated_bytes_and_checks_selected_drift( + tmp_path, monkeypatch, +): + def git(*args): + return subprocess.run( + ["git", "-C", str(tmp_path), *args], check=True, capture_output=True, + ).stdout.decode().strip() + + def commit(message): + git("add", LIBRARY.as_posix()) + git("-c", "user.name=Index fixture", "-c", "user.email=index@example.invalid", + "commit", "-s", "-m", message) + return git("rev-parse", "HEAD") + + git("init") + library = tmp_path / LIBRARY + for name in ("selected", "unrelated"): + owner = library / "methods" / name + owner.mkdir(parents=True) + (owner / "SKILL.md").write_text( + f"---\nname: {name}\ndescription: fixture\n---\nOriginal body\n", + encoding="utf-8", + ) + (library / "skill-index.json").write_text(json.dumps({ + "schema_version": 1, + "skills": {name: {"path": f"methods/{name}", "backend": "common"} + for name in ("selected", "unrelated")}, + }), encoding="utf-8") + original = commit("Create two skill sources") + monkeypatch.setattr(index_builder, "PROJECT", tmp_path) + assert index_builder.main(["--source-revision", original]) == 0 + destination = tmp_path / DESTINATION + unrelated = destination / "common/skills/unrelated.json" + original_bytes = unrelated.read_bytes() + + source = library / "methods/selected/SKILL.md" + source.write_text(source.read_text(encoding="utf-8") + "New guidance\n", encoding="utf-8") + candidate = commit("Change only selected guidance") + args = ["--source-revision", candidate, "--skill", "selected"] + assert index_builder.main([*args, "--check"]) == 1 + assert index_builder.main(args) == 0 + assert unrelated.read_bytes() == original_bytes + selected = destination / "common/skills/selected.json" + assert json.loads(selected.read_text())["sources"][0]["commit"] == candidate + assert json.loads(original_bytes)["sources"][0]["commit"] == original + assert index_builder.main([*args, "--check"]) == 0 + + selected.write_text("{}", encoding="utf-8") + assert index_builder.main([*args, "--check"]) == 1 + assert selected.read_text() == "{}" # Check is read-only. + for invalid in ("missing", ""): + with pytest.raises(SystemExit) as error: + index_builder.main(["--source-revision", candidate, "--skill", invalid]) + assert error.value.code == 2 + assert unrelated.read_bytes() == original_bytes + assert index_builder.main(args) == 0 + # Omitting --skill retains the existing full-regeneration contract. + assert index_builder.main(["--source-revision", candidate]) == 0 + assert json.loads(unrelated.read_text())["sources"][0]["commit"] == candidate + + registry = library / "skill-index.json" + entries = json.loads(registry.read_text(encoding="utf-8")) + entries["skills"]["selected"]["backend"] = "cuda" + registry.write_text(json.dumps(entries), encoding="utf-8") + moved = commit("Move selected skill to CUDA") + args = ["--source-revision", moved, "--skill", "selected"] + assert index_builder.main([*args, "--check"]) == 1 + with pytest.raises(ValueError, match="Remove obsolete generated files explicitly"): + index_builder.main(args) + # Selective generation must not hide the old projection after a backend move. + selected.unlink() + assert index_builder.main(args) == 0 + assert (destination / "cuda/skills/selected.json").is_file() + assert index_builder.main([*args, "--check"]) == 0 + + def test_package_index_and_checkout_skill_have_distinct_read_boundaries(): reader = KnowledgeReader(skill_root=ROOT) identity = "inferrsi.common.skills.claim_contract" diff --git a/tools/build_skill_index.py b/tools/build_skill_index.py index db6cd5cf..4b4affbb 100644 --- a/tools/build_skill_index.py +++ b/tools/build_skill_index.py @@ -82,13 +82,16 @@ def source_inputs(root: Path) -> set[str]: } -def render_index(root: Path, revision: str, source_files: dict[str, str]) -> dict[str, str]: +def render_index( + root: Path, revision: str | dict[str, str], source_files: dict[str, str] +) -> dict[str, str]: """Render the projection without requiring historical Git objects.""" outputs = {} links = [] entries = _registry(root) for selected in skill_records(root): name = selected["name"] + source_revision = revision[name] if isinstance(revision, dict) else revision metadata = skill_metadata(root, name) backend = entries[name]["backend"] directory = f"{LIBRARY.as_posix()}/{entries[name]['path']}" @@ -115,8 +118,8 @@ def render_index(root: Path, revision: str, source_files: dict[str, str]) -> dic "origin": metadata.get("origin", ["unknown"]), "sources": [{ "type": "git", - "url": f"{REPOSITORY}/blob/{revision}/{entry}", - "commit": revision, + "url": f"{REPOSITORY}/blob/{source_revision}/{entry}", + "commit": source_revision, "paths": paths, "content_sha256": hashes[entry], }], @@ -162,7 +165,8 @@ def render_index(root: Path, revision: str, source_files: dict[str, str]) -> dic "These are knowledge abstracts (`--kind knowledge`), with namespaced package", "IDs distinct from explicitly selected `checkout_ref` skills. Ascend/CUDA", "procedures use their corresponding backend; shared methods use `common`.", "", - f"Source snapshot: [{revision}]({REPOSITORY}/tree/{revision}/{LIBRARY.as_posix()}).", "", + "Each record retains its own source snapshot in `sources`; unchanged skills", + "can keep their existing provenance when another skill is updated.", "", "```bash", "inferrsi knowledge search 'profiling' --repo-type inferrsi --limit 50", "inferrsi collect --repo-type inferrsi --kind knowledge --limit 50 --material", @@ -173,9 +177,14 @@ def render_index(root: Path, revision: str, source_files: dict[str, str]) -> dic "`source_files` hashes identify the canonical Git bytes, not a validation result.", "", "```bash", "python tools/build_skill_index.py --source-revision ", - f"python tools/build_skill_index.py --source-revision {revision} --check", + "# Update or check one skill without repinning unrelated records:", + "python tools/build_skill_index.py --source-revision " + "--skill ", + "python tools/build_skill_index.py --source-revision " + "--skill --check", "```", "", "`--check` detects missing, stale and unexpected generated records without writing.", + "With `--skill`, only that record and the shared navigation are updated/checked.", "References and cases ship with their owning skill, not as duplicate knowledge abstracts.", "", "| Skill | Backend | Original declared status |", "| --- | --- | --- |", *links, "", @@ -186,6 +195,7 @@ def render_index(root: Path, revision: str, source_files: dict[str, str]) -> dic def main(argv: list[str] | None = None) -> int: parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("--source-revision", required=True) + parser.add_argument("--skill", help="Update/check only this registered skill and navigation") parser.add_argument("--check", action="store_true") args = parser.parse_args(argv) outputs = build_index(PROJECT, args.source_revision) @@ -195,6 +205,13 @@ def main(argv: list[str] | None = None) -> int: for path in destination.rglob("*") if path.is_file() } stale = existing - outputs.keys() + if args.skill is not None: + entry = _registry(PROJECT / LIBRARY).get(args.skill) + if entry is None: + parser.error(f"Unknown registered skill: {args.skill}") + selected = f"{entry['backend']}/skills/{args.skill}.json" + outputs = {path: outputs[path] for path in (selected, "README.md")} + stale = {path for path in stale if path.endswith(f"/skills/{args.skill}.json")} changed = [ path for path, content in outputs.items() if not (destination / path).is_file()