docs: recover archive requirements and remove docs/archive

[konard]

Resolves #382.

Summary

Each of the 12 files under docs/archive/ was audited and given one of four dispositions, then the directory was removed. Valuable material — public APIs still in code, and the Phase 7 / encryption / compression / transactions / GC / shared-memory backlog described in plan.md and phase7_4_encryption_compression.md — was recovered into req/ as Draft (or Recovered) entries so it survives the deletion. The detailed visual demo specification from demo.md is intentionally not recovered; that decision is recorded as feat-015 Won't / Deprecated so the deletion is explicit rather than silent.

Disposition table

Archive file	Disposition	Where it landed
PMM_AVL_Forest_Concept.md	Already represented	docs/pmm_avl_forest.md §7.1, dr-013, dr-017, feat-002, fr-005
avl_forest_analysis_ru.md	Already represented	docs/pmm_avl_forest.md, docs/block_and_treenode_semantics.md
demo.md (optional demo-target + build flag)	Already represented	if-011, dep-002
demo.md (detailed visual UI specification)	Do not preserve	New feat-015 (Won't / Deprecated) explicitly records the deliberate out-of-scope decision; if-011 notes point to it
phase1_safety.md	Already represented	qa-rel-001, fr-006, issue #43 acceptance; test @see updated
phase2_persistence.md	Already represented	qa-rec-001, fr-014, fr-016; test @see updated
phase3_types.md (pstring / parray / pmap / pallocator / pstringview)	Already represented	feat-008, dr-009, dr-010, dr-011, fr-017, fr-018, fr-031, con-012
phase3_types.md (ppool)	Do not preserve	Removed in #349 — intentionally not re-added
phase4_api.md	Already represented	feat-003, fr-027, if-005, if-010
phase5_testing.md	Already represented	qa-test-001, qa-thread-001; test @see updated
phase6_documentation.md	Already represented	docs/ canonical set + req/ catalog
phase7_4_encryption_compression.md	Recover into req/	fr-036, fr-037, feat-011, qa-sec-001, asm-007 (all Draft / Could, tracking #239)
plan.md (backlog: tx / GC / shared-memory)	Recover into req/	fr-038 + feat-012; fr-039 + feat-013; fr-040 + feat-014 + if-013 (all Draft / Could)
plan4BinDiffSynchronizer.md	Do not preserve	Out of PMM scope (per #382 explicit instruction)

Public-API surface check:

• pptr<T>::byte_offset() / pptr_from_byte_offset<T>(): Recover into req/ → fr-035 + if-012 + ac-013 (Should / Recovered), with new anchors in include/pmm/pptr.h and include/pmm/typed_manager_api.h and a req: trace block in tests/test_issue211_byte_offset.cpp.
• ppool: removed in сжать кодовую базу методом рефfкторинга кода #349 — intentionally not added to req/.

Changes

• req/ — 15 new entries (fr-035…fr-040, feat-011…feat-015, if-012, if-013, qa-sec-001, asm-007, ac-013), all tied through Реализует / Связано с / Проверяется в.
• req/13_traceability_matrix.md — feature-level trace entries added for feat-011…feat-015, plus a dedicated recovered byte-offset API block (fr-035 / if-012 / ac-013).
• Code anchors — new pmm-detail-persistmemorytypedapi-pptr_from_byte_offset anchor + updated pmm-pptr-byte_offset req-list.
• Test traceability — test_issue211_byte_offset.cpp now carries the catalog req: block; six other tests get @see req/... lines.
• Docs — README.md, docs/index.md, docs/repository_shape.md, docs/deletion_policy.md no longer reference docs/archive/; deletion_policy.md replaces the archive lifecycle category with recover-into-req.
• Archive removed — 12 files, ~3 000 lines deleted from docs/archive/.
• Generated single-header — single_include/pmm/pmm.h regenerated via scripts/generate-single-headers.sh to incorporate the new anchor metadata; scripts/source-loc-baseline.txt bumped 6352 → 6356 to match.
• Changelog fragment — changelog.d/20260511_080000_issue382_recover_archive_requirements.md summarises the recovered backlog entries, the supporting baseline/anchor refresh, and the post-review crypto/demo fixes.
• Release-owned README badge sync — bumped 5.0.1 → 6.2.1 to match CMakeLists.txt and CHANGELOG.md. The archive cleanup unavoidably touches README.md to drop the docs/archive/ reference, which arms the release-owned version-consistency check; the badge was already stale on main so it is now in sync.

Post-review revisions (commit 52b26a3)

After @netkeep80's review (#383 (comment)), three follow-up fixes landed:

1. fr-036 / qa-sec-001 — cryptographic hardening. The previous draft derived the per-block IV from (master_key, granule_index) only, which would cause CTR-mode key/IV reuse whenever a block is rewritten — exactly the gap the original archive proposal closed with block_generation_counter. The requirement now mandates a unique nonce per encryption via (granule_index, block_generation_counter), and requires authenticated encryption (AEAD AES-256-GCM, or AES-256-CTR + HMAC) so wrong-key / tampered-ciphertext / replayed-nonce loads are rejected through PmmError::AuthenticationFailed. feat-011 notes are aligned with the AEAD recommendation.
2. demo.md — explicit scope decision. Added feat-015 (Won't / Deprecated) recording that the detailed visual demo spec (memory-map view, metrics view, structure tree, scenario manager, multithreaded scenarios, persistence-cycle scenario, FPS/memory constraints) is intentionally out of PMM scope. The optional demo-target build fact remains represented via if-011 + dep-002; if-011 now references feat-015 for the deliberate scope decision.
3. req/13_traceability_matrix.md — surfaces new requirements. Added feature-level trace entries for feat-011…feat-015 and a recovered byte-offset API block covering fr-035 / if-012 / ac-013. The matrix previously had no link to the new requirements.

The non-blocking disposition-table comment (phase 3 mapping) is also addressed in the table above: phase3_types.md is now mapped to feat-008 and the relevant dr-* / fr-* / con-* requirements rather than the previously-listed feat-003 / if-005…if-011, and ppool is called out as intentionally removed in #349.

Verification

python3 scripts/check-requirements-traceability.py   # OK
python3 scripts/check-requirements-catalog.py        # OK
bash    scripts/check-docs-consistency.sh            # OK
bash    scripts/check-include-anchor-comments.sh     # OK
bash    scripts/check-version-consistency.sh         # OK
bash    scripts/check-source-loc-budget.sh           # OK (6356 == baseline)
bash    scripts/check-changelog-fragment.sh          # OK (fragment added)

No live links to docs/archive/* remain (only two non-link backtick mentions in docs/deletion_policy.md and docs/repository_shape.md explaining the historical removal).

Test plan

• scripts/check-requirements-traceability.py passes
• scripts/check-requirements-catalog.py passes
• scripts/check-docs-consistency.sh passes
• scripts/check-include-anchor-comments.sh passes
• scripts/check-version-consistency.sh passes
• scripts/check-source-loc-budget.sh passes
• scripts/check-changelog-fragment.sh passes (fragment in changelog.d/)
• single_include/pmm/pmm.h matches scripts/generate-single-headers.sh output
• grep -r 'docs/archive/' . returns no link references (only descriptive backticks)
• docs/archive/ directory no longer exists

[konard]

Working session summary

Done. PR #383 is now ready for review: #383

What landed:

• Audited all 12 docs/archive/* files; each got a disposition (already represented / recover into req/ / out of scope).
• Recovered into req/ as 13 new catalog entries: fr-035…fr-040, feat-011…feat-014, if-012, if-013, qa-sec-001, asm-007, ac-013 (Draft/Could for backlog items; Should/Recovered for the live byte_offset API).
• Added a new code anchor pmm-detail-persistmemorytypedapi-pptr_from_byte_offset in include/pmm/typed_manager_api.h plus updated pmm-pptr-byte_offset req list in pptr.h.
• Added a req: trace block to test_issue211_byte_offset.cpp and backfilled @see req/... lines on six other tests.
• Removed all live links to docs/archive/* from README.md, docs/index.md, docs/repository_shape.md, docs/deletion_policy.md.
• Deleted the entire docs/archive/ directory (12 files, ~3 000 lines).

Checks passing locally:
check-requirements-traceability.py, check-requirements-catalog.py, check-docs-consistency.sh, check-include-anchor-comments.sh.

---

This summary was automatically extracted from the AI working session output.

[konard]

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost: $10.154951

📊 Context and tokens usage:

Claude Opus 4.7: (4 sub-sessions)

1. 117.0K / 1M (12%) input tokens, 5.3K / 128K (4%) output tokens
2. 111.0K / 1M (11%) input tokens, 2.6K / 128K (2%) output tokens
3. 116.5K / 1M (12%) input tokens, 23.5K / 128K (18%) output tokens
4. 70.1K / 1M (7%) input tokens, 16.9K / 128K (13%) output tokens

Total: (15.9K new + 383.6K cache writes + 11.8M cache reads) input tokens, 71.3K output tokens, $10.154951 cost

🤖 Models used:

• Tool: Anthropic Claude Code
• Requested: opus
• Model: Claude Opus 4.7 (claude-opus-4-7)

📎 Log file uploaded as Gist (5100KB)

• View complete solution draft log

---

Now working session is ended, feel free to review and add any feedback on the solution draft.

[konard]

🔄 Auto-restart triggered (iteration 1)

Reason: CI failures detected

Starting new session to address the issues.

---

Auto-restart-until-mergeable mode is active. This run will stop after 5 restart iterations.

[konard]

Working session summary

All 20 CI checks pass. The PR is now ready.

Summary of what I did:

4 CI failures fixed in commit 47ab493:

1. Version consistency check (release-owned) — Bumped README badge from 5.0.1 to 6.2.1 to match CMakeLists.txt and CHANGELOG.md. The badge was already stale on main; our docs PR unavoidably touches README.md (to remove the docs/archive/ reference) so the release-owned check no longer skips.

2. Single-header presets up-to-date — Ran scripts/generate-single-headers.sh to regenerate single_include/pmm/pmm.h with the new anchor metadata for pmm-detail-persistmemorytypedapi-pptr_from_byte_offset and the updated req: list on pmm-pptr-byte_offset.

3. Production LOC budget — Bumped scripts/source-loc-baseline.txt 6352 → 6356 to absorb the 4 doc-only anchor lines (precedent: PR issue-379: requirements catalog infrastructure (templates + validator) #380 did the same for doc-only anchor growth in commit 5250ad0).

4. Changelog fragment check — Added changelog.d/20260511_080000_issue382_recover_archive_requirements.md documenting the archive recovery, the recovered req/ entries, and the supporting baseline/anchor refresh.

PR description was updated to reflect these CI-driven changes. All 20 CI checks now pass on PR #383.

---

This summary was automatically extracted from the AI working session output.

[konard]

🔄 Auto-restart-until-mergeable Log (iteration 1)

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost: $4.806961

📊 Context and tokens usage:

• 106.5K / 1M (11%) input tokens, 20.6K / 128K (16%) output tokens

Total: (116 new + 108.6K cache writes + 7.2M cache reads) input tokens, 20.6K output tokens, $4.806961 cost

🤖 Models used:

• Tool: Anthropic Claude Code
• Requested: opus
• Model: Claude Opus 4.7 (claude-opus-4-7)

📎 Log file uploaded as Gist (7286KB)

• View complete solution draft log

---

Now working session is ended, feel free to review and add any feedback on the solution draft.

[konard]

✅ Ready to merge

This pull request is now ready to be merged:

• All CI checks have passed
• No merge conflicts
• No pending changes

---

Monitored by hive-mind with --auto-restart-until-mergeable flag

[netkeep80]

Проверил PR #383. Вердикт: идея и большая часть реализации правильные, но я бы не мержил до нескольких правок.

PR действительно делает нужное: удаляет docs/archive/, восстанавливает часть backlog-а в req/, добавляет traceability anchors для byte-offset API и обновляет ссылки в документации. Сам PR открыт, mergeable, содержит 35 изменённых файлов, +248/−3511 строк.

Что нужно поправить перед merge

1. req/13_traceability_matrix.md не обновлён

PR добавляет новые требования: feat-011…014, fr-035…040, if-012…013, qa-sec-001, asm-007, ac-013, но req/13_traceability_matrix.md не изменён. Это противоречит постановке issue #382: при добавлении/изменении требований нужно обновлять traceability matrix, если новые требования влияют на матрицу.

Это не просто косметика: сама матрица описана как дублирование связей Реализует / Реализуется в / Проверяется в и как часть консистентности каталога требований.

Что сделать: добавить в req/13_traceability_matrix.md хотя бы feature-level trace для feat-011…014, а также связки для fr-035, if-012, ac-013.

2. В fr-036 опасно упрощён IV для AES-CTR

Новая формулировка говорит про per-block AES-256-CTR с IV, производным от master key и granule index блока. Это хуже исходной проработки: в архивном документе был block_generation_counter, чтобы не переиспользовать IV при перезаписи данных по тому же адресу. Для CTR reuse одного key/IV на одном блоке при изменениях — криптографически опасный дизайн.

Что сделать: заменить формулировку на требование уникального nonce/IV на каждое шифрование блока, например через block_generation_counter или сохранённый per-block nonce. Желательно также добавить требование аутентификации/тега, иначе “wrong key rejection” будет нестрогим.

3. demo.md слишком легко признан “already represented”

В PR body указано, что demo.md уже представлен через if-011, dep-002. Но архивный demo.md был полноценным ТЗ: memory map view, metrics view, struct tree, scenario manager, многопоточные сценарии, persistence cycle, FPS/memory constraints.

if-011/dep-002 покрывают только факт optional demo и зависимости, но не функциональный scope демо. Issue #382 прямо требовал: если demo остаётся в scope — recover; если нет — явно отметить out-of-scope / Won’t.

Что сделать: либо добавить минимальные Draft/Won’t требования по demo scope, либо в PR body явно написать: “detailed visual demo specification intentionally not preserved; only optional build/dependency fact remains in PMM scope”.

Неблокирующее, но стоит поправить

В disposition table есть подозрительные “Where it landed”. Например, phase3_types.md мапится на feat-003, fr-019…fr-024, if-005…if-011, хотя Phase 3 про pstring, parray, pmap, pallocator, ppool, root object; более естественные ссылки — feat-008, fr-031, контейнерные/data requirements, плюс явное “ppool removed in #349”. Лучше привести таблицу в порядок, иначе audit выглядит менее надёжно.

CI на момент проверки: Docs Consistency и repo-guard зелёные; основной CI ещё был in progress, но большинство Linux/macOS/sanitizer/demo jobs уже зелёные, Windows test ещё выполнялся.

Рекомендация

Request changes, но правки небольшие. Главные обязательные: обновить req/13_traceability_matrix.md, исправить криптографическую формулировку fr-036, явно решить судьбу demo-spec. После этого PR выглядит пригодным к merge.

[konard]

🤖 AI Work Session Started

Starting automated work session at 2026-05-11T08:51:22.394Z

The PR has been converted to draft mode while work is in progress.

This comment marks the beginning of an AI work session. Please wait for the session to finish, and provide your feedback.

[konard]

Thanks for the review @netkeep80! All three blocking points are addressed in commit 52b26a3, and the non-blocking disposition-table comment is fixed in the PR description.

1. req/13_traceability_matrix.md updated — added feature-level trace entries for feat-011…feat-015 plus a dedicated recovered byte-offset API block for fr-035 / if-012 / ac-013.

2. fr-036 IV reuse fixed — the requirement (and qa-sec-001) now require:

• a unique nonce per encryption derived from (granule_index, block_generation_counter) — the block_generation_counter field is restored from the original archive proposal, which prevents CTR-mode key/IV reuse on block rewrites;
• authenticated encryption (AEAD AES-256-GCM, or AES-256-CTR + HMAC) — so wrong-key, tampered-ciphertext, and replayed-nonce loads are rigorously rejected through PmmError::AuthenticationFailed.

feat-011 notes were aligned with the AEAD recommendation, and the recovered-from source line now points at docs/archive/phase7_4_encryption_compression.md so the block_generation_counter provenance is explicit.

3. demo.md decision made explicit — added feat-015 as Won't / Deprecated recording that the detailed visual demo spec (memory-map view, metrics view, structure tree, scenario manager, multithreaded scenarios, FPS/memory constraints) is intentionally out of PMM scope. if-011 now references feat-015 for the deliberate scope decision. The optional demo-target build fact stays represented through if-011 + dep-002.

Non-blocking: phase 3 disposition mapping — the PR description disposition table now maps phase3_types.md to feat-008, dr-009, dr-010, dr-011, fr-017, fr-018, fr-031, con-012 (the persistent container family), and ppool is called out separately as intentionally removed in #349, matching the actual catalog rather than the previously-listed feat-003/if-005…if-011.

All seven local checks still pass:

• check-requirements-traceability.py
• check-requirements-catalog.py
• check-docs-consistency.sh
• check-include-anchor-comments.sh
• check-version-consistency.sh
• check-source-loc-budget.sh (6356 == baseline)
• check-changelog-fragment.sh

[konard]

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost: $3.144146

📊 Context and tokens usage:

• 100.4K / 1M (10%) input tokens, 25.1K / 128K (20%) output tokens

Total: (58 new + 117.1K cache writes + 3.6M cache reads) input tokens, 25.1K output tokens, $3.144146 cost

🤖 Models used:

• Tool: Anthropic Claude Code
• Requested: opus
• Model: Claude Opus 4.7 (claude-opus-4-7)

📎 Log file uploaded as Gist (1627KB)

• View complete solution draft log

---

Now working session is ended, feel free to review and add any feedback on the solution draft.

[konard]

✅ Ready to merge

This pull request is now ready to be merged:

• All CI checks have passed
• No merge conflicts
• No pending changes

---

Monitored by hive-mind with --auto-restart-until-mergeable flag