Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions SESSION-HANDOFF.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,31 @@

> 状态:当前接班入口。旧的批量生产 session 快照已失效,不得用于恢复自动循环;持续运行使用只读 supervisor + 有界 writer epoch。

## 2026-07-14 总结不足后再推进 4 篇补强论文

- status:`local-ready`(已完成本地内容、receipt、atlas 和主要门禁;远端 PR / merge / Pages deploy 仍以实时外部状态为准)。
- 起始 ref:`6dd71d8868a0142b88f2afefbdce353dba147678`(PR #32 merge 后的 `origin/main`)。
- 分支:`study/papers-20260714-one-more-round`。
- 本轮不足总结:上一轮 40 篇完成了规模和部署闭环,但多数卡片仍是 `STATIC_ANALYSIS` / `UNVERIFIED`;部分卡片 91 行、低于建议 100 行;L4 主要是 toy / manual simulation;主题上对 ML 工程 agent、终端 agent、长上下文有效窗口、视觉 Web GUI agent 的覆盖仍不够。
- 本轮 objective:新增 4 篇更厚的 `study-v2` paper note,分别补强 `MLE-bench`、`Terminal-Bench`、`RULER`、`VisualWebArena`,保持 `UNVERIFIED` 边界,不声明运行真实 benchmark。
- scope:允许新增 `src/content/docs/papers/*.md`、`data/review-receipts/papers/*.json`,刷新 `data/note-index.json`、papers atlas 派生页和公开计数文案;不修改候选队列、policy/threshold、既有论文正文语义。
- activated_by:`explicit-user-request-2026-07-14-summarize-gaps-and-advance-one-more-round`
- review_after:`2026-07-14`
- acceptance_checks:
- `lr search arxiv` + arXiv API 元数据核验 4/4;
- `node scripts/quality-gate.mjs` 逐篇通过,行数分别为 135 / 137 / 147 / 138,无 advisory;
- `npm run audit:content-contract`:0 blocking,56 v2;
- `npm run atlas`:2028 notes,69 chunks;
- `npm run audit:counts`:projects=961、papers=1067、total=2028;
- `npm run audit:links` / `npm run audit:wikilinks`:无 blocking;
- `git ls-files -co --exclude-standard -z | node scripts/audit-public-redlines.mjs --stdin0`:0 blocking;
- `npm run build:strict -- --log /tmp/study-one-more-round-build-clean.log`:通过;首次失败由 stale `.astro` cache 触发 duplicate-id warning,删除 ignored cache 后恢复;
- `git diff --check`:通过。
- budget:1 个内容小批次、4 篇新增 paper、1 个可写切片、1 个本地 writer。
- external_outcome:当前为本地 review-ready branch;PR、merge 和 Pages deploy 需要以单独外部动作完成并复核。
- stop_conditions:规范工具链不可用;arXiv 来源不可核验;content contract / redline / strict build 失败且无法在 scope 内修复;需要改 policy/threshold、候选队列或敏感内容;用户停止。
- superseded_by:`none`

## 2026-07-14 新增 40 篇论文全流程完成记录

- status:`complete`
Expand Down
154 changes: 141 additions & 13 deletions data/note-index.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,16 @@
"taxonomy_version": "taxonomy-v1",
"stats": {
"summary": {
"total": 2024,
"classified": 1977,
"total": 2028,
"classified": 1981,
"unclassified": 47,
"unknown_difficulty": 1975,
"empty_description": 1970
},
"by_area": {
"papers": {
"total": 1063,
"classified": 1044,
"total": 1067,
"classified": 1048,
"unclassified": 19,
"unknown_difficulty": 1014,
"empty_description": 1013
Expand Down Expand Up @@ -20611,6 +20611,38 @@
"chunk_route": "/study/atlas/papers/topic-papers-nlp-foundations-and-scaling-01/"
}
},
{
"id": "papers::mle-bench",
"area": "papers",
"slug": "mle-bench",
"title": "MLE-bench — 用 Kaggle 任务衡量机器学习工程 agent",
"description": "用 MLE-bench 理解 ML 工程 agent 为什么不能只靠单元测试和代码 benchmark 来评估。",
"difficulty": "intermediate",
"canonical_topics": [
"papers-agents-and-llm-systems"
],
"classification": {
"state": "classified",
"source": "frontmatter-category",
"topic_id": "papers-agents-and-llm-systems",
"matched_category": "agent",
"raw_category": "AI Agent / MLE Benchmark"
},
"trust": {
"contract_state": "v2",
"verification_status": "UNVERIFIED"
},
"freshness": {
"state": "NOT_EVALUATED",
"reviewed_at": "2026-07-14",
"review_after": null
},
"route": "/study/papers/mle-bench/",
"atlas": {
"chunk_id": "topic-papers-agents-and-llm-systems-01",
"chunk_route": "/study/atlas/papers/topic-papers-agents-and-llm-systems-01/"
}
},
{
"id": "papers::mlflow",
"area": "papers",
Expand Down Expand Up @@ -26401,6 +26433,38 @@
"chunk_route": "/study/atlas/papers/topic-papers-network-protocols-01/"
}
},
{
"id": "papers::ruler-long-context",
"area": "papers",
"slug": "ruler-long-context",
"title": "RULER — 真实长上下文能力不能只看 NIAH",
"description": "用 RULER 理解长上下文模型的有效窗口、检索幻觉和聚合推理为什么要分开评测。",
"difficulty": "intermediate",
"canonical_topics": [
"papers-agents-and-llm-systems"
],
"classification": {
"state": "classified",
"source": "frontmatter-category",
"topic_id": "papers-agents-and-llm-systems",
"matched_category": "llm",
"raw_category": "LLM / Long Context Evaluation"
},
"trust": {
"contract_state": "v2",
"verification_status": "UNVERIFIED"
},
"freshness": {
"state": "NOT_EVALUATED",
"reviewed_at": "2026-07-14",
"review_after": null
},
"route": "/study/papers/ruler-long-context/",
"atlas": {
"chunk_id": "topic-papers-agents-and-llm-systems-01",
"chunk_route": "/study/atlas/papers/topic-papers-agents-and-llm-systems-01/"
}
},
{
"id": "papers::rwkv-2023",
"area": "papers",
Expand Down Expand Up @@ -30633,6 +30697,38 @@
"chunk_route": "/study/atlas/papers/topic-papers-distributed-training-and-gpu-01/"
}
},
{
"id": "papers::terminal-bench",
"area": "papers",
"slug": "terminal-bench",
"title": "Terminal-Bench — 在真实命令行任务里测试 agent",
"description": "用 Terminal-Bench 理解终端环境为什么能暴露 agent 的长程执行、环境理解和验证能力。",
"difficulty": "intermediate",
"canonical_topics": [
"papers-agents-and-llm-systems"
],
"classification": {
"state": "classified",
"source": "frontmatter-category",
"topic_id": "papers-agents-and-llm-systems",
"matched_category": "agent",
"raw_category": "AI Agent / Terminal Benchmark"
},
"trust": {
"contract_state": "v2",
"verification_status": "UNVERIFIED"
},
"freshness": {
"state": "NOT_EVALUATED",
"reviewed_at": "2026-07-14",
"review_after": null
},
"route": "/study/papers/terminal-bench/",
"atlas": {
"chunk_id": "topic-papers-agents-and-llm-systems-01",
"chunk_route": "/study/atlas/papers/topic-papers-agents-and-llm-systems-01/"
}
},
{
"id": "papers::tesla-architecture-2008",
"area": "papers",
Expand Down Expand Up @@ -32513,6 +32609,38 @@
"chunk_route": "/study/atlas/papers/topic-papers-nlp-foundations-and-scaling-02/"
}
},
{
"id": "papers::visualwebarena",
"area": "papers",
"slug": "visualwebarena",
"title": "VisualWebArena — 让网页 agent 真正看见界面",
"description": "用 VisualWebArena 理解多模态 web agent 为什么不能只读 DOM 文本,还要处理视觉线索。",
"difficulty": "intermediate",
"canonical_topics": [
"papers-agents-and-llm-systems"
],
"classification": {
"state": "classified",
"source": "frontmatter-category",
"topic_id": "papers-agents-and-llm-systems",
"matched_category": "agent",
"raw_category": "AI Agent / Multimodal Web"
},
"trust": {
"contract_state": "v2",
"verification_status": "UNVERIFIED"
},
"freshness": {
"state": "NOT_EVALUATED",
"reviewed_at": "2026-07-14",
"review_after": null
},
"route": "/study/papers/visualwebarena/",
"atlas": {
"chunk_id": "topic-papers-agents-and-llm-systems-01",
"chunk_route": "/study/atlas/papers/topic-papers-agents-and-llm-systems-01/"
}
},
{
"id": "papers::vit",
"area": "papers",
Expand Down Expand Up @@ -32761,8 +32889,8 @@
},
"route": "/study/papers/voyager/",
"atlas": {
"chunk_id": "topic-papers-agents-and-llm-systems-01",
"chunk_route": "/study/atlas/papers/topic-papers-agents-and-llm-systems-01/"
"chunk_id": "topic-papers-agents-and-llm-systems-02",
"chunk_route": "/study/atlas/papers/topic-papers-agents-and-llm-systems-02/"
}
},
{
Expand Down Expand Up @@ -33112,8 +33240,8 @@
},
"route": "/study/papers/webgpt-2021/",
"atlas": {
"chunk_id": "topic-papers-agents-and-llm-systems-01",
"chunk_route": "/study/atlas/papers/topic-papers-agents-and-llm-systems-01/"
"chunk_id": "topic-papers-agents-and-llm-systems-02",
"chunk_route": "/study/atlas/papers/topic-papers-agents-and-llm-systems-02/"
}
},
{
Expand Down Expand Up @@ -33176,8 +33304,8 @@
},
"route": "/study/papers/webxskill/",
"atlas": {
"chunk_id": "topic-papers-agents-and-llm-systems-01",
"chunk_route": "/study/atlas/papers/topic-papers-agents-and-llm-systems-01/"
"chunk_id": "topic-papers-agents-and-llm-systems-02",
"chunk_route": "/study/atlas/papers/topic-papers-agents-and-llm-systems-02/"
}
},
{
Expand Down Expand Up @@ -33400,8 +33528,8 @@
},
"route": "/study/papers/wizardlm-2023/",
"atlas": {
"chunk_id": "topic-papers-agents-and-llm-systems-01",
"chunk_route": "/study/atlas/papers/topic-papers-agents-and-llm-systems-01/"
"chunk_id": "topic-papers-agents-and-llm-systems-02",
"chunk_route": "/study/atlas/papers/topic-papers-agents-and-llm-systems-02/"
}
},
{
Expand Down Expand Up @@ -64331,7 +64459,7 @@
"page": 2,
"pages": 2,
"route": "/study/atlas/papers/topic-papers-agents-and-llm-systems-02/",
"entries": 1
"entries": 5
},
{
"id": "topic-papers-ai-safety-and-interpretability-01",
Expand Down
55 changes: 55 additions & 0 deletions data/review-receipts/papers/mle-bench.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
{
"schema_version": "study-review-receipt-v1",
"generation": 1,
"predecessor_digest_sha256": null,
"note": {
"area": "papers",
"slug": "mle-bench",
"digest_sha256": "b093cd115076731f58de53743a0cf5a47825c14c74d0f59731cf1f8f01b3415c"
},
"source_revision": "arXiv:2410.07095v6",
"research_input_sha256": "8693c68493b19f35534fc4630738ec7c7e77bdc70ddbe247e870e49aae2d3ae6",
"reviewers": [
{
"role": "ZERO_BASE",
"reviewer_version": "study-static-review-20260714-one-more-round",
"decision": "PASS_WITH_NOTES",
"score": 86,
"warnings": [
"Explains Kaggle-style ML engineering with a toy workflow; no Kaggle task was executed."
],
"execution": {
"review_mode": "STATIC_REVIEW",
"code_mode": "MANUAL_SIMULATION"
}
},
{
"role": "ENGINEER",
"reviewer_version": "study-static-review-20260714-one-more-round",
"decision": "PASS_WITH_NOTES",
"score": 84,
"warnings": [
"Engineering claims are limited to static paper reading; AIDE scaffold and benchmark harness were not run."
],
"execution": {
"review_mode": "STATIC_REVIEW",
"code_mode": "NOT_APPLICABLE"
}
},
{
"role": "ACADEMIC",
"reviewer_version": "study-static-review-20260714-one-more-round",
"decision": "PASS_WITH_NOTES",
"score": 84,
"warnings": [
"Citation identity was checked through arXiv metadata; reported medal rates are not independently reproduced."
],
"execution": {
"review_mode": "STATIC_REVIEW",
"code_mode": "NOT_APPLICABLE"
}
}
],
"waivers": [],
"created_at": "2026-07-14T14:30:00.000Z"
}
55 changes: 55 additions & 0 deletions data/review-receipts/papers/ruler-long-context.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
{
"schema_version": "study-review-receipt-v1",
"generation": 1,
"predecessor_digest_sha256": null,
"note": {
"area": "papers",
"slug": "ruler-long-context",
"digest_sha256": "fdea3ef9179c032b4040312f8720d8e43b0a09628bbf8aef951386d00a1cea81"
},
"source_revision": "arXiv:2404.06654v3",
"research_input_sha256": "585001909e88932e9aa31c90a0fb62d22fe20ca6e3e5588dcbcc0cc841768ac8",
"reviewers": [
{
"role": "ZERO_BASE",
"reviewer_version": "study-static-review-20260714-one-more-round",
"decision": "PASS_WITH_NOTES",
"score": 87,
"warnings": [
"Explains RULER through a toy long-context tracing example; benchmark tasks were not executed."
],
"execution": {
"review_mode": "STATIC_REVIEW",
"code_mode": "MANUAL_SIMULATION"
}
},
{
"role": "ENGINEER",
"reviewer_version": "study-static-review-20260714-one-more-round",
"decision": "PASS_WITH_NOTES",
"score": 84,
"warnings": [
"Engineering claims are limited to static evaluation design reading; no model inference was run."
],
"execution": {
"review_mode": "STATIC_REVIEW",
"code_mode": "NOT_APPLICABLE"
}
},
{
"role": "ACADEMIC",
"reviewer_version": "study-static-review-20260714-one-more-round",
"decision": "PASS_WITH_NOTES",
"score": 85,
"warnings": [
"Citation identity was checked through arXiv metadata; reported long-context results are not independently reproduced."
],
"execution": {
"review_mode": "STATIC_REVIEW",
"code_mode": "NOT_APPLICABLE"
}
}
],
"waivers": [],
"created_at": "2026-07-14T14:30:00.000Z"
}
Loading