Skip to content

docs(research): follow evidence through the decision - #7276

Merged
huangruiteng merged 1 commit into
mainfrom
codex/research-decision-evidence-20261024
Oct 24, 2026
Merged

huangruiteng merged 1 commit into
mainfrom
codex/research-decision-evidence-20261024

Conversation

@loopx-agent

Copy link
Copy Markdown
Collaborator

Submission-adjacent trajectory samples can omit the earlier comparisons and selection that produced a candidate. This can leave a mechanism report unable to use existing evidence and make its next-experiment proposal repeat work already done.

Add a focused curation section to the existing trajectory-assessment reference: follow the design, comparison, selection and actual subsequent work; record coverage with existing selection.rule and episode source_ids; reconcile applicable prior comparisons before proposing another experiment. Preserve negative results, missing links, local/native attribution boundaries and earlier report versions. This is curator guidance, with no automatic completeness gate or new execution authority.

Validation on 262e05858bd6b32ba111a516246e1a8e41862417:

  • Committed whitespace and RFC bilingual checks passed; no RFC pair changed.
  • Quick premerge passed all 4 direct checks and the selected public-boundary check, with no failures or manual holds.
  • Exact-scope change-quality receipt cqr_466ebf419c4a9fe80f0c passed.
  • Final prose and public/private review completed. This documentation proposal contains no raw study evidence; no model or benchmark run was launched. Agent adherence, scientific calibration and researcher benefit require separate validation.

Ready for maintainer review; no merge or installation performed.

Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent; model=gpt-6.1-sol; provider=OpenAI; reasoning_effort=max; declaration_source=runtime_reported; observation_id=8eed7a7787e6f4b0855bb9cee6781fc12f5a9b929a6abe2cbdee3ac70e1614c4

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

动机

整理研究轨迹的操作者,需要判断一次选择是否使用了先前实验,而相关比较往往早于最终提交附近的消息。 只看提交附近的片段,容易把“当前材料没提供比较”说成“整次运行从未比较”,进而重复已经做过的实验。新增指南要求在授权历史中追到候选、比较、选择和后续行动,并保留仍然缺失的环节为未知。 整理者现在能在原有选择说明和逐段来源里交代决策覆盖,补充材料使用明确的后继输入包,保留旧报告、失败和消耗。 本次只改研究证据的整理指南;不修改评估器、采样器或格式,不授予分析和实验执行权限,也不声称模型已按指南执行。

改动思路

这段说明复用原来的 selection.rule、逐段 source_ids 和后继包规则,把比较结果如何进入选择、又如何进入实际工作说清楚。 当前改动只有一个参考文档的30行;人工整理与机器引用校验、模型分析与实验执行仍各有原来的边界。 Add one section under existing evidence-selection guidance and reuse selection.rule/source_ids/successor rules; a new schema or automated completeness classifier would add an unjustified owner.

具体改动

完整审查唯一改动 docs/reference/trajectory-assessment.md +30/-0,准确头 262e05858bd6b32ba111a516246e1a8e41862417,实际merge-base 984691c6ac54686d6747f4325564e68d81397f85。规格取同一文档修改前完整提交 984691c6ac54686d6747f4325564e68d81397f85,完整旧文本与新增段落均已阅读,未用新增段落作为它自己的验收条件。TA标签仅是评审映射。

  • TA-SOURCE implemented:Literal selected ranges remain bound to existing episode source_ids and selection.rule; citation presence alone is not entailment. 实际路径:docs/reference/trajectory-assessment.md#evidence-selection;证据:manual whole-clause review; unchanged runtime tree。

  • TA-UNKNOWN implemented:Known population, supplied evidence and actual events remain distinct; unknown is not zero or proof of absence. 实际路径:docs/reference/trajectory-assessment.md#coverage-and-cost;证据:omitted-earlier-comparison counterexample and seven clause boundaries。

  • TA-SUCCESSOR implemented:Changed source material requires an explicit successor/full reassessment; retain old reports, failures and usage. 实际路径:docs/reference/trajectory-assessment.md#continuation-and-adoption;证据:pre-change continuation contract and added successor paragraph。

  • TA-AUTHORITY implemented:Curator judgment, uncalibrated observer assessment and matched scientific attribution do not grant execution authority. 实际路径:docs/reference/trajectory-assessment.md;证据:pre-change producer/scientific boundaries compared to all30 added lines。

关键内容讲解

新增段落放在 Coverage and cost 与 Producer boundary 之间。第一段把提交附近信息定位为入口,要求沿假设、候选、比较、选择到实际后续工作找来源;不存在的链路仍未知。第二段复用 selection.rule 与 source_ids,并明确材料不足不等于整次运行没有实验。第三段在提出新比较前对照既有设计与结果,保留输入、时点和复用限制,把真实后期收益与尚未确定的机制分开。最后一段要求有界原文片段、引用复用、明确后继包及旧报告/失败/用量保留,同时明确这些是整理者判断。

Git逐路径检查证明运行时代码、来源格式、提示词加载器、分析provider和CLI都没有改动。quick premerge四项直接检查与一项公开边界检查通过。没有为了这段文档重复运行模型或实验,也没有查询远端CI。

对主干的风险

主要反例是:旧报告只拿到最后提交附近片段,实际上早先做过一次比较,后来的改动又带来真实增益但没有因果归因。按新增指南,整理者应在已有授权历史内找出先前候选、输入/预算和结果,绑定对应原文,再解释是否被选择与后续使用;找不到的部分继续未知,旧报告的材料不足不能被追改成“从未做过”。新增来源进入明确后继包,并遵守原有完整重评估与旧用量保留规则。这个人工推演验证文义和边界,不能冒充实际模型理解或科研收益的实证。

“Within the authorized history”和“bounded excerpts”限制了材料范围,未要求无边界复制整次运行。已有比较不自动免除必要复验,新的比较也不由这段文字自动获准。原文引用不能被模型摘要替代;引用校验通过仍不证明选择完整或归因正确。指南的“优先缺失条件/反例/必要重复”是整理建议,没有新完整性状态、机器拒绝门或额外确认步骤。

这次不改变已有前端旅程、保留规则、评分、原生测量所有者或实验冻结条件,不需要为参考说明添加新设置页。未验证真实操作者或模型采用、节省预算或长期效果,这些限制保留为未知。回滚只需撤回新增说明,不会删除既有输入包、报告或失败证据。

我的整体评价

APPROVE,当前明确的文档缺口已补齐,未把更大研究项目说成完成。这段说明复用原来的 selection.rule、逐段 source_ids 和后继包规则,把比较结果如何进入选择、又如何进入实际工作说清楚。 当前改动只有一个参考文档的30行;人工整理与机器引用校验、模型分析与实验执行仍各有原来的边界。 一段参考指南已足以解释决策链与来源选择,新建自动选择器、状态字段或完整性检查器没有现成调用需求;无需在当前PR扩大架构或设立形式化后续任务。

long_horizon 为 preserved:旧证据、未知、失败与成本均保留,未依据文字本身宣称长程收益。user_experience 为 improved:同一整理任务利用已有历史与来源引用,不新增必填表单或确认轮次;要求有范围的原文片段,不要求把整次运行全部搬入模型。 仅 docs/reference/trajectory-assessment.md 新增整理说明,没有页面渲染器、组件或样式变化。 没有新增渲染呈现需要核验;共享安装、真实模型理解及长期收益未验证。

严格质量回执 cqr_671c038829f482c95831 为PASS。预期改进是让整理决定有可追溯依据;实际模型是否采用、结论是否更准仍须各自证据。本评审不授予实验、发布、正式自我批准或合并权限;当前头若继续改动,应重新审查新增文义与边界。

English verdict: APPROVE — 262e05858bd6b32ba111a516246e1a8e41862417. The30-line curator guide follows evidence through comparisons, selection and actual subsequent work while preserving unknowns, literal source bindings, explicit successor packs, prior failures/usage and execution boundaries. The immutable pre-change contract, complete diff and bounded counterexample were reviewed; docs-only premerge and strict CQR pass. Model adoption, effectiveness and cost benefits are unverified; no merge performed.

@huangruiteng
huangruiteng merged commit a925e74 into main Oct 24, 2026
23 of 25 checks passed
@huangruiteng
huangruiteng deleted the codex/research-decision-evidence-20261024 branch October 24, 2026 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants