Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .agents/skills/precheck-pr/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ description: Self-review a System1-Omni branch before opening a PR or marking it

# PR self-review

Read the root [agent entrypoint](../../../AGENTS.md) and applicable
[lessons](../../../lessons.md) before this review. Use the canonical
[development loop](../../../docs/development-loop.md) for evidence freshness and
handoff; independent read-only reviews remain outside the contributor-intent gate.

Review the contributor's changes against System1-Omni's conventions. Read
`CONTRIBUTING.md`, `README.md`, and applicable `AGENTS.md` instructions from the
checkout before starting. Paths below are relative to the repository root.
Expand Down
5 changes: 5 additions & 0 deletions .agents/skills/self-review/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ description: Self-review System1-Omni changes before requesting maintainer revie

# System1-Omni self-review

Read the root [agent entrypoint](../../../AGENTS.md) and applicable
[lessons](../../../lessons.md) before this review. Use the canonical
[development loop](../../../docs/development-loop.md) for evidence freshness and
handoff; independent read-only reviews remain outside the contributor-intent gate.

Start with the [human contributor-intent gate](#human-contributor-intent-gate).

Follow the target checkout's `CONTRIBUTING.md` when present and its PR template.
Expand Down
5 changes: 5 additions & 0 deletions .agents/skills/system1-omni-review/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ description: Review ThinkFlowLab/system1-omni pull requests for Rust serving, mo

# System1-Omni PR review

Read the root [agent entrypoint](../../../AGENTS.md) and applicable
[lessons](../../../lessons.md) before this review. Use the canonical
[development loop](../../../docs/development-loop.md) for evidence freshness and
handoff; independent read-only reviews remain outside the contributor-intent gate.

Use in `ThinkFlowLab/system1-omni` (repository ID `1386454853`). Produce a local reviewer report. The existing `precheck-pr` and `self-review` skills support contributor preparation; this skill supplies repository-specific independent review and does not replace their checklist. It grants no authority to edit code, push, post reviews/comments, approve, change labels or merge. It does not authorize model downloads, GPU reservations, paid execution or benchmark campaigns.

## Establish evidence
Expand Down
5 changes: 5 additions & 0 deletions .agents/skills/write-optimization-blog/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,11 @@ Turn implementation and experiment records into a readable engineering post.
Explain the bottleneck, the change that addresses it, and the measured result.
Keep measured behavior distinguishable from implemented support and future work.

When the destination is System1-Omni, first read its root
[AGENTS.md](../../../AGENTS.md), [CONTRIBUTING.md](../../../CONTRIBUTING.md),
and applicable [lessons](../../../lessons.md). For another repository, use that
repository's own instructions and canonical lessons.

## Choose the destination and evidence

Inspect the target repository's blog/docs conventions and contribution rules.
Expand Down
39 changes: 39 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Working in System1-Omni

Before implementation or contributor self-review, read [CONTRIBUTING.md](CONTRIBUTING.md),
including its human contributor-intent gate. Before changing or reviewing an area,
consult [lessons.md](lessons.md) and the relevant contracts below. Follow any more
specific instructions in the directory you are working in; report conflicts.
Independent read-only review stays outside the contributor-intent gate.

## Find the source of truth

- Setup and entrypoints: [README](README.md) and [first CPU decision](docs/getting-started.md).
- Ownership, implemented versus planned layers, numerical and lifetime invariants:
[architecture contracts](docs/architecture.md).
- Commands, platforms and validation handoffs:
[change-to-validation matrix](CONTRIBUTING.md#choose-validation-by-changed-area).
- Contributor self-review and independent maintainer review:
[contribution process](CONTRIBUTING.md#self-review-before-requesting-review).
- Resuming work, evidence freshness, safe retries and measurements:
[development loop](docs/development-loop.md).
- Preventing repeated mistakes and recording verified incidents: [lessons](lessons.md).

## Navigate the implementation

Transport lives in `src/frontend/`; admission and shared execution interfaces in
`src/runtime/`; model-specific processing/execution in `src/models/`; hardware
operations in `src/backends/`. Read the affected component README and model
recipe before editing. New tests belong under root `tests/`; preserve explicit
Cargo target wiring and the test-module wiring rules in CONTRIBUTING.

Use the relevant repository skill deliberately: [add-new-model](.agents/skills/add-new-model/SKILL.md),
[precheck-pr](.agents/skills/precheck-pr/SKILL.md),
[self-review](.agents/skills/self-review/SKILL.md), or
[independent review](.agents/skills/system1-omni-review/SKILL.md).
Their presence does not guarantee automatic invocation by any tool.

Keep code, contracts and user-facing documentation aligned in the same change.
Record exact revisions, personally run commands, observed CI and unverified
hardware separately. These instructions grant no permission to publish, merge,
change settings, install monitoring, or start paid/GPU runs.
15 changes: 11 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ Review the [README](README.md) and
[architecture and integration contracts](docs/architecture.md) for the project's
ownership boundaries and current implementation status before making changes.

Before changing or reviewing an area, read the applicable [lessons](lessons.md).
Use the [development loop](docs/development-loop.md) to checkpoint evidence and
resume work. Agent-assisted contributors also follow [AGENTS.md](AGENTS.md);
these entrypoints link to the same canonical guidance.

## Before implementation or a PR

Complete the [human contributor-intent gate](.agents/skills/self-review/SKILL.md#human-contributor-intent-gate)
Expand Down Expand Up @@ -97,11 +102,13 @@ from the repository root and record the exact revision and environment.
| Frontend transport (`src/frontend/`) | `cargo test -p omni-jev --test frontend --locked`; include `cargo test -p omni-jev --test forwarding_limits --test in_process --locked` when those paths change. | Check request/response, failure and cancellation behavior against the [frontend contract](src/frontend/README.md), then run Rust CI. |
| Native admission or execution interface (`src/runtime/`) | `cargo test --locked -p omni-runtime` | Check affected workers against the [runtime contract](src/runtime/README.md) and [architecture invariants](docs/architecture.md), then run Rust CI. |
| Benchmark tooling (`benchmarks/`, `tests/benchmarks/`) | Install `benchmarks/requirements.txt`; run `python benchmarks/bench.py validate benchmarks/smoke.jsonl` and `python -m unittest discover -s tests/benchmarks -p 'test_*.py' -v`. | Follow the [benchmark protocol](benchmarks/README.md) for performance claims. Tooling smoke tests are not model-quality or throughput measurements. |
| CUDA contract or backend tooling | Use the `cuda-contract` job's current commands and prerequisites in [CI](.github/workflows/ci.yml). | Text-based contract checks do not execute kernels. Changed native/device behavior needs the affected model recipe's hardware and numerical evidence. |
| Model processing, execution, batching or backend behavior | Select tests under root `tests/` using the model's manifest, README and recipe. | Follow the [architecture contracts](docs/architecture.md), [model/hardware coverage](docs/supported-models.md) and model-specific validation recipe. Identify unsupported and untested configurations; arrange unavailable hardware checks with the maintainer. |
| CUDA contract or backend tooling | Use the `cuda-contract` job's current commands and prerequisites in [CI](.github/workflows/ci.yml). | This is a CPU/static job: it does not compile CUDA or execute kernels. Choose further checks for the changed behavior from the affected model recipe and available resources. |
| Model processing, execution, batching or backend behavior | Select tests under root `tests/` using the model's manifest, README and recipe. | Follow the [architecture contracts](docs/architecture.md), [model/hardware coverage](docs/supported-models.md) and model-specific validation recipe. Identify unsupported and untested configurations. For performance claims or concrete numerical/performance risk, use the [conditional comparison guidance](.agents/skills/self-review/SKILL.md#conditional-ab-checks). Report unavailable checks and their effect on the claim; agree additional hardware validation separately. |

The author supplies reproducible results and names any unresolved validation
handoff. Maintainer review remains separate from author evidence. Do not claim
The matrix distinguishes existing checks from change-specific validation choices;
it creates no new hardware gate or obligation to supply hardware. The author
reports reproducible results and unresolved validation limits. Maintainer review
remains separate from author evidence. Do not claim
merge authority or required-check settings from this matrix or passing CI alone.
Before handing off, record the PR head, commands, results and evidence links,
remaining blockers, and next action. Recheck evidence after a head change;
Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,10 @@ request: self-review the full diff, keep serving, processing, scheduling, and
model execution separate according to the [architecture contracts](docs/architecture.md),
and run the checks appropriate to your changes.

For agent-assisted work, start with [AGENTS.md](AGENTS.md). The
[development loop](docs/development-loop.md) explains resumable handoffs, and
[lessons](lessons.md) records verified prevention rules.

**Have an idea or found a problem?** [Open an
issue](https://github.com/ThinkFlowLab/system1-omni/issues/new) with the
details, or [send a pull
Expand Down
11 changes: 8 additions & 3 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,11 @@ design. Concrete input/output types follow each executor's supported layout.

## Implementation status

The [Rust frontend](../src/frontend/README.md) currently forwards HTTP requests
to separately running workers. Cua-S1 and Open-Jev have native Rust/CUDA workers
The [Rust frontend](../src/frontend/README.md) binary forwards HTTP requests
to separately running workers. Its library also provides an in-process transport
for the runtime's byte-oriented `Engine` interface; admission, readiness and queue
reporting stay worker-owned. This interface does not implement shared model
processing or dynamic batching. Cua-S1 and Open-Jev have native Rust/CUDA workers
that share the [Qwen3.5/3.8 executor](../src/models/qwen3_5/native/), which accepts
single prompts and bounded packed prefill. Cua-S1 and Open-Jev-9B use
single-prompt calls; Open-Jev-27B-v1.1 packs candidates within one request for
Expand Down Expand Up @@ -118,7 +121,9 @@ executor, with FIFO waiting. Engines own scheduler instances; model-specific
execution adapters declare the unit and submit owned closures. Waiting is async,
so queued requests do not occupy blocking threads waiting for a model lock.
Schedulers are local to each worker/executor; this does not coordinate separate
processes. Queue-length limits and token budgets are not implemented yet.
processes. `SerialScheduler` does not impose queue-length limits or token budgets.
The separate `Engine` interface can report a worker's capacity and refusals; it
does not itself implement an admission policy.

- Batch only work accepted by the same loaded executor, with compatible
checkpoint/adapter identity, device, dtype, and input layout. Models declare
Expand Down
98 changes: 98 additions & 0 deletions docs/development-loop.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# Development loop and handoff

This is a contributor-operated workflow using existing GitHub issues, PR bodies,
comments and check links. It defines no installed watcher, background runner,
webhook, hook or new permission. Automation deployment and its owner, storage,
concurrency controls and delivery verification remain a separate decision.

Start from [CONTRIBUTING](../CONTRIBUTING.md); agents also read
[AGENTS.md](../AGENTS.md). Consult [lessons](../lessons.md) before changing or
reviewing an area. Architecture and validation stay in their
[canonical contracts](architecture.md) and
[change-to-validation matrix](../CONTRIBUTING.md#choose-validation-by-changed-area).

## One change, one evidence trail

1. Complete the existing human intent gate, define the bounded change and select
validation from the matrix. Record affected components and prerequisites.
2. Implement the bounded change. Add or update tests when behavior changes and
update affected contracts/documentation. Capture revision,
platform/tool versions, exact command, start/end time, result and evidence URL.
3. Self-review the complete diff and fill the existing PR template. A green CPU
suite does not validate GPU behavior; author evidence does not replace
maintainer review. Keep blocked validation and its handoff owner visible.
4. For multi-session work or an actual handoff, optionally keep a **Handoff
checkpoint** in the authorized PR body or issue. Reuse existing PR evidence
and record only fields needed to resume the work. If used, read it back to
verify persistence. Without a remote item, keep relevant state in the
contributor's private task workspace until publication is authorized; do not
commit run state.
5. On resumption, read any checkpoint and current remote state before choosing the
next action. Preserve past results as history, clearly separated from evidence
applicable to the current head. A closed issue or merged PR does not complete
every dependency automatically.

## Handoff checkpoint fields

When a checkpoint is useful, reuse the existing item rather than opening a
duplicate tracker. The following are optional fields to select for the actual
handoff, not a checklist required for every change:

- Repository identity and item number/URL; observation time and sources.
- Open/closed/merged state; actual PR base/head SHA. Issues have no inherent head;
link their relevant PRs and revisions instead.
- Each relevant check's name, run/attempt, checked SHA, status/conclusion,
platform, observation time and evidence link. Keep unknown/stale separate.
- Review/requirement status and validation owners, independently of author test
results. Unknown branch-protection or merge authority remains unknown.
- Dependencies, blockers, next action and responsible person, or explicitly
unassigned handoff. Include what evidence will unblock it.
- For an authorized external action: a stable transition/target key, purpose,
planned/in-flight/confirmed/uncertain state and returned external identifier.
Preserve enough history to avoid replaying it after a restart.

An action key includes repository/item, action type and meaningful evidence
version (for example head + check run/attempt + conclusion), not only the head.
Read before editing a shared record; coordinate concurrent writers. If a write's
result is uncertain, inspect the destination before retrying. A deduplication
record is not a guarantee of exactly-once delivery or authorization to act.

## Reconcile changes and demonstrate the protocol

This synthetic walkthrough exercises the rules, not a deployed tracker:

| Observation | Checkpoint / next action |
| --- | --- |
| Head A, run 10 pending | Save A/run 10 and the pending validation handoff; read it back after a pause. |
| Same A/run 10 failed | Refresh CI evidence despite unchanged head; record a new failure transition and diagnosis handoff. |
| Identical failure observed again | Retain the existing transition key; do not duplicate an action or notification. |
| Same A, rerun 11 succeeds | Record the new attempt and supersede current CI status; do not erase history or infer human approval. |
| Head B replaces A | A's check evidence is historical; select and obtain B's checks. Reassess review applicability under actual repository policy. |
| Base changes while head stays B | Reassess merge-result validation; refresh reviews, requirements and mergeability independently. |

Before a separately authorized merge or other consequential action, refresh the
actual head and relevant gates. Check runs and legacy commit statuses are separate
sources: an empty status list is not a failing check. If a source cannot be read,
record the last known result and its age without calling it current.

## Bounded execution and useful measurements

For benchmarks or resource-intensive work, follow time/attempt limits when the
applicable recipe or agreed execution plan specifies them. Do not impose a new
budget on ordinary contribution work. Retry only safe transient failures within
any applicable limit; a retry cannot expand authorization. Diagnose deterministic
failures instead of repeatedly running them. Stop on access denial, exhausted
agreed resources, scope change or missing human decision, and report the blocker.
Do not run parallel jobs against conflicting mutable resources.

CI timeouts apply to their configured jobs, not as universal local-test budgets.
Use the [benchmark protocol](../benchmarks/README.md) only where applicable;
GPU reservations, downloads and paid runs require separately agreed scope and
available resources. Record failures and skipped/ignored tests honestly.

For an iteration improvement claim, define the observation window and metric
before collecting it. Examples are elapsed time from first submitted revision to
first actionable validation result, or PR open-to-merge time. Record timestamps,
data sources, sample count, exclusions and unfinished items. Count independent
incidents using [lessons](../lessons.md), not alerts. No baseline or speedup is
claimed by this initialization; one successful PR cannot establish causality.
46 changes: 46 additions & 0 deletions lessons.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Lessons

Read relevant active lessons before changing or reviewing an area. After a verified
failure or review correction, propose a concise prevention rule in the same PR or
its explicit follow-up. Keep normal architecture and review policy in their
canonical documents; this file records evidence-backed learning.

## Active lessons

No verified lessons recorded yet. This initialization does not infer historical
incidents from old PRs or count its acceptance examples as real failures.

## Entry format

For each actual lesson, record:

- Stable lesson ID, affected scope, and concise prevention rule.
- Incident key and reviewer-accessible source link; observed failure separately
from suspected cause, with confidence when attribution is uncertain.
- Concrete preventive check and its canonical test/tool/document location.
- Named owner or explicitly unassigned next action, plus active/resolved status.
- Independent incident count and the distinct incident keys supporting it.

Do not add empty incident entries to satisfy this format. Keep secrets, private
conversation links, personal data and one-off execution dumps out of the file.

## Bounds, recurrence and archive

The active file limit is 200 lines, including this guidance.

- Keep this active file within that limit. Consolidate
duplicate lessons before archiving resolved or superseded detail.
- Count independent failures, not retries, repeated observations or duplicate
reports of one incident. Uncertain identity stays unresolved, not incremented.
- Use the observed failure mechanism and impact to propose an appropriate
preventive change for review. There is no default incident-count threshold or
automatic escalation; any new test or policy depends on repository needs and
available resources.
- When there is material to archive, create `docs/lessons-archive/YYYY-MM.md`,
retain source/incident history, and leave a short active-file reference.
No archive is created until there is real content. Never hide an unresolved
risk to meet the line limit.

Contributors propose updates through ordinary review; maintainers decide policy
changes. See [the development loop](docs/development-loop.md) for durable task
state and [CONTRIBUTING](CONTRIBUTING.md) for validation and review.
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,3 +77,4 @@ nav:
- LFM2.5-350M choice worker: recipe/lfm2/README.md
- D3 reference worker: recipe/d3/reference.md
- Contributing: CONTRIBUTING.md
- Development loop: docs/development-loop.md
Loading
Loading