[Quality-547] add bert_tiny_v2 model and update input classifier by evelyn-with-warp · Pull Request #10756 · warpdotdev/warp

evelyn-with-warp · 2026-05-12T19:10:38Z

Description

Add retrained nld classifier to dogfood;
Meanwhile changing model file storage from raw blobs to lfs for optimizing git efficiency

It could resolve the misfires reported in quality-547; For quality 543 on file paths misfires, some are still pending to resolve by improving heuristics.

About switching to git lfs to store model binary files

Scenario	Potential Impact	Failure Risk	Mitigation
Fresh clone with `git-lfs` installed	`git clone` (and subsequent `git pull`) automatically invokes the smudge filter, which fetches the real ONNX blobs from LFS in place of the pointer files. `rust-embed` embeds the actual model bytes at build time.	None — classifier loads normally.	No mitigation needed; this is the happy path.
Fresh clone without `git-lfs` installed	ONNX paths land as ~130-byte pointer stubs. `rust-embed` embeds the pointer file at build time; runtime tries to load it as ONNX.	`candle::InferenceRunner::new` fails and the classifier silently falls back to `HeuristicClassifier`. App still builds and runs — no compile-time break, but a behavioral regression.	`script/bootstrap` now refuses to proceed if `git-lfs` is missing and runs `git lfs install --local && git lfs pull` when present; `WARP.md` documents it.
Existing clone, user pulls this branch with `git-lfs` installed but never enabled in the repo	`git lfs install --local` hasn't been run, so the smudge filter is inactive.	The v1 file may stay correct (already a blob locally) but the new v2 file appears as a pointer.	Same bootstrap step covers it; users who skip bootstrap can run `git lfs install && git lfs pull` once.
Existing clone, user pulls without ever installing `git-lfs`	The v2 model lands as a pointer (silent classifier fallback).	Build succeeds, but classifier silently degrades to heuristic fallback.	Bootstrap requirement + `WARP.md` note + the runtime `log::warn!("Failed to load onnx classifier (v2): ...")` makes the failure greppable in logs.
Existing clone with `git-lfs` installed and enabled	`git pull` invokes the smudge filter automatically, fetching the v2 blob from LFS to replace the pointer. `rust-embed` picks up the real bytes on the next build.	None — classifier loads normally.	No mitigation needed; this is the happy path for existing clones.
Forks / branches based on pre-LFS commits	Cherry-picks or merges crossing the LFS boundary need `git-lfs` locally to render the binaries correctly.	Low — no history rewrite is needed since we're only renormalizing on this commit.	Old branches keep working with raw blobs because their `.gitattributes` doesn't yet have the LFS rule.

TL;DR — Risk Mitigation Summary

1. Fresh clones (hard enforcement at bootstrap)
The actual gate lives in script/bootstrap, which now (a) refuses to proceed if git-lfs is not installed, and (b) runs git lfs install --local && git lfs pull automatically when it is. The WARP.md note complements this by explaining the requirement and providing recovery commands for users who bypass bootstrap.

2. Existing clones (tiered recovery)

Hard failure path: If a build or runtime error surfaces (e.g. candle::InferenceRunner::new panics or the agent reports a missing model), the agent's remediation is to run git lfs install --local && git lfs pull once and retry.
Silent fallback path: If the build succeeds but the classifier silently regresses to HeuristicClassifier, we accept it as a non-blocking degradation. The log::warn!("Failed to load onnx classifier (v2): ...") line makes the condition greppable in logs, and the next bootstrap run will self-heal.

3. CI (requires explicit opt-in to LFS)
Update CI with: lfs: true to any job that compiles the app, runs classifier tests, or bundles release artifacts. Please see 3bf9211

Linked Issue

The linked issue is labeled ready-to-spec or ready-to-implement.
Where appropriate, screenshots or a short video of the implementation are included below (especially for user-visible or UI changes).

Testing

I have manually tested my changes locally with ./script/run
For the CI changes on LFS, we see below logs in CI logs to verify CI does fetch raw binary model files

28217b297a * app/assets/windows/arm64/OpenConsole.pdb (30 MB)
6eca0f5ba1 * app/assets/windows/arm64/conpty.pdb (6.4 MB)
3cf69e7c9b * app/assets/windows/x64/OpenConsole.pdb (34 MB)
e9347e3fa6 * app/assets/windows/x64/conpty.pdb (6.5 MB)
05b2aaba0e * crates/input_classifier/models/onnx/bert_tiny_v1.onnx (18 MB)
1fd8afc92b * crates/input_classifier/models/onnx/bert_tiny_v2.onnx (18 MB)

macos CI
Linux CI
Windows CI

Screenshots / Videos

https://www.loom.com/share/94f99078f8614b2bbc05eab7f8d1a7d1
in this video we tested on the misfires listed in quality-547

left: warp dev
right: warp local build on this change

Video for dogfood feature
https://www.loom.com/share/196ffbc863aa4b39894f4e821ab54d85

Agent Mode

Warp Agent Mode - This PR was created via Warp's AI Agent Mode

oz-for-oss · 2026-05-12T19:10:51Z

@evelyn-with-warp

I'm starting a first review of this pull request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

oz-for-oss

Overview

This PR adds a v2 ONNX input classifier model, switches classifier model files under crates/input_classifier/models/** to Git LFS, updates the v2 feature gate to load the new model, and adds local bootstrap documentation for LFS setup.

Concerns

CI and release workflows still check out the repository without LFS enabled, so cargo/bundle jobs that do not run ./script/bootstrap can embed Git LFS pointer stubs instead of the classifier binaries and silently ship the heuristic fallback.

Verdict

Found: 0 critical, 1 important, 0 suggestions

Request changes

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

oz-for-oss · 2026-05-12T19:17:08Z

 *.pdb filter=lfs diff=lfs merge=lfs -text
-input_classifier/models/* filter=lfs diff=lfs merge=lfs -text
-input_classifier/models/**/*tokenizer.json linguist-generated=true
+crates/input_classifier/models/** filter=lfs diff=lfs merge=lfs -text


⚠️ [IMPORTANT] Moving these embedded models to LFS also requires CI/release checkouts to fetch LFS objects; the existing workflows use actions/checkout without lfs: true and prepare_environment does not run git lfs pull, so bundle jobs can embed pointer stubs and ship the heuristic fallback. Update the shared workflow checkout path with lfs: true or run git lfs pull before cargo/bundle commands.

see the mitigation solution in PR description; for CI, we have updated to fetch from LFS as well (as seen in Tests section in PR description)

evelyn-with-warp · 2026-05-12T20:31:14Z

The change we see in crates/input_classifier/models/onnx/bert_tiny_v1.onnx in github refers to updating raw blob to lfs pointer

actions/checkout does not fetch LFS content by default, so the crates/input_classifier/models/** ONNX models (newly LFS-tracked) and the pre-existing *.pdb debug symbols would otherwise appear as ~133-byte pointer stubs on CI runners. rust-embed bakes the models into the binary at compile time, so without this step the bundled app embeds the pointer text and silently falls back to the heuristic classifier at runtime. Adding a 'git lfs pull' step to the shared prepare_environment composite action covers every workflow that goes through it (ci.yml + create_release.yml build/test/bundle jobs). Co-Authored-By: Oz <oz-agent@warp.dev>

evelyn-with-warp · 2026-05-12T22:10:38Z

-input_classifier/models/* filter=lfs diff=lfs merge=lfs -text
-input_classifier/models/**/*tokenizer.json linguist-generated=true
+crates/input_classifier/models/** filter=lfs diff=lfs merge=lfs -text
+crates/input_classifier/models/**/*tokenizer.json -filter -diff -merge text linguist-generated=true


this is probably the reason why previous onnx model was checked in as raw blob not lfs: the paths did not include crates/ so those lines matched nothing

i'm pretty sure it originally did use LFS; this probably broke when we moved the crate from ./input_classifier to ./crates/input_classifier. (i would be surprised if i failed to set this up properly when i originally did it.)

@vorporeal does that mean git lfs was already enabled in CI?
CI didn't fail so far
pls lmk if other tests could cross-verify

anyway , i didn't see lfs:true at every checkout, so probably worth adding/re-adding it

i believe we never needed lfs in CI because we weren't using the classifier in unit or integration tests (because it wasn't a default feature)

Stacks on the previous `prepare_environment` LFS pull (Option A) so we get an explicit, per-call-site audit trail in the workflow files and GitHub-rendered `with: lfs: true` block in the CI logs. The redundant `git lfs pull` afterwards in prepare_environment is a no-op when checkout has already materialized the LFS objects. Touches every `actions/checkout` invocation in: - .github/workflows/ci.yml (8 sites) - .github/workflows/create_release.yml (12 sites) Merged with existing `with:` blocks (`ref:` on release_linux_arm and `fetch-depth:` on generate_changelogs) rather than duplicating them. Co-Authored-By: Oz <oz-agent@warp.dev>

Adds a follow-up Verify LFS payloads step to prepare_environment so that every CI job's log contains an explicit per-file listing of the materialized LFS payloads, e.g.: 1fd8afc92b * crates/input_classifier/models/onnx/bert_tiny_v2.onnx (17 MB) 05b2aaba0e * crates/input_classifier/models/onnx/bert_tiny_v1.onnx (17 MB) Makes it trivial to confirm at a glance that the ONNX classifier binaries reached the runner as actual blobs rather than 133-byte pointer stubs. The leading asterisk means the data is present in the working tree (not just a pointer). Co-Authored-By: Oz <oz-agent@warp.dev>

This reverts commit c0e05a7. We already verified LFS payloads load correctly via the prepare_environment-only path (commits 3bf9211 and 596a622), so the per-checkout `lfs: true` opt-ins are redundant. Drop them to keep the diff to .github/workflows/* minimal and rely on `prepare_environment` to do the single `git lfs pull`. Co-Authored-By: Oz <oz-agent@warp.dev>

vorporeal · 2026-05-13T00:12:33Z


 ### Platform Setup
- `./script/bootstrap` - Platform-specific setup (calls platform-specific bootstrap scripts)
+- `./script/bootstrap` - Platform-specific setup (calls platform-specific bootstrap scripts). Also verifies `git-lfs` is installed and runs `git lfs install --local && git lfs pull` so model binaries under `crates/input_classifier/models/**` are materialized in the working tree.


this is not a useful update to our top-level public documentation. agents like to add extra context here - in the extreme, this lists every single thing that the bootstrap script does, which does not provide value to the end user reading our readme.

removed in d9608a3
the initiative was giving agent some hint when there's lfs related failure, but it's a fair point that agent might able to get this from session context as well

vorporeal · 2026-05-13T00:13:06Z

 - `./script/install_cargo_build_deps` - Install Cargo build dependencies
 - `./script/install_cargo_test_deps` - Install Cargo test dependencies

+#### Git LFS


i also don't think we need this here - the bootstrap script will print out a notice with this exact same information, right?

vorporeal · 2026-05-13T00:16:45Z

 #[derive(Clone, Copy, RustEmbed)]
 #[folder = "models/onnx"]
 struct Models;


we should be changing this so that we're only bundling one model into the binary instead of both.

you probably want to use #[cfg_attr(feature = "...", include = "...")], to switch from unconditionally including everything in the models folder to using an allowlist

vorporeal · 2026-05-13T00:19:57Z

+    # Print a per-file listing of the LFS payloads so the CI log shows that
+    # the ONNX/fasttext model binaries (and Windows .pdb files) are real
+    # binaries rather than 133-byte pointer stubs. The `*` marker before
+    # each path means "data is present in the working tree".
+    - name: Verify LFS payloads
+      shell: bash
+      run: |
+        cd "${{ steps.repo-root.outputs.path }}"
+        git lfs ls-files --size


now that we've verified that things are working properly, we can remove this, yeah? if you want to keep it, we should at least condition it on the action being executed in debug mode.

Removes the extra detail appended to the `./script/bootstrap` entry and the dedicated "#### Git LFS" subsection. Per review feedback, this level of context belongs in the bootstrap script's own output rather than in the top-level WARP.md documentation. Co-Authored-By: Oz <oz-agent@warp.dev>

Per PR review feedback, the `git lfs ls-files --size` step is only useful for on-demand auditing of LFS payload integrity. Now that we've verified things work end-to-end, gate the step on `runner.debug == '1'` so it only runs when the workflow is re-run with debug logging (or ACTIONS_RUNNER_DEBUG=true), keeping normal CI logs clean. Co-Authored-By: Oz <oz-agent@warp.dev>

feat: add bert_tiny_v2 model and update input classifier

e6b12ba

cla-bot Bot added the cla-signed label May 12, 2026

oz-for-oss Bot reviewed May 12, 2026

View reviewed changes

remove redundant comment

edc9114

evelyn-with-warp mentioned this pull request May 12, 2026

[Quality-547] Update bert_tiny model to the retrained classifier #10529

Closed

4 tasks

evelyn-with-warp force-pushed the evelyn/quality-547 branch from 84580d2 to 3bf9211 Compare May 12, 2026 22:06

evelyn-with-warp commented May 12, 2026

View reviewed changes

evelyn-with-warp requested review from szgupta and vorporeal May 12, 2026 22:11

evelyn-with-warp and others added 3 commits May 12, 2026 15:55

vorporeal reviewed May 13, 2026

View reviewed changes

evelyn-with-warp and others added 2 commits May 12, 2026 17:28

evelyn-with-warp force-pushed the evelyn/quality-547 branch from 12472c0 to 617cceb Compare May 13, 2026 00:32

islolate compile paths on classifier v1 & v2

e849231

evelyn-with-warp requested a review from vorporeal May 13, 2026 01:05

Conversation

evelyn-with-warp commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

About switching to git lfs to store model binary files

TL;DR — Risk Mitigation Summary

Linked Issue

Testing

Screenshots / Videos

Agent Mode

Uh oh!

oz-for-oss Bot commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

oz-for-oss Bot left a comment

Choose a reason for hiding this comment

Overview

Concerns

Verdict

Uh oh!

oz-for-oss Bot May 12, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

evelyn-with-warp commented May 12, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

evelyn-with-warp May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

evelyn-with-warp commented May 12, 2026 •

edited

Loading

oz-for-oss Bot commented May 12, 2026 •

edited

Loading

evelyn-with-warp May 12, 2026 •

edited

Loading