Skip to content

Add bounded structural code tools and typed source patterns - #74

Open
AmirrezaFarnamTaheri wants to merge 12 commits into
tinyhumansai:mainfrom
AmirrezaFarnamTaheri:feat/structural-code
Open

AmirrezaFarnamTaheri wants to merge 12 commits into
tinyhumansai:mainfrom
AmirrezaFarnamTaheri:feat/structural-code

Conversation

@AmirrezaFarnamTaheri

@AmirrezaFarnamTaheri AmirrezaFarnamTaheri commented Oct 18, 2026 •

Copy link
Copy Markdown

Canonical upstream compatibility — October 18, 2026

Merged TinyTools canonical main e1c9683f546434e6147d02e4d837eae16e3d48ba into the structural code branch without dropping the new bounded structural engine. Current fork head: 0d349716deafc4d91a595bf85400e8445cd1a6e8; zero behind this upstream revision. Canonical workflow_dispatch was restored, the temporary merge workflow removed, and fork CI completed successfully.

Summary

Add reusable structural code inspection and boilerplate rendering for agent tools. The new pure tinytools-code crate owns parsing, bounded analysis, revision-checked edits, and typed chunk rendering. An optional tinytools-std adapter exposes these operations through the host's existing FsGate.

The dependency-light tinytools vocabulary keeps its existing dependency tree.

Related issue

None. Consumer integration: OpenHuman #14.

API or behavior changes

This is additive. The native adapter requires the new optional code feature.

  • CodeEngine supports explicit Rust, TypeScript, TSX, JavaScript, Python, Go, JSON, and TOML grammars. It provides bounded caching, incremental parsing, lexical outlines, diagnostics, folded bodies, and revision-checked targeted patches.
  • ChunkCatalog, typed manifests, and one-pass rendering provide twelve starter patterns, validated parameters and slots, and bounded custom templates. Node constraints must refer to visible, named nodes in the selected grammar. Dependency declarations are informational; rendering does not install them.
  • code_tools and code_tools_with_options expose six deferred tools: ast_outline, ast_symbol, ast_patch, chunk_search, chunk_inspect, and chunk_apply. Outline level defaults to 1; level 0 omits the folded skeleton. Chunk search defaults to five results, capped by the configured page limit.
  • Reads and writes use the host gate. Mutations require revisions and publish complete staged files after validation and response-size checks. Only literal dry_run: true requests a read-only preview. Custom manifests resolve templates within their own directory, and application checks both template and destination revisions.

Structural validation establishes syntax and selected source ranges. It does not provide typechecking or semantic reference resolution. Source-byte and entry limits bound the retained source cache, but do not establish an allocator-level bound on tree-sitter memory. Publication does not provide filesystem compare-and-swap against unrelated writers or a power-loss durability guarantee.

Validation

  • cargo fmt --all -- --check
  • cargo clippy --all-targets --all-features -- -D warnings
  • cargo build --all-targets --all-features
  • cargo test --all-features

The final source passes workspace formatting and strict Clippy on Rust 1.99. Focused tests on Rust 1.96.1 pass: 43 pure-engine tests and 35 native-adapter tests, plus execution of the fixed-fixture benchmark.

Final-revision CI passes the workspace build, both feature configurations of the test suite, the dependency-light vocabulary check, strict documentation, supply-chain checks, and the Rust 1.88 build. The unchanged 90% per-file coverage gate passes all 156 executable source files. The minimum for the new pure-engine files is 94.02%; the minimum for the native adapter is 93.66%. The UTF-8 edit-boundary module is fully covered.

Tests

New regressions cover all eight grammars, partial UTF-8 prefix and suffix edits, cache and output bounds, outline levels and pagination, selector ambiguity, stale revisions, import placement, comment and indentation preservation, typed expansion, and grammar-specific manifest constraints.

Native tests cover custom-template containment, bounded UTF-8 and escaped JSON arguments, late permission revocation, cancellation, staged publication races, no-clobber creation, response-size preflight, and permission preservation. The final available action-budget unit remains usable.

Documentation

The PR adds an accepted specification, a linked implementation plan, crate and module documentation, API examples, and benchmark usage notes.

Checklist

  • The change is focused on one logical change
  • No new #[allow(...)], #[ignore], or relaxed lints
  • No secrets, tokens, or .env contents in the diff or the description

Native test modules use a test-only clippy::unwrap_used allowance consistent with neighboring filesystem tests. Production lints, CI, and coverage rules are unchanged; no tests are ignored.

@coderabbitai

coderabbitai Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 133a409a-9de4-4342-bebc-38590264092a
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@tinysweeper

tinysweeper Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Tiny Sweeper review

Reviewed head: a49d5f9680119c415afaaa56c0147c70929ae942

Verdict: Ready for maintainer review · Priority: none
Coverage: Reviewed 87 of 88 files; not reviewed: Cargo.lock.
Pull request purpose (title): Add bounded structur…

Diagnostics and supporting details

Full summary

Pull request purpose (title): Add bounded structural code tools and typed source patterns. The change adds a pure tinytools-code crate for bounded lexical parsing, revision-checked patches, and typed boilerplate rendering, plus an optional code feature in tinytools-std exposing six FsGate-backed deferred tools, and merges upstream cross-platform compatibility fixes (non-Linux group_exit::wait returns a Ready future, empty/whitespace/NUL DNS hosts are rejected, environment and git test helpers are platform-gated). No actionable findings are reported. The latest-push delta could not be computed from a trusted comparison.

Unreviewed paths

  • Cargo.lock

Review snapshot

Change surface Files Review signal Count
Production 54 Active findings 0
Tests 17 Noted findings 0
Documentation 13 Resolved findings 0
Configuration 3 Pending checks/questions 0

Completeness: Complete
Test assessment: No supported feature-to-test mapping was available; this does not mean tests are absent or passed.

What changed

Adds a new pure tinytools-code crate (Cargo.toml, lib.rs, analysis/engine/catalog/render/patch modules with eight compiled grammars and query packs) plus an optional tinytools-std code feature wiring six FsGate-backed tools, and merges upstream compatibility fixes gating non-Linux timeout group_exit::wait, rejecting empty DNS hosts in url_guard, and platform-gating workspace_state and timeout test helpers.

Features

  • Added — Pure bounded structural engine: Provides explicit Rust, TypeScript, TSX, JavaScript, Python, Go, JSON, and TOML grammars with lexical symbols, references, diagnostics, folded source, and bounded cached/incremental parsing, including UTF-8-safe minimal edits. (crates/tinytools-code/src/engine/mod.rs, crates/tinytools-code/src/analysis/mod.rs, crates/tinytools-code/src/language/mod.rs, crates/tinytools-code/src/engine/edit.rs)
  • Added — Revision-checked patch operations: Requires an exact source revision and a unique selector for body/declaration replacement, import insertion, member appends, and single-node wrapping, validating original and resulting syntax and bounding output before construction. (crates/tinytools-code/src/patch/mod.rs, crates/tinytools-code/src/patch/types.rs, crates/tinytools-code/src/patch/imports.rs, crates/tinytools-code/src/patch/indentation.rs)
  • Added — Typed chunk catalog and one-pass rendering: Ships twelve starter patterns with validated metadata, typed parameters, original standalone comment slots, grammar-checked node constraints, and bounded expansion that never rescans inserted text. (crates/tinytools-code/src/catalog/mod.rs, crates/tinytools-code/src/render/mod.rs, crates/tinytools-code/src/catalog/starter-manifests.json, crates/tinytools-code/src/catalog/validation.rs)
  • Added — Optional FsGate-backed native code tools: code_tools and code_tools_with_options expose six deferred tools whose reads and writes pass the host gate; writes require revisions, stage complete files, recheck guards, and publish atomically with no-clobber creation or permission-preserving replacement. (crates/tinytools-std/src/code/mod.rs, crates/tinytools-std/src/code/publish.rs, crates/tinytools-std/src/code/declaration.rs, crates/tinytools-std/src/code/mutate.rs)
  • Added — Feature gating and optional dependency wiring: The tinytools-std code feature pulls in tinytools-code, serde, and tempfile only when enabled, keeping the base vocabulary crate unchanged. (crates/tinytools-std/Cargo.toml, crates/tinytools-std/src/lib.rs)
  • Modified — Cross-platform timeout DNS and test fixes: Non-Linux group_exit::wait returns a ready future, empty/whitespace/NUL DNS hosts are rejected before resolution, and environment and git helpers are platform-gated. (crates/tinytools-std/src/timeout/group_exit.rs, crates/tinytools-std/src/url_guard/mod.rs, crates/tinytools-std/src/timeout/mod_tests.rs, crates/tinytools-std/src/filesystem/workspace_state/mod_tests.rs)
  • Added — Specification and implementation plan: Documents the accepted structural-code contract, implementation plan, crate READMEs, and benchmark usage. (docs/specs/structural-code.md, docs/plans/structural-code.md, crates/tinytools-code/README.md, crates/tinytools-std/src/code/README.md)

Tests

No supported feature-to-test mapping was produced. Test execution is not inferred.

Findings

No active actionable findings.

Before merge

None.

Agent review details

critique

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Lane summary: Reviewed 2 files; 0 findings. _The code index is behind this pull request (indexed at `8ac07cc1a2a5`), so retrieved context may be out of date._

security

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The security lane reported that the added url_guard assertions are test-only host-independent rejection paths calling no new sink, and introduced no security defect.
  • Lane summary: The added assertions are test-only additions to url_guard/mod_tests.rs exercising resolve_host_ips with whitespace-only and NUL-containing host strings; both are host-independent rejection paths that call into no new sink. The diff introduces no security defect. 1 file was not security-reviewed: crates/tinytools-std/README.md (prose or tabular data). _The code index is behind this pull request (indexed at `8ac07cc1a2a5`), so retrieved context may be out of date._

tests

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Lane summary: This revision is a small follow-up on top of an already-large structural-code change: it adds a trim-empty check to `resolve_host_ips`, `#[cfg(unix)]` gates on three helpers in the workspace_state tests, a platform-aware environment variable in the timeout test, a `Ready`-returning signature for the non-Linux `group_exit::wait`, and a URL-guard regression for whitespace and NUL hosts. The new assertions are real: they exercise the added `host.trim().is_empty()` branch and the OS-resolver error path, and they cannot pass if the guard is removed. The `#[cfg(unix)]` additions and the `Ready` signature change are consistent with their callers and needed for the non-Unix build. No reportable defect found. _The code index is behind this pull request (indexed at `8ac07cc1a2a5`), so retrieved context may be out of date._

commits

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The commits lane reported no sensitive content in committed files.
  • Lane summary: No sensitive content in committed files. _The code index is behind this pull request (indexed at `8ac07cc1a2a5`), so retrieved context may be out of date._

description

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The description explains additive API changes, feature gating, and explicitly bounds the guarantees: no typechecking, no allocator-level tree-sitter bound, and no filesystem compare-and-swap or power-loss durability.
  • Lane summary: This revision adds a `code` feature row to the tinytools-std README and extends the DNS-resolution test with a whitespace-only host and an embedded-NUL host, both of which the existing `host.trim().is_empty()` guard and the OS resolver reject without network access. The test helper `rejection()` already exists in the same file, so the new assertions compile, and both match the behaviour of `resolve_host_ips` shown above. Nothing in the diff contradicts the change description. _The code index is behind this pull request (indexed at `8ac07cc1a2a5`), so retrieved context may be out of date._
Evidence and run details
  • Models: ladder/vectors-oai3, deepseek-v4.1-flash
  • Spend: $0.021001
  • Tokens: 744803 input · 4843 output · 283648 cached · 743 embedding
  • Updated: 2026-10-23T12:15:04Z
  • Continuity: summary cache chain restarted at the storage ceiling.
Head State Pass summary
2f36f7b6d540 incomplete 0 active finding(s), 0 resolved finding(s) (at 2026-10-18T19:28:34Z)
0d349716deaf incomplete 0 active finding(s), 0 resolved finding(s) (at 2026-10-18T19:33:04Z)
0d349716deaf incomplete 0 active finding(s), 0 resolved finding(s) (at 2026-10-18T19:38:01Z)
8ac07cc1a2a5 incomplete 0 active finding(s), 0 resolved finding(s) (at 2026-10-23T12:09:43Z)
a49d5f968011 ready for maintainer review 0 active finding(s), 0 resolved finding(s) (at 2026-10-23T12:15:04Z)

tinysweeper 0.1.2

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tinysweeper found nothing blocking, but could not review everything, so this is not an approval: .github/workflows/openhuman-tinytools-sync-once.yml, README.md, crates/tinytools-code/Cargo.toml, crates/tinytools-code/README.md, crates/tinytools-code/benches/structural.rs, crates/tinytools-code/src/analysis/README.md, crates/tinytools-code/src/analysis/extract.rs, crates/tinytools-code/src/analysis/fold.rs and 77 more.

$0.0001 · 0 in / 0 out · 703 embedded · ladder/vectors-oai3

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tinysweeper found nothing blocking, but could not review everything, so this is not an approval: .github/workflows/openhuman-tinytools-sync-once.yml, tinysweeper/description, tinysweeper/tests.

$0.0001 · 0 in / 0 out · 704 embedded · ladder/vectors-oai3

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tinysweeper found nothing blocking, but could not review everything, so this is not an approval: README.md, crates/tinytools-code/Cargo.toml, crates/tinytools-code/README.md, crates/tinytools-code/benches/structural.rs, crates/tinytools-code/src/analysis/README.md, crates/tinytools-code/src/analysis/extract.rs, crates/tinytools-code/src/analysis/fold.rs, crates/tinytools-code/src/analysis/mod.rs and 76 more.

$0.0001 · 0 in / 0 out · 702 embedded · ladder/vectors-oai3

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tinysweeper found nothing blocking, but could not review everything, so this is not an approval: README.md, crates/tinytools-code/Cargo.toml, crates/tinytools-code/README.md, crates/tinytools-code/benches/structural.rs, crates/tinytools-code/src/analysis/README.md, crates/tinytools-code/src/analysis/extract.rs, crates/tinytools-code/src/analysis/fold.rs, crates/tinytools-code/src/analysis/mod.rs and 76 more.

$0.0001 · 0 in / 0 out · 702 embedded · ladder/vectors-oai3

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tinysweeper found nothing blocking, but could not review everything, so this is not an approval: README.md, crates/tinytools-code/Cargo.toml, crates/tinytools-code/README.md, crates/tinytools-code/benches/structural.rs, crates/tinytools-code/src/analysis/README.md, crates/tinytools-code/src/analysis/extract.rs, crates/tinytools-code/src/analysis/fold.rs, crates/tinytools-code/src/analysis/mod.rs and 76 more.

$0.0001 · 0 in / 0 out · 702 embedded · ladder/vectors-oai3

Copy link
Copy Markdown
Author

Updated the branch through 8ac07cc. Merged current upstream main without rewriting history and resolved the manifest conflict by aligning the optional structural-code dependency with workspace v0.6.0. Validation also exposed two Windows issues: blank DNS names are now rejected explicitly before OS resolution, and the environment test checks an inherited Windows variable rather than Git Bash's synthesized HOME. Non-Linux cleanup and Unix-only test helpers now pass all-targets Clippy. Local all-features tests and doctests, all-targets build, Clippy with -D warnings, and formatting pass. New fork and upstream CI runs are being checked. The earlier automated review reported incomplete coverage, not an actionable source finding; this update does not treat that as approval.

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tinysweeper could not complete coverage (1 unanswered item(s)), so this is not an approval.

Diagnostics

Total spend: $0.0021

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tinysweeper found nothing blocking. Approving.

Diagnostics

Total spend: $0.0210

Copy link
Copy Markdown
Author

Final follow-up a49d5f9 restores deterministic OS resolver-error coverage with an embedded-NUL hostname; it performs no external DNS query. Both the fork CI run (39951692725) and upstream PR CI run (39951699555) now pass every job, including per-file 90% coverage. GitHub reports no merge conflict.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant