diff --git a/.claude/agents/rust-rtk.md b/.claude/agents/rust-rtk.md index 8efe67f0e..5adca48b9 100644 --- a/.claude/agents/rust-rtk.md +++ b/.claude/agents/rust-rtk.md @@ -1,7 +1,7 @@ --- name: rust-rtk description: Expert Rust developer for RTK - CLI proxy patterns, filter design, performance optimization -model: claude-sonnet-4-5-20250929 +model: sonnet tools: Read, Write, Edit, MultiEdit, Bash, Grep, Glob --- @@ -509,7 +509,7 @@ rtk newcmd args - Update `CLAUDE.md` Module Responsibilities table - Update `README.md` with command support -- Update `CHANGELOG.md` +- CHANGELOG.md is auto-generated by release-please — do not edit manually ## Performance Targets diff --git a/.claude/agents/technical-writer.md b/.claude/agents/technical-writer.md index f5341af46..9d81a6a60 100644 --- a/.claude/agents/technical-writer.md +++ b/.claude/agents/technical-writer.md @@ -115,7 +115,7 @@ rtk --version # Should show rtk X.Y.Z **Option 2: From Source** ```bash -git clone https://github.com/rtk-ai/rtk.git +git clone https://github.com/algolia/rtk.git cd rtk cargo install --path . rtk --version # Verify installation @@ -130,7 +130,7 @@ rtk gain # Should show token savings analytics **From Source** (Cargo required): ```bash -git clone https://github.com/rtk-ai/rtk.git +git clone https://github.com/algolia/rtk.git cd rtk cargo install --path . @@ -141,7 +141,7 @@ rtk --version **Binary Download** (faster): ```bash -curl -sSL https://github.com/rtk-ai/rtk/releases/download/v0.16.0/rtk-linux-x86_64 -o rtk +curl -sSL https://github.com/algolia/rtk/releases/download/v0.16.0/rtk-linux-x86_64 -o rtk chmod +x rtk sudo mv rtk /usr/local/bin/ rtk --version @@ -172,7 +172,7 @@ rtk --version - **Fix**: Uninstall and reinstall correct RTK ```bash cargo uninstall rtk - cargo install --path . # From rtk-ai/rtk repo + cargo install --path . # From algolia/rtk repo rtk gain --help # Should work ``` ``` diff --git a/.claude/commands/diagnose.md b/.claude/commands/diagnose.md index f91422769..78366c848 100644 --- a/.claude/commands/diagnose.md +++ b/.claude/commands/diagnose.md @@ -199,7 +199,7 @@ options: ### Fix 1 : Installer RTK localement ```bash -cd /Users/florianbruniaux/Sites/rtk-ai/rtk +# Depuis la racine du repo RTK cargo install --path . # Vérifier installation which rtk && rtk --version @@ -345,7 +345,7 @@ chmod +x .claude/hooks/*.sh **Upgrade recommendation**: If running v0.15.x or older, upgrade to v0.16.x: ```bash -cd /Users/florianbruniaux/Sites/rtk-ai/rtk +# From the RTK repo root git pull origin main cargo install --path . --force rtk --version # Should show 0.16.x or newer diff --git a/.claude/commands/tech/codereview.md b/.claude/commands/tech/codereview.md index fb0813fc3..35e92360d 100644 --- a/.claude/commands/tech/codereview.md +++ b/.claude/commands/tech/codereview.md @@ -1,6 +1,7 @@ --- model: sonnet description: RTK Code Review — Review locale pre-PR avec auto-fix +argument-hint: "[--fix] [file-pattern]" --- # RTK Code Review diff --git a/.claude/commands/tech/remove-worktree.md b/.claude/commands/tech/remove-worktree.md index edc5802da..90a63632d 100644 --- a/.claude/commands/tech/remove-worktree.md +++ b/.claude/commands/tech/remove-worktree.md @@ -1,6 +1,7 @@ --- model: haiku description: Remove a specific worktree (directory + git reference + branch) +argument-hint: "" --- # Remove Worktree diff --git a/.claude/commands/tech/worktree-status.md b/.claude/commands/tech/worktree-status.md index 9b2194eb9..faa5ca49d 100644 --- a/.claude/commands/tech/worktree-status.md +++ b/.claude/commands/tech/worktree-status.md @@ -1,6 +1,7 @@ --- model: haiku description: Worktree Cargo Check Status +argument-hint: "" --- # Worktree Status Check diff --git a/.claude/commands/tech/worktree.md b/.claude/commands/tech/worktree.md index 1f5df7afb..69dfc04d8 100644 --- a/.claude/commands/tech/worktree.md +++ b/.claude/commands/tech/worktree.md @@ -1,6 +1,7 @@ --- model: haiku description: Git Worktree Setup for RTK +argument-hint: "" --- # Git Worktree Setup diff --git a/.claude/commands/worktree-status.md b/.claude/commands/worktree-status.md index fb423f5f7..0de86d248 100644 --- a/.claude/commands/worktree-status.md +++ b/.claude/commands/worktree-status.md @@ -1,6 +1,7 @@ --- model: haiku description: Check background cargo check status for a git worktree +argument-hint: "" --- # Worktree Status Check diff --git a/.claude/commands/worktree.md b/.claude/commands/worktree.md index 17666b014..eabdff07e 100644 --- a/.claude/commands/worktree.md +++ b/.claude/commands/worktree.md @@ -1,6 +1,7 @@ --- model: haiku description: Git Worktree Setup for RTK (Rust project) +argument-hint: "" --- # Git Worktree Setup diff --git a/.claude/hooks/rtk-suggest.sh b/.claude/hooks/rtk-suggest.sh index 34fb50f3b..80c356582 100755 --- a/.claude/hooks/rtk-suggest.sh +++ b/.claude/hooks/rtk-suggest.sh @@ -97,10 +97,8 @@ elif echo "$FIRST_CMD" | grep -qE '^head\s+'; then fi # --- JS/TS tooling --- -elif echo "$FIRST_CMD" | grep -qE '^(pnpm\s+)?vitest(\s|$)'; then - SUGGESTION="rtk vitest run" -elif echo "$FIRST_CMD" | grep -qE '^pnpm\s+test(\s|$)'; then - SUGGESTION="rtk vitest run" +elif echo "$FIRST_CMD" | grep -qE '^(pnpm\s+)?vitest(\s+run)?(\s|$)'; then + SUGGESTION="rtk vitest" elif echo "$FIRST_CMD" | grep -qE '^pnpm\s+tsc(\s|$)'; then SUGGESTION="rtk tsc" elif echo "$FIRST_CMD" | grep -qE '^(npx\s+)?tsc(\s|$)'; then diff --git a/.claude/rules/search-strategy.md b/.claude/rules/search-strategy.md index 0b4504df7..09a212c91 100644 --- a/.claude/rules/search-strategy.md +++ b/.claude/rules/search-strategy.md @@ -23,8 +23,7 @@ src/ │ ├── utils.rs ← strip_ansi, truncate, execute_command │ ├── filter.rs ← Language-aware code filtering engine │ ├── toml_filter.rs ← TOML DSL filter engine -│ ├── display_helpers.rs ← Terminal formatting helpers -│ └── telemetry.rs ← Analytics ping +│ └── display_helpers.rs ← Terminal formatting helpers ├── hooks/ ← Hook system │ ├── init.rs ← rtk init command │ ├── rewrite_cmd.rs ← rtk rewrite command diff --git a/.claude/skills/code-simplifier/SKILL.md b/.claude/skills/code-simplifier/SKILL.md index 15a3ae19d..4c3c22e06 100644 --- a/.claude/skills/code-simplifier/SKILL.md +++ b/.claude/skills/code-simplifier/SKILL.md @@ -7,6 +7,13 @@ triggers: - "over-engineered" - "refactor this" - "make this idiomatic" +allowed-tools: + - Read + - Grep + - Glob + - Edit +effort: low +tags: [rust, simplify, refactor, idioms, rtk] --- # RTK Code Simplifier diff --git a/.claude/skills/design-patterns/SKILL.md b/.claude/skills/design-patterns/SKILL.md index 10045f48b..c44f79f93 100644 --- a/.claude/skills/design-patterns/SKILL.md +++ b/.claude/skills/design-patterns/SKILL.md @@ -6,6 +6,12 @@ triggers: - "how to structure" - "best pattern for" - "refactor to pattern" +allowed-tools: + - Read + - Grep + - Glob +effort: medium +tags: [rust, design-patterns, architecture, newtype, builder, rtk] --- # RTK Rust Design Patterns diff --git a/.claude/skills/issue-triage/SKILL.md b/.claude/skills/issue-triage/SKILL.md index 8fb1aadaa..635167757 100644 --- a/.claude/skills/issue-triage/SKILL.md +++ b/.claude/skills/issue-triage/SKILL.md @@ -1,7 +1,14 @@ --- +name: issue-triage description: > Issue triage: audit open issues, categorize, detect duplicates, cross-ref PRs, risk assessment, post comments. Args: "all" for deep analysis of all, issue numbers to focus (e.g. "42 57"), "en"/"fr" for language, no arg = audit only in French. +allowed-tools: + - Bash + - Read + - Grep +effort: medium +tags: [triage, issues, github, categorize, duplicates, risk] --- # Issue Triage @@ -155,7 +162,16 @@ Si toujours ambigu, demander à l'utilisateur via `AskUserQuestion`. Après affichage du tableau de triage, copier dans le presse-papier : ```bash -pbcopy <<'EOF' +# Cross-platform clipboard +clip() { + if command -v pbcopy &>/dev/null; then pbcopy + elif command -v xclip &>/dev/null; then xclip -selection clipboard + elif command -v wl-copy &>/dev/null; then wl-copy + else cat + fi +} + +clip <<'EOF' {tableau de triage complet} EOF ``` diff --git a/.claude/skills/issue-triage/templates/issue-comment.md b/.claude/skills/issue-triage/templates/issue-comment.md index 64e9146f4..93917a239 100644 --- a/.claude/skills/issue-triage/templates/issue-comment.md +++ b/.claude/skills/issue-triage/templates/issue-comment.md @@ -32,7 +32,7 @@ To move forward, we need the following: {What happens once the info is provided — e.g., "Once confirmed, we'll prioritize this for the next release."} --- -*Triaged via [rtk](https://github.com/rtk-ai/rtk) `/issue-triage`* +*Triaged via [rtk](https://github.com/algolia/rtk) `/issue-triage`* ``` --- @@ -53,7 +53,7 @@ This issue covers the same problem as #{original_number}: **{original_title}**. If your situation differs in an important way (different command, different OS, different error message), please reopen and add that context. Otherwise, follow the original issue for updates. --- -*Triaged via [rtk](https://github.com/rtk-ai/rtk) `/issue-triage`* +*Triaged via [rtk](https://github.com/algolia/rtk) `/issue-triage`* ``` --- @@ -74,7 +74,7 @@ If this is still relevant: Thanks for taking the time to report it. --- -*Triaged via [rtk](https://github.com/rtk-ai/rtk) `/issue-triage`* +*Triaged via [rtk](https://github.com/algolia/rtk) `/issue-triage`* ``` --- @@ -99,7 +99,7 @@ After review, this request falls outside RTK's current design goals. If the use case evolves or the scope changes in a future version, feel free to reopen with updated context. --- -*Triaged via [rtk](https://github.com/rtk-ai/rtk) `/issue-triage`* +*Triaged via [rtk](https://github.com/algolia/rtk) `/issue-triage`* ``` --- diff --git a/.claude/skills/performance.md b/.claude/skills/performance.md deleted file mode 100644 index 30c90b0eb..000000000 --- a/.claude/skills/performance.md +++ /dev/null @@ -1,435 +0,0 @@ ---- -description: CLI performance optimization - startup time, memory usage, token savings benchmarking ---- - -# Performance Optimization Skill - -Systematic performance analysis and optimization for RTK CLI tool, focusing on **startup time (<10ms)**, **memory usage (<5MB)**, and **token savings (60-90%)**. - -## When to Use - -- **Automatically triggered**: After filter changes, regex modifications, or dependency additions -- **Manual invocation**: When performance degradation suspected or before release -- **Proactive**: After any code change that could impact startup time or memory - -## RTK Performance Targets - -| Metric | Target | Verification Method | Failure Threshold | -|--------|--------|---------------------|-------------------| -| **Startup time** | <10ms | `hyperfine 'rtk '` | >15ms = blocker | -| **Memory usage** | <5MB resident | `/usr/bin/time -l rtk ` (macOS) | >7MB = blocker | -| **Token savings** | 60-90% | Tests with `count_tokens()` | <60% = blocker | -| **Binary size** | <5MB stripped | `ls -lh target/release/rtk` | >8MB = investigate | - -## Performance Analysis Workflow - -### 1. Establish Baseline - -Before making any changes, capture current performance: - -```bash -# Startup time baseline -hyperfine 'rtk git status' --warmup 3 --export-json /tmp/baseline_startup.json - -# Memory usage baseline (macOS) -/usr/bin/time -l rtk git status 2>&1 | grep "maximum resident set size" > /tmp/baseline_memory.txt - -# Memory usage baseline (Linux) -/usr/bin/time -v rtk git status 2>&1 | grep "Maximum resident set size" > /tmp/baseline_memory.txt - -# Binary size baseline -ls -lh target/release/rtk | tee /tmp/baseline_binary_size.txt -``` - -### 2. Make Changes - -Implement optimization or feature changes. - -### 3. Rebuild and Measure - -```bash -# Rebuild with optimizations -cargo build --release - -# Measure startup time -hyperfine 'target/release/rtk git status' --warmup 3 --export-json /tmp/after_startup.json - -# Measure memory usage -/usr/bin/time -l target/release/rtk git status 2>&1 | grep "maximum resident set size" > /tmp/after_memory.txt - -# Check binary size -ls -lh target/release/rtk | tee /tmp/after_binary_size.txt -``` - -### 4. Compare Results - -```bash -# Startup time comparison -hyperfine 'rtk git status' 'target/release/rtk git status' --warmup 3 - -# Example output: -# Benchmark 1: rtk git status -# Time (mean ± σ): 6.2 ms ± 0.3 ms [User: 4.1 ms, System: 1.8 ms] -# Benchmark 2: target/release/rtk git status -# Time (mean ± σ): 7.8 ms ± 0.4 ms [User: 5.2 ms, System: 2.1 ms] -# -# Summary -# 'rtk git status' ran 1.26 times faster than 'target/release/rtk git status' - -# Memory comparison -diff /tmp/baseline_memory.txt /tmp/after_memory.txt - -# Binary size comparison -diff /tmp/baseline_binary_size.txt /tmp/after_binary_size.txt -``` - -### 5. Identify Regressions - -**Startup time regression** (>15% increase or >2ms absolute): -```bash -# Profile with flamegraph -cargo install flamegraph -cargo flamegraph -- target/release/rtk git status - -# Open flamegraph.svg -open flamegraph.svg -# Look for: -# - Regex compilation (should be in lazy_static init) -# - Excessive allocations -# - File I/O on startup (should be zero) -``` - -**Memory regression** (>20% increase or >1MB absolute): -```bash -# Profile allocations (requires nightly) -cargo +nightly build --release -Z build-std -RUSTFLAGS="-C link-arg=-fuse-ld=lld" cargo +nightly build --release - -# Use DHAT for heap profiling -cargo install dhat -# Add to main.rs: -# #[global_allocator] -# static ALLOC: dhat::Alloc = dhat::Alloc; -``` - -**Token savings regression** (<60% savings): -```bash -# Run token accuracy tests -cargo test test_token_savings - -# Example failure output: -# Git log filter: expected ≥60% savings, got 52.3% - -# Fix: Improve filter condensation logic -``` - -## Common Performance Issues - -### Issue 1: Regex Recompilation - -**Symptom**: Startup time >20ms, flamegraph shows regex compilation in hot path - -**Detection**: -```bash -# Flamegraph shows Regex::new() calls during execution -cargo flamegraph -- target/release/rtk git log -10 -# Look for "regex::Regex::new" in non-lazy_static sections -``` - -**Fix**: -```rust -// ❌ WRONG: Recompiled on every call -fn filter_line(line: &str) -> Option<&str> { - let re = Regex::new(r"pattern").unwrap(); // RECOMPILED! - re.find(line).map(|m| m.as_str()) -} - -// ✅ RIGHT: Compiled once with lazy_static -use lazy_static::lazy_static; - -lazy_static! { - static ref LINE_PATTERN: Regex = Regex::new(r"pattern").unwrap(); -} - -fn filter_line(line: &str) -> Option<&str> { - LINE_PATTERN.find(line).map(|m| m.as_str()) -} -``` - -### Issue 2: Excessive Allocations - -**Symptom**: Memory usage >5MB, many small allocations in flamegraph - -**Detection**: -```bash -# DHAT heap profiling -cargo +nightly build --release -valgrind --tool=dhat target/release/rtk git status -``` - -**Fix**: -```rust -// ❌ WRONG: Allocates Vec for every line -fn filter_lines(input: &str) -> String { - input.lines() - .map(|line| line.to_string()) // Allocates String - .collect::>() - .join("\n") -} - -// ✅ RIGHT: Borrow slices, single allocation -fn filter_lines(input: &str) -> String { - input.lines() - .collect::>() // Vec of &str (no String allocation) - .join("\n") -} -``` - -### Issue 3: Startup I/O - -**Symptom**: Startup time varies wildly (5ms to 50ms), flamegraph shows file reads - -**Detection**: -```bash -# strace on Linux -strace -c target/release/rtk git status 2>&1 | grep -E "open|read" - -# dtrace on macOS (requires SIP disabled) -sudo dtrace -n 'syscall::open*:entry { @[execname] = count(); }' & -target/release/rtk git status -sudo pkill dtrace -``` - -**Fix**: -```rust -// ❌ WRONG: File I/O on startup -fn main() { - let config = load_config().unwrap(); // Reads ~/.config/rtk/config.toml - // ... -} - -// ✅ RIGHT: Lazy config loading (only if needed) -fn main() { - // No I/O on startup - // Config loaded on-demand when first accessed -} -``` - -### Issue 4: Dependency Bloat - -**Symptom**: Binary size >5MB, many unused dependencies in `Cargo.toml` - -**Detection**: -```bash -# Analyze dependency tree -cargo tree - -# Find heavy dependencies -cargo install cargo-bloat -cargo bloat --release --crates - -# Example output: -# File .text Size Crate -# 0.5% 2.1% 42.3KB regex -# 0.4% 1.8% 36.1KB clap -# ... -``` - -**Fix**: -```toml -# ❌ WRONG: Full feature set (bloat) -[dependencies] -clap = { version = "4", features = ["derive", "color", "suggestions"] } - -# ✅ RIGHT: Minimal features -[dependencies] -clap = { version = "4", features = ["derive"], default-features = false } -``` - -## Optimization Techniques - -### Technique 1: Lazy Static Initialization - -**Use case**: Regex patterns, static configuration, one-time allocations - -**Implementation**: -```rust -use lazy_static::lazy_static; -use regex::Regex; - -lazy_static! { - static ref COMMIT_HASH: Regex = Regex::new(r"[0-9a-f]{7,40}").unwrap(); - static ref AUTHOR_LINE: Regex = Regex::new(r"^Author: (.+)$").unwrap(); - static ref DATE_LINE: Regex = Regex::new(r"^Date: (.+)$").unwrap(); -} - -// All regex compiled once at startup, reused forever -``` - -**Impact**: ~5-10ms saved per regex pattern (if compiled at runtime) - -### Technique 2: Zero-Copy String Processing - -**Use case**: Filter output without allocating intermediate Strings - -**Implementation**: -```rust -// ❌ WRONG: Allocates String for every line -fn filter(input: &str) -> String { - input.lines() - .filter(|line| !line.is_empty()) - .map(|line| line.to_string()) // Allocates! - .collect::>() - .join("\n") -} - -// ✅ RIGHT: Borrow slices, single final allocation -fn filter(input: &str) -> String { - input.lines() - .filter(|line| !line.is_empty()) - .collect::>() // Vec<&str> (no String alloc) - .join("\n") // Single allocation for joined result -} -``` - -**Impact**: ~1-2MB memory saved, ~1-2ms startup saved - -### Technique 3: Minimal Dependencies - -**Use case**: Reduce binary size and compile time - -**Implementation**: -```toml -# Only include features you actually use -[dependencies] -clap = { version = "4", features = ["derive"], default-features = false } -serde = { version = "1", features = ["derive"], default-features = false } - -# Avoid heavy dependencies -# ❌ Avoid: tokio (adds 5-10ms startup overhead) -# ❌ Avoid: full regex (use regex-lite if possible) -# ✅ Use: anyhow (lightweight error handling) -# ✅ Use: lazy_static (zero runtime overhead) -``` - -**Impact**: ~1-2MB binary size reduction, ~2-5ms startup saved - -## Performance Testing Checklist - -Before committing filter changes: - -### Startup Time -- [ ] Benchmark with `hyperfine 'rtk ' --warmup 3` -- [ ] Verify <10ms mean time -- [ ] Check variance (σ) is small (<1ms) -- [ ] Compare against baseline (regression <2ms) - -### Memory Usage -- [ ] Profile with `/usr/bin/time -l rtk ` -- [ ] Verify <5MB resident set size -- [ ] Compare against baseline (regression <1MB) - -### Token Savings -- [ ] Run `cargo test test_token_savings` -- [ ] Verify all filters achieve ≥60% savings -- [ ] Check real fixtures used (not synthetic) - -### Binary Size -- [ ] Check `ls -lh target/release/rtk` -- [ ] Verify <5MB stripped binary -- [ ] Run `cargo bloat --release --crates` if >5MB - -## Continuous Performance Monitoring - -### Pre-Commit Hook - -Add to `.claude/hooks/bash/pre-commit-performance.sh`: - -```bash -#!/bin/bash -# Performance regression check before commit - -echo "🚀 Running performance checks..." - -# Benchmark startup time -CURRENT_TIME=$(hyperfine 'rtk git status' --warmup 3 --export-json /tmp/perf.json 2>&1 | grep "Time (mean" | awk '{print $4}') - -# Extract numeric value (remove "ms") -CURRENT_MS=$(echo $CURRENT_TIME | sed 's/ms//') - -# Check if > 10ms -if (( $(echo "$CURRENT_MS > 10" | bc -l) )); then - echo "❌ Startup time regression: ${CURRENT_MS}ms (target: <10ms)" - exit 1 -fi - -# Check binary size -BINARY_SIZE=$(ls -l target/release/rtk | awk '{print $5}') -MAX_SIZE=$((5 * 1024 * 1024)) # 5MB - -if [ $BINARY_SIZE -gt $MAX_SIZE ]; then - echo "❌ Binary size regression: $(($BINARY_SIZE / 1024 / 1024))MB (target: <5MB)" - exit 1 -fi - -echo "✅ Performance checks passed" -``` - -### CI/CD Integration - -Add to `.github/workflows/ci.yml`: - -```yaml -- name: Performance Regression Check - run: | - cargo build --release - cargo install hyperfine - - # Benchmark startup time - hyperfine 'target/release/rtk git status' --warmup 3 --max-runs 10 - - # Check binary size - BINARY_SIZE=$(ls -l target/release/rtk | awk '{print $5}') - MAX_SIZE=$((5 * 1024 * 1024)) - if [ $BINARY_SIZE -gt $MAX_SIZE ]; then - echo "Binary too large: $(($BINARY_SIZE / 1024 / 1024))MB" - exit 1 - fi -``` - -## Performance Optimization Priorities - -**Priority order** (highest to lowest impact): - -1. **🔴 Lazy static regex** (5-10ms per pattern if compiled at runtime) -2. **🔴 Remove startup I/O** (10-50ms for config file reads) -3. **🟡 Zero-copy processing** (1-2MB memory, 1-2ms startup) -4. **🟡 Minimal dependencies** (1-2MB binary, 2-5ms startup) -5. **🟢 Algorithm optimization** (varies, measure first) - -**When in doubt**: Profile first with `flamegraph`, then optimize the hottest path. - -## Tools Reference - -| Tool | Purpose | Command | -|------|---------|---------| -| **hyperfine** | Benchmark startup time | `hyperfine 'rtk ' --warmup 3` | -| **time** | Memory usage (macOS) | `/usr/bin/time -l rtk ` | -| **time** | Memory usage (Linux) | `/usr/bin/time -v rtk ` | -| **flamegraph** | CPU profiling | `cargo flamegraph -- rtk ` | -| **cargo bloat** | Binary size analysis | `cargo bloat --release --crates` | -| **cargo tree** | Dependency tree | `cargo tree` | -| **DHAT** | Heap profiling | `cargo +nightly build && valgrind --tool=dhat` | -| **strace** | System call tracing (Linux) | `strace -c target/release/rtk ` | -| **dtrace** | System call tracing (macOS) | `sudo dtrace -n 'syscall::open*:entry'` | - -**Install tools**: -```bash -# macOS -brew install hyperfine - -# Linux / cross-platform via cargo -cargo install hyperfine -cargo install flamegraph -cargo install cargo-bloat -``` diff --git a/.claude/skills/performance/SKILL.md b/.claude/skills/performance/SKILL.md index e4f45c0c9..30c90b0eb 100644 --- a/.claude/skills/performance/SKILL.md +++ b/.claude/skills/performance/SKILL.md @@ -1,209 +1,435 @@ --- -name: performance -description: RTK CLI performance analysis and optimization. Startup time (<10ms), binary size (<5MB), regex compilation, memory usage. Use when adding dependencies, changing initialization, or suspecting regressions. -triggers: - - "startup time" - - "performance regression" - - "too slow" - - "benchmark" - - "binary size" - - "memory usage" +description: CLI performance optimization - startup time, memory usage, token savings benchmarking --- -# RTK Performance Analysis +# Performance Optimization Skill -## Hard Targets (Non-Negotiable) +Systematic performance analysis and optimization for RTK CLI tool, focusing on **startup time (<10ms)**, **memory usage (<5MB)**, and **token savings (60-90%)**. -| Metric | Target | Blocker | -|--------|--------|---------| -| Startup time | <10ms | Release blocker | -| Binary size (stripped) | <5MB | Release blocker | -| Memory (resident) | <5MB | Release blocker | -| Token savings per filter | ≥60% | Release blocker | +## When to Use -## Benchmark Startup Time +- **Automatically triggered**: After filter changes, regex modifications, or dependency additions +- **Manual invocation**: When performance degradation suspected or before release +- **Proactive**: After any code change that could impact startup time or memory + +## RTK Performance Targets + +| Metric | Target | Verification Method | Failure Threshold | +|--------|--------|---------------------|-------------------| +| **Startup time** | <10ms | `hyperfine 'rtk '` | >15ms = blocker | +| **Memory usage** | <5MB resident | `/usr/bin/time -l rtk ` (macOS) | >7MB = blocker | +| **Token savings** | 60-90% | Tests with `count_tokens()` | <60% = blocker | +| **Binary size** | <5MB stripped | `ls -lh target/release/rtk` | >8MB = investigate | + +## Performance Analysis Workflow + +### 1. Establish Baseline + +Before making any changes, capture current performance: ```bash -# Install hyperfine (once) -brew install hyperfine +# Startup time baseline +hyperfine 'rtk git status' --warmup 3 --export-json /tmp/baseline_startup.json + +# Memory usage baseline (macOS) +/usr/bin/time -l rtk git status 2>&1 | grep "maximum resident set size" > /tmp/baseline_memory.txt + +# Memory usage baseline (Linux) +/usr/bin/time -v rtk git status 2>&1 | grep "Maximum resident set size" > /tmp/baseline_memory.txt + +# Binary size baseline +ls -lh target/release/rtk | tee /tmp/baseline_binary_size.txt +``` + +### 2. Make Changes + +Implement optimization or feature changes. -# Baseline (before changes) -hyperfine 'rtk git status' --warmup 3 --export-json /tmp/before.json +### 3. Rebuild and Measure -# After changes — rebuild first +```bash +# Rebuild with optimizations cargo build --release -# Compare against installed -hyperfine 'target/release/rtk git status' 'rtk git status' --warmup 3 +# Measure startup time +hyperfine 'target/release/rtk git status' --warmup 3 --export-json /tmp/after_startup.json + +# Measure memory usage +/usr/bin/time -l target/release/rtk git status 2>&1 | grep "maximum resident set size" > /tmp/after_memory.txt -# Target: <10ms mean time +# Check binary size +ls -lh target/release/rtk | tee /tmp/after_binary_size.txt ``` -## Check Binary Size +### 4. Compare Results ```bash -# Release build with strip=true (already in Cargo.toml) -cargo build --release -ls -lh target/release/rtk -# Should be <5MB +# Startup time comparison +hyperfine 'rtk git status' 'target/release/rtk git status' --warmup 3 + +# Example output: +# Benchmark 1: rtk git status +# Time (mean ± σ): 6.2 ms ± 0.3 ms [User: 4.1 ms, System: 1.8 ms] +# Benchmark 2: target/release/rtk git status +# Time (mean ± σ): 7.8 ms ± 0.4 ms [User: 5.2 ms, System: 2.1 ms] +# +# Summary +# 'rtk git status' ran 1.26 times faster than 'target/release/rtk git status' + +# Memory comparison +diff /tmp/baseline_memory.txt /tmp/after_memory.txt + +# Binary size comparison +diff /tmp/baseline_binary_size.txt /tmp/after_binary_size.txt +``` -# If too large — check what's contributing -cargo bloat --release --crates -cargo bloat --release -n 20 -# Install: cargo install cargo-bloat +### 5. Identify Regressions + +**Startup time regression** (>15% increase or >2ms absolute): +```bash +# Profile with flamegraph +cargo install flamegraph +cargo flamegraph -- target/release/rtk git status + +# Open flamegraph.svg +open flamegraph.svg +# Look for: +# - Regex compilation (should be in lazy_static init) +# - Excessive allocations +# - File I/O on startup (should be zero) ``` -## Memory Usage +**Memory regression** (>20% increase or >1MB absolute): +```bash +# Profile allocations (requires nightly) +cargo +nightly build --release -Z build-std +RUSTFLAGS="-C link-arg=-fuse-ld=lld" cargo +nightly build --release + +# Use DHAT for heap profiling +cargo install dhat +# Add to main.rs: +# #[global_allocator] +# static ALLOC: dhat::Alloc = dhat::Alloc; +``` +**Token savings regression** (<60% savings): ```bash -# macOS -/usr/bin/time -l target/release/rtk git status 2>&1 | grep "maximum resident" -# Target: <5,000,000 bytes (5MB) +# Run token accuracy tests +cargo test test_token_savings + +# Example failure output: +# Git log filter: expected ≥60% savings, got 52.3% -# Linux -/usr/bin/time -v target/release/rtk git status 2>&1 | grep "Maximum resident" -# Target: <5,000 kbytes +# Fix: Improve filter condensation logic ``` -## Regex Compilation Audit +## Common Performance Issues -Regex compilation on every function call is a common perf killer: +### Issue 1: Regex Recompilation -```bash -# Find all Regex::new calls -grep -n "Regex::new" src/*.rs +**Symptom**: Startup time >20ms, flamegraph shows regex compilation in hot path -# Verify ALL are inside lazy_static! blocks -# Any Regex::new outside lazy_static! = performance bug +**Detection**: +```bash +# Flamegraph shows Regex::new() calls during execution +cargo flamegraph -- target/release/rtk git log -10 +# Look for "regex::Regex::new" in non-lazy_static sections ``` +**Fix**: ```rust -// ❌ Recompiles on every filter_line() call -fn filter_line(line: &str) -> bool { - let re = Regex::new(r"^error").unwrap(); // BAD - re.is_match(line) +// ❌ WRONG: Recompiled on every call +fn filter_line(line: &str) -> Option<&str> { + let re = Regex::new(r"pattern").unwrap(); // RECOMPILED! + re.find(line).map(|m| m.as_str()) } -// ✅ Compiled once at first use +// ✅ RIGHT: Compiled once with lazy_static +use lazy_static::lazy_static; + lazy_static! { - static ref ERROR_RE: Regex = Regex::new(r"^error").unwrap(); + static ref LINE_PATTERN: Regex = Regex::new(r"pattern").unwrap(); } -fn filter_line(line: &str) -> bool { - ERROR_RE.is_match(line) // GOOD + +fn filter_line(line: &str) -> Option<&str> { + LINE_PATTERN.find(line).map(|m| m.as_str()) } ``` -## Dependency Impact Assessment +### Issue 2: Excessive Allocations -Before adding any new crate: +**Symptom**: Memory usage >5MB, many small allocations in flamegraph +**Detection**: ```bash -# Check startup impact (measure before adding) -hyperfine 'rtk git status' --warmup 3 - -# Add dependency to Cargo.toml -# Rebuild -cargo build --release +# DHAT heap profiling +cargo +nightly build --release +valgrind --tool=dhat target/release/rtk git status +``` -# Measure after -hyperfine 'target/release/rtk git status' --warmup 3 +**Fix**: +```rust +// ❌ WRONG: Allocates Vec for every line +fn filter_lines(input: &str) -> String { + input.lines() + .map(|line| line.to_string()) // Allocates String + .collect::>() + .join("\n") +} -# If startup increased >1ms — investigate -# If startup increased >3ms — reject the dependency +// ✅ RIGHT: Borrow slices, single allocation +fn filter_lines(input: &str) -> String { + input.lines() + .collect::>() // Vec of &str (no String allocation) + .join("\n") +} ``` -### Forbidden dependencies +### Issue 3: Startup I/O -| Crate | Reason | Alternative | -|-------|--------|-------------| -| `tokio` | +5-10ms startup | Blocking `std::process::Command` | -| `async-std` | +5-10ms startup | Blocking I/O | -| `rayon` | Thread pool init overhead | Sequential iteration | -| `reqwest` | Pulls tokio | `ureq` (blocking) if HTTP needed | - -### Dependency weight check +**Symptom**: Startup time varies wildly (5ms to 50ms), flamegraph shows file reads +**Detection**: ```bash -# After cargo build --release -cargo build --release --timings -# Open target/cargo-timings/cargo-timing.html -# Look for crates with long compile times (correlates with complexity) +# strace on Linux +strace -c target/release/rtk git status 2>&1 | grep -E "open|read" + +# dtrace on macOS (requires SIP disabled) +sudo dtrace -n 'syscall::open*:entry { @[execname] = count(); }' & +target/release/rtk git status +sudo pkill dtrace ``` -## Allocation Profiling +**Fix**: +```rust +// ❌ WRONG: File I/O on startup +fn main() { + let config = load_config().unwrap(); // Reads ~/.config/rtk/config.toml + // ... +} + +// ✅ RIGHT: Lazy config loading (only if needed) +fn main() { + // No I/O on startup + // Config loaded on-demand when first accessed +} +``` + +### Issue 4: Dependency Bloat + +**Symptom**: Binary size >5MB, many unused dependencies in `Cargo.toml` +**Detection**: ```bash -# macOS — use Instruments -instruments -t Allocations target/release/rtk git log -10 +# Analyze dependency tree +cargo tree -# Or use cargo-instruments -cargo install cargo-instruments -cargo instruments --release -t Allocations -- git log -10 +# Find heavy dependencies +cargo install cargo-bloat +cargo bloat --release --crates + +# Example output: +# File .text Size Crate +# 0.5% 2.1% 42.3KB regex +# 0.4% 1.8% 36.1KB clap +# ... ``` -Common RTK allocation hotspots: +**Fix**: +```toml +# ❌ WRONG: Full feature set (bloat) +[dependencies] +clap = { version = "4", features = ["derive", "color", "suggestions"] } -```rust -// ❌ Allocates new String on every line -let lines: Vec = input.lines().map(|l| l.to_string()).collect(); +# ✅ RIGHT: Minimal features +[dependencies] +clap = { version = "4", features = ["derive"], default-features = false } +``` + +## Optimization Techniques -// ✅ Borrow slices -let lines: Vec<&str> = input.lines().collect(); +### Technique 1: Lazy Static Initialization -// ❌ Clone large output unnecessarily -let raw_copy = output.stdout.clone(); +**Use case**: Regex patterns, static configuration, one-time allocations -// ✅ Use reference until you actually need to own -let display = &output.stdout; +**Implementation**: +```rust +use lazy_static::lazy_static; +use regex::Regex; + +lazy_static! { + static ref COMMIT_HASH: Regex = Regex::new(r"[0-9a-f]{7,40}").unwrap(); + static ref AUTHOR_LINE: Regex = Regex::new(r"^Author: (.+)$").unwrap(); + static ref DATE_LINE: Regex = Regex::new(r"^Date: (.+)$").unwrap(); +} + +// All regex compiled once at startup, reused forever ``` -## Token Savings Measurement +**Impact**: ~5-10ms saved per regex pattern (if compiled at runtime) + +### Technique 2: Zero-Copy String Processing +**Use case**: Filter output without allocating intermediate Strings + +**Implementation**: ```rust -// In tests — always verify claims -fn count_tokens(text: &str) -> usize { - text.split_whitespace().count() +// ❌ WRONG: Allocates String for every line +fn filter(input: &str) -> String { + input.lines() + .filter(|line| !line.is_empty()) + .map(|line| line.to_string()) // Allocates! + .collect::>() + .join("\n") +} + +// ✅ RIGHT: Borrow slices, single final allocation +fn filter(input: &str) -> String { + input.lines() + .filter(|line| !line.is_empty()) + .collect::>() // Vec<&str> (no String alloc) + .join("\n") // Single allocation for joined result } +``` -#[test] -fn test_savings_claim() { - let input = include_str!("../tests/fixtures/mycmd_raw.txt"); - let output = filter_output(input).unwrap(); +**Impact**: ~1-2MB memory saved, ~1-2ms startup saved - let input_tokens = count_tokens(input); - let output_tokens = count_tokens(&output); - let savings = 100.0 * (1.0 - output_tokens as f64 / input_tokens as f64); +### Technique 3: Minimal Dependencies - assert!( - savings >= 60.0, - "Expected ≥60% savings, got {:.1}% ({} → {} tokens)", - savings, input_tokens, output_tokens - ); -} +**Use case**: Reduce binary size and compile time + +**Implementation**: +```toml +# Only include features you actually use +[dependencies] +clap = { version = "4", features = ["derive"], default-features = false } +serde = { version = "1", features = ["derive"], default-features = false } + +# Avoid heavy dependencies +# ❌ Avoid: tokio (adds 5-10ms startup overhead) +# ❌ Avoid: full regex (use regex-lite if possible) +# ✅ Use: anyhow (lightweight error handling) +# ✅ Use: lazy_static (zero runtime overhead) ``` -## Before/After Regression Check +**Impact**: ~1-2MB binary size reduction, ~2-5ms startup saved + +## Performance Testing Checklist + +Before committing filter changes: + +### Startup Time +- [ ] Benchmark with `hyperfine 'rtk ' --warmup 3` +- [ ] Verify <10ms mean time +- [ ] Check variance (σ) is small (<1ms) +- [ ] Compare against baseline (regression <2ms) + +### Memory Usage +- [ ] Profile with `/usr/bin/time -l rtk ` +- [ ] Verify <5MB resident set size +- [ ] Compare against baseline (regression <1MB) + +### Token Savings +- [ ] Run `cargo test test_token_savings` +- [ ] Verify all filters achieve ≥60% savings +- [ ] Check real fixtures used (not synthetic) -Template for any performance-sensitive change: +### Binary Size +- [ ] Check `ls -lh target/release/rtk` +- [ ] Verify <5MB stripped binary +- [ ] Run `cargo bloat --release --crates` if >5MB + +## Continuous Performance Monitoring + +### Pre-Commit Hook + +Add to `.claude/hooks/bash/pre-commit-performance.sh`: ```bash -# 1. Baseline -cargo build --release -hyperfine 'target/release/rtk git status' --warmup 5 --export-json /tmp/before.json -/usr/bin/time -l target/release/rtk git status 2>&1 | grep "maximum resident" -ls -lh target/release/rtk +#!/bin/bash +# Performance regression check before commit -# 2. Make changes -# ... edit code ... +echo "🚀 Running performance checks..." -# 3. Rebuild and compare -cargo build --release -hyperfine 'target/release/rtk git status' --warmup 5 --export-json /tmp/after.json -/usr/bin/time -l target/release/rtk git status 2>&1 | grep "maximum resident" -ls -lh target/release/rtk - -# 4. Compare -# Startup: jq '.results[0].mean' /tmp/before.json /tmp/after.json -# If after > before + 1ms: investigate -# If after > 10ms: regression, do not merge +# Benchmark startup time +CURRENT_TIME=$(hyperfine 'rtk git status' --warmup 3 --export-json /tmp/perf.json 2>&1 | grep "Time (mean" | awk '{print $4}') + +# Extract numeric value (remove "ms") +CURRENT_MS=$(echo $CURRENT_TIME | sed 's/ms//') + +# Check if > 10ms +if (( $(echo "$CURRENT_MS > 10" | bc -l) )); then + echo "❌ Startup time regression: ${CURRENT_MS}ms (target: <10ms)" + exit 1 +fi + +# Check binary size +BINARY_SIZE=$(ls -l target/release/rtk | awk '{print $5}') +MAX_SIZE=$((5 * 1024 * 1024)) # 5MB + +if [ $BINARY_SIZE -gt $MAX_SIZE ]; then + echo "❌ Binary size regression: $(($BINARY_SIZE / 1024 / 1024))MB (target: <5MB)" + exit 1 +fi + +echo "✅ Performance checks passed" +``` + +### CI/CD Integration + +Add to `.github/workflows/ci.yml`: + +```yaml +- name: Performance Regression Check + run: | + cargo build --release + cargo install hyperfine + + # Benchmark startup time + hyperfine 'target/release/rtk git status' --warmup 3 --max-runs 10 + + # Check binary size + BINARY_SIZE=$(ls -l target/release/rtk | awk '{print $5}') + MAX_SIZE=$((5 * 1024 * 1024)) + if [ $BINARY_SIZE -gt $MAX_SIZE ]; then + echo "Binary too large: $(($BINARY_SIZE / 1024 / 1024))MB" + exit 1 + fi +``` + +## Performance Optimization Priorities + +**Priority order** (highest to lowest impact): + +1. **🔴 Lazy static regex** (5-10ms per pattern if compiled at runtime) +2. **🔴 Remove startup I/O** (10-50ms for config file reads) +3. **🟡 Zero-copy processing** (1-2MB memory, 1-2ms startup) +4. **🟡 Minimal dependencies** (1-2MB binary, 2-5ms startup) +5. **🟢 Algorithm optimization** (varies, measure first) + +**When in doubt**: Profile first with `flamegraph`, then optimize the hottest path. + +## Tools Reference + +| Tool | Purpose | Command | +|------|---------|---------| +| **hyperfine** | Benchmark startup time | `hyperfine 'rtk ' --warmup 3` | +| **time** | Memory usage (macOS) | `/usr/bin/time -l rtk ` | +| **time** | Memory usage (Linux) | `/usr/bin/time -v rtk ` | +| **flamegraph** | CPU profiling | `cargo flamegraph -- rtk ` | +| **cargo bloat** | Binary size analysis | `cargo bloat --release --crates` | +| **cargo tree** | Dependency tree | `cargo tree` | +| **DHAT** | Heap profiling | `cargo +nightly build && valgrind --tool=dhat` | +| **strace** | System call tracing (Linux) | `strace -c target/release/rtk ` | +| **dtrace** | System call tracing (macOS) | `sudo dtrace -n 'syscall::open*:entry'` | + +**Install tools**: +```bash +# macOS +brew install hyperfine + +# Linux / cross-platform via cargo +cargo install hyperfine +cargo install flamegraph +cargo install cargo-bloat ``` diff --git a/.claude/skills/pr-review/SKILL.md b/.claude/skills/pr-review/SKILL.md new file mode 100644 index 000000000..936419554 --- /dev/null +++ b/.claude/skills/pr-review/SKILL.md @@ -0,0 +1,227 @@ +--- +description: > + Batch review des PRs RTK par ordre de complexité croissante (XS → S → M → L). + Pour chaque PR : vérifie l'état (conflits, CLA, reviews), lit le diff complet, + analyse le code en contexte, présente un résumé avec lien + taille + recommandation. + Attend validation explicite avant tout merge. Poste des commentaires boldguy-adapt + sur les PRs bloquées (conflit, CLA, CHANGES_REQUESTED). + Args: "triage" pour lancer un triage complet avant la review. "from:" pour + reprendre à partir d'un numéro de PR spécifique. +allowed-tools: + - Bash + - Read + - Grep + - Glob + - Write + - AskUserQuestion +--- + +# /pr-review + +Batch review des PRs RTK — du plus simple au plus complexe, une par une, avec validation utilisateur avant chaque merge. + +--- + +## Quand utiliser + +- Après un `/rtk-triage` pour agir sur les résultats +- Régulièrement pour dégraisser le backlog +- Avant une release pour vider la file quick wins + +--- + +## Workflow + +### Phase 0 — Préconditions + +```bash +git rev-parse --is-inside-work-tree +gh auth status +date +%Y-%m-%d +``` + +Si l'argument `triage` est passé, exécuter `/rtk-triage` d'abord et utiliser sa liste de quick wins comme séquence. Sinon, construire la liste soi-même. + +--- + +### Phase 1 — Construire la liste de PRs (si pas de triage) + +```bash +gh pr list --state open --limit 200 \ + --json number,title,author,additions,deletions,changedFiles,mergeable,mergeStateStatus,isDraft,statusCheckRollup,reviewDecision,body \ + | jq 'sort_by(.additions + .deletions)' +``` + +**Classement par taille** : + +| Taille | Critère | Traitement | +|--------|---------|------------| +| XS | < 30 lignes, 1 fichier | En premier | +| S | 30-100 lignes, 1-3 fichiers | Ensuite | +| M | 100-200 lignes, logique non triviale | Après | +| L | > 200 lignes | Dernier ou skip | +| XL | > 500 lignes | Skip (session dédiée) | + +**Filtrer d'emblée** : +- Exclure les PRs draft +- Exclure les PRs de nous (les nôtres ont une review flow différente) +- Si `from:` passé en argument : commencer à ce numéro + +--- + +### Phase 2 — Pour chaque PR (une par une, dans l'ordre) + +#### Étape A — Vérification état (AVANT de lire le diff) + +```bash +# 1. Etat mergeable + CLA +gh pr view --json mergeable,mergeStateStatus,statusCheckRollup,reviewDecision + +# 2. Reviews existantes (CHANGES_REQUESTED ?) +gh api repos/algolia/rtk/pulls//reviews \ + --jq '.[] | {author: .user.login, state: .state, body: .body}' + +# 3. Commentaires inline (si CHANGES_REQUESTED) +gh api repos/algolia/rtk/pulls//comments \ + --jq '.[] | {author: .user.login, body: .body, path: .path, line: .line}' +``` + +**Décision rapide selon état** : + +| État | Action | +|------|--------| +| MERGEABLE + CLA ok + pas de CHANGES_REQUESTED | → lire le diff | +| CONFLICTING | → préparer commentaire rebase, skip diff | +| CLA non signé | → préparer commentaire CLA, skip diff | +| CHANGES_REQUESTED par un maintainer | → skip (ne pas override), noter | +| Draft | → skip silencieusement | + +#### Étape B — Lire le diff complet + +```bash +gh pr diff +``` + +Si le diff touche une logique complexe (filter functions, regex, routing) → lire le fichier source en contexte avec `Read` pour comprendre l'impact réel. + +#### Étape C — Présenter à l'utilisateur + +Format de présentation **obligatoire** pour chaque PR : + +``` +**PR #** — https://github.com/algolia/rtk/pull/ + +**Author**: | **Size**: (+ -, fichiers) | **CLA**: | **Mergeable**: + +**Ce que ça fait** — [description en 2-4 phrases : le problème résolu, les fichiers touchés, la logique modifiée, les tests ajoutés] + +**Qualité du diff** : [analyse honnête : propre/à vérifier/problème détecté] + +Merge # ? +``` + +**Règles de présentation** : +- Toujours inclure le lien GitHub cliquable +- Toujours mentionner si des tests couvrent le changement +- Si une fonction complexe est touchée, expliquer l'impact +- Ne pas embellir — si le diff est moyen, le dire +- Langue : français pour l'analyse (comme ici) + +#### Étape D — Attendre la validation + +**NE JAMAIS MERGER SANS RÉPONSE EXPLICITE.** Les réponses attendues : + +| Réponse | Action | +|---------|--------| +| "ok" / "go" / "merge" | Merger avec `gh pr merge --merge` | +| "skip" / "next" | Passer à la PR suivante sans merger | +| "comment" | Poster un commentaire (demander le texte si pas fourni) | +| "close" | Fermer la PR | +| Retour avec instructions | Appliquer puis redemander confirmation | + +#### Étape E — Merger (si validé) + +```bash +gh pr merge --merge --squash +``` + +Confirmer immédiatement : `Merged #. ✓` + +Puis **vérifier que la PR suivante n'est pas passée en CONFLICTING** à cause du merge (surtout si les deux touchent `rules.rs`, `registry.rs`, `main.rs`, ou `CHANGELOG.md`). + +--- + +### Phase 3 — PRs bloquées : commentaire boldguy-adapt + +Pour les PRs avec conflit, CLA manquant, ou besoin de rebase, poster un commentaire en anglais, ton boldguy-adapt. + +**Règles du commentaire** : +- **Anglais uniquement** (GitHub) +- Remercier la contribution en ouverture (sincèrement, pas de manière générique) +- Dire clairement ce qui bloque (1-2 points max) +- Donner les étapes exactes pour débloquer +- Pas d'em dash (`—`), pas de staccato, longueurs de phrases variées +- Ne pas sonner comme un bot + +**Template conflit + CLA** : +``` +Hey @, thanks for the contribution! [mention spécifique de ce que la PR apporte] + +Two things before we can merge: + +1. The branch needs a rebase on `develop` — there's a conflict on [fichier]. A `git rebase origin/develop` should do it. + +2. The CLA hasn't been signed yet. The CLAassistant bot left instructions in the PR — just follow the link, takes about a minute. + +Once both are sorted, this will move quickly. +``` + +**Template conflit seul** : +``` +Hey @, good fix on [description spécifique]. One thing to address before merge: the branch has a conflict on [fichier] after recent changes to develop. A `git rebase origin/develop` should resolve it cleanly. +``` + +**Template CLA seul** : +``` +Hey @, thanks for [description spécifique]. The only thing blocking merge is the CLA signature — the CLAassistant bot left the link in the PR. Once that's done, we're good to go. +``` + +--- + +### Phase 4 — Récap de session + +Après avoir traité toutes les PRs (ou à la demande) : + +``` +## Session recap — YYYY-MM-DD + +| PR | Titre | Action | Raison | +|----|-------|--------|--------| +| #N | titre | Mergé ✓ | — | +| #N | titre | Skip | CHANGES_REQUESTED (KuSh) | +| #N | titre | Commenté | Conflit + CLA | +| #N | titre | Fermé | Doublon avec #M | + +Mergées : N | Skippées : N | Commentées : N +``` + +--- + +## Règles + +- **Une PR à la fois** — ne jamais présenter plusieurs PRs en attente de validation +- **Jamais merger sans "ok" explicite** — "ça a l'air bien" n'est pas un ok +- **Ne pas overrider un CHANGES_REQUESTED** d'un maintainer sans instructions explicites de l'utilisateur +- **Vérifier les conflits post-merge** sur la PR suivante si les deux touchent les mêmes fichiers +- **Langue** : analyse en français, commentaires GitHub en anglais +- **Ton boldguy** : factuel, direct, bienveillant, pas de marqueurs AI (em dash, staccato, punchline finale parfaite) + +--- + +## Fichiers fréquemment en conflit (surveiller) + +- `CHANGELOG.md` — toutes les PRs y touchent +- `src/discover/rules.rs` — ajouts fréquents de règles +- `src/discover/registry.rs` — tests de classify/rewrite +- `src/main.rs` — routing des commandes +- `src/hooks/rewrite_cmd.rs` — rewrites hooks diff --git a/.claude/skills/pr-triage/SKILL.md b/.claude/skills/pr-triage/SKILL.md index f55a5dd19..830ed22e0 100644 --- a/.claude/skills/pr-triage/SKILL.md +++ b/.claude/skills/pr-triage/SKILL.md @@ -1,7 +1,15 @@ --- +name: pr-triage description: > PR triage: audit open PRs, deep review selected ones, draft and post review comments. Args: "all" to review all, PR numbers to focus (e.g. "42 57"), "en"/"fr" for language, no arg = audit only in French. +allowed-tools: + - Bash + - Read + - Grep + - Glob +effort: medium +tags: [triage, pr, github, review, code-review, rtk] --- # PR Triage @@ -145,7 +153,16 @@ _Externes — Problématiques_ : un des critères suivants : Après affichage du tableau de triage, copier dans le presse-papier : ```bash -pbcopy <<'EOF' +# Cross-platform clipboard +clip() { + if command -v pbcopy &>/dev/null; then pbcopy + elif command -v xclip &>/dev/null; then xclip -selection clipboard + elif command -v wl-copy &>/dev/null; then wl-copy + else cat + fi +} + +clip <<'EOF' {tableau de triage complet} EOF ``` diff --git a/.claude/skills/pr-triage/templates/review-comment.md b/.claude/skills/pr-triage/templates/review-comment.md index fbf582de4..5f7df791a 100644 --- a/.claude/skills/pr-triage/templates/review-comment.md +++ b/.claude/skills/pr-triage/templates/review-comment.md @@ -42,7 +42,7 @@ Use this template to generate GitHub PR review comments. Fill in each section ba {- Description of what's done right.} --- -*Automated review via [rtk](https://github.com/rtk-ai/rtk) `/pr-triage`* +*Automated review via [rtk](https://github.com/algolia/rtk) `/pr-triage`* ``` --- diff --git a/.claude/skills/repo-recap.md b/.claude/skills/repo-recap/SKILL.md similarity index 96% rename from .claude/skills/repo-recap.md rename to .claude/skills/repo-recap/SKILL.md index 7dfa186fd..24df1a8cf 100644 --- a/.claude/skills/repo-recap.md +++ b/.claude/skills/repo-recap/SKILL.md @@ -1,5 +1,6 @@ --- description: Generate a comprehensive repo recap (PRs, issues, releases) for sharing with team. Pass "en" or "fr" as argument for language (default fr). +allowed-tools: Bash Read Grep --- # Repo Recap @@ -139,7 +140,16 @@ Structure the full recap as Markdown with: After displaying the recap, automatically copy it to clipboard: ```bash -cat << 'EOF' | pbcopy +# Cross-platform clipboard +clip() { + if command -v pbcopy &>/dev/null; then pbcopy + elif command -v xclip &>/dev/null; then xclip -selection clipboard + elif command -v wl-copy &>/dev/null; then wl-copy + else cat + fi +} + +cat << 'EOF' | clip {formatted recap content} EOF ``` diff --git a/.claude/skills/rtk-tdd/SKILL.md b/.claude/skills/rtk-tdd/SKILL.md index 79caf4972..e13ec58a5 100644 --- a/.claude/skills/rtk-tdd/SKILL.md +++ b/.claude/skills/rtk-tdd/SKILL.md @@ -5,6 +5,13 @@ description: > implementation, testing, refactoring, and bug fixing tasks. Provides Rust-idiomatic testing patterns with anyhow/thiserror, cfg(test), and Arrange-Act-Assert workflow. +allowed-tools: + - Read + - Write + - Edit + - Bash +effort: medium +tags: [tdd, testing, rust, red-green-refactor, rtk] --- # Rust TDD Workflow diff --git a/.claude/skills/rtk-triage/SKILL.md b/.claude/skills/rtk-triage/SKILL.md index 9aed21a26..705d18210 100644 --- a/.claude/skills/rtk-triage/SKILL.md +++ b/.claude/skills/rtk-triage/SKILL.md @@ -1,4 +1,5 @@ --- +name: rtk-triage description: > Triage complet RTK : exécute issue-triage + pr-triage en parallèle, puis croise les données pour détecter doubles couvertures, trous sécurité, @@ -9,6 +10,8 @@ allowed-tools: - Write - Read - AskUserQuestion +effort: high +tags: [triage, orchestration, issues, pr, security, cross-analysis, rtk] --- # /rtk-triage diff --git a/.claude/skills/security-guardian.md b/.claude/skills/security-guardian/SKILL.md similarity index 99% rename from .claude/skills/security-guardian.md rename to .claude/skills/security-guardian/SKILL.md index 6a74d4a01..b584353df 100644 --- a/.claude/skills/security-guardian.md +++ b/.claude/skills/security-guardian/SKILL.md @@ -1,5 +1,6 @@ --- description: CLI security expert for RTK - command injection, shell escaping, hook security +allowed-tools: Read Grep Glob Bash --- # Security Guardian diff --git a/.claude/skills/ship.md b/.claude/skills/ship/SKILL.md similarity index 94% rename from .claude/skills/ship.md rename to .claude/skills/ship/SKILL.md index 380a8ba2b..f34ab53a4 100644 --- a/.claude/skills/ship.md +++ b/.claude/skills/ship/SKILL.md @@ -1,5 +1,6 @@ --- description: Build, commit, push & version bump workflow - automates the complete release cycle +allowed-tools: Read Write Edit Bash Grep Glob --- # Ship Release @@ -61,8 +62,9 @@ git status # Should show "nothing to commit, working tree clean" **Files to update**: 1. `Cargo.toml` (line 3): `version = "X.Y.Z"` -2. `CHANGELOG.md` (add new section) -3. `README.md` (if version mentioned) +2. `README.md` (if version mentioned) + +> **Note**: `CHANGELOG.md` is auto-generated by release-please from conventional commit messages — do not edit manually. **Example**: ```toml @@ -119,13 +121,12 @@ hyperfine 'target/release/rtk git status' --warmup 3 ```bash # Stage version files -git add Cargo.toml Cargo.lock CHANGELOG.md README.md +git add Cargo.toml Cargo.lock README.md # Commit with version tag git commit -m "chore(release): bump version to v0.17.0 - Updated Cargo.toml version -- Updated CHANGELOG.md with release notes - Verified all quality checks pass - Benchmarked performance (<10ms startup) @@ -187,7 +188,7 @@ gh release view v0.17.0 ### 3. Installation Verification ```bash # Test installation from release -curl -sSL https://github.com/rtk-ai/rtk/releases/download/v0.17.0/rtk-macos-latest -o rtk +curl -sSL https://github.com/algolia/rtk/releases/download/v0.17.0/rtk-macos-latest -o rtk chmod +x rtk ./rtk --version # Should show v0.17.0 @@ -361,14 +362,7 @@ target/release/rtk --version **Symptom**: CHANGELOG.md has conflicts after rebase -**Solution**: -```bash -# Always add new entries at top -# Manual merge: -# 1. Keep all entries from both branches -# 2. Sort by version (newest first) -# 3. Ensure date format consistency -``` +**Solution**: Do not edit CHANGELOG.md manually. It is auto-generated by release-please from conventional commit messages when merging to master. ## Security Considerations diff --git a/.claude/skills/tdd-rust/SKILL.md b/.claude/skills/tdd-rust/SKILL.md index 4e8c3bb56..87d556941 100644 --- a/.claude/skills/tdd-rust/SKILL.md +++ b/.claude/skills/tdd-rust/SKILL.md @@ -8,6 +8,13 @@ triggers: - "write tests for" - "test coverage" - "fix failing test" +allowed-tools: + - Read + - Write + - Edit + - Bash +effort: medium +tags: [tdd, testing, rust, filters, snapshots, token-savings, rtk] --- # RTK TDD Workflow diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 413e94f04..cf6901c54 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -52,7 +52,7 @@ PRs target the **`develop`** branch, not `main`. All commits require a DCO sign- rtk routes CLI commands via a Clap `Commands` enum in `main.rs` to specialized filter modules in `src/cmds/*/`, each executing the underlying command and compressing output. Token savings are tracked in SQLite via `src/core/tracking.rs`. -For full details see [ARCHITECTURE.md](../ARCHITECTURE.md) and [docs/TECHNICAL.md](../docs/TECHNICAL.md). Module responsibilities are documented in each folder's `README.md` and each file's `//!` doc header. +For full details see [ARCHITECTURE.md](../docs/contributing/ARCHITECTURE.md) and [docs/contributing/TECHNICAL.md](../docs/contributing/TECHNICAL.md). Module responsibilities are documented in each folder's `README.md` and each file's `//!` doc header. ## Key Conventions diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 000000000..57dba0f42 --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,17 @@ +version: 2 +updates: + - package-ecosystem: "cargo" + directory: "/" + schedule: + interval: "weekly" + labels: + - "dependencies" + open-pull-requests-limit: 5 + + - package-ecosystem: "github-actions" + directory: "/" + schedule: + interval: "weekly" + labels: + - "dependencies" + - "ci" diff --git a/.github/workflows/CICD.md b/.github/workflows/CICD.md index 53776a00d..b20e9cff6 100644 --- a/.github/workflows/CICD.md +++ b/.github/workflows/CICD.md @@ -14,27 +14,29 @@ Trigger: pull_request to develop or master └────────┬─────────┘ │ ┌────────▼─────────┐ - │ clippy │ - └──┬───┬───┬───┬───┘ - │ │ │ │ - ┌──────────────┘ │ │ └──────────────┐ - │ ┌───────┘ └───────┐ │ - ▼ ▼ ▼ ▼ - ┌──────────────┐ ┌──────────────┐ ┌───────────┐ ┌──────────┐ - │ test │ │Security Scan │ │ benchmark │ │ validate │ - │ ubuntu │ │ cargo audit │ │ >=80% │ │ ai agent │ - │ windows │ │ (advisory) │ │ savings │ │ doc │ - │ macos │ │ │ │ │ │ │ - └──────┬───────┘ └──────┬───────┘ └─────┬─────┘ └────┬─────┘ - │ │ │ │ - └────────────────┴───────┬───────┴─────────────┘ - │ - ┌──────────▼─────────┐ - │ All must pass │ - │ to merge │ - └────────────────────┘ + │ clippy │ + │ -D unsafe_code │ + └┬───┬───┬───┬───┬─┘ + │ │ │ │ │ + ┌───────────────┘ │ │ │ └───────────────┐ + │ ┌───────────┘ │ └──────────┐ │ + ▼ ▼ ▼ ▼ ▼ + ┌──────────┐ ┌──────────┐ ┌───────────┐ ┌─────────┐ ┌──────────┐ + │ test │ │ security │ │ semgrep │ │benchmark│ │ doc │ + │ ubuntu │ │ cargo │ │ AST-aware │ │ >=80% │ │ review │ + │ windows │ │ audit │ │ diff-only │ │ savings │ │ ai agent │ + │ macos │ │ patterns │ │ │ │ │ │ │ + └────┬─────┘ └────┬─────┘ └─────┬─────┘ └────┬────┘ └────┬─────┘ + │ │ │ │ │ + └────────────┴─────────┬───┴─────────────┴────────────┘ + │ + ┌──────────▼─────────┐ + │ All must pass │ + │ to merge │ + └────────────────────┘ + DCO check (independent, develop PRs only) + + Dependabot (weekly: Cargo deps + GitHub Actions) ``` ## Merge to develop — pre-release (cd.yml) diff --git a/.github/workflows/cd.yml b/.github/workflows/cd.yml index 393f9e500..0ed2bc0fc 100644 --- a/.github/workflows/cd.yml +++ b/.github/workflows/cd.yml @@ -98,17 +98,25 @@ jobs: # ═══════════════════════════════════════════════ release-please: - if: github.ref == 'refs/heads/master' && github.event_name == 'push' + if: github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') runs-on: ubuntu-latest outputs: release_created: ${{ steps.release.outputs.release_created }} tag_name: ${{ steps.release.outputs.tag_name }} steps: + - uses: actions/create-github-app-token@v3 + id: app-token + with: + client-id: ${{ secrets.APP_CLIENT_ID }} + private-key: ${{ secrets.APP_PRIVATE_KEY }} + permission-contents: write + permission-pull-requests: write - uses: googleapis/release-please-action@v4 id: release with: release-type: rust package-name: rtk + token: ${{ steps.app-token.outputs.token }} build-release: name: Build and upload release assets @@ -127,9 +135,17 @@ jobs: if: ${{ needs.release-please.outputs.release_created == 'true' }} runs-on: ubuntu-latest steps: + - uses: actions/create-github-app-token@v3 + id: app-token + with: + client-id: ${{ secrets.APP_CLIENT_ID }} + private-key: ${{ secrets.APP_PRIVATE_KEY }} + permission-contents: write + - uses: actions/checkout@v4 with: fetch-depth: 0 + token: ${{ steps.app-token.outputs.token }} - name: Update latest tag run: | diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index bad4b5d62..b56acffad 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -14,6 +14,18 @@ env: jobs: # ─── Fast gates (fail early, save CI minutes) ─── + check-test-presence: + name: test presence + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 50 + - name: Check filter modules have tests + run: | + git fetch origin "${{ github.base_ref }}" --depth=1 || true + bash scripts/check-test-presence.sh "origin/${{ github.base_ref }}" + fmt: name: fmt runs-on: ubuntu-latest @@ -34,7 +46,7 @@ jobs: with: components: clippy - uses: Swatinem/rust-cache@v2 - - run: cargo clippy --all-targets + - run: cargo clippy --all-targets -- -D unsafe_code # ─── Parallel gates (all need code to compile) ─── @@ -173,6 +185,18 @@ jobs: echo "- Require approval from 2 maintainers" >> $GITHUB_STEP_SUMMARY echo "- Test in isolated environment before merge" >> $GITHUB_STEP_SUMMARY + semgrep: + name: semgrep security scan + needs: clippy + runs-on: ubuntu-latest + container: + image: semgrep/semgrep + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + - run: semgrep scan --config .semgrep.yml --baseline-commit ${{ github.event.pull_request.base.sha }} --error + benchmark: name: benchmark needs: clippy @@ -187,8 +211,11 @@ jobs: - name: Build rtk run: cargo build --release + - name: Install system tools + run: sudo apt-get install -y tree + - name: Install Python tools - run: pip install ruff pytest + run: pip install ruff pytest mypy - name: Install Go uses: actions/setup-go@v5 diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 79b667c02..691a9c554 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -81,9 +81,6 @@ jobs: - name: Build run: cargo build --release --target ${{ matrix.target }} - env: - RTK_TELEMETRY_URL: ${{ vars.RTK_TELEMETRY_URL }} - RTK_TELEMETRY_TOKEN: ${{ secrets.RTK_TELEMETRY_TOKEN }} - name: Package (Unix) if: matrix.os != 'windows-latest' @@ -120,9 +117,6 @@ jobs: - name: Build DEB run: cargo deb - env: - RTK_TELEMETRY_URL: ${{ vars.RTK_TELEMETRY_URL }} - RTK_TELEMETRY_TOKEN: ${{ secrets.RTK_TELEMETRY_TOKEN }} - name: Upload DEB uses: actions/upload-artifact@v4 @@ -147,9 +141,6 @@ jobs: - name: Build release run: cargo build --release - env: - RTK_TELEMETRY_URL: ${{ vars.RTK_TELEMETRY_URL }} - RTK_TELEMETRY_TOKEN: ${{ secrets.RTK_TELEMETRY_TOKEN }} - name: Generate RPM run: cargo generate-rpm @@ -165,6 +156,13 @@ jobs: needs: [build, build-deb, build-rpm] runs-on: ubuntu-latest steps: + - uses: actions/create-github-app-token@v3 + id: app-token + with: + client-id: ${{ secrets.APP_CLIENT_ID }} + private-key: ${{ secrets.APP_PRIVATE_KEY }} + permission-contents: write + - name: Checkout uses: actions/checkout@v4 @@ -208,8 +206,7 @@ jobs: tag_name: ${{ steps.version.outputs.version }} files: release/* prerelease: ${{ inputs.prerelease }} - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + token: ${{ steps.app-token.outputs.token }} notify-discord: name: Notify Discord @@ -232,10 +229,10 @@ jobs: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} run: | TAG="${{ steps.version.outputs.tag }}" - RELEASE_URL="https://github.com/rtk-ai/rtk/releases/tag/${TAG}" + RELEASE_URL="https://github.com/algolia/rtk/releases/tag/${TAG}" # Fetch release notes from GitHub API - NOTES=$(gh api "repos/rtk-ai/rtk/releases/tags/${TAG}" --jq '.body' 2>/dev/null | head -c 1800 || echo "") + NOTES=$(gh api "repos/algolia/rtk/releases/tags/${TAG}" --jq '.body' 2>/dev/null | head -c 1800 || echo "") DESC=$(echo "${NOTES:-No release notes}" | jq -Rs .) jq -n \ @@ -265,7 +262,7 @@ jobs: - name: Download checksums run: | gh release download "${{ steps.version.outputs.tag }}" \ - --repo rtk-ai/rtk \ + --repo algolia/rtk \ --pattern checksums.txt env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} @@ -283,21 +280,21 @@ jobs: cat > rtk.rb << 'FORMULA' class Rtk < Formula desc "Rust Token Killer - High-performance CLI proxy to minimize LLM token consumption" - homepage "https://www.rtk-ai.app" + homepage "https://github.com/algolia/rtk" version "VERSION_PLACEHOLDER" license "MIT" if OS.mac? && Hardware::CPU.arm? - url "https://github.com/rtk-ai/rtk/releases/download/TAG_PLACEHOLDER/rtk-aarch64-apple-darwin.tar.gz" + url "https://github.com/algolia/rtk/releases/download/TAG_PLACEHOLDER/rtk-aarch64-apple-darwin.tar.gz" sha256 "SHA_MAC_ARM_PLACEHOLDER" elsif OS.mac? && Hardware::CPU.intel? - url "https://github.com/rtk-ai/rtk/releases/download/TAG_PLACEHOLDER/rtk-x86_64-apple-darwin.tar.gz" + url "https://github.com/algolia/rtk/releases/download/TAG_PLACEHOLDER/rtk-x86_64-apple-darwin.tar.gz" sha256 "SHA_MAC_INTEL_PLACEHOLDER" elsif OS.linux? && Hardware::CPU.arm? - url "https://github.com/rtk-ai/rtk/releases/download/TAG_PLACEHOLDER/rtk-aarch64-unknown-linux-gnu.tar.gz" + url "https://github.com/algolia/rtk/releases/download/TAG_PLACEHOLDER/rtk-aarch64-unknown-linux-gnu.tar.gz" sha256 "SHA_LINUX_ARM_PLACEHOLDER" elsif OS.linux? && Hardware::CPU.intel? - url "https://github.com/rtk-ai/rtk/releases/download/TAG_PLACEHOLDER/rtk-x86_64-unknown-linux-musl.tar.gz" + url "https://github.com/algolia/rtk/releases/download/TAG_PLACEHOLDER/rtk-x86_64-unknown-linux-musl.tar.gz" sha256 "SHA_LINUX_INTEL_PLACEHOLDER" end @@ -319,7 +316,7 @@ jobs: # Measure your token savings rtk gain - Full documentation: https://www.rtk-ai.app + Full documentation: https://github.com/algolia/rtk EOS end @@ -340,14 +337,14 @@ jobs: - name: Push to homebrew-tap run: | CONTENT=$(base64 -w 0 rtk.rb) - SHA=$(gh api repos/rtk-ai/homebrew-tap/contents/Formula/rtk.rb --jq '.sha' 2>/dev/null || echo "") + SHA=$(gh api repos/algolia/homebrew-tap/contents/Formula/rtk.rb --jq '.sha' 2>/dev/null || echo "") if [ -n "$SHA" ]; then - gh api -X PUT repos/rtk-ai/homebrew-tap/contents/Formula/rtk.rb \ + gh api -X PUT repos/algolia/homebrew-tap/contents/Formula/rtk.rb \ -f message="rtk ${{ steps.version.outputs.version }}" \ -f content="$CONTENT" \ -f sha="$SHA" else - gh api -X PUT repos/rtk-ai/homebrew-tap/contents/Formula/rtk.rb \ + gh api -X PUT repos/algolia/homebrew-tap/contents/Formula/rtk.rb \ -f message="rtk ${{ steps.version.outputs.version }}" \ -f content="$CONTENT" fi diff --git a/.gitignore b/.gitignore index fdba09783..947ca4fe7 100644 --- a/.gitignore +++ b/.gitignore @@ -28,8 +28,10 @@ Thumbs.db # Test artifacts *.cast.bak -# Benchmark results -scripts/benchmark/ +# Benchmark results (fixture data, not infra) +scripts/benchmark/diff/ +scripts/benchmark/rtk/ +scripts/benchmark/unix/ benchmark-report.md # SQLite databases diff --git a/.release-please-manifest.json b/.release-please-manifest.json index 57647ad29..521104886 100644 --- a/.release-please-manifest.json +++ b/.release-please-manifest.json @@ -1,3 +1,3 @@ { - ".": "0.34.2" + ".": "0.38.0-algolia.1" } diff --git a/.rtk/filters.toml b/.rtk/filters.toml new file mode 100644 index 000000000..9a137dcd5 --- /dev/null +++ b/.rtk/filters.toml @@ -0,0 +1,13 @@ +# Project-local RTK filters — commit this file with your repo. +# Filters here override user-global and built-in filters. +# Docs: https://github.com/algolia/rtk#custom-filters +schema_version = 1 + +# Example: suppress build noise from a custom tool +# [filters.my-tool] +# description = "Compact my-tool output" +# match_command = "^my-tool\\s+build" +# strip_ansi = true +# strip_lines_matching = ["^\\s*$", "^Downloading", "^Installing"] +# max_lines = 30 +# on_empty = "my-tool: ok" diff --git a/.semgrep.yml b/.semgrep.yml new file mode 100644 index 000000000..314fe90ca --- /dev/null +++ b/.semgrep.yml @@ -0,0 +1,107 @@ +rules: + - id: dynamic-command-execution + patterns: + - pattern: Command::new($ARG) + - pattern-not: Command::new("...") + message: > + Dynamic shell invocation via Command::new($ARG). + RTK only executes known CLI tools — use string literals, not variables. + languages: [rust] + severity: ERROR + + - id: unsafe-block + pattern: unsafe { ... } + message: > + Unsafe block detected. RTK has no legitimate need for unsafe code. + languages: [rust] + severity: ERROR + + - id: ld-preload-manipulation + pattern-either: + - pattern: $CMD.env("LD_PRELOAD", ...) + - pattern: $CMD.env("LD_LIBRARY_PATH", ...) + message: > + LD_PRELOAD/LD_LIBRARY_PATH manipulation detected. + This can hijack shared library loading — forbidden in RTK. + languages: [rust] + severity: ERROR + + - id: raw-socket-usage + pattern-either: + - pattern: TcpStream::$METHOD(...) + - pattern: UdpSocket::$METHOD(...) + - pattern: TcpListener::$METHOD(...) + message: > + Raw socket usage detected. RTK is a CLI proxy — it should not + open network connections directly. Network access is banned in + this fork (no telemetry, no remote calls). + languages: [rust] + severity: ERROR + + - id: reqwest-forbidden + pattern: reqwest::$METHOD(...) + message: > + reqwest is forbidden in RTK. Network access is banned in this fork + (no telemetry, no remote calls). Adding reqwest also increases + binary size and attack surface. + languages: [rust] + severity: ERROR + + - id: interpreter-execution + pattern-either: + - pattern: Command::new("curl") + - pattern: Command::new("wget") + - pattern: Command::new("python") + - pattern: Command::new("python3") + - pattern: Command::new("node") + - pattern: Command::new("bash") + - pattern: Command::new("sh") + - pattern: Command::new("perl") + - pattern: Command::new("ruby") + message: > + Direct interpreter/downloader execution detected. + RTK proxies user commands — it should never spawn interpreters + or download tools on its own. + languages: [rust] + severity: ERROR + + - id: ureq-forbidden + pattern: ureq::$METHOD(...) + message: > + ureq usage detected. Network access is banned in this fork + (no telemetry, no remote calls). + languages: [rust] + severity: ERROR + + # ── WARNING rules (non-blocking, flag for review) ── + + - id: path-env-manipulation + pattern-either: + - pattern: $CMD.env("PATH", ...) + - pattern: std::env::set_var("PATH", ...) + - pattern: env::set_var("PATH", ...) + message: > + PATH environment variable manipulation detected. + Hijacking PATH can redirect command resolution to attacker-controlled binaries. + languages: [rust] + severity: WARNING + + - id: sensitive-path-reference + pattern-regex: \.(ssh|bashrc|zshrc|bash_profile|profile)|authorized_keys|/etc/passwd|/etc/shadow + message: > + Reference to sensitive system path detected. + RTK filters should not access dotfiles, SSH keys, or system credential files. + languages: [rust] + severity: WARNING + + - id: filesystem-deletion + pattern-either: + - pattern: fs::remove_file(...) + - pattern: fs::remove_dir_all(...) + - pattern: std::fs::remove_file(...) + - pattern: std::fs::remove_dir_all(...) + message: > + File/directory deletion detected. Expected in hooks/init cleanup, + surprising in a filter module. Verify intent. + languages: [rust] + severity: WARNING diff --git a/CHANGELOG.md b/CHANGELOG.md index 5ed7ca6ab..6ef8bb35c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,247 @@ All notable changes to rtk (Rust Token Killer) will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [0.38.0](https://github.com/rtk-ai/rtk/compare/v0.37.2...v0.38.0) (2026-04-29) + + +### Features + +* **cicd:** enforce cicd sast & package check ([3bbbb49](https://github.com/rtk-ai/rtk/commit/3bbbb492f33f0e619ab0d1dbce4389ad49e763ae)) +* **gains:** add --reset flag ([e3149cb](https://github.com/rtk-ai/rtk/commit/e3149cb7fbed18eae95f753664ddd8eaaaf6cc39)) +* **glab:** add GitLab CLI (glab) command support ([048f2f9](https://github.com/rtk-ai/rtk/commit/048f2f980bd95c5918f309d1d7ebc096d196f00d)) +* **glab:** add GitLab CLI (glab) command support ([bc31f3f](https://github.com/rtk-ai/rtk/commit/bc31f3f0f39077884e8d52c3508e840b355f682e)), closes [#851](https://github.com/rtk-ai/rtk/issues/851) + + +### Bug Fixes + +* **benchmark:** benchmark capture all fd only stream ([c590bd6](https://github.com/rtk-ai/rtk/commit/c590bd69329bb82608666958c7e06bf169a7d577)) +* **benchmark:** capture all fd for stream cmd benchmark ([e6c2523](https://github.com/rtk-ai/rtk/commit/e6c2523be1180772e40c175e2f9a523d349fb13d)) +* **benchmark:** extract format_diff_changes + remove wrong diff test ([e7ae6bf](https://github.com/rtk-ai/rtk/commit/e7ae6bf018882dba248f151ba4ec4929300b3e36)) +* **cicd:** : no semgrep alert on sh call cicd ([7681daf](https://github.com/rtk-ai/rtk/commit/7681dafc76f164cfad588fe37d9a165dcb476e10)) +* **discover:** also encode '_', '\', and non-ASCII chars in project path slug ([73a05c3](https://github.com/rtk-ai/rtk/commit/73a05c3262b6410cb24370d939c428d1dc0c7a77)), closes [#1457](https://github.com/rtk-ai/rtk/issues/1457) +* **discover:** encode '.' as '-' in project path slug ([2d031f3](https://github.com/rtk-ai/rtk/commit/2d031f32e9ad4452c2cc229c030ea6c0936c8bec)), closes [#1457](https://github.com/rtk-ai/rtk/issues/1457) +* **filters:** benchmark ci update + fix stream + filter quality ([137af04](https://github.com/rtk-ai/rtk/commit/137af0493189a86020da1feaa1de74df92466137)) +* **filters:** benchmark ci update + fix stream filter quality ([88d9f6a](https://github.com/rtk-ai/rtk/commit/88d9f6a0d94fd2b5b3d40c956e966756670a2704)) +* **git:** fix empty output when branch name contains '/' in git diff ([e070226](https://github.com/rtk-ai/rtk/commit/e0702260a94377b6bbec5cb79d91d81cba17b0ec)) +* **git:** fix empty output when branch name contains '/' in git diff ([13188a8](https://github.com/rtk-ai/rtk/commit/13188a88b22f692157b89874f4c76287a0b3ecae)), closes [#1431](https://github.com/rtk-ai/rtk/issues/1431) +* grep false negatives, output mangling, and truncation annotations ([de41533](https://github.com/rtk-ai/rtk/commit/de415335ea069c06370855366945a3704579ee18)) +* **install:** resolve version via redirect to avoid GitHub API rate limits ([5e1a641](https://github.com/rtk-ai/rtk/commit/5e1a64180f094ae456780a78b675f243312089c6)) +* **npm:** regex match end line ([5e84e94](https://github.com/rtk-ai/rtk/commit/5e84e9471736fe58e89094854f4123ecb07c2d3b)) +* **npx:** dispatch unknown tools to npx instead of npm ([2c4569c](https://github.com/rtk-ai/rtk/commit/2c4569caa64d013ad4ada0b7580f9f16d8334c19)), closes [#815](https://github.com/rtk-ai/rtk/issues/815) +* remove wrong cicd benchmark + npm test regex ([7e3690a](https://github.com/rtk-ai/rtk/commit/7e3690a23ab158ca8e1e890650554e20e3a0c17b)) +* **stream:** add semgrep flag for sh tests ([7cfcdbe](https://github.com/rtk-ai/rtk/commit/7cfcdbec8681b15b794b6aef982ccb38feb79fd7)) +* **stream:** add semgrep flag for sh tests ([d327724](https://github.com/rtk-ai/rtk/commit/d327724f814b6875903366db0b0616780b454ad1)) +* **stream:** route to respective fd ([605e335](https://github.com/rtk-ai/rtk/commit/605e335f0546d2ed8554a95e7749a0b494c510e3)) +* **stream:** route to respective fd ([81a1be6](https://github.com/rtk-ai/rtk/commit/81a1be6a744942515347dd296ddcf7d9f126200d)) +* **tracking:** test env path ([70b36b4](https://github.com/rtk-ai/rtk/commit/70b36b4dbc3e147219ad87cf539d073523b86a85)) + +## [0.37.2](https://github.com/rtk-ai/rtk/compare/v0.37.1...v0.37.2) (2026-04-20) + + +### Bug Fixes + +* **discover:** exclude_commands bypass for env-prefix, sub cmd + regex ([ca4c59c](https://github.com/rtk-ai/rtk/commit/ca4c59c230306d310069bed3c0ba930068dc4dc4)) +* **discover:** exclude_commands bypass for env-prefix, sub cmd + regex ([42d3161](https://github.com/rtk-ai/rtk/commit/42d3161872713bc0b20ef49b0714add40c40d5e3)) +* **discover:** word boundary in exclude_commands ([0ea115b](https://github.com/rtk-ai/rtk/commit/0ea115bca5fa66daa69fda2f0eeaaf103346b3a4)) +* **docs:** add missing docs for exclude commands patterns ([2e401ac](https://github.com/rtk-ai/rtk/commit/2e401ac38feec88de8d5e46f0301c8a532b95614)) +* **hooks:** add regression test for windows native ([115e448](https://github.com/rtk-ai/rtk/commit/115e44853b8cdd2d7af3af2b52c9c31e924a45d3)) +* **hooks:** windows use 'rtk hook claude' no fallback ([da3c432](https://github.com/rtk-ai/rtk/commit/da3c432201240f0da9627d8cc6bc70e5b7f8bdfe)) +* **hooks:** windows use 'rtk hook claude' no fallback ([0e29650](https://github.com/rtk-ai/rtk/commit/0e29650e11959730f4c4a2e6d6c0519e14dc8595)) +* **tests:** windows regression test fix path ([13a73dd](https://github.com/rtk-ai/rtk/commit/13a73ddfd78460560a1f5fde94b54b1f848b41b5)) + +## [0.37.1](https://github.com/rtk-ai/rtk/compare/v0.37.0...v0.37.1) (2026-04-18) + + +### Bug Fixes + +* **docs:** user facing docs ([c8d6878](https://github.com/rtk-ai/rtk/commit/c8d68787fb8b31c52125e9fc7ea62e0aa590485f)) + +## [0.37.0](https://github.com/rtk-ai/rtk/compare/v0.36.0...v0.37.0) (2026-04-17) + + +### Features + +* **discover:** handle more npm/npx/pnpm/pnpx patterns ([9e96caa](https://github.com/rtk-ai/rtk/commit/9e96caa0a18a95c84da82ba57716a9d3ef86d0c8)) +* **refacto-core:** binary hook w/ native cmd exec + streaming ([e7b7f9a](https://github.com/rtk-ai/rtk/commit/e7b7f9ab665a0f7303d41d23ad156d24e5e8964e)) + + +### Bug Fixes + +* **docs:** use release please changelog no manual ([7591a14](https://github.com/rtk-ai/rtk/commit/7591a14e4ceb732ab7ca160ac01a852926abe77a)) +* isolate cursor hook tests from local settings (determinist) ([d8ddefe](https://github.com/rtk-ai/rtk/commit/d8ddefe78efe25c35bb2a2f9083f2eacb9dd7274)) +* P0+P1 fixes from pre-merge review of hook engine ([df8e035](https://github.com/rtk-ai/rtk/commit/df8e03558d4d6cc2f5cbac91c63ab1b3b51d3bcd)) +* P0+P1 fixes from pre-merge review of hook engine ([d34389c](https://github.com/rtk-ai/rtk/commit/d34389c3d0936c2b0790e14f450bb50a28a7edf7)) +* rename ship.md to ship/SKILL.md to match develop ([5916ecd](https://github.com/rtk-ai/rtk/commit/5916ecd86fb319c2519a0b4fb2891309833a3bb4)) +* **runner:** preserve fd separation on command failure ([e92d099](https://github.com/rtk-ai/rtk/commit/e92d0993c93f0b732316dfa932d265aeca7488d6)) +* **stream:** missing stderr fields ([a1d46f3](https://github.com/rtk-ai/rtk/commit/a1d46f39c291e3356b9c26a062bde05ba1de591a)) + +## [0.36.0](https://github.com/rtk-ai/rtk/compare/v0.35.0...v0.36.0) (2026-04-13) + + +### Features + +* **benchmark:** add multipass VM integration test suite ([6e7863b](https://github.com/rtk-ai/rtk/commit/6e7863bf313b0d18a47cf0ca2cdaea03cc2ed900)) +* **benchmark:** add multipass VM integration test suite ([d22759b](https://github.com/rtk-ai/rtk/commit/d22759b8c5254ad9c4a455f10cb7de75e92df581)) +* **benchmark:** add Swift ecosystem tests (6 commands + savings) ([1fbb6d9](https://github.com/rtk-ai/rtk/commit/1fbb6d935b4a0d031a7862cba312eebe1303ba9b)) +* **init:** add native support for Kilo Code and Google Antigravity ([d0a3797](https://github.com/rtk-ai/rtk/commit/d0a3797ec580f96948489d1e7c3329ac22a6c4eb)) +* **init:** add support for kilocode and antigravity agents ([66b90f1](https://github.com/rtk-ai/rtk/commit/66b90f1ed3de81acdce61164c068c24ed7ef29db)) +* **pnpm:** Add filter argument support ([2ba8d37](https://github.com/rtk-ai/rtk/commit/2ba8d372df186b4056a3b8906fc25cde8586dd42)) +* **skills:** add /pr-review skill for batch PR review ([21e67a1](https://github.com/rtk-ai/rtk/commit/21e67a1113041b74542d0285e5f74587dfb30b65)) +* **telemetry:** enrich daily ping with gap detection and quality metrics ([644c50f](https://github.com/rtk-ai/rtk/commit/644c50f786e5c567617e7ea907c5f312797b1265)) + + +### Bug Fixes + +* **benchmark:** address PR review feedback ([87ee81f](https://github.com/rtk-ai/rtk/commit/87ee81f08be5e7b1ca79513b1a91925d455f4f5c)) +* **benchmark:** address review feedback from @FlorianBruniaux ([d13c185](https://github.com/rtk-ai/rtk/commit/d13c185aac64d14288b574df44623723a69e7b95)) +* **ccusage:** add --yes flag and warn when falling back to npx ([f68fa00](https://github.com/rtk-ai/rtk/commit/f68fa0087c03d6882993b7b3eaee98e1dbab41b4)) +* **clippy:** show full error blocks instead of truncated headline ([95d9d13](https://github.com/rtk-ai/rtk/commit/95d9d134b0b76d83b6162614b0a79269b2135f40)) +* **clippy:** show full error blocks instead of truncated headline ([f4074f8](https://github.com/rtk-ai/rtk/commit/f4074f898a9b73b72bbcd8b18afab4831dcda406)), closes [#602](https://github.com/rtk-ai/rtk/issues/602) +* **curl:** skip JSON schema conversion for internal/localhost URLs ([577c311](https://github.com/rtk-ai/rtk/commit/577c311ecaaa8ae94f22dbe252152424d4333d04)) +* **discover:** preserve golangci-lint flags in rewrite ([d85303e](https://github.com/rtk-ai/rtk/commit/d85303ec4893deb904260f5dc11b7df906a50c07)) +* **docs:** update TELEMETRY.md to match code after review fixes ([be5c057](https://github.com/rtk-ai/rtk/commit/be5c0576d95566f37f266fd9f92e2a1b263697bd)) +* **find:** include hidden files when pattern targets dotfiles ([#1101](https://github.com/rtk-ai/rtk/issues/1101)) ([dbeeaed](https://github.com/rtk-ai/rtk/commit/dbeeaed16aee79674ec2fd3778b7b11b10b847c6)) +* **git:** re-insert -- separator when clap consumes it from git diff args ([#1215](https://github.com/rtk-ai/rtk/issues/1215)) ([9979c69](https://github.com/rtk-ai/rtk/commit/9979c699307a4adad2c2df0f2bc3b663df653311)) +* **git:** remove -u short alias from --ultra-compact to fix git push -u ([6b76fdb](https://github.com/rtk-ai/rtk/commit/6b76fdb87d7c54cfc2a1b0e6117dd78b8430910b)) +* **golangci-lint:** restore run wrapper and align guidance ([4f4e4d2](https://github.com/rtk-ai/rtk/commit/4f4e4d2b5a3529030fe4089f60d2f4b8740b5d53)) +* **golangci-lint:** support inline global flags before run ([24f2ada](https://github.com/rtk-ai/rtk/commit/24f2adaf8fb541c2564fa7dfb423947932e68fb4)) +* **go:** prevent double-counted failures when test-level fail also triggers package-level fail ([#958](https://github.com/rtk-ai/rtk/issues/958)) ([4fc15ef](https://github.com/rtk-ai/rtk/commit/4fc15ef2c1c80336ffaafa4179db4cee6f39236a)) +* **go:** prevent double-counting failures when package-level fail cascades from test failures ([#958](https://github.com/rtk-ai/rtk/issues/958)) ([9722d5e](https://github.com/rtk-ai/rtk/commit/9722d5ebd8916f9b398bdc01b1102d42ab2b8795)) +* **hooks:** ensure default permission verdict prompts user for confirmation ([40462c0](https://github.com/rtk-ai/rtk/commit/40462c05e66f116928de365a0d271bdfd61cec72)) +* **hooks:** require all segments to match allow rules ([#1213](https://github.com/rtk-ai/rtk/issues/1213)) ([40c9dbc](https://github.com/rtk-ai/rtk/commit/40c9dbc7dbbf9332d6859060765c582a880f0fde)) +* **init:** honor CODEX_HOME for Codex global paths ([d442799](https://github.com/rtk-ai/rtk/commit/d442799e34d522c87a6eb60c2ff373385d201315)) +* **init:** install Codex global instructions in CODEX_HOME ([a257688](https://github.com/rtk-ai/rtk/commit/a2576883a27c5f915ba0ae7883a51006411b3ae5)) +* **json:** rename --schema to --keys-only, closes [#621](https://github.com/rtk-ai/rtk/issues/621) ([c16713a](https://github.com/rtk-ai/rtk/commit/c16713a973b563a6cba283c830b67c8c470e419f)) +* **ls:** filter quality wrong truncation ([aa6317f](https://github.com/rtk-ai/rtk/commit/aa6317fb83a5d9883623a4d3bee7a25bc99dcb4c)) +* **permissions:** glob_matches middle-wildcard matches commands without trailing args ([#1105](https://github.com/rtk-ai/rtk/issues/1105)) ([3db8070](https://github.com/rtk-ai/rtk/commit/3db8070b51b9a312fcca20a8460d3d6259cc38b7)) +* **pnpm:** list command not working ([ba235d8](https://github.com/rtk-ai/rtk/commit/ba235d85974c0a85b25e290a8bb83648800438a6)) +* **pytest:** -q mode summary line not detected ([57502a5](https://github.com/rtk-ai/rtk/commit/57502a5bef1fb56109a57cf2ea7377fd271253a7)) +* report package-level failures (timeouts, signals) in go test summary ([0b1c32b](https://github.com/rtk-ai/rtk/commit/0b1c32b3cc9a3e73418d401d1d481c1611c7ec0b)) +* report package-level failures (timeouts, signals) in go test summary ([c85a387](https://github.com/rtk-ai/rtk/commit/c85a387363e2079234b6141aad26418172c0e61a)), closes [#958](https://github.com/rtk-ai/rtk/issues/958) +* **security:** correct email domain from .dev to .app ([47383e8](https://github.com/rtk-ai/rtk/commit/47383e80197fc56e38f880f33a6b54261b82523c)) +* **tee:** prevent panic on UTF-8 multi-byte truncation boundary ([da486bf](https://github.com/rtk-ai/rtk/commit/da486bf394330c804cd1cd12e4b6835f18de5205)) +* **telemetry:** 7 bugs in enrichment — privacy leak, broken meta_usage, pricing ([15f666d](https://github.com/rtk-ai/rtk/commit/15f666dd8dbd18648cb7bd14a6f9f3cac2f7d10b)) +* **telemetry:** clean code ([8156081](https://github.com/rtk-ai/rtk/commit/81560812610686fa5ca3633c2bf0b79c05eaa7d9)) +* **telemetry:** consent, erasure, auth, docs ([2e4cc4b](https://github.com/rtk-ai/rtk/commit/2e4cc4bb5226444c8c0bfc827baf0c101c3759e8)) +* **telemetry:** non-terminal consent, single config load ([7821e98](https://github.com/rtk-ai/rtk/commit/7821e9872fd1f1ae9b40eb8a4458049869acc36b)) +* **telemetry:** RGPD-compliant, consent gate, erasure, privacy controls ([6a5bc84](https://github.com/rtk-ai/rtk/commit/6a5bc847e06cf6066e6f4aeed5a3ad0803a3649b)) + +## [0.35.0](https://github.com/rtk-ai/rtk/compare/v0.34.3...v0.35.0) (2026-04-06) + + +### Features + +* **aws:** expand CLI filters from 8 to 25 subcommands ([402c48e](https://github.com/rtk-ai/rtk/commit/402c48e66988e638a5b4f4dd193238fc1d0fe18f)) + + +### Bug Fixes + +* **cmd:** read/cat multiple file and consistent behavior ([3f58018](https://github.com/rtk-ai/rtk/commit/3f58018f4af1d7206457929cf80bb4534203c3ee)) +* **docs:** clean some docs + disclaimer ([deda44f](https://github.com/rtk-ai/rtk/commit/deda44f73607981f3d27ecc6341ce927aab34d37)) +* **gh:** pass through gh pr merge instead of canned response ([#938](https://github.com/rtk-ai/rtk/issues/938)) ([8465ca9](https://github.com/rtk-ai/rtk/commit/8465ca953fa9d70dcc971a941c19465d456eb7d4)) +* **gh:** pass through gh pr merge instead of canned response ([#938](https://github.com/rtk-ai/rtk/issues/938)) ([e1f2845](https://github.com/rtk-ai/rtk/commit/e1f2845df06a8d8b8325945dc4940ec5f530e4cc)) +* **git:** inherit stdin for commit and push to preserve SSH signing ([#733](https://github.com/rtk-ai/rtk/issues/733)) ([eefeae4](https://github.com/rtk-ai/rtk/commit/eefeae45656ff2607c3f519c8eae235e3f0fe411)) +* **git:** inherit stdin for commit and push to preserve SSH signing ([#733](https://github.com/rtk-ai/rtk/issues/733)) ([6cee6c6](https://github.com/rtk-ai/rtk/commit/6cee6c60b80f914ed9505e3925d85cadec43ab97)) +* **git:** preserve full diff hunk headers ([62f4452](https://github.com/rtk-ai/rtk/commit/62f445227679f3df293fe35e9b18cc5ab39d7963)) +* **git:** preserve full diff hunk headers ([09b3ff9](https://github.com/rtk-ai/rtk/commit/09b3ff9424e055f5fe25e535e5b60e077f8344f9)) +* **go:** avoid false build errors from download logs ([9c1cf2f](https://github.com/rtk-ai/rtk/commit/9c1cf2f403534fa7874638b1b983c2d7f918a185)) +* **go:** avoid false build errors from download logs ([d44fd3e](https://github.com/rtk-ai/rtk/commit/d44fd3e034208e3bcd59c2c46f7720eec4f10c98)) +* **go:** cover more build failure shapes ([2425ad6](https://github.com/rtk-ai/rtk/commit/2425ad68e5386d19e5ec9ff1ca151a6d2c9a56d3)) +* **go:** preserve failing test location context ([1481bc5](https://github.com/rtk-ai/rtk/commit/1481bc590924031456a6022510275c29c09e330e)) +* **go:** preserve failing test location context ([374fe64](https://github.com/rtk-ai/rtk/commit/374fe64cfbedcd676733973e81a63a6dfecbb1b7)) +* **go:** restore build error coverage ([1177c9c](https://github.com/rtk-ai/rtk/commit/1177c9c873ac63b6c0bcc9e1b664a705baa0ad7a)) +* **grep:** close subprocess stdin to prevent memory leak ([#897](https://github.com/rtk-ai/rtk/issues/897)) ([7217562](https://github.com/rtk-ai/rtk/commit/72175623551f40b581b4a7f6ed966c1e4a9c7358)) +* **grep:** close subprocess stdin to prevent memory leak ([#897](https://github.com/rtk-ai/rtk/issues/897)) ([09979cf](https://github.com/rtk-ai/rtk/commit/09979cf29701a1b775bcac761d24ec0e055d1bec)) +* **hook_check:** detect missing integrations ([9cf9ccc](https://github.com/rtk-ai/rtk/commit/9cf9ccc1ac39f8bba37e932c7d318a3aa7a34ae9)) +* **init:** remove opt-out instruction from telemetry message ([7571c8e](https://github.com/rtk-ai/rtk/commit/7571c8e101c41ee64c51e2bd64697f85f9142423)) +* **init:** remove telemetry info lines from init output ([7dbef2c](https://github.com/rtk-ai/rtk/commit/7dbef2ce00824d26f2057e4c3c76e429e2e23088)) +* **main:** kill zombie processes + path for rtk md ([d16fc6d](https://github.com/rtk-ai/rtk/commit/d16fc6dacbfec912c21522939b15b7bbd9719487)) +* **main:** kill zombie processes + path for rtk md + missing intergrations ([a919335](https://github.com/rtk-ai/rtk/commit/a919335519ed4a5259a212e56407cb312aa99bac)) +* **merge:** changelog conflicts ([d92c5d2](https://github.com/rtk-ai/rtk/commit/d92c5d264a49483c8d6079e04d946a79bc990a74)) +* **proxy:** kill child process on SIGINT/SIGTERM to prevent orphans ([d813919](https://github.com/rtk-ai/rtk/commit/d813919a24546e044e7844fc7ed05fef4ec24033)) +* **proxy:** kill child process on SIGINT/SIGTERM to prevent orphans ([3318510](https://github.com/rtk-ai/rtk/commit/33185101fc122d0c11a25a4e02ac9f3a7dc7e3bb)) +* **review:** address ChildGuard disarm, stdin dedup, hook masking ([d85fe33](https://github.com/rtk-ai/rtk/commit/d85fe3384b87c16fafd25ec7bcadbff6e69f3f1f)) +* **security:** default to ask when no permission rule matches ([#886](https://github.com/rtk-ai/rtk/issues/886)) ([158c745](https://github.com/rtk-ai/rtk/commit/158c74527f6591d372e40a78cd604d73a20649a9)) +* **security:** default to ask when no permission rule matches ([#886](https://github.com/rtk-ai/rtk/issues/886)) ([41a6c6b](https://github.com/rtk-ai/rtk/commit/41a6c6bf6da78a4754794fdc6a1469df2e327920)) +* **tracking:** use std::env::temp_dir() for compatibility (instead of unix tmp) ([e918661](https://github.com/rtk-ai/rtk/commit/e918661440d7b50321f0535032f52c5e87aaf3cb)) + +## [Unreleased] + +### Bug Fixes + +* **git:** remove `-u` short alias from `--ultra-compact` to fix `git push -u` upstream tracking ([#1086](https://github.com/rtk-ai/rtk/issues/1086)) + +## [0.35.0](https://github.com/rtk-ai/rtk/compare/v0.34.3...v0.35.0) (2026-04-06) + + +### Features + +* **aws:** expand CLI filters from 8 to 25 subcommands ([402c48e](https://github.com/rtk-ai/rtk/commit/402c48e66988e638a5b4f4dd193238fc1d0fe18f)) + + +### Bug Fixes + +* **cmd:** read/cat multiple file and consistent behavior ([3f58018](https://github.com/rtk-ai/rtk/commit/3f58018f4af1d7206457929cf80bb4534203c3ee)) +* **docs:** clean some docs + disclaimer ([deda44f](https://github.com/rtk-ai/rtk/commit/deda44f73607981f3d27ecc6341ce927aab34d37)) +* **gh:** pass through gh pr merge instead of canned response ([#938](https://github.com/rtk-ai/rtk/issues/938)) ([8465ca9](https://github.com/rtk-ai/rtk/commit/8465ca953fa9d70dcc971a941c19465d456eb7d4)) +* **gh:** pass through gh pr merge instead of canned response ([#938](https://github.com/rtk-ai/rtk/issues/938)) ([e1f2845](https://github.com/rtk-ai/rtk/commit/e1f2845df06a8d8b8325945dc4940ec5f530e4cc)) +* **git:** inherit stdin for commit and push to preserve SSH signing ([#733](https://github.com/rtk-ai/rtk/issues/733)) ([eefeae4](https://github.com/rtk-ai/rtk/commit/eefeae45656ff2607c3f519c8eae235e3f0fe411)) +* **git:** inherit stdin for commit and push to preserve SSH signing ([#733](https://github.com/rtk-ai/rtk/issues/733)) ([6cee6c6](https://github.com/rtk-ai/rtk/commit/6cee6c60b80f914ed9505e3925d85cadec43ab97)) +* **git:** preserve full diff hunk headers ([62f4452](https://github.com/rtk-ai/rtk/commit/62f445227679f3df293fe35e9b18cc5ab39d7963)) +* **git:** preserve full diff hunk headers ([09b3ff9](https://github.com/rtk-ai/rtk/commit/09b3ff9424e055f5fe25e535e5b60e077f8344f9)) +* **go:** avoid false build errors from download logs ([9c1cf2f](https://github.com/rtk-ai/rtk/commit/9c1cf2f403534fa7874638b1b983c2d7f918a185)) +* **go:** avoid false build errors from download logs ([d44fd3e](https://github.com/rtk-ai/rtk/commit/d44fd3e034208e3bcd59c2c46f7720eec4f10c98)) +* **go:** cover more build failure shapes ([2425ad6](https://github.com/rtk-ai/rtk/commit/2425ad68e5386d19e5ec9ff1ca151a6d2c9a56d3)) +* **go:** preserve failing test location context ([1481bc5](https://github.com/rtk-ai/rtk/commit/1481bc590924031456a6022510275c29c09e330e)) +* **go:** preserve failing test location context ([374fe64](https://github.com/rtk-ai/rtk/commit/374fe64cfbedcd676733973e81a63a6dfecbb1b7)) +* **go:** restore build error coverage ([1177c9c](https://github.com/rtk-ai/rtk/commit/1177c9c873ac63b6c0bcc9e1b664a705baa0ad7a)) +* **grep:** close subprocess stdin to prevent memory leak ([#897](https://github.com/rtk-ai/rtk/issues/897)) ([7217562](https://github.com/rtk-ai/rtk/commit/72175623551f40b581b4a7f6ed966c1e4a9c7358)) +* **grep:** close subprocess stdin to prevent memory leak ([#897](https://github.com/rtk-ai/rtk/issues/897)) ([09979cf](https://github.com/rtk-ai/rtk/commit/09979cf29701a1b775bcac761d24ec0e055d1bec)) +* **hook_check:** detect missing integrations ([9cf9ccc](https://github.com/rtk-ai/rtk/commit/9cf9ccc1ac39f8bba37e932c7d318a3aa7a34ae9)) +* **init:** remove opt-out instruction from telemetry message ([7571c8e](https://github.com/rtk-ai/rtk/commit/7571c8e101c41ee64c51e2bd64697f85f9142423)) +* **init:** remove telemetry info lines from init output ([7dbef2c](https://github.com/rtk-ai/rtk/commit/7dbef2ce00824d26f2057e4c3c76e429e2e23088)) +* **main:** kill zombie processes + path for rtk md ([d16fc6d](https://github.com/rtk-ai/rtk/commit/d16fc6dacbfec912c21522939b15b7bbd9719487)) +* **main:** kill zombie processes + path for rtk md + missing intergrations ([a919335](https://github.com/rtk-ai/rtk/commit/a919335519ed4a5259a212e56407cb312aa99bac)) +* **merge:** changelog conflicts ([d92c5d2](https://github.com/rtk-ai/rtk/commit/d92c5d264a49483c8d6079e04d946a79bc990a74)) +* **proxy:** kill child process on SIGINT/SIGTERM to prevent orphans ([d813919](https://github.com/rtk-ai/rtk/commit/d813919a24546e044e7844fc7ed05fef4ec24033)) +* **proxy:** kill child process on SIGINT/SIGTERM to prevent orphans ([3318510](https://github.com/rtk-ai/rtk/commit/33185101fc122d0c11a25a4e02ac9f3a7dc7e3bb)) +* **review:** address ChildGuard disarm, stdin dedup, hook masking ([d85fe33](https://github.com/rtk-ai/rtk/commit/d85fe3384b87c16fafd25ec7bcadbff6e69f3f1f)) +* **security:** default to ask when no permission rule matches ([#886](https://github.com/rtk-ai/rtk/issues/886)) ([158c745](https://github.com/rtk-ai/rtk/commit/158c74527f6591d372e40a78cd604d73a20649a9)) +* **security:** default to ask when no permission rule matches ([#886](https://github.com/rtk-ai/rtk/issues/886)) ([41a6c6b](https://github.com/rtk-ai/rtk/commit/41a6c6bf6da78a4754794fdc6a1469df2e327920)) +* **tracking:** use std::env::temp_dir() for compatibility (instead of unix tmp) ([e918661](https://github.com/rtk-ai/rtk/commit/e918661440d7b50321f0535032f52c5e87aaf3cb)) + +## [Unreleased] + +### Features + +* **aws:** expand CLI filters from 8 to 25 subcommands — CloudWatch Logs, CloudFormation events, Lambda, IAM, DynamoDB (with type unwrapping), ECS tasks, EC2 security groups, S3API objects, S3 sync/cp, EKS, SQS, Secrets Manager ([#885](https://github.com/rtk-ai/rtk/pull/885)) +* **aws:** add shared runner `run_aws_filtered()` eliminating per-handler boilerplate +* **tee:** add `force_tee_hint()` — truncated output saves full data to file with recovery hint + +## [0.34.3](https://github.com/rtk-ai/rtk/compare/v0.34.2...v0.34.3) (2026-04-02) + + +### Bug Fixes + +* **automod:** add auto discovery for cmds ([234909d](https://github.com/rtk-ai/rtk/commit/234909d2c754ade2fdc939b0a1435a8e34ffc305)) +* **ci:** fix validate-docs.sh broken module count check ([bbe3da6](https://github.com/rtk-ai/rtk/commit/bbe3da642b5fc4b065b13a65647ea0ebf5264e65)) +* **cleaning:** constant extract ([aabc016](https://github.com/rtk-ai/rtk/commit/aabc0167bc013fd2d0c61a687580f6e69305500a)) +* **cmds:** migrate remaining exit_code to exit_code_from_output ([ba9fa34](https://github.com/rtk-ai/rtk/commit/ba9fa345f3d1d14bd0af236ec9aa8a9a0e5581d6)) +* **cmds:** more covering for run_filtered ([e48485a](https://github.com/rtk-ai/rtk/commit/e48485adc6a33d12b70664598020595cf7dfcd7e)) +* **docs:** add documentation ([2f7278a](https://github.com/rtk-ai/rtk/commit/2f7278ac5992bf2e84b763fb05642d89900ba495)) +* **docs:** add maintainers docs ([14265b4](https://github.com/rtk-ai/rtk/commit/14265b48c3a15e459a31da11250a51ab5830a508)) +* **refacto-p1:** unified cmds execution flow (+ rm dead code) ([75bd607](https://github.com/rtk-ai/rtk/commit/75bd607d55235f313855f5fe8c9eceafd73700a7)) +* **refacto-p2:** more standardize ([47a76ea](https://github.com/rtk-ai/rtk/commit/47a76ea35ed2fe02a3600792163f727fa3a94ff2)) +* **refacto-p2:** more standardize ([92c671a](https://github.com/rtk-ai/rtk/commit/92c671a175a5e2bf09720fd1a8591140bcb473a0)) +* **refacto:** wrappers for standardization, exit codes lexer tokenizer, constants, code clean ([bff0258](https://github.com/rtk-ai/rtk/commit/bff02584243f1b73418418b0c05365acf56fbb36)) +* **registry:** quoted env prefix + inline regex cleanup + routing docs ([f3217a4](https://github.com/rtk-ai/rtk/commit/f3217a467b543a3181605b257162f2b3ab5d5df0)) +* **review:** address PR [#910](https://github.com/rtk-ai/rtk/issues/910) review feedback ([0a8b8fd](https://github.com/rtk-ai/rtk/commit/0a8b8fd0693fa504f376146cbbcafe9ddf4632c8)) +* **review:** PR [#934](https://github.com/rtk-ai/rtk/issues/934) ([5bd35a3](https://github.com/rtk-ai/rtk/commit/5bd35a33ad6abe5278749726bed19912664531c2)) +* **review:** PR [#934](https://github.com/rtk-ai/rtk/issues/934) ([bae7930](https://github.com/rtk-ai/rtk/commit/bae79301194bbb48d1cbb39554096c3225f7cb73)) +* **rules:** add wc RtkRule with pattern field for develop compat ([d75e864](https://github.com/rtk-ai/rtk/commit/d75e864f20451a5e17918c75f2ea32672f65e1f4)) +* **standardize:** git+kube sub wrappers run_filtered ([7fd221f](https://github.com/rtk-ai/rtk/commit/7fd221f44660bcf411aa333d2c35a49ff89e7961)) +* **standardize:** merge pattern into rues ([08aabb9](https://github.com/rtk-ai/rtk/commit/08aabb95c3ae6e0b734f696264e1e1a8c0f0b22e)) + ## [0.34.2](https://github.com/rtk-ai/rtk/compare/v0.34.1...v0.34.2) (2026-03-30) diff --git a/CLAUDE.md b/CLAUDE.md index 0dddf14e5..b80dd1307 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,12 +11,12 @@ This is a fork with critical fixes for git argument parsing and modern JavaScrip ### Name Collision Warning **Two different "rtk" projects exist:** -- This project: Rust Token Killer (rtk-ai/rtk) +- This project: Rust Token Killer (algolia/rtk, fork of rtk-ai/rtk) - reachingforthejack/rtk: Rust Type Kit (DIFFERENT - generates Rust types) **Verify correct installation:** ```bash -rtk --version # Should show "rtk 0.28.2" (or newer) +rtk --version # Should show the version from Cargo.toml (rtk 0.38.0-algolia.1 or newer) rtk gain # Should show token savings stats (NOT "command not found") ``` @@ -70,11 +70,13 @@ cargo generate-rpm # RPM package (needs cargo-generate-rpm, after rel rtk uses a **command proxy architecture**: `main.rs` routes CLI commands via a Clap `Commands` enum to specialized filter modules in `src/cmds/*/`, each of which executes the underlying command and compresses its output. Token savings are tracked in SQLite via `src/core/tracking.rs`. For the full architecture, component details, and module development patterns, see: -- [ARCHITECTURE.md](ARCHITECTURE.md) — System design, module organization, filtering strategies, error handling -- [docs/TECHNICAL.md](docs/TECHNICAL.md) — End-to-end flow, folder map, hook system, filter pipeline +- [ARCHITECTURE.md](docs/contributing/ARCHITECTURE.md) — System design, module organization, filtering strategies, error handling +- [docs/contributing/TECHNICAL.md](docs/contributing/TECHNICAL.md) — End-to-end flow, folder map, hook system, filter pipeline Module responsibilities are documented in each folder's `README.md` and each file's `//!` doc header. Browse `src/cmds/*/` to discover available filters. +Supported ecosystems: git/gh/gt, cargo, go/golangci-lint, npm/pnpm/npx, ruff/pytest/pip/mypy, rspec/rubocop/rake, dotnet, playwright/vitest/jest, docker/kubectl/aws. + ### Proxy Mode **Purpose**: Execute commands without filtering but track usage for metrics. @@ -158,6 +160,124 @@ git branch # Verify correct branch (main, feature/*, etc.) - "Should I test Y edge case?" → ASK if not mentioned in requirements - "Should I verify Z across N platforms?" → ASK if N > 2 +## Fork Policy (algolia/rtk) + +This repository is a **fork** of [rtk-ai/rtk](https://github.com/rtk-ai/rtk). +The fork stays as thin as possible: upstream is authoritative, we add the +minimum needed for Algolia. + +### What we change from upstream +- **Telemetry stripped**: no phone-home, no `ureq` / `hostname` deps, + no opt-out config, no `RTK_TELEMETRY_*` env vars +- **No Homebrew tap**: install via `cargo install --git` or `install.sh` +- **Branding**: all install URLs and repo references point to `algolia/rtk` +- **Targeted bug fixes**: only when upstream hasn't addressed the issue. + Each lives under `bug-reports/` with a reproducer. + +### What we DON'T change +- All upstream features, filters, commands, TOML DSL, rewrite engine +- Test suite (must pass) +- Architecture and module structure + +### Upstream Catchup Procedure (re-fork strategy) + +When upstream has new releases to absorb, **do not merge** — re-fork. +Merging accumulates conflict-resolution noise and can silently re-apply +patches that upstream has fixed. The re-fork procedure forces patch +re-validation at every catchup: + +```bash +# 1. Fetch upstream +git remote add upstream https://github.com/rtk-ai/rtk.git # one-time +git fetch upstream + +# 2. Branch off upstream/master directly +git checkout -b fork/upstream-realign-vX.Y.Z upstream/master + +# 3. Strip telemetry (always required) +# - Delete src/core/telemetry.rs, src/core/telemetry_cmd.rs +# - Remove `pub mod telemetry;` and `pub mod telemetry_cmd;` from +# src/core/mod.rs +# - Remove `core::telemetry::maybe_ping();` from main.rs +# - Remove the `Telemetry` Commands enum variant + match arm in main.rs +# - Remove TelemetryConfig + tests from src/core/config.rs +# - Remove prompt_telemetry_consent / save_telemetry_consent from +# src/hooks/init.rs and the call site +# - Remove `ureq` (and `hostname` if present) from Cargo.toml +# - Remove RTK_TELEMETRY_URL/TOKEN env vars from .github/workflows/release.yml +# - Delete docs/TELEMETRY.md, docs/guide/resources/telemetry.md +# - Strip Privacy & Telemetry sections from README*.md and other docs + +# 4. Fork hygiene rebrand (see below) +# sed-based rewrite of rtk-ai/rtk → algolia/rtk in install instructions, +# rtk-ai.app links, master → main where appropriate + +# 5. Re-test bug-reports/*.md against the new base. +# Upstream fixes most issues over time — re-apply ONLY patches still +# needed. Each preserved patch should reference its bug-report file +# in the commit message. + +# 6. Bump version in Cargo.toml: 0.X.Y → 0.X.Y-algolia.1 +# Also update .release-please-manifest.json + +# 7. Build + test +cargo fmt --all && cargo clippy --all-targets && cargo test --all + +# 8. Open a PR from fork/upstream-realign-vX.Y.Z against main. +# The PR diff against main is large (300+ commits absorbed), but the +# diff against upstream/master is small (telemetry strip + 2-3 +# targeted patches + rebrand). Reviewers should review it the second +# way: `git diff upstream/master..fork/upstream-realign-vX.Y.Z`. + +# 9. Once approved, the merge replaces main. Force-push if your team's +# convention requires linear history; merge commit if not. +``` + +**Key principle**: every catchup re-validates every patch. Patches that +upstream silently fixed get dropped automatically because the re-fork +starts from a clean upstream tree. + +## Fork Hygiene (Mandatory) + +This is the **Algolia fork** (`algolia/rtk`), not the upstream (`rtk-ai/rtk`). +Upstream references leak in during rebases and releases. **Every rebase +and every release MUST include a fork hygiene check.** + +### Pre-commit Checklist (after rebase or before release) + +Run this grep to catch upstream leaks: + +```bash +rg -i 'brew install rtk[^-]|rtk-ai\.app|contact@rtk|"rtk 0\.\d+\.\d+"' --glob '*.md' --glob '*.rb' +``` + +**Zero matches required.** Any hit must be fixed before commit. Historical +references in `CHANGELOG.md` (auto-generated from upstream commits) are +the one exception. + +### Banned Patterns in User-Facing Docs + +| Pattern | Why | Replace With | +|---------|-----|--------------| +| `brew install rtk` | No Homebrew tap for fork | `cargo install --git https://github.com/algolia/rtk` or `curl \| sh` | +| `https://www.rtk-ai.app` | Upstream website | Remove or use `https://github.com/algolia/rtk` | +| `contact@rtk-ai.app` | Upstream email | `#proj-internal-skills` on Slack | +| `rtk-ai/rtk` in install instructions | Upstream repo | `algolia/rtk` | +| `brew uninstall rtk` | No Homebrew install exists | `cargo uninstall rtk` | +| Hardcoded version strings (`"rtk 0.28.2"`) | Goes stale on every release | Use current `Cargo.toml` version | + +### Where to Check + +All `README*.md`, `INSTALL.md`, `CLAUDE.md`, `openclaw/README.md`, +`Formula/rtk.rb`, GitHub repo metadata (`gh repo edit --homepage`). + +### On Release + +1. Run the grep above — fix any matches +2. Update version strings in docs to match `Cargo.toml` +3. Verify `gh repo view --json homepageUrl` returns the algolia URL + (not upstream) + ## Plan Execution Protocol When user provides a numbered plan (QW1-QW4, Phase 1-5, sprint tasks, etc.): diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 08e5ec0ac..6cd87369b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -7,7 +7,7 @@ - [Report an Issue](../../issues/new) - [Open Pull Requests](../../pulls) - [Start a Discussion](../../discussions) -- [Technical Documentation](docs/TECHNICAL.md) — Architecture, end-to-end flow, folder map, how to write tests +- [Technical Documentation](docs/contributing/TECHNICAL.md) — Architecture, end-to-end flow, folder map, how to write tests --- @@ -107,15 +107,50 @@ For the step-by-step checklist (create filter, register rewrite pattern, registe --- +## Commit Messages & Changelog + +RTK uses [Conventional Commits](https://www.conventionalcommits.org/) and [release-please](https://github.com/googleapis/release-please) to **auto-generate CHANGELOG.md, version bumps, and GitHub releases**. Never edit `CHANGELOG.md` manually — it is fully managed by release-please from your commit messages. + +### Commit format + +``` +(): +``` + +| Type | Semver Impact | When to Use | +|------|---------------|-------------| +| `feat` | Minor | New features, new filters, new command support | +| `fix` | Patch | Bug fixes, corrections | +| `perf` | Patch | Performance improvements | +| `refactor` | — | Code restructuring (no changelog entry) | +| `docs` | — | Documentation only | +| `chore` | — | Maintenance, CI, deps | +| `feat!` / `fix!` | Major | Breaking changes (add `!` after type) | + +**Scope** should match the module or area: `git`, `cargo`, `gh`, `hook`, `tracking`, `cicd`, etc. + +### Examples + +``` +feat(kubectl): add pod log filtering +fix(git): preserve merge commit messages in log filter +perf(cargo): lazy-compile clippy regex patterns +feat!(hook): change rewrite config format +``` + +These commit messages directly become CHANGELOG entries when release-please creates a release PR. Write them as if they will be read by users. + +--- + ## Branch Naming Convention Git branch names cannot include spaces or colons, so we use slash-prefixed names. Pick the prefix that matches your change type and follow it with an optional scope and a short, kebab-case description. -| Prefix | Semver Impact | When to Use | -|--------|---------------|-------------| -| `fix/` | Patch | Bug fixes, corrections, minor adjustments | -| `feat/` | Minor | New features, new filters, new command support | -| `chore/` | Major | Breaking changes, API changes, removed functionality | +| Prefix | When to Use | +|--------|-------------| +| `fix/` | Bug fixes, corrections, minor adjustments | +| `feat/` | New features, new filters, new command support | +| `chore/` | CI/CD, deps, maintenance, breaking changes | Combine the prefix with a scope if it adds clarity (e.g. `git`, `kubectl`, `filter`, `tracking`, `config`) and finish with a descriptive slug: `fix/-` or `feat/`. @@ -137,7 +172,7 @@ chore/release-pipeline-cleanup **For large features or refactors**, prefer multi-part PRs over one enormous PR. Split the work into logical, reviewable chunks that can each be merged independently. Examples: - feat(Part 1): Add data model and tests - feat(Part 2): Add CLI command and integration -- feat(Part 3): Update documentation and CHANGELOG +- feat(Part 3): Update documentation **Why**: Small, focused PRs are easier to review, safer to merge, and faster to ship. Large PRs slow down review, hide bugs, and increase merge conflict risk. @@ -166,7 +201,7 @@ Every change **must** include tests. See [Testing](#testing) below. ### 4. Add Documentation -Every change **must** include documentation updates. See [Documentation](#documentation) below. +Documentation updates are required for new filters, new features, and changes that affect already-documented behavior. Bug fixes and refactors typically don't need doc updates. See [Documentation](#documentation) below. ### Contributor License Agreement (CLA) @@ -203,7 +238,7 @@ your branch --> develop (review + CI + integration testing) --> version branch - Every change **must** include tests. We follow **TDD (Red-Green-Refactor)**: write a failing test first, implement the minimum to pass, then refactor. -For how to write tests (fixtures, snapshots, token savings verification), see [docs/TECHNICAL.md — Testing](docs/TECHNICAL.md#testing). +For how to write tests (fixtures, snapshots, token savings verification), see [docs/contributing/TECHNICAL.md — Testing](docs/contributing/TECHNICAL.md#testing). ### Test Types @@ -235,19 +270,20 @@ cargo fmt --all --check && cargo clippy --all-targets && cargo test ## Documentation -Every change **must** include documentation updates. Use this table to find which docs to update: +Documentation updates are required for new filters, new features, and changes that affect already-documented behavior. Use this table to find which docs to update: | What you changed | Update these docs | |------------------|-------------------| -| New Rust filter (`src/cmds/`) | Ecosystem `README.md` (e.g., `src/cmds/git/README.md`), [README.md](README.md) command list, [CHANGELOG.md](CHANGELOG.md) | -| New TOML filter (`src/filters/`) | [src/filters/README.md](src/filters/README.md) if naming conventions change, [README.md](README.md) command list, [CHANGELOG.md](CHANGELOG.md) | +| New Rust filter (`src/cmds/`) | Ecosystem `README.md` (e.g., `src/cmds/git/README.md`), [README.md](README.md) command list | +| New TOML filter (`src/filters/`) | [src/filters/README.md](src/filters/README.md) if naming conventions change, [README.md](README.md) command list | | New rewrite pattern | `src/discover/rules.rs` — see [Adding a New Command Filter](src/cmds/README.md#adding-a-new-command-filter) | -| Core infrastructure (`src/core/`) | [src/core/README.md](src/core/README.md), [docs/TECHNICAL.md](docs/TECHNICAL.md) if flow changes | +| Core infrastructure (`src/core/`) | [src/core/README.md](src/core/README.md), [docs/contributing/TECHNICAL.md](docs/contributing/TECHNICAL.md) if flow changes | | Hook system (`src/hooks/`) | [src/hooks/README.md](src/hooks/README.md), [hooks/README.md](hooks/README.md) for agent-facing docs | -| Architecture or design change | [ARCHITECTURE.md](ARCHITECTURE.md), [docs/TECHNICAL.md](docs/TECHNICAL.md) | -| Bug fix or breaking change | [CHANGELOG.md](CHANGELOG.md) | +| Architecture or design change | [ARCHITECTURE.md](docs/contributing/ARCHITECTURE.md), [docs/contributing/TECHNICAL.md](docs/contributing/TECHNICAL.md) | + +> **Note**: Do NOT edit `CHANGELOG.md` manually — it is auto-generated by [release-please](https://github.com/googleapis/release-please) from your commit messages. See [Commit Messages & Changelog](#commit-messages--changelog). -**Navigation**: [CONTRIBUTING.md](CONTRIBUTING.md) (you are here) → [docs/TECHNICAL.md](docs/TECHNICAL.md) (architecture + flow) → each folder's `README.md` (implementation details). +**Navigation**: [CONTRIBUTING.md](CONTRIBUTING.md) (you are here) → [docs/contributing/TECHNICAL.md](docs/contributing/TECHNICAL.md) (architecture + flow) → each folder's `README.md` (implementation details). Keep documentation concise and practical -- examples over explanations. diff --git a/Cargo.lock b/Cargo.lock index 49b7c1aef..1a15fce9b 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -101,10 +101,15 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8" [[package]] -name = "base64" -version = "0.22.1" +name = "automod" +version = "1.0.16" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6" +checksum = "e8b5778837666541195063243828c5b6139221b47dc4ec3ba81738e532469ab1" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] [[package]] name = "bitflags" @@ -312,17 +317,6 @@ dependencies = [ "windows-sys 0.48.0", ] -[[package]] -name = "displaydoc" -version = "0.2.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "97369cbbc041bc366949bc74d34658d6cda5621039731c6310521892a3a20ae0" -dependencies = [ - "proc-macro2", - "quote", - "syn", -] - [[package]] name = "env_home" version = "0.1.0" @@ -385,15 +379,6 @@ version = "0.1.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2" -[[package]] -name = "form_urlencoded" -version = "1.2.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "cb4cb245038516f5f85277875cdaa4f7d2c9a0fa0468de06ed190163b1581fcf" -dependencies = [ - "percent-encoding", -] - [[package]] name = "generic-array" version = "0.14.7" @@ -480,17 +465,6 @@ version = "0.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea" -[[package]] -name = "hostname" -version = "0.4.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "617aaa3557aef3810a6369d0a99fac8a080891b68bd9f9812a1eeda0c0730cbd" -dependencies = [ - "cfg-if", - "libc", - "windows-link", -] - [[package]] name = "iana-time-zone" version = "0.1.65" @@ -515,114 +489,12 @@ dependencies = [ "cc", ] -[[package]] -name = "icu_collections" -version = "2.1.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4c6b649701667bbe825c3b7e6388cb521c23d88644678e83c0c4d0a621a34b43" -dependencies = [ - "displaydoc", - "potential_utf", - "yoke", - "zerofrom", - "zerovec", -] - -[[package]] -name = "icu_locale_core" -version = "2.1.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "edba7861004dd3714265b4db54a3c390e880ab658fec5f7db895fae2046b5bb6" -dependencies = [ - "displaydoc", - "litemap", - "tinystr", - "writeable", - "zerovec", -] - -[[package]] -name = "icu_normalizer" -version = "2.1.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5f6c8828b67bf8908d82127b2054ea1b4427ff0230ee9141c54251934ab1b599" -dependencies = [ - "icu_collections", - "icu_normalizer_data", - "icu_properties", - "icu_provider", - "smallvec", - "zerovec", -] - -[[package]] -name = "icu_normalizer_data" -version = "2.1.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7aedcccd01fc5fe81e6b489c15b247b8b0690feb23304303a9e560f37efc560a" - -[[package]] -name = "icu_properties" -version = "2.1.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "020bfc02fe870ec3a66d93e677ccca0562506e5872c650f893269e08615d74ec" -dependencies = [ - "icu_collections", - "icu_locale_core", - "icu_properties_data", - "icu_provider", - "zerotrie", - "zerovec", -] - -[[package]] -name = "icu_properties_data" -version = "2.1.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "616c294cf8d725c6afcd8f55abc17c56464ef6211f9ed59cccffe534129c77af" - -[[package]] -name = "icu_provider" -version = "2.1.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "85962cf0ce02e1e0a629cc34e7ca3e373ce20dda4c4d7294bbd0bf1fdb59e614" -dependencies = [ - "displaydoc", - "icu_locale_core", - "writeable", - "yoke", - "zerofrom", - "zerotrie", - "zerovec", -] - [[package]] name = "id-arena" version = "2.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954" -[[package]] -name = "idna" -version = "1.1.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3b0875f23caa03898994f6ddc501886a45c7d3d62d04d2d90788d47be1b1e4de" -dependencies = [ - "idna_adapter", - "smallvec", - "utf8_iter", -] - -[[package]] -name = "idna_adapter" -version = "1.2.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3acae9609540aa318d1bc588455225fb2085b9ed0c4f6bd0d9d5bcd86f1a0344" -dependencies = [ - "icu_normalizer", - "icu_properties", -] - [[package]] name = "ignore" version = "0.4.25" @@ -717,12 +589,6 @@ version = "0.12.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53" -[[package]] -name = "litemap" -version = "0.8.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6373607a59f0be73a39b6fe456b8192fcc3585f602af20751600e974dd455e77" - [[package]] name = "log" version = "0.4.29" @@ -772,27 +638,12 @@ version = "0.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "04744f49eae99ab78e0d5c0b603ab218f515ea8cfe5a456d7629ad883a3b6e7d" -[[package]] -name = "percent-encoding" -version = "2.3.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220" - [[package]] name = "pkg-config" version = "0.3.32" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7edddbd0b52d732b21ad9a5fab5c704c14cd949e5e9a1ec5929a24fded1b904c" -[[package]] -name = "potential_utf" -version = "0.1.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b73949432f5e2a09657003c25bca5e19a0e9c84f8058ca374f49e0ebe605af77" -dependencies = [ - "zerovec", -] - [[package]] name = "prettyplease" version = "0.2.37" @@ -876,34 +727,21 @@ version = "0.8.10" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "dc897dd8d9e8bd1ed8cdad82b5966c3e0ecae09fb1907d58efaa013543185d0a" -[[package]] -name = "ring" -version = "0.17.14" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a4689e6c2294d81e88dc6261c768b63bc4fcdb852be6d1352498b114f61383b7" -dependencies = [ - "cc", - "cfg-if", - "getrandom 0.2.17", - "libc", - "untrusted", - "windows-sys 0.52.0", -] - [[package]] name = "rtk" -version = "0.34.2" +version = "0.38.0-algolia.1" dependencies = [ "anyhow", + "automod", "chrono", "clap", "colored", "dirs", "flate2", "getrandom 0.4.2", - "hostname", "ignore", "lazy_static", + "libc", "quick-xml", "regex", "rusqlite", @@ -911,9 +749,7 @@ dependencies = [ "serde_json", "sha2", "tempfile", - "thiserror", "toml", - "ureq", "walkdir", "which", ] @@ -945,41 +781,6 @@ dependencies = [ "windows-sys 0.61.2", ] -[[package]] -name = "rustls" -version = "0.23.37" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "758025cb5fccfd3bc2fd74708fd4682be41d99e5dff73c377c0646c6012c73a4" -dependencies = [ - "log", - "once_cell", - "ring", - "rustls-pki-types", - "rustls-webpki", - "subtle", - "zeroize", -] - -[[package]] -name = "rustls-pki-types" -version = "1.14.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "be040f8b0a225e40375822a563fa9524378b9d63112f53e19ffff34df5d33fdd" -dependencies = [ - "zeroize", -] - -[[package]] -name = "rustls-webpki" -version = "0.103.9" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d7df23109aa6c1567d1c575b9952556388da57401e4ace1d15f79eedad0d8f53" -dependencies = [ - "ring", - "rustls-pki-types", - "untrusted", -] - [[package]] name = "rustversion" version = "1.0.22" @@ -1083,24 +884,12 @@ version = "1.15.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "67b1b7a3b5fe4f1376887184045fcf45c69e92af734b7aaddc05fb777b6fbd03" -[[package]] -name = "stable_deref_trait" -version = "1.2.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596" - [[package]] name = "strsim" version = "0.11.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f" -[[package]] -name = "subtle" -version = "2.6.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292" - [[package]] name = "syn" version = "2.0.117" @@ -1112,17 +901,6 @@ dependencies = [ "unicode-ident", ] -[[package]] -name = "synstructure" -version = "0.13.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "728a70f3dbaf5bab7f0c4b1ac8d7ae5ea60a4b5549c8a5914361c99147a709d2" -dependencies = [ - "proc-macro2", - "quote", - "syn", -] - [[package]] name = "tempfile" version = "3.26.0" @@ -1156,16 +934,6 @@ dependencies = [ "syn", ] -[[package]] -name = "tinystr" -version = "0.8.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "42d3e9c45c09de15d06dd8acf5f4e0e399e85927b7f00711024eb7ae10fa4869" -dependencies = [ - "displaydoc", - "zerovec", -] - [[package]] name = "toml" version = "0.8.23" @@ -1225,46 +993,6 @@ version = "0.2.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853" -[[package]] -name = "untrusted" -version = "0.9.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8ecb6da28b8a351d773b68d5825ac39017e680750f980f3a1a85cd8dd28a47c1" - -[[package]] -name = "ureq" -version = "2.12.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "02d1a66277ed75f640d608235660df48c8e3c19f3b4edb6a263315626cc3c01d" -dependencies = [ - "base64", - "flate2", - "log", - "once_cell", - "rustls", - "rustls-pki-types", - "url", - "webpki-roots 0.26.11", -] - -[[package]] -name = "url" -version = "2.5.8" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ff67a8a4397373c3ef660812acab3268222035010ab8680ec4215f38ba3d0eed" -dependencies = [ - "form_urlencoded", - "idna", - "percent-encoding", - "serde", -] - -[[package]] -name = "utf8_iter" -version = "1.0.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be" - [[package]] name = "utf8parse" version = "0.2.2" @@ -1396,24 +1124,6 @@ dependencies = [ "semver", ] -[[package]] -name = "webpki-roots" -version = "0.26.11" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "521bc38abb08001b01866da9f51eb7c5d647a19260e00054a8c7fd5f9e57f7a9" -dependencies = [ - "webpki-roots 1.0.6", -] - -[[package]] -name = "webpki-roots" -version = "1.0.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "22cfaf3c063993ff62e73cb4311efde4db1efb31ab78a3e5c457939ad5cc0bed" -dependencies = [ - "rustls-pki-types", -] - [[package]] name = "which" version = "8.0.1" @@ -1502,15 +1212,6 @@ dependencies = [ "windows-targets 0.48.5", ] -[[package]] -name = "windows-sys" -version = "0.52.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "282be5f36a8ce781fad8c8ae18fa3f9beff57ec1b52cb3de0789201425d9a33d" -dependencies = [ - "windows-targets 0.52.6", -] - [[package]] name = "windows-sys" version = "0.59.0" @@ -1753,35 +1454,6 @@ dependencies = [ "wasmparser", ] -[[package]] -name = "writeable" -version = "0.6.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9edde0db4769d2dc68579893f2306b26c6ecfbe0ef499b013d731b7b9247e0b9" - -[[package]] -name = "yoke" -version = "0.8.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "72d6e5c6afb84d73944e5cedb052c4680d5657337201555f9f2a16b7406d4954" -dependencies = [ - "stable_deref_trait", - "yoke-derive", - "zerofrom", -] - -[[package]] -name = "yoke-derive" -version = "0.8.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b659052874eb698efe5b9e8cf382204678a0086ebf46982b79d6ca3182927e5d" -dependencies = [ - "proc-macro2", - "quote", - "syn", - "synstructure", -] - [[package]] name = "zerocopy" version = "0.8.40" @@ -1802,66 +1474,6 @@ dependencies = [ "syn", ] -[[package]] -name = "zerofrom" -version = "0.1.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "50cc42e0333e05660c3587f3bf9d0478688e15d870fab3346451ce7f8c9fbea5" -dependencies = [ - "zerofrom-derive", -] - -[[package]] -name = "zerofrom-derive" -version = "0.1.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d71e5d6e06ab090c67b5e44993ec16b72dcbaabc526db883a360057678b48502" -dependencies = [ - "proc-macro2", - "quote", - "syn", - "synstructure", -] - -[[package]] -name = "zeroize" -version = "1.8.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b97154e67e32c85465826e8bcc1c59429aaaf107c1e4a9e53c8d8ccd5eff88d0" - -[[package]] -name = "zerotrie" -version = "0.2.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2a59c17a5562d507e4b54960e8569ebee33bee890c70aa3fe7b97e85a9fd7851" -dependencies = [ - "displaydoc", - "yoke", - "zerofrom", -] - -[[package]] -name = "zerovec" -version = "0.11.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6c28719294829477f525be0186d13efa9a3c602f7ec202ca9e353d310fb9a002" -dependencies = [ - "yoke", - "zerofrom", - "zerovec-derive", -] - -[[package]] -name = "zerovec-derive" -version = "0.11.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "eadce39539ca5cb3985590102671f2567e659fca9666581ad3411d59207951f3" -dependencies = [ - "proc-macro2", - "quote", - "syn", -] - [[package]] name = "zmij" version = "1.0.21" diff --git a/Cargo.toml b/Cargo.toml index 2c672aef3..b774553e7 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,12 +1,12 @@ [package] name = "rtk" -version = "0.34.2" +version = "0.38.0-algolia.1" edition = "2021" authors = ["Patrick Szymkowiak"] description = "Rust Token Killer - High-performance CLI proxy to minimize LLM token consumption" license = "MIT" -homepage = "https://www.rtk-ai.app" -repository = "https://github.com/rtk-ai/rtk" +homepage = "https://github.com/algolia/rtk" +repository = "https://github.com/algolia/rtk" readme = "README.md" keywords = ["cli", "llm", "token", "filter", "productivity"] categories = ["command-line-utilities", "development-tools"] @@ -25,15 +25,16 @@ dirs = "5" rusqlite = { version = "0.31", features = ["bundled"] } toml = "0.8" chrono = "0.4" -thiserror = "1.0" tempfile = "3" sha2 = "0.10" -ureq = "2" -hostname = "0.4" getrandom = "0.4" flate2 = "1.0" quick-xml = "0.37" which = "8" +automod = "1" + +[target.'cfg(unix)'.dependencies] +libc = "0.2" [build-dependencies] toml = "0.8" @@ -58,7 +59,6 @@ priority = "optional" assets = [ ["target/release/rtk", "usr/bin/", "755"], ] - # cargo-generate-rpm configuration [package.metadata.generate-rpm] assets = [ diff --git a/DISCLAIMER.md b/DISCLAIMER.md new file mode 100644 index 000000000..af81e1432 --- /dev/null +++ b/DISCLAIMER.md @@ -0,0 +1,25 @@ +# Disclaimer + +## No Warranty + +This software is provided "AS IS", without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose, and non-infringement. The entire risk as to the quality and performance of the software is with you. + +## Limitation of Liability + +In no event shall the authors or copyright holders be liable for any claim, damages, or other liability, whether in an action of contract, tort, or otherwise, arising from, out of, or in connection with the software or the use or other dealings in the software. This includes, without limitation, any direct, indirect, incidental, special, exemplary, or consequential damages (including but not limited to loss of data, loss of profits, or business interruption). + +## Precompiled Binaries + +Precompiled binaries are provided solely for convenience and are covered by the same license as the source code (Apache License 2.0). They are provided without warranties or conditions of any kind. You are responsible for verifying the integrity and suitability of any binary before use. Always verify checksums when available. + +## Third-Party Dependencies + +This software incorporates third-party open-source components, each governed by their respective licenses. The authors make no representations or warranties regarding these dependencies and accept no liability for any issues arising from their use. + +## Use at Your Own Risk + +This software interacts with your development environment, file system, and external commands. It is your responsibility to ensure that its use is appropriate for your environment and complies with any applicable policies, regulations, or agreements. The authors are not responsible for any unintended side effects resulting from its use. + +--- + +See [LICENSE](LICENSE) for the full terms of the Apache License 2.0 under which this software is distributed. diff --git a/Formula/rtk.rb b/Formula/rtk.rb index 25a8b2e8a..d39a25b75 100644 --- a/Formula/rtk.rb +++ b/Formula/rtk.rb @@ -1,34 +1,35 @@ # typed: false # frozen_string_literal: true -# Homebrew formula for rtk - Rust Token Killer -# To install: brew tap rtk-ai/tap && brew install rtk +# Homebrew formula for rtk - Rust Token Killer (algolia fork) +# This formula is shipped only for parity with upstream — the algolia +# fork has no Homebrew tap. Install via `cargo install --git` instead. class Rtk < Formula desc "High-performance CLI proxy to minimize LLM token consumption" - homepage "https://www.rtk-ai.app" + homepage "https://github.com/algolia/rtk" version "0.1.0" license "MIT" on_macos do on_intel do - url "https://github.com/rtk-ai/rtk/releases/download/v#{version}/rtk-x86_64-apple-darwin.tar.gz" + url "https://github.com/algolia/rtk/releases/download/v#{version}/rtk-x86_64-apple-darwin.tar.gz" sha256 "PLACEHOLDER_SHA256_INTEL" end on_arm do - url "https://github.com/rtk-ai/rtk/releases/download/v#{version}/rtk-aarch64-apple-darwin.tar.gz" + url "https://github.com/algolia/rtk/releases/download/v#{version}/rtk-aarch64-apple-darwin.tar.gz" sha256 "PLACEHOLDER_SHA256_ARM" end end on_linux do on_intel do - url "https://github.com/rtk-ai/rtk/releases/download/v#{version}/rtk-x86_64-unknown-linux-gnu.tar.gz" + url "https://github.com/algolia/rtk/releases/download/v#{version}/rtk-x86_64-unknown-linux-gnu.tar.gz" sha256 "PLACEHOLDER_SHA256_LINUX_INTEL" end on_arm do - url "https://github.com/rtk-ai/rtk/releases/download/v#{version}/rtk-aarch64-unknown-linux-gnu.tar.gz" + url "https://github.com/algolia/rtk/releases/download/v#{version}/rtk-aarch64-unknown-linux-gnu.tar.gz" sha256 "PLACEHOLDER_SHA256_LINUX_ARM" end end diff --git a/INSTALL.md b/INSTALL.md index 98457d09a..d35fa0da0 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -5,7 +5,7 @@ **There are TWO completely different projects named "rtk":** 1. ✅ **Rust Token Killer** (this project) - LLM token optimizer - - Repos: `rtk-ai/rtk` + - Repos: `algolia/rtk` - Has `rtk gain` command for token savings stats 2. ❌ **Rust Type Kit** (reachingforthejack/rtk) - DIFFERENT PROJECT @@ -44,7 +44,7 @@ cargo uninstall rtk ### Quick Install (Linux/macOS) ```bash -curl -fsSL https://raw.githubusercontent.com/rtk-ai/rtk/master/install.sh | sh +curl -fsSL https://raw.githubusercontent.com/algolia/rtk/main/install.sh | sh ``` After installation, **verify you have the correct rtk**: @@ -55,8 +55,8 @@ rtk gain # Must show token savings stats (not "command not found") ### Alternative: Manual Installation ```bash -# From rtk-ai repository (NOT reachingforthejack!) -cargo install --git https://github.com/rtk-ai/rtk +# From algolia repository (NOT reachingforthejack!) +cargo install --git https://github.com/algolia/rtk # OR (if published and correct on crates.io) cargo install rtk @@ -179,7 +179,7 @@ rtk init --show ### First-Time User (Recommended) ```bash # 1. Install RTK -cargo install --git https://github.com/rtk-ai/rtk +cargo install --git https://github.com/algolia/rtk rtk gain # Verify (must show token stats) # 2. Setup with prompts @@ -231,11 +231,11 @@ rtk ls . # Test with git rtk git status -# Test with pnpm (fork only) +# Test with pnpm rtk pnpm list -# Test with Vitest (feat/vitest-support branch only) -rtk vitest run +# Test with Vitest +rtk vitest ``` ## Uninstalling @@ -303,8 +303,15 @@ rtk pnpm install pkg # Silent installation ### Tests ```bash -rtk test cargo test # Failures only (-90%) -rtk vitest run # Filtered Vitest output (-99.6%) +rtk cargo test # Filtered Cargo test output (-90%) +rtk go test # Filtered Go tests (NDJSON, -90%) +rtk jest # Filtered Jest output (-99.6%) +rtk vitest # Filtered Vitest output (-99.6%) +rtk playwright test # Filtered Playwright output (-94%) +rtk pytest # Filtered Python tests (-90%) +rtk rake test # Filtered Ruby tests (-90%) +rtk rspec # Filtered RSpec tests (-60%) +rtk test # Generic test wrapper - failures only (-90%) ``` ### Statistics @@ -319,7 +326,7 @@ rtk gain --history # With command history ### Production T3 Stack Project | Operation | Standard | RTK | Reduction | |-----------|----------|-----|-----------| -| `vitest run` | 102,199 chars | 377 chars | **-99.6%** | +| `vitest` | 102,199 chars | 377 chars | **-99.6%** | | `git status` | 529 chars | 217 chars | **-59%** | | `pnpm list` | ~8,000 tokens | ~2,400 | **-70%** | | `pnpm outdated` | ~12,000 tokens | ~1,200-2,400 | **-80-90%** | @@ -369,11 +376,11 @@ cargo install --path . --force ## Support and Contributing -- **Website**: https://www.rtk-ai.app -- **Contact**: contact@rtk-ai.app +- **Website**: https://github.com/algolia/rtk +- **Contact**: #proj-internal-skills on Slack - **Troubleshooting**: See [TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md) for common issues -- **GitHub issues**: https://github.com/rtk-ai/rtk/issues -- **Pull Requests**: https://github.com/rtk-ai/rtk/pulls +- **GitHub issues**: https://github.com/algolia/rtk/issues +- **Pull Requests**: https://github.com/algolia/rtk/pulls ⚠️ **If you installed the wrong rtk (Type Kit)**, see [TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md#problem-rtk-gain-command-not-found) diff --git a/README.md b/README.md index 05f77de1a..5c6cd087f 100644 --- a/README.md +++ b/README.md @@ -7,18 +7,16 @@

- CI - Release + CI + Release License: MIT Discord - Homebrew

- Website • Install • - Troubleshooting • - Architecture • + Troubleshooting • + Architecture • Discord

@@ -57,16 +55,12 @@ rtk filters and compresses command outputs before they reach your LLM context. S ## Installation -### Homebrew (recommended) - -```bash -brew install rtk -``` +> The Algolia fork has no Homebrew tap. Use one of the methods below. ### Quick Install (Linux/macOS) ```bash -curl -fsSL https://raw.githubusercontent.com/rtk-ai/rtk/refs/heads/master/install.sh | sh +curl -fsSL https://raw.githubusercontent.com/algolia/rtk/refs/heads/main/install.sh | sh ``` > Installs to `~/.local/bin`. Add to PATH if needed: @@ -77,20 +71,22 @@ curl -fsSL https://raw.githubusercontent.com/rtk-ai/rtk/refs/heads/master/instal ### Cargo ```bash -cargo install --git https://github.com/rtk-ai/rtk +cargo install --git https://github.com/algolia/rtk ``` ### Pre-built Binaries -Download from [releases](https://github.com/rtk-ai/rtk/releases): +Download from [releases](https://github.com/algolia/rtk/releases): - macOS: `rtk-x86_64-apple-darwin.tar.gz` / `rtk-aarch64-apple-darwin.tar.gz` - Linux: `rtk-x86_64-unknown-linux-musl.tar.gz` / `rtk-aarch64-unknown-linux-gnu.tar.gz` - Windows: `rtk-x86_64-pc-windows-msvc.zip` +> **Windows users**: Extract the zip and place `rtk.exe` somewhere in your PATH (e.g. `C:\Users\\.local\bin`). Run RTK from **Command Prompt**, **PowerShell**, or **Windows Terminal** — do not double-click the `.exe` (it will flash and close). For the best experience, use [WSL](https://learn.microsoft.com/en-us/windows/wsl/install) where the full hook system works natively. See [Windows setup](#windows) below for details. + ### Verify Installation ```bash -rtk --version # Should show "rtk 0.28.2" +rtk --version # Should show the version from Cargo.toml (rtk 0.38.0-algolia.1 or newer) rtk gain # Should show token savings stats ``` @@ -106,6 +102,8 @@ rtk init -g --codex # Codex (OpenAI) rtk init -g --agent cursor # Cursor rtk init --agent windsurf # Windsurf rtk init --agent cline # Cline / Roo Code +rtk init --agent kilocode # Kilo Code +rtk init --agent antigravity # Google Antigravity # 2. Restart your AI tool, then test git status # Automatically rewritten to rtk git status @@ -167,15 +165,16 @@ rtk gh run list # Workflow run status ### Test Runners ```bash -rtk test cargo test # Show failures only (-90%) -rtk err npm run build # Errors/warnings only -rtk vitest run # Vitest compact (failures only) +rtk jest # Jest compact (failures only) +rtk vitest # Vitest compact (failures only) rtk playwright test # E2E results (failures only) rtk pytest # Python tests (-90%) rtk go test # Go tests (NDJSON, -90%) rtk cargo test # Cargo tests (-90%) rtk rake test # Ruby minitest (-90%) rtk rspec # RSpec tests (JSON, -60%+) +rtk err # Filter errors only from any command +rtk test # Generic test wrapper - failures only (-90%) ``` ### Build & Lint @@ -201,6 +200,18 @@ rtk bundle install # Ruby gems (strip Using lines) rtk prisma generate # Schema generation (no ASCII art) ``` +### AWS +```bash +rtk aws sts get-caller-identity # One-line identity +rtk aws ec2 describe-instances # Compact instance list +rtk aws lambda list-functions # Name/runtime/memory (strips secrets) +rtk aws logs get-log-events # Timestamped messages only +rtk aws cloudformation describe-stack-events # Failures first +rtk aws dynamodb scan # Unwraps type annotations +rtk aws iam list-roles # Strips policy documents +rtk aws s3 ls # Truncated with tee recovery +``` + ### Containers ```bash rtk docker ps # Compact container list @@ -218,7 +229,7 @@ rtk json config.json # Structure without values rtk deps # Dependencies summary rtk env -f AWS # Filtered env vars rtk log app.log # Deduplicated logs -rtk curl # Auto-detect JSON + schema +rtk curl # Truncate + save full output rtk wget # Download, strip progress bars rtk summary # Heuristic summary rtk proxy # Raw passthrough + tracking @@ -294,159 +305,78 @@ rtk init --show # Verify installation After install, **restart Claude Code**. -## Supported AI Tools - -RTK supports 10 AI coding tools. Each integration transparently rewrites shell commands to `rtk` equivalents for 60-90% token savings. - -| Tool | Install | Method | -|------|---------|--------| -| **Claude Code** | `rtk init -g` | PreToolUse hook (bash) | -| **GitHub Copilot (VS Code)** | `rtk init -g --copilot` | PreToolUse hook (`rtk hook copilot`) — transparent rewrite | -| **GitHub Copilot CLI** | `rtk init -g --copilot` | PreToolUse deny-with-suggestion (CLI limitation) | -| **Cursor** | `rtk init -g --agent cursor` | preToolUse hook (hooks.json) | -| **Gemini CLI** | `rtk init -g --gemini` | BeforeTool hook (`rtk hook gemini`) | -| **Codex** | `rtk init -g --codex` | AGENTS.md + RTK.md instructions | -| **Windsurf** | `rtk init --agent windsurf` | .windsurfrules (project-scoped) | -| **Cline / Roo Code** | `rtk init --agent cline` | .clinerules (project-scoped) | -| **OpenCode** | `rtk init -g --opencode` | Plugin TS (tool.execute.before) | -| **OpenClaw** | `openclaw plugins install ./openclaw` | Plugin TS (before_tool_call) | -| **Mistral Vibe** | Planned (#800) | Blocked on upstream BeforeToolCallback | - -### Claude Code (default) - -```bash -rtk init -g # Install hook + RTK.md -rtk init -g --auto-patch # Non-interactive (CI/CD) -rtk init --show # Verify installation -rtk init -g --uninstall # Remove -``` - -### GitHub Copilot (VS Code + CLI) - -```bash -rtk init -g --copilot # Install hook + instructions -``` - -Creates `.github/hooks/rtk-rewrite.json` (PreToolUse hook) and `.github/copilot-instructions.md` (prompt-level awareness). - -The hook (`rtk hook copilot`) auto-detects the format: -- **VS Code Copilot Chat**: transparent rewrite via `updatedInput` (same as Claude Code) -- **Copilot CLI**: deny-with-suggestion (CLI does not support `updatedInput` yet — see [copilot-cli#2013](https://github.com/github/copilot-cli/issues/2013)) - -### Cursor - -```bash -rtk init -g --agent cursor -``` - -Creates `~/.cursor/hooks/rtk-rewrite.sh` + patches `~/.cursor/hooks.json` with preToolUse matcher. Works with both Cursor editor and `cursor-agent` CLI. +## Windows -### Gemini CLI +RTK works on Windows with some limitations. The auto-rewrite hook (`rtk-rewrite.sh`) requires a Unix shell, so on native Windows RTK falls back to **CLAUDE.md injection mode** — your AI assistant receives RTK instructions but commands are not rewritten automatically. -```bash -rtk init -g --gemini -rtk init -g --gemini --uninstall -``` - -Creates `~/.gemini/hooks/rtk-hook-gemini.sh` + patches `~/.gemini/settings.json` with BeforeTool hook. - -### Codex (OpenAI) - -```bash -rtk init -g --codex -``` - -Creates `~/.codex/RTK.md` + `~/.codex/AGENTS.md` with `@RTK.md` reference. Codex reads these as global instructions. +### Recommended: WSL (full support) -### Windsurf +For the best experience, use [WSL](https://learn.microsoft.com/en-us/windows/wsl/install) (Windows Subsystem for Linux). Inside WSL, RTK works exactly like Linux — full hook support, auto-rewrite, everything: ```bash -rtk init --agent windsurf +# Inside WSL +curl -fsSL https://raw.githubusercontent.com/algolia/rtk/refs/heads/main/install.sh | sh +rtk init -g ``` -Creates `.windsurfrules` in the current project. Cascade reads rules and prefixes commands with `rtk`. +### Native Windows (limited support) -### Cline / Roo Code +On native Windows (cmd.exe / PowerShell), RTK filters work but the hook does not auto-rewrite commands: -```bash -rtk init --agent cline +```powershell +# 1. Download and extract rtk-x86_64-pc-windows-msvc.zip from releases +# 2. Add rtk.exe to your PATH +# 3. Initialize (falls back to CLAUDE.md injection) +rtk init -g +# 4. Use rtk explicitly +rtk cargo test +rtk git status ``` -Creates `.clinerules` in the current project. Cline reads rules and prefixes commands with `rtk`. +**Important**: Do not double-click `rtk.exe` — it is a CLI tool that prints usage and exits immediately. Always run it from a terminal (Command Prompt, PowerShell, or Windows Terminal). -### OpenCode +| Feature | WSL | Native Windows | +|---------|-----|----------------| +| Filters (cargo, git, etc.) | Full | Full | +| Auto-rewrite hook | Yes | No (CLAUDE.md fallback) | +| `rtk init -g` | Hook mode | CLAUDE.md mode | +| `rtk gain` / analytics | Full | Full | -```bash -rtk init -g --opencode -``` - -Creates `~/.config/opencode/plugins/rtk.ts`. Uses `tool.execute.before` hook. +## Supported AI Tools -### OpenClaw +RTK supports 12 AI coding tools. Each integration transparently rewrites shell commands to `rtk` equivalents for 60-90% token savings. -```bash -openclaw plugins install ./openclaw -``` +| Tool | Install | Method | +|------|---------|--------| +| **Claude Code** | `rtk init -g` | PreToolUse hook (bash) | +| **GitHub Copilot (VS Code)** | `rtk init -g --copilot` | PreToolUse hook — transparent rewrite | +| **GitHub Copilot CLI** | `rtk init -g --copilot` | PreToolUse deny-with-suggestion (CLI limitation) | +| **Cursor** | `rtk init -g --agent cursor` | preToolUse hook (hooks.json) | +| **Gemini CLI** | `rtk init -g --gemini` | BeforeTool hook | +| **Codex** | `rtk init -g --codex` | AGENTS.md + RTK.md instructions | +| **Windsurf** | `rtk init --agent windsurf` | .windsurfrules (project-scoped) | +| **Cline / Roo Code** | `rtk init --agent cline` | .clinerules (project-scoped) | +| **OpenCode** | `rtk init -g --opencode` | Plugin TS (tool.execute.before) | +| **OpenClaw** | `openclaw plugins install ./openclaw` | Plugin TS (before_tool_call) | +| **Mistral Vibe** | Planned (tracked upstream) | Blocked on upstream | +| **Kilo Code** | `rtk init --agent kilocode` | .kilocode/rules/rtk-rules.md (project-scoped) | +| **Google Antigravity** | `rtk init --agent antigravity` | .agents/rules/antigravity-rtk-rules.md (project-scoped) | -Plugin in `openclaw/` directory. Uses `before_tool_call` hook, delegates to `rtk rewrite`. - -### Mistral Vibe (planned) - -Blocked on upstream BeforeToolCallback support ([mistral-vibe#531](https://github.com/mistralai/mistral-vibe/issues/531), [PR #533](https://github.com/mistralai/mistral-vibe/pull/533)). Tracked in [#800](https://github.com/rtk-ai/rtk/issues/800). - -### Commands Rewritten - -| Raw Command | Rewritten To | -|-------------|-------------| -| `git status/diff/log/add/commit/push/pull` | `rtk git ...` | -| `gh pr/issue/run` | `rtk gh ...` | -| `cargo test/build/clippy` | `rtk cargo ...` | -| `cat/head/tail ` | `rtk read ` | -| `rg/grep ` | `rtk grep ` | -| `ls` | `rtk ls` | -| `vitest/jest` | `rtk vitest run` | -| `tsc` | `rtk tsc` | -| `eslint/biome` | `rtk lint` | -| `prettier` | `rtk prettier` | -| `playwright` | `rtk playwright` | -| `prisma` | `rtk prisma` | -| `ruff check/format` | `rtk ruff ...` | -| `pytest` | `rtk pytest` | -| `pip list/install` | `rtk pip ...` | -| `go test/build/vet` | `rtk go ...` | -| `golangci-lint` | `rtk golangci-lint` | -| `rake test` / `rails test` | `rtk rake test` | -| `rspec` / `bundle exec rspec` | `rtk rspec` | -| `rubocop` / `bundle exec rubocop` | `rtk rubocop` | -| `bundle install/update` | `rtk bundle ...` | -| `docker ps/images/logs` | `rtk docker ...` | -| `kubectl get/logs` | `rtk kubectl ...` | -| `curl` | `rtk curl` | -| `pnpm list/outdated` | `rtk pnpm ...` | - -Commands already using `rtk`, heredocs (`<<`), and unrecognized commands pass through unchanged. +For per-agent setup details, override controls, and graceful degradation, see [`docs/guide/getting-started/supported-agents.md`](docs/guide/getting-started/supported-agents.md). ## Configuration -### Config File - `~/.config/rtk/config.toml` (macOS: `~/Library/Application Support/rtk/config.toml`): ```toml -[tracking] -database_path = "/path/to/custom.db" # default: ~/.local/share/rtk/history.db - [hooks] exclude_commands = ["curl", "playwright"] # skip rewrite for these [tee] enabled = true # save raw output on failure (default: true) mode = "failures" # "failures", "always", or "never" -max_files = 20 # rotation limit ``` -### Tee: Full Output Recovery - When a command fails, RTK saves the full unfiltered output so the LLM can read it without re-executing: ``` @@ -454,50 +384,39 @@ FAILED: 2/15 tests [full output: ~/.local/share/rtk/tee/1707753600_cargo_test.log] ``` +For the full config reference (all sections, env vars, per-project filters), see [`docs/guide/getting-started/configuration.md`](docs/guide/getting-started/configuration.md). + ### Uninstall ```bash rtk init -g --uninstall # Remove hook, RTK.md, settings.json entry cargo uninstall rtk # Remove binary -brew uninstall rtk # If installed via Homebrew ``` ## Documentation -- **[TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md)** - Fix common issues -- **[INSTALL.md](INSTALL.md)** - Detailed installation guide -- **[ARCHITECTURE.md](ARCHITECTURE.md)** - Technical architecture -- **[SECURITY.md](SECURITY.md)** - Security policy and PR review process -- **[AUDIT_GUIDE.md](docs/AUDIT_GUIDE.md)** - Token savings analytics guide - -## Privacy & Telemetry - -RTK collects **anonymous, aggregate usage metrics** once per day, **enabled by default**. This helps prioritize development. See opt-out options below. +- **[INSTALL.md](INSTALL.md)** — detailed installation reference +- **[docs/contributing/ARCHITECTURE.md](docs/contributing/ARCHITECTURE.md)** — system design and technical decisions +- **[CONTRIBUTING.md](CONTRIBUTING.md)** — contribution guide +- **[SECURITY.md](SECURITY.md)** — security policy -**What is collected:** -- Device hash (salted SHA-256 — per-user random salt stored locally, not reversible) -- RTK version, OS, architecture -- Command count (last 24h) and top command names (e.g. "git", "cargo" — no arguments, no file paths) -- Token savings percentage +## Privacy -**What is NOT collected:** source code, file paths, command arguments, secrets, environment variables, or any personally identifiable information. - -**Opt-out** (any of these): -```bash -# Environment variable -export RTK_TELEMETRY_DISABLED=1 - -# Or in config file (~/.config/rtk/config.toml) -[telemetry] -enabled = false -``` +This Algolia fork strips all telemetry from upstream. RTK does not phone +home: no anonymous pings, no usage stats, no remote calls. All metrics +(`rtk gain`, `rtk discover`) stay in `~/.local/share/rtk/tracking.db` +on your machine. ## Contributing -Contributions welcome! Please open an issue or PR on [GitHub](https://github.com/rtk-ai/rtk). +Contributions welcome! Please open an issue or PR on [GitHub](https://github.com/algolia/rtk). Join the community on [Discord](https://discord.gg/RySmvNF5kF). ## License MIT License - see [LICENSE](LICENSE) for details. + +## Disclaimer + +See [DISCLAIMER.md](DISCLAIMER.md). diff --git a/README_es.md b/README_es.md index c099d6649..dda34d5b0 100644 --- a/README_es.md +++ b/README_es.md @@ -15,10 +15,10 @@

- Sitio web • + Instalar • Solucion de problemas • - Arquitectura • + Arquitectura • Discord

@@ -121,10 +121,12 @@ rtk git push # -> "ok main" ### Tests ```bash -rtk test cargo test # Solo fallos (-90%) -rtk vitest run # Vitest compacto +rtk jest # Jest compacto +rtk vitest # Vitest compacto rtk pytest # Tests Python (-90%) rtk go test # Tests Go (-90%) +rtk cargo test # Tests Rust (-90%) +rtk test # Solo fallos (-90%) ``` ### Build & Lint @@ -146,7 +148,7 @@ rtk discover # Descubrir ahorros perdidos - **[TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md)** - Resolver problemas comunes - **[INSTALL.md](INSTALL.md)** - Guia de instalacion detallada -- **[ARCHITECTURE.md](ARCHITECTURE.md)** - Arquitectura tecnica +- **[ARCHITECTURE.md](docs/contributing/ARCHITECTURE.md)** - Arquitectura tecnica ## Contribuir @@ -157,3 +159,7 @@ Unete a la comunidad en [Discord](https://discord.gg/RySmvNF5kF). ## Licencia Licencia MIT - ver [LICENSE](LICENSE) para detalles. + +## Descargo de responsabilidad + +Ver [DISCLAIMER.md](DISCLAIMER.md). diff --git a/README_fr.md b/README_fr.md index 4c5e749da..fcd83c688 100644 --- a/README_fr.md +++ b/README_fr.md @@ -15,10 +15,10 @@

- Site web • + Installer • Depannage • - Architecture • + Architecture • Discord

@@ -135,11 +135,12 @@ rtk git push # -> "ok main" ### Tests ```bash -rtk test cargo test # Echecs uniquement (-90%) -rtk vitest run # Vitest compact +rtk jest # Jest compact +rtk vitest # Vitest compact rtk pytest # Tests Python (-90%) rtk go test # Tests Go (-90%) rtk cargo test # Tests Cargo (-90%) +rtk test # Echecs uniquement (-90%) ``` ### Build & Lint @@ -184,7 +185,7 @@ mode = "failures" - **[TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md)** - Resoudre les problemes courants - **[INSTALL.md](INSTALL.md)** - Guide d'installation detaille -- **[ARCHITECTURE.md](ARCHITECTURE.md)** - Architecture technique +- **[ARCHITECTURE.md](docs/contributing/ARCHITECTURE.md)** - Architecture technique ## Contribuer @@ -195,3 +196,7 @@ Rejoignez la communaute sur [Discord](https://discord.gg/RySmvNF5kF). ## Licence Licence MIT - voir [LICENSE](LICENSE) pour les details. + +## Avertissement + +Voir [DISCLAIMER.md](DISCLAIMER.md). diff --git a/README_ja.md b/README_ja.md index 6c690affa..e8c6683ae 100644 --- a/README_ja.md +++ b/README_ja.md @@ -15,10 +15,10 @@

- ウェブサイト • + インストール • トラブルシューティング • - アーキテクチャ • + アーキテクチャ • Discord

@@ -121,10 +121,11 @@ rtk git push # -> "ok main" ### テスト ```bash -rtk test cargo test # 失敗のみ表示（-90%） -rtk vitest run # Vitest コンパクト +rtk jest # Jest コンパクト +rtk vitest # Vitest コンパクト rtk pytest # Python テスト（-90%） rtk go test # Go テスト（-90%） +rtk test # 失敗のみ表示（-90%） ``` ### ビルド & リント @@ -146,7 +147,7 @@ rtk discover # 見逃した節約機会を発見 - **[TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md)** - よくある問題の解決 - **[INSTALL.md](INSTALL.md)** - 詳細インストールガイド -- **[ARCHITECTURE.md](ARCHITECTURE.md)** - 技術アーキテクチャ +- **[ARCHITECTURE.md](docs/contributing/ARCHITECTURE.md)** - 技術アーキテクチャ ## コントリビュート @@ -157,3 +158,7 @@ rtk discover # 見逃した節約機会を発見 ## ライセンス MIT ライセンス - 詳細は [LICENSE](LICENSE) を参照。 + +## 免責事項 + +詳細は [DISCLAIMER.md](DISCLAIMER.md) を参照。 diff --git a/README_ko.md b/README_ko.md index 5d3b1a0b2..8d8450d09 100644 --- a/README_ko.md +++ b/README_ko.md @@ -15,10 +15,10 @@

- 웹사이트 • + 설치 • 문제 해결 • - 아키텍처 • + 아키텍처 • Discord

@@ -121,10 +121,11 @@ rtk git push # -> "ok main" ### 테스트 ```bash -rtk test cargo test # 실패만 표시 (-90%) -rtk vitest run # Vitest 컴팩트 +rtk jest # Jest 컴팩트 +rtk vitest # Vitest 컴팩트 rtk pytest # Python 테스트 (-90%) rtk go test # Go 테스트 (-90%) +rtk test # 실패만 표시 (-90%) ``` ### 빌드 & 린트 @@ -146,7 +147,7 @@ rtk discover # 놓친 절약 기회 발견 - **[TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md)** - 일반적인 문제 해결 - **[INSTALL.md](INSTALL.md)** - 상세 설치 가이드 -- **[ARCHITECTURE.md](ARCHITECTURE.md)** - 기술 아키텍처 +- **[ARCHITECTURE.md](docs/contributing/ARCHITECTURE.md)** - 기술 아키텍처 ## 기여 @@ -157,3 +158,7 @@ rtk discover # 놓친 절약 기회 발견 ## 라이선스 MIT 라이선스 - 자세한 내용은 [LICENSE](LICENSE)를 참조하세요. + +## 면책 조항 + +자세한 내용은 [DISCLAIMER.md](DISCLAIMER.md)를 참조하세요. diff --git a/README_zh.md b/README_zh.md index 00b9c001f..394ed8dab 100644 --- a/README_zh.md +++ b/README_zh.md @@ -15,10 +15,10 @@

- 官网 • + 安装 • 故障排除 • - 架构 • + 架构 • Discord

@@ -122,10 +122,11 @@ rtk git push # -> "ok main" ### 测试 ```bash -rtk test cargo test # 仅显示失败（-90%） -rtk vitest run # Vitest 紧凑输出 +rtk jest # Jest 紧凑输出 +rtk vitest # Vitest 紧凑输出 rtk pytest # Python 测试（-90%） rtk go test # Go 测试（-90%） +rtk test # 仅显示失败（-90%） ``` ### 构建 & 检查 @@ -154,7 +155,7 @@ rtk discover # 发现遗漏的节省机会 - **[TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md)** - 解决常见问题 - **[INSTALL.md](INSTALL.md)** - 详细安装指南 -- **[ARCHITECTURE.md](ARCHITECTURE.md)** - 技术架构 +- **[ARCHITECTURE.md](docs/contributing/ARCHITECTURE.md)** - 技术架构 ## 贡献 @@ -165,3 +166,7 @@ rtk discover # 发现遗漏的节省机会 ## 许可证 MIT 许可证 - 详见 [LICENSE](LICENSE)。 + +## 免责声明 + +详见 [DISCLAIMER.md](DISCLAIMER.md)。 diff --git a/ROADMAP.md b/ROADMAP.md deleted file mode 100644 index 7f8e1d6eb..000000000 --- a/ROADMAP.md +++ /dev/null @@ -1,15 +0,0 @@ -# RTK Roadmap - - -Stability & Reliability - - Critical Fixes: Resolve bugs and stabilize Vitest/pnpm support. - - Fork Strategy: Establish the fork as the new standard if upstream remains inactive. - - Pro Tooling: Add a configuration file (TOML) and structured logging. - - Easy Install: Launch a Homebrew formula and pre-compiled binaries for one-click setup. - - Early Adoption: Prove token savings on real projects to onboard the first 5 teams. - ---- diff --git a/SECURITY.md b/SECURITY.md index 2d06b77c3..876da07fd 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -21,7 +21,7 @@ RTK is a CLI tool that executes shell commands and handles user input. PRs from - **Shell injection** (command execution vulnerabilities) - **Supply chain attacks** (malicious dependencies) - **Backdoors** (logic bombs, exfiltration code) -- **Data leaks** (tracking.db exposure, telemetry abuse) +- **Data leaks** (tracking.db exposure) --- @@ -50,7 +50,7 @@ The following files are considered **high-risk** and trigger mandatory 2-reviewe ### Tier 1: Shell Execution & System Interaction - **`src/runner.rs`** - Shell command execution engine (primary injection vector) - **`src/summary.rs`** - Command output aggregation (data exfiltration risk) -- **`src/tracking.rs`** - SQLite database operations (privacy/telemetry concerns) +- **`src/tracking.rs`** - SQLite database operations (local privacy concerns) - **`src/discover/registry.rs`** - Rewrite logic for all commands (command injection risk via rewrite rules) - **`hooks/rtk-rewrite.sh`** / **`.claude/hooks/rtk-rewrite.sh`** - Thin delegator hook (executes in Claude Code context, intercepts all commands) @@ -114,7 +114,7 @@ bash scripts/detect-dangerous-patterns.sh /tmp/pr.diff | `SystemTime::now() > ...` | Logic bombs | Delayed malicious behavior | | Base64/hex strings | Obfuscation | Hides malicious URLs/commands | -See [Dangerous Patterns Reference](https://github.com/rtk-ai/rtk/wiki/Dangerous-Patterns) for exploitation examples. +See [Dangerous Patterns Reference](https://github.com/algolia/rtk/wiki/Dangerous-Patterns) for exploitation examples. --- @@ -209,7 +209,7 @@ Critical vulnerabilities (remote code execution, data exfiltration) may be fast- ## Contact - **Security issues**: security@rtk-ai.dev -- **General questions**: https://github.com/rtk-ai/rtk/discussions +- **General questions**: https://github.com/algolia/rtk/discussions - **Maintainers**: @FlorianBruniaux (active fork maintainer) --- diff --git a/bug-reports/2026-03-31-curl-python3-not-found-in-shell-functions.md b/bug-reports/2026-03-31-curl-python3-not-found-in-shell-functions.md new file mode 100644 index 000000000..d44dbdb23 --- /dev/null +++ b/bug-reports/2026-03-31-curl-python3-not-found-in-shell-functions.md @@ -0,0 +1,58 @@ +# RTK Bug: `curl` and `python3` not found inside shell function bodies + +**Date**: 2026-03-31 +**Severity**: HIGH — completely breaks multi-step API scripts +**Category**: Command rewrite breaks function definitions + +--- + +## Symptom + +When a bash command defines a shell function that uses `curl` and pipes to `python3`, both commands fail with `command not found` despite being present at `/usr/bin/curl` and `/usr/bin/python3`. + +``` +create_link:2: command not found: curl +create_link:11: command not found: python3 +``` + +## Reproduction + +```bash +create_link() { + local url="$1" + curl -s -X POST "https://api.short.io/links" \ + -H "Content-Type: application/json" \ + -H "Authorization: $API_KEY" \ + -d '{"originalURL": "'$url'"}' | python3 -c "import sys,json; print(json.load(sys.stdin))" +} +create_link "https://example.com" +``` + +## Expected + +`curl` and `python3` execute normally inside the function body. + +## Actual + +RTK hook rewrites `curl` and/or `python3` inside the function definition, producing invalid command references that fail at invocation time. + +## Workaround + +Use `/usr/bin/curl` and `/usr/bin/python3` absolute paths, or use a dedicated CLI tool instead of curl. + +## Context + +Trying to create short.io links via API. Both `which curl` and `which python3` confirm they exist at `/usr/bin/`. + +## Resolution (v0.34.2-algolia.1) + +**Root cause**: Two issues compounding: +1. `curl` piped to `python3`/`jq` was rewritten to `rtk curl`, which auto-compresses JSON output — breaking downstream pipe consumers that expect raw JSON. +2. Shell function definitions containing rewritable commands could theoretically be corrupted (though current parser already skipped most function forms via the `$((` / compound detection). + +**Fix**: +- Added `curl`/`wget` to the pipe-incompatible list in `rewrite_compound()` — they are not rewritten when piped. +- Added explicit shell function definition detection (`() {`, `function `) in `rewrite_command()` — bail early on function bodies. +- 8 new tests covering function definitions, curl pipe skipping, and compound edge cases. + +**Status**: FIXED diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.md deleted file mode 100644 index cf52f026d..000000000 --- a/docs/TROUBLESHOOTING.md +++ /dev/null @@ -1,337 +0,0 @@ -# RTK Troubleshooting Guide - -## Problem: "rtk gain" command not found - -### Symptom -```bash -$ rtk --version -rtk 1.0.0 # (or similar) - -$ rtk gain -rtk: 'gain' is not a rtk command. See 'rtk --help'. -``` - -### Root Cause -You installed the **wrong rtk package**. You have **Rust Type Kit** (reachingforthejack/rtk) instead of **Rust Token Killer** (rtk-ai/rtk). - -### Solution - -**1. Uninstall the wrong package:** -```bash -cargo uninstall rtk -``` - -**2. Install the correct one (Token Killer):** - -#### Quick Install (Linux/macOS) -```bash -curl -fsSL https://github.com/rtk-ai/rtk/blob/master/install.sh | sh -``` - -#### Alternative: Manual Installation -```bash -cargo install --git https://github.com/rtk-ai/rtk -``` - -**3. Verify installation:** -```bash -rtk --version -rtk gain # MUST show token savings stats, not error -``` - -If `rtk gain` now works, installation is correct. - ---- - -## Problem: Confusion Between Two "rtk" Projects - -### The Two Projects - -| Project | Repository | Purpose | Key Command | -|---------|-----------|---------|-------------| -| **Rust Token Killer** ✅ | rtk-ai/rtk | LLM token optimizer for Claude Code | `rtk gain` | -| **Rust Type Kit** ❌ | reachingforthejack/rtk | Rust codebase query and type generator | `rtk query` | - -### How to Identify Which One You Have - -```bash -# Check if "gain" command exists -rtk gain - -# Token Killer → Shows token savings stats -# Type Kit → Error: "gain is not a rtk command" -``` - ---- - -## Problem: cargo install rtk installs wrong package - -### Why This Happens -If **Rust Type Kit** is published to crates.io under the name `rtk`, running `cargo install rtk` will install the wrong package. - -### Solution -**NEVER use** `cargo install rtk` without verifying. - -**Always use explicit repository URLs:** - -```bash -# CORRECT - Token Killer -cargo install --git https://github.com/rtk-ai/rtk - -# OR install from fork -git clone https://github.com/rtk-ai/rtk.git -cd rtk && git checkout feat/all-features -cargo install --path . --force -``` - -**After any installation, ALWAYS verify:** -```bash -rtk gain # Must work if you want Token Killer -``` - ---- - -## Problem: RTK not working in Claude Code - -### Symptom -Claude Code doesn't seem to be using rtk, outputs are verbose. - -### Checklist - -**1. Verify rtk is installed and correct:** -```bash -rtk --version -rtk gain # Must show stats -``` - -**2. Initialize rtk for Claude Code:** -```bash -# Global (all projects) -rtk init --global - -# Per-project -cd /your/project -rtk init -``` - -**3. Verify CLAUDE.md file exists:** -```bash -# Check global -cat ~/.claude/CLAUDE.md | grep rtk - -# Check project -cat ./CLAUDE.md | grep rtk -``` - -**4. Install auto-rewrite hook (recommended for automatic RTK usage):** - -**Option A: Automatic (recommended)** -```bash -rtk init -g -# → Installs hook + RTK.md automatically -# → Follow printed instructions to add hook to ~/.claude/settings.json -# → Restart Claude Code - -# Verify installation -rtk init --show # Should show "✅ Hook: executable, with guards" -``` - -**Option B: Manual (fallback)** -```bash -# Copy hook to Claude Code hooks directory -mkdir -p ~/.claude/hooks -cp .claude/hooks/rtk-rewrite.sh ~/.claude/hooks/ -chmod +x ~/.claude/hooks/rtk-rewrite.sh -``` - -Then add to `~/.claude/settings.json` (replace `~` with full path): -```json -{ - "hooks": { - "PreToolUse": [ - { - "matcher": "Bash", - "hooks": [ - { - "type": "command", - "command": "/Users/yourname/.claude/hooks/rtk-rewrite.sh" - } - ] - } - ] - } -} -``` - -**Note**: Use absolute path in `settings.json`, not `~/.claude/...` - ---- - -## Problem: RTK not working in OpenCode - -### Symptom -OpenCode runs commands without rtk, outputs are verbose. - -### Checklist - -**1. Verify rtk is installed and correct:** -```bash -rtk --version -rtk gain # Must show stats -``` - -**2. Install the OpenCode plugin (global only):** -```bash -rtk init -g --opencode -``` - -**3. Verify plugin file exists:** -```bash -ls -la ~/.config/opencode/plugins/rtk.ts -``` - -**4. Restart OpenCode** -OpenCode must be restarted to load the plugin. - -**5. Verify status:** -```bash -rtk init --show # Should show "OpenCode: plugin installed" -``` - ---- - -## Problem: RTK commands fail on Windows ("program not found" or "No such file") - -### Symptom -``` -rtk vitest --run -# Error: program not found -# Or: The system cannot find the file specified - -rtk lint . -# Error: No such file or directory -``` - -### Root Cause -On Windows, Node.js tools (vitest, eslint, tsc, etc.) are installed as `.CMD` or `.BAT` wrapper scripts, not as native `.exe` binaries. Rust's `std::process::Command::new("vitest")` does not honor the Windows `PATHEXT` environment variable, so it cannot find `vitest.CMD` even when it's on PATH. - -### Solution -Update to rtk v0.23.1+ which resolves this via the `which` crate for proper PATH+PATHEXT resolution. All 16+ command modules now use `resolved_command()` instead of `Command::new()`. - -```bash -cargo install --git https://github.com/rtk-ai/rtk -rtk --version # Should be 0.23.1+ -``` - -### Affected Commands -All commands that spawn external tools: `rtk vitest`, `rtk lint`, `rtk tsc`, `rtk pnpm`, `rtk playwright`, `rtk prisma`, `rtk next`, `rtk prettier`, `rtk ruff`, `rtk pytest`, `rtk pip`, `rtk mypy`, `rtk golangci-lint`, and others. - ---- - -## Problem: "command not found: rtk" after installation - -### Symptom -```bash -$ cargo install --path . --force - Compiling rtk v0.7.1 - Finished release [optimized] target(s) - Installing ~/.cargo/bin/rtk - -$ rtk --version -zsh: command not found: rtk -``` - -### Root Cause -`~/.cargo/bin` is not in your PATH. - -### Solution - -**1. Check if cargo bin is in PATH:** -```bash -echo $PATH | grep -o '[^:]*\.cargo[^:]*' -``` - -**2. If not found, add to PATH:** - -For **bash** (`~/.bashrc`): -```bash -export PATH="$HOME/.cargo/bin:$PATH" -``` - -For **zsh** (`~/.zshrc`): -```bash -export PATH="$HOME/.cargo/bin:$PATH" -``` - -For **fish** (`~/.config/fish/config.fish`): -```fish -set -gx PATH $HOME/.cargo/bin $PATH -``` - -**3. Reload shell config:** -```bash -source ~/.bashrc # or ~/.zshrc or restart terminal -``` - -**4. Verify:** -```bash -which rtk -rtk --version -rtk gain -``` - ---- - -## Problem: Compilation errors during installation - -### Symptom -```bash -$ cargo install --path . -error: failed to compile rtk v0.7.1 -``` - -### Solutions - -**1. Update Rust toolchain:** -```bash -rustup update stable -rustup default stable -``` - -**2. Clean and rebuild:** -```bash -cargo clean -cargo build --release -cargo install --path . --force -``` - -**3. Check Rust version (minimum required):** -```bash -rustc --version # Should be 1.70+ for most features -``` - -**4. If still fails, report issue:** -- GitHub: https://github.com/rtk-ai/rtk/issues - ---- - -## Need More Help? - -**Report issues:** -- Fork-specific: https://github.com/rtk-ai/rtk/issues -- Upstream: https://github.com/rtk-ai/rtk/issues - -**Run the diagnostic script:** -```bash -# From the rtk repository root -bash scripts/check-installation.sh -``` - -This script will check: -- ✅ RTK installed and in PATH -- ✅ Correct version (Token Killer, not Type Kit) -- ✅ Available features (pnpm, vitest, next, etc.) -- ✅ Claude Code integration (CLAUDE.md files) -- ✅ Auto-rewrite hook status - -The script provides specific fix commands for any issues found. diff --git a/ARCHITECTURE.md b/docs/contributing/ARCHITECTURE.md similarity index 98% rename from ARCHITECTURE.md rename to docs/contributing/ARCHITECTURE.md index a803dcbdc..86f77b51d 100644 --- a/ARCHITECTURE.md +++ b/docs/contributing/ARCHITECTURE.md @@ -1,6 +1,6 @@ # rtk Architecture Documentation -> **Deep reference** for RTK's system design, filtering taxonomy, performance characteristics, and architecture decisions. For a guided tour of the end-to-end flow, start with [docs/TECHNICAL.md](docs/TECHNICAL.md). +> **Deep reference** for RTK's system design, filtering taxonomy, performance characteristics, and architecture decisions. For a guided tour of the end-to-end flow, start with [TECHNICAL.md](TECHNICAL.md). **rtk (Rust Token Killer)** is a high-performance CLI proxy that minimizes LLM token consumption through intelligent output filtering and compression. @@ -26,7 +26,7 @@ ## System Overview -> For the proxy pattern diagram and key components table, see [docs/TECHNICAL.md](docs/TECHNICAL.md#2-architecture-overview). +> For the proxy pattern diagram and key components table, see [TECHNICAL.md](TECHNICAL.md#2-architecture-overview). ### Design Principles @@ -38,7 +38,7 @@ ### Hook Architecture (v0.9.5+) -> For the hook interception diagram and agent-specific JSON formats, see [docs/TECHNICAL.md](docs/TECHNICAL.md#32-hook-interception-command-rewriting) and [hooks/README.md](hooks/README.md). +> For the hook interception diagram and agent-specific JSON formats, see [TECHNICAL.md](TECHNICAL.md#32-hook-interception-command-rewriting) and [hooks/README.md](hooks/README.md). Two hook strategies: @@ -159,7 +159,7 @@ Database: ~/.local/share/rtk/history.db ### Module Map -> For the full file-level module tree, see [docs/TECHNICAL.md](docs/TECHNICAL.md#4-folder-map) and each folder's README. +> For the full file-level module tree, see [TECHNICAL.md](TECHNICAL.md#4-folder-map) and each folder's README. **Token savings by ecosystem:** @@ -181,7 +181,7 @@ Savings by ecosystem: ### Module Breakdown - **Command Modules**: `src/cmds/` — organized by ecosystem (git, rust, js, python, go, dotnet, cloud, system, ruby). Each ecosystem README lists its files. -- **Core Infrastructure**: `src/core/` — utils, filter, tracking, tee, config, toml_filter, display_helpers, telemetry +- **Core Infrastructure**: `src/core/` — utils, filter, tracking, tee, config, toml_filter, display_helpers - **Hook System**: `src/hooks/` — init, rewrite, permissions, hook_cmd, hook_check, hook_audit, verify, trust, integrity - **Analytics**: `src/analytics/` — gain, cc_economics, ccusage, session_cmd @@ -1034,7 +1034,7 @@ Overhead Sources: ## Resources -- **[docs/TECHNICAL.md](docs/TECHNICAL.md)**: Guided tour of end-to-end flow +- **[TECHNICAL.md](TECHNICAL.md)**: Guided tour of end-to-end flow - **[CONTRIBUTING.md](CONTRIBUTING.md)**: Design philosophy, contribution workflow, checklist - **CLAUDE.md**: Quick reference for AI agents (dev commands, build verification) - **README.md**: User guide, installation, examples diff --git a/docs/contributing/CODING_PRACTICES.md b/docs/contributing/CODING_PRACTICES.md new file mode 100644 index 000000000..bc0975541 --- /dev/null +++ b/docs/contributing/CODING_PRACTICES.md @@ -0,0 +1,186 @@ +# RTK Coding Practices v1.0 + +This document follows the [Design Philosophy](../../CONTRIBUTING.md#design-philosophy) in `CONTRIBUTING.md`. Once you understand the mental model there, this guide describes the coding practices we use day-to-day in RTK and what reviewers will look for on your PR. + +Our goal is to keep the codebase consistent and easy to extend. PRs that deviate from these practices may be asked for changes during review — this is guidance, not a gate. If a rule seems wrong for your specific case, flag it in the PR and we'll discuss. + +> **Heads up:** RTK has grown quickly and some code in the repository predates these practices. You may spot modules that don't fully follow them — this is expected, and core/ecosystem maintainers will refactor them over time. When in doubt, follow the practices below for new code rather than mirroring older patterns. + +--- + +## Quick Start for Contributors + +New to RTK? The fastest path to a mergeable first PR: + +1. **Read the flow once.** Start at [`CONTRIBUTING.md`](../../CONTRIBUTING.md), then skim [`docs/contributing/TECHNICAL.md`](TECHNICAL.md) to see how a command flows from `main.rs` → a `*_cmd.rs` filter → tracking → stdout. +2. **Look at a good example.** [`src/cmds/git/git.rs`](../../src/cmds/git/git.rs) is a representative filter — it shows the `run()` entry point, `lazy_static!` regex setup, filter helpers, and embedded tests all in one file. +3. **Know the shared helpers before reimplementing.** Two files cover most of what you need: + - [`src/core/runner.rs`](../../src/core/runner.rs) — command execution wrappers: `run_filtered()` (run a command, then apply your filter function), `run_passthrough()` (run unfiltered but tracked), `run_streamed()` (streaming filter). + - [`src/core/utils.rs`](../../src/core/utils.rs) — shared utilities: `resolved_command()`, `strip_ansi()`, `truncate()`, `count_tokens()`, and more. +4. **Follow the checklist.** [`src/cmds/README.md — Adding a New Command Filter`](../../src/cmds/README.md#adding-a-new-command-filter) walks you through creating a filter, registering it, and adding tests. +5. **Write the test first.** We follow Red-Green-Refactor. A snapshot test plus a token-savings assertion (see [Testing](#testing) below) is enough for most filters. + +If you're unsure whether your approach fits, open a draft PR or a discussion early — we'd rather help shape the design than ask for a rewrite at review. + +--- + +## Design Philosophy + +For the full framing (Correctness vs. Token Savings, Transparency, Never Block, Zero Overhead, Extensibility), see the [Design Philosophy](../../CONTRIBUTING.md#design-philosophy) section in `CONTRIBUTING.md`. + +Two practical reminders that come up often in review: + +**Portability.** RTK should behave the same across platforms. Use `#[cfg(target_os = "...")]` for platform-specific code; never assume a single OS. + +**Extensibility.** RTK should be modular. Before writing a new feature or filter, check whether an existing entry point fits — `runner::run_filtered()`, `runner::run_passthrough()`, helpers in `src/core/utils.rs`, etc. If your logic could be reused elsewhere, lift it into a shared component rather than burying it in one `*_cmd.rs` file. + +--- + +## Files, Functions, and Documentation + +Each folder contains a root `README.md` that explains the main principles, flows, and specificities of the source files it owns. These READMEs should describe concepts and cases — not list individual source files or counts, to avoid stale lists as the code evolves. Because the root README reflects core features and logic, it should not change often; meaningful edits usually imply a core refactor. + +Tests live in the same file as the code they test (inside `#[cfg(test)] mod tests { ... }`), not in a separate test file. This keeps the filter, its fixtures, and its assertions close together. + +--- + +## Edge Cases + +When you add an edge-case branch or a non-obvious exception, leave a short comment above it explaining *why* it exists. This prevents a future contributor from removing it because the reason isn't visible from the code alone. + +Referencing an issue is often the clearest form: + +```rust +// ISSUE #463: some `git log` output contains NUL bytes when --format=%x00 is used; +// skip the line rather than panicking on invalid UTF-8. +if line.contains('\0') { + continue; +} +``` + +--- + +## Comments + +Prefer code that reads clearly over code that needs comments to explain it. In particular, avoid redundant comments that restate what the function signature already says. + +Comments are welcome when they add information the code cannot carry on its own. The common cases: + +- **File header (`//!`)** — purpose and scope of the current file. +- **Edge case** — a non-obvious branch or exception, as described above. +- **Issue reference** — e.g. `// ISSUE #463: the fix for this`. +- **"Why, not what"** — when the intent or tradeoff behind a decision isn't obvious from the code. + +In short: avoid noise comments; keep the ones that would save a future reader a trip to `git blame`. + +--- + +## Variables + +Use explicit, descriptive names for variables, just like for functions. + +Do not hardcode repetitive patterns or values that control behavior — extract them into named constants at the top of the file. For anything a user might want to tune (thresholds, limits, display cutoffs), use `config::limits()` so it flows through `~/.config/rtk/config.toml`. + +Example from `src/cmds/git/git.rs`: + +```rust +let limits = config::limits(); +let max_files = limits.status_max_files; +let max_untracked = limits.status_max_untracked; +``` + +--- + +## Function and File Size + +**Prefer functions under ~60 lines.** Shorter functions are easier to read, test, and reuse. If a function grows beyond that, it's usually a sign the logic should be split into helpers — but this is a guideline, not a hard cap. + +Legitimate exceptions include: +- Dispatcher / match functions that route to subcommands, where each arm delegates to a focused helper. +- State-machine parsers where splitting would harm readability. + +When you keep a longer function, aim to make each block obviously cohesive — and consider leaving a short comment on *why* splitting it would hurt. + +**Files are expected to be large** in RTK because each module keeps its tests and fixtures alongside the implementation. When a file becomes hard to navigate, split responsibilities across multiple files where possible. If it isn't possible, a big file is acceptable for now. + +--- + +## Imports and Dependencies + +RTK is a low-dependency project. Before adding a crate, check whether the functionality is already covered by `std`, an existing dependency, or `src/core/utils.rs`. If a few lines of straightforward code will do the job, prefer that over a new dependency. + +When a new dependency is genuinely needed, justify it in the PR description. For non-trivial additions, it's worth opening a discussion with maintainers first. + +--- + +## Error Handling + +Use `anyhow::Result` everywhere, and always attach context with `.context("description")?` or `.with_context(|| format!(...))`. + +Never silently swallow errors (`Err(_) => {}`). Either log with `eprintln!` and fall back to raw output (the common case for filters), or propagate the error. + +Example of the standard fallback pattern for a filter: + +```rust +let filtered = filter_output(&output.stdout) + .unwrap_or_else(|e| { + eprintln!("rtk: filter warning: {}", e); + output.stdout.clone() // passthrough on failure — never block the user + }); +``` + +For the full error-handling architecture (propagation chain, exit code preservation), see [ARCHITECTURE.md — Error Handling](ARCHITECTURE.md#error-handling). + +--- + +## Testing + +See [`CONTRIBUTING.md` — Testing](../../CONTRIBUTING.md#testing) for the full strategy. In short, for a new filter you typically want: + +- **Unit + snapshot tests** in the same file, using the `insta` crate. +- **A token-savings assertion** verifying the filter hits the ≥60% target on a real fixture. + +Minimal example: + +```rust +#[cfg(test)] +mod tests { + use super::*; + use insta::assert_snapshot; + + fn count_tokens(s: &str) -> usize { s.split_whitespace().count() } + + #[test] + fn filter_git_log_snapshot() { + let input = include_str!("../../../tests/fixtures/git_log_raw.txt"); + let output = filter_git_log(input); + assert_snapshot!(output); + } + + #[test] + fn filter_git_log_savings() { + let input = include_str!("../../../tests/fixtures/git_log_raw.txt"); + let output = filter_git_log(input); + let savings = 100.0 - (count_tokens(&output) as f64 / count_tokens(input) as f64 * 100.0); + assert!(savings >= 60.0, "expected ≥60% savings, got {:.1}%", savings); + } +} +``` + +Fixtures go in `tests/fixtures/` and should be captured from real command output rather than hand-written. + +--- + +## Security + +RTK executes shell commands on behalf of the user, so security is a first-class concern. + +**Command execution.** All commands go through argument arrays via `Command::new().args()` — never through shell string concatenation. This prevents injection. Always use `resolved_command()` from `src/core/utils.rs` instead of a raw `Command::new()`. + +**Hook integrity.** RTK verifies hook files via SHA-256 hashes before operational commands. If a hook has been tampered with, RTK exits with code 1. See [`src/hooks/integrity.rs`](../../src/hooks/integrity.rs). + +**Project filter trust.** `.rtk/filters.toml` files are not loaded until the user explicitly trusts them, and content changes require re-trust. See [`src/hooks/trust.rs`](../../src/hooks/trust.rs). + +**Permission whitelist.** `is_operational_command()` in `main.rs` uses a whitelist pattern — new commands are *not* integrity-checked until explicitly added. This is an intentional security posture: fail-open with an audit trail is preferred over false confidence. + +**`unsafe` code.** Not allowed except for Unix signal handling in proxy mode, which is correctly scoped to `#[cfg(unix)]`. diff --git a/docs/TECHNICAL.md b/docs/contributing/TECHNICAL.md similarity index 71% rename from docs/TECHNICAL.md rename to docs/contributing/TECHNICAL.md index 541f712d4..db2ac0af1 100644 --- a/docs/TECHNICAL.md +++ b/docs/contributing/TECHNICAL.md @@ -3,7 +3,7 @@ > **Start here** for a guided tour of how RTK works end-to-end. > > - [CONTRIBUTING.md](../CONTRIBUTING.md) — Design philosophy, PR process, branch naming, testing requirements -> - [ARCHITECTURE.md](../ARCHITECTURE.md) — Deep reference: filtering taxonomy, performance benchmarks, architecture decisions +> - [ARCHITECTURE.md](ARCHITECTURE.md) — Deep reference: filtering taxonomy, performance benchmarks, architecture decisions > - Each folder has its own `README.md` with implementation details and file descriptions --- @@ -94,15 +94,129 @@ All rewrite logic lives in Rust (`src/discover/registry.rs`). Hooks are thin del > **Details**: [`hooks/README.md`](../hooks/README.md) covers each agent's JSON format, the rewrite registry, compound command handling, and the `RTK_DISABLED` override. +#### Rewrite Pipeline + +The rewrite pipeline is how RTK intercepts and rewrites commands. The call chain is: + +``` +hook shell → rewrite_cmd.rs → rewrite_command() → rewrite_compound() → rewrite_segment() → classify_command() +``` + +Traced step by step for `cargo fmt --all && cargo test 2>&1 | tail -20`: + +``` +LLM Agent: "cargo fmt --all && cargo test 2>&1 | tail -20" + | + | Hook shell (hooks/claude/rtk-rewrite.sh) + | Reads JSON from agent, extracts command, calls `rtk rewrite "$CMD"` + | On failure (jq missing, rtk missing, old version): exit 0 (passthrough) + | + v +rewrite_cmd::run(cmd) [src/hooks/rewrite_cmd.rs] + | 1. Load config → hooks.exclude_commands + | 2. check_command(cmd) → Deny → exit(2) + | 3. registry::rewrite_command(cmd, excluded) + | → None → exit(1) (no RTK equivalent, passthrough) + | → Some + Allow → print, exit(0) + | → Some + Ask → print, exit(3) + | + v +rewrite_command(cmd, excluded) [src/discover/registry.rs] + | Early exits: + | - Empty → None + | - Contains "<<" or "$((" (heredoc/arithmetic) → None + | - Simple "rtk ..." (no operators) → return as-is + | - Otherwise → rewrite_compound(cmd, excluded) + | + v +rewrite_compound(cmd, excluded) [src/discover/registry.rs] + | + | Step 1 — Tokenize (lexer.rs) + | tokenize() produces typed tokens with byte offsets: + | Arg("cargo") Arg("fmt") Arg("--all") + | Operator("&&") + | Arg("cargo") Arg("test") Redirect("2>&1") + | Pipe("|") + | Arg("tail") Arg("-20") + | + | Step 2 — Split on operators, rewrite each segment + | Operator (&&, ||, ;) → rewrite both sides + | Pipe (|) → rewrite left side only, keep right side raw + | exception: find/fd before pipe → skip rewrite + | Shellism (&) → rewrite both sides (background) + | + | Calls rewrite_segment() per segment: + | segment 1: "cargo fmt --all" + | segment 2: "cargo test 2>&1" + | after pipe: "tail -20" kept raw + | + v +rewrite_segment(seg, excluded) [src/discover/registry.rs] + | + | Step 3 — Strip trailing redirects + | strip_trailing_redirects() re-tokenizes the segment: + | "cargo test 2>&1" → cmd_part="cargo test", redirect=" 2>&1" + | (simple commands like "cargo fmt --all" → no redirect, suffix is "") + | + | Step 4 — Already RTK → return as-is + | + | Step 5 — Special cases (short-circuit before classification) + | head -N / --lines=N → rewrite_line_range() → "rtk read file --max-lines N" + | tail -N / -n N / --lines N → rewrite_line_range() → "rtk read file --tail-lines N" + | head/tail with unsupported flag (-c, -f) → None (skip rewrite) + | cat with incompatible flag (-A, -v, -e) → None (skip rewrite) + | + | Step 6 — classify_command(cmd_part) [see below] + | → Supported → check excluded list → continue + | → Unsupported/Ignored → None (skip rewrite) + | + | Step 7 — Build rewritten command + | a. Find matching rule from rules.rs + | b. Extract env prefix (ENV_PREFIX regex, second pass — first was in classify) + | e.g. "GIT_SSH_COMMAND=\"ssh -o ...\" git push" → prefix="GIT_SSH_COMMAND=..." + | c. Guard: RTK_DISABLED=1 in prefix → None + | d. Guard: gh with --json/--jq/--template → None + | e. Apply rule's rewrite_prefixes: "cargo fmt" → "rtk cargo fmt" + | f. Reassemble: env_prefix + rtk_cmd + args + redirect_suffix + | + v +classify_command(cmd) [src/discover/registry.rs] + | 1. Check IGNORED_EXACT (cd, echo, fi, done, ...) + | 2. Check IGNORED_PREFIXES (rtk, mkdir, mv, ...) + | 3. Strip env prefix with ENV_PREFIX regex (for pattern matching only) + | 4. Normalize absolute paths: /usr/bin/grep → grep + | 5. Strip git global opts: git -C /tmp status → git status + | 6. Guard: cat/head/tail with redirect (>, >>) → Unsupported (write, not read) + | 7. Match against REGEX_SET (60+ compiled patterns from rules.rs) + | 8. Extract subcommand → lookup custom savings/status overrides + | 9. Return Classification::Supported { rtk_equivalent, category, savings, status } + | + v +Result: "rtk cargo fmt --all && rtk cargo test 2>&1 | tail -20" + | + | Hook response + | Hook wraps result in agent-specific JSON, returns to LLM agent + | + v +LLM Agent executes rewritten command + (bash handles && and |, each rtk invocation is a separate process) +``` + +Key design decisions: +- **Lexer-based tokenization**: A single-pass state machine (`lexer.rs`) handles all shell constructs (quotes, escapes, redirects, operators). Used for both compound splitting and redirect stripping. +- **Segment-level rewriting**: Compound commands are split by operators, each segment rewritten independently. Bash recombines them at execution time. +- **Pipe semantics**: Only the left side of `|` is rewritten. The pipe consumer (grep, head, wc) runs raw. `find`/`fd` before a pipe is never rewritten (output format incompatible with xargs). +- **Double env prefix handling**: `classify_command()` strips env prefixes to match the underlying command against rules. `rewrite_segment()` extracts the same prefix separately to re-prepend it to the rewritten command. +- **Fallback contract**: If any segment fails to match, it stays raw. `rewrite_command()` returns `None` only when zero segments were rewritten. + ### 3.3 CLI Parsing and Routing Once the rewritten command reaches RTK: -1. **Telemetry**: `telemetry::maybe_ping()` fires a non-blocking daily usage ping -2. **Clap parsing**: `Cli::try_parse()` matches against the `Commands` enum -3. **Hook check**: `hook_check::maybe_warn()` warns if the installed hook is outdated (rate-limited to 1/day) -4. **Integrity check**: `integrity::runtime_check()` verifies the hook's SHA-256 hash for operational commands -5. **Routing**: A `match cli.command` dispatches to the specialized filter module +1. **Clap parsing**: `Cli::try_parse()` matches against the `Commands` enum +2. **Hook check**: `hook_check::maybe_warn()` warns if the installed hook is outdated (rate-limited to 1/day) +3. **Integrity check**: `integrity::runtime_check()` verifies the hook's SHA-256 hash for operational commands +4. **Routing**: A `match cli.command` dispatches to the specialized filter module If Clap parsing fails (command not in the enum), the fallback path runs instead. diff --git a/docs/filter-workflow.md b/docs/filter-workflow.md deleted file mode 100644 index 0b0d32c1b..000000000 --- a/docs/filter-workflow.md +++ /dev/null @@ -1,102 +0,0 @@ -# How a TOML filter goes from file to execution - -This document explains what happens between "I created `src/filters/my-tool.toml`" and "RTK filters the output of `my-tool`". - -## Build pipeline - -```mermaid -flowchart TD - A[["📄 src/filters/my-tool.toml\n(new file)"]] --> B - - subgraph BUILD ["🔨 cargo build"] - B["build.rs\n① ls src/filters/*.toml\n② sort alphabetically\n③ concat → schema_version = 1 + all files"] --> C - C{"TOML valid?\nDuplicate names?"} -->|"❌ panic! (build fails)"| D[["🛑 Error message\npoints to bad file"]] - C -->|"✅ ok"| E[["OUT_DIR/builtin_filters.toml\n(generated file)"]] - E --> F["rustc\ninclude_str!(concat!(env!(OUT_DIR),\n'/builtin_filters.toml'))"] - F --> G[["🦀 rtk binary\nBUILTIN_TOML embedded"]] - end - - subgraph TESTS ["🧪 cargo test"] - H["test_builtin_filter_count\nassert_eq!(filters.len(), N)"] -->|"❌ count wrong"| I[["FAIL\n'Expected N, got N+1'\nUpdate the count'"]] - J["test_builtin_all_expected_\nfilters_present\nassert!(names.contains('my-tool'))"] -->|"❌ name missing"| K[["FAIL\n'my-tool is missing—\nwas its .toml deleted?'"]] - L["test_builtin_all_filters_\nhave_inline_tests\nassert!(tested.contains(name))"] -->|"❌ no tests"| M[["FAIL\n'Add tests.my-tool\nentries'"]] - end - - subgraph VERIFY ["✅ rtk verify"] - N["runs [[tests.my-tool]]\ninput → filter → compare expected"] - N -->|"❌ mismatch"| O[["FAIL\nshows actual vs expected"]] - N -->|"✅ pass"| P[["60/60 tests passed"]] - end - - G --> H - G --> J - G --> L - G --> N - - subgraph RUNTIME ["⚡ rtk my-tool --verbose"] - Q["Claude Code hook\nmy-tool ... → rtk my-tool ..."] --> R - R["TomlFilterRegistry::load()\n① .rtk/filters.toml (project)\n② ~/.config/rtk/filters.toml (user)\n③ BUILTIN_TOML (binary)\n④ passthrough"] --> S - S{"match_command\n'^my-tool\\b'\nmatches?"} -->|"No match"| T[["exec raw\n(passthrough)"]] - S -->|"✅ match"| U["exec command\ncapture stdout"] - U --> V - - subgraph PIPELINE ["8-stage filter pipeline"] - V["strip_ansi"] --> W["replace"] - W --> X{"match_output\nshort-circuit?"} - X -->|"✅ pattern matched"| Y[["emit message\nstop pipeline"]] - X -->|"no match"| Z["strip/keep_lines"] - Z --> AA["truncate_lines_at"] - AA --> AB["tail_lines"] - AB --> AC["max_lines"] - AC --> AD{"output\nempty?"} - AD -->|"yes"| AE[["emit on_empty"]] - AD -->|"no"| AF[["print filtered\noutput + exit code"]] - end - end - - G --> Q - - style BUILD fill:#1e3a5f,color:#fff - style TESTS fill:#1a3a1a,color:#fff - style VERIFY fill:#2d1b69,color:#fff - style RUNTIME fill:#3a1a1a,color:#fff - style PIPELINE fill:#4a2a00,color:#fff - style D fill:#8b0000,color:#fff - style I fill:#8b0000,color:#fff - style K fill:#8b0000,color:#fff - style M fill:#8b0000,color:#fff - style O fill:#8b0000,color:#fff -``` - -## Step-by-step summary - -| Step | Who | What happens | Fails if | -|------|-----|--------------|----------| -| 1 | Contributor | Creates `src/filters/my-tool.toml` | — | -| 2 | `build.rs` | Concatenates all `.toml` files alphabetically | TOML syntax error, duplicate filter name | -| 3 | `rustc` | Embeds result in binary via `BUILTIN_TOML` const | — | -| 4 | `cargo test` | 3 guards check count, names, inline test presence | Count not updated, name not in list, no `[[tests.*]]` | -| 5 | `rtk verify` | Runs each `[[tests.my-tool]]` entry | Filter logic doesn't match expected output | -| 6 | Runtime | Hook rewrites command, registry looks up filter, pipeline runs | No match → passthrough (not an error) | - -## Filter lookup priority at runtime - -```mermaid -flowchart LR - CMD["rtk my-tool args"] --> P1 - P1{"1. .rtk/filters.toml\n(project-local)"} - P1 -->|"✅ match"| WIN["apply filter"] - P1 -->|"no match"| P2 - P2{"2. ~/.config/rtk/filters.toml\n(user-global)\n(macOS alt: ~/Library/Application Support/rtk/filters.toml)"} - P2 -->|"✅ match"| WIN - P2 -->|"no match"| P3 - P3{"3. BUILTIN_TOML\n(binary)"} - P3 -->|"✅ match"| WIN - P3 -->|"no match"| P4[["exec raw\n(passthrough)"]] -``` - -First match wins. A project filter with the same name as a built-in shadows the built-in and triggers a warning: - -``` -[rtk] warning: filter 'make' is shadowing a built-in filter -``` diff --git a/docs/guide/analytics/discover.md b/docs/guide/analytics/discover.md new file mode 100644 index 000000000..575ca73b2 --- /dev/null +++ b/docs/guide/analytics/discover.md @@ -0,0 +1,58 @@ +--- +title: Discover and Session +description: Find missed savings opportunities with rtk discover, and track RTK adoption with rtk session +sidebar: + order: 2 +--- + +# Discover and Session + +## rtk discover — find missed savings + +`rtk discover` analyzes your Claude Code command history to identify commands that ran without RTK filtering and calculates how many tokens you lost. + +```bash +rtk discover # analyze current project history +rtk discover --all # all projects +rtk discover --all --since 7 # last 7 days, all projects +``` + +**Example output:** + +``` +Missed savings analysis (last 7 days) +──────────────────────────────────── +Command Count Est. lost +cargo test 12 ~48,000 tokens +git log 8 ~12,000 tokens +pnpm list 3 ~6,000 tokens +──────────────────────────────────── +Total missed: 23 ~66,000 tokens + +Run `rtk init --global` to capture these automatically. +``` + +If commands appear in the missed list after installing RTK, it usually means the hook isn't active for that agent. See [Troubleshooting](../resources/troubleshooting.md) — "Agent not using RTK". + +## rtk session — adoption tracking + +`rtk session` shows RTK adoption across recent Claude Code sessions: how many shell commands ran through RTK vs. raw. + +```bash +rtk session +``` + +**Example output:** + +``` +Recent sessions (last 10) +───────────────────────────────────────────────────── +Session Total RTK Coverage +2026-04-06 14:32 (45 cmds) 45 43 95.6% +2026-04-05 09:14 (38 cmds) 38 38 100.0% +2026-04-04 16:50 (52 cmds) 52 49 94.2% +───────────────────────────────────────────────────── +Average coverage: 96.6% +``` + +Low coverage on a session usually means RTK was disabled (`RTK_DISABLED=1`) or the hook wasn't active for a specific subagent. diff --git a/docs/guide/analytics/gain.md b/docs/guide/analytics/gain.md new file mode 100644 index 000000000..706508fce --- /dev/null +++ b/docs/guide/analytics/gain.md @@ -0,0 +1,215 @@ +--- +title: Token Savings Analytics +description: Measure and analyze your RTK token savings with rtk gain +sidebar: + order: 1 +--- + +# Token Savings Analytics + +`rtk gain` shows how many tokens RTK has saved across all your commands, with daily, weekly, and monthly breakdowns. + +## Quick reference + +```bash +# Default summary +rtk gain + +# Temporal breakdowns +rtk gain --daily # all days since tracking started +rtk gain --weekly # aggregated by week +rtk gain --monthly # aggregated by month +rtk gain --all # all breakdowns at once + +# Classic flags +rtk gain --graph # ASCII graph, last 30 days +rtk gain --history # last 10 commands +rtk gain --quota # monthly quota savings estimate (default tier: 20x) +rtk gain --quota -t pro # use pro tier token budget for estimate + +# Export +rtk gain --all --format json > savings.json +rtk gain --all --format csv > savings.csv +``` + +## Daily breakdown + +```bash +rtk gain --daily +``` + +``` +📅 Daily Breakdown (3 days) +════════════════════════════════════════════════════════════════ +Date Cmds Input Output Saved Save% +──────────────────────────────────────────────────────────────── +2026-01-28 89 380.9K 26.7K 355.8K 93.4% +2026-01-29 102 894.5K 32.4K 863.7K 96.6% +2026-01-30 5 749 55 694 92.7% +──────────────────────────────────────────────────────────────── +TOTAL 196 1.3M 59.2K 1.2M 95.6% +``` + +- **Cmds**: RTK commands executed +- **Input**: Estimated tokens from raw command output +- **Output**: Actual tokens after filtering +- **Saved**: Input - Output (tokens that never reached the LLM) +- **Save%**: Saved / Input × 100 + +## Weekly and monthly breakdowns + +```bash +rtk gain --weekly +rtk gain --monthly +``` + +Same columns as daily, aggregated by Sunday-Saturday week or calendar month. + +## Export formats + +| Format | Flag | Use case | +|--------|------|----------| +| `text` | default | Terminal display | +| `json` | `--format json` | Programmatic analysis, dashboards | +| `csv` | `--format csv` | Excel, Python/R, Google Sheets | + +**JSON structure:** +```json +{ + "summary": { + "total_commands": 196, + "total_input": 1276098, + "total_output": 59244, + "total_saved": 1220217, + "avg_savings_pct": 95.62 + }, + "daily": [...], + "weekly": [...], + "monthly": [...] +} +``` + +## Typical savings by command + +| Command | Typical savings | Mechanism | +|---------|----------------|-----------| +| `git status` | 77-93% | Compact stat format | +| `eslint` | 84% | Group by rule | +| `jest` | 94-99% | Show failures only | +| `vitest` | 94-99% | Show failures only | +| `find` | 75% | Tree format | +| `pnpm list` | 70-90% | Compact dependencies | +| `grep` | 70% | Truncate + group | + +## How token estimation works + +RTK estimates tokens using `text.len() / 4` (4 characters per token average). This is accurate to ±10% compared to actual LLM tokenization — sufficient for trend analysis. + +``` +Input Tokens = estimate_tokens(raw_command_output) +Output Tokens = estimate_tokens(rtk_filtered_output) +Saved Tokens = Input - Output +Savings % = (Saved / Input) × 100 +``` + +## Database + +Savings data is stored locally in SQLite: + +- **Location**: `~/.local/share/rtk/history.db` (Linux / macOS) +- **Retention**: 90 days (automatic cleanup) +- **Scope**: Global across all projects and Claude sessions + +```bash +# Inspect raw data +sqlite3 ~/.local/share/rtk/history.db \ + "SELECT timestamp, rtk_cmd, saved_tokens FROM commands + ORDER BY timestamp DESC LIMIT 10" + +# Backup +cp ~/.local/share/rtk/history.db ~/backups/rtk-history-$(date +%Y%m%d).db + +# Reset +rm ~/.local/share/rtk/history.db # recreated on next command +``` + +## Analysis workflows + +```bash +# Weekly progress: generate a CSV report every Monday +rtk gain --weekly --format csv > reports/week-$(date +%Y-%W).csv + +# Monthly budget review +rtk gain --monthly --format json | jq '.monthly[] | + {month, saved_tokens, quota_pct: (.saved_tokens / 6000000 * 100)}' + +# Cron: daily JSON snapshot for a dashboard +0 0 * * * rtk gain --all --format json > /var/www/dashboard/rtk-stats.json +``` + +**Python/pandas:** +```python +import pandas as pd +import subprocess + +result = subprocess.run(['rtk', 'gain', '--all', '--format', 'csv'], + capture_output=True, text=True) +lines = result.stdout.split('\n') +daily_start = lines.index('# Daily Data') + 2 +daily_end = lines.index('', daily_start) +daily_df = pd.read_csv(pd.StringIO('\n'.join(lines[daily_start:daily_end]))) +daily_df['date'] = pd.to_datetime(daily_df['date']) +daily_df.plot(x='date', y='savings_pct', kind='line') +``` + +**GitHub Actions (weekly stats):** +```yaml +on: + schedule: + - cron: '0 0 * * 1' +jobs: + stats: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - run: cargo install rtk + - run: rtk gain --weekly --format json > stats/week-$(date +%Y-%W).json + - run: git add stats/ && git commit -m "Weekly rtk stats" && git push +``` + +## Quota estimate + +`--quota` estimates how many tokens RTK has saved relative to your monthly subscription budget, so you can see the cost impact of those savings. + +```bash +rtk gain --quota # uses 20x tier by default +rtk gain --quota -t pro # Claude Pro plan budget +rtk gain --quota -t 5x # 5× usage plan budget +rtk gain --quota -t 20x # 20× usage plan budget +``` + +The tiers (`pro`, `5x`, `20x`) correspond to Anthropic Claude API subscription levels, each with a different monthly token allocation. RTK uses those allocations as a denominator to express your savings as a percentage of your budget. + +:::tip[Find missed savings] +`rtk gain` shows what RTK saved. To find commands that ran *without* RTK and calculate what you lost, see [rtk discover](./discover.md). +::: + +## Troubleshooting + +**No data showing:** +```bash +ls -lh ~/.local/share/rtk/history.db +sqlite3 ~/.local/share/rtk/history.db "SELECT COUNT(*) FROM commands" +git status # run any tracked command to generate data +``` + +**Incorrect statistics:** Token estimation is a heuristic. For precise counts, use `tiktoken`: +```bash +pip install tiktoken +git status > output.txt +python -c " +import tiktoken +enc = tiktoken.get_encoding('cl100k_base') +print(len(enc.encode(open('output.txt').read())), 'actual tokens') +" +``` diff --git a/docs/guide/getting-started/configuration.md b/docs/guide/getting-started/configuration.md new file mode 100644 index 000000000..b242fa125 --- /dev/null +++ b/docs/guide/getting-started/configuration.md @@ -0,0 +1,109 @@ +--- +title: Configuration +description: Customize RTK behavior via config.toml, environment variables, and per-project filters +sidebar: + order: 4 +--- + +# Configuration + +## Config file location + +| Platform | Path | +|----------|------| +| Linux | `~/.config/rtk/config.toml` | +| macOS | `~/Library/Application Support/rtk/config.toml` | + +```bash +rtk config # show current configuration +rtk config --create # create config file with defaults +``` + +## Full config structure + +```toml +[tracking] +enabled = true # enable/disable token tracking +history_days = 90 # retention in days (auto-cleanup) +database_path = "/custom/path/history.db" # optional override + +[display] +colors = true # colored output +emoji = true # use emojis in output +max_width = 120 # maximum output width + +[filters] +# These apply to file-reading commands (ls, find, grep, cat/rtk read). +# Paths matching these patterns are excluded from output, keeping noise low. +ignore_dirs = [".git", "node_modules", "target", "__pycache__", ".venv", "vendor"] +ignore_files = ["*.lock", "*.min.js", "*.min.css"] + +[tee] +enabled = true # save raw output on failure +mode = "failures" # "failures" (default), "always", "never" +max_files = 20 # rotation: keep last N files +# directory = "/custom/tee/path" # optional override + +[hooks] +exclude_commands = [] # commands to never auto-rewrite +``` + +## Environment variables + +| Variable | Description | +|----------|-------------| +| `RTK_DISABLED=1` | Disable RTK for a single command (`RTK_DISABLED=1 git status`) | +| `RTK_TEE_DIR` | Override the tee directory | +| `RTK_HOOK_AUDIT=1` | Enable hook audit logging | +| `SKIP_ENV_VALIDATION=1` | Skip env validation (useful with Next.js) | + +## Tee system + +When a command fails, RTK saves the full raw output to a local file and prints the path: + +``` +FAILED: 2/15 tests +[full output: ~/.local/share/rtk/tee/1707753600_cargo_test.log] +``` + +Your AI assistant can then read the file if it needs more detail, without re-running the command. + +| Setting | Default | Description | +|---------|---------|-------------| +| `tee.enabled` | `true` | Enable/disable | +| `tee.mode` | `"failures"` | `"failures"`, `"always"`, `"never"` | +| `tee.max_files` | `20` | Rotation: keep last N files | +| Min size | 500 bytes | Outputs shorter than this are not saved | +| Max file size | 1 MB | Truncated above this | + +## Excluding commands from auto-rewrite + +Prevent specific commands from being rewritten by the hook: + +```toml +[hooks] +exclude_commands = ["git rebase", "git cherry-pick", "docker exec"] +``` + +Patterns match against the full command after stripping env prefixes (`sudo`, `VAR=val`), so `"psql"` excludes both `psql -h localhost` and `PGPASSWORD=x psql -h localhost`. + +Subcommand patterns work too: `"git push"` excludes `git push origin main` but not `git status`. + +Patterns starting with `^` are treated as regex: + +```toml +[hooks] +exclude_commands = ["^curl", "^wget", "git rebase"] +``` + +Invalid regex patterns fall back to prefix matching. + +Or for a single invocation: + +```bash +RTK_DISABLED=1 git rebase main +``` + +## Per-project filters + +Create `.rtk/filters.toml` in your project root to add custom filters or override built-ins. See [`src/filters/README.md`](https://github.com/algolia/rtk/blob/main/src/filters/README.md) for the full TOML DSL reference. diff --git a/docs/guide/getting-started/installation.md b/docs/guide/getting-started/installation.md new file mode 100644 index 000000000..a7c7bb682 --- /dev/null +++ b/docs/guide/getting-started/installation.md @@ -0,0 +1,95 @@ +--- +title: Installation +description: Install RTK via curl, Homebrew, Cargo, or from source, and verify the correct version +sidebar: + order: 1 +--- + +# Installation + +## Name collision warning + +Two unrelated projects share the name `rtk`. Make sure you install the right one: + +- **Rust Token Killer** (`algolia/rtk`) — this project, a token-saving CLI proxy +- **Rust Type Kit** (`reachingforthejack/rtk`) — a different tool for generating Rust types + +The easiest way to verify you have the correct one: run `rtk gain`. It should display token savings stats. If it returns "command not found", you either have the wrong package or RTK is not installed. + +## Check before installing + +```bash +rtk --version # should print: rtk x.y.z +rtk gain # should show token savings stats +``` + +If both commands work, RTK is already installed. Skip to [Project initialization](#project-initialization). + +## Quick install (Linux and macOS) + +```bash +curl -fsSL https://raw.githubusercontent.com/algolia/rtk/main/install.sh | sh +``` + +## Homebrew (macOS and Linux) + +```bash +brew install rtk-ai/tap/rtk +``` + +## Cargo + +:::caution[Name collision risk] +`cargo install rtk` may install **Rust Type Kit** instead of Rust Token Killer — two unrelated projects share the same crate name. Use the explicit Git URL to guarantee the correct package: +::: + +```bash +cargo install --git https://github.com/algolia/rtk rtk +``` + +## Pre-built binaries (Windows, Linux, macOS) + +Download from [GitHub releases](https://github.com/algolia/rtk/releases): + +- macOS: `rtk-x86_64-apple-darwin.tar.gz` / `rtk-aarch64-apple-darwin.tar.gz` +- Linux: `rtk-x86_64-unknown-linux-musl.tar.gz` / `rtk-aarch64-unknown-linux-gnu.tar.gz` +- Windows: `rtk-x86_64-pc-windows-msvc.zip` + +**Windows users**: Extract the zip and place `rtk.exe` in a directory on your PATH. Run RTK from Command Prompt, PowerShell, or Windows Terminal — do not double-click the `.exe` (it prints usage and exits immediately). For full hook support, use [WSL](https://learn.microsoft.com/en-us/windows/wsl/install) instead. + +## Verify installation + +```bash +rtk --version # rtk x.y.z +rtk gain # token savings dashboard +``` + +If `rtk gain` fails but `rtk --version` succeeds, you installed Rust Type Kit by mistake. Uninstall it first: + +```bash +cargo uninstall rtk +``` + +Then reinstall using one of the methods above. + +## Project initialization + +Run once per project to enable the Claude Code hook: + +```bash +rtk init +``` + +For a global install that patches `settings.json` automatically: + +```bash +rtk init --global +``` + +## Uninstall + +```bash +rtk init -g --uninstall # remove hook, RTK.md, and settings.json entry +cargo uninstall rtk # remove binary (if installed via Cargo) +brew uninstall rtk # remove binary (if installed via Homebrew) +``` diff --git a/docs/guide/getting-started/quick-start.md b/docs/guide/getting-started/quick-start.md new file mode 100644 index 000000000..6e1b7b558 --- /dev/null +++ b/docs/guide/getting-started/quick-start.md @@ -0,0 +1,70 @@ +--- +title: Quick Start +description: Get RTK running in 5 minutes and see your first token savings +sidebar: + order: 2 +--- + +# Quick Start + +This guide walks you through your first RTK commands after installation. + +## Prerequisites + +RTK is installed and verified: + +```bash +rtk --version # rtk x.y.z +rtk gain # shows token savings dashboard +``` + +If not, see [Installation](./installation.md). + +## Step 1: Initialize for your AI assistant + +```bash +# For Claude Code (global — applies to all projects) +rtk init --global + +# For a single project only +cd /your/project && rtk init +``` + +This installs the hook that automatically rewrites commands. Restart your AI assistant after this step. + +## Step 2: Use your tools normally + +Once the hook is installed, nothing changes in how you work. Your AI assistant runs commands as usual — the hook intercepts them transparently and rewrites them before execution. + +For example, when Claude Code runs `cargo test`, the hook rewrites it to `rtk cargo test` before it executes. The LLM receives filtered output with only the failures — not 500 lines of passing tests. You never see or type `rtk`. + +RTK covers all major ecosystems — Git, Cargo/Rust, JavaScript, Python, Go, Ruby, .NET, Docker/Kubernetes, and more. See [What RTK Optimizes](../resources/what-rtk-covers.md) for the full list. + +## Step 3: Check your savings + +After a few commands, see how much was saved: + +```bash +rtk gain +``` + +``` +Total commands : 12 +Input tokens : 45,230 +Output tokens : 4,890 +Saved : 40,340 (89.2%) +``` + +## Step 4: Unsupported commands + +Commands RTK doesn't recognize run through passthrough — output is unchanged, usage is tracked: + +```bash +rtk proxy make install +``` + +## Next steps + +- [What RTK Optimizes](../resources/what-rtk-covers.md) — all supported commands and savings by ecosystem +- [Supported agents](./supported-agents.md) — Claude Code, Cursor, Copilot, and more +- [Configuration](./configuration.md) — customize RTK behavior diff --git a/docs/guide/getting-started/supported-agents.md b/docs/guide/getting-started/supported-agents.md new file mode 100644 index 000000000..0a1b50219 --- /dev/null +++ b/docs/guide/getting-started/supported-agents.md @@ -0,0 +1,175 @@ +--- +title: Supported Agents +description: How to integrate RTK with Claude Code, Cursor, Copilot, Cline, Windsurf, Codex, OpenCode, Kilo Code, and Antigravity +sidebar: + order: 3 +--- + +# Supported Agents + +RTK supports all major AI coding agents across 3 integration tiers. Mistral Vibe support is planned. + +## How it works + +Each agent integration intercepts CLI commands before execution and rewrites them to their RTK equivalent. The agent runs `rtk cargo test` instead of `cargo test`, sees filtered output, and uses up to 90% fewer tokens — without any change to your workflow. + +All rewrite logic lives in the RTK binary (`rtk rewrite`). Agent hooks are thin delegates that parse the agent-specific JSON format and call `rtk rewrite` for the actual decision. + +``` +Agent runs "cargo test" + -> Hook intercepts (PreToolUse / plugin event) + -> Calls rtk rewrite "cargo test" + -> Returns "rtk cargo test" + -> Agent executes filtered command + -> LLM sees 90% fewer tokens +``` + +## Supported agents + +| Agent | Integration tier | Can rewrite transparently? | +|-------|-----------------|---------------------------| +| Claude Code | Shell hook (`PreToolUse`) | Yes | +| VS Code Copilot Chat | Shell hook (`PreToolUse`) | Yes | +| GitHub Copilot CLI | Shell hook (deny-with-suggestion) | No (agent retries) | +| Cursor | Shell hook (`preToolUse`) | Yes | +| Gemini CLI | Rust binary (`BeforeTool`) | Yes | +| OpenCode | TypeScript plugin (`tool.execute.before`) | Yes | +| OpenClaw | TypeScript plugin (`before_tool_call`) | Yes | +| Cline / Roo Code | Rules file (prompt-level) | N/A | +| Windsurf | Rules file (prompt-level) | N/A | +| Codex CLI | AGENTS.md instructions | N/A | +| Kilo Code | Rules file (prompt-level) | N/A | +| Google Antigravity | Rules file (prompt-level) | N/A | +| Mistral Vibe | Planned ([#800](https://github.com/algolia/rtk/issues)) | Pending upstream | + +## Installation by agent + +### Claude Code + +```bash +rtk init --global # installs hook + patches settings.json +``` + +Restart Claude Code. Verify: + +```bash +rtk init --show # shows hook status +``` + +### Cursor + +```bash +rtk init --global --cursor +``` + +Restart Cursor. The hook uses `preToolUse` with Cursor's `updated_input` format. + +### VS Code Copilot Chat + +```bash +rtk init --global --copilot +``` + +### Gemini CLI + +```bash +rtk init --global --gemini +``` + +### OpenCode + +```bash +rtk init --global --opencode +``` + +Creates `~/.config/opencode/plugins/rtk.ts`. Uses the `tool.execute.before` hook. + +### OpenClaw + +```bash +openclaw plugins install ./openclaw +``` + +Plugin in the `openclaw/` directory. Uses the `before_tool_call` hook, delegates to `rtk rewrite`. + +### Cline / Roo Code + +```bash +rtk init --cline # creates .clinerules in current project +``` + +Cline reads `.clinerules` as custom instructions. RTK adds guidance telling Cline to prefer `rtk ` over raw commands. + +### Windsurf + +```bash +rtk init --windsurf # creates .windsurfrules in current project +``` + +### Codex CLI + +```bash +rtk init --codex # creates AGENTS.md or patches existing one +``` + +### Kilo Code + +```bash +rtk init --agent kilocode # creates .kilocode/rules/rtk-rules.md in current project +``` + +Kilo Code reads `.kilocode/rules/` as custom instructions. RTK adds guidance telling Kilo Code to prefer `rtk ` over raw commands. + +### Google Antigravity + +```bash +rtk init --agent antigravity # creates .agents/rules/antigravity-rtk-rules.md in current project +``` + +Antigravity reads `.agents/rules/` as custom instructions. RTK adds guidance telling Antigravity to prefer `rtk ` over raw commands. + +### Mistral Vibe (planned) + +Support is blocked on upstream `BeforeToolCallback` ([mistral-vibe#531](https://github.com/mistralai/mistral-vibe/issues/531)). Tracked in [#800](https://github.com/algolia/rtk/issues). + +## Integration tiers explained + +| Tier | Mechanism | How rewrites work | +|------|-----------|------------------| +| **Full hook** | Shell script or Rust binary, intercepts via agent API | Transparent — agent never sees the raw command | +| **Plugin** | TypeScript/JS in agent's plugin system | Transparent — in-place mutation | +| **Rules file** | Prompt-level instructions | Guidance only — agent is told to prefer `rtk ` | + +Rules file integrations (Cline, Windsurf, Codex, Kilo Code, Antigravity) rely on the model following instructions. Full hook integrations (Claude Code, Cursor, Gemini) are guaranteed — the command is rewritten before the agent sees it. + +## Windows support + +The shell hook (`rtk-rewrite.sh`) requires a Unix shell. On native Windows: + +- `rtk init -g` automatically falls back to **CLAUDE.md injection mode** (prompt-level instructions) +- Filters work normally (`rtk cargo test`, `rtk git status`) +- Auto-rewrite does not work — the AI assistant is instructed to use RTK but commands are not intercepted + +For full hook support on Windows, use [WSL](https://learn.microsoft.com/en-us/windows/wsl/install). Inside WSL, all agents with shell hook integration (Claude Code, Cursor, Gemini) work identically to Linux. + +## Graceful degradation + +Hooks never block command execution. If RTK is missing, the hook exits cleanly and the raw command runs unchanged: + +- RTK binary not found: warning to stderr, exit 0 +- Invalid JSON input: pass through unchanged +- RTK version too old: warning to stderr, exit 0 +- Filter logic error: fallback to raw command output + +## Override: disable RTK for one command + +```bash +RTK_DISABLED=1 git status # runs raw git status, no rewrite +``` + +Or exclude commands permanently in `~/.config/rtk/config.toml`: + +```toml +[hooks] +exclude_commands = ["git rebase", "git cherry-pick"] +``` diff --git a/docs/guide/index.md b/docs/guide/index.md new file mode 100644 index 000000000..4ef74a532 --- /dev/null +++ b/docs/guide/index.md @@ -0,0 +1,64 @@ +--- +title: RTK Documentation +description: RTK (Rust Token Killer) — reduce LLM token consumption by 60-90% on common dev commands, with zero workflow changes +sidebar: + order: 1 +--- + +# RTK — Rust Token Killer + +RTK is a CLI proxy that sits between your AI assistant and your development tools. It filters command output before it reaches the LLM, keeping only what matters and discarding boilerplate, progress bars, and noise. + +**Result:** 60-90% fewer tokens consumed per command, without changing how you work. You run `git status` as usual — RTK's hook intercepts it, filters the output, and the LLM sees a compact 3-line summary instead of 40 lines. + +## How it works + +``` +Your AI assistant runs: git status + ↓ + Hook intercepts (PreToolUse) + ↓ + rtk git status (transparent rewrite) + ↓ + Raw output: 40 lines → Filtered: 3 lines + ~800 tokens → ~60 tokens (92% saved) + ↓ + LLM sees the compact output +``` + +Zero config changes to your workflow. The hook handles everything automatically. + +## What RTK optimizes + +Dozens of commands across all major ecosystems — Git, Cargo/Rust, JavaScript, Python, Go, Ruby, .NET, Docker/Kubernetes, and more. See [What RTK Optimizes](./resources/what-rtk-covers.md) for the full list with savings percentages. + +## Get started + +1. **[Installation](./getting-started/installation.md)** — Install RTK and verify you have the right package +2. **[Quick Start](./getting-started/quick-start.md)** — Connect to your AI assistant in 5 minutes +3. **[Supported Agents](./getting-started/supported-agents.md)** — Claude Code, Cursor, Copilot, Gemini, and more + +## Measure your savings + +```bash +rtk gain # total savings across all sessions +rtk gain --daily # day-by-day breakdown +rtk gain --weekly # weekly aggregation +``` + +See [Token Savings Analytics](./analytics/gain.md) for export formats and analysis workflows. + +## Analyze your usage + +```bash +rtk discover # find commands that ran without RTK (missed savings) +rtk session # RTK adoption rate per Claude Code session +``` + +See [Discover and Session](./analytics/discover.md) for details. + +## Further reading + +- [Configuration](./getting-started/configuration.md) — config.toml, global flags, env vars, tee recovery +- [Troubleshooting](./resources/troubleshooting.md) — common issues and fixes +- [ARCHITECTURE.md](../contributing/ARCHITECTURE.md) — system design for contributors diff --git a/docs/guide/resources/troubleshooting.md b/docs/guide/resources/troubleshooting.md new file mode 100644 index 000000000..09bb608fb --- /dev/null +++ b/docs/guide/resources/troubleshooting.md @@ -0,0 +1,184 @@ +--- +title: Troubleshooting +description: Common RTK issues and how to fix them +sidebar: + order: 2 +--- + +# Troubleshooting + +## `rtk gain` says "not a rtk command" + +**Symptom:** +```bash +$ rtk gain +rtk: 'gain' is not a rtk command. See 'rtk --help'. +``` + +**Cause:** You installed **Rust Type Kit** (`reachingforthejack/rtk`) instead of **Rust Token Killer** (`algolia/rtk`). They share the same binary name. + +**Fix:** +```bash +cargo uninstall rtk +curl -fsSL https://raw.githubusercontent.com/algolia/rtk/main/install.sh | sh +rtk gain # should now show token savings stats +``` + +## How to tell which rtk you have + +| If `rtk gain`... | You have | +|------------------|----------| +| Shows token savings dashboard | Rust Token Killer ✅ | +| Returns "not a rtk command" | Rust Type Kit ❌ | + +## AI assistant not using RTK + +**Symptom:** Claude Code (or another agent) runs `cargo test` instead of `rtk cargo test`. + +**Checklist:** + +1. Verify RTK is installed: + ```bash + rtk --version + rtk gain + ``` + +2. Initialize the hook: + ```bash + rtk init --global # Claude Code + rtk init --global --cursor # Cursor + rtk init --global --opencode # OpenCode + ``` + +3. Restart your AI assistant. + +4. Verify hook status: + ```bash + rtk init --show + ``` + +5. Check `settings.json` has the hook registered (Claude Code): + ```bash + cat ~/.claude/settings.json | grep rtk + ``` + +## RTK not found after `cargo install` + +**Symptom:** +```bash +$ rtk --version +zsh: command not found: rtk +``` + +**Cause:** `~/.cargo/bin` is not in your PATH. + +**Fix:** + +For bash (`~/.bashrc`) or zsh (`~/.zshrc`): +```bash +export PATH="$HOME/.cargo/bin:$PATH" +``` + +For fish (`~/.config/fish/config.fish`): +```fish +set -gx PATH $HOME/.cargo/bin $PATH +``` + +Then reload: +```bash +source ~/.zshrc # or ~/.bashrc +rtk --version +``` + +## RTK on Windows + +### Double-clicking rtk.exe does nothing + +**Symptom:** You double-click `rtk.exe`, a terminal flashes and closes instantly. + +**Cause:** RTK is a command-line tool. With no arguments, it prints usage and exits. The console window opens and closes before you can read anything. + +**Fix:** Open a terminal first, then run RTK from there: +- Press `Win+R`, type `cmd`, press Enter +- Or open PowerShell or Windows Terminal +- Then run: `rtk --version` + +### Hook not working (no auto-rewrite) + +**Symptom:** `rtk init -g` shows "Falling back to --claude-md mode" on Windows. + +**Cause:** The auto-rewrite hook (`rtk-rewrite.sh`) requires a Unix shell. Native Windows doesn't have one. + +**Fix:** Use [WSL](https://learn.microsoft.com/en-us/windows/wsl/install) for full hook support: +```bash +# Inside WSL +curl -fsSL https://raw.githubusercontent.com/algolia/rtk/refs/heads/main/install.sh | sh +rtk init -g # full hook mode works in WSL +``` + +On native Windows, RTK falls back to CLAUDE.md injection. Your AI assistant gets RTK instructions but won't auto-rewrite commands. It can still use RTK manually: `rtk cargo test`, `rtk git status`, etc. + +### Node.js tools not found + +**Symptom:** +``` +rtk vitest --run +Error: program not found +``` + +**Cause:** On Windows, Node.js tools are installed as `.CMD`/`.BAT` wrappers. Older RTK versions couldn't find them. + +**Fix:** Update to RTK v0.23.1+: +```bash +cargo install --git https://github.com/algolia/rtk +rtk --version # should be 0.23.1+ +``` + +## Compilation error during installation + +```bash +rustup update stable +rustup default stable +cargo clean +cargo build --release +cargo install --path . --force +``` + +Minimum required Rust version: 1.70+. + +## OpenCode not using RTK + +```bash +rtk init --global --opencode +# restart OpenCode +rtk init --show # should show "OpenCode: plugin installed" +``` + +## `cargo install rtk` installs the wrong package + +If Rust Type Kit is published to crates.io under the name `rtk`, `cargo install rtk` may install the wrong one. + +Always use the explicit URL: + +```bash +cargo install --git https://github.com/algolia/rtk +``` + +## Run the diagnostic script + +From the RTK repository root: + +```bash +bash scripts/check-installation.sh +``` + +Checks: +- RTK installed and in PATH +- Correct version (Token Killer, not Type Kit) +- Available features +- Claude Code integration +- Hook status + +## Still stuck? + +Open an issue: https://github.com/algolia/rtk/issues diff --git a/docs/guide/resources/what-rtk-covers.md b/docs/guide/resources/what-rtk-covers.md new file mode 100644 index 000000000..dd5c39e89 --- /dev/null +++ b/docs/guide/resources/what-rtk-covers.md @@ -0,0 +1,157 @@ +--- +title: What RTK Optimizes +description: Commands and ecosystems automatically optimized by RTK with typical token savings +sidebar: + order: 1 +--- + +# What RTK Optimizes + +Once RTK is installed with a hook, these commands are automatically intercepted and filtered. You run them normally — the hook rewrites them transparently before execution. + +Typical savings: 60-99%. + +## Git + +| Command | Savings | What changes | +|---------|---------|--------------| +| `git status` | 75-93% | Compact stat format, grouped by state | +| `git log` | 80-92% | Hash + author + subject only | +| `git diff` | 70% | Context reduced, headers stripped | +| `git show` | 70% | Same as diff | +| `git stash list` | 75% | Compact one-line per entry | + +## GitHub CLI + +| Command | Savings | What changes | +|---------|---------|--------------| +| `gh pr view` | 87% | Removes ASCII art and verbose metadata | +| `gh pr checks` | 79% | Status + name only, failures highlighted | +| `gh run list` | 82% | Compact workflow run summary | +| `gh issue view` | 80% | Body only, no decoration | + +## Graphite (Stacked PRs) + +| Command | Savings | What changes | +|---------|---------|--------------| +| `gt log` | 75% | Stack summary only | +| `gt status` | 70% | Current branch context | + +## Cargo / Rust + +| Command | Savings | What changes | +|---------|---------|--------------| +| `cargo test` | 90% | Failures only, passed tests suppressed | +| `cargo nextest` | 90% | Same as test | +| `cargo build` | 80% | Errors and warnings only | +| `cargo check` | 80% | Errors and warnings only | +| `cargo clippy` | 80% | Lint warnings grouped by file | + +## JavaScript / TypeScript + +| Command | Savings | What changes | +|---------|---------|--------------| +| `jest` | 94-99% | Failures only | +| `vitest` | 94-99% | Failures only | +| `tsc` | 75% | Type errors grouped by file | +| `eslint` | 84% | Violations grouped by rule | +| `pnpm list` | 70-90% | Compact dependency tree | +| `pnpm outdated` | 70% | Package + current + latest only | +| `next build` | 80% | Route summary + errors only | +| `prisma migrate` | 75% | Migration status only | +| `playwright test` | 90% | Failures + trace links only | + +## Python + +| Command | Savings | What changes | +|---------|---------|--------------| +| `pytest` | 80-90% | Failures only | +| `ruff check` | 75% | Violations grouped by file | +| `mypy` | 75% | Type errors grouped by file | +| `pip install` | 70% | Installed packages only, progress stripped | + +## Go + +| Command | Savings | What changes | +|---------|---------|--------------| +| `go test` | 80-90% | Failures only | +| `golangci-lint run` | 75% | Violations grouped by file | +| `go build` | 75% | Errors only | + +## Ruby + +| Command | Savings | What changes | +|---------|---------|--------------| +| `rspec` | 80-90% | Failures only | +| `rubocop` | 75% | Offenses grouped by file | +| `rake` | 70% | Task output, build errors highlighted | + +## .NET + +| Command | Savings | What changes | +|---------|---------|--------------| +| `dotnet build` | 80% | Errors and warnings only | +| `dotnet test` | 85-90% | Failures only | +| `dotnet format` | 75% | Changed files only | + +## Docker / Kubernetes + +| Command | Savings | What changes | +|---------|---------|--------------| +| `docker ps` | 65% | Essential columns (name, image, status, port) | +| `docker images` | 60% | Name + tag + size only | +| `docker logs` | 70% | Deduplicated, last N lines | +| `docker compose up` | 75% | Service status, errors highlighted | +| `kubectl get pods` | 65% | Name + status + restarts only | +| `kubectl logs` | 70% | Deduplicated entries | + +## Files and Search + +| Command | Savings | What changes | +|---------|---------|--------------| +| `ls` | 80% | Tree format with file counts | +| `find` | 75% | Tree format | +| `grep` | 70% | Truncated lines, grouped by file | +| `diff` | 65% | Context reduced | +| `wc` | 60% | Compact counts | +| `cat` / `head` / `tail ` | 60-80% | Smart file reading via `rtk read` | +| `rtk smart ` | 85% | 2-line heuristic code summary (signatures only) | + +## Cloud and Data + +| Command | Savings | What changes | +|---------|---------|--------------| +| `aws` | 70% | JSON condensed, relevant fields only | +| `psql` | 65% | Query results without decoration | +| `curl` | 60% | Response body only, headers stripped | + +## Global flags + +These flags apply to all RTK commands and can push savings even higher: + +| Flag | Description | +|------|-------------| +| `--ultra-compact` | ASCII icons, inline format — extra token reduction on top of normal filtering | +| `-v` / `--verbose` | Show filtering details on stderr (`-v`, `-vv`, `-vvv` for increasing detail) | + +```bash +# Ultra-compact: even smaller output +rtk git log --ultra-compact + +# Debug: see what RTK is doing +rtk git status -vvv +``` + +:::note +Use `--ultra-compact` (long form) rather than `-u` when working with Git commands. Git's own `-u` flag means `--set-upstream` and the short form can cause confusion. +::: + +## Commands that are not rewritten + +If a command isn't in the list above, RTK runs it through passthrough — the output reaches the LLM unchanged. You can explicitly track unsupported commands: + +```bash +rtk proxy make install # runs make install, tracks usage, no filtering +``` + +To check which commands were missed opportunities: `rtk discover`. diff --git a/docs/images/gain-dashboard.jpg b/docs/images/gain-dashboard.jpg deleted file mode 100644 index 0d57d7b0b..000000000 Binary files a/docs/images/gain-dashboard.jpg and /dev/null differ diff --git a/docs/maintainers/MAINTAINERS_APPLY.md b/docs/maintainers/MAINTAINERS_APPLY.md new file mode 100644 index 000000000..3503f0bb3 --- /dev/null +++ b/docs/maintainers/MAINTAINERS_APPLY.md @@ -0,0 +1,72 @@ +# RTK Maintainers Application + +RTK is growing fast, with more contributors, PRs, and ideas than ever. To keep things moving smoothly, we're looking for new maintainers. + +We've introduced two types of maintainers to progressively build a clean process and strong collaboration between contributors. +For now, we're starting by recruiting **Ecosystem Maintainers** only. As the project evolves, we'll soon begin accepting **Core Maintainers** as well. + +> Maintainers are expected to be active and involved over time, not just occasional contributors. + +--- + +## How to apply guide + +#### ✅ Requirements + +To apply, you should have: + +- 3+ merged PRs to RTK (filters, fixes, docs — all contributions count) +- 3+ PR reviews with helpful, constructive feedback + +--- + +### ✍️ How to Apply + +1. Open a discussion in [algolia/rtk Maintainers Applications · Discussions · GitHub](https://github.com/algolia/rtk/discussions/categories/maintainers-applications) titled **Maintainer Application: [Your GitHub Handle]** +2. In your application, include: + - The ecosystem(s) you're interested in + - Your experience with those ecosystems + - Links to your merged PRs and reviews + - Your Discord username (and make sure you've joined the server) + - Your PRs that have been accepted in RTK +3. For **Core Maintainer** applications, also include: + - Your experience with Rust + - Your experience with Open Source +4. A Core Maintainer will get back to you as soon as possible +5. If it's a good fit, we'll continue the conversation on Discord and guide you through the next steps + +--- + +### 👀 What to Expect + +- A review of your ecosystem experience and understanding of RTK concepts +- A discussion with current maintainers +- Introduction to the team + +--- + +## What Maintainers Do + +### 🌱 Ecosystem Maintainers + +Ecosystem Maintainers are responsible for specific environments inside the `cmds/` folder (e.g. `git`, `system`, etc.). They own and manage their ecosystem end-to-end: + +- Responsible for the quality of filters +- Review and ensure quality of contributions +- Maintain consistency with the rest of the RTK ecosystem +- Help shape and grow their specific domain +- Handle issues and PRs related to their environment *(security and quality review from core maintainers still required for release)* + +### 🔧 Core Maintainers (once we've fully integrated some Ecosystem Maintainers) + +Core Maintainers are responsible for the core of RTK. They have a broader scope and higher responsibilities and permissions, including: + +- Maintaining core functionalities and architecture +- Reviewing and merging PRs for release with the core team +- Defining project direction and standards with the core team +- Ensuring consistency across the entire project +- Refactoring for optimization, standardization & conformity + +--- + +If you enjoy contributing and want to help RTK scale in a healthy way, we'd be excited to have you onboard 🚀 diff --git a/docs/AUDIT_GUIDE.md b/docs/usage/AUDIT_GUIDE.md similarity index 94% rename from docs/AUDIT_GUIDE.md rename to docs/usage/AUDIT_GUIDE.md index 8bcebdffe..b641f2450 100644 --- a/docs/AUDIT_GUIDE.md +++ b/docs/usage/AUDIT_GUIDE.md @@ -29,6 +29,10 @@ rtk gain --all --format csv > savings.csv # Combined flags rtk gain --graph --history --quota # Classic view with extras rtk gain --daily --weekly --monthly # Multiple breakdowns + +# Reset all tracking data +rtk gain --reset # prompts [y/N] before deleting +rtk gain --reset --yes # skip prompt (CI/scripts) ``` ## Command Options @@ -51,6 +55,15 @@ rtk gain --daily --weekly --monthly # Multiple breakdowns | `--quota` | Monthly quota analysis (Pro/5x/20x tiers) | | `--tier ` | Quota tier: pro, 5x, 20x (default: 20x) | +### Reset Flag + +| Flag | Description | +|------|-------------| +| `--reset` | Permanently delete all tracking data (commands + parse failures) | +| `--yes` | Skip the confirmation prompt (for CI/scripts) | + +> **Warning**: `--reset` is irreversible. It clears both the `commands` and `parse_failures` tables atomically. A `[y/N]` confirmation prompt is shown by default. In non-interactive environments (piped stdin), it defaults to `N` unless `--yes` is passed. + ### Export Formats | Format | Flag | Use Case | @@ -267,7 +280,8 @@ Savings % = (Saved / Input) × 100 |---------|----------------|-----------| | `rtk git status` | 77-93% | Compact stat format | | `rtk eslint` | 84% | Group by rule | -| `rtk vitest run` | 94-99% | Show failures only | +| `rtk jest` | 94-99% | Show failures only | +| `rtk vitest` | 94-99% | Show failures only | | `rtk find` | 75% | Tree format | | `rtk pnpm list` | 70-90% | Compact dependencies | | `rtk grep` | 70% | Truncate + group | @@ -429,4 +443,4 @@ print(f'rtk estimate: {len(text) // 4}') - [README.md](../README.md) - Full rtk documentation - [CLAUDE.md](../CLAUDE.md) - Claude Code integration guide -- [ARCHITECTURE.md](../ARCHITECTURE.md) - Technical architecture +- [ARCHITECTURE.md](../contributing/ARCHITECTURE.md) - Technical architecture diff --git a/docs/FEATURES.md b/docs/usage/FEATURES.md similarity index 97% rename from docs/FEATURES.md rename to docs/usage/FEATURES.md index 061a604a9..901288565 100644 --- a/docs/FEATURES.md +++ b/docs/usage/FEATURES.md @@ -25,7 +25,6 @@ Binaire Rust unique, zero dependances externes, overhead < 10ms par commande. 15. [Systeme de hooks](#systeme-de-hooks) 16. [Configuration](#configuration) 17. [Systeme Tee (recuperation de sortie)](#systeme-tee) -18. [Telemetrie](#telemetrie) --- @@ -576,12 +575,13 @@ Filtre la sortie de `cargo nextest` pour n'afficher que les echecs. --- -### `rtk vitest run` -- Tests Vitest +### `rtk jest` / `rtk vitest` -- Tests Jest/Vitest **Economies :** ~99.5% ```bash -rtk vitest run [args...] +rtk jest [args...] +rtk vitest [args...] ``` --- @@ -963,13 +963,13 @@ Les lignes repetees sont fusionnees : `[ERROR] Connection refused (x42)`. --- -### `rtk curl` -- HTTP avec detection JSON +### `rtk curl` -- HTTP avec troncature ```bash rtk curl [args...] ``` -Auto-detecte les reponses JSON et affiche le schema au lieu du contenu complet. +Tronque les reponses longues et sauvegarde la sortie complete dans un fichier pour recuperation. --- @@ -1258,7 +1258,8 @@ rtk verify | `ls` | `rtk ls` | | `tree` | `rtk tree` | | `wc` | `rtk wc` | -| `vitest/jest` | `rtk vitest run` | +| `jest` | `rtk jest` | +| `vitest` | `rtk vitest` | | `tsc` | `rtk tsc` | | `eslint/biome` | `rtk lint` | | `prettier` | `rtk prettier` | @@ -1321,9 +1322,6 @@ mode = "failures" # "failures" (defaut), "always", ou "never" max_files = 20 # Rotation : garder les N derniers fichiers # directory = "/custom/tee/path" # Chemin personnalise (optionnel) -[telemetry] -enabled = true # Telemetrie anonyme (1 ping/jour, opt-out possible) - [hooks] exclude_commands = [] # Commandes a exclure de la recriture automatique ``` @@ -1333,7 +1331,6 @@ exclude_commands = [] # Commandes a exclure de la recriture automatique | Variable | Description | |----------|-------------| | `RTK_TEE_DIR` | Surcharge le repertoire tee | -| `RTK_TELEMETRY_DISABLED=1` | Desactiver la telemetrie | | `RTK_HOOK_AUDIT=1` | Activer l'audit du hook | | `SKIP_ENV_VALIDATION=1` | Desactiver la validation d'env (Next.js, etc.) | @@ -1369,26 +1366,6 @@ FAILED: 2/15 tests --- -## Telemetrie - -RTK envoie un ping anonyme une fois par jour (23h d'intervalle) pour des statistiques d'utilisation. - -**Donnees envoyees :** hash de device, version, OS, architecture, nombre de commandes/24h, top commandes, pourcentage d'economies. - -**Desactiver :** -```bash -# Via variable d'environnement -export RTK_TELEMETRY_DISABLED=1 - -# Via config.toml -[telemetry] -enabled = false -``` - -Aucune donnee personnelle, aucun contenu de commande, aucun chemin de fichier n'est transmis. - ---- - ## Resume des economies par categorie | Categorie | Commandes | Economies typiques | diff --git a/docs/tracking.md b/docs/usage/TRACKING.md similarity index 97% rename from docs/tracking.md rename to docs/usage/TRACKING.md index 82c12883d..6091893c9 100644 --- a/docs/tracking.md +++ b/docs/usage/TRACKING.md @@ -369,7 +369,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Install RTK - run: cargo install --git https://github.com/rtk-ai/rtk + run: cargo install --git https://github.com/algolia/rtk - name: Export weekly stats run: | @@ -444,7 +444,7 @@ if __name__ == "__main__": ```rust // In your Cargo.toml // [dependencies] -// rtk = { git = "https://github.com/rtk-ai/rtk" } +// rtk = { git = "https://github.com/algolia/rtk" } use rtk::tracking::{Tracker, TimedExecution}; use anyhow::Result; @@ -538,8 +538,8 @@ let _ = conn.execute( ## Security & Privacy -- **Local storage only**: Tracking database never leaves the machine -- **Telemetry enabled by default**: RTK sends a daily anonymous usage ping (version, OS, command counts, token savings). Device identity is a salted SHA-256 hash. Opt out with `RTK_TELEMETRY_DISABLED=1` or `[telemetry] enabled = false` in `~/.config/rtk/config.toml` +- **Local storage only**: Tracking database never leaves the machine. The + Algolia fork ships with telemetry stripped — there is no remote ping. - **User control**: Users can delete `~/.local/share/rtk/tracking.db` anytime - **90-day retention**: Old data automatically purged diff --git a/hooks/README.md b/hooks/README.md index 9d5e63809..6a6744281 100644 --- a/hooks/README.md +++ b/hooks/README.md @@ -38,7 +38,7 @@ Each agent subdirectory has its own README with hook-specific details: - **[`cursor/`](cursor/README.md)** — Shell hook, Cursor JSON format, empty `{}` response requirement - **[`cline/`](cline/README.md)** — Rules file (prompt-level), `.clinerules` project-local installation - **[`windsurf/`](windsurf/README.md)** — Rules file (prompt-level), `.windsurfrules` workspace-scoped -- **[`codex/`](codex/README.md)** — Awareness document, `AGENTS.md` integration, `~/.codex/` location +- **[`codex/`](codex/README.md)** — Awareness document, `AGENTS.md` integration, `$CODEX_HOME` or `~/.codex/` location - **[`opencode/`](opencode/README.md)** — TypeScript plugin, `zx` library, `tool.execute.before` event, in-place mutation ## Supported Agents @@ -184,7 +184,7 @@ Example: `cargo fmt --all && cargo test` becomes `rtk cargo fmt --all && rtk car ### Override Controls - **`RTK_DISABLED=1`**: Per-command override (`RTK_DISABLED=1 git status` runs raw) -- **`exclude_commands`**: In `~/.config/rtk/config.toml`, list commands to never rewrite +- **`exclude_commands`**: In `~/.config/rtk/config.toml`, list commands to never rewrite. Matches against the full command after stripping env prefixes. Subcommand patterns work (`"git push"` excludes `git push origin main`). Patterns starting with `^` are treated as regex. - **Already-RTK**: `rtk git status` passes through unchanged (no `rtk rtk git`) ## Exit Code Contract diff --git a/hooks/antigravity/README.md b/hooks/antigravity/README.md new file mode 100644 index 000000000..0b279254b --- /dev/null +++ b/hooks/antigravity/README.md @@ -0,0 +1,9 @@ +# Google Antigravity Hooks + +> Part of [`hooks/`](../README.md) — see also [`src/hooks/`](../../src/hooks/README.md) for installation code + +## Specifics + +- Prompt-level guidance only (no programmatic hook) -- relies on Antigravity reading custom instructions +- `rules.md` contains the instruction to prefix all shell commands with `rtk`, usage examples, and meta commands +- Installed to `.agents/rules/antigravity-rtk-rules.md` (project-local) by `rtk init --agent antigravity` diff --git a/hooks/antigravity/rules.md b/hooks/antigravity/rules.md new file mode 100644 index 000000000..8ee44bcd4 --- /dev/null +++ b/hooks/antigravity/rules.md @@ -0,0 +1,32 @@ +# RTK - Rust Token Killer (Google Antigravity) + +**Usage**: Token-optimized CLI proxy for shell commands. + +## Rule + +Always prefix shell commands with `rtk` to minimize token consumption. + +Examples: + +```bash +rtk git status +rtk cargo test +rtk ls src/ +rtk grep "pattern" src/ +rtk find "*.rs" . +rtk docker ps +rtk gh pr list +``` + +## Meta Commands + +```bash +rtk gain # Show token savings +rtk gain --history # Command history with savings +rtk discover # Find missed RTK opportunities +rtk proxy # Run raw (no filtering, for debugging) +``` + +## Why + +RTK filters and compresses command output before it reaches the LLM context, saving 60-90% tokens on common operations. Always use `rtk ` instead of raw commands. diff --git a/hooks/claude/rtk-rewrite.sh b/hooks/claude/rtk-rewrite.sh index f7a42b5d4..e2d806b14 100644 --- a/hooks/claude/rtk-rewrite.sh +++ b/hooks/claude/rtk-rewrite.sh @@ -19,25 +19,33 @@ if ! command -v jq &>/dev/null; then fi if ! command -v rtk &>/dev/null; then - echo "[rtk] WARNING: rtk is not installed or not in PATH. Hook cannot rewrite commands. Install: https://github.com/rtk-ai/rtk#installation" >&2 + echo "[rtk] WARNING: rtk is not installed or not in PATH. Hook cannot rewrite commands. Install: https://github.com/algolia/rtk#installation" >&2 exit 0 fi # Version guard: rtk rewrite was added in 0.23.0. # Older binaries: warn once and exit cleanly (no silent failure). -RTK_VERSION=$(rtk --version 2>/dev/null | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1) -if [ -n "$RTK_VERSION" ]; then - MAJOR=$(echo "$RTK_VERSION" | cut -d. -f1) - MINOR=$(echo "$RTK_VERSION" | cut -d. -f2) - # Require >= 0.23.0 - if [ "$MAJOR" -eq 0 ] && [ "$MINOR" -lt 23 ]; then - echo "[rtk] WARNING: rtk $RTK_VERSION is too old (need >= 0.23.0). Upgrade: cargo install rtk" >&2 - exit 0 +# Cache the version check to avoid spawning multiple processes on every hook call. +CACHE_DIR=${XDG_CACHE_HOME:-$HOME/.cache} +CACHE_FILE="$CACHE_DIR/rtk-hook-version-ok" +if [ ! -f "$CACHE_FILE" ]; then + RTK_VERSION_RAW=$(rtk --version 2>/dev/null) + RTK_VERSION=${RTK_VERSION_RAW#rtk } + RTK_VERSION=${RTK_VERSION%% *} + if [ -n "$RTK_VERSION" ]; then + IFS=. read -r MAJOR MINOR PATCH <<<"$RTK_VERSION" + # Require >= 0.23.0 + if [ "$MAJOR" -eq 0 ] && [ "$MINOR" -lt 23 ]; then + echo "[rtk] WARNING: rtk $RTK_VERSION is too old (need >= 0.23.0). Upgrade: cargo install rtk" >&2 + exit 0 + fi fi + mkdir -p "$CACHE_DIR" 2>/dev/null + touch "$CACHE_FILE" 2>/dev/null fi INPUT=$(cat) -CMD=$(echo "$INPUT" | jq -r '.tool_input.command // empty') +CMD=$(jq -r '.tool_input.command // empty' <<<"$INPUT") if [ -z "$CMD" ]; then exit 0 @@ -70,29 +78,24 @@ case $EXIT_CODE in ;; esac -ORIGINAL_INPUT=$(echo "$INPUT" | jq -c '.tool_input') -UPDATED_INPUT=$(echo "$ORIGINAL_INPUT" | jq --arg cmd "$REWRITTEN" '.command = $cmd') - if [ "$EXIT_CODE" -eq 3 ]; then # Ask: rewrite the command, omit permissionDecision so Claude Code prompts. - jq -n \ - --argjson updated "$UPDATED_INPUT" \ - '{ + jq -c --arg cmd "$REWRITTEN" \ + '.tool_input.command = $cmd | { "hookSpecificOutput": { "hookEventName": "PreToolUse", - "updatedInput": $updated + "updatedInput": .tool_input } - }' + }' <<<"$INPUT" else # Allow: rewrite the command and auto-allow. - jq -n \ - --argjson updated "$UPDATED_INPUT" \ - '{ + jq -c --arg cmd "$REWRITTEN" \ + '.tool_input.command = $cmd | { "hookSpecificOutput": { "hookEventName": "PreToolUse", "permissionDecision": "allow", "permissionDecisionReason": "RTK auto-rewrite", - "updatedInput": $updated + "updatedInput": .tool_input } - }' + }' <<<"$INPUT" fi diff --git a/hooks/claude/test-rtk-rewrite.sh b/hooks/claude/test-rtk-rewrite.sh index 85103163b..702fe9299 100644 --- a/hooks/claude/test-rtk-rewrite.sh +++ b/hooks/claude/test-rtk-rewrite.sh @@ -117,6 +117,10 @@ test_rewrite "npx prisma migrate" \ "npx prisma migrate" \ "rtk prisma migrate" +test_rewrite "rtk git status" \ + "rtk git status" \ + "rtk git status" + echo "" # ---- SECTION 2: Env var prefix handling (THE BIG FIX) ---- @@ -134,8 +138,8 @@ test_rewrite "env + git log" \ "GIT_PAGER=cat rtk git log --oneline -10" test_rewrite "multi env + vitest" \ - "NODE_ENV=test CI=1 npx vitest run" \ - "NODE_ENV=test CI=1 rtk vitest run" + "NODE_ENV=test CI=1 npx vitest" \ + "NODE_ENV=test CI=1 rtk vitest" test_rewrite "env + ls" \ "LANG=C ls -la" \ @@ -143,7 +147,7 @@ test_rewrite "env + ls" \ test_rewrite "env + npm run" \ "NODE_ENV=test npm run test:e2e" \ - "NODE_ENV=test rtk npm test:e2e" + "NODE_ENV=test rtk npm run test:e2e" test_rewrite "env + docker compose (unsupported subcommand, NOT rewritten)" \ "COMPOSE_PROJECT_NAME=test docker compose up -d" \ @@ -159,23 +163,15 @@ echo "" echo "--- New patterns ---" test_rewrite "npm run test:e2e" \ "npm run test:e2e" \ - "rtk npm test:e2e" + "rtk npm run test:e2e" test_rewrite "npm run build" \ "npm run build" \ - "rtk npm build" + "rtk npm run build" -test_rewrite "npm test" \ - "npm test" \ - "rtk npm test" - -test_rewrite "vue-tsc -b" \ - "vue-tsc -b" \ - "rtk tsc -b" - -test_rewrite "npx vue-tsc --noEmit" \ - "npx vue-tsc --noEmit" \ - "rtk tsc --noEmit" +test_rewrite "npm jest run" \ + "npm jest run" \ + "rtk jest" test_rewrite "docker compose up -d (NOT rewritten — unsupported by rtk)" \ "docker compose up -d" \ @@ -209,17 +205,17 @@ test_rewrite "docker exec -it db psql" \ "docker exec -it db psql" \ "rtk docker exec -it db psql" -test_rewrite "find (NOT rewritten — different arg format)" \ +test_rewrite "find . -name '*.ts'" \ "find . -name '*.ts'" \ - "" + "rtk find . -name '*.ts'" -test_rewrite "tree (NOT rewritten — different arg format)" \ +test_rewrite "tree src/" \ "tree src/" \ - "" + "rtk tree src/" -test_rewrite "wget (NOT rewritten — different arg format)" \ +test_rewrite "wget https://example.com/file" \ "wget https://example.com/file" \ - "" + "rtk wget https://example.com/file" test_rewrite "gh api repos/owner/repo" \ "gh api repos/owner/repo" \ @@ -281,32 +277,28 @@ echo "" echo "--- Vitest run dedup ---" test_rewrite "vitest (no args)" \ "vitest" \ - "rtk vitest run" + "rtk vitest" -test_rewrite "vitest run (no double run)" \ +test_rewrite "vitest run (no run)" \ "vitest run" \ - "rtk vitest run" + "rtk vitest" -test_rewrite "vitest run --reporter" \ - "vitest run --reporter=verbose" \ - "rtk vitest run --reporter=verbose" +test_rewrite "vitest --reporter" \ + "vitest --reporter=verbose" \ + "rtk vitest --reporter=verbose" -test_rewrite "npx vitest run" \ - "npx vitest run" \ - "rtk vitest run" +test_rewrite "npx vitest" \ + "npx vitest" \ + "rtk vitest" -test_rewrite "pnpm vitest run --coverage" \ - "pnpm vitest run --coverage" \ - "rtk vitest run --coverage" +test_rewrite "pnpm vitest --coverage" \ + "pnpm vitest --coverage" \ + "rtk vitest --coverage" echo "" # ---- SECTION 5: Should NOT rewrite ---- echo "--- Should NOT rewrite ---" -test_rewrite "already rtk" \ - "rtk git status" \ - "" - test_rewrite "heredoc" \ "cat <<'EOF' hello diff --git a/hooks/codex/README.md b/hooks/codex/README.md index e922e6365..50030e958 100644 --- a/hooks/codex/README.md +++ b/hooks/codex/README.md @@ -6,4 +6,4 @@ - Prompt-level guidance via awareness document -- no programmatic hook - `rtk-awareness.md` is injected into `AGENTS.md` with an `@RTK.md` reference -- Installed to `~/.codex/` by `rtk init --codex` +- Installed to `$CODEX_HOME` when set, otherwise `~/.codex/`, by `rtk init --codex` diff --git a/hooks/cursor/rtk-rewrite.sh b/hooks/cursor/rtk-rewrite.sh index 4b80b260c..658697554 100644 --- a/hooks/cursor/rtk-rewrite.sh +++ b/hooks/cursor/rtk-rewrite.sh @@ -15,7 +15,7 @@ if ! command -v jq &>/dev/null; then fi if ! command -v rtk &>/dev/null; then - echo "[rtk] WARNING: rtk is not installed or not in PATH. Hook cannot rewrite commands. Install: https://github.com/rtk-ai/rtk#installation" >&2 + echo "[rtk] WARNING: rtk is not installed or not in PATH. Hook cannot rewrite commands. Install: https://github.com/algolia/rtk#installation" >&2 exit 0 fi diff --git a/hooks/kilocode/README.md b/hooks/kilocode/README.md new file mode 100644 index 000000000..2b38d9427 --- /dev/null +++ b/hooks/kilocode/README.md @@ -0,0 +1,9 @@ +# Kilo Code Hooks + +> Part of [`hooks/`](../README.md) — see also [`src/hooks/`](../../src/hooks/README.md) for installation code + +## Specifics + +- Prompt-level guidance only (no programmatic hook) -- relies on Kilo Code reading custom instructions +- `rules.md` contains the instruction to prefix all shell commands with `rtk`, usage examples, and meta commands +- Installed to `.kilocode/rules/rtk-rules.md` (project-local) by `rtk init --agent kilocode` diff --git a/hooks/kilocode/rules.md b/hooks/kilocode/rules.md new file mode 100644 index 000000000..53d764596 --- /dev/null +++ b/hooks/kilocode/rules.md @@ -0,0 +1,32 @@ +# RTK - Rust Token Killer (Kilo Code) + +**Usage**: Token-optimized CLI proxy for shell commands. + +## Rule + +Always prefix shell commands with `rtk` to minimize token consumption. + +Examples: + +```bash +rtk git status +rtk cargo test +rtk ls src/ +rtk grep "pattern" src/ +rtk find "*.rs" . +rtk docker ps +rtk gh pr list +``` + +## Meta Commands + +```bash +rtk gain # Show token savings +rtk gain --history # Command history with savings +rtk discover # Find missed RTK opportunities +rtk proxy # Run raw (no filtering, for debugging) +``` + +## Why + +RTK filters and compresses command output before it reaches the LLM context, saving 60-90% tokens on common operations. Always use `rtk ` instead of raw commands. diff --git a/install.sh b/install.sh index 1654245d9..ceddc987e 100644 --- a/install.sh +++ b/install.sh @@ -1,10 +1,10 @@ #!/usr/bin/env sh -# rtk installer - https://github.com/rtk-ai/rtk -# Usage: curl -fsSL https://raw.githubusercontent.com/rtk-ai/rtk/refs/heads/master/install.sh | sh +# rtk installer - https://github.com/algolia/rtk +# Usage: curl -fsSL https://raw.githubusercontent.com/algolia/rtk/refs/heads/main/install.sh | sh set -e -REPO="rtk-ai/rtk" +REPO="algolia/rtk" BINARY_NAME="rtk" INSTALL_DIR="${RTK_INSTALL_DIR:-$HOME/.local/bin}" @@ -46,10 +46,25 @@ detect_arch() { } # Get latest release version +# Primary: parse the 302 redirect on /releases/latest (no API call, no rate limit). +# Fallback: the GitHub REST API (subject to 60 req/hour anonymous limit). get_latest_version() { - VERSION=$(curl -fsSL "https://api.github.com/repos/${REPO}/releases/latest" | grep '"tag_name":' | sed -E 's/.*"([^"]+)".*/\1/') + # Try the web redirect first — does not count against the API rate limit. + VERSION=$(curl -sI "https://github.com/${REPO}/releases/latest" \ + | grep -i '^location:' \ + | sed -E 's|.*/tag/([^[:space:]]+).*|\1|' \ + | tr -d '\r') + + # Fallback to the REST API if the redirect didn't yield a tag. + if [ -z "$VERSION" ]; then + warn "Redirect lookup failed, falling back to GitHub API..." + VERSION=$(curl -fsSL "https://api.github.com/repos/${REPO}/releases/latest" \ + | grep '"tag_name":' \ + | sed -E 's/.*"([^"]+)".*/\1/') + fi + if [ -z "$VERSION" ]; then - error "Failed to get latest version" + error "Failed to get latest version (GitHub API may be rate-limited; set RTK_VERSION=vX.Y.Z to pin)" fi } @@ -113,7 +128,12 @@ main() { detect_os detect_arch get_target - get_latest_version + if [ -n "$RTK_VERSION" ]; then + VERSION="$RTK_VERSION" + info "Using pinned version from RTK_VERSION: $VERSION" + else + get_latest_version + fi install verify diff --git a/openclaw/README.md b/openclaw/README.md index 301d7c0fa..4440db027 100644 --- a/openclaw/README.md +++ b/openclaw/README.md @@ -19,7 +19,7 @@ RTK must be installed and available in `$PATH`: ```bash brew install rtk # or -curl -fsSL https://raw.githubusercontent.com/rtk-ai/rtk/refs/heads/master/install.sh | sh +curl -fsSL https://raw.githubusercontent.com/algolia/rtk/refs/heads/main/install.sh | sh ``` ### Install the plugin @@ -61,7 +61,7 @@ In `openclaw.json`: ## What gets rewritten -Everything that `rtk rewrite` supports (30+ commands). See the [full command list](https://github.com/rtk-ai/rtk#commands). +Everything that `rtk rewrite` supports (30+ commands). See the [full command list](https://github.com/algolia/rtk#commands). ## What's NOT rewritten diff --git a/openclaw/openclaw.plugin.json b/openclaw/openclaw.plugin.json index 3fce418d7..dcc2f1f9a 100644 --- a/openclaw/openclaw.plugin.json +++ b/openclaw/openclaw.plugin.json @@ -3,7 +3,7 @@ "name": "RTK Token Optimizer", "version": "1.0.0", "description": "Transparently rewrites shell commands to their RTK equivalents for 60-90% LLM token savings", - "homepage": "https://github.com/rtk-ai/rtk", + "homepage": "https://github.com/algolia/rtk", "license": "MIT", "configSchema": { "type": "object", diff --git a/openclaw/package.json b/openclaw/package.json index 18d359ff4..7195151b9 100644 --- a/openclaw/package.json +++ b/openclaw/package.json @@ -1,15 +1,15 @@ { - "name": "@rtk-ai/rtk-rewrite", + "name": "@algolia/rtk-rewrite", "version": "1.0.0", "description": "RTK plugin for OpenClaw — rewrites shell commands for 60-90% LLM token savings", "main": "index.ts", "license": "MIT", "repository": { "type": "git", - "url": "https://github.com/rtk-ai/rtk", + "url": "https://github.com/algolia/rtk", "directory": "openclaw" }, - "homepage": "https://github.com/rtk-ai/rtk", + "homepage": "https://github.com/algolia/rtk", "keywords": [ "rtk", "openclaw", diff --git a/scripts/benchmark-sessions/lib/runner.py b/scripts/benchmark-sessions/lib/runner.py new file mode 100644 index 000000000..192fbcd41 --- /dev/null +++ b/scripts/benchmark-sessions/lib/runner.py @@ -0,0 +1,155 @@ +from __future__ import annotations + +import asyncio +import subprocess +import tempfile +from pathlib import Path + +from .config import TaskConfig +from .manifest import ( + RunManifest, + SessionEntry, + TbEntry, + TbTaskEntry, + write_manifest, +) +from .session import run_all_sessions, setup_codebase, setup_rtk +from .terminal_bench import run_terminal_bench +from .vm import create_vm_pool, destroy_vm_pool + +ROOT_DIR = Path(__file__).resolve().parent.parent + + +def _create_tarball(source_dir: Path) -> str: + tarball = tempfile.mktemp(suffix=".tar.gz") + subprocess.run( + ["tar", "czf", tarball, "-C", str(source_dir), "."], + check=True, + ) + return tarball + + +def _print_step(step: int, total: int, msg: str): + print(f"\n[{step}/{total}] {msg}") + + +def _session_to_entry(r) -> SessionEntry: + return SessionEntry( + vm_name=r.vm_name, + group=r.group, + stdout_json=f"{r.vm_name}-stdout.json", + otel_log=f"{r.vm_name}-otel.log", + rtk_db=f"{r.vm_name}-tracking.db" if r.rtk_db_path else None, + exit_code=r.exit_code, + error=r.error or None, + ) + + +def _tb_to_entry(r) -> TbEntry: + return TbEntry( + vm_name=r.vm_name, + group=r.group, + total=r.total, + passed=r.passed, + failed=r.failed, + tasks=[TbTaskEntry(name=t.name, passed=t.passed, duration_s=t.duration_s) for t in r.tasks], + error=r.error, + ) + + +async def run_benchmark( + task: TaskConfig, + vms: int, + api_key: str, + output_dir: Path, + cloud_init: Path | None = None, + terminal_bench: bool = False, + keep_vms: bool = False, +) -> RunManifest: + if cloud_init is None: + cloud_init = ROOT_DIR / "cloud-init-base.yaml" + + output_dir.mkdir(parents=True, exist_ok=True) + + total_steps = 5 if terminal_bench else 4 + vm_names: list[str] = [] + + manifest = RunManifest( + task_name=task.name, + model=task.model, + vm_count=vms, + ) + + try: + _print_step(1, total_steps, f"Creating {vms * 2} VMs ({vms} RTK ON + {vms} RTK OFF)") + vm_names = await create_vm_pool(vms, cloud_init) + print(f" VMs ready: {', '.join(vm_names)}") + + _print_step(2, total_steps, "Setting up codebases") + local_tarball = None + if not task.codebase.is_github: + local_tarball = _create_tarball(task.codebase.local_path()) + + await asyncio.gather(*( + setup_codebase(name, task.codebase, local_tarball) + for name in vm_names + )) + print(" Codebases deployed") + + _print_step(3, total_steps, "Configuring RTK on ON VMs") + setup_script = ROOT_DIR / "setup-rtk.sh" + on_vms = [n for n in vm_names if "-on-" in n] + off_vms = [n for n in vm_names if "-off-" in n] + await asyncio.gather(*(setup_rtk(vm, setup_script) for vm in on_vms)) + print(f" RTK configured on {len(on_vms)} VMs") + + _print_step(4, total_steps, f"Running Claude sessions (timeout: {task.timeout_minutes}min)") + results = await run_all_sessions(vm_names, task, api_key, output_dir) + + on_ok = [r for r in results if r.group == "on" and not r.error] + off_ok = [r for r in results if r.group == "off" and not r.error] + errors = [r for r in results if r.error] + print(f" Completed: {len(on_ok)} ON, {len(off_ok)} OFF, {len(errors)} errors") + for r in errors: + print(f" {r.vm_name}: {r.error}") + + manifest.sessions = [_session_to_entry(r) for r in results] + + if terminal_bench: + _print_step(5, total_steps, "Running terminal-bench precision tests") + tb_on = await asyncio.gather(*( + run_terminal_bench(vm, "on", task.model, api_key) + for vm in on_vms + )) + tb_off = await asyncio.gather(*( + run_terminal_bench(vm, "off", task.model, api_key) + for vm in off_vms + )) + + manifest.terminal_bench = [_tb_to_entry(r) for r in list(tb_on) + list(tb_off)] + + ok_on = [r for r in tb_on if not r.error] + ok_off = [r for r in tb_off if not r.error] + if ok_on and ok_off: + on_total = sum(r.total for r in ok_on) + on_passed = sum(r.passed for r in ok_on) + off_total = sum(r.total for r in ok_off) + off_passed = sum(r.passed for r in ok_off) + on_rate = on_passed / on_total if on_total else 0 + off_rate = off_passed / off_total if off_total else 0 + print(f" terminal-bench: ON pass rate={on_rate:.0%}, OFF pass rate={off_rate:.0%}, delta={on_rate - off_rate:+.0%}") + + tb_errors = [r for r in list(tb_on) + list(tb_off) if r.error] + for r in tb_errors: + print(f" {r.vm_name}: {r.error}") + + write_manifest(manifest, output_dir) + print(f"\n Manifest written to {output_dir / 'manifest.json'}") + + finally: + if not keep_vms and vm_names: + print("\nCleaning up VMs...") + await destroy_vm_pool(vm_names) + print(" VMs destroyed") + + return manifest diff --git a/scripts/benchmark.sh b/scripts/benchmark.sh index a1e616bcc..0af7e4417 100755 --- a/scripts/benchmark.sh +++ b/scripts/benchmark.sh @@ -11,34 +11,31 @@ else exit 1 fi BENCH_DIR="$(pwd)/scripts/benchmark" +RTK_ROOT="$(pwd)" -# Mode local : générer les fichiers debug if [ -z "$CI" ]; then rm -rf "$BENCH_DIR" mkdir -p "$BENCH_DIR/unix" "$BENCH_DIR/rtk" "$BENCH_DIR/diff" fi -# Nom de fichier safe safe_name() { echo "$1" | tr ' /' '_-' | tr -cd 'a-zA-Z0-9_-' } -# Fonction pour compter les tokens (~4 chars = 1 token) count_tokens() { local input="$1" local len=${#input} echo $(( (len + 3) / 4 )) } -# Compteurs globaux TOTAL_UNIX=0 TOTAL_RTK=0 TOTAL_TESTS=0 GOOD_TESTS=0 FAIL_TESTS=0 -SKIP_TESTS=0 +WARN_TESTS=0 +NEGATIVE_TESTS=0 -# Fonction de benchmark — une ligne par test bench() { local name="$1" local unix_cmd="$2" @@ -55,24 +52,41 @@ bench() { local icon="" local tag="" - if [ -z "$rtk_out" ]; then + if [ -z "$rtk_out" ] && [ -n "$unix_out" ]; then icon="❌" tag="FAIL" FAIL_TESTS=$((FAIL_TESTS + 1)) TOTAL_UNIX=$((TOTAL_UNIX + unix_tokens)) TOTAL_RTK=$((TOTAL_RTK + unix_tokens)) - elif [ "$rtk_tokens" -ge "$unix_tokens" ] && [ "$unix_tokens" -gt 0 ]; then + elif [ "$rtk_tokens" -gt "$unix_tokens" ] && [ "$unix_tokens" -gt 0 ]; then + icon="🔴" + tag="NEG" + NEGATIVE_TESTS=$((NEGATIVE_TESTS + 1)) + TOTAL_UNIX=$((TOTAL_UNIX + unix_tokens)) + TOTAL_RTK=$((TOTAL_RTK + rtk_tokens)) + elif [ "$unix_tokens" -gt 0 ] && [ "$rtk_tokens" -eq "$unix_tokens" ]; then icon="⚠️" - tag="SKIP" - SKIP_TESTS=$((SKIP_TESTS + 1)) + tag="WARN" + WARN_TESTS=$((WARN_TESTS + 1)) TOTAL_UNIX=$((TOTAL_UNIX + unix_tokens)) - TOTAL_RTK=$((TOTAL_RTK + unix_tokens)) - else - icon="✅" - tag="GOOD" - GOOD_TESTS=$((GOOD_TESTS + 1)) + TOTAL_RTK=$((TOTAL_RTK + rtk_tokens)) + elif [ "$unix_tokens" -gt 0 ]; then + local savings=$(( (unix_tokens - rtk_tokens) * 100 / unix_tokens )) + if [ "$savings" -lt 60 ]; then + icon="⚠️" + tag="WARN" + WARN_TESTS=$((WARN_TESTS + 1)) + else + icon="✅" + tag="GOOD" + GOOD_TESTS=$((GOOD_TESTS + 1)) + fi TOTAL_UNIX=$((TOTAL_UNIX + unix_tokens)) TOTAL_RTK=$((TOTAL_RTK + rtk_tokens)) + else + icon="⏭️" + tag="SKIP" + WARN_TESTS=$((WARN_TESTS + 1)) fi if [ "$tag" = "FAIL" ]; then @@ -88,12 +102,13 @@ bench() { "$icon" "$name" "$unix_cmd" "$rtk_cmd" "$unix_tokens" "$rtk_tokens" "$pct" fi - # Fichiers debug en local uniquement if [ -z "$CI" ]; then local filename=$(safe_name "$name") local prefix="GOOD" [ "$tag" = "FAIL" ] && prefix="FAIL" - [ "$tag" = "SKIP" ] && prefix="BAD" + [ "$tag" = "NEG" ] && prefix="NEG" + [ "$tag" = "WARN" ] && prefix="WARN" + [ "$tag" = "SKIP" ] && prefix="SKIP" local ts=$(date "+%d/%m/%Y %H:%M:%S") @@ -124,7 +139,6 @@ bench() { fi } -# Section header section() { echo "" echo "── $1 ──" @@ -149,6 +163,18 @@ bench "ls src/ -l" "ls -l src/" "$RTK ls src/ -l" bench "ls -a" "ls -la" "$RTK ls -a" bench "ls multi" "ls -la src/ scripts/" "$RTK ls src/ scripts/" +# =================== +# tree +# =================== +if command -v tree &>/dev/null; then + section "tree" + bench "tree" "tree -L 2" "$RTK tree -L 2" + bench "tree src/" "tree src/ -L 2" "$RTK tree src/ -L 2" +else + echo "" + echo "⏭️ tree (not installed, skipped)" +fi + # =================== # read # =================== @@ -175,6 +201,7 @@ bench "git status" "git status" "$RTK git status" bench "git log -n 10" "git log -10" "$RTK git log -n 10" bench "git log -n 5" "git log -5" "$RTK git log -n 5" bench "git diff" "git diff HEAD~1 2>/dev/null || echo ''" "$RTK git diff HEAD~1" +bench "git show" "git show HEAD --stat 2>/dev/null || true" "$RTK git show HEAD --stat" # =================== # grep @@ -183,7 +210,6 @@ section "grep" bench "grep fn" "grep -rn 'fn ' src/ || true" "$RTK grep 'fn ' src/" bench "grep struct" "grep -rn 'struct ' src/ || true" "$RTK grep 'struct ' src/" bench "grep -l 40" "grep -rn 'fn ' src/ || true" "$RTK grep 'fn ' src/ -l 40" -bench "grep --max 20" "grep -rn 'fn ' src/ | head -20 || true" "$RTK grep 'fn ' src/ --max 20" bench "grep -c" "grep -ron 'fn ' src/ || true" "$RTK grep 'fn ' src/ -c" # =================== @@ -229,7 +255,7 @@ bench "env --show-all" "env" "$RTK env --show-all" # =================== section "err" if command -v cargo &>/dev/null; then - bench "err cargo build" "cargo build 2>&1 || true" "$RTK err cargo build" + bench "err cargo build" "cargo build 2>&1 || true" "$RTK err cargo build 2>&1" else echo "⏭️ err cargo build (cargo not in PATH, skipped)" fi @@ -239,7 +265,7 @@ fi # =================== section "test" if command -v cargo &>/dev/null; then - bench "test cargo test" "cargo test 2>&1 || true" "$RTK test cargo test" + bench "test cargo test" "cargo test 2>&1 || true" "$RTK test cargo test 2>&1" else echo "⏭️ test cargo test (cargo not in PATH, skipped)" fi @@ -287,20 +313,14 @@ fi # =================== section "cargo" if command -v cargo &>/dev/null; then - bench "cargo build" "cargo build 2>&1 || true" "$RTK cargo build" - bench "cargo test" "cargo test 2>&1 || true" "$RTK cargo test" - bench "cargo clippy" "cargo clippy 2>&1 || true" "$RTK cargo clippy" - bench "cargo check" "cargo check 2>&1 || true" "$RTK cargo check" + bench "cargo build" "cargo build 2>&1 || true" "$RTK cargo build 2>&1" + bench "cargo test" "cargo test 2>&1 || true" "$RTK cargo test 2>&1" + bench "cargo clippy" "cargo clippy 2>&1 || true" "$RTK cargo clippy 2>&1" + bench "cargo check" "cargo check 2>&1 || true" "$RTK cargo check 2>&1" else echo "⏭️ cargo build/test/clippy/check (cargo not in PATH, skipped)" fi -# =================== -# diff -# =================== -section "diff" -bench "diff" "diff Cargo.toml LICENSE 2>&1 || true" "$RTK diff Cargo.toml LICENSE" - # =================== # smart # =================== @@ -327,7 +347,16 @@ fi # =================== if command -v wget &> /dev/null; then section "wget" - bench "wget" "wget -qO- https://httpbin.org/robots.txt" "$RTK wget https://httpbin.org/robots.txt -O" + bench "wget" "wget -qO- https://httpbin.org/json" "$RTK wget https://httpbin.org/json" + rm -f json 2>/dev/null +fi + +# =================== +# npm (standalone — does not require package.json) +# =================== +if command -v npm &> /dev/null; then + section "npm" + bench "npm list" "npm list -g --depth 0 2>&1 || true" "$RTK npm list -g --depth 0" fi # =================== @@ -337,7 +366,7 @@ if [ -f "package.json" ]; then section "modern JS stack" if command -v tsc &> /dev/null || [ -f "node_modules/.bin/tsc" ]; then - bench "tsc" "tsc --noEmit 2>&1 || true" "$RTK tsc --noEmit" + bench "tsc" "tsc --noEmit 2>&1 || true" "$RTK tsc --noEmit 2>&1" fi if command -v prettier &> /dev/null || [ -f "node_modules/.bin/prettier" ]; then @@ -367,7 +396,7 @@ if [ -f "package.json" ]; then fi if command -v vitest &> /dev/null || [ -f "node_modules/.bin/vitest" ]; then - bench "vitest run" "vitest run --reporter=json 2>&1 || true" "$RTK vitest run" + bench "vitest" "vitest run --reporter=json 2>&1 || true" "$RTK vitest" fi if command -v pnpm &> /dev/null; then @@ -379,14 +408,31 @@ fi # =================== # gh (skip si pas dispo ou pas dans un repo) # =================== -if command -v gh &> /dev/null && git rev-parse --git-dir &> /dev/null; then +if command -v gh &> /dev/null && git rev-parse --git-dir &> /dev/null && gh auth status &> /dev/null; then section "gh" bench "gh pr list" "gh pr list 2>&1 || true" "$RTK gh pr list" bench "gh run list" "gh run list 2>&1 || true" "$RTK gh run list" fi # =================== -# docker (skip si pas dispo) +# glab +# =================== +if command -v glab &> /dev/null; then + section "glab" + bench "glab mr list" "glab mr list 2>&1 || true" "$RTK glab mr list" + bench "glab issue list" "glab issue list 2>&1 || true" "$RTK glab issue list" +fi + +# =================== +# gt (Graphite) +# =================== +if command -v gt &> /dev/null; then + section "gt" + bench "gt log" "gt log 2>&1 || true" "$RTK gt log" +fi + +# =================== +# docker # =================== if command -v docker &> /dev/null; then section "docker" @@ -395,7 +441,7 @@ if command -v docker &> /dev/null; then fi # =================== -# kubectl (skip si pas dispo) +# kubectl # =================== if command -v kubectl &> /dev/null; then section "kubectl" @@ -412,7 +458,6 @@ if command -v python3 &> /dev/null && command -v ruff &> /dev/null && command -v PYTHON_FIXTURE=$(mktemp -d) cd "$PYTHON_FIXTURE" - # pyproject.toml cat > pyproject.toml << 'PYEOF' [project] name = "rtk-bench" @@ -422,7 +467,6 @@ version = "0.1.0" line-length = 88 PYEOF - # sample.py avec quelques issues ruff cat > sample.py << 'PYEOF' import os import sys @@ -442,7 +486,6 @@ def unused_function(): # F841: local variable assigned but never used return None PYEOF - # test_sample.py cat > test_sample.py << 'PYEOF' from sample import process_data @@ -456,7 +499,15 @@ PYEOF bench "ruff check" "ruff check . 2>&1 || true" "$RTK ruff check ." bench "pytest" "pytest -v 2>&1 || true" "$RTK pytest -v" - cd - > /dev/null + if command -v pip &>/dev/null; then + bench "pip list" "pip list 2>&1 || true" "$RTK pip list" + fi + + if command -v mypy &>/dev/null; then + bench "mypy" "mypy sample.py 2>&1 || true" "$RTK mypy sample.py" + fi + + cd "$RTK_ROOT" rm -rf "$PYTHON_FIXTURE" fi @@ -469,14 +520,12 @@ if command -v go &> /dev/null && command -v golangci-lint &> /dev/null; then GO_FIXTURE=$(mktemp -d) cd "$GO_FIXTURE" - # go.mod cat > go.mod << 'GOEOF' module bench go 1.21 GOEOF - # main.go cat > main.go << 'GOEOF' package main @@ -496,7 +545,6 @@ func main() { } GOEOF - # main_test.go cat > main_test.go << 'GOEOF' package main @@ -522,16 +570,55 @@ GOEOF bench "go build" "go build ./... 2>&1 || true" "$RTK go build ./..." bench "go vet" "go vet ./... 2>&1 || true" "$RTK go vet ./..." - cd - > /dev/null + cd "$RTK_ROOT" rm -rf "$GO_FIXTURE" fi +# =================== +# Ruby +# =================== +if command -v ruby &> /dev/null; then + section "ruby" + if command -v rake &>/dev/null; then + bench "rake -T" "rake -T 2>&1 || true" "$RTK rake -T" + fi + if command -v rubocop &>/dev/null; then + bench "rubocop" "rubocop --format simple 2>&1 || true" "$RTK rubocop --format simple" + fi + if command -v rspec &>/dev/null; then + bench "rspec --dry-run" "rspec --dry-run 2>&1 || true" "$RTK rspec --dry-run" + fi +fi + +# =================== +# dotnet +# =================== +if command -v dotnet &> /dev/null; then + section "dotnet" + bench "dotnet --info" "dotnet --info 2>&1 || true" "$RTK dotnet --info" +fi + +# =================== +# aws +# =================== +if command -v aws &> /dev/null; then + section "aws" + bench "aws --version" "aws --version 2>&1 || true" "$RTK aws --version" +fi + +# =================== +# psql +# =================== +if command -v psql &> /dev/null; then + section "psql" + bench "psql --version" "psql --version 2>&1 || true" "$RTK psql --version" +fi + # =================== # rewrite (verify rewrite works with and without quotes) # =================== section "rewrite" -# bench_rewrite: verifies rewrite produces expected output (not token comparison) bench_rewrite() { local name="$1" local cmd="$2" @@ -558,7 +645,7 @@ bench_rewrite "rewrite cargo test" "$RTK rewrite cargo test" "rtk cargo bench_rewrite "rewrite compound" "$RTK rewrite 'cargo test && git push'" "rtk cargo test && rtk git push" # =================== -# Résumé global +# Summary # =================== echo "" echo "═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════" @@ -574,19 +661,30 @@ if [ "$TOTAL_TESTS" -gt 0 ]; then fi echo "" - echo " ✅ $GOOD_TESTS good ⚠️ $SKIP_TESTS skip ❌ $FAIL_TESTS fail $GOOD_TESTS/$TOTAL_TESTS ($GOOD_PCT%)" + echo " ✅ $GOOD_TESTS good ⚠️ $WARN_TESTS warn 🔴 $NEGATIVE_TESTS negative ❌ $FAIL_TESTS fail $GOOD_TESTS/$TOTAL_TESTS ($GOOD_PCT%)" echo " Tokens: $TOTAL_UNIX → $TOTAL_RTK (-$TOTAL_SAVE_PCT%)" echo "" - # Fichiers debug en local if [ -z "$CI" ]; then echo " Debug: $BENCH_DIR/{unix,rtk,diff}/" fi echo "" - # Exit code non-zero si moins de 80% good - if [ "$GOOD_PCT" -lt 80 ]; then - echo " BENCHMARK FAILED: $GOOD_PCT% good (minimum 80%)" - exit 1 + EXIT_CODE=0 + + if [ "$NEGATIVE_TESTS" -gt 0 ]; then + echo " BENCHMARK FAILED: $NEGATIVE_TESTS filter(s) produced more tokens than raw output" + EXIT_CODE=1 + fi + + if [ "$FAIL_TESTS" -gt 0 ]; then + echo " BENCHMARK FAILED: $FAIL_TESTS filter(s) returned empty output" + EXIT_CODE=1 fi + + if [ "$GOOD_PCT" -lt 60 ] && [ "$EXIT_CODE" -eq 0 ]; then + echo " WARNING: $GOOD_PCT% good (target 60%)" + fi + + exit $EXIT_CODE fi diff --git a/scripts/benchmark/cleanup.ts b/scripts/benchmark/cleanup.ts new file mode 100644 index 000000000..7cc38edba --- /dev/null +++ b/scripts/benchmark/cleanup.ts @@ -0,0 +1,11 @@ +#!/usr/bin/env bun +/** + * Delete the RTK test VM. + * Usage: bun run scripts/benchmark/cleanup.ts + */ + +import { vmDelete } from "./lib/vm"; + +console.log("Deleting rtk-test VM..."); +await vmDelete(); +console.log("Done."); diff --git a/scripts/benchmark/cloud-init.yaml b/scripts/benchmark/cloud-init.yaml new file mode 100644 index 000000000..a528c5aa0 --- /dev/null +++ b/scripts/benchmark/cloud-init.yaml @@ -0,0 +1,315 @@ +#cloud-config +# RTK Integration Test VM — Ubuntu 24.04 +# Installs all tools needed for comprehensive RTK testing (~200 commands) +# Usage: multipass launch --name rtk-test --cloud-init scripts/benchmark/cloud-init.yaml --cpus 2 --memory 4G --disk 20G 24.04 + +package_update: true +package_upgrade: false + +packages: + # System tools + - curl + - wget + - jq + - git + - make + - cmake + - rsync + - sqlite3 + - shellcheck + - yamllint + - postgresql-client + - docker.io + - containerd + - python3 + - python3-pip + - python3-venv + - pipx + # Build essentials (for Rust compilation) + - build-essential + - pkg-config + - libssl-dev + - libsqlite3-dev + # Misc + - hyperfine + - unzip + - tree + +runcmd: + # ── Rust toolchain ── + - su - ubuntu -c 'curl --proto "=https" --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y' + + # ── Node.js 22 + package managers ── + - curl -fsSL https://deb.nodesource.com/setup_22.x | bash - + - apt-get install -y nodejs + - npm install -g pnpm yarn + - npm install -g eslint prettier typescript + - npm install -g markdownlint-cli + + # ── Go 1.22 ── + - curl -fsSL https://go.dev/dl/go1.22.5.linux-amd64.tar.gz | tar -C /usr/local -xz + - echo 'export PATH=$PATH:/usr/local/go/bin:/home/ubuntu/go/bin' >> /home/ubuntu/.bashrc + - su - ubuntu -c 'export PATH=$PATH:/usr/local/go/bin && go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest' + + # ── Python tools ── + - pipx install ruff + - pipx install mypy + - pipx install poetry + - pip3 install --break-system-packages pytest uv pre-commit + + # ── .NET 8 SDK ── + - | + wget https://dot.net/v1/dotnet-install.sh -O /tmp/dotnet-install.sh + chmod +x /tmp/dotnet-install.sh + /tmp/dotnet-install.sh --channel 8.0 --install-dir /usr/local/share/dotnet + ln -sf /usr/local/share/dotnet/dotnet /usr/local/bin/dotnet + echo 'export DOTNET_ROOT=/usr/local/share/dotnet' >> /home/ubuntu/.bashrc + + # ── Terraform ── + - | + wget -qO- https://apt.releases.hashicorp.com/gpg | gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg + echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" > /etc/apt/sources.list.d/hashicorp.list + apt-get update && apt-get install -y terraform + + # ── Helm ── + - curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash + + # ── Hadolint ── + - | + wget -qO /usr/local/bin/hadolint https://github.com/hadolint/hadolint/releases/latest/download/hadolint-Linux-x86_64 + chmod +x /usr/local/bin/hadolint + + # ── Docker setup ── + - usermod -aG docker ubuntu + - systemctl enable docker + - systemctl start docker + + # ── kubectl (standalone binary) ── + - | + curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" + install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl + rm kubectl + + # ── ansible ── + - pip3 install --break-system-packages ansible-core + + # ── Mock tools (too heavy to install) ── + - | + cat > /usr/local/bin/gcloud << 'MOCK' + #!/bin/bash + if [ "$1" = "version" ] || [ "$1" = "--version" ]; then + echo "Google Cloud SDK 400.0.0" + echo "bq 2.0.80" + echo "core 2023.01.01" + echo "gsutil 5.17" + else + echo "gcloud mock: $*" + fi + MOCK + chmod +x /usr/local/bin/gcloud + + - | + cat > /usr/local/bin/shopify << 'MOCK' + #!/bin/bash + echo "Shopify CLI 3.0.0 (mock)" + if [ "$1" = "theme" ] && [ "$2" = "check" ]; then + echo "Running theme check..." + echo " 1 issue found" + echo " [warn] Missing alt text on image" + fi + MOCK + chmod +x /usr/local/bin/shopify + + - | + cat > /usr/local/bin/pio << 'MOCK' + #!/bin/bash + if [ "$1" = "--version" ]; then echo "PlatformIO Core, version 6.1.0" + elif [ "$1" = "run" ]; then + echo "Processing esp32dev (platform: espressif32; board: esp32dev)" + echo "Linking .pio/build/esp32dev/firmware.elf" + echo "========================= [SUCCESS] =========================" + fi + MOCK + chmod +x /usr/local/bin/pio + + - | + cat > /usr/local/bin/quarto << 'MOCK' + #!/bin/bash + if [ "$1" = "--version" ]; then echo "1.3.450" + elif [ "$1" = "render" ]; then echo "Rendering document..."; echo "Output created: document.html" + fi + MOCK + chmod +x /usr/local/bin/quarto + + - | + cat > /usr/local/bin/sops << 'MOCK' + #!/bin/bash + if [ "$1" = "--version" ]; then echo "sops 3.7.3"; fi + MOCK + chmod +x /usr/local/bin/sops + + - | + cat > /usr/local/bin/swift << 'MOCK' + #!/bin/bash + if [ "$1" = "--version" ]; then echo "Swift version 5.9.2 (swift-5.9.2-RELEASE)" + elif [ "$1" = "build" ]; then echo "Compiling Swift module..."; echo "Build complete! (0.42s)" + fi + MOCK + chmod +x /usr/local/bin/swift + + # ── Fake test projects ── + + # Node.js project with errors + - | + su - ubuntu -c ' + mkdir -p /tmp/test-node/src && cd /tmp/test-node + npm init -y >/dev/null 2>&1 + echo "{\"compilerOptions\":{\"strict\":true,\"noEmit\":true,\"target\":\"ES2020\",\"module\":\"ESNext\",\"moduleResolution\":\"node\"},\"include\":[\"src\"]}" > tsconfig.json + echo "const x: number = \"not a number\";\nconst unused = 42;\nfunction greet(name: string): string { return name }\ngreet(123);" > src/index.ts + echo "{\"rules\":{\"no-unused-vars\":\"error\",\"semi\":[\"error\",\"always\"]}}" > .eslintrc.json + echo "const x = 1;const y=2; const z =3" > src/ugly.ts + ' + + # Python project with errors + - | + su - ubuntu -c ' + mkdir -p /tmp/test-python && cd /tmp/test-python + cat > main.py << "PYEOF" + import os + import sys + unused_import = 1 + def add(a: int, b: int) -> str: + return a + b + x: int = "hello" + PYEOF + cat > test_main.py << "PYEOF" + def test_pass(): + assert 1 + 1 == 2 + def test_fail(): + assert 1 + 1 == 3, "math is broken" + PYEOF + cat > pyproject.toml << "PYEOF" + [tool.ruff] + line-length = 80 + select = ["E", "F", "W"] + [tool.mypy] + strict = true + [tool.pytest.ini_options] + testpaths = ["."] + PYEOF + ' + + # Go project with errors + - | + su - ubuntu -c ' + export PATH=$PATH:/usr/local/go/bin + mkdir -p /tmp/test-go && cd /tmp/test-go + go mod init test-go 2>/dev/null + cat > main.go << "GOEOF" + package main + import "fmt" + func main() { fmt.Println("hello") } + func unused() { var x int; _ = x } + GOEOF + cat > main_test.go << "GOEOF" + package main + import "testing" + func TestPass(t *testing.T) { if 1+1 != 2 { t.Fatal("math") } } + func TestFail(t *testing.T) { t.Fatal("expected failure") } + GOEOF + ' + + # Rust project with errors + - | + su - ubuntu -c ' + export PATH=$HOME/.cargo/bin:$PATH + mkdir -p /tmp/test-rust && cd /tmp/test-rust + cargo init --name test-rust 2>/dev/null + cat > src/main.rs << "RSEOF" + fn main() { + let x = vec![1, 2, 3]; + let _y = x.iter().map(|i| i.clone()).collect::>(); + println!("hello"); + } + #[cfg(test)] + mod tests { + #[test] fn test_pass() { assert_eq!(1 + 1, 2); } + #[test] fn test_fail() { assert_eq!(1 + 1, 3); } + } + RSEOF + ' + + # Dockerfiles for hadolint + - | + su - ubuntu -c ' + cat > /tmp/Dockerfile.bad << "DEOF" + FROM ubuntu:latest + RUN apt-get update && apt-get install -y curl wget git + RUN cd /tmp && wget http://example.com/script.sh && bash script.sh + EXPOSE 80 443 8080 + DEOF + ' + + # Shell/YAML/Markdown test files + - | + su - ubuntu -c ' + printf "#!/bin/bash\necho \$foo\nls *.txt\ncd \$(pwd)\n[ -f file ] && rm file\n" > /tmp/test.sh + printf "foo: bar\nbaz: qux\nlist:\n - item1\n - item2\ntruthy: yes\n" > /tmp/test.yaml + printf "#Header without space\nSome text\n\n* List item\n+ Mixed markers\n" > /tmp/test.md + ' + + # Git repo for testing + - | + su - ubuntu -c ' + mkdir -p /tmp/test-git && cd /tmp/test-git + git init && git config user.email "test@rtk.dev" && git config user.name "RTK Test" + for i in $(seq 1 20); do echo "line $i" >> file.txt && git add file.txt && git commit -m "feat: commit number $i"; done + echo "modified" >> file.txt && echo "new file" > new.txt + ' + + # Large log file for dedup testing + - | + su - ubuntu -c ' + for i in $(seq 1 500); do + printf "[2026-03-25 10:00:00] INFO Starting service...\n[2026-03-25 10:00:01] WARN Connection timeout\n[2026-03-25 10:00:01] ERROR Failed to connect: refused\n" + done > /tmp/large.log + for i in $(seq 1 50); do echo "[2026-03-25 10:05:00] FATAL Out of memory"; done >> /tmp/large.log + ' + + # .env file + - | + su - ubuntu -c ' + printf "DATABASE_URL=postgres://user:pass@localhost:5432/db\nAPI_KEY=sk-1234567890abcdef\nSECRET_TOKEN=ghp_xxxx\nNODE_ENV=production\nPORT=3000\n" > /tmp/.env + ' + + # Makefile + - | + su - ubuntu -c ' + printf ".PHONY: all test\nall:\n\t@echo Building...\n\t@echo Build complete\ntest:\n\t@echo Running tests...\n\t@echo 2 tests passed\n" > /tmp/Makefile + ' + + # Terraform project + - | + su - ubuntu -c ' + mkdir -p /tmp/test-terraform && cd /tmp/test-terraform + printf "terraform {\n required_version = \">= 1.0\"\n}\nresource \"null_resource\" \"test\" {\n triggers = { always = timestamp() }\n}\noutput \"test\" { value = \"hello\" }\n" > main.tf + ' + + # Helm chart + - su - ubuntu -c 'mkdir -p /tmp/test-helm && cd /tmp/test-helm && helm create test-chart 2>/dev/null || true' + + # .NET project + - | + export DOTNET_ROOT=/usr/local/share/dotnet + su - ubuntu -c ' + export DOTNET_ROOT=/usr/local/share/dotnet && export PATH=$PATH:$DOTNET_ROOT + mkdir -p /tmp/test-dotnet && cd /tmp/test-dotnet + dotnet new console -n TestApp --force 2>/dev/null || true + ' + + # Signal completion + - touch /home/ubuntu/.cloud-init-complete + - chown ubuntu:ubuntu /home/ubuntu/.cloud-init-complete + - echo "RTK cloud-init setup complete" | tee /var/log/rtk-setup.log + +final_message: "RTK test VM ready in $UPTIME seconds" diff --git a/scripts/benchmark/lib/report.ts b/scripts/benchmark/lib/report.ts new file mode 100644 index 000000000..1fb751c39 --- /dev/null +++ b/scripts/benchmark/lib/report.ts @@ -0,0 +1,113 @@ +/** + * Report generation for RTK integration test results. + */ + +import type { TestResult } from "./test"; +import { getCounts, getResults } from "./test"; + +interface BuildInfo { + buildTime: number; + binarySize: number; + version: string; + branch: string; + commit: string; +} + +export function generateReport(buildInfo: BuildInfo): string { + const { total, passed, failed, skipped } = getCounts(); + const results = getResults(); + const passRate = total > 0 ? Math.round((passed * 100) / total) : 0; + + const lines: string[] = []; + + lines.push("======================================================"); + lines.push(" RTK INTEGRATION TEST REPORT"); + lines.push("======================================================"); + lines.push(""); + lines.push(`Date: ${new Date().toISOString()}`); + lines.push(`Branch: ${buildInfo.branch}`); + lines.push(`Commit: ${buildInfo.commit}`); + lines.push(`Version: ${buildInfo.version}`); + lines.push(`Binary: ${buildInfo.binarySize} bytes`); + lines.push(`Build: ${buildInfo.buildTime}s`); + lines.push(""); + + // Summary + lines.push("--- Summary ---"); + lines.push(`Total: ${total}`); + lines.push(`Passed: ${passed} (${passRate}%)`); + lines.push(`Failed: ${failed}`); + lines.push(`Skipped: ${skipped}`); + lines.push(""); + + // Group results by phase (name prefix before ":") + const phases = new Map(); + for (const r of results) { + const colonIdx = r.name.indexOf(":"); + const phase = colonIdx > 0 ? r.name.slice(0, colonIdx) : "misc"; + if (!phases.has(phase)) phases.set(phase, []); + phases.get(phase)!.push(r); + } + + for (const [phase, phaseResults] of phases) { + const pPassed = phaseResults.filter((r) => r.status === "PASS").length; + const pTotal = phaseResults.length; + lines.push(`--- ${phase} (${pPassed}/${pTotal}) ---`); + + for (const r of phaseResults) { + const shortName = r.name.includes(":") ? r.name.split(":")[1] : r.name; + lines.push(` ${r.status.padEnd(4)} | ${shortName} | ${r.detail}`); + } + lines.push(""); + } + + // Failures detail + const failures = results.filter((r) => r.status === "FAIL"); + if (failures.length > 0) { + lines.push("--- Failures ---"); + for (const f of failures) { + lines.push(` ${f.name}: ${f.detail}`); + } + lines.push(""); + } + + // Token savings summary + const savingsResults = results.filter((r) => r.savings !== undefined); + if (savingsResults.length > 0) { + const avgSavings = Math.round( + savingsResults.reduce((sum, r) => sum + (r.savings ?? 0), 0) / + savingsResults.length + ); + const minSavings = Math.min( + ...savingsResults.map((r) => r.savings ?? 100) + ); + const maxSavings = Math.max(...savingsResults.map((r) => r.savings ?? 0)); + lines.push("--- Token Savings ---"); + lines.push(`Average: ${avgSavings}%`); + lines.push(`Min: ${minSavings}%`); + lines.push(`Max: ${maxSavings}%`); + lines.push(""); + } + + // Verdict + lines.push("======================================================"); + if (failed === 0) { + lines.push(" Verdict: READY FOR RELEASE"); + } else { + lines.push(` Verdict: NOT READY (${failed} failures)`); + } + lines.push("======================================================"); + + return lines.join("\n"); +} + +/** Save report to file */ +export async function saveReport( + buildInfo: BuildInfo, + outPath: string +): Promise { + const report = generateReport(buildInfo); + await Bun.write(outPath, report); + console.log(`\nReport saved to: ${outPath}`); + return report; +} diff --git a/scripts/benchmark/lib/test.ts b/scripts/benchmark/lib/test.ts new file mode 100644 index 000000000..cffe6148b --- /dev/null +++ b/scripts/benchmark/lib/test.ts @@ -0,0 +1,167 @@ +/** + * Test helpers for RTK integration testing. + */ + +import { vmExec, RTK_BIN } from "./vm"; + +export type TestStatus = "PASS" | "FAIL" | "SKIP"; + +export interface TestResult { + name: string; + status: TestStatus; + detail: string; + exitCode?: number; + outputSize?: number; + savings?: number; + duration?: number; +} + +const results: TestResult[] = []; + +export function getResults(): TestResult[] { + return results; +} + +export function getCounts() { + const total = results.length; + const passed = results.filter((r) => r.status === "PASS").length; + const failed = results.filter((r) => r.status === "FAIL").length; + const skipped = results.filter((r) => r.status === "SKIP").length; + return { total, passed, failed, skipped }; +} + +function record(result: TestResult) { + results.push(result); + const icon = + result.status === "PASS" + ? "\x1b[32mPASS\x1b[0m" + : result.status === "FAIL" + ? "\x1b[31mFAIL\x1b[0m" + : "\x1b[33mSKIP\x1b[0m"; + console.log(` ${icon} | ${result.name} | ${result.detail}`); +} + +/** + * Test a command exits with expected code and doesn't crash. + * expectedExit: number or "any" (just checks no signal death) + */ +export async function testCmd( + name: string, + cmd: string, + expectedExit: number | "any" = 0 +): Promise { + const start = Date.now(); + const { stdout, stderr, exitCode } = await vmExec(cmd); + const duration = Date.now() - start; + const outputSize = stdout.length + stderr.length; + + let status: TestStatus; + let detail: string; + + if (expectedExit === "any") { + // Just check it didn't die from signal (exit >= 128) + if (exitCode < 128) { + status = "PASS"; + detail = `exit=${exitCode} | ${outputSize}b | ${duration}ms`; + } else { + status = "FAIL"; + detail = `SIGNAL exit=${exitCode} | ${outputSize}b`; + } + } else if (exitCode === expectedExit) { + status = "PASS"; + detail = `exit=${exitCode} | ${outputSize}b | ${duration}ms`; + } else { + status = "FAIL"; + detail = `expected exit=${expectedExit}, got ${exitCode} | ${outputSize}b`; + } + + const result: TestResult = { + name, + status, + detail, + exitCode, + outputSize, + duration, + }; + record(result); + return result; +} + +/** + * Test token savings: compare raw command output vs RTK filtered output. + */ +export async function testSavings( + name: string, + rawCmd: string, + rtkCmd: string, + targetPct: number +): Promise { + const raw = await vmExec(rawCmd); + const rtk = await vmExec(rtkCmd); + + const rawSize = raw.stdout.length; + const rtkSize = rtk.stdout.length; + + if (rawSize === 0) { + const result: TestResult = { + name, + status: "SKIP", + detail: "raw output empty", + }; + record(result); + return result; + } + + const savings = Math.round(100 - (rtkSize * 100) / rawSize); + + let status: TestStatus; + let detail: string; + + if (savings >= targetPct) { + status = "PASS"; + detail = `raw=${rawSize}b filtered=${rtkSize}b savings=${savings}% (target: >=${targetPct}%)`; + } else { + status = "FAIL"; + detail = `savings=${savings}% < target ${targetPct}% (raw=${rawSize}b filtered=${rtkSize}b)`; + } + + const result: TestResult = { name, status, detail, savings }; + record(result); + return result; +} + +/** + * Test rewrite engine: input -> expected output. + */ +export async function testRewrite( + input: string, + expected: string +): Promise { + const escaped = input.replace(/'/g, "'\\''"); + const { stdout } = await vmExec(`${RTK_BIN} rewrite '${escaped}'`); + const actual = stdout.trim(); + + let status: TestStatus; + let detail: string; + + if (actual === expected) { + status = "PASS"; + detail = `'${input}' -> '${actual}'`; + } else { + status = "FAIL"; + detail = `'${input}' -> expected '${expected}', got '${actual}'`; + } + + const result: TestResult = { name: `rewrite: ${input}`, status, detail }; + record(result); + return result; +} + +/** + * Skip a test with a reason. + */ +export function skipTest(name: string, reason: string): TestResult { + const result: TestResult = { name, status: "SKIP", detail: reason }; + record(result); + return result; +} diff --git a/scripts/benchmark/lib/vm.ts b/scripts/benchmark/lib/vm.ts new file mode 100644 index 000000000..fcbf3a815 --- /dev/null +++ b/scripts/benchmark/lib/vm.ts @@ -0,0 +1,181 @@ +/** + * Multipass VM management for RTK integration testing. + */ + +import { $ } from "bun"; + +const VM_NAME = "rtk-test"; +const CLOUD_INIT = "scripts/benchmark/cloud-init.yaml"; + +export interface VmInfo { + name: string; + state: string; + ipv4: string; +} + +/** Check if VM exists and is running */ +export async function vmExists(): Promise { + const result = await $`multipass list --format json`.quiet(); + const data = JSON.parse(result.stdout.toString()); + return data.list?.some((vm: VmInfo) => vm.name === VM_NAME) ?? false; +} + +/** Check if VM is running */ +export async function vmRunning(): Promise { + const result = await $`multipass list --format json`.quiet(); + const data = JSON.parse(result.stdout.toString()); + const vm = data.list?.find((v: VmInfo) => v.name === VM_NAME); + return vm?.state === "Running"; +} + +/** Create a new VM with cloud-init (20 min timeout for full provisioning) */ +export async function vmCreate(): Promise { + console.log(`[vm] Creating ${VM_NAME} with cloud-init (this takes ~10-15 min)...`); + // --timeout 1200 = 20 min for cloud-init to finish installing Rust, Go, Node, .NET, etc. + await $`multipass launch --name ${VM_NAME} --cpus 2 --memory 4G --disk 20G --timeout 1200 --cloud-init ${CLOUD_INIT} 24.04`; +} + +/** Start existing VM */ +export async function vmStart(): Promise { + console.log(`[vm] Starting ${VM_NAME}...`); + await $`multipass start ${VM_NAME}`; +} + +/** Execute a command in the VM, returns stdout (60s timeout per test by default) */ +export async function vmExec( + cmd: string, + timeoutMs = 60_000 +): Promise<{ + stdout: string; + stderr: string; + exitCode: number; +}> { + const exec = $`multipass exec ${VM_NAME} -- bash -c ${cmd}` + .quiet() + .nothrow() + .then((r) => ({ + stdout: r.stdout.toString(), + stderr: r.stderr.toString(), + exitCode: r.exitCode, + })); + + const timeout = new Promise<{ stdout: string; stderr: string; exitCode: number }>((_, reject) => + setTimeout(() => reject(new Error(`vmExec timed out after ${timeoutMs}ms: ${cmd}`)), timeoutMs) + ); + + return Promise.race([exec, timeout]); +} + +/** Transfer a file to the VM */ +export async function vmTransfer( + localPath: string, + remotePath: string +): Promise { + await $`multipass transfer ${localPath} ${VM_NAME}:${remotePath}`; +} + +/** Wait for cloud-init to complete (max 40 min — installs Rust, Go, Node, .NET, etc.) */ +export async function vmWaitReady(maxWaitSec = 2400): Promise { + console.log("[vm] Waiting for cloud-init..."); + const start = Date.now(); + while ((Date.now() - start) / 1000 < maxWaitSec) { + const { exitCode } = await vmExec( + "test -f /home/ubuntu/.cloud-init-complete" + ); + if (exitCode === 0) { + const elapsed = Math.round((Date.now() - start) / 1000); + console.log(`[vm] Cloud-init complete after ${elapsed}s`); + return true; + } + await Bun.sleep(10_000); + } + console.error("[vm] Cloud-init timed out!"); + return false; +} + +/** Transfer RTK source and build in release mode */ +export async function vmBuildRtk(projectRoot: string): Promise<{ + buildTime: number; + binarySize: number; + version: string; +}> { + console.log("[vm] Transferring RTK source..."); + + // Create tarball excluding heavy dirs and macOS resource forks (._*) + await $`COPYFILE_DISABLE=1 tar czf /tmp/rtk-src.tar.gz --exclude target --exclude .git --exclude node_modules --exclude "index.html*" --exclude "._*" -C ${projectRoot} .`; + await vmTransfer("/tmp/rtk-src.tar.gz", "/tmp/rtk-src.tar.gz"); + await vmExec( + "mkdir -p /home/ubuntu/rtk && cd /home/ubuntu/rtk && tar xzf /tmp/rtk-src.tar.gz" + ); + + console.log("[vm] Building RTK (release)..."); + const start = Date.now(); + const { stdout, exitCode } = await vmExec( + "export PATH=$HOME/.cargo/bin:$PATH && cd /home/ubuntu/rtk && cargo build --release 2>&1 | tail -5" + ); + const buildTime = Math.round((Date.now() - start) / 1000); + + if (exitCode !== 0) { + throw new Error(`Build failed:\n${stdout}`); + } + + const { stdout: sizeStr } = await vmExec( + "stat -c%s /home/ubuntu/rtk/target/release/rtk" + ); + const binarySize = parseInt(sizeStr.trim(), 10); + + const { stdout: version } = await vmExec( + "/home/ubuntu/rtk/target/release/rtk --version" + ); + + console.log( + `[vm] Build OK in ${buildTime}s — ${binarySize} bytes — ${version.trim()}` + ); + + return { buildTime, binarySize, version: version.trim() }; +} + +/** Delete the VM */ +export async function vmDelete(): Promise { + console.log(`[vm] Deleting ${VM_NAME}...`); + await $`multipass delete ${VM_NAME} --purge`.nothrow(); +} + +/** Ensure VM is ready (create or reuse) */ +export async function vmEnsureReady(): Promise { + if (await vmExists()) { + if (!(await vmRunning())) { + await vmStart(); + } + console.log(`[vm] Reusing existing VM ${VM_NAME}`); + // Check if cloud-init is still running + const { exitCode } = await vmExec( + "test -f /home/ubuntu/.cloud-init-complete" + ); + if (exitCode !== 0) { + console.log("[vm] Cloud-init still running, waiting..."); + const ready = await vmWaitReady(); + if (!ready) { + throw new Error( + "Cloud-init timed out. Check: multipass exec rtk-test -- cat /var/log/cloud-init-output.log" + ); + } + } + } else { + await vmCreate(); + // multipass launch --timeout should wait, but double-check + const { exitCode } = await vmExec( + "test -f /home/ubuntu/.cloud-init-complete" + ); + if (exitCode !== 0) { + const ready = await vmWaitReady(); + if (!ready) { + throw new Error( + "Cloud-init timed out. Check: multipass exec rtk-test -- cat /var/log/cloud-init-output.log" + ); + } + } + } +} + +export const RTK_BIN = "/home/ubuntu/rtk/target/release/rtk"; diff --git a/scripts/benchmark/rebuild.ts b/scripts/benchmark/rebuild.ts new file mode 100644 index 000000000..1d06277ff --- /dev/null +++ b/scripts/benchmark/rebuild.ts @@ -0,0 +1,17 @@ +#!/usr/bin/env bun +/** + * Fast rebuild: reuse existing VM, just transfer source and recompile. + * Usage: bun run scripts/benchmark/rebuild.ts + */ + +import { vmEnsureReady, vmBuildRtk } from "./lib/vm"; + +const PROJECT_ROOT = new URL("../../", import.meta.url).pathname.replace(/\/$/, ""); + +await vmEnsureReady(); +const info = await vmBuildRtk(PROJECT_ROOT); + +console.log(`\nRebuild complete:`); +console.log(` Version: ${info.version}`); +console.log(` Binary: ${info.binarySize} bytes`); +console.log(` Time: ${info.buildTime}s`); diff --git a/scripts/benchmark/run.ts b/scripts/benchmark/run.ts new file mode 100644 index 000000000..3d964963c --- /dev/null +++ b/scripts/benchmark/run.ts @@ -0,0 +1,425 @@ +#!/usr/bin/env bun +/** + * RTK Full Integration Test Suite — Multipass VM + * + * Usage: + * bun run scripts/benchmark/run.ts # Full suite + * bun run scripts/benchmark/run.ts --quick # Skip slow phases (perf, concurrency) + * bun run scripts/benchmark/run.ts --phase 3 # Run specific phase only + * + * Prerequisites: + * brew install multipass + */ + +import { $ } from "bun"; +import { vmEnsureReady, vmBuildRtk, vmExec, RTK_BIN } from "./lib/vm"; +import { testCmd, testSavings, testRewrite, skipTest, getCounts } from "./lib/test"; +import { saveReport } from "./lib/report"; + +const args = process.argv.slice(2); +const quick = args.includes("--quick"); +const phaseArg = args.includes("--phase") + ? parseInt(args[args.indexOf("--phase") + 1], 10) + : null; +const phaseOnly = phaseArg !== null && !Number.isNaN(phaseArg) ? phaseArg : null; +if (args.includes("--phase") && phaseOnly === null) { + console.error("Error: --phase requires a number (e.g. --phase 3)"); + process.exit(1); +} +const reportPath = args.includes("--report") + ? args[args.indexOf("--report") + 1] + : `${new URL("../../", import.meta.url).pathname.replace(/\/$/, "")}/benchmark-report.txt`; + +const PROJECT_ROOT = new URL("../../", import.meta.url).pathname.replace(/\/$/, ""); +const RTK = RTK_BIN; + +function shouldRun(phase: number): boolean { + return phaseOnly === null || phaseOnly === phase; +} + +function heading(phase: number, title: string) { + console.log(`\n\x1b[34m[Phase ${phase}] ${title}\x1b[0m`); +} + +// ══════════════════════════════════════════════════════════════ +// Phase 0: VM Setup +// ══════════════════════════════════════════════════════════════ + +console.log("\x1b[34m[rtk-test] RTK Full Integration Test Suite\x1b[0m"); +console.log(`Project: ${PROJECT_ROOT}`); + +await vmEnsureReady(); + +// ══════════════════════════════════════════════════════════════ +// Phase 1: Transfer & Build +// ══════════════════════════════════════════════════════════════ + +heading(1, "Transfer & Build"); +const branch = (await $`git -C ${PROJECT_ROOT} branch --show-current`.text()).trim(); +const commit = (await $`git -C ${PROJECT_ROOT} log --oneline -1`.text()).trim(); +const buildInfo = await vmBuildRtk(PROJECT_ROOT); + +// Binary size check +// ARM Linux release binaries are ~6.5MB (vs ~4MB x86 stripped). +// CLAUDE.md target is <5MB for stripped x86 release builds. +// VM builds are ARM + not fully stripped, so we use a relaxed 8MB limit here. +const sizeLimit = 8_388_608; // 8MB (relaxed for ARM Linux VM) +if (buildInfo.binarySize < sizeLimit) { + console.log(` \x1b[32mPASS\x1b[0m | binary size | ${buildInfo.binarySize} bytes < 8MB`); +} else { + console.log(` \x1b[31mFAIL\x1b[0m | binary size | ${buildInfo.binarySize} bytes >= 8MB`); +} + +// ══════════════════════════════════════════════════════════════ +// Phase 2: Cargo Quality (fmt, clippy, test) +// ══════════════════════════════════════════════════════════════ + +if (shouldRun(2)) { + heading(2, "Cargo Quality"); + + await testCmd( + "quality:cargo fmt", + "export PATH=$HOME/.cargo/bin:$PATH && cd /home/ubuntu/rtk && cargo fmt --all --check 2>&1" + ); + + await testCmd( + "quality:cargo clippy", + "export PATH=$HOME/.cargo/bin:$PATH && cd /home/ubuntu/rtk && cargo clippy --all-targets -- -D warnings 2>&1" + ); + + await testCmd( + "quality:cargo test", + "export PATH=$HOME/.cargo/bin:$PATH && cd /home/ubuntu/rtk && cargo test --all 2>&1" + ); +} + +// ══════════════════════════════════════════════════════════════ +// Phase 3: Rust Built-in Commands +// ══════════════════════════════════════════════════════════════ + +if (shouldRun(3)) { + heading(3, "Rust Built-in Commands"); + + // Git + await testCmd("git:status", `cd /tmp/test-git && ${RTK} git status`); + await testCmd("git:log", `cd /tmp/test-git && ${RTK} git log -5`); + await testCmd("git:log --oneline", `cd /tmp/test-git && ${RTK} git log --oneline -10`); + await testCmd("git:diff", `cd /tmp/test-git && ${RTK} git diff`, "any"); + await testCmd("git:branch", `cd /tmp/test-git && ${RTK} git branch`); + await testCmd("git:add --dry-run", `cd /tmp/test-git && ${RTK} git add --dry-run .`, "any"); + + // Files + await testCmd("files:ls", `${RTK} ls /home/ubuntu/rtk`); + await testCmd("files:ls src/", `${RTK} ls /home/ubuntu/rtk/src/`); + await testCmd("files:ls -R", `${RTK} ls -R /home/ubuntu/rtk/src/`); + await testCmd("files:read", `${RTK} read /home/ubuntu/rtk/src/main.rs`); + await testCmd("files:read aggressive", `${RTK} read /home/ubuntu/rtk/src/main.rs -l aggressive`); + await testCmd("files:smart", `${RTK} smart /home/ubuntu/rtk/src/main.rs`); + await testCmd("files:find *.rs", `${RTK} find '*.rs' /home/ubuntu/rtk/src/`); + await testCmd("files:wc", `${RTK} wc /home/ubuntu/rtk/src/main.rs`); + await testCmd("files:diff", `${RTK} diff /home/ubuntu/rtk/src/main.rs /home/ubuntu/rtk/src/utils.rs`); + + // Search + await testCmd("search:grep", `${RTK} grep 'fn main' /home/ubuntu/rtk/src/`); + + // Data + await testCmd("data:json", `${RTK} json /tmp/test-node/package.json`); + await testCmd("data:deps", `cd /home/ubuntu/rtk && ${RTK} deps`); + await testCmd("data:env", `${RTK} env`); + + // Runners + await testCmd("runner:summary", `${RTK} summary 'echo hello world'`); + // BUG: rtk err swallows exit code — tracked in #846 + await testCmd("runner:err", `${RTK} err false`, "any"); + await testCmd("runner:test", `${RTK} test 'echo ok'`, "any"); + + // Logs + await testCmd("log:large", `${RTK} log /tmp/large.log`); + + // Network + await testCmd("net:curl", `${RTK} curl https://httpbin.org/get`, "any"); + + // GitHub + await testCmd("gh:pr list", `cd /home/ubuntu/rtk && ${RTK} gh pr list`, "any"); + + // Cargo (test project has intentional test failure → exit 101) + await testCmd("cargo:build", `export PATH=$HOME/.cargo/bin:$PATH && cd /tmp/test-rust && ${RTK} cargo build`); + await testCmd("cargo:test", `export PATH=$HOME/.cargo/bin:$PATH && cd /tmp/test-rust && ${RTK} cargo test`, 101); + await testCmd("cargo:clippy", `export PATH=$HOME/.cargo/bin:$PATH && cd /tmp/test-rust && ${RTK} cargo clippy`); + + // Python (test project has intentional failures) + await testCmd("python:pytest", `cd /tmp/test-python && ${RTK} pytest`, 1); + await testCmd("python:ruff check", `cd /tmp/test-python && ${RTK} ruff check .`, 1); + await testCmd("python:mypy", `cd /tmp/test-python && ${RTK} mypy .`, 1); + await testCmd("python:pip list", `${RTK} pip list`); + + // Go (test project has intentional test failure) + await testCmd("go:test", `export PATH=$PATH:/usr/local/go/bin && cd /tmp/test-go && ${RTK} go test ./...`, 1); + await testCmd("go:build", `export PATH=$PATH:/usr/local/go/bin && cd /tmp/test-go && ${RTK} go build .`, 1); + await testCmd("go:vet", `export PATH=$PATH:/usr/local/go/bin && cd /tmp/test-go && ${RTK} go vet ./...`, 1); + await testCmd("go:golangci-lint", `export PATH=$PATH:/usr/local/go/bin:$HOME/go/bin && cd /tmp/test-go && ${RTK} golangci-lint run`, 1); + + // TypeScript + await testCmd("ts:tsc", `cd /tmp/test-node && ${RTK} tsc --noEmit`, "any"); + + // Linters + await testCmd("lint:eslint", `cd /tmp/test-node && ${RTK} lint 'eslint src/'`, "any"); + await testCmd("lint:prettier", `cd /tmp/test-node && ${RTK} prettier --check src/`, "any"); + + // Docker + await testCmd("docker:ps", `${RTK} docker ps`, "any"); + await testCmd("docker:images", `${RTK} docker images`, "any"); + + // Kubernetes + await testCmd("k8s:pods", `${RTK} kubectl pods`, "any"); + + // .NET + await testCmd("dotnet:build", `export DOTNET_ROOT=/usr/local/share/dotnet && export PATH=$PATH:$DOTNET_ROOT && cd /tmp/test-dotnet/TestApp 2>/dev/null && ${RTK} dotnet build || echo 'dotnet skip'`, "any"); + + // Meta + await testCmd("meta:gain", `${RTK} gain`); + await testCmd("meta:gain --history", `${RTK} gain --history`); + await testCmd("meta:proxy", `${RTK} proxy echo 'proxy test'`); + await testCmd("meta:verify", `${RTK} verify`, "any"); +} + +// ══════════════════════════════════════════════════════════════ +// Phase 4: TOML Filter Commands +// ══════════════════════════════════════════════════════════════ + +if (shouldRun(4)) { + heading(4, "TOML Filter Commands"); + + // System + await testCmd("toml:df", `${RTK} df -h`); + await testCmd("toml:du", `${RTK} du -sh /tmp`, "any"); + await testCmd("toml:ps", `${RTK} ps aux`); + await testCmd("toml:ping", `${RTK} ping -c 2 127.0.0.1`); + + // Build tools + await testCmd("toml:make", `cd /tmp && ${RTK} make -f Makefile`, "any"); + await testCmd("toml:rsync", `${RTK} rsync --version`); + + // Linters + await testCmd("toml:shellcheck", `${RTK} shellcheck /tmp/test.sh`, "any"); + await testCmd("toml:hadolint", `${RTK} hadolint /tmp/Dockerfile.bad`, "any"); + await testCmd("toml:yamllint", `${RTK} yamllint /tmp/test.yaml`, "any"); + await testCmd("toml:markdownlint", `${RTK} markdownlint /tmp/test.md`, "any"); + + // Cloud/Infra + await testCmd("toml:terraform", `${RTK} terraform --version`, "any"); + await testCmd("toml:helm", `${RTK} helm version`, "any"); + await testCmd("toml:ansible", `${RTK} ansible-playbook --version`, "any"); + + // Mocked tools + await testCmd("toml:gcloud", `${RTK} gcloud version`); + await testCmd("toml:shopify", `${RTK} shopify theme check`, "any"); + await testCmd("toml:pio", `${RTK} pio run`, "any"); + await testCmd("toml:quarto", `${RTK} quarto render`, "any"); + await testCmd("toml:sops", `${RTK} sops --version`); + // Swift ecosystem + await testCmd("toml:swift build", `${RTK} swift build`, "any"); + await testCmd("toml:swift test", `${RTK} swift test`, "any"); + await testCmd("toml:swift run", `${RTK} swift run`, "any"); + await testCmd("toml:swift package", `${RTK} swift package resolve`, "any"); + await testCmd("toml:swiftlint", `${RTK} swiftlint`, "any"); + await testCmd("toml:swiftformat", `${RTK} swiftformat`, "any"); + await testCmd("toml:kubectl", `${RTK} kubectl version --client`, "any"); +} + +// ══════════════════════════════════════════════════════════════ +// Phase 5: Hook Rewrite Engine +// ══════════════════════════════════════════════════════════════ + +if (shouldRun(5)) { + heading(5, "Hook Rewrite Engine"); + + // Basic rewrites + await testRewrite("git status", "rtk git status"); + await testRewrite("git log --oneline -10", "rtk git log --oneline -10"); + await testRewrite("cargo test", "rtk cargo test"); + await testRewrite("cargo build --release", "rtk cargo build --release"); + await testRewrite("docker ps", "rtk docker ps"); + // NOTE: rtk rewrites "kubectl get pods" to "rtk kubectl get pods" (preserves get) + await testRewrite("kubectl get pods", "rtk kubectl get pods"); + await testRewrite("ruff check", "rtk ruff check"); + await testRewrite("pytest", "rtk pytest"); + await testRewrite("go test", "rtk go test"); + await testRewrite("pnpm list", "rtk pnpm list"); + await testRewrite("gh pr list", "rtk gh pr list"); + await testRewrite("df -h", "rtk df -h"); + await testRewrite("ps aux", "rtk ps aux"); + + // Compound + await testRewrite("cargo test && git status", "rtk cargo test && rtk git status"); + // NOTE: shell strips single quotes in vmExec, so 'msg' becomes msg + await testRewrite("git add . && git commit -m msg", "rtk git add . && rtk git commit -m msg"); + + // No rewrite (shell builtins) — rtk rewrite returns empty string + exit 1 + // We test via testCmd since testRewrite expects non-empty output + await testCmd("rewrite:cd (no rewrite)", `${RTK} rewrite 'cd /tmp'`, 1); + await testCmd("rewrite:export (no rewrite)", `${RTK} rewrite 'export FOO=bar'`, 1); +} + +// ══════════════════════════════════════════════════════════════ +// Phase 6: Exit Code Preservation +// ══════════════════════════════════════════════════════════════ + +if (shouldRun(6)) { + heading(6, "Exit Code Preservation"); + + // Success + await testCmd("exit:git status=0", `cd /tmp/test-git && ${RTK} git status`, 0); + await testCmd("exit:ls=0", `${RTK} ls /tmp`, 0); + await testCmd("exit:gain=0", `${RTK} gain`, 0); + + // Failures + // rg returns exit 1 (no match) or 2 (error) — accept both + await testCmd("exit:grep NOTFOUND", `${RTK} grep NOTFOUND_XYZ_123 /tmp`, "any"); +} + +// ══════════════════════════════════════════════════════════════ +// Phase 7: Token Savings +// ══════════════════════════════════════════════════════════════ + +if (shouldRun(7)) { + heading(7, "Token Savings"); + + await testSavings( + "savings:git log", + "cd /tmp/test-git && git log -20", + `cd /tmp/test-git && ${RTK} git log -20`, + 60 + ); + await testSavings( + "savings:ls", + "ls -la /home/ubuntu/rtk/src/", + `${RTK} ls /home/ubuntu/rtk/src/`, + 60 + ); + await testSavings( + "savings:log dedup", + "cat /tmp/large.log", + `${RTK} log /tmp/large.log`, + 80 + ); + await testSavings( + "savings:read aggressive", + "cat /home/ubuntu/rtk/src/main.rs", + `${RTK} read /home/ubuntu/rtk/src/main.rs -l aggressive`, + 50 + ); + await testSavings( + "savings:swift test", + "swift test", + `${RTK} swift test`, + 60 + ); + await testSavings( + "savings:swiftlint", + "swiftlint", + `${RTK} swiftlint`, + 20 + ); +} + +// ══════════════════════════════════════════════════════════════ +// Phase 8: Pipe Compatibility +// ══════════════════════════════════════════════════════════════ + +if (shouldRun(8)) { + heading(8, "Pipe Compatibility"); + + await testCmd("pipe:git status|wc", `cd /tmp/test-git && ${RTK} git status | wc -l`); + await testCmd("pipe:ls|wc", `${RTK} ls /home/ubuntu/rtk/src/ | wc -l`); + await testCmd("pipe:grep|head", `${RTK} grep 'fn' /home/ubuntu/rtk/src/ | head -5`); +} + +// ══════════════════════════════════════════════════════════════ +// Phase 9: Edge Cases +// ══════════════════════════════════════════════════════════════ + +if (shouldRun(9)) { + heading(9, "Edge Cases"); + + await testCmd("edge:summary true", `${RTK} summary 'true'`, "any"); + await testCmd("edge:grep NOTFOUND", `${RTK} grep NOTFOUND_XYZ /home/ubuntu/rtk/src/`, 1); + await testCmd("edge:unicode", `echo 'hello world' > /tmp/uni.txt && ${RTK} grep 'hello' /tmp`, "any"); +} + +// ══════════════════════════════════════════════════════════════ +// Phase 10: Performance (skip with --quick) +// ══════════════════════════════════════════════════════════════ + +if (shouldRun(10) && !quick) { + heading(10, "Performance"); + + // hyperfine + const { exitCode: hfExist } = await vmExec("command -v hyperfine"); + if (hfExist === 0) { + const { stdout: hfOut } = await vmExec( + `cd /tmp/test-git && hyperfine --warmup 3 --min-runs 5 '${RTK} git status' 'git status' --export-json /dev/stdout 2>/dev/null` + ); + try { + const hf = JSON.parse(hfOut); + const rtkMean = (hf.results?.[0]?.mean * 1000).toFixed(1); + const rawMean = (hf.results?.[1]?.mean * 1000).toFixed(1); + console.log(` Startup: rtk=${rtkMean}ms raw=${rawMean}ms`); + } catch { + console.log(" hyperfine output parse failed"); + } + } else { + skipTest("perf:hyperfine", "not installed"); + } + + // Memory + const { stdout: memOut } = await vmExec( + `cd /tmp/test-git && /usr/bin/time -v ${RTK} git status 2>&1 | grep 'Maximum resident'` + ); + const memKb = parseInt(memOut.match(/(\d+)/)?.[1] ?? "0", 10); + if (memKb > 0 && memKb < 20000) { + await testCmd("perf:memory", `echo '${memKb} KB < 20MB'`); + } else if (memKb > 0) { + await testCmd("perf:memory", `echo '${memKb} KB >= 20MB' && exit 1`, 0); + } +} else if (quick && shouldRun(10)) { + skipTest("perf:hyperfine", "--quick mode"); + skipTest("perf:memory", "--quick mode"); +} + +// ══════════════════════════════════════════════════════════════ +// Phase 11: Concurrency (skip with --quick) +// ══════════════════════════════════════════════════════════════ + +if (shouldRun(11) && !quick) { + heading(11, "Concurrency"); + + await testCmd( + "concurrency:10x git status", + `cd /tmp/test-git && for i in $(seq 1 10); do ${RTK} git status >/dev/null & done; wait` + ); +} else if (quick && shouldRun(11)) { + skipTest("concurrency:10x", "--quick mode"); +} + +// ══════════════════════════════════════════════════════════════ +// Report +// ══════════════════════════════════════════════════════════════ + +const report = await saveReport( + { ...buildInfo, branch, commit }, + reportPath +); + +console.log("\n" + report); + +const { total, passed, failed, skipped } = getCounts(); +const passRate = total > 0 ? Math.round((passed * 100) / total) : 0; + +if (failed === 0) { + console.log(`\n\x1b[32m READY FOR RELEASE — ${passed}/${total} (${passRate}%)\x1b[0m\n`); + process.exit(0); +} else { + console.log(`\n\x1b[31m NOT READY — ${failed} failures — ${passed}/${total} (${passRate}%)\x1b[0m\n`); + process.exit(1); +} diff --git a/scripts/check-installation.sh b/scripts/check-installation.sh index e7a56fb7a..ce0d2eb73 100755 --- a/scripts/check-installation.sh +++ b/scripts/check-installation.sh @@ -24,7 +24,7 @@ else echo -e " ${RED}❌ RTK is NOT installed${NC}" echo "" echo " Install with:" - echo " curl -fsSL https://github.com/rtk-ai/rtk/blob/master/install.sh| sh" + echo " curl -fsSL https://github.com/algolia/rtk/blob/main/install.sh| sh" exit 1 fi echo "" @@ -45,7 +45,7 @@ else echo "" echo " You installed the wrong package. Fix it with:" echo " cargo uninstall rtk" - echo " curl -fsSL https://github.com/rtk-ai/rtk/blob/master/install.sh | sh" + echo " curl -fsSL https://github.com/algolia/rtk/blob/main/install.sh | sh" CORRECT_RTK=false fi echo "" @@ -142,7 +142,7 @@ if [ ${#MISSING_FEATURES[@]} -gt 0 ]; then echo "" echo "To get all features, install the fork:" echo " cargo uninstall rtk" - echo " curl -fsSL https://github.com/rtk-ai/rtk/blob/master/install.sh | sh" + echo " curl -fsSL https://github.com/algolia/rtk/blob/main/install.sh | sh" echo " cd rtk && git checkout feat/all-features" echo " cargo install --path . --force" else diff --git a/scripts/check-test-presence.sh b/scripts/check-test-presence.sh new file mode 100755 index 000000000..08147c4dc --- /dev/null +++ b/scripts/check-test-presence.sh @@ -0,0 +1,69 @@ +#!/usr/bin/env bash +set -euo pipefail + +# check-test-presence.sh — CI guard: new/modified *_cmd.rs files must have #[cfg(test)] +# +# Usage: +# bash scripts/check-test-presence.sh [BASE_BRANCH] +# bash scripts/check-test-presence.sh --self-test +# +# BASE_BRANCH defaults to origin/develop + +if [ "${1:-}" = "--self-test" ]; then + # Self-test: create a tempfile without tests and verify the check catches it + TMPFILE="src/cmds/system/_rtk_check_self_test_cmd.rs" + echo "pub fn run() {}" > "$TMPFILE" + trap 'rm -f "$TMPFILE"' EXIT + + if grep -q '#\[cfg(test)\]' "$TMPFILE"; then + echo "FAIL: self-test broken (false negative)" + exit 1 + fi + rm "$TMPFILE" + trap - EXIT + echo "PASS: --self-test detection works correctly" + exit 0 +fi + +BASE_BRANCH="${1:-origin/develop}" +EXIT_CODE=0 + +# Find *_cmd.rs files that were added or modified in this PR +CHANGED_FILES=$(git diff --name-only --diff-filter=AM --no-renames "$BASE_BRANCH"...HEAD \ + 2>/dev/null | grep -E 'src/cmds/.+_cmd\.rs$' || true) + +if [ -z "$CHANGED_FILES" ]; then + echo "check-test-presence: no *_cmd.rs changes detected — OK" + exit 0 +fi + +echo "check-test-presence: checking $(echo "$CHANGED_FILES" | wc -l | tr -d ' ') filter module(s)..." +echo "" + +while IFS= read -r file; do + if [ ! -f "$file" ]; then + continue + fi + + if grep -q '#\[cfg(test)\]' "$file"; then + echo " PASS $file" + else + echo " FAIL $file" + echo " Missing #[cfg(test)] module." + echo " Every *_cmd.rs filter must include inline unit tests." + echo " Reference: src/cmds/cloud/aws_cmd.rs" + echo "" + EXIT_CODE=1 + fi +done <<< "$CHANGED_FILES" + +echo "" + +if [ "$EXIT_CODE" -ne 0 ]; then + echo "check-test-presence: FAILED — add tests before merging." + echo "See .claude/rules/cli-testing.md for the testing guide." +else + echo "check-test-presence: all filter modules have tests — OK" +fi + +exit "$EXIT_CODE" diff --git a/scripts/validate-docs.sh b/scripts/validate-docs.sh index 508f40a71..e7c3a1592 100755 --- a/scripts/validate-docs.sh +++ b/scripts/validate-docs.sh @@ -3,41 +3,25 @@ set -e echo "🔍 Validating RTK documentation consistency..." -# 1. Nombre de modules cohérent -MAIN_MODULES=$(grep -c '^mod ' src/main.rs) -echo "📊 Module count in main.rs: $MAIN_MODULES" - -# Extract module count from ARCHITECTURE.md -if [ -f "ARCHITECTURE.md" ]; then - ARCH_MODULES=$(grep 'Total:.*modules' ARCHITECTURE.md | grep -o '[0-9]\+' | head -1) - if [ -z "$ARCH_MODULES" ]; then - echo "⚠️ Could not extract module count from ARCHITECTURE.md" - else - echo "📊 Module count in ARCHITECTURE.md: $ARCH_MODULES" - if [ "$MAIN_MODULES" != "$ARCH_MODULES" ]; then - echo "❌ Module count mismatch: main.rs=$MAIN_MODULES, ARCHITECTURE.md=$ARCH_MODULES" - exit 1 - fi - fi -fi +# 1. Source file count sanity check +SRC_FILES=$(find src -name "*.rs" ! -name "mod.rs" ! -name "main.rs" | wc -l | tr -d ' ') +echo "📊 Rust source files in src/: $SRC_FILES" # 3. Commandes Python/Go présentes partout PYTHON_GO_CMDS=("ruff" "pytest" "pip" "go" "golangci") echo "🐍 Checking Python/Go commands documentation..." for cmd in "${PYTHON_GO_CMDS[@]}"; do - for file in README.md CLAUDE.md; do - if [ ! -f "$file" ]; then - echo "⚠️ $file not found, skipping" - continue - fi - if ! grep -q "$cmd" "$file"; then - echo "❌ $file ne mentionne pas commande $cmd" - exit 1 - fi - done + if [ ! -f "README.md" ]; then + echo "⚠️ README.md not found, skipping" + break + fi + if ! grep -q "$cmd" "README.md"; then + echo "❌ README.md ne mentionne pas commande $cmd" + exit 1 + fi done -echo "✅ Python/Go commands: documented in README.md and CLAUDE.md" +echo "✅ Python/Go commands: documented in README.md" # 4. Hooks cohérents avec doc HOOK_FILE=".claude/hooks/rtk-rewrite.sh" diff --git a/src/analytics/README.md b/src/analytics/README.md index 584b52d40..5cea9ce77 100644 --- a/src/analytics/README.md +++ b/src/analytics/README.md @@ -1,10 +1,10 @@ # Analytics -> See also [docs/TECHNICAL.md](../../docs/TECHNICAL.md) for the full architecture overview +> See also [docs/contributing/TECHNICAL.md](../../docs/contributing/TECHNICAL.md) for the full architecture overview ## Scope -**Read-only dashboards** over the tracking database. Analytics presents the value that `cmds/` creates — it queries token savings, correlates with external spending data, and surfaces adoption opportunities. It never modifies the tracking DB. +**Read-only dashboards** over the tracking database. Queries token savings, correlates with external spending data, and surfaces adoption metrics. Never modifies the tracking DB. Owns: `rtk gain` (savings dashboard), `rtk cc-economics` (cost reduction), `rtk session` (adoption analysis), and Claude Code usage data parsing. @@ -15,7 +15,7 @@ Boundary rule: if a new module writes to the DB, it belongs in `core/` or `cmds/ ## Purpose Token savings analytics, economic modeling, and adoption metrics. -These modules read from the SQLite tracking database to produce dashboards, spending estimates, and session-level adoption reports that help users understand the value RTK provides. +These modules read from the SQLite tracking database to produce dashboards, spending estimates, and session-level adoption reports. ## Adding New Functionality To add a new analytics view: (1) create a new `*_cmd.rs` file in this directory, (2) query `core/tracking` for the metrics you need using the existing `TrackingDb` API, (3) register the command in `main.rs` under the `Commands` enum, and (4) add `#[cfg(test)]` unit tests with sample tracking data. Analytics modules should be read-only against the tracking database and never modify it. diff --git a/src/analytics/cc_economics.rs b/src/analytics/cc_economics.rs index 693dc61e2..037593102 100644 --- a/src/analytics/cc_economics.rs +++ b/src/analytics/cc_economics.rs @@ -14,9 +14,6 @@ use crate::core::utils::{format_cpt, format_tokens, format_usd}; // ── Constants ── -#[allow(dead_code)] -const BILLION: f64 = 1e9; - // API pricing ratios (verified Feb 2026, consistent across Claude models <=200K context) // Source: https://docs.anthropic.com/en/docs/about-claude/models const WEIGHT_OUTPUT: f64 = 5.0; // Output = 5x input diff --git a/src/analytics/ccusage.rs b/src/analytics/ccusage.rs index 49bd5bc8d..c291615b7 100644 --- a/src/analytics/ccusage.rs +++ b/src/analytics/ccusage.rs @@ -4,6 +4,7 @@ //! Claude Code API usage metrics. Handles subprocess execution, JSON parsing, //! and graceful degradation when ccusage is unavailable. +use crate::core::stream::exec_capture; use crate::core::utils::{resolved_command, tool_exists}; use anyhow::{Context, Result}; use serde::Deserialize; @@ -95,7 +96,9 @@ fn build_command() -> Option { } // Fallback: try npx + eprintln!("[info] ccusage not installed globally, fetching via npx..."); let npx_check = resolved_command("npx") + .arg("--yes") .arg("ccusage") .arg("--help") .stdout(std::process::Stdio::null()) @@ -104,6 +107,7 @@ fn build_command() -> Option { if npx_check.map(|s| s.success()).unwrap_or(false) { let mut cmd = resolved_command("npx"); + cmd.arg("--yes"); cmd.arg("ccusage"); return Some(cmd); } @@ -111,12 +115,6 @@ fn build_command() -> Option { None } -/// Check if ccusage CLI is available (binary or via npx) -#[allow(dead_code)] -pub fn is_available() -> bool { - build_command().is_some() -} - /// Fetch usage data from ccusage for the last 90 days /// /// Returns `Ok(None)` if ccusage is unavailable (graceful degradation) @@ -137,34 +135,30 @@ pub fn fetch(granularity: Granularity) -> Result>> { Granularity::Monthly => "monthly", }; - let output = cmd - .arg(subcommand) + cmd.arg(subcommand) .arg("--json") .arg("--since") - .arg("20250101") // 90 days back approx - .output(); + .arg("20250101"); // 90 days back approx - let output = match output { + let result = match exec_capture(&mut cmd) { Err(e) => { eprintln!("[warn] ccusage execution failed: {}", e); return Ok(None); } - Ok(o) => o, + Ok(r) => r, }; - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr); + if !result.success() { eprintln!( "[warn] ccusage exited with {}: {}", - output.status, - stderr.trim() + result.exit_code, + result.stderr.trim() ); return Ok(None); } - let stdout = String::from_utf8_lossy(&output.stdout); let periods = - parse_json(&stdout, granularity).context("Failed to parse ccusage JSON output")?; + parse_json(&result.stdout, granularity).context("Failed to parse ccusage JSON output")?; Ok(Some(periods)) } @@ -328,11 +322,4 @@ mod tests { assert_eq!(periods[0].metrics.cache_creation_tokens, 0); // default assert_eq!(periods[0].metrics.cache_read_tokens, 0); } - - #[test] - fn test_is_available() { - // Just smoke test - actual availability depends on system - let _available = is_available(); - // No assertion - just ensure it doesn't panic - } } diff --git a/src/analytics/gain.rs b/src/analytics/gain.rs index a43ebbeb0..9c8c2630a 100644 --- a/src/analytics/gain.rs +++ b/src/analytics/gain.rs @@ -24,11 +24,25 @@ pub fn run( all: bool, format: &str, failures: bool, + reset: bool, + yes: bool, _verbose: u8, ) -> Result<()> { let tracker = Tracker::new().context("Failed to initialize tracking database")?; let project_scope = resolve_project_scope(project)?; // added: resolve project path + if reset { + if !yes && !confirm_reset()? { + println!("Aborted."); + return Ok(()); + } + tracker + .reset_all() + .context("Failed to reset token savings")?; + println!("{}", styled("Token savings stats reset to zero.", true)); + return Ok(()); + } + if failures { return show_failures(&tracker); } @@ -725,3 +739,26 @@ fn show_failures(tracker: &Tracker) -> Result<()> { Ok(()) } + +/// Prompt the user to confirm a destructive reset operation. +/// Defaults to No in non-interactive (piped) environments. +fn confirm_reset() -> Result { + use std::io::{self, BufRead, IsTerminal, Write}; + + eprint!("This will permanently delete all tracking data. Continue? [y/N] "); + io::stderr().flush().ok(); + + if !io::stdin().is_terminal() { + eprintln!("(non-interactive mode, defaulting to N)"); + return Ok(false); + } + + let stdin = io::stdin(); + let mut line = String::new(); + stdin + .lock() + .read_line(&mut line) + .context("Failed to read confirmation")?; + + Ok(matches!(line.trim().to_lowercase().as_str(), "y" | "yes")) +} diff --git a/src/cmds/README.md b/src/cmds/README.md index a84e8e744..010e9495a 100644 --- a/src/cmds/README.md +++ b/src/cmds/README.md @@ -2,7 +2,7 @@ ## Scope -**Command execution and output filtering** — this is the core value RTK delivers. Every module here calls an external CLI tool (`Command::new("some_tool")`), transforms its stdout/stderr to reduce token consumption, and records savings via `core/tracking`. +**Command execution and output filtering.** Every module here calls an external CLI tool (`Command::new("some_tool")`), transforms its stdout/stderr to reduce token consumption, and records savings via `core/tracking`. Owns: all command-specific filter logic, organized by ecosystem (git, rust, js, python, go, dotnet, cloud, system). Cross-ecosystem routing (e.g., `lint_cmd` detecting Python and delegating to `ruff_cmd`) is an intra-component concern. @@ -35,47 +35,182 @@ Each subdirectory has its own README with file descriptions, parsing strategies, - **[`system/`](system/README.md)** — ls, tree, read, grep, find, wc, env, json, log, deps, summary, format, smart — format_cmd routing, filter levels, language detection - **[`ruby/`](ruby/README.md)** — rake/rails test, rspec, rubocop — JSON injection pattern, `ruby_exec()` bundle exec auto-detection -## Common Pattern +## Execution Flow -Every command module follows this structure: +The shared wrappers in [`core/runner.rs`](../core/runner.rs) encapsulate the execution skeleton. Modules build the `Command` (custom arg logic), then delegate to a runner entry point. All runners handle tracking, tee recovery, and exit code propagation automatically. + +``` + run_streaming() Filter applied tee_and_hint() + | (per-line or post-hoc) | + v | v + +---------+ stdout +-------+-------+ filtered +-------+ + | Spawn |--------->| filter |----------->| Print | + +---------+ stderr +---------------+ +-------+ + | (live) | + v v + +----------+ +---------+ + | raw = | | Track | + | stdout + | | savings | + | stderr | +---------+ + +----------+ | + v + +-----------+ + | Ok(code) | + | returned | + +-----------+ +``` + +### Filter modes + +All execution goes through `core::stream::run_streaming()` with one of four `FilterMode` variants. The runner entry points (`run_filtered`, `run_streamed`, `run_passthrough`) select the appropriate mode automatically — module authors don't interact with `FilterMode` directly. + +| FilterMode | How it works | Used by | +|------------|-------------|---------| +| **`CaptureOnly`** | Buffers all stdout silently, then passes the full string to `filter_fn` post-hoc. Stderr streams to terminal in real time. | `run_filtered()` (default path) | +| **`Buffered`** | Buffers all stdout, applies filter, then prints the result. Stderr streams live. Chosen automatically by `run_filtered()` when `filter_stdout_only` is set. | `run_filtered()` (stdout-only path) | +| **`Streaming`** | Feeds each stdout line to a `StreamFilter::feed_line()` as it arrives. Emitted lines print immediately. Calls `flush()` after process exits for final output. | `run_streamed()` | +| **`Passthrough`** | Inherits the parent TTY directly — no piping, no buffering. `raw`/`filtered` are empty. | `run_passthrough()` | + +### When to use which + +| Scenario | Runner | FilterMode | Why | +|----------|--------|------------|-----| +| Parse structured output (JSON, tables) | `run_filtered()` | CaptureOnly/Buffered | Filter needs full text to parse structure | +| Long-running, line-parseable output | `run_streamed()` | Streaming | Low memory, real-time output | +| No filtering, just track usage | `run_passthrough()` | Passthrough | Zero overhead, inherits TTY | +| Custom logic (multi-command, file I/O) | Manual with `exec_capture()` | CaptureOnly | Full control over execution | + +### Phases + +1. **Spawn** — `run_streaming()` starts the child process with piped stdout/stderr (or inherited TTY for Passthrough) +2. **Filter** — stdout is processed per the FilterMode; stderr is forwarded to the terminal in real time via a dedicated reader thread +3. **Print** — filtered output is written to stdout (live for Streaming, post-hoc for CaptureOnly/Buffered); if tee enabled, appends recovery hint on failure +4. **Track** — `timer.track()` records raw vs filtered for token savings +5. **Exit code** — returns `Ok(exit_code)` to caller; `main.rs` calls `process::exit(code)` once + +**`RunOptions` builder:** + +| Constructor | Behavior | +|-------------|----------| +| `RunOptions::default()` | Combined stdout+stderr to filter, no tee | +| `RunOptions::with_tee("label")` | Combined filtering + tee recovery | +| `RunOptions::stdout_only()` | Stdout-only to filter, stderr passthrough, no tee | +| `RunOptions::stdout_only().tee("label")` | Stdout-only + tee recovery | + +**Example — filtered command (recommended):** + +```rust +pub fn run(args: &[String], verbose: u8) -> Result { + let mut cmd = resolved_command("mycmd"); + for arg in args { cmd.arg(arg); } + if verbose > 0 { eprintln!("Running: mycmd {}", args.join(" ")); } + + runner::run_filtered( + cmd, "mycmd", &args.join(" "), + filter_mycmd_output, + runner::RunOptions::stdout_only().tee("mycmd"), + ) +} +``` + +Exit code handling is **fully automatic** when using `run_filtered()` — the wrapper extracts the exit code (including Unix signal handling via 128+signal), tracks savings, and returns `Ok(exit_code)`. Module authors just return the result. + +**Streaming filters (line-by-line):** + +Use `runner::run_streamed()` when the command is long-running or produces unbounded output that should be filtered line-by-line. Three levels of abstraction, from simplest to most flexible: + +**Level 1: `RegexBlockFilter`** — regex start pattern + indent continuation (3-5 lines) + +For block-based errors where blocks start with a regex match and continue on indented lines. Handles skip prefixes, block counting, and summary automatically. + +```rust +use crate::core::stream::{BlockStreamFilter, RegexBlockFilter}; + +pub fn run(args: &[String], verbose: u8) -> Result { + let mut cmd = resolved_command("mycmd"); + for arg in args { cmd.arg(arg); } + + let filter = RegexBlockFilter::new("mycmd", r"^error\[") + .skip_prefixes(&["warning:", "note:"]); + + runner::run_streamed( + cmd, "mycmd", &args.join(" "), + Box::new(BlockStreamFilter::new(filter)), + runner::RunOptions::with_tee("mycmd"), + ) +} +``` + +`RegexBlockFilter` provides: regex-based block start detection, indent-based continuation (space/tab), configurable line skipping via prefixes, and automatic summary (`"mycmd: 3 blocks in output"` or `"mycmd: no errors found"`). + +**Level 2: `BlockHandler` trait** — custom block detection with state tracking + +When you need custom block start/continuation logic or stateful parsing beyond regex + indent. Implement the `BlockHandler` trait and wrap in `BlockStreamFilter`. + +```rust +use crate::core::stream::{BlockHandler, BlockStreamFilter}; + +struct MyHandler { error_count: usize } + +impl BlockHandler for MyHandler { + fn should_skip(&mut self, line: &str) -> bool { line.is_empty() } + fn is_block_start(&mut self, line: &str) -> bool { + if line.starts_with("FAIL") { self.error_count += 1; true } else { false } + } + fn is_block_continuation(&mut self, line: &str, _block: &[String]) -> bool { + line.starts_with(" ") || line.starts_with("at ") + } + fn format_summary(&self, _exit_code: i32, _raw: &str) -> Option { + Some(format!("{} failures\n", self.error_count)) + } +} +``` + +See `cmds/rust/cargo_cmd.rs::CargoBuildHandler` and `cmds/js/tsc_cmd.rs::TscHandler` for production examples. + +**Level 3: `StreamFilter` trait** — full line-by-line control + +When block-based parsing doesn't fit (e.g., state machines, multi-phase output, line transforms). Implement `StreamFilter` directly. ```rust -pub fn run(args: MyArgs, verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - let output = resolved_command("mycmd").args(&args).output().context("Failed to execute mycmd")?; - let raw = format!("{}\n{}", String::from_utf8_lossy(&output.stdout), String::from_utf8_lossy(&output.stderr)); - - let filtered = filter_output(&raw).unwrap_or_else(|e| { - eprintln!("rtk: filter warning: {}", e); - raw.clone() // Fallback to raw on filter failure - }); - - let exit_code = output.status.code().unwrap_or(1); - if let Some(hint) = tee::tee_and_hint(&raw, "mycmd", exit_code) { - println!("{}\n{}", filtered, hint); - } else { - println!("{}", filtered); +use crate::core::stream::StreamFilter; + +struct MyFilter { state: State } + +impl StreamFilter for MyFilter { + fn feed_line(&mut self, line: &str) -> Option { + // Return Some(text) to emit, None to suppress + if line.contains("error") { Some(format!("{}\n", line)) } else { None } } + fn flush(&mut self) -> String { String::new() } + fn on_exit(&mut self, exit_code: i32, raw: &str) -> Option { None } +} +``` + +See `cmds/rust/runner.rs::ErrorStreamFilter` for a complete reference implementation (state machine that tracks error blocks across lines). - timer.track("mycmd args", "rtk mycmd args", &raw, &filtered); - if !output.status.success() { std::process::exit(exit_code); } - Ok(()) +**Example — passthrough command (no filtering):** + +```rust +pub fn run_passthrough(args: &[OsString], verbose: u8) -> Result { + runner::run_passthrough("mycmd", args, verbose) } ``` -Six phases: **timer** → **execute** → **filter (with fallback)** → **tee on failure** → **track** → **exit code**. See [core/README.md](../core/README.md#consumer-contracts) for the contracts each phase must honor. +**Example — manual execution (custom logic):** -## Token Savings by Category +```rust +pub fn run(args: &[String], verbose: u8) -> Result { + let output = resolved_command("mycmd").args(args) + .output().context("Failed to run mycmd")?; + let exit_code = exit_code_from_output(&output, "mycmd"); + // ... custom filtering, tracking ... + Ok(exit_code) +} +``` + +Modules with deviations (subcommand dispatch, parser trait systems, two-command fallback, synthetic output). -| Category | Commands | Typical Savings | Strategy | -|----------|----------|----------------|----------| -| Test Runners | vitest, pytest, cargo test, go test, playwright | 90-99% | Show failures only, aggregate passes | -| Build Tools | cargo build, npm, pnpm, dotnet | 70-90% | Strip progress bars, summarize errors | -| VCS | git status/log/diff/show | 70-80% | Compact commit hashes, stat summaries | -| Linters | eslint/biome, ruff, tsc, mypy, golangci-lint | 80-85% | Group by file/rule, strip context | -| Package Managers | pip, cargo install, pnpm list | 75-80% | Remove decorative output, compact trees | -| File Operations | ls, find, grep, cat/head/tail | 60-75% | Tree format, grouped results, truncation | -| Infrastructure | docker, kubectl, aws, terraform | 75-85% | Essential info only | ## Cross-Command Dependencies @@ -89,7 +224,25 @@ These behaviors must be uniform across all command modules. Full audit details i ### Exit Code Propagation -Modules must capture the underlying command's exit code, propagate it via `std::process::exit()` only on failure, and return `Ok(())` on success. When the process is killed by signal (`.code()` returns `None`), default to exit code 1. +All module `run()` functions return `Result` where the `i32` is the underlying command's exit code. `main.rs` calls `std::process::exit(code)` once at the single exit point — **modules never call `process::exit()` directly**. + +| Return value | Meaning | Who exits | +|--------------|---------|-----------| +| `Ok(0)` | Command succeeded | `main.rs` exits 0 | +| `Ok(N)` | Command failed with code N | `main.rs` exits N | +| `Err(e)` | RTK itself failed (not the command) | `main.rs` prints error, exits 1 | + +**How exit codes are extracted:** + +| Execution style | Helper | Signal handling | +|----------------|--------|-----------------| +| `cmd.output()` (filtered) | `exit_code_from_output(&output, "tool")` | 128+signal on Unix | +| `cmd.status()` (passthrough) | `exit_code_from_status(&status, "tool")` | 128+signal on Unix | +| `run_filtered()` (wrapper) | Automatic — no manual code needed | Built-in | + +**When using `run_filtered()`**: exit code handling is fully automatic. The wrapper extracts the exit code, handles signals, and returns `Ok(exit_code)`. Module authors just return the wrapper's result — no exit code logic needed. + +**When doing manual execution**: use `exit_code_from_output()` or `exit_code_from_status()` and return `Ok(exit_code)`. Never call `process::exit()`, never use `.code().unwrap_or(1)` (loses signal info). ### Filter Failure Passthrough @@ -105,50 +258,36 @@ Modules must capture stderr and include it in the raw string passed to `timer.tr ### Tracking Completeness -All modules must call `timer.track()` on every path — success, failure, and fallback. Never exit before tracking. +All modules must call `timer.track()` on every path — success, failure, and fallback. Since modules return `Ok(exit_code)` instead of calling `process::exit()`, tracking always runs before the program exits. ### Verbose Flag All modules accept `verbose: u8`. Use it to print debug info (command being run, savings %, filter tier). Do not accept and ignore it. -### Gaps (to be fixed) - -**Exit code** — 5 different patterns coexist, should be reviewed for uniform behavior: -- `vitest_cmd.rs`, `tsc_cmd.rs`, `psql_cmd.rs` — exit unconditionally, even on success -- `lint_cmd.rs` — swallows signal kills silently -- `golangci_cmd.rs` — maps signal kill to exit 130 (correct but unique) - -**Filter passthrough** — silent passthrough, no warning: -- `gh_cmd.rs`, `pip_cmd.rs`, `container.rs`, `dotnet_cmd.rs` — `run_passthrough()` skips filtering without warning -- `pnpm_cmd.rs` — 3-tier degradation but no tee recovery on final tier - -**Tee recovery** — missing from some high-risk modules: -- `pnpm_cmd.rs` — 3-tier parser, no tee -- `gh_cmd.rs` — aggressive markdown filtering, no tee -- `ruff_cmd.rs`, `golangci_cmd.rs` — JSON parsers, no tee -- `psql_cmd.rs` — has tee but exits before calling it on error path - -**Stderr handling** — 3 patterns coexist. Some modules combine stderr into raw (correct), others print via `eprintln!()` and exclude from tracking (inflates savings %). See `docs/ISO_ANALYZE.md` section 4. - -**Tracking** — exit before track on error path: -- `ls.rs`, `tree.rs` — lost metrics on failure -- `container.rs` — inconsistent across subcommands - -**Verbose** — accept parameter but ignore it: -- `container.rs` — all internal functions prefix `_verbose` -- `diff_cmd.rs` — `_verbose` unused ## Adding a New Command Filter -Adding a new filter or command requires changes in multiple places: +Adding a new filter or command requires changes in multiple places. For TOML-vs-Rust decision criteria, see [CONTRIBUTING.md](../../CONTRIBUTING.md#toml-vs-rust-which-one). -1. **Create the filter** — TOML file in [`src/filters/`](../filters/README.md) or Rust module in `src/cmds//` -2. **Add rewrite pattern** — Entry in `src/discover/rules.rs` (PATTERNS + RULES arrays at matching index) so hooks auto-rewrite the command -3. **Register in main.rs** — (Rust modules only) Three changes: - - Add `pub mod mymod;` to the ecosystem's `mod.rs` (e.g., `src/cmds/system/mod.rs`) +### Rust module (structured output, flag injection, state machines) + +1. **Create module** in `src/cmds//mycmd_cmd.rs`: + - Write the `filter_mycmd()` function (pure: `&str -> String`, no side effects) + - Write `pub fn run(...) -> Result` using `runner::run_filtered()` — build the `Command`, choose `RunOptions`, delegate + - Use `RunOptions::stdout_only()` when the filter parses structured stdout (JSON, NDJSON) — stderr would corrupt parsing + - Use `RunOptions::default()` when filtering combined text output + - Add `.tee("label")` when the filter parses structured output (enables raw output recovery on failure) + - **Exit codes**: handled automatically by `run_filtered()` — just return its result +2. **Register module**: + - Ecosystem `mod.rs` files use `automod::dir!()` — any `.rs` file in the directory becomes a public module automatically. No manual `pub mod` needed, but be aware: WIP or helper files will also be exposed. Only commit command-ready modules. - Add variant to `Commands` enum in `main.rs` with `#[arg(trailing_var_arg = true, allow_hyphen_values = true)]` - - Add routing match arm in `main.rs` to call `mymod::run()` + - Add routing match arm in `main.rs`: `Commands::Mycmd { args } => mycmd_cmd::run(&args, cli.verbose)?,` +3. **Add rewrite pattern** — Entry in `src/discover/rules.rs` (PATTERNS + RULES arrays at matching index) so hooks auto-rewrite the command 4. **Write tests** — Real fixture, snapshot test, token savings >= 60% (see [testing rules](../../.claude/rules/cli-testing.md)) -5. **Update docs** — README.md command list, CHANGELOG.md +5. **Update docs** — Ecosystem README (CHANGELOG.md is auto-generated by release-please) + +### TOML filter (simple line-based filtering) -Follow the [Common Pattern](#common-pattern) above for the module template (timer, fallback, tee, tracking, exit code). For TOML-vs-Rust decision criteria, see [CONTRIBUTING.md](../../CONTRIBUTING.md#toml-vs-rust-which-one). +1. **Create filter** in [`src/filters/`](../filters/README.md) +2. **Add rewrite pattern** in `src/discover/rules.rs` +3. **Write tests** and **update docs** diff --git a/src/cmds/cloud/README.md b/src/cmds/cloud/README.md index 7bfa92282..a86acec6a 100644 --- a/src/cmds/cloud/README.md +++ b/src/cmds/cloud/README.md @@ -1,11 +1,11 @@ # Cloud and Infrastructure -> Part of [`src/cmds/`](../README.md) — see also [docs/TECHNICAL.md](../../../docs/TECHNICAL.md) +> Part of [`src/cmds/`](../README.md) — see also [docs/contributing/TECHNICAL.md](../../../docs/contributing/TECHNICAL.md) ## Specifics -- `aws_cmd.rs` forces `--output json` for structured parsing +- `aws_cmd.rs` — 25 specialized filters covering STS, S3, EC2, ECS, RDS, CloudFormation, CloudWatch Logs, Lambda, IAM, DynamoDB, EKS, SQS, Secrets Manager. Forces `--output json` for structured parsing, uses `force_tee_hint()` for truncation recovery, strips Lambda secrets. Shared runner `run_aws_filtered()` handles boilerplate for JSON-based filters; text-based filters (S3 ls, S3 sync/cp) have dedicated runners - `container.rs` handles both Docker and Kubernetes; `DockerCommands` and `KubectlCommands` sub-enums in `main.rs` route to `container::run()` -- uses passthrough for unknown subcommands -- `curl_cmd.rs` auto-detects JSON responses and shows schema (structure without values) +- `curl_cmd.rs` truncates long responses, saves full output to file for recovery - `wget_cmd.rs` wraps wget with output filtering - `psql_cmd.rs` filters PostgreSQL query output diff --git a/src/cmds/cloud/aws_cmd.rs b/src/cmds/cloud/aws_cmd.rs index bb1757ec1..2657218b8 100644 --- a/src/cmds/cloud/aws_cmd.rs +++ b/src/cmds/cloud/aws_cmd.rs @@ -3,17 +3,47 @@ //! Replaces verbose `--output table`/`text` with JSON, then compresses. //! Specialized filters for high-frequency commands (STS, S3, EC2, ECS, RDS, CloudFormation). +use crate::core::tee::force_tee_hint; use crate::core::tracking; -use crate::core::utils::{join_with_overflow, resolved_command, truncate_iso_date}; +use crate::core::utils::{ + exit_code_from_output, exit_code_from_status, human_bytes, join_with_overflow, + resolved_command, shorten_arn, truncate_iso_date, +}; use crate::json_cmd; use anyhow::{Context, Result}; +use lazy_static::lazy_static; +use regex::Regex; use serde_json::Value; const MAX_ITEMS: usize = 20; const JSON_COMPRESS_DEPTH: usize = 4; +/// Result of a filter function: filtered text + whether items were truncated. +/// When `truncated` is true, the shared runner force-tees the full raw output +/// so the LLM has a recovery path to access all data. +struct FilterResult { + text: String, + truncated: bool, +} + +impl FilterResult { + fn new(text: String) -> Self { + Self { + text, + truncated: false, + } + } + + fn truncated(text: String) -> Self { + Self { + text, + truncated: true, + } + } +} + /// Run an AWS CLI command with token-optimized output -pub fn run(subcommand: &str, args: &[String], verbose: u8) -> Result<()> { +pub fn run(subcommand: &str, args: &[String], verbose: u8) -> Result { // Build the full sub-path: e.g. "sts" + ["get-caller-identity"] -> "sts get-caller-identity" let full_sub = if args.is_empty() { subcommand.to_string() @@ -23,28 +53,146 @@ pub fn run(subcommand: &str, args: &[String], verbose: u8) -> Result<()> { // Route to specialized handlers match subcommand { - "sts" if !args.is_empty() && args[0] == "get-caller-identity" => { - run_sts_identity(&args[1..], verbose) - } + "sts" if !args.is_empty() && args[0] == "get-caller-identity" => run_aws_filtered( + &["sts", "get-caller-identity"], + &args[1..], + verbose, + filter_sts_identity, + ), "s3" if !args.is_empty() && args[0] == "ls" => run_s3_ls(&args[1..], verbose), - "ec2" if !args.is_empty() && args[0] == "describe-instances" => { - run_ec2_describe(&args[1..], verbose) - } - "ecs" if !args.is_empty() && args[0] == "list-services" => { - run_ecs_list_services(&args[1..], verbose) - } - "ecs" if !args.is_empty() && args[0] == "describe-services" => { - run_ecs_describe_services(&args[1..], verbose) + "ec2" if !args.is_empty() && args[0] == "describe-instances" => run_aws_filtered( + &["ec2", "describe-instances"], + &args[1..], + verbose, + filter_ec2_instances, + ), + "ecs" if !args.is_empty() && args[0] == "list-services" => run_aws_filtered( + &["ecs", "list-services"], + &args[1..], + verbose, + filter_ecs_list_services, + ), + "ecs" if !args.is_empty() && args[0] == "describe-services" => run_aws_filtered( + &["ecs", "describe-services"], + &args[1..], + verbose, + filter_ecs_describe_services, + ), + "rds" if !args.is_empty() && args[0] == "describe-db-instances" => run_aws_filtered( + &["rds", "describe-db-instances"], + &args[1..], + verbose, + filter_rds_instances, + ), + "cloudformation" if !args.is_empty() && args[0] == "list-stacks" => run_aws_filtered( + &["cloudformation", "list-stacks"], + &args[1..], + verbose, + filter_cfn_list_stacks, + ), + "cloudformation" if !args.is_empty() && args[0] == "describe-stacks" => run_aws_filtered( + &["cloudformation", "describe-stacks"], + &args[1..], + verbose, + filter_cfn_describe_stacks, + ), + "cloudformation" if !args.is_empty() && args[0] == "describe-stack-events" => { + run_aws_filtered( + &["cloudformation", "describe-stack-events"], + &args[1..], + verbose, + filter_cfn_events, + ) } - "rds" if !args.is_empty() && args[0] == "describe-db-instances" => { - run_rds_describe(&args[1..], verbose) + "logs" + if !args.is_empty() + && (args[0] == "get-log-events" || args[0] == "filter-log-events") => + { + run_aws_filtered(&["logs", &args[0]], &args[1..], verbose, filter_logs_events) } - "cloudformation" if !args.is_empty() && args[0] == "list-stacks" => { - run_cfn_list_stacks(&args[1..], verbose) + "lambda" if !args.is_empty() && args[0] == "list-functions" => run_aws_filtered( + &["lambda", "list-functions"], + &args[1..], + verbose, + filter_lambda_list, + ), + "lambda" if !args.is_empty() && args[0] == "get-function" => run_aws_filtered( + &["lambda", "get-function"], + &args[1..], + verbose, + filter_lambda_get, + ), + "iam" if !args.is_empty() && args[0] == "list-roles" => run_aws_filtered( + &["iam", "list-roles"], + &args[1..], + verbose, + filter_iam_roles, + ), + "iam" if !args.is_empty() && args[0] == "list-users" => run_aws_filtered( + &["iam", "list-users"], + &args[1..], + verbose, + filter_iam_users, + ), + "dynamodb" if !args.is_empty() && (args[0] == "scan" || args[0] == "query") => { + run_aws_filtered( + &["dynamodb", &args[0]], + &args[1..], + verbose, + filter_dynamodb_items, + ) } - "cloudformation" if !args.is_empty() && args[0] == "describe-stacks" => { - run_cfn_describe_stacks(&args[1..], verbose) + "ecs" if !args.is_empty() && args[0] == "describe-tasks" => run_aws_filtered( + &["ecs", "describe-tasks"], + &args[1..], + verbose, + filter_ecs_tasks, + ), + "ec2" if !args.is_empty() && args[0] == "describe-security-groups" => run_aws_filtered( + &["ec2", "describe-security-groups"], + &args[1..], + verbose, + filter_security_groups, + ), + "s3api" if !args.is_empty() && args[0] == "list-objects-v2" => run_aws_filtered( + &["s3api", "list-objects-v2"], + &args[1..], + verbose, + filter_s3_objects, + ), + "eks" if !args.is_empty() && args[0] == "describe-cluster" => run_aws_filtered( + &["eks", "describe-cluster"], + &args[1..], + verbose, + filter_eks_cluster, + ), + "sqs" if !args.is_empty() && args[0] == "receive-message" => run_aws_filtered( + &["sqs", "receive-message"], + &args[1..], + verbose, + filter_sqs_messages, + ), + "dynamodb" if !args.is_empty() && args[0] == "get-item" => run_aws_filtered( + &["dynamodb", "get-item"], + &args[1..], + verbose, + filter_dynamodb_get_item, + ), + "logs" if !args.is_empty() && args[0] == "get-query-results" => run_aws_filtered( + &["logs", "get-query-results"], + &args[1..], + verbose, + filter_logs_query_results, + ), + "s3" if !args.is_empty() && (args[0] == "sync" || args[0] == "cp") => { + run_s3_transfer(&args[0], &args[1..], verbose) } + "secretsmanager" if !args.is_empty() && args[0] == "get-secret-value" => run_aws_filtered( + &["secretsmanager", "get-secret-value"], + &args[1..], + verbose, + filter_secrets_get, + ), _ => run_generic(subcommand, args, verbose, &full_sub), } } @@ -54,11 +202,20 @@ pub fn run(subcommand: &str, args: &[String], verbose: u8) -> Result<()> { /// and do not accept --output json, so we must not inject it for them. fn is_structured_operation(args: &[String]) -> bool { let op = args.first().map(|s| s.as_str()).unwrap_or(""); - op.starts_with("describe-") || op.starts_with("list-") || op.starts_with("get-") + // Exclude s3 sync/cp (they're text operations) + if op == "sync" || op == "cp" { + return false; + } + op.starts_with("describe-") + || op.starts_with("list-") + || op.starts_with("get-") + || op == "scan" + || op == "query" + || op == "receive-message" } /// Generic strategy: force --output json for structured ops, compress via json_cmd schema -fn run_generic(subcommand: &str, args: &[String], verbose: u8, full_sub: &str) -> Result<()> { +fn run_generic(subcommand: &str, args: &[String], verbose: u8, full_sub: &str) -> Result { let timer = tracking::TimedExecution::start(); let mut cmd = resolved_command("aws"); @@ -95,7 +252,7 @@ fn run_generic(subcommand: &str, args: &[String], verbose: u8, full_sub: &str) - &stderr, ); eprintln!("{}", stderr.trim()); - std::process::exit(output.status.code().unwrap_or(1)); + return Ok(crate::core::utils::exit_code_from_output(&output, "aws")); } let filtered = match json_cmd::filter_json_string(&raw, JSON_COMPRESS_DEPTH) { @@ -117,7 +274,7 @@ fn run_generic(subcommand: &str, args: &[String], verbose: u8, full_sub: &str) - &filtered, ); - Ok(()) + Ok(0) } fn run_aws_json( @@ -141,6 +298,9 @@ fn run_aws_json( skip_next = true; continue; } + if arg.starts_with("--output=") { + continue; + } cmd.arg(arg); } cmd.args(["--output", "json"]); @@ -163,39 +323,62 @@ fn run_aws_json( Ok((stdout, stderr, output.status)) } -fn run_sts_identity(extra_args: &[String], verbose: u8) -> Result<()> { +/// Shared runner for AWS commands that return JSON. +/// Follows the six-phase contract: timer → execute → filter (fallback) → tee → track → exit code. +fn run_aws_filtered( + sub_args: &[&str], + extra_args: &[String], + verbose: u8, + filter_fn: fn(&str) -> Option, +) -> Result { + let cmd_label = format!("aws {}", sub_args.join(" ")); + let rtk_label = format!("rtk {}", cmd_label); + let slug = cmd_label.replace(' ', "_"); let timer = tracking::TimedExecution::start(); - let (raw, stderr, status) = run_aws_json(&["sts", "get-caller-identity"], extra_args, verbose)?; + let (stdout, stderr, status) = run_aws_json(sub_args, extra_args, verbose)?; + + // Combine stdout+stderr for accurate tracking (per contract) + let raw = if stderr.is_empty() { + stdout.clone() + } else { + format!("{}\n{}", stdout, stderr) + }; if !status.success() { - timer.track( - "aws sts get-caller-identity", - "rtk aws sts get-caller-identity", - &stderr, - &stderr, - ); - std::process::exit(status.code().unwrap_or(1)); + let exit_code = exit_code_from_status(&status, "aws"); + if let Some(hint) = crate::core::tee::tee_and_hint(&raw, &slug, exit_code) { + eprintln!("{}\n{}", stderr.trim(), hint); + } else { + eprintln!("{}", stderr.trim()); + } + timer.track(&cmd_label, &rtk_label, &raw, &stderr); + return Ok(exit_code); } - let filtered = match filter_sts_identity(&raw) { - Some(f) => f, - None => raw.clone(), - }; - println!("{}", filtered); + let result = filter_fn(&stdout).unwrap_or_else(|| { + eprintln!("rtk: filter warning: aws filter returned None, passing through raw output"); + FilterResult::new(stdout.clone()) + }); - timer.track( - "aws sts get-caller-identity", - "rtk aws sts get-caller-identity", - &raw, - &filtered, - ); - Ok(()) + if result.truncated { + if let Some(hint) = crate::core::tee::force_tee_hint(&raw, &slug) { + println!("{}\n{}", result.text, hint); + } else { + println!("{}", result.text); + } + } else if let Some(hint) = crate::core::tee::tee_and_hint(&raw, &slug, 0) { + println!("{}\n{}", result.text, hint); + } else { + println!("{}", result.text); + } + + timer.track(&cmd_label, &rtk_label, &raw, &result.text); + Ok(0) } -fn run_s3_ls(extra_args: &[String], verbose: u8) -> Result<()> { +fn run_s3_ls(extra_args: &[String], verbose: u8) -> Result { let timer = tracking::TimedExecution::start(); - // s3 ls doesn't support --output json, run as-is and filter text let mut cmd = resolved_command("aws"); cmd.args(["s3", "ls"]); for arg in extra_args { @@ -207,227 +390,121 @@ fn run_s3_ls(extra_args: &[String], verbose: u8) -> Result<()> { } let output = cmd.output().context("Failed to run aws s3 ls")?; - let raw = String::from_utf8_lossy(&output.stdout).to_string(); - + let stdout = String::from_utf8_lossy(&output.stdout).to_string(); + let stderr = String::from_utf8_lossy(&output.stderr).to_string(); + let raw = if stderr.is_empty() { + stdout.clone() + } else { + format!("{}\n{}", stdout, stderr) + }; if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr).to_string(); - timer.track("aws s3 ls", "rtk aws s3 ls", &stderr, &stderr); - eprintln!("{}", stderr.trim()); - std::process::exit(output.status.code().unwrap_or(1)); - } - - let filtered = filter_s3_ls(&raw); - println!("{}", filtered); - - timer.track("aws s3 ls", "rtk aws s3 ls", &raw, &filtered); - Ok(()) -} - -fn run_ec2_describe(extra_args: &[String], verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - let (raw, stderr, status) = run_aws_json(&["ec2", "describe-instances"], extra_args, verbose)?; - - if !status.success() { - timer.track( - "aws ec2 describe-instances", - "rtk aws ec2 describe-instances", - &stderr, - &stderr, - ); - std::process::exit(status.code().unwrap_or(1)); + let exit_code = exit_code_from_output(&output, "aws"); + if let Some(hint) = crate::core::tee::tee_and_hint(&raw, "aws_s3_ls", exit_code) { + eprintln!("{}\n{}", stderr.trim(), hint); + } else { + eprintln!("{}", stderr.trim()); + } + timer.track("aws s3 ls", "rtk aws s3 ls", &raw, &stderr); + return Ok(exit_code); } - let filtered = match filter_ec2_instances(&raw) { - Some(f) => f, - None => raw.clone(), - }; - println!("{}", filtered); - - timer.track( - "aws ec2 describe-instances", - "rtk aws ec2 describe-instances", - &raw, - &filtered, - ); - Ok(()) -} - -fn run_ecs_list_services(extra_args: &[String], verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - let (raw, stderr, status) = run_aws_json(&["ecs", "list-services"], extra_args, verbose)?; - - if !status.success() { - timer.track( - "aws ecs list-services", - "rtk aws ecs list-services", - &stderr, - &stderr, - ); - std::process::exit(status.code().unwrap_or(1)); + let result = filter_s3_ls(&stdout); + if result.truncated { + if let Some(hint) = crate::core::tee::force_tee_hint(&raw, "aws_s3_ls") { + println!("{}\n{}", result.text, hint); + } else { + println!("{}", result.text); + } + } else { + println!("{}", result.text); } - let filtered = match filter_ecs_list_services(&raw) { - Some(f) => f, - None => raw.clone(), - }; - println!("{}", filtered); - - timer.track( - "aws ecs list-services", - "rtk aws ecs list-services", - &raw, - &filtered, - ); - Ok(()) + timer.track("aws s3 ls", "rtk aws s3 ls", &raw, &result.text); + Ok(0) } -fn run_ecs_describe_services(extra_args: &[String], verbose: u8) -> Result<()> { +/// Run s3 sync/cp (text output, not JSON) +fn run_s3_transfer(operation: &str, extra_args: &[String], verbose: u8) -> Result { let timer = tracking::TimedExecution::start(); - let (raw, stderr, status) = run_aws_json(&["ecs", "describe-services"], extra_args, verbose)?; + let cmd_label = format!("aws s3 {}", operation); + let rtk_label = format!("rtk aws s3 {}", operation); + let slug = format!("aws_s3_{}", operation); - if !status.success() { - timer.track( - "aws ecs describe-services", - "rtk aws ecs describe-services", - &stderr, - &stderr, - ); - std::process::exit(status.code().unwrap_or(1)); + let mut cmd = resolved_command("aws"); + cmd.args(["s3", operation]); + for arg in extra_args { + cmd.arg(arg); } - let filtered = match filter_ecs_describe_services(&raw) { - Some(f) => f, - None => raw.clone(), - }; - println!("{}", filtered); - - timer.track( - "aws ecs describe-services", - "rtk aws ecs describe-services", - &raw, - &filtered, - ); - Ok(()) -} - -fn run_rds_describe(extra_args: &[String], verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - let (raw, stderr, status) = - run_aws_json(&["rds", "describe-db-instances"], extra_args, verbose)?; - - if !status.success() { - timer.track( - "aws rds describe-db-instances", - "rtk aws rds describe-db-instances", - &stderr, - &stderr, - ); - std::process::exit(status.code().unwrap_or(1)); + if verbose > 0 { + eprintln!("Running: {} {}", cmd_label, extra_args.join(" ")); } - let filtered = match filter_rds_instances(&raw) { - Some(f) => f, - None => raw.clone(), + let output = cmd + .output() + .context(format!("Failed to run {}", cmd_label))?; + let stdout = String::from_utf8_lossy(&output.stdout).to_string(); + let stderr = String::from_utf8_lossy(&output.stderr).to_string(); + let raw = if stderr.is_empty() { + stdout.clone() + } else { + format!("{}\n{}", stdout, stderr) }; - println!("{}", filtered); - - timer.track( - "aws rds describe-db-instances", - "rtk aws rds describe-db-instances", - &raw, - &filtered, - ); - Ok(()) -} - -fn run_cfn_list_stacks(extra_args: &[String], verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - let (raw, stderr, status) = - run_aws_json(&["cloudformation", "list-stacks"], extra_args, verbose)?; - - if !status.success() { - timer.track( - "aws cloudformation list-stacks", - "rtk aws cloudformation list-stacks", - &stderr, - &stderr, - ); - std::process::exit(status.code().unwrap_or(1)); + if !output.status.success() { + let exit_code = exit_code_from_output(&output, "aws"); + if let Some(hint) = crate::core::tee::tee_and_hint(&raw, &slug, exit_code) { + eprintln!("{}\n{}", stderr.trim(), hint); + } else { + eprintln!("{}", stderr.trim()); + } + timer.track(&cmd_label, &rtk_label, &raw, &stderr); + return Ok(exit_code); } - let filtered = match filter_cfn_list_stacks(&raw) { - Some(f) => f, - None => raw.clone(), - }; - println!("{}", filtered); - - timer.track( - "aws cloudformation list-stacks", - "rtk aws cloudformation list-stacks", - &raw, - &filtered, - ); - Ok(()) -} - -fn run_cfn_describe_stacks(extra_args: &[String], verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - let (raw, stderr, status) = - run_aws_json(&["cloudformation", "describe-stacks"], extra_args, verbose)?; - - if !status.success() { - timer.track( - "aws cloudformation describe-stacks", - "rtk aws cloudformation describe-stacks", - &stderr, - &stderr, - ); - std::process::exit(status.code().unwrap_or(1)); + let result = filter_s3_transfer(&stdout); + if result.truncated { + if let Some(hint) = force_tee_hint(&raw, &slug) { + println!("{}\n{}", result.text, hint); + } else { + println!("{}", result.text); + } + } else { + println!("{}", result.text); } - let filtered = match filter_cfn_describe_stacks(&raw) { - Some(f) => f, - None => raw.clone(), - }; - println!("{}", filtered); - - timer.track( - "aws cloudformation describe-stacks", - "rtk aws cloudformation describe-stacks", - &raw, - &filtered, - ); - Ok(()) + timer.track(&cmd_label, &rtk_label, &raw, &result.text); + Ok(0) } // --- Filter functions (all use serde_json::Value for resilience) --- +// Each returns Option: Some = filtered, None = fallback to raw. +// FilterResult.truncated = true means items were cut; shared runner will tee full output. -fn filter_sts_identity(json_str: &str) -> Option { +fn filter_sts_identity(json_str: &str) -> Option { let v: Value = serde_json::from_str(json_str).ok()?; let account = v["Account"].as_str().unwrap_or("?"); let arn = v["Arn"].as_str().unwrap_or("?"); - Some(format!("AWS: {} {}", account, arn)) + Some(FilterResult::new(format!("AWS: {} {}", account, arn))) } -fn filter_s3_ls(output: &str) -> String { +fn filter_s3_ls(output: &str) -> FilterResult { let lines: Vec<&str> = output.lines().collect(); let total = lines.len(); - let mut result: Vec<&str> = lines.iter().take(MAX_ITEMS + 10).copied().collect(); + let limit = MAX_ITEMS + 10; - if total > MAX_ITEMS + 10 { - result.truncate(MAX_ITEMS + 10); - result.push(""); // will be replaced - return format!( + if total > limit { + let text = format!( "{}\n... +{} more items", - result[..result.len() - 1].join("\n"), - total - MAX_ITEMS - 10 + lines[..limit].join("\n"), + total - limit ); + FilterResult::truncated(text) + } else { + FilterResult::new(lines.join("\n")) } - - result.join("\n") } -fn filter_ec2_instances(json_str: &str) -> Option { +fn filter_ec2_instances(json_str: &str) -> Option { let v: Value = serde_json::from_str(json_str).ok()?; let reservations = v["Reservations"].as_array()?; @@ -438,35 +515,56 @@ fn filter_ec2_instances(json_str: &str) -> Option { let id = inst["InstanceId"].as_str().unwrap_or("?"); let state = inst["State"]["Name"].as_str().unwrap_or("?"); let itype = inst["InstanceType"].as_str().unwrap_or("?"); - let ip = inst["PrivateIpAddress"].as_str().unwrap_or("-"); + let private_ip = inst["PrivateIpAddress"].as_str().unwrap_or("-"); + let public_ip = inst["PublicIpAddress"].as_str().unwrap_or("-"); + let subnet = inst["SubnetId"].as_str().unwrap_or("-"); + let vpc = inst["VpcId"].as_str().unwrap_or("-"); - // Extract Name tag let name = inst["Tags"] .as_array() .and_then(|tags| tags.iter().find(|t| t["Key"].as_str() == Some("Name"))) .and_then(|t| t["Value"].as_str()) .unwrap_or("-"); - instances.push(format!("{} {} {} {} ({})", id, state, itype, ip, name)); + let sgs: Vec<&str> = inst["SecurityGroups"] + .as_array() + .map(|arr| arr.iter().filter_map(|sg| sg["GroupId"].as_str()).collect()) + .unwrap_or_default(); + let sg_str = if sgs.is_empty() { + "-".to_string() + } else { + sgs.join(",") + }; + + instances.push(format!( + "{} {} {} {} pub:{} vpc:{} subnet:{} sg:[{}] ({})", + id, state, itype, private_ip, public_ip, vpc, subnet, sg_str, name + )); } } } let total = instances.len(); + let truncated = total > MAX_ITEMS; let mut result = format!("EC2: {} instances\n", total); for inst in instances.iter().take(MAX_ITEMS) { result.push_str(&format!(" {}\n", inst)); } - if total > MAX_ITEMS { + if truncated { result.push_str(&format!(" ... +{} more\n", total - MAX_ITEMS)); } - Some(result.trim_end().to_string()) + let text = result.trim_end().to_string(); + Some(if truncated { + FilterResult::truncated(text) + } else { + FilterResult::new(text) + }) } -fn filter_ecs_list_services(json_str: &str) -> Option { +fn filter_ecs_list_services(json_str: &str) -> Option { let v: Value = serde_json::from_str(json_str).ok()?; let arns = v["serviceArns"].as_array()?; @@ -475,15 +573,18 @@ fn filter_ecs_list_services(json_str: &str) -> Option { for arn in arns.iter().take(MAX_ITEMS) { let arn_str = arn.as_str().unwrap_or("?"); - // Extract short name from ARN: arn:aws:ecs:...:service/cluster/name -> name - let short = arn_str.rsplit('/').next().unwrap_or(arn_str); - result.push(short.to_string()); + result.push(shorten_arn(arn_str).to_string()); } - Some(join_with_overflow(&result, total, MAX_ITEMS, "services")) + let text = join_with_overflow(&result, total, MAX_ITEMS, "services"); + Some(if total > MAX_ITEMS { + FilterResult::truncated(text) + } else { + FilterResult::new(text) + }) } -fn filter_ecs_describe_services(json_str: &str) -> Option { +fn filter_ecs_describe_services(json_str: &str) -> Option { let v: Value = serde_json::from_str(json_str).ok()?; let services = v["services"].as_array()?; @@ -502,10 +603,15 @@ fn filter_ecs_describe_services(json_str: &str) -> Option { )); } - Some(join_with_overflow(&result, total, MAX_ITEMS, "services")) + let text = join_with_overflow(&result, total, MAX_ITEMS, "services"); + Some(if total > MAX_ITEMS { + FilterResult::truncated(text) + } else { + FilterResult::new(text) + }) } -fn filter_rds_instances(json_str: &str) -> Option { +fn filter_rds_instances(json_str: &str) -> Option { let v: Value = serde_json::from_str(json_str).ok()?; let dbs = v["DBInstances"].as_array()?; @@ -518,16 +624,23 @@ fn filter_rds_instances(json_str: &str) -> Option { let version = db["EngineVersion"].as_str().unwrap_or("?"); let class = db["DBInstanceClass"].as_str().unwrap_or("?"); let status = db["DBInstanceStatus"].as_str().unwrap_or("?"); + let endpoint = db["Endpoint"]["Address"].as_str().unwrap_or("-"); + let port = db["Endpoint"]["Port"].as_i64().unwrap_or(0); result.push(format!( - "{} {} {} {} {}", - name, engine, version, class, status + "{} {} {} {} {} {}:{}", + name, engine, version, class, status, endpoint, port )); } - Some(join_with_overflow(&result, total, MAX_ITEMS, "instances")) + let text = join_with_overflow(&result, total, MAX_ITEMS, "instances"); + Some(if total > MAX_ITEMS { + FilterResult::truncated(text) + } else { + FilterResult::new(text) + }) } -fn filter_cfn_list_stacks(json_str: &str) -> Option { +fn filter_cfn_list_stacks(json_str: &str) -> Option { let v: Value = serde_json::from_str(json_str).ok()?; let stacks = v["StackSummaries"].as_array()?; @@ -544,10 +657,15 @@ fn filter_cfn_list_stacks(json_str: &str) -> Option { result.push(format!("{} {} {}", name, status, truncate_iso_date(date))); } - Some(join_with_overflow(&result, total, MAX_ITEMS, "stacks")) + let text = join_with_overflow(&result, total, MAX_ITEMS, "stacks"); + Some(if total > MAX_ITEMS { + FilterResult::truncated(text) + } else { + FilterResult::new(text) + }) } -fn filter_cfn_describe_stacks(json_str: &str) -> Option { +fn filter_cfn_describe_stacks(json_str: &str) -> Option { let v: Value = serde_json::from_str(json_str).ok()?; let stacks = v["Stacks"].as_array()?; @@ -563,7 +681,6 @@ fn filter_cfn_describe_stacks(json_str: &str) -> Option { .unwrap_or("?"); result.push(format!("{} {} {}", name, status, truncate_iso_date(date))); - // Show outputs if present if let Some(outputs) = stack["Outputs"].as_array() { for out in outputs { let key = out["OutputKey"].as_str().unwrap_or("?"); @@ -573,307 +690,2062 @@ fn filter_cfn_describe_stacks(json_str: &str) -> Option { } } - Some(join_with_overflow(&result, total, MAX_ITEMS, "stacks")) + let text = join_with_overflow(&result, total, MAX_ITEMS, "stacks"); + Some(if total > MAX_ITEMS { + FilterResult::truncated(text) + } else { + FilterResult::new(text) + }) } -#[cfg(test)] -mod tests { - use super::*; +// --- P0 filters: CloudWatch Logs, CloudFormation Events, Lambda --- + +const MAX_LOG_EVENTS: usize = 50; + +/// Convert days since Unix epoch to (year, month, day). Civil calendar, UTC. +fn days_to_ymd(days: i64) -> (i64, i64, i64) { + // Algorithm from http://howardhinnant.github.io/date_algorithms.html + let z = days + 719468; + let era = if z >= 0 { z } else { z - 146096 } / 146097; + let doe = z - era * 146097; + let yoe = (doe - doe / 1460 + doe / 36524 - doe / 146096) / 365; + let y = yoe + era * 400; + let doy = doe - (365 * yoe + yoe / 4 - yoe / 100); + let mp = (5 * doy + 2) / 153; + let d = doy - (153 * mp + 2) / 5 + 1; + let m = if mp < 10 { mp + 3 } else { mp - 9 }; + let y = if m <= 2 { y + 1 } else { y }; + (y, m, d) +} - #[test] - fn test_snapshot_sts_identity() { - let json = r#"{ - "UserId": "AIDAEXAMPLEUSERID1234", - "Account": "123456789012", - "Arn": "arn:aws:iam::123456789012:user/dev-user" -}"#; - let result = filter_sts_identity(json).unwrap(); - assert_eq!( - result, - "AWS: 123456789012 arn:aws:iam::123456789012:user/dev-user" - ); +fn filter_logs_events(json_str: &str) -> Option { + let v: Value = serde_json::from_str(json_str).ok()?; + let events = v["events"].as_array()?; + + let total = events.len(); + let truncated = total > MAX_LOG_EVENTS; + let mut lines = Vec::new(); + + for event in events.iter().take(MAX_LOG_EVENTS) { + // Convert epoch ms to YYYY-MM-DD HH:MM:SS UTC + let time_str = match event["timestamp"].as_i64() { + Some(ts) if ts > 0 => { + let epoch_secs = ts / 1000; + // Days since Unix epoch + let days = epoch_secs / 86400; + let time_of_day = epoch_secs % 86400; + let h = time_of_day / 3600; + let m = (time_of_day % 3600) / 60; + let s = time_of_day % 60; + // Convert days to Y-M-D (simplified: good through 2099) + let (y, mo, d) = days_to_ymd(days); + format!("{:04}-{:02}-{:02} {:02}:{:02}:{:02}", y, mo, d, h, m, s) + } + _ => "??:??:??".to_string(), + }; + + let msg = event["message"].as_str().unwrap_or("").trim_end(); + // If the message is JSON, compact it to one line + let compact_msg = if msg.starts_with('{') { + serde_json::from_str::(msg) + .ok() + .and_then(|v| serde_json::to_string(&v).ok()) + .unwrap_or_else(|| msg.to_string()) + } else { + msg.to_string() + }; + + lines.push(format!("{} {}", time_str, compact_msg)); } - #[test] - fn test_snapshot_ec2_instances() { - let json = r#"{"Reservations":[{"Instances":[{"InstanceId":"i-0a1b2c3d4e5f00001","InstanceType":"t3.micro","PrivateIpAddress":"10.0.1.10","State":{"Code":16,"Name":"running"},"Tags":[{"Key":"Name","Value":"web-server-1"}],"BlockDeviceMappings":[],"SecurityGroups":[]},{"InstanceId":"i-0a1b2c3d4e5f00002","InstanceType":"t3.large","PrivateIpAddress":"10.0.2.20","State":{"Code":80,"Name":"stopped"},"Tags":[{"Key":"Name","Value":"worker-1"}],"BlockDeviceMappings":[],"SecurityGroups":[]}]}]}"#; - let result = filter_ec2_instances(json).unwrap(); - assert!(result.contains("EC2: 2 instances")); - assert!(result.contains("i-0a1b2c3d4e5f00001 running t3.micro 10.0.1.10 (web-server-1)")); - assert!(result.contains("i-0a1b2c3d4e5f00002 stopped t3.large 10.0.2.20 (worker-1)")); + if truncated { + lines.push(format!("... +{} more events", total - MAX_LOG_EVENTS)); } - #[test] - fn test_filter_sts_identity() { - let json = r#"{ - "UserId": "AIDAEXAMPLE", - "Account": "123456789012", - "Arn": "arn:aws:iam::123456789012:user/dev" - }"#; - let result = filter_sts_identity(json).unwrap(); - assert_eq!( - result, - "AWS: 123456789012 arn:aws:iam::123456789012:user/dev" - ); + let text = lines.join("\n"); + Some(if truncated { + FilterResult::truncated(text) + } else { + FilterResult::new(text) + }) +} + +fn filter_cfn_events(json_str: &str) -> Option { + let v: Value = serde_json::from_str(json_str).ok()?; + let events = v["StackEvents"].as_array()?; + + let mut failed = Vec::new(); + let mut failed_count = 0usize; + let mut success_count = 0usize; + + for event in events { + let status = event["ResourceStatus"].as_str().unwrap_or("?"); + let logical_id = event["LogicalResourceId"].as_str().unwrap_or("?"); + let resource_type_raw = event["ResourceType"].as_str().unwrap_or("?"); + let resource_type = resource_type_raw + .strip_prefix("AWS::") + .unwrap_or(resource_type_raw); + let ts = event["Timestamp"] + .as_str() + .map(truncate_iso_date) + .unwrap_or("?"); + + if status.contains("FAILED") || status.contains("ROLLBACK") { + failed_count += 1; + if failed.len() < MAX_ITEMS { + let reason = event["ResourceStatusReason"].as_str().unwrap_or(""); + let mut line = format!("{} {} {} {}", ts, logical_id, resource_type, status); + if !reason.is_empty() { + line.push_str(&format!(" REASON: {}", reason)); + } + failed.push(line); + } + } else { + success_count += 1; + } } - #[test] - fn test_filter_sts_identity_missing_fields() { - let json = r#"{}"#; - let result = filter_sts_identity(json).unwrap(); - assert_eq!(result, "AWS: ? ?"); + let total_events = events.len(); + let mut lines = Vec::new(); + lines.push(format!( + "CloudFormation: {} events ({} failed, {} successful)", + total_events, failed_count, success_count + )); + + if !failed.is_empty() { + lines.push("--- FAILURES ---".to_string()); + for f in &failed { + lines.push(format!(" {}", f)); + } } - #[test] - fn test_filter_sts_identity_invalid_json() { - let result = filter_sts_identity("not json"); - assert!(result.is_none()); + if success_count > 0 { + lines.push(format!("+ {} successful resources", success_count)); } - #[test] - fn test_filter_s3_ls_basic() { - let output = "2024-01-01 bucket1\n2024-01-02 bucket2\n2024-01-03 bucket3\n"; - let result = filter_s3_ls(output); - assert!(result.contains("bucket1")); - assert!(result.contains("bucket3")); + // Truncate if huge number of events + let truncated = total_events > MAX_ITEMS * 5; // >100 events + let text = lines.join("\n"); + Some(if truncated { + FilterResult::truncated(text) + } else { + FilterResult::new(text) + }) +} + +fn filter_lambda_list(json_str: &str) -> Option { + let v: Value = serde_json::from_str(json_str).ok()?; + let functions = v["Functions"].as_array()?; + + let total = functions.len(); + let truncated = total > MAX_ITEMS; + let mut result = Vec::new(); + + for func in functions.iter().take(MAX_ITEMS) { + let name = func["FunctionName"].as_str().unwrap_or("?"); + let runtime = func["Runtime"].as_str().unwrap_or("?"); + let memory = func["MemorySize"].as_i64().unwrap_or(0); + let timeout = func["Timeout"].as_i64().unwrap_or(0); + let state = func["State"].as_str().unwrap_or("active"); + // SECURITY: Environment is intentionally NOT read (may contain secrets) + result.push(format!( + "{} {} {}MB {}s {}", + name, runtime, memory, timeout, state + )); } - #[test] - fn test_filter_s3_ls_overflow() { - let mut lines = Vec::new(); - for i in 1..=50 { - lines.push(format!("2024-01-01 bucket{}", i)); + let text = join_with_overflow(&result, total, MAX_ITEMS, "functions"); + Some(if truncated { + FilterResult::truncated(text) + } else { + FilterResult::new(text) + }) +} + +fn filter_lambda_get(json_str: &str) -> Option { + let v: Value = serde_json::from_str(json_str).ok()?; + let config = &v["Configuration"]; + + let name = config["FunctionName"].as_str().unwrap_or("?"); + let runtime = config["Runtime"].as_str().unwrap_or("?"); + let handler = config["Handler"].as_str().unwrap_or("?"); + let memory = config["MemorySize"].as_i64().unwrap_or(0); + let timeout = config["Timeout"].as_i64().unwrap_or(0); + let state = config["State"].as_str().unwrap_or("active"); + let last_modified = config["LastModified"] + .as_str() + .map(truncate_iso_date) + .unwrap_or("?"); + // SECURITY: Environment and Code.Location intentionally NOT read + + let mut text = format!( + "{} {} {} {}MB {}s {} {}", + name, runtime, handler, memory, timeout, state, last_modified + ); + + // Show layer names if present + // Layer ARNs use colons: arn:aws:lambda:region:acct:layer:name:version + if let Some(layers) = config["Layers"].as_array() { + if !layers.is_empty() { + let layer_names: Vec = layers + .iter() + .filter_map(|l| { + let arn = l["Arn"].as_str()?; + let parts: Vec<&str> = arn.rsplitn(3, ':').collect(); + if parts.len() >= 2 { + Some(format!("{}:{}", parts[1], parts[0])) + } else { + Some(arn.to_string()) + } + }) + .collect(); + text.push_str(&format!("\n layers: {}", layer_names.join(", "))); } - let input = lines.join("\n"); - let result = filter_s3_ls(&input); - assert!(result.contains("... +20 more items")); } - #[test] - fn test_filter_ec2_instances() { - let json = r#"{ - "Reservations": [{ - "Instances": [{ - "InstanceId": "i-abc123", - "State": {"Name": "running"}, - "InstanceType": "t3.micro", - "PrivateIpAddress": "10.0.1.5", - "Tags": [{"Key": "Name", "Value": "web-server"}] - }, { - "InstanceId": "i-def456", - "State": {"Name": "stopped"}, - "InstanceType": "t3.large", - "PrivateIpAddress": "10.0.1.6", - "Tags": [{"Key": "Name", "Value": "worker"}] - }] - }] - }"#; - let result = filter_ec2_instances(json).unwrap(); - assert!(result.contains("EC2: 2 instances")); - assert!(result.contains("i-abc123 running t3.micro 10.0.1.5 (web-server)")); - assert!(result.contains("i-def456 stopped t3.large 10.0.1.6 (worker)")); - } + Some(FilterResult::new(text)) +} - #[test] - fn test_filter_ec2_no_name_tag() { - let json = r#"{ - "Reservations": [{ +// --- P1 filters: IAM, DynamoDB, ECS tasks --- + +/// Extract principal services/accounts from AssumeRolePolicyDocument. +/// Returns compact list like ["lambda.amazonaws.com", "ecs-tasks.amazonaws.com"] +/// instead of the full 200+ token JSON policy document. +fn extract_assume_principals(role: &Value) -> Vec { + let mut principals = Vec::new(); + // AssumeRolePolicyDocument can be a JSON string or an object + let doc = if let Some(s) = role["AssumeRolePolicyDocument"].as_str() { + serde_json::from_str::(s).ok() + } else if role["AssumeRolePolicyDocument"].is_object() { + Some(role["AssumeRolePolicyDocument"].clone()) + } else { + None + }; + if let Some(doc) = doc { + let statements = doc["Statement"].as_array(); + if let Some(stmts) = statements { + for stmt in stmts { + let principal = &stmt["Principal"]; + // Principal can be "*", {"Service": "..."}, {"AWS": "..."}, etc. + if let Some(s) = principal.as_str() { + principals.push(s.to_string()); + } else if let Some(svc) = principal["Service"].as_str() { + principals.push(svc.to_string()); + } else if let Some(svcs) = principal["Service"].as_array() { + for s in svcs { + if let Some(s) = s.as_str() { + principals.push(s.to_string()); + } + } + } else if let Some(aws) = principal["AWS"].as_str() { + principals.push(shorten_arn(aws).to_string()); + } else if let Some(awss) = principal["AWS"].as_array() { + for a in awss { + if let Some(a) = a.as_str() { + principals.push(shorten_arn(a).to_string()); + } + } + } + } + } + } + principals.dedup(); + principals +} + +fn filter_iam_roles(json_str: &str) -> Option { + let v: Value = serde_json::from_str(json_str).ok()?; + let roles = v["Roles"].as_array()?; + + let total = roles.len(); + let truncated = total > MAX_ITEMS; + let mut result = Vec::new(); + + for role in roles.iter().take(MAX_ITEMS) { + let name = role["RoleName"].as_str().unwrap_or("?"); + let date = role["CreateDate"] + .as_str() + .map(truncate_iso_date) + .unwrap_or("?"); + let desc = role["Description"].as_str().unwrap_or(""); + + // Extract principals from AssumeRolePolicyDocument (compact, not full JSON) + let principals = extract_assume_principals(role); + let principal_str = if principals.is_empty() { + String::new() + } else { + format!(" assume:[{}]", principals.join(",")) + }; + + if desc.is_empty() { + result.push(format!("{} {}{}", name, date, principal_str)); + } else { + result.push(format!("{} {} [{}]{}", name, date, desc, principal_str)); + } + } + + let text = join_with_overflow(&result, total, MAX_ITEMS, "roles"); + Some(if truncated { + FilterResult::truncated(text) + } else { + FilterResult::new(text) + }) +} + +fn filter_iam_users(json_str: &str) -> Option { + let v: Value = serde_json::from_str(json_str).ok()?; + let users = v["Users"].as_array()?; + + let total = users.len(); + let truncated = total > MAX_ITEMS; + let mut result = Vec::new(); + + for user in users.iter().take(MAX_ITEMS) { + let name = user["UserName"].as_str().unwrap_or("?"); + let date = user["CreateDate"] + .as_str() + .map(truncate_iso_date) + .unwrap_or("?"); + result.push(format!("{} created:{}", name, date)); + } + + let text = join_with_overflow(&result, total, MAX_ITEMS, "users"); + Some(if truncated { + FilterResult::truncated(text) + } else { + FilterResult::new(text) + }) +} + +/// Recursively unwrap DynamoDB typed values to plain JSON. +/// `{"S": "foo"}` -> `"foo"`, `{"N": "42"}` -> `42`, `{"M": {...}}` -> unwrapped object, etc. +fn unwrap_dynamodb_value(val: &Value, depth: usize) -> Value { + if depth > 10 { + return val.clone(); + } + + if let Some(obj) = val.as_object() { + if obj.len() == 1 { + if let Some((key, inner)) = obj.iter().next() { + match key.as_str() { + "S" | "B" => return inner.clone(), + "N" => { + if let Some(s) = inner.as_str() { + // Try i64 first, then f64 + if let Ok(n) = s.parse::() { + return Value::Number(n.into()); + } + if let Ok(f) = s.parse::() { + if let Some(n) = serde_json::Number::from_f64(f) { + return Value::Number(n); + } + } + return Value::String(s.to_string()); + } + return inner.clone(); + } + "BOOL" => return inner.clone(), + "NULL" => return Value::Null, + "L" => { + if let Some(arr) = inner.as_array() { + return Value::Array( + arr.iter() + .map(|v| unwrap_dynamodb_value(v, depth + 1)) + .collect(), + ); + } + } + "M" => { + if let Some(map) = inner.as_object() { + let unwrapped: serde_json::Map = map + .iter() + .map(|(k, v)| (k.clone(), unwrap_dynamodb_value(v, depth + 1))) + .collect(); + return Value::Object(unwrapped); + } + } + "SS" => return inner.clone(), + "NS" => { + // Parse NS set: try i64 first, then f64 + if let Some(arr) = inner.as_array() { + let nums: Vec = arr + .iter() + .filter_map(|v| { + let s = v.as_str()?; + if let Ok(n) = s.parse::() { + Some(Value::Number(n.into())) + } else if let Ok(f) = s.parse::() { + serde_json::Number::from_f64(f).map(Value::Number) + } else { + Some(Value::String(s.to_string())) + } + }) + .collect(); + return Value::Array(nums); + } + return inner.clone(); + } + "BS" => return inner.clone(), + _ => {} + } + } + } + // Not a DynamoDB type wrapper — unwrap each field as a potential item + let unwrapped: serde_json::Map = obj + .iter() + .map(|(k, v)| (k.clone(), unwrap_dynamodb_value(v, depth + 1))) + .collect(); + return Value::Object(unwrapped); + } + + val.clone() +} + +fn filter_dynamodb_items(json_str: &str) -> Option { + let v: Value = serde_json::from_str(json_str).ok()?; + let items = v["Items"].as_array()?; + + let count = v["Count"].as_i64().unwrap_or(items.len() as i64); + let scanned = v["ScannedCount"].as_i64().unwrap_or(count); + let total = items.len(); + let truncated = total > MAX_ITEMS; + + let mut lines = Vec::new(); + lines.push(format!("Count: {}/{}", count, scanned)); + + // Show ConsumedCapacity if present + if let Some(capacity) = v["ConsumedCapacity"].as_object() { + if let Some(units) = capacity["CapacityUnits"].as_f64() { + lines.push(format!("Capacity: {} RCU", units)); + } + } + + // Show pagination status if LastEvaluatedKey exists + if v["LastEvaluatedKey"].is_object() { + lines.push("(paginated — more results available)".to_string()); + } + + for item in items.iter().take(MAX_ITEMS) { + let unwrapped = unwrap_dynamodb_value(item, 0); + let compact = serde_json::to_string(&unwrapped).unwrap_or_else(|_| "?".to_string()); + lines.push(compact); + } + + if truncated { + lines.push(format!("... +{} more items", total - MAX_ITEMS)); + } + + let text = lines.join("\n"); + Some(if truncated { + FilterResult::truncated(text) + } else { + FilterResult::new(text) + }) +} + +fn filter_ecs_tasks(json_str: &str) -> Option { + let v: Value = serde_json::from_str(json_str).ok()?; + let tasks = v["tasks"].as_array()?; + + let total = tasks.len(); + let truncated = total > MAX_ITEMS; + let mut result = Vec::new(); + + for task in tasks.iter().take(MAX_ITEMS) { + let task_arn = task["taskArn"].as_str().unwrap_or("?"); + let task_id = shorten_arn(task_arn); + let status = task["lastStatus"].as_str().unwrap_or("?"); + + let containers: Vec = task["containers"] + .as_array() + .map(|cs| { + cs.iter() + .map(|c| { + let name = c["name"].as_str().unwrap_or("?"); + let cstatus = c["lastStatus"].as_str().unwrap_or("?"); + let exit = c["exitCode"].as_i64(); + match exit { + Some(code) => format!("{}:{}(exit:{})", name, cstatus, code), + None => format!("{}:{}", name, cstatus), + } + }) + .collect() + }) + .unwrap_or_default(); + + let stopped_reason = task["stoppedReason"].as_str().unwrap_or(""); + let reason_str = if stopped_reason.is_empty() { + String::new() + } else { + format!(" reason:{}", stopped_reason) + }; + + result.push(format!( + "{} {} containers:[{}]{}", + task_id, + status, + containers.join(", "), + reason_str + )); + } + + let text = join_with_overflow(&result, total, MAX_ITEMS, "tasks"); + Some(if truncated { + FilterResult::truncated(text) + } else { + FilterResult::new(text) + }) +} + +// --- P2 filters: Security Groups, S3 objects, EKS, SQS --- + +fn format_sg_rule(perm: &Value) -> String { + let protocol = perm["IpProtocol"].as_str().unwrap_or("?"); + let proto = if protocol == "-1" { "all" } else { protocol }; + + let from_port = perm["FromPort"].as_i64(); + let to_port = perm["ToPort"].as_i64(); + let port = match (from_port, to_port) { + (Some(f), Some(t)) if f == t => format!("{}", f), + (Some(f), Some(t)) => format!("{}-{}", f, t), + _ => "*".to_string(), + }; + + let mut sources = Vec::new(); + if let Some(ranges) = perm["IpRanges"].as_array() { + for r in ranges { + if let Some(cidr) = r["CidrIp"].as_str() { + sources.push(cidr.to_string()); + } + } + } + if let Some(ranges) = perm["Ipv6Ranges"].as_array() { + for r in ranges { + if let Some(cidr) = r["CidrIpv6"].as_str() { + sources.push(cidr.to_string()); + } + } + } + if let Some(groups) = perm["UserIdGroupPairs"].as_array() { + for g in groups { + let gid = g["GroupId"].as_str().unwrap_or("?"); + sources.push(gid.to_string()); + } + } + + let src = if sources.is_empty() { + "?".to_string() + } else { + sources.join(",") + }; + + if proto == "all" { + format!("all<-{}", src) + } else { + format!("{}/{}<-{}", proto, port, src) + } +} + +fn filter_security_groups(json_str: &str) -> Option { + let v: Value = serde_json::from_str(json_str).ok()?; + let groups = v["SecurityGroups"].as_array()?; + + let total = groups.len(); + let truncated = total > MAX_ITEMS; + let mut result = Vec::new(); + + for sg in groups.iter().take(MAX_ITEMS) { + let name = sg["GroupName"].as_str().unwrap_or("?"); + let id = sg["GroupId"].as_str().unwrap_or("?"); + + let ingress: Vec = sg["IpPermissions"] + .as_array() + .map(|perms| perms.iter().map(format_sg_rule).collect()) + .unwrap_or_default(); + let egress: Vec = sg["IpPermissionsEgress"] + .as_array() + .map(|perms| perms.iter().map(format_sg_rule).collect()) + .unwrap_or_default(); + + let ingress_str = if ingress.is_empty() { + "none".to_string() + } else { + ingress.join(", ") + }; + let egress_str = if egress.is_empty() { + "none".to_string() + } else { + egress.join(", ") + }; + + result.push(format!( + "{} ({}) ingress: {} | egress: {}", + name, id, ingress_str, egress_str + )); + } + + let text = join_with_overflow(&result, total, MAX_ITEMS, "groups"); + Some(if truncated { + FilterResult::truncated(text) + } else { + FilterResult::new(text) + }) +} + +fn filter_s3_objects(json_str: &str) -> Option { + let v: Value = serde_json::from_str(json_str).ok()?; + let empty_vec = vec![]; + let contents = v["Contents"].as_array().unwrap_or(&empty_vec); + + let total = contents.len(); + let truncated = total > MAX_ITEMS; + let mut result = Vec::new(); + + for obj in contents.iter().take(MAX_ITEMS) { + let key = obj["Key"].as_str().unwrap_or("?"); + let size = obj["Size"].as_u64().unwrap_or(0); + let modified = obj["LastModified"] + .as_str() + .map(truncate_iso_date) + .unwrap_or("?"); + result.push(format!("{} {} {}", key, human_bytes(size), modified)); + } + + let text = join_with_overflow(&result, total, MAX_ITEMS, "objects"); + Some(if truncated { + FilterResult::truncated(text) + } else { + FilterResult::new(text) + }) +} + +fn filter_eks_cluster(json_str: &str) -> Option { + let v: Value = serde_json::from_str(json_str).ok()?; + let cluster = &v["cluster"]; + + let name = cluster["name"].as_str().unwrap_or("?"); + let status = cluster["status"].as_str().unwrap_or("?"); + let version = cluster["version"].as_str().unwrap_or("?"); + let endpoint = cluster["endpoint"].as_str().unwrap_or("?"); + // certificateAuthority intentionally NOT read (base64 cert, 1000+ chars) + + let text = format!("{} {} k8s/{} {}", name, status, version, endpoint); + Some(FilterResult::new(text)) +} + +lazy_static! { + static ref S3_TRANSFER_RE: Regex = Regex::new(r"^(upload|download|delete|copy|move):").unwrap(); +} + +fn filter_sqs_messages(json_str: &str) -> Option { + let v: Value = serde_json::from_str(json_str).ok()?; + let empty_vec = vec![]; + let messages = v["Messages"].as_array().unwrap_or(&empty_vec); + + let total = messages.len(); + let truncated = total > MAX_ITEMS; + let mut result = Vec::new(); + + for msg in messages.iter().take(MAX_ITEMS) { + let id = msg["MessageId"].as_str().unwrap_or("?"); + let id_short = &id[..id.len().min(8)]; // UUIDs are ASCII-safe + let body = msg["Body"].as_str().unwrap_or("?"); + let body_truncated = crate::core::utils::truncate(body, 200); + // ReceiptHandle intentionally NOT read (200+ chars of opaque garbage) + result.push(format!("{} {}", id_short, body_truncated)); + } + + let text = join_with_overflow(&result, total, MAX_ITEMS, "messages"); + Some(if truncated { + FilterResult::truncated(text) + } else { + FilterResult::new(text) + }) +} + +fn filter_dynamodb_get_item(json_str: &str) -> Option { + let v: Value = serde_json::from_str(json_str).ok()?; + + let mut lines = Vec::new(); + + // Extract and unwrap the Item + if let Some(item) = v["Item"].as_object() { + let unwrapped = unwrap_dynamodb_value(&Value::Object(item.clone()), 0); + let compact = serde_json::to_string(&unwrapped).unwrap_or_else(|_| "?".to_string()); + lines.push(compact); + } + + // Show ConsumedCapacity if present + if let Some(capacity) = v["ConsumedCapacity"].as_object() { + if let Some(units) = capacity["CapacityUnits"].as_f64() { + lines.push(format!("Capacity: {} RCU", units)); + } + } + + if lines.is_empty() { + return None; + } + + Some(FilterResult::new(lines.join("\n"))) +} + +fn filter_logs_query_results(json_str: &str) -> Option { + let v: Value = serde_json::from_str(json_str).ok()?; + + let mut lines = Vec::new(); + + // Show status + if let Some(status) = v["status"].as_str() { + lines.push(format!("Status: {}", status)); + } + + // Extract results array (array of arrays of {field, value} objects) + if let Some(results) = v["results"].as_array() { + let total = results.len(); + let truncated = total > MAX_ITEMS; + + for row in results.iter().take(MAX_ITEMS) { + if let Some(fields) = row.as_array() { + let field_pairs: Vec = fields + .iter() + .filter_map(|field| { + let field_name = field["field"].as_str()?; + // Skip internal @ptr field + if field_name == "@ptr" { + return None; + } + let field_value = match field["value"].as_str() { + Some(s) => s.to_string(), + None => field["value"].to_string(), // numbers, booleans + }; + Some(format!("{}={}", field_name, field_value)) + }) + .collect(); + lines.push(field_pairs.join(" ")); + } + } + + if truncated { + lines.push(format!("... +{} more rows", total - MAX_ITEMS)); + } + + let text = lines.join("\n"); + return Some(if truncated { + FilterResult::truncated(text) + } else { + FilterResult::new(text) + }); + } + + None +} + +fn filter_s3_transfer(output: &str) -> FilterResult { + let lines: Vec<&str> = output.lines().collect(); + let total = lines.len(); + + // Pass through short output unchanged + if total < 10 { + return FilterResult::new(output.to_string()); + } + + // Count operations + let mut uploaded = 0; + let mut downloaded = 0; + let mut deleted = 0; + let mut copied = 0; + let mut moved = 0; + let mut errors = Vec::new(); + + for line in &lines { + if let Some(captures) = S3_TRANSFER_RE.captures(line) { + match captures.get(1).map(|m| m.as_str()) { + Some("upload") => uploaded += 1, + Some("download") => downloaded += 1, + Some("delete") => deleted += 1, + Some("copy") => copied += 1, + Some("move") => moved += 1, + _ => {} + } + } else if line.contains("error") || line.contains("failed") { + errors.push(line.to_string()); + } + } + + let mut summary_parts = Vec::new(); + if uploaded > 0 { + summary_parts.push(format!("{} uploaded", uploaded)); + } + if downloaded > 0 { + summary_parts.push(format!("{} downloaded", downloaded)); + } + if deleted > 0 { + summary_parts.push(format!("{} deleted", deleted)); + } + if copied > 0 { + summary_parts.push(format!("{} copied", copied)); + } + if moved > 0 { + summary_parts.push(format!("{} moved", moved)); + } + + let mut result_lines = Vec::new(); + + if !summary_parts.is_empty() { + result_lines.push(format!( + "S3 transfer: {}, {} errors", + summary_parts.join(", "), + errors.len() + )); + } + + // Include error lines verbatim + for error in errors.iter().take(10) { + result_lines.push(error.clone()); + } + + if result_lines.is_empty() { + return FilterResult::new(output.to_string()); + } + + FilterResult::new(result_lines.join("\n")) +} + +fn filter_secrets_get(json_str: &str) -> Option { + let v: Value = serde_json::from_str(json_str).ok()?; + + let mut lines = Vec::new(); + + // Extract Name + if let Some(name) = v["Name"].as_str() { + lines.push(format!("Name: {}", name)); + } + + // Extract SecretString + if let Some(secret_str) = v["SecretString"].as_str() { + // Try to parse as JSON and compact it + if let Ok(secret_json) = serde_json::from_str::(secret_str) { + let compact = + serde_json::to_string(&secret_json).unwrap_or_else(|_| secret_str.to_string()); + lines.push(format!("Secret: {}", compact)); + } else { + lines.push(format!("Secret: {}", secret_str)); + } + } + + if lines.is_empty() { + return None; + } + + Some(FilterResult::new(lines.join("\n"))) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::core::utils::count_tokens; + + #[test] + fn test_snapshot_sts_identity() { + let json = r#"{ + "UserId": "AIDAEXAMPLEUSERID1234", + "Account": "123456789012", + "Arn": "arn:aws:iam::123456789012:user/dev-user" +}"#; + let result = filter_sts_identity(json).unwrap(); + assert_eq!( + result.text, + "AWS: 123456789012 arn:aws:iam::123456789012:user/dev-user" + ); + assert!(!result.truncated); + } + + #[test] + fn test_snapshot_ec2_instances() { + let json = r#"{"Reservations":[{"Instances":[{"InstanceId":"i-0a1b2c3d4e5f00001","InstanceType":"t3.micro","PrivateIpAddress":"10.0.1.10","PublicIpAddress":"54.1.2.3","VpcId":"vpc-123","SubnetId":"subnet-a","State":{"Code":16,"Name":"running"},"Tags":[{"Key":"Name","Value":"web-server-1"}],"BlockDeviceMappings":[],"SecurityGroups":[{"GroupId":"sg-001"}]},{"InstanceId":"i-0a1b2c3d4e5f00002","InstanceType":"t3.large","PrivateIpAddress":"10.0.2.20","VpcId":"vpc-123","SubnetId":"subnet-b","State":{"Code":80,"Name":"stopped"},"Tags":[{"Key":"Name","Value":"worker-1"}],"BlockDeviceMappings":[],"SecurityGroups":[{"GroupId":"sg-002"}]}]}]}"#; + let result = filter_ec2_instances(json).unwrap(); + assert!(result.text.contains("EC2: 2 instances")); + assert!(result.text.contains("i-0a1b2c3d4e5f00001 running t3.micro 10.0.1.10 pub:54.1.2.3 vpc:vpc-123 subnet:subnet-a sg:[sg-001] (web-server-1)")); + assert!(result + .text + .contains("i-0a1b2c3d4e5f00002 stopped t3.large 10.0.2.20")); + assert!(!result.truncated); + } + + #[test] + fn test_filter_sts_identity() { + let json = r#"{ + "UserId": "AIDAEXAMPLE", + "Account": "123456789012", + "Arn": "arn:aws:iam::123456789012:user/dev" + }"#; + let result = filter_sts_identity(json).unwrap(); + assert_eq!( + result.text, + "AWS: 123456789012 arn:aws:iam::123456789012:user/dev" + ); + } + + #[test] + fn test_filter_sts_identity_missing_fields() { + let json = r#"{}"#; + let result = filter_sts_identity(json).unwrap(); + assert_eq!(result.text, "AWS: ? ?"); + } + + #[test] + fn test_filter_sts_identity_invalid_json() { + let result = filter_sts_identity("not json"); + assert!(result.is_none()); + } + + #[test] + fn test_filter_s3_ls_basic() { + let output = "2024-01-01 bucket1\n2024-01-02 bucket2\n2024-01-03 bucket3\n"; + let result = filter_s3_ls(output); + assert!(result.text.contains("bucket1")); + assert!(result.text.contains("bucket3")); + assert!(!result.truncated); + } + + #[test] + fn test_filter_s3_ls_overflow() { + let mut lines = Vec::new(); + for i in 1..=50 { + lines.push(format!("2024-01-01 bucket{}", i)); + } + let input = lines.join("\n"); + let result = filter_s3_ls(&input); + assert!(result.text.contains("... +20 more items")); + assert!(result.truncated); + } + + #[test] + fn test_filter_ec2_instances() { + let json = r#"{ + "Reservations": [{ "Instances": [{ "InstanceId": "i-abc123", "State": {"Name": "running"}, "InstanceType": "t3.micro", "PrivateIpAddress": "10.0.1.5", - "Tags": [] + "PublicIpAddress": "54.1.2.3", + "VpcId": "vpc-001", + "SubnetId": "subnet-001", + "SecurityGroups": [{"GroupId": "sg-001", "GroupName": "web"}], + "Tags": [{"Key": "Name", "Value": "web-server"}] + }, { + "InstanceId": "i-def456", + "State": {"Name": "stopped"}, + "InstanceType": "t3.large", + "PrivateIpAddress": "10.0.1.6", + "VpcId": "vpc-001", + "SubnetId": "subnet-002", + "SecurityGroups": [{"GroupId": "sg-002", "GroupName": "worker"}], + "Tags": [{"Key": "Name", "Value": "worker"}] + }] + }] + }"#; + let result = filter_ec2_instances(json).unwrap(); + assert!(result.text.contains("EC2: 2 instances")); + assert!(result.text.contains("i-abc123 running t3.micro 10.0.1.5 pub:54.1.2.3 vpc:vpc-001 subnet:subnet-001 sg:[sg-001] (web-server)")); + assert!(result.text.contains("i-def456 stopped t3.large 10.0.1.6")); + assert!(result.text.contains("sg:[sg-002]")); + } + + #[test] + fn test_filter_ec2_no_name_tag() { + let json = r#"{ + "Reservations": [{ + "Instances": [{ + "InstanceId": "i-abc123", + "State": {"Name": "running"}, + "InstanceType": "t3.micro", + "PrivateIpAddress": "10.0.1.5", + "Tags": [] }] }] }"#; - let result = filter_ec2_instances(json).unwrap(); - assert!(result.contains("(-)")); + let result = filter_ec2_instances(json).unwrap(); + assert!(result.text.contains("(-)")); + } + + #[test] + fn test_filter_ec2_invalid_json() { + assert!(filter_ec2_instances("not json").is_none()); + } + + #[test] + fn test_filter_ecs_list_services() { + let json = r#"{ + "serviceArns": [ + "arn:aws:ecs:us-east-1:123:service/cluster/api-service", + "arn:aws:ecs:us-east-1:123:service/cluster/worker-service" + ] + }"#; + let result = filter_ecs_list_services(json).unwrap(); + assert!(result.text.contains("api-service")); + assert!(result.text.contains("worker-service")); + assert!(!result.text.contains("arn:aws")); + } + + #[test] + fn test_filter_ecs_describe_services() { + let json = r#"{ + "services": [{ + "serviceName": "api", + "status": "ACTIVE", + "runningCount": 3, + "desiredCount": 3, + "launchType": "FARGATE" + }] + }"#; + let result = filter_ecs_describe_services(json).unwrap(); + assert_eq!(result.text, "api ACTIVE 3/3 (FARGATE)"); + } + + #[test] + fn test_filter_rds_instances() { + let json = r#"{ + "DBInstances": [{ + "DBInstanceIdentifier": "mydb", + "Engine": "postgres", + "EngineVersion": "15.4", + "DBInstanceClass": "db.t3.micro", + "DBInstanceStatus": "available", + "Endpoint": {"Address": "mydb.cluster-abc.us-east-1.rds.amazonaws.com", "Port": 5432} + }] + }"#; + let result = filter_rds_instances(json).unwrap(); + assert_eq!(result.text, "mydb postgres 15.4 db.t3.micro available mydb.cluster-abc.us-east-1.rds.amazonaws.com:5432"); + } + + #[test] + fn test_filter_cfn_list_stacks() { + let json = r#"{ + "StackSummaries": [{ + "StackName": "my-stack", + "StackStatus": "CREATE_COMPLETE", + "CreationTime": "2024-01-15T10:30:00Z" + }, { + "StackName": "other-stack", + "StackStatus": "UPDATE_COMPLETE", + "LastUpdatedTime": "2024-02-20T14:00:00Z", + "CreationTime": "2024-01-01T00:00:00Z" + }] + }"#; + let result = filter_cfn_list_stacks(json).unwrap(); + assert!(result.text.contains("my-stack CREATE_COMPLETE 2024-01-15")); + assert!(result + .text + .contains("other-stack UPDATE_COMPLETE 2024-02-20")); + } + + #[test] + fn test_filter_cfn_describe_stacks_with_outputs() { + let json = r#"{ + "Stacks": [{ + "StackName": "my-stack", + "StackStatus": "CREATE_COMPLETE", + "CreationTime": "2024-01-15T10:30:00Z", + "Outputs": [ + {"OutputKey": "ApiUrl", "OutputValue": "https://api.example.com"}, + {"OutputKey": "BucketName", "OutputValue": "my-bucket"} + ] + }] + }"#; + let result = filter_cfn_describe_stacks(json).unwrap(); + assert!(result.text.contains("my-stack CREATE_COMPLETE 2024-01-15")); + assert!(result.text.contains("ApiUrl=https://api.example.com")); + assert!(result.text.contains("BucketName=my-bucket")); + } + + #[test] + fn test_filter_cfn_describe_stacks_no_outputs() { + let json = r#"{ + "Stacks": [{ + "StackName": "my-stack", + "StackStatus": "CREATE_COMPLETE", + "CreationTime": "2024-01-15T10:30:00Z" + }] + }"#; + let result = filter_cfn_describe_stacks(json).unwrap(); + assert!(result.text.contains("my-stack CREATE_COMPLETE 2024-01-15")); + assert!(!result.text.contains("=")); + } + + #[test] + fn test_ec2_token_savings() { + let json = r#"{ + "Reservations": [{ + "ReservationId": "r-001", + "OwnerId": "123456789012", + "Groups": [], + "Instances": [{ + "InstanceId": "i-0a1b2c3d4e5f00001", + "ImageId": "ami-0abcdef1234567890", + "InstanceType": "t3.micro", + "KeyName": "my-key-pair", + "LaunchTime": "2024-01-15T10:30:00+00:00", + "Placement": { "AvailabilityZone": "us-east-1a", "GroupName": "", "Tenancy": "default" }, + "PrivateDnsName": "ip-10-0-1-10.ec2.internal", + "PrivateIpAddress": "10.0.1.10", + "PublicDnsName": "ec2-54-0-0-10.compute-1.amazonaws.com", + "PublicIpAddress": "54.0.0.10", + "State": { "Code": 16, "Name": "running" }, + "SubnetId": "subnet-0abc123def456001", + "VpcId": "vpc-0abc123def456001", + "Architecture": "x86_64", + "BlockDeviceMappings": [{ "DeviceName": "/dev/xvda", "Ebs": { "AttachTime": "2024-01-15T10:30:05+00:00", "DeleteOnTermination": true, "Status": "attached", "VolumeId": "vol-001" } }], + "EbsOptimized": false, + "EnaSupport": true, + "Hypervisor": "xen", + "NetworkInterfaces": [{ "NetworkInterfaceId": "eni-001", "PrivateIpAddress": "10.0.1.10", "Status": "in-use" }], + "RootDeviceName": "/dev/xvda", + "RootDeviceType": "ebs", + "SecurityGroups": [{ "GroupId": "sg-001", "GroupName": "web-server-sg" }], + "SourceDestCheck": true, + "Tags": [{ "Key": "Name", "Value": "web-server-1" }, { "Key": "Environment", "Value": "production" }, { "Key": "Team", "Value": "backend" }], + "VirtualizationType": "hvm", + "CpuOptions": { "CoreCount": 1, "ThreadsPerCore": 2 }, + "MetadataOptions": { "State": "applied", "HttpTokens": "required", "HttpEndpoint": "enabled" } + }] + }] +}"#; + let result = filter_ec2_instances(json).unwrap(); + let input_tokens = count_tokens(json); + let output_tokens = count_tokens(&result.text); + let savings = 100.0 - (output_tokens as f64 / input_tokens as f64 * 100.0); + assert!( + savings >= 60.0, + "EC2 filter: expected >=60% savings, got {:.1}%", + savings + ); + } + + #[test] + fn test_sts_token_savings() { + let json = r#"{ + "UserId": "AIDAEXAMPLEUSERID1234", + "Account": "123456789012", + "Arn": "arn:aws:iam::123456789012:user/dev-user" +}"#; + let result = filter_sts_identity(json).unwrap(); + let input_tokens = count_tokens(json); + let output_tokens = count_tokens(&result.text); + let savings = 100.0 - (output_tokens as f64 / input_tokens as f64 * 100.0); + assert!( + savings >= 60.0, + "STS identity filter: expected >=60% savings, got {:.1}%", + savings + ); + } + + #[test] + fn test_rds_overflow() { + let mut dbs = Vec::new(); + for i in 1..=25 { + dbs.push(format!( + r#"{{"DBInstanceIdentifier": "db-{}", "Engine": "postgres", "EngineVersion": "15.4", "DBInstanceClass": "db.t3.micro", "DBInstanceStatus": "available"}}"#, + i + )); + } + let json = format!(r#"{{"DBInstances": [{}]}}"#, dbs.join(",")); + let result = filter_rds_instances(&json).unwrap(); + assert!(result.text.contains("... +5 more instances")); + assert!(result.truncated); + } + + // === P0 filter tests === + + #[test] + fn test_filter_logs_events() { + let json = r#"{ + "events": [ + {"timestamp": 1705312200000, "message": "INFO: Starting service\n", "ingestionTime": 1705312201000}, + {"timestamp": 1705312260000, "message": "ERROR: Connection refused\n", "ingestionTime": 1705312261000}, + {"timestamp": 1705312320000, "message": "{\"level\":\"warn\",\"msg\":\"retrying\"}\n", "ingestionTime": 1705312321000} + ], + "nextForwardToken": "f/1234567890abcdef1234567890abcdef", + "nextBackwardToken": "b/1234567890abcdef1234567890abcdef" + }"#; + let result = filter_logs_events(json).unwrap(); + assert!(result.text.contains("INFO: Starting service")); + assert!(result.text.contains("ERROR: Connection refused")); + // JSON log message should be compacted to single line + assert!(result.text.contains("retrying")); + // Pagination tokens should NOT appear + assert!(!result.text.contains("nextForwardToken")); + assert!(!result.text.contains("f/1234567890")); + assert!(!result.truncated); + } + + #[test] + fn test_filter_logs_events_truncation() { + let mut events = Vec::new(); + for i in 0..60 { + events.push(format!( + r#"{{"timestamp": {}, "message": "line {}", "ingestionTime": {}}}"#, + 1705312200000i64 + i * 1000, + i, + 1705312200000i64 + i * 1000 + 100 + )); + } + let json = format!(r#"{{"events": [{}]}}"#, events.join(",")); + let result = filter_logs_events(&json).unwrap(); + assert!(result.text.contains("... +10 more events")); + assert!(result.truncated); + } + + #[test] + fn test_filter_logs_events_token_savings() { + let mut events = Vec::new(); + for i in 0..20 { + events.push(format!( + r#"{{"timestamp": {}, "message": "2024-01-15T10:30:{:02}Z INFO [com.example.service.Handler] Processing request id={} user=admin@example.com action=GET /api/v1/items?limit=100&offset=0 duration={}ms", "ingestionTime": {}}}"#, + 1705312200000i64 + i * 1000, + i, + 1000 + i, + 50 + i * 10, + 1705312200000i64 + i * 1000 + 100 + )); + } + let json = format!( + r#"{{"events": [{}], "nextForwardToken": "f/abcdef1234567890abcdef1234567890abcdef1234567890", "nextBackwardToken": "b/abcdef1234567890abcdef1234567890abcdef1234567890"}}"#, + events.join(",") + ); + let result = filter_logs_events(&json).unwrap(); + let input_tokens = count_tokens(&json); + let output_tokens = count_tokens(&result.text); + let savings = 100.0 - (output_tokens as f64 / input_tokens as f64 * 100.0); + // Logs savings come from stripping ingestionTime, pagination tokens, and JSON keys. + // With realistic fixtures the savings are modest per-event but the pagination + // tokens alone save ~20 tokens each. + assert!( + savings >= 15.0, + "Logs filter: expected >=15% savings, got {:.1}%", + savings + ); + } + + #[test] + fn test_filter_logs_events_invalid_json() { + assert!(filter_logs_events("not json").is_none()); + } + + #[test] + fn test_filter_cfn_events() { + let json = r#"{ + "StackEvents": [ + { + "Timestamp": "2024-01-15T10:30:00Z", + "LogicalResourceId": "MyBucket", + "ResourceType": "AWS::S3::Bucket", + "ResourceStatus": "CREATE_FAILED", + "ResourceStatusReason": "Bucket already exists", + "ResourceProperties": "{\"BucketName\":\"my-bucket\",\"VersioningConfiguration\":{\"Status\":\"Enabled\"},\"Tags\":[{\"Key\":\"Env\",\"Value\":\"prod\"}]}" + }, + { + "Timestamp": "2024-01-15T10:29:00Z", + "LogicalResourceId": "MyVpc", + "ResourceType": "AWS::EC2::VPC", + "ResourceStatus": "CREATE_COMPLETE", + "ResourceProperties": "{\"CidrBlock\":\"10.0.0.0/16\"}" + }, + { + "Timestamp": "2024-01-15T10:28:00Z", + "LogicalResourceId": "MyStack", + "ResourceType": "AWS::CloudFormation::Stack", + "ResourceStatus": "ROLLBACK_IN_PROGRESS", + "ResourceStatusReason": "The following resource(s) failed to create: [MyBucket]" + } + ] + }"#; + let result = filter_cfn_events(json).unwrap(); + assert!(result.text.contains("3 events")); + assert!(result.text.contains("2 failed")); + assert!(result.text.contains("1 successful")); + assert!(result.text.contains("FAILURES")); + assert!(result.text.contains("MyBucket")); + assert!(result.text.contains("Bucket already exists")); + // ResourceProperties should NOT appear + assert!(!result.text.contains("BucketName")); + assert!(!result.text.contains("CidrBlock")); + // AWS:: prefix stripped from resource type + assert!(result.text.contains("S3::Bucket")); + assert!(!result.text.contains("AWS::S3")); + } + + #[test] + fn test_filter_cfn_events_token_savings() { + let json = r#"{ + "StackEvents": [ + {"Timestamp": "2024-01-15T10:30:00Z", "LogicalResourceId": "Res1", "ResourceType": "AWS::Lambda::Function", "ResourceStatus": "CREATE_FAILED", "ResourceStatusReason": "Error", "ResourceProperties": "{\"FunctionName\":\"my-fn\",\"Runtime\":\"python3.12\",\"Handler\":\"index.handler\",\"MemorySize\":512,\"Timeout\":30,\"Role\":\"arn:aws:iam::123:role/my-role\",\"Environment\":{\"Variables\":{\"TABLE\":\"my-table\"}}}"}, + {"Timestamp": "2024-01-15T10:29:00Z", "LogicalResourceId": "Res2", "ResourceType": "AWS::EC2::VPC", "ResourceStatus": "CREATE_COMPLETE", "ResourceProperties": "{\"CidrBlock\":\"10.0.0.0/16\",\"EnableDnsSupport\":true,\"EnableDnsHostnames\":true}"}, + {"Timestamp": "2024-01-15T10:28:00Z", "LogicalResourceId": "Res3", "ResourceType": "AWS::S3::Bucket", "ResourceStatus": "CREATE_COMPLETE", "ResourceProperties": "{\"BucketName\":\"my-bucket\",\"VersioningConfiguration\":{\"Status\":\"Enabled\"}}"} + ] + }"#; + let result = filter_cfn_events(json).unwrap(); + let input_tokens = count_tokens(json); + let output_tokens = count_tokens(&result.text); + let savings = 100.0 - (output_tokens as f64 / input_tokens as f64 * 100.0); + // Real CF deployments have 30+ events with huge ResourceProperties + // (stringified JSON). Small fixture shows ~46% but real-world is 90%+. + assert!( + savings >= 40.0, + "CFN events filter: expected >=40% savings, got {:.1}%", + savings + ); + } + + #[test] + fn test_filter_lambda_list() { + let json = r#"{ + "Functions": [ + {"FunctionName": "my-api", "Runtime": "python3.12", "MemorySize": 512, "Timeout": 30, "State": "Active", "Environment": {"Variables": {"SECRET_KEY": "s3cr3t", "DB_PASSWORD": "hunter2"}}}, + {"FunctionName": "my-worker", "Runtime": "nodejs20.x", "MemorySize": 256, "Timeout": 60, "State": "Active"} + ] + }"#; + let result = filter_lambda_list(json).unwrap(); + assert!(result.text.contains("my-api python3.12 512MB 30s Active")); + assert!(result + .text + .contains("my-worker nodejs20.x 256MB 60s Active")); + // SECURITY: secrets must NOT appear + assert!(!result.text.contains("SECRET_KEY")); + assert!(!result.text.contains("s3cr3t")); + assert!(!result.text.contains("DB_PASSWORD")); + assert!(!result.text.contains("hunter2")); + assert!(!result.truncated); + } + + #[test] + fn test_filter_lambda_list_token_savings() { + let json = r#"{ + "Functions": [ + {"FunctionName": "fn-1", "FunctionArn": "arn:aws:lambda:us-east-1:123:function:fn-1", "Runtime": "python3.12", "Role": "arn:aws:iam::123:role/role-1", "Handler": "index.handler", "CodeSize": 5242880, "Description": "A function", "Timeout": 30, "MemorySize": 512, "LastModified": "2024-01-15T10:30:00.000+0000", "CodeSha256": "abc123def456", "Version": "$LATEST", "TracingConfig": {"Mode": "Active"}, "RevisionId": "rev-123", "State": "Active", "LastUpdateStatus": "Successful", "PackageType": "Zip", "Architectures": ["x86_64"], "EphemeralStorage": {"Size": 512}, "Environment": {"Variables": {"TABLE_NAME": "my-table", "API_KEY": "secret-api-key-12345"}}} + ] + }"#; + let result = filter_lambda_list(json).unwrap(); + let input_tokens = count_tokens(json); + let output_tokens = count_tokens(&result.text); + let savings = 100.0 - (output_tokens as f64 / input_tokens as f64 * 100.0); + assert!( + savings >= 60.0, + "Lambda list filter: expected >=60% savings, got {:.1}%", + savings + ); + } + + #[test] + fn test_filter_lambda_get() { + let json = r#"{ + "Configuration": { + "FunctionName": "my-api", + "Runtime": "python3.12", + "Handler": "app.handler", + "MemorySize": 512, + "Timeout": 30, + "State": "Active", + "LastModified": "2024-01-15T10:30:00.000+0000", + "Environment": {"Variables": {"SECRET": "hunter2"}}, + "Layers": [ + {"Arn": "arn:aws:lambda:us-east-1:123:layer:my-layer:5"}, + {"Arn": "arn:aws:lambda:us-east-1:123:layer:common-utils:3"} + ] + }, + "Code": {"Location": "https://awslambda-us-east-1-tasks.s3.amazonaws.com/snapshots/123/my-func?versionId=abc&X-Amz-Security-Token=very-long-token"}, + "Tags": {"Team": "backend"} + }"#; + let result = filter_lambda_get(json).unwrap(); + assert!(result + .text + .contains("my-api python3.12 app.handler 512MB 30s Active 2024-01-15")); + assert!(result.text.contains("layers: my-layer:5, common-utils:3")); + // SECURITY + assert!(!result.text.contains("SECRET")); + assert!(!result.text.contains("hunter2")); + assert!(!result.text.contains("awslambda")); + assert!(!result.text.contains("X-Amz-Security-Token")); + } + + #[test] + fn test_filter_lambda_get_no_layers() { + let json = r#"{ + "Configuration": { + "FunctionName": "simple-fn", + "Runtime": "nodejs20.x", + "Handler": "index.handler", + "MemorySize": 128, + "Timeout": 10, + "State": "Active", + "LastModified": "2024-02-20T14:00:00.000+0000" + }, + "Code": {"Location": "https://example.com/code"} + }"#; + let result = filter_lambda_get(json).unwrap(); + assert!(result.text.contains("simple-fn")); + assert!(!result.text.contains("layers")); + } + + #[test] + fn test_filter_lambda_list_invalid_json() { + assert!(filter_lambda_list("not json").is_none()); + } + + #[test] + fn test_filter_cfn_events_invalid_json() { + assert!(filter_cfn_events("not json").is_none()); + } + + // === P1 filter tests === + + #[test] + fn test_filter_iam_roles() { + let json = r#"{ + "Roles": [ + {"RoleName": "admin-role", "CreateDate": "2024-01-15T10:30:00Z", "Description": "Admin access", "AssumeRolePolicyDocument": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"lambda.amazonaws.com\"},\"Action\":\"sts:AssumeRole\"}]}"}, + {"RoleName": "lambda-exec", "CreateDate": "2024-02-20T14:00:00Z", "AssumeRolePolicyDocument": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"lambda.amazonaws.com\"},\"Action\":\"sts:AssumeRole\"}]}"} + ] + }"#; + let result = filter_iam_roles(json).unwrap(); + assert!(result + .text + .contains("admin-role 2024-01-15 [Admin access] assume:[lambda.amazonaws.com]")); + assert!(result + .text + .contains("lambda-exec 2024-02-20 assume:[lambda.amazonaws.com]")); + // Full policy JSON should NOT appear, only extracted principals + assert!(!result.text.contains("Statement")); + assert!(!result.text.contains("Version")); + } + + #[test] + fn test_filter_iam_roles_token_savings() { + let json = r#"{ + "Roles": [ + {"RoleName": "role-1", "RoleId": "AROA1234567890", "Arn": "arn:aws:iam::123:role/role-1", "Path": "/", "CreateDate": "2024-01-15T10:30:00Z", "MaxSessionDuration": 3600, "Description": "Test role", "AssumeRolePolicyDocument": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"lambda.amazonaws.com\"},\"Action\":\"sts:AssumeRole\"}]}", "Tags": [{"Key": "Team", "Value": "backend"}]} + ] + }"#; + let result = filter_iam_roles(json).unwrap(); + let input_tokens = count_tokens(json); + let output_tokens = count_tokens(&result.text); + let savings = 100.0 - (output_tokens as f64 / input_tokens as f64 * 100.0); + assert!( + savings >= 60.0, + "IAM roles filter: expected >=60% savings, got {:.1}%", + savings + ); + } + + #[test] + fn test_filter_iam_users() { + let json = r#"{ + "Users": [ + {"UserName": "alice", "UserId": "AIDA1234", "Arn": "arn:aws:iam::123:user/alice", "Path": "/", "CreateDate": "2024-01-15T10:30:00Z"}, + {"UserName": "bob", "UserId": "AIDA5678", "Arn": "arn:aws:iam::123:user/bob", "Path": "/", "CreateDate": "2024-02-20T14:00:00Z"} + ] + }"#; + let result = filter_iam_users(json).unwrap(); + assert!(result.text.contains("alice created:2024-01-15")); + assert!(result.text.contains("bob created:2024-02-20")); + assert!(!result.text.contains("AIDA")); + assert!(!result.text.contains("arn:aws")); + } + + #[test] + fn test_filter_dynamodb_items() { + let json = r#"{ + "Items": [ + {"id": {"S": "user-1"}, "name": {"S": "Alice"}, "age": {"N": "30"}, "active": {"BOOL": true}}, + {"id": {"S": "user-2"}, "name": {"S": "Bob"}, "scores": {"L": [{"N": "100"}, {"N": "95"}]}, "meta": {"M": {"role": {"S": "admin"}}}} + ], + "Count": 2, + "ScannedCount": 100 + }"#; + let result = filter_dynamodb_items(json).unwrap(); + assert!(result.text.contains("Count: 2/100")); + // Type wrappers should be unwrapped + assert!(result.text.contains("\"Alice\"")); + assert!(result.text.contains("\"Bob\"")); + assert!(!result.text.contains(r#""S""#)); + assert!(!result.text.contains(r#""N""#)); + assert!(!result.text.contains(r#""BOOL""#)); + // Nested types should be unwrapped too + assert!(result.text.contains("\"admin\"")); + } + + #[test] + fn test_filter_dynamodb_token_savings() { + let json = r#"{ + "Items": [ + {"pk": {"S": "USER#1"}, "sk": {"S": "PROFILE"}, "name": {"S": "Alice"}, "email": {"S": "alice@example.com"}, "age": {"N": "30"}, "active": {"BOOL": true}, "tags": {"SS": ["admin", "user"]}, "meta": {"M": {"role": {"S": "admin"}, "team": {"S": "backend"}}}, "scores": {"L": [{"N": "100"}, {"N": "95"}, {"N": "88"}]}}, + {"pk": {"S": "USER#2"}, "sk": {"S": "PROFILE"}, "name": {"S": "Bob"}, "email": {"S": "bob@example.com"}, "age": {"N": "25"}, "active": {"BOOL": false}, "tags": {"SS": ["user"]}, "meta": {"M": {"role": {"S": "viewer"}, "team": {"S": "frontend"}}}, "scores": {"L": [{"N": "80"}, {"N": "75"}]}} + ], + "Count": 2, + "ScannedCount": 2 + }"#; + let result = filter_dynamodb_items(json).unwrap(); + let input_tokens = count_tokens(json); + let output_tokens = count_tokens(&result.text); + let savings = 100.0 - (output_tokens as f64 / input_tokens as f64 * 100.0); + assert!( + savings >= 30.0, + "DynamoDB filter: expected >=30% savings, got {:.1}%", + savings + ); + } + + #[test] + fn test_filter_dynamodb_null_type() { + let json = r#"{ + "Items": [{"id": {"S": "1"}, "deleted_at": {"NULL": true}}], + "Count": 1, + "ScannedCount": 1 + }"#; + let result = filter_dynamodb_items(json).unwrap(); + assert!(result.text.contains("null")); + assert!(!result.text.contains("NULL")); + } + + #[test] + fn test_filter_ecs_tasks() { + let json = r#"{ + "tasks": [ + { + "taskArn": "arn:aws:ecs:us-east-1:123:task/my-cluster/abc123def456", + "lastStatus": "RUNNING", + "desiredStatus": "RUNNING", + "containers": [ + {"name": "web", "lastStatus": "RUNNING"}, + {"name": "sidecar", "lastStatus": "RUNNING"} + ], + "attachments": [{"id": "eni-123", "type": "ElasticNetworkInterface", "status": "ATTACHED", "details": []}], + "overrides": {"containerOverrides": []} + }, + { + "taskArn": "arn:aws:ecs:us-east-1:123:task/my-cluster/def789ghi012", + "lastStatus": "STOPPED", + "stoppedReason": "Essential container in task exited", + "containers": [ + {"name": "worker", "lastStatus": "STOPPED", "exitCode": 1} + ], + "attachments": [], + "overrides": {} + } + ] + }"#; + let result = filter_ecs_tasks(json).unwrap(); + assert!(result + .text + .contains("abc123def456 RUNNING containers:[web:RUNNING, sidecar:RUNNING]")); + assert!(result + .text + .contains("def789ghi012 STOPPED containers:[worker:STOPPED(exit:1)]")); + assert!(result + .text + .contains("reason:Essential container in task exited")); + // Attachments and overrides should NOT appear + assert!(!result.text.contains("ElasticNetworkInterface")); + assert!(!result.text.contains("containerOverrides")); + } + + #[test] + fn test_filter_iam_roles_invalid_json() { + assert!(filter_iam_roles("not json").is_none()); + } + + #[test] + fn test_filter_dynamodb_invalid_json() { + assert!(filter_dynamodb_items("not json").is_none()); + } + + #[test] + fn test_filter_ecs_tasks_invalid_json() { + assert!(filter_ecs_tasks("not json").is_none()); + } + + // === P2 filter tests === + + #[test] + fn test_filter_security_groups() { + let json = r#"{ + "SecurityGroups": [{ + "GroupName": "web-sg", + "GroupId": "sg-001", + "IpPermissions": [ + {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}, + {"IpProtocol": "tcp", "FromPort": 22, "ToPort": 22, "IpRanges": [{"CidrIp": "10.0.0.0/8"}], "Ipv6Ranges": [], "UserIdGroupPairs": []} + ], + "IpPermissionsEgress": [ + {"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []} + ] + }] + }"#; + let result = filter_security_groups(json).unwrap(); + assert!(result.text.contains("web-sg (sg-001)")); + assert!(result.text.contains("tcp/443<-0.0.0.0/0")); + assert!(result.text.contains("tcp/22<-10.0.0.0/8")); + assert!(result.text.contains("all<-0.0.0.0/0")); } #[test] - fn test_filter_ec2_invalid_json() { - assert!(filter_ec2_instances("not json").is_none()); + fn test_filter_security_groups_token_savings() { + let json = r#"{ + "SecurityGroups": [{ + "GroupName": "web-sg", "GroupId": "sg-001", "Description": "Web server security group", "VpcId": "vpc-001", "OwnerId": "123456789012", + "IpPermissions": [ + {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "HTTPS from anywhere"}], "Ipv6Ranges": [{"CidrIpv6": "::/0", "Description": "HTTPS IPv6"}], "PrefixListIds": [], "UserIdGroupPairs": []}, + {"IpProtocol": "tcp", "FromPort": 80, "ToPort": 80, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "HTTP from anywhere"}], "Ipv6Ranges": [], "PrefixListIds": [], "UserIdGroupPairs": []} + ], + "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [{"CidrIpv6": "::/0"}], "PrefixListIds": [], "UserIdGroupPairs": []}], + "Tags": [{"Key": "Name", "Value": "web-sg"}, {"Key": "Environment", "Value": "production"}] + }] + }"#; + let result = filter_security_groups(json).unwrap(); + let input_tokens = count_tokens(json); + let output_tokens = count_tokens(&result.text); + let savings = 100.0 - (output_tokens as f64 / input_tokens as f64 * 100.0); + assert!( + savings >= 60.0, + "SG filter: expected >=60% savings, got {:.1}%", + savings + ); } #[test] - fn test_filter_ecs_list_services() { + fn test_filter_s3_objects() { let json = r#"{ - "serviceArns": [ - "arn:aws:ecs:us-east-1:123:service/cluster/api-service", - "arn:aws:ecs:us-east-1:123:service/cluster/worker-service" + "Contents": [ + {"Key": "data/users.csv", "Size": 5242880, "LastModified": "2024-01-15T10:30:00Z", "ETag": "\"abc123\"", "StorageClass": "STANDARD"}, + {"Key": "logs/app.log", "Size": 1024, "LastModified": "2024-02-20T14:00:00Z", "ETag": "\"def456\"", "StorageClass": "STANDARD"} ] }"#; - let result = filter_ecs_list_services(json).unwrap(); - assert!(result.contains("api-service")); - assert!(result.contains("worker-service")); - assert!(!result.contains("arn:aws")); + let result = filter_s3_objects(json).unwrap(); + assert!(result.text.contains("data/users.csv 5.0 MB 2024-01-15")); + assert!(result.text.contains("logs/app.log 1.0 KB 2024-02-20")); + // ETag and StorageClass should NOT appear + assert!(!result.text.contains("abc123")); + assert!(!result.text.contains("STANDARD")); } #[test] - fn test_filter_ecs_describe_services() { + fn test_filter_eks_cluster() { let json = r#"{ - "services": [{ - "serviceName": "api", + "cluster": { + "name": "my-cluster", "status": "ACTIVE", - "runningCount": 3, - "desiredCount": 3, - "launchType": "FARGATE" - }] + "version": "1.28", + "endpoint": "https://ABC123.gr7.us-east-1.eks.amazonaws.com", + "certificateAuthority": {"data": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY21...VERY_LONG_BASE64_CERT_DATA"}, + "logging": {"clusterLogging": [{"types": ["api","audit","authenticator","controllerManager","scheduler"], "enabled": true}]}, + "platformVersion": "eks.5" + } }"#; - let result = filter_ecs_describe_services(json).unwrap(); - assert_eq!(result, "api ACTIVE 3/3 (FARGATE)"); + let result = filter_eks_cluster(json).unwrap(); + assert!(result + .text + .contains("my-cluster ACTIVE k8s/1.28 https://ABC123.gr7.us-east-1.eks.amazonaws.com")); + // certificateAuthority should NOT appear + assert!(!result.text.contains("LS0tLS1CRUdJTi")); + assert!(!result.text.contains("VERY_LONG")); } #[test] - fn test_filter_rds_instances() { + fn test_filter_sqs_messages() { let json = r#"{ - "DBInstances": [{ - "DBInstanceIdentifier": "mydb", - "Engine": "postgres", - "EngineVersion": "15.4", - "DBInstanceClass": "db.t3.micro", - "DBInstanceStatus": "available" - }] + "Messages": [ + { + "MessageId": "12345678-abcd-efgh-ijkl-1234567890ab", + "ReceiptHandle": "AQEBwJnKyrHigUMZj6rYigCgxlaS3SLy0a...VERY_LONG_RECEIPT_HANDLE_200_CHARS_OF_OPAQUE_GARBAGE_THAT_NOBODY_NEEDS", + "MD5OfBody": "abc123", + "Body": "{\"orderId\": 42, \"status\": \"pending\"}" + } + ] }"#; - let result = filter_rds_instances(json).unwrap(); - assert_eq!(result, "mydb postgres 15.4 db.t3.micro available"); + let result = filter_sqs_messages(json).unwrap(); + assert!(result.text.contains("12345678")); + assert!(result.text.contains("orderId")); + // ReceiptHandle should NOT appear + assert!(!result.text.contains("AQEBwJnK")); + assert!(!result.text.contains("OPAQUE_GARBAGE")); + assert!(!result.text.contains("MD5OfBody")); } #[test] - fn test_filter_cfn_list_stacks() { + fn test_filter_security_groups_invalid_json() { + assert!(filter_security_groups("not json").is_none()); + } + + #[test] + fn test_filter_s3_objects_invalid_json() { + assert!(filter_s3_objects("not json").is_none()); + } + + #[test] + fn test_filter_eks_cluster_invalid_json() { + assert!(filter_eks_cluster("not json").is_none()); + } + + #[test] + fn test_filter_sqs_messages_invalid_json() { + assert!(filter_sqs_messages("not json").is_none()); + } + + #[test] + fn test_filter_dynamodb_get_item() { let json = r#"{ - "StackSummaries": [{ - "StackName": "my-stack", - "StackStatus": "CREATE_COMPLETE", - "CreationTime": "2024-01-15T10:30:00Z" - }, { - "StackName": "other-stack", - "StackStatus": "UPDATE_COMPLETE", - "LastUpdatedTime": "2024-02-20T14:00:00Z", - "CreationTime": "2024-01-01T00:00:00Z" - }] + "Item": { + "id": {"N": "123"}, + "name": {"S": "test-item"}, + "price": {"N": "19.99"}, + "tags": {"L": [{"S": "new"}, {"S": "sale"}]}, + "metadata": {"M": {"key": {"S": "value"}}} + }, + "ConsumedCapacity": { + "CapacityUnits": 1.0 + } }"#; - let result = filter_cfn_list_stacks(json).unwrap(); - assert!(result.contains("my-stack CREATE_COMPLETE 2024-01-15")); - assert!(result.contains("other-stack UPDATE_COMPLETE 2024-02-20")); + let result = filter_dynamodb_get_item(json).unwrap(); + assert!(result.text.contains(r#""id":123"#)); + assert!(result.text.contains(r#""name":"test-item""#)); + assert!(result.text.contains("Capacity: 1 RCU")); } #[test] - fn test_filter_cfn_describe_stacks_with_outputs() { + fn test_filter_dynamodb_get_item_no_item() { + let json = r#"{}"#; + assert!(filter_dynamodb_get_item(json).is_none()); + } + + #[test] + fn test_filter_dynamodb_get_item_invalid_json() { + assert!(filter_dynamodb_get_item("not json").is_none()); + } + + #[test] + fn test_filter_logs_query_results() { let json = r#"{ - "Stacks": [{ - "StackName": "my-stack", - "StackStatus": "CREATE_COMPLETE", - "CreationTime": "2024-01-15T10:30:00Z", - "Outputs": [ - {"OutputKey": "ApiUrl", "OutputValue": "https://api.example.com"}, - {"OutputKey": "BucketName", "OutputValue": "my-bucket"} + "status": "Complete", + "results": [ + [ + {"field": "@timestamp", "value": "2024-01-01 12:00:00"}, + {"field": "@message", "value": "Error occurred"}, + {"field": "@ptr", "value": "internal-pointer"} + ], + [ + {"field": "@timestamp", "value": "2024-01-01 12:01:00"}, + {"field": "@message", "value": "Another error"} ] - }] + ] }"#; - let result = filter_cfn_describe_stacks(json).unwrap(); - assert!(result.contains("my-stack CREATE_COMPLETE 2024-01-15")); - assert!(result.contains("ApiUrl=https://api.example.com")); - assert!(result.contains("BucketName=my-bucket")); + let result = filter_logs_query_results(json).unwrap(); + assert!(result.text.contains("Status: Complete")); + assert!(result.text.contains("@timestamp=2024-01-01 12:00:00")); + assert!(result.text.contains("@message=Error occurred")); + assert!(!result.text.contains("@ptr")); // Should be filtered out } #[test] - fn test_filter_cfn_describe_stacks_no_outputs() { + fn test_filter_logs_query_results_empty() { + let json = r#"{"status": "Complete", "results": []}"#; + let result = filter_logs_query_results(json).unwrap(); + assert_eq!(result.text, "Status: Complete"); + } + + #[test] + fn test_filter_logs_query_results_invalid_json() { + assert!(filter_logs_query_results("not json").is_none()); + } + + #[test] + fn test_filter_s3_transfer_short_output() { + let output = "upload: file1.txt to s3://bucket/file1.txt\n"; + let result = filter_s3_transfer(output); + // Short output passes through unchanged + assert_eq!(result.text, output); + } + + #[test] + fn test_filter_s3_transfer_with_operations() { + let output = "\ +upload: file1.txt to s3://bucket/file1.txt +upload: file2.txt to s3://bucket/file2.txt +download: s3://bucket/file3.txt to file3.txt +delete: s3://bucket/old.txt +upload: file4.txt to s3://bucket/file4.txt +upload: file5.txt to s3://bucket/file5.txt +download: s3://bucket/file6.txt to file6.txt +copy: s3://bucket/a.txt to s3://bucket/b.txt +error: failed to upload file7.txt +upload: file8.txt to s3://bucket/file8.txt +upload: file9.txt to s3://bucket/file9.txt +upload: file10.txt to s3://bucket/file10.txt +"; + let result = filter_s3_transfer(output); + assert!(result.text.contains("7 uploaded")); + assert!(result.text.contains("2 downloaded")); + assert!(result.text.contains("1 deleted")); + assert!(result.text.contains("1 copied")); + assert!(result.text.contains("1 errors")); + assert!(result.text.contains("error: failed to upload file7.txt")); + } + + #[test] + fn test_filter_secrets_get() { let json = r#"{ - "Stacks": [{ - "StackName": "my-stack", - "StackStatus": "CREATE_COMPLETE", - "CreationTime": "2024-01-15T10:30:00Z" - }] + "Name": "my-secret", + "SecretString": "{\"username\":\"admin\",\"password\":\"secret123\"}", + "ARN": "arn:aws:secretsmanager:us-east-1:123456789012:secret:my-secret-AbCdEf", + "VersionId": "version-uuid", + "CreatedDate": "2024-01-01T00:00:00Z" }"#; - let result = filter_cfn_describe_stacks(json).unwrap(); - assert!(result.contains("my-stack CREATE_COMPLETE 2024-01-15")); - assert!(!result.contains("=")); + let result = filter_secrets_get(json).unwrap(); + assert!(result.text.contains("Name: my-secret")); + assert!(result + .text + .contains(r#"{"username":"admin","password":"secret123"}"#)); + assert!(!result.text.contains("ARN")); + assert!(!result.text.contains("VersionId")); } - fn count_tokens(text: &str) -> usize { - text.split_whitespace().count() + #[test] + fn test_filter_secrets_get_plain_text() { + let json = r#"{ + "Name": "my-secret", + "SecretString": "plain-text-password" + }"#; + let result = filter_secrets_get(json).unwrap(); + assert!(result.text.contains("Name: my-secret")); + assert!(result.text.contains("Secret: plain-text-password")); } #[test] - fn test_ec2_token_savings() { + fn test_filter_secrets_get_invalid_json() { + assert!(filter_secrets_get("not json").is_none()); + } + + #[test] + fn test_dynamodb_n_type_parsing() { + // Test i64 + let json = r#"{"N": "123"}"#; + let val: Value = serde_json::from_str(json).unwrap(); + let result = unwrap_dynamodb_value(&val, 0); + assert_eq!(result, Value::Number(123.into())); + + // Test f64 + let json = r#"{"N": "123.45"}"#; + let val: Value = serde_json::from_str(json).unwrap(); + let result = unwrap_dynamodb_value(&val, 0); + assert!(result.is_number()); + } + + #[test] + fn test_dynamodb_ns_type_parsing() { + // Test NS with integers and floats + let json = r#"{"NS": ["123", "456", "78.9"]}"#; + let val: Value = serde_json::from_str(json).unwrap(); + let result = unwrap_dynamodb_value(&val, 0); + let arr = result.as_array().unwrap(); + assert_eq!(arr.len(), 3); + assert_eq!(arr[0], Value::Number(123.into())); + assert_eq!(arr[1], Value::Number(456.into())); + assert!(arr[2].is_number()); + } + + #[test] + fn test_filter_dynamodb_items_with_capacity() { let json = r#"{ - "Reservations": [{ - "ReservationId": "r-001", - "OwnerId": "123456789012", - "Groups": [], - "Instances": [{ - "InstanceId": "i-0a1b2c3d4e5f00001", - "ImageId": "ami-0abcdef1234567890", - "InstanceType": "t3.micro", - "KeyName": "my-key-pair", - "LaunchTime": "2024-01-15T10:30:00+00:00", - "Placement": { "AvailabilityZone": "us-east-1a", "GroupName": "", "Tenancy": "default" }, - "PrivateDnsName": "ip-10-0-1-10.ec2.internal", - "PrivateIpAddress": "10.0.1.10", - "PublicDnsName": "ec2-54-0-0-10.compute-1.amazonaws.com", - "PublicIpAddress": "54.0.0.10", - "State": { "Code": 16, "Name": "running" }, - "SubnetId": "subnet-0abc123def456001", - "VpcId": "vpc-0abc123def456001", - "Architecture": "x86_64", - "BlockDeviceMappings": [{ "DeviceName": "/dev/xvda", "Ebs": { "AttachTime": "2024-01-15T10:30:05+00:00", "DeleteOnTermination": true, "Status": "attached", "VolumeId": "vol-001" } }], - "EbsOptimized": false, - "EnaSupport": true, - "Hypervisor": "xen", - "NetworkInterfaces": [{ "NetworkInterfaceId": "eni-001", "PrivateIpAddress": "10.0.1.10", "Status": "in-use" }], - "RootDeviceName": "/dev/xvda", - "RootDeviceType": "ebs", - "SecurityGroups": [{ "GroupId": "sg-001", "GroupName": "web-server-sg" }], - "SourceDestCheck": true, - "Tags": [{ "Key": "Name", "Value": "web-server-1" }, { "Key": "Environment", "Value": "production" }, { "Key": "Team", "Value": "backend" }], - "VirtualizationType": "hvm", - "CpuOptions": { "CoreCount": 1, "ThreadsPerCore": 2 }, - "MetadataOptions": { "State": "applied", "HttpTokens": "required", "HttpEndpoint": "enabled" } - }] - }] -}"#; - let result = filter_ec2_instances(json).unwrap(); - let input_tokens = count_tokens(json); - let output_tokens = count_tokens(&result); - let savings = 100.0 - (output_tokens as f64 / input_tokens as f64 * 100.0); - assert!( - savings >= 60.0, - "EC2 filter: expected >=60% savings, got {:.1}%", - savings - ); + "Items": [ + {"id": {"N": "1"}, "name": {"S": "item1"}} + ], + "Count": 1, + "ScannedCount": 1, + "ConsumedCapacity": { + "CapacityUnits": 2.5 + } + }"#; + let result = filter_dynamodb_items(json).unwrap(); + assert!(result.text.contains("Count: 1/1")); + assert!(result.text.contains("Capacity: 2.5 RCU")); } #[test] - fn test_sts_token_savings() { + fn test_filter_dynamodb_items_with_pagination() { let json = r#"{ - "UserId": "AIDAEXAMPLEUSERID1234", - "Account": "123456789012", - "Arn": "arn:aws:iam::123456789012:user/dev-user" -}"#; - let result = filter_sts_identity(json).unwrap(); - let input_tokens = count_tokens(json); - let output_tokens = count_tokens(&result); - let savings = 100.0 - (output_tokens as f64 / input_tokens as f64 * 100.0); - assert!( - savings >= 60.0, - "STS identity filter: expected >=60% savings, got {:.1}%", - savings + "Items": [ + {"id": {"N": "1"}, "name": {"S": "item1"}} + ], + "Count": 1, + "ScannedCount": 1, + "LastEvaluatedKey": { + "id": {"N": "1"} + } + }"#; + let result = filter_dynamodb_items(json).unwrap(); + assert!(result.text.contains("Count: 1/1")); + assert!(result.text.contains("(paginated — more results available)")); + } + + // === Snapshot-style tests: verify full output format === + + #[test] + fn test_snapshot_logs_events_format() { + let json = r#"{ + "events": [ + {"timestamp": 1705312200000, "message": "INFO: server started\n", "ingestionTime": 1705312201000}, + {"timestamp": 1705312260000, "message": "ERROR: connection lost\n", "ingestionTime": 1705312261000} + ], + "nextForwardToken": "f/token123" + }"#; + let result = filter_logs_events(json).unwrap(); + assert_eq!( + result.text, + "2024-01-15 09:50:00 INFO: server started\n2024-01-15 09:51:00 ERROR: connection lost" ); } #[test] - fn test_rds_overflow() { - let mut dbs = Vec::new(); - for i in 1..=25 { - dbs.push(format!( - r#"{{"DBInstanceIdentifier": "db-{}", "Engine": "postgres", "EngineVersion": "15.4", "DBInstanceClass": "db.t3.micro", "DBInstanceStatus": "available"}}"#, - i + fn test_snapshot_lambda_list_format() { + let json = r#"{"Functions": [ + {"FunctionName": "api", "Runtime": "python3.12", "MemorySize": 512, "Timeout": 30, "State": "Active"} + ]}"#; + let result = filter_lambda_list(json).unwrap(); + assert_eq!(result.text, "api python3.12 512MB 30s Active"); + } + + #[test] + fn test_snapshot_dynamodb_scan_format() { + let json = r#"{"Items": [{"id": {"N": "1"}, "name": {"S": "Alice"}}], "Count": 1, "ScannedCount": 1}"#; + let result = filter_dynamodb_items(json).unwrap(); + assert_eq!(result.text, "Count: 1/1\n{\"id\":1,\"name\":\"Alice\"}"); + } + + #[test] + fn test_snapshot_security_groups_format() { + let json = r#"{"SecurityGroups": [{ + "GroupName": "web", "GroupId": "sg-1", + "IpPermissions": [{"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}], + "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}] + }]}"#; + let result = filter_security_groups(json).unwrap(); + assert_eq!( + result.text, + "web (sg-1) ingress: tcp/443<-0.0.0.0/0 | egress: all<-0.0.0.0/0" + ); + } + + #[test] + fn test_snapshot_cfn_events_format() { + let json = r#"{"StackEvents": [ + {"Timestamp": "2024-01-15T10:30:00Z", "LogicalResourceId": "Bucket", "ResourceType": "AWS::S3::Bucket", "ResourceStatus": "CREATE_FAILED", "ResourceStatusReason": "Already exists"}, + {"Timestamp": "2024-01-15T10:29:00Z", "LogicalResourceId": "VPC", "ResourceType": "AWS::EC2::VPC", "ResourceStatus": "CREATE_COMPLETE"} + ]}"#; + let result = filter_cfn_events(json).unwrap(); + assert!(result + .text + .starts_with("CloudFormation: 2 events (1 failed, 1 successful)")); + assert!(result.text.contains("--- FAILURES ---")); + assert!(result + .text + .contains("Bucket S3::Bucket CREATE_FAILED REASON: Already exists")); + } + + // === Empty collection edge cases === + + #[test] + fn test_filter_lambda_list_empty() { + let json = r#"{"Functions": []}"#; + let result = filter_lambda_list(json).unwrap(); + assert_eq!(result.text, ""); + } + + #[test] + fn test_filter_iam_roles_empty() { + let json = r#"{"Roles": []}"#; + let result = filter_iam_roles(json).unwrap(); + assert_eq!(result.text, ""); + } + + #[test] + fn test_filter_iam_users_empty() { + let json = r#"{"Users": []}"#; + let result = filter_iam_users(json).unwrap(); + assert_eq!(result.text, ""); + } + + #[test] + fn test_filter_dynamodb_items_empty() { + let json = r#"{"Items": [], "Count": 0, "ScannedCount": 0}"#; + let result = filter_dynamodb_items(json).unwrap(); + assert_eq!(result.text, "Count: 0/0"); + } + + #[test] + fn test_filter_ecs_tasks_empty() { + let json = r#"{"tasks": []}"#; + let result = filter_ecs_tasks(json).unwrap(); + assert_eq!(result.text, ""); + } + + #[test] + fn test_filter_security_groups_empty() { + let json = r#"{"SecurityGroups": []}"#; + let result = filter_security_groups(json).unwrap(); + assert_eq!(result.text, ""); + } + + #[test] + fn test_filter_s3_objects_empty() { + let json = r#"{}"#; + let result = filter_s3_objects(json).unwrap(); + assert_eq!(result.text, ""); + } + + #[test] + fn test_filter_sqs_messages_empty() { + let json = r#"{}"#; + let result = filter_sqs_messages(json).unwrap(); + assert_eq!(result.text, ""); + } + + #[test] + fn test_filter_logs_events_empty() { + let json = r#"{"events": []}"#; + let result = filter_logs_events(json).unwrap(); + assert_eq!(result.text, ""); + } + + #[test] + fn test_filter_ec2_instances_empty() { + let json = r#"{"Reservations": []}"#; + let result = filter_ec2_instances(json).unwrap(); + assert_eq!(result.text, "EC2: 0 instances"); + } + + #[test] + fn test_filter_cfn_events_empty() { + let json = r#"{"StackEvents": []}"#; + let result = filter_cfn_events(json).unwrap(); + assert_eq!( + result.text, + "CloudFormation: 0 events (0 failed, 0 successful)" + ); + } + + #[test] + fn test_filter_cfn_events_failure_count_exceeds_max_items() { + // Verify that failed_count reports the real count, not the capped collection size + let mut events = Vec::new(); + for i in 0..30 { + events.push(format!( + r#"{{"Timestamp": "2024-01-15T10:30:00Z", "LogicalResourceId": "Res{}", "ResourceType": "AWS::Lambda::Function", "ResourceStatus": "CREATE_FAILED", "ResourceStatusReason": "Error {}", "ResourceProperties": "{{}}"}}"#, + i, i )); } - let json = format!(r#"{{"DBInstances": [{}]}}"#, dbs.join(",")); - let result = filter_rds_instances(&json).unwrap(); - assert!(result.contains("... +5 more instances")); + let json = format!(r#"{{"StackEvents": [{}]}}"#, events.join(",")); + let result = filter_cfn_events(&json).unwrap(); + // Should report all 30 failures, not capped at MAX_ITEMS (20) + assert!(result.text.contains("30 failed")); } } diff --git a/src/cmds/cloud/container.rs b/src/cmds/cloud/container.rs index b4e0057ab..3931ebf6e 100644 --- a/src/cmds/cloud/container.rs +++ b/src/cmds/cloud/container.rs @@ -1,9 +1,13 @@ //! Filters Docker and kubectl output into compact summaries. +use crate::core::runner::{self, RunOptions}; +use crate::core::stream::exec_capture; use crate::core::tracking; use crate::core::utils::resolved_command; use anyhow::{Context, Result}; +use serde_json::Value; use std::ffi::OsString; +use std::process::Command; #[derive(Debug, Clone, Copy)] pub enum ContainerCmd { @@ -15,7 +19,7 @@ pub enum ContainerCmd { KubectlLogs, } -pub fn run(cmd: ContainerCmd, args: &[String], verbose: u8) -> Result<()> { +pub fn run(cmd: ContainerCmd, args: &[String], verbose: u8) -> Result { match cmd { ContainerCmd::DockerPs => docker_ps(verbose), ContainerCmd::DockerImages => docker_images(verbose), @@ -26,39 +30,55 @@ pub fn run(cmd: ContainerCmd, args: &[String], verbose: u8) -> Result<()> { } } -fn docker_ps(_verbose: u8) -> Result<()> { +fn run_kubectl_json(cmd: Command, label: &str, filter_fn: F) -> Result +where + F: Fn(&Value) -> String, +{ + runner::run_filtered( + cmd, + "kubectl", + label, + |stdout| match serde_json::from_str::(stdout) { + Ok(json) => filter_fn(&json), + Err(e) => { + eprintln!("[rtk] kubectl: JSON parse failed: {}", e); + stdout.to_string() + } + }, + RunOptions::stdout_only() + .early_exit_on_failure() + .no_trailing_newline(), + ) +} + +fn docker_ps(_verbose: u8) -> Result { let timer = tracking::TimedExecution::start(); - let raw = resolved_command("docker") - .args(["ps"]) - .output() - .map(|o| String::from_utf8_lossy(&o.stdout).to_string()) + let raw = exec_capture(resolved_command("docker").args(["ps"])) + .map(|r| r.stdout) .unwrap_or_default(); - let output = resolved_command("docker") - .args([ - "ps", - "--format", - "{{.ID}}\t{{.Names}}\t{{.Status}}\t{{.Image}}\t{{.Ports}}", - ]) - .output() - .context("Failed to run docker ps")?; - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr); - eprint!("{}", stderr); + let result = exec_capture(resolved_command("docker").args([ + "ps", + "--format", + "{{.ID}}\t{{.Names}}\t{{.Status}}\t{{.Image}}\t{{.Ports}}", + ])) + .context("Failed to run docker ps")?; + + if !result.success() { + eprint!("{}", result.stderr); timer.track("docker ps", "rtk docker ps", &raw, &raw); - std::process::exit(output.status.code().unwrap_or(1)); + return Ok(result.exit_code); } - let stdout = String::from_utf8_lossy(&output.stdout); + let stdout = result.stdout; let mut rtk = String::new(); if stdout.trim().is_empty() { rtk.push_str("[docker] 0 containers"); println!("{}", rtk); timer.track("docker ps", "rtk docker ps", &raw, &rtk); - return Ok(()); + return Ok(0); } let count = stdout.lines().count(); @@ -92,31 +112,30 @@ fn docker_ps(_verbose: u8) -> Result<()> { print!("{}", rtk); timer.track("docker ps", "rtk docker ps", &raw, &rtk); - Ok(()) + Ok(0) } -fn docker_images(_verbose: u8) -> Result<()> { +fn docker_images(_verbose: u8) -> Result { let timer = tracking::TimedExecution::start(); - let raw = resolved_command("docker") - .args(["images"]) - .output() - .map(|o| String::from_utf8_lossy(&o.stdout).to_string()) + let raw = exec_capture(resolved_command("docker").args(["images"])) + .map(|r| r.stdout) .unwrap_or_default(); - let output = resolved_command("docker") - .args(["images", "--format", "{{.Repository}}:{{.Tag}}\t{{.Size}}"]) - .output() - .context("Failed to run docker images")?; + let result = exec_capture(resolved_command("docker").args([ + "images", + "--format", + "{{.Repository}}:{{.Tag}}\t{{.Size}}", + ])) + .context("Failed to run docker images")?; - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr); - eprint!("{}", stderr); + if !result.success() { + eprint!("{}", result.stderr); timer.track("docker images", "rtk docker images", &raw, &raw); - std::process::exit(output.status.code().unwrap_or(1)); + return Ok(result.exit_code); } - let stdout = String::from_utf8_lossy(&output.stdout); + let stdout = result.stdout; let lines: Vec<&str> = stdout.lines().collect(); let mut rtk = String::new(); @@ -124,7 +143,7 @@ fn docker_images(_verbose: u8) -> Result<()> { rtk.push_str("[docker] 0 images"); println!("{}", rtk); timer.track("docker images", "rtk docker images", &raw, &rtk); - return Ok(()); + return Ok(0); } let mut total_size_mb: f64 = 0.0; @@ -173,89 +192,47 @@ fn docker_images(_verbose: u8) -> Result<()> { print!("{}", rtk); timer.track("docker images", "rtk docker images", &raw, &rtk); - Ok(()) + Ok(0) } -fn docker_logs(args: &[String], _verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - +fn docker_logs(args: &[String], _verbose: u8) -> Result { let container = args.first().map(|s| s.as_str()).unwrap_or(""); if container.is_empty() { println!("Usage: rtk docker logs "); - return Ok(()); - } - - let output = resolved_command("docker") - .args(["logs", "--tail", "100", container]) - .output() - .context("Failed to run docker logs")?; - - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let raw = format!("{}\n{}", stdout, stderr); - - if !output.status.success() { - if !stderr.trim().is_empty() { - eprint!("{}", stderr); - } - timer.track( - &format!("docker logs {}", container), - "rtk docker logs", - &raw, - &raw, - ); - std::process::exit(output.status.code().unwrap_or(1)); + return Ok(0); } - let analyzed = crate::log_cmd::run_stdin_str(&raw); - let rtk = format!("[docker] Logs for {}:\n{}", container, analyzed); - println!("{}", rtk); - timer.track( - &format!("docker logs {}", container), - "rtk docker logs", - &raw, - &rtk, - ); - Ok(()) + let mut cmd = resolved_command("docker"); + cmd.args(["logs", "--tail", "100", container]); + + let label = format!("logs {}", container); + runner::run_filtered( + cmd, + "docker", + &label, + |raw| { + format!( + "[docker] Logs for {}:\n{}", + container, + crate::log_cmd::run_stdin_str(raw) + ) + }, + RunOptions::default().early_exit_on_failure(), + ) } -fn kubectl_pods(args: &[String], _verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - +fn kubectl_pods(args: &[String], _verbose: u8) -> Result { let mut cmd = resolved_command("kubectl"); cmd.args(["get", "pods", "-o", "json"]); for arg in args { cmd.arg(arg); } + run_kubectl_json(cmd, "get pods", format_kubectl_pods) +} - let output = cmd.output().context("Failed to run kubectl get pods")?; - let raw = String::from_utf8_lossy(&output.stdout).to_string(); - let mut rtk = String::new(); - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr); - if !stderr.trim().is_empty() { - eprint!("{}", stderr); - } - timer.track("kubectl get pods", "rtk kubectl pods", &raw, &raw); - std::process::exit(output.status.code().unwrap_or(1)); - } - - let json: serde_json::Value = match serde_json::from_str(&raw) { - Ok(v) => v, - Err(_) => { - rtk.push_str("No pods found"); - println!("{}", rtk); - timer.track("kubectl get pods", "rtk kubectl pods", &raw, &rtk); - return Ok(()); - } - }; - +fn format_kubectl_pods(json: &Value) -> String { let Some(pods) = json["items"].as_array().filter(|a| !a.is_empty()) else { - rtk.push_str("No pods found"); - println!("{}", rtk); - timer.track("kubectl get pods", "rtk kubectl pods", &raw, &rtk); - return Ok(()); + return "No pods found\n".to_string(); }; let (mut running, mut pending, mut failed, mut restarts_total) = (0, 0, 0, 0i64); let mut issues: Vec = Vec::new(); @@ -310,61 +287,33 @@ fn kubectl_pods(args: &[String], _verbose: u8) -> Result<()> { parts.push(format!("{} restarts", restarts_total)); } - rtk.push_str(&format!("{} pods: {}\n", pods.len(), parts.join(", "))); + let mut out = format!("{} pods: {}\n", pods.len(), parts.join(", ")); if !issues.is_empty() { - rtk.push_str("[warn] Issues:\n"); + out.push_str("[warn] Issues:\n"); for issue in issues.iter().take(10) { - rtk.push_str(&format!(" {}\n", issue)); + out.push_str(&format!(" {}\n", issue)); } if issues.len() > 10 { - rtk.push_str(&format!(" ... +{} more", issues.len() - 10)); + out.push_str(&format!(" ... +{} more", issues.len() - 10)); } } - - print!("{}", rtk); - timer.track("kubectl get pods", "rtk kubectl pods", &raw, &rtk); - Ok(()) + out } -fn kubectl_services(args: &[String], _verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - +fn kubectl_services(args: &[String], _verbose: u8) -> Result { let mut cmd = resolved_command("kubectl"); cmd.args(["get", "services", "-o", "json"]); for arg in args { cmd.arg(arg); } + run_kubectl_json(cmd, "get services", format_kubectl_services) +} - let output = cmd.output().context("Failed to run kubectl get services")?; - let raw = String::from_utf8_lossy(&output.stdout).to_string(); - let mut rtk = String::new(); - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr); - if !stderr.trim().is_empty() { - eprint!("{}", stderr); - } - timer.track("kubectl get svc", "rtk kubectl svc", &raw, &raw); - std::process::exit(output.status.code().unwrap_or(1)); - } - - let json: serde_json::Value = match serde_json::from_str(&raw) { - Ok(v) => v, - Err(_) => { - rtk.push_str("No services found"); - println!("{}", rtk); - timer.track("kubectl get svc", "rtk kubectl svc", &raw, &rtk); - return Ok(()); - } - }; - +fn format_kubectl_services(json: &Value) -> String { let Some(services) = json["items"].as_array().filter(|a| !a.is_empty()) else { - rtk.push_str("No services found"); - println!("{}", rtk); - timer.track("kubectl get svc", "rtk kubectl svc", &raw, &rtk); - return Ok(()); + return "No services found\n".to_string(); }; - rtk.push_str(&format!("{} services:\n", services.len())); + let mut out = format!("{} services:\n", services.len()); for svc in services.iter().take(15) { let ns = svc["metadata"]["namespace"].as_str().unwrap_or("-"); @@ -389,7 +338,7 @@ fn kubectl_services(args: &[String], _verbose: u8) -> Result<()> { .collect() }) .unwrap_or_default(); - rtk.push_str(&format!( + out.push_str(&format!( " {}/{} {} [{}]\n", ns, name, @@ -398,21 +347,16 @@ fn kubectl_services(args: &[String], _verbose: u8) -> Result<()> { )); } if services.len() > 15 { - rtk.push_str(&format!(" ... +{} more", services.len() - 15)); + out.push_str(&format!(" ... +{} more", services.len() - 15)); } - - print!("{}", rtk); - timer.track("kubectl get svc", "rtk kubectl svc", &raw, &rtk); - Ok(()) + out } -fn kubectl_logs(args: &[String], _verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - +fn kubectl_logs(args: &[String], _verbose: u8) -> Result { let pod = args.first().map(|s| s.as_str()).unwrap_or(""); if pod.is_empty() { println!("Usage: rtk kubectl logs "); - return Ok(()); + return Ok(0); } let mut cmd = resolved_command("kubectl"); @@ -421,33 +365,20 @@ fn kubectl_logs(args: &[String], _verbose: u8) -> Result<()> { cmd.arg(arg); } - let output = cmd.output().context("Failed to run kubectl logs")?; - let raw = String::from_utf8_lossy(&output.stdout).to_string(); - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr); - if !stderr.trim().is_empty() { - eprint!("{}", stderr); - } - timer.track( - &format!("kubectl logs {}", pod), - "rtk kubectl logs", - &raw, - &raw, - ); - std::process::exit(output.status.code().unwrap_or(1)); - } - - let analyzed = crate::log_cmd::run_stdin_str(&raw); - let rtk = format!("Logs for {}:\n{}", pod, analyzed); - println!("{}", rtk); - timer.track( - &format!("kubectl logs {}", pod), - "rtk kubectl logs", - &raw, - &rtk, - ); - Ok(()) + let label = format!("logs {}", pod); + runner::run_filtered( + cmd, + "kubectl", + &label, + |stdout| { + format!( + "Logs for {}:\n{}", + pod, + crate::log_cmd::run_stdin_str(stdout) + ) + }, + RunOptions::stdout_only().early_exit_on_failure(), + ) } /// Format `docker compose ps --format` output into compact form. @@ -587,64 +518,38 @@ fn compact_ports(ports: &str) -> String { } } -/// Runs an unsupported docker subcommand by passing it through directly -pub fn run_docker_passthrough(args: &[OsString], verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - - if verbose > 0 { - eprintln!("docker passthrough: {:?}", args); - } - let status = resolved_command("docker") - .args(args) - .status() - .context("Failed to run docker")?; - - let args_str = tracking::args_display(args); - timer.track_passthrough( - &format!("docker {}", args_str), - &format!("rtk docker {} (passthrough)", args_str), - ); - - if !status.success() { - std::process::exit(status.code().unwrap_or(1)); - } - Ok(()) +pub fn run_docker_passthrough(args: &[OsString], verbose: u8) -> Result { + crate::core::runner::run_passthrough("docker", args, verbose) } /// Run `docker compose ps` with compact output -pub fn run_compose_ps(verbose: u8) -> Result<()> { +pub fn run_compose_ps(verbose: u8) -> Result { let timer = tracking::TimedExecution::start(); // Raw output for token tracking - let raw_output = resolved_command("docker") - .args(["compose", "ps"]) - .output() + let raw_result = exec_capture(resolved_command("docker").args(["compose", "ps"])) .context("Failed to run docker compose ps")?; - if !raw_output.status.success() { - let stderr = String::from_utf8_lossy(&raw_output.stderr); - eprintln!("{}", stderr); - std::process::exit(raw_output.status.code().unwrap_or(1)); + if !raw_result.success() { + eprintln!("{}", raw_result.stderr); + return Ok(raw_result.exit_code); } - let raw = String::from_utf8_lossy(&raw_output.stdout).to_string(); + let raw = raw_result.stdout; // Structured output for parsing (same pattern as docker_ps) - let output = resolved_command("docker") - .args([ - "compose", - "ps", - "--format", - "{{.Name}}\t{{.Image}}\t{{.Status}}\t{{.Ports}}", - ]) - .output() - .context("Failed to run docker compose ps --format")?; - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr); - eprintln!("{}", stderr); - std::process::exit(output.status.code().unwrap_or(1)); - } - let structured = String::from_utf8_lossy(&output.stdout).to_string(); + let result = exec_capture(resolved_command("docker").args([ + "compose", + "ps", + "--format", + "{{.Name}}\t{{.Image}}\t{{.Status}}\t{{.Ports}}", + ])) + .context("Failed to run docker compose ps --format")?; + + if !result.success() { + eprintln!("{}", result.stderr); + return Ok(result.exit_code); + } + let structured = result.stdout; if verbose > 0 { eprintln!("raw docker compose ps:\n{}", raw); @@ -653,132 +558,61 @@ pub fn run_compose_ps(verbose: u8) -> Result<()> { let rtk = format_compose_ps(&structured); println!("{}", rtk); timer.track("docker compose ps", "rtk docker compose ps", &raw, &rtk); - Ok(()) + Ok(0) } -/// Run `docker compose logs` with deduplication -pub fn run_compose_logs(service: Option<&str>, verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - +pub fn run_compose_logs(service: Option<&str>, verbose: u8) -> Result { let mut cmd = resolved_command("docker"); cmd.args(["compose", "logs", "--tail", "100"]); if let Some(svc) = service { cmd.arg(svc); } - let output = cmd.output().context("Failed to run docker compose logs")?; - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr); - eprintln!("{}", stderr); - std::process::exit(output.status.code().unwrap_or(1)); - } - - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let raw = format!("{}\n{}", stdout, stderr); - - if verbose > 0 { - eprintln!("raw docker compose logs:\n{}", raw); - } - - let rtk = format_compose_logs(&raw); - println!("{}", rtk); let svc_label = service.unwrap_or("all"); - timer.track( - &format!("docker compose logs {}", svc_label), - "rtk docker compose logs", - &raw, - &rtk, - ); - Ok(()) + runner::run_filtered( + cmd, + "docker", + &format!("compose logs {}", svc_label), + |raw| { + if verbose > 0 { + eprintln!("raw docker compose logs:\n{}", raw); + } + format_compose_logs(raw) + }, + RunOptions::default().early_exit_on_failure(), + ) } -/// Run `docker compose build` with summary output -pub fn run_compose_build(service: Option<&str>, verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - +pub fn run_compose_build(service: Option<&str>, verbose: u8) -> Result { let mut cmd = resolved_command("docker"); cmd.args(["compose", "build"]); if let Some(svc) = service { cmd.arg(svc); } - let output = cmd.output().context("Failed to run docker compose build")?; - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr); - eprintln!("{}", stderr); - std::process::exit(output.status.code().unwrap_or(1)); - } - - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let raw = format!("{}\n{}", stdout, stderr); - - if verbose > 0 { - eprintln!("raw docker compose build:\n{}", raw); - } - - let rtk = format_compose_build(&raw); - println!("{}", rtk); let svc_label = service.unwrap_or("all"); - timer.track( - &format!("docker compose build {}", svc_label), - "rtk docker compose build", - &raw, - &rtk, - ); - Ok(()) + runner::run_filtered( + cmd, + "docker", + &format!("compose build {}", svc_label), + |raw| { + if verbose > 0 { + eprintln!("raw docker compose build:\n{}", raw); + } + format_compose_build(raw) + }, + RunOptions::default().early_exit_on_failure(), + ) } -/// Runs an unsupported docker compose subcommand by passing it through directly -pub fn run_compose_passthrough(args: &[OsString], verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - - if verbose > 0 { - eprintln!("docker compose passthrough: {:?}", args); - } - let status = resolved_command("docker") - .arg("compose") - .args(args) - .status() - .context("Failed to run docker compose")?; - - let args_str = tracking::args_display(args); - timer.track_passthrough( - &format!("docker compose {}", args_str), - &format!("rtk docker compose {} (passthrough)", args_str), - ); - - if !status.success() { - std::process::exit(status.code().unwrap_or(1)); - } - Ok(()) +pub fn run_compose_passthrough(args: &[OsString], verbose: u8) -> Result { + let mut combined = vec![OsString::from("compose")]; + combined.extend_from_slice(args); + crate::core::runner::run_passthrough("docker", &combined, verbose) } -/// Runs an unsupported kubectl subcommand by passing it through directly -pub fn run_kubectl_passthrough(args: &[OsString], verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - - if verbose > 0 { - eprintln!("kubectl passthrough: {:?}", args); - } - let status = resolved_command("kubectl") - .args(args) - .status() - .context("Failed to run kubectl")?; - - let args_str = tracking::args_display(args); - timer.track_passthrough( - &format!("kubectl {}", args_str), - &format!("rtk kubectl {} (passthrough)", args_str), - ); - - if !status.success() { - std::process::exit(status.code().unwrap_or(1)); - } - Ok(()) +pub fn run_kubectl_passthrough(args: &[OsString], verbose: u8) -> Result { + crate::core::runner::run_passthrough("kubectl", args, verbose) } #[cfg(test)] diff --git a/src/cmds/cloud/curl_cmd.rs b/src/cmds/cloud/curl_cmd.rs index 7141ad72d..a24506d8c 100644 --- a/src/cmds/cloud/curl_cmd.rs +++ b/src/cmds/cloud/curl_cmd.rs @@ -1,11 +1,15 @@ -//! Runs curl and auto-compresses JSON responses. +//! Runs curl and applies a simple truncation with tee hint if the output is too long. +use crate::core::tee::force_tee_hint; use crate::core::tracking; -use crate::core::utils::{resolved_command, truncate}; -use crate::json_cmd; +use crate::core::{stream::exec_capture, utils::resolved_command}; use anyhow::{Context, Result}; -pub fn run(args: &[String], verbose: u8) -> Result<()> { +const MAX_RESPONSE_SIZE: usize = 500; + +/// Not using run_filtered: on failure, curl can return HTML error pages (404, 500) +/// that the JSON schema filter would mangle. The early exit skips filtering entirely. +pub fn run(args: &[String], verbose: u8) -> Result { let timer = tracking::TimedExecution::start(); let mut cmd = resolved_command("curl"); cmd.arg("-s"); // Silent mode (no progress bar) @@ -18,70 +22,61 @@ pub fn run(args: &[String], verbose: u8) -> Result<()> { eprintln!("Running: curl -s {}", args.join(" ")); } - let output = cmd.output().context("Failed to run curl")?; - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); + let result = exec_capture(&mut cmd).context("Failed to run curl")?; - if !output.status.success() { - let msg = if stderr.trim().is_empty() { - stdout.trim().to_string() + // Early exit: don't feed HTTP error bodies (HTML 404 etc.) through JSON schema filter + if !result.success() { + let msg = if result.stderr.trim().is_empty() { + result.stdout.trim().to_string() } else { - stderr.trim().to_string() + result.stderr.trim().to_string() }; eprintln!("FAILED: curl {}", msg); - std::process::exit(output.status.code().unwrap_or(1)); + return Ok(result.exit_code); } - let raw = stdout.to_string(); + let raw = result.stdout.clone(); + + let result = filter_curl_output(&result.stdout); - // Auto-detect JSON and pipe through filter - let filtered = filter_curl_output(&stdout); - println!("{}", filtered); + println!("{}", result.content); + if let Some(hint) = &result.tee_hint { + println!("{}", hint); + } timer.track( &format!("curl {}", args.join(" ")), &format!("rtk curl {}", args.join(" ")), &raw, - &filtered, + &result.content, ); - Ok(()) + Ok(0) } -fn filter_curl_output(output: &str) -> String { - let trimmed = output.trim(); - - // Try JSON detection: starts with { or [ - if (trimmed.starts_with('{') || trimmed.starts_with('[')) - && (trimmed.ends_with('}') || trimmed.ends_with(']')) - { - if let Ok(schema) = json_cmd::filter_json_string(trimmed, 5) { - // Only use schema if it's actually shorter than the original (#297) - if schema.len() <= trimmed.len() { - return schema; - } +fn filter_curl_output(raw: &str) -> FilterResult { + let trimmed = raw.trim(); + let tee_hint = force_tee_hint(raw, "curl"); + + // If the output is too long and we have a tee hint, truncate the output. + let content = if trimmed.len() >= MAX_RESPONSE_SIZE && tee_hint.is_some() { + let mut end = MAX_RESPONSE_SIZE; + // Ensure we don't cut in the middle of a UTF-8 character. + // .len() counts bytes, not chars. + while !trimmed.is_char_boundary(end) { + end -= 1; } - } + format!("{}... ({} bytes total)", &trimmed[..end], trimmed.len()) + } else { + trimmed.to_string() + }; - // Not JSON: truncate long output - let lines: Vec<&str> = trimmed.lines().collect(); - if lines.len() > 30 { - let mut result: Vec<&str> = lines[..30].to_vec(); - result.push(""); - let msg = format!( - "... ({} more lines, {} bytes total)", - lines.len() - 30, - trimmed.len() - ); - return format!("{}\n{}", result.join("\n"), msg); - } + FilterResult { content, tee_hint } +} - // Short output: return as-is but truncate long lines - lines - .iter() - .map(|l| truncate(l, 200)) - .collect::>() - .join("\n") +struct FilterResult { + content: String, + tee_hint: Option, } #[cfg(test)] @@ -89,47 +84,42 @@ mod tests { use super::*; #[test] - fn test_filter_curl_json() { - // Large JSON where schema is shorter than original — schema should be returned - let output = r#"{"name": "a very long user name here", "count": 42, "items": [1, 2, 3], "description": "a very long description that takes up many characters in the original JSON payload", "status": "active", "url": "https://example.com/api/v1/users/123"}"#; + fn test_filter_curl_json_small_no_tee_hint() { + let output = r#"{"r2Ready":true,"status":"ok"}"#; let result = filter_curl_output(output); - assert!(result.contains("name")); - assert!(result.contains("string")); - assert!(result.contains("int")); + assert_eq!(result.content, output); + assert!(result.tee_hint.is_none()); } #[test] - fn test_filter_curl_json_array() { - let output = r#"[{"id": 1}, {"id": 2}]"#; + fn test_filter_curl_non_json() { + let output = "Hello, World!\nThis is plain text."; let result = filter_curl_output(output); - assert!(result.contains("id")); + assert_eq!(result.content, output); } #[test] - fn test_filter_curl_non_json() { - let output = "Hello, World!\nThis is plain text."; - let result = filter_curl_output(output); - assert!(result.contains("Hello, World!")); - assert!(result.contains("plain text")); + fn test_filter_curl_long_output_truncated() { + let long: String = "x".repeat(1000); + let result = filter_curl_output(&long); + assert!(result.content.starts_with('x')); + assert!(result.content.contains("bytes total")); + assert!(result.content.contains("1000")); + assert!(result.content.len() < 600); } #[test] - fn test_filter_curl_json_small_returns_original() { - // Small JSON where schema would be larger than original (issue #297) - let output = r#"{"r2Ready":true,"status":"ok"}"#; - let result = filter_curl_output(output); - // Schema would be "{\n r2Ready: bool,\n status: string\n}" which is longer - // Should return the original JSON unchanged - assert_eq!(result.trim(), output.trim()); + fn test_filter_curl_multibyte_boundary() { + let content = "a".repeat(499) + "é"; + let result = filter_curl_output(&content); + assert!(result.content.contains("bytes total")); + assert!(result.content.len() < 600); } #[test] - fn test_filter_curl_long_output() { - let lines: Vec = (0..50).map(|i| format!("Line {}", i)).collect(); - let output = lines.join("\n"); - let result = filter_curl_output(&output); - assert!(result.contains("Line 0")); - assert!(result.contains("Line 29")); - assert!(result.contains("more lines")); + fn test_filter_curl_exact_500_bytes() { + let content = "a".repeat(500); + let result = filter_curl_output(&content); + assert!(result.content.contains("bytes total")); } } diff --git a/src/cmds/cloud/mod.rs b/src/cmds/cloud/mod.rs index 8917a6c27..28af53ca7 100644 --- a/src/cmds/cloud/mod.rs +++ b/src/cmds/cloud/mod.rs @@ -1,7 +1 @@ -//! Cloud and infrastructure tool filters. - -pub mod aws_cmd; -pub mod container; -pub mod curl_cmd; -pub mod psql_cmd; -pub mod wget_cmd; +automod::dir!(pub "src/cmds/cloud"); diff --git a/src/cmds/cloud/psql_cmd.rs b/src/cmds/cloud/psql_cmd.rs index 9ec243bed..4b4e5f2c2 100644 --- a/src/cmds/cloud/psql_cmd.rs +++ b/src/cmds/cloud/psql_cmd.rs @@ -3,9 +3,9 @@ //! Detects table and expanded display formats, strips borders/padding, //! and produces compact tab-separated or key=value output. -use crate::core::tracking; +use crate::core::runner::{self, RunOptions}; use crate::core::utils::resolved_command; -use anyhow::{Context, Result}; +use anyhow::Result; use lazy_static::lazy_static; use regex::Regex; @@ -19,9 +19,11 @@ lazy_static! { static ref RECORD_HEADER: Regex = Regex::new(r"^-\[ RECORD (\d+) \]-").unwrap(); } -pub fn run(args: &[String], verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - +// Edge cases vs previous manual implementation: +// - On failure: stderr is no longer eprinted on the success path (only on failure via early_exit) +// - On success: tracking raw includes stderr (previously stdout-only, but stderr is empty on success) +// - Tee hint uses merged stdout+stderr as raw (was stdout-only) +pub fn run(args: &[String], verbose: u8) -> Result { let mut cmd = resolved_command("psql"); for arg in args { cmd.arg(arg); @@ -31,38 +33,15 @@ pub fn run(args: &[String], verbose: u8) -> Result<()> { eprintln!("Running: psql {}", args.join(" ")); } - let output = cmd - .output() - .context("Failed to run psql (is PostgreSQL client installed?)")?; - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - - let exit_code = output.status.code().unwrap_or(1); - - if !stderr.is_empty() { - eprint!("{}", stderr); - } - - if exit_code != 0 { - std::process::exit(exit_code); - } - - let filtered = filter_psql_output(&stdout); - - if let Some(hint) = crate::core::tee::tee_and_hint(&stdout, "psql", exit_code) { - println!("{}\n{}", filtered, hint); - } else { - println!("{}", filtered); - } - - timer.track( - &format!("psql {}", args.join(" ")), - &format!("rtk psql {}", args.join(" ")), - &stdout, - &filtered, - ); - - Ok(()) + runner::run_filtered( + cmd, + "psql", + &args.join(" "), + filter_psql_output, + RunOptions::stdout_only() + .tee("psql") + .early_exit_on_failure(), + ) } fn filter_psql_output(output: &str) -> String { diff --git a/src/cmds/cloud/wget_cmd.rs b/src/cmds/cloud/wget_cmd.rs index 32996ac34..4faed8081 100644 --- a/src/cmds/cloud/wget_cmd.rs +++ b/src/cmds/cloud/wget_cmd.rs @@ -1,9 +1,10 @@ +use crate::core::stream::exec_capture; use crate::core::tracking; use crate::core::utils::resolved_command; use anyhow::{Context, Result}; /// Compact wget - strips progress bars, shows only result -pub fn run(url: &str, args: &[String], verbose: u8) -> Result<()> { +pub fn run(url: &str, args: &[String], verbose: u8) -> Result { let timer = tracking::TimedExecution::start(); if verbose > 0 { @@ -19,18 +20,14 @@ pub fn run(url: &str, args: &[String], verbose: u8) -> Result<()> { } cmd_args.push(url); - let output = resolved_command("wget") - .args(&cmd_args) - .output() - .context("Failed to run wget")?; + let mut cmd = resolved_command("wget"); + cmd.args(&cmd_args); + let result = exec_capture(&mut cmd).context("Failed to run wget")?; - let stderr = String::from_utf8_lossy(&output.stderr); - let stdout = String::from_utf8_lossy(&output.stdout); + let raw_output = format!("{}\n{}", result.stderr, result.stdout); - let raw_output = format!("{}\n{}", stderr, stdout); - - if output.status.success() { - let filename = extract_filename_from_output(&stderr, url, args); + if result.success() { + let filename = extract_filename_from_output(&result.stderr, url, args); let size = get_file_size(&filename); let msg = format!( "{} ok | {} | {}", @@ -41,18 +38,18 @@ pub fn run(url: &str, args: &[String], verbose: u8) -> Result<()> { println!("{}", msg); timer.track(&format!("wget {}", url), "rtk wget", &raw_output, &msg); } else { - let error = parse_error(&stderr, &stdout); + let error = parse_error(&result.stderr, &result.stdout); let msg = format!("{} FAILED: {}", compact_url(url), error); println!("{}", msg); timer.track(&format!("wget {}", url), "rtk wget", &raw_output, &msg); - std::process::exit(output.status.code().unwrap_or(1)); + return Ok(result.exit_code); } - Ok(()) + Ok(0) } /// Run wget and output to stdout (for piping) -pub fn run_stdout(url: &str, args: &[String], verbose: u8) -> Result<()> { +pub fn run_stdout(url: &str, args: &[String], verbose: u8) -> Result { let timer = tracking::TimedExecution::start(); if verbose > 0 { @@ -65,16 +62,13 @@ pub fn run_stdout(url: &str, args: &[String], verbose: u8) -> Result<()> { } cmd_args.push(url); - let output = resolved_command("wget") - .args(&cmd_args) - .output() - .context("Failed to run wget")?; + let mut cmd = resolved_command("wget"); + cmd.args(&cmd_args); + let result = exec_capture(&mut cmd).context("Failed to run wget")?; - if output.status.success() { - let content = String::from_utf8_lossy(&output.stdout); - let lines: Vec<&str> = content.lines().collect(); + if result.success() { + let lines: Vec<&str> = result.stdout.lines().collect(); let total = lines.len(); - let raw_output = content.to_string(); let mut rtk_output = String::new(); if total > 20 { @@ -82,7 +76,7 @@ pub fn run_stdout(url: &str, args: &[String], verbose: u8) -> Result<()> { "{} ok | {} lines | {}\n", compact_url(url), total, - format_size(output.stdout.len() as u64) + format_size(result.stdout.len() as u64) )); rtk_output.push_str("--- first 10 lines ---\n"); for line in lines.iter().take(10) { @@ -99,19 +93,18 @@ pub fn run_stdout(url: &str, args: &[String], verbose: u8) -> Result<()> { timer.track( &format!("wget -O - {}", url), "rtk wget -o", - &raw_output, + &result.stdout, &rtk_output, ); } else { - let stderr = String::from_utf8_lossy(&output.stderr); - let error = parse_error(&stderr, ""); + let error = parse_error(&result.stderr, ""); let msg = format!("{} FAILED: {}", compact_url(url), error); println!("{}", msg); - timer.track(&format!("wget -O - {}", url), "rtk wget -o", &stderr, &msg); - std::process::exit(output.status.code().unwrap_or(1)); + timer.track(&format!("wget -O - {}", url), "rtk wget -o", &result.stderr, &msg); + return Ok(result.exit_code); } - Ok(()) + Ok(0) } fn extract_filename_from_output(stderr: &str, url: &str, args: &[String]) -> String { @@ -261,3 +254,114 @@ fn truncate_line(line: &str, max: usize) -> String { format!("{}...", t) } } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn test_compact_url_strips_protocol() { + assert_eq!(compact_url("https://example.com/file.zip"), "example.com/file.zip"); + assert_eq!(compact_url("http://example.com/file.zip"), "example.com/file.zip"); + } + + #[test] + fn test_compact_url_truncates_long_url() { + let long = "https://example.com/very/long/path/that/exceeds/fifty/characters/file.zip"; + let result = compact_url(long); + assert!(result.contains("..."), "Long URL should be truncated with ..."); + assert!(result.len() < long.len()); + } + + #[test] + fn test_compact_url_short_unchanged() { + let short = "https://x.com/f"; + assert_eq!(compact_url(short), "x.com/f"); + } + + #[test] + fn test_format_size_zero() { + assert_eq!(format_size(0), "?"); + } + + #[test] + fn test_format_size_bytes() { + assert_eq!(format_size(512), "512B"); + } + + #[test] + fn test_format_size_kilobytes() { + let result = format_size(2048); + assert!(result.ends_with("KB"), "Expected KB, got {}", result); + } + + #[test] + fn test_format_size_megabytes() { + let result = format_size(2 * 1024 * 1024); + assert!(result.ends_with("MB"), "Expected MB, got {}", result); + } + + #[test] + fn test_parse_error_404() { + assert_eq!(parse_error("HTTP request failed: 404", ""), "404 Not Found"); + } + + #[test] + fn test_parse_error_dns() { + assert_eq!( + parse_error("unable to resolve host example.com", ""), + "DNS lookup failed" + ); + } + + #[test] + fn test_parse_error_ssl() { + assert_eq!( + parse_error("SSL certificate verification failed", ""), + "SSL/TLS error" + ); + } + + #[test] + fn test_parse_error_unknown() { + assert_eq!(parse_error("", ""), "Unknown error"); + } + + #[test] + fn test_truncate_line_short() { + assert_eq!(truncate_line("hello", 10), "hello"); + } + + #[test] + fn test_truncate_line_exact() { + assert_eq!(truncate_line("hello", 5), "hello"); + } + + #[test] + fn test_truncate_line_long() { + let result = truncate_line("hello world this is long", 10); + assert!(result.ends_with("...")); + assert!(result.len() <= 10); + } + + #[test] + fn test_extract_filename_from_output_flag() { + let args = vec!["-O".to_string(), "myfile.zip".to_string()]; + assert_eq!( + extract_filename_from_output("", "https://example.com/x", &args), + "myfile.zip" + ); + } + + #[test] + fn test_extract_filename_from_url_fallback() { + let result = extract_filename_from_output("", "https://example.com/file.tar.gz", &[]); + assert_eq!(result, "file.tar.gz"); + } + + #[test] + fn test_extract_filename_empty_url_fallback() { + let result = extract_filename_from_output("", "https://example.com/", &[]); + assert_eq!(result, "index.html"); + } +} diff --git a/src/cmds/dotnet/README.md b/src/cmds/dotnet/README.md index 707aaab58..dc6b0cfa2 100644 --- a/src/cmds/dotnet/README.md +++ b/src/cmds/dotnet/README.md @@ -1,6 +1,6 @@ # .NET Ecosystem -> Part of [`src/cmds/`](../README.md) — see also [docs/TECHNICAL.md](../../../docs/TECHNICAL.md) +> Part of [`src/cmds/`](../README.md) — see also [docs/contributing/TECHNICAL.md](../../../docs/contributing/TECHNICAL.md) ## Specifics diff --git a/src/cmds/dotnet/dotnet_cmd.rs b/src/cmds/dotnet/dotnet_cmd.rs index 5f0508830..a60935711 100644 --- a/src/cmds/dotnet/dotnet_cmd.rs +++ b/src/cmds/dotnet/dotnet_cmd.rs @@ -1,6 +1,7 @@ //! Filters dotnet CLI output — build, test, and format results. use crate::binlog; +use crate::core::stream::exec_capture; use crate::core::tracking; use crate::core::utils::{resolved_command, truncate}; use crate::dotnet_format_report; @@ -18,19 +19,19 @@ const DOTNET_CLI_UI_LANGUAGE: &str = "DOTNET_CLI_UI_LANGUAGE"; const DOTNET_CLI_UI_LANGUAGE_VALUE: &str = "en-US"; static TEMP_PATH_COUNTER: AtomicU64 = AtomicU64::new(0); -pub fn run_build(args: &[String], verbose: u8) -> Result<()> { +pub fn run_build(args: &[String], verbose: u8) -> Result { run_dotnet_with_binlog("build", args, verbose) } -pub fn run_test(args: &[String], verbose: u8) -> Result<()> { +pub fn run_test(args: &[String], verbose: u8) -> Result { run_dotnet_with_binlog("test", args, verbose) } -pub fn run_restore(args: &[String], verbose: u8) -> Result<()> { +pub fn run_restore(args: &[String], verbose: u8) -> Result { run_dotnet_with_binlog("restore", args, verbose) } -pub fn run_format(args: &[String], verbose: u8) -> Result<()> { +pub fn run_format(args: &[String], verbose: u8) -> Result { let timer = tracking::TimedExecution::start(); let (report_path, cleanup_report_path) = resolve_format_report_path(args); let mut cmd = resolved_command("dotnet"); @@ -46,10 +47,8 @@ pub fn run_format(args: &[String], verbose: u8) -> Result<()> { } let command_started_at = SystemTime::now(); - let output = cmd.output().context("Failed to run dotnet format")?; - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let raw = format!("{}\n{}", stdout, stderr); + let result = exec_capture(&mut cmd).context("Failed to run dotnet format")?; + let raw = format!("{}\n{}", result.stdout, result.stderr); let check_mode = !has_write_mode_override(args); let filtered = @@ -69,14 +68,10 @@ pub fn run_format(args: &[String], verbose: u8) -> Result<()> { } } - if !output.status.success() { - std::process::exit(output.status.code().unwrap_or(1)); - } - - Ok(()) + Ok(result.exit_code) } -pub fn run_passthrough(args: &[OsString], verbose: u8) -> Result<()> { +pub fn run_passthrough(args: &[OsString], verbose: u8) -> Result { if args.is_empty() { anyhow::bail!("dotnet: no subcommand specified"); } @@ -95,16 +90,13 @@ pub fn run_passthrough(args: &[OsString], verbose: u8) -> Result<()> { eprintln!("Running: dotnet {} ...", subcommand); } - let output = cmd - .output() + let result = exec_capture(&mut cmd) .with_context(|| format!("Failed to run dotnet {}", subcommand))?; - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let raw = format!("{}\n{}", stdout, stderr); + let raw = format!("{}\n{}", result.stdout, result.stderr); - print!("{}", stdout); - eprint!("{}", stderr); + print!("{}", result.stdout); + eprint!("{}", result.stderr); timer.track( &format!("dotnet {}", subcommand), @@ -113,14 +105,10 @@ pub fn run_passthrough(args: &[OsString], verbose: u8) -> Result<()> { &raw, ); - if !output.status.success() { - std::process::exit(output.status.code().unwrap_or(1)); - } - - Ok(()) + Ok(result.exit_code) } -fn run_dotnet_with_binlog(subcommand: &str, args: &[String], verbose: u8) -> Result<()> { +fn run_dotnet_with_binlog(subcommand: &str, args: &[String], verbose: u8) -> Result { let timer = tracking::TimedExecution::start(); let binlog_path = build_binlog_path(subcommand); let should_expect_binlog = subcommand != "test" || has_binlog_arg(args); @@ -143,27 +131,25 @@ fn run_dotnet_with_binlog(subcommand: &str, args: &[String], verbose: u8) -> Res } let command_started_at = SystemTime::now(); - let output = cmd - .output() + let result = exec_capture(&mut cmd) .with_context(|| format!("Failed to run dotnet {}", subcommand))?; - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let raw = format!("{}\n{}", stdout, stderr); + let raw = format!("{}\n{}", result.stdout, result.stderr); + let command_success = result.success(); let filtered = match subcommand { "build" => { let binlog_summary = if should_expect_binlog && binlog_path.exists() { normalize_build_summary( binlog::parse_build(&binlog_path).unwrap_or_default(), - output.status.success(), + command_success, ) } else { binlog::BuildSummary::default() }; let raw_summary = normalize_build_summary( binlog::parse_build_from_text(&raw), - output.status.success(), + command_success, ); let summary = merge_build_summaries(binlog_summary, raw_summary); format_build_output(&summary, &binlog_path) @@ -184,18 +170,18 @@ fn run_dotnet_with_binlog(subcommand: &str, args: &[String], verbose: u8) -> Res command_started_at, ); - let summary = normalize_test_summary(summary, output.status.success()); + let summary = normalize_test_summary(summary, command_success); let binlog_diagnostics = if should_expect_binlog && binlog_path.exists() { normalize_build_summary( binlog::parse_build(&binlog_path).unwrap_or_default(), - output.status.success(), + command_success, ) } else { binlog::BuildSummary::default() }; let raw_diagnostics = normalize_build_summary( binlog::parse_build_from_text(&raw), - output.status.success(), + command_success, ); let test_build_summary = merge_build_summaries(binlog_diagnostics, raw_diagnostics); format_test_output( @@ -209,14 +195,14 @@ fn run_dotnet_with_binlog(subcommand: &str, args: &[String], verbose: u8) -> Res let binlog_summary = if should_expect_binlog && binlog_path.exists() { normalize_restore_summary( binlog::parse_restore(&binlog_path).unwrap_or_default(), - output.status.success(), + command_success, ) } else { binlog::RestoreSummary::default() }; let raw_summary = normalize_restore_summary( binlog::parse_restore_from_text(&raw), - output.status.success(), + command_success, ); let summary = merge_restore_summaries(binlog_summary, raw_summary); @@ -227,9 +213,9 @@ fn run_dotnet_with_binlog(subcommand: &str, args: &[String], verbose: u8) -> Res _ => raw.clone(), }; - let output_to_print = if !output.status.success() { - let stdout_trimmed = stdout.trim(); - let stderr_trimmed = stderr.trim(); + let output_to_print = if !command_success { + let stdout_trimmed = result.stdout.trim(); + let stderr_trimmed = result.stderr.trim(); if !stdout_trimmed.is_empty() { format!("{}\n\n{}", stdout_trimmed, filtered) } else if !stderr_trimmed.is_empty() { @@ -261,11 +247,7 @@ fn run_dotnet_with_binlog(subcommand: &str, args: &[String], verbose: u8) -> Res eprintln!("Binlog cleaned up: {}", binlog_path.display()); } - if !output.status.success() { - std::process::exit(output.status.code().unwrap_or(1)); - } - - Ok(()) + Ok(result.exit_code) } fn build_binlog_path(subcommand: &str) -> PathBuf { diff --git a/src/cmds/dotnet/mod.rs b/src/cmds/dotnet/mod.rs index ce6bfa485..113c57d6a 100644 --- a/src/cmds/dotnet/mod.rs +++ b/src/cmds/dotnet/mod.rs @@ -1,6 +1 @@ -//! .NET ecosystem filters. - -pub mod binlog; -pub mod dotnet_cmd; -pub mod dotnet_format_report; -pub mod dotnet_trx; +automod::dir!(pub "src/cmds/dotnet"); diff --git a/src/cmds/git/README.md b/src/cmds/git/README.md index ec595ca02..09febe2d4 100644 --- a/src/cmds/git/README.md +++ b/src/cmds/git/README.md @@ -1,6 +1,6 @@ # Git and VCS -> Part of [`src/cmds/`](../README.md) — see also [docs/TECHNICAL.md](../../../docs/TECHNICAL.md) +> Part of [`src/cmds/`](../README.md) — see also [docs/contributing/TECHNICAL.md](../../../docs/contributing/TECHNICAL.md) ## Specifics @@ -8,8 +8,31 @@ - Auto-detects `--merges` flag to avoid conflicting with `--no-merges` injection - Global git options (`-C`, `--git-dir`, `--work-tree`, `--no-pager`) are prepended before the subcommand - Exit code propagation is critical for CI/CD pipelines +- **glab_cmd.rs** declares `-R`/`--repo` and `-g`/`--group` at the clap level; they are **appended** to the glab args (not prepended) so subcommand dispatch stays intact +- `has_output_flag()` short-circuits to passthrough when the user explicitly requests `-F` / `--output` / `--json` (avoids double JSON injection) +- `should_passthrough_view()` redirects `mr/issue view` to passthrough when `--web` or `--comments` is set +- JSON handlers use the local `run_glab_json()` helper wrapping `runner::run_filtered` + `RunOptions::stdout_only().early_exit_on_failure().no_trailing_newline()`; on JSON parse error, falls back to the raw stdout (glab sometimes emits plain text for empty results) +- `ci status` uses text-keyword parsing (glab doesn't support `-F json` for this subcommand); when no English status keyword is recognized (non-English locale), returns raw verbatim +- `ci trace` uses ANSI-stripping + GitLab section-marker filtering + runner/git/artifact boilerplate removal; kept as text-only filter, not JSON +- `release list` falls back to raw output when the glab 1.82+ format doesn't match the legacy tab-delimited parser +- Pipeline / merge-status indicators use text tags (`[ok]`, `[fail]`, `[cancel]`, `[run]`, `[pend]`, `[skip]`, `[conflict]`) to match `gh_cmd.rs` and avoid multi-byte rendering quirks ## Cross-command - `gh_cmd.rs` imports `compact_diff()` from `git.rs` for diff formatting; markdown helpers (`filter_markdown_body`, `filter_markdown_segment`) are defined in `gh_cmd.rs` itself +- `glab_cmd.rs` also uses `compact_diff()` from `git.rs` for `mr diff`; its `filter_markdown_body` is currently **duplicated** from `gh_cmd.rs` (shared-module refactor deferred) - `diff_cmd.rs` is a standalone ultra-condensed diff (separate from `git diff`) + +## glab vs gh JSON schema quick-ref + +| Aspect | gh | glab | +|--------|----|------| +| Notation | `#42` | `!42` | +| States | `OPEN`/`MERGED`/`CLOSED` | `opened`/`merged`/`closed` | +| Author | `author.login` | `author.username` | +| URL field | `url` | `web_url` | +| Body field | `body` | `description` | +| Merge check | `mergeable` | `merge_status` (`can_be_merged` / `cannot_be_merged`) | +| CI status | `statusCheckRollup` | `head_pipeline.status` | +| Labels | `labels` (array of objects) | `labels` (array of strings) | +| Reviewers | `reviewRequests`/`reviews` | `reviewers` (array of objects with `username`) | diff --git a/src/cmds/git/diff_cmd.rs b/src/cmds/git/diff_cmd.rs index 96a148bc3..59c95b100 100644 --- a/src/cmds/git/diff_cmd.rs +++ b/src/cmds/git/diff_cmd.rs @@ -40,17 +40,7 @@ pub fn run(file1: &Path, file2: &Path, verbose: u8) -> Result<()> { diff.added, diff.removed, diff.modified )); - // Never truncate diff content — users make decisions based on this data. - // Only the summary header provides compression; all changes are shown in full. - for change in &diff.changes { - match change { - DiffChange::Added(ln, c) => rtk.push_str(&format!("+{:4} {}\n", ln, c)), - DiffChange::Removed(ln, c) => rtk.push_str(&format!("-{:4} {}\n", ln, c)), - DiffChange::Modified(ln, old, new) => { - rtk.push_str(&format!("~{:4} {} → {}\n", ln, old, new)) - } - } - } + rtk.push_str(&format_diff_changes(&diff)); print!("{}", rtk); timer.track( @@ -93,6 +83,20 @@ struct DiffResult { changes: Vec, } +fn format_diff_changes(diff: &DiffResult) -> String { + let mut out = String::new(); + for change in &diff.changes { + match change { + DiffChange::Added(ln, c) => out.push_str(&format!("+{:4} {}\n", ln, c)), + DiffChange::Removed(ln, c) => out.push_str(&format!("-{:4} {}\n", ln, c)), + DiffChange::Modified(ln, old, new) => { + out.push_str(&format!("~{:4} {} → {}\n", ln, old, new)) + } + } + } + out +} + fn compute_diff(lines1: &[&str], lines2: &[&str]) -> DiffResult { let mut changes = Vec::new(); let mut added = 0; @@ -417,6 +421,23 @@ diff --git a/b.rs b/b.rs assert!(!result.changes.is_empty()); } + #[test] + fn test_format_diff_shows_all_changes() { + let mut a = Vec::new(); + let mut b = Vec::new(); + for i in 0..100 { + a.push(format!("old_line_{}", i)); + b.push(format!("new_line_{}", i)); + } + let a_refs: Vec<&str> = a.iter().map(|s| s.as_str()).collect(); + let b_refs: Vec<&str> = b.iter().map(|s| s.as_str()).collect(); + let diff = compute_diff(&a_refs, &b_refs); + let output = format_diff_changes(&diff); + + assert!(output.contains("old_line_0"), "should contain first change"); + assert!(output.contains("new_line_99"), "should contain last change"); + } + #[test] fn test_long_lines_not_truncated() { let long_line = "x".repeat(500); diff --git a/src/cmds/git/gh_cmd.rs b/src/cmds/git/gh_cmd.rs index e008a2f19..3535fd4ec 100644 --- a/src/cmds/git/gh_cmd.rs +++ b/src/cmds/git/gh_cmd.rs @@ -3,13 +3,14 @@ //! Provides token-optimized alternatives to verbose `gh` commands. //! Focuses on extracting essential information from JSON outputs. -use crate::core::tracking; +use crate::core::runner::{self, RunOptions}; use crate::core::utils::{ok_confirmation, resolved_command, truncate}; use crate::git; -use anyhow::{Context, Result}; +use anyhow::Result; use lazy_static::lazy_static; use regex::Regex; use serde_json::Value; +use std::process::Command; lazy_static! { static ref HTML_COMMENT_RE: Regex = Regex::new(r"(?s)").unwrap(); @@ -160,8 +161,25 @@ fn extract_identifier_and_extra_args(args: &[String]) -> Option<(String, Vec Result<()> { +fn run_gh_json(cmd: Command, label: &str, filter_fn: F) -> Result +where + F: Fn(&Value) -> String, +{ + runner::run_filtered( + cmd, + "gh", + label, + |stdout| match serde_json::from_str::(stdout) { + Ok(json) => filter_fn(&json), + Err(_) => stdout.to_string(), + }, + RunOptions::stdout_only() + .early_exit_on_failure() + .no_trailing_newline(), + ) +} + +pub fn run(subcommand: &str, args: &[String], verbose: u8, ultra_compact: bool) -> Result { // When user explicitly passes --json, they want raw gh JSON output, not RTK filtering if has_json_flag(args) { return run_passthrough("gh", subcommand, args); @@ -180,7 +198,7 @@ pub fn run(subcommand: &str, args: &[String], verbose: u8, ultra_compact: bool) } } -fn run_pr(args: &[String], verbose: u8, ultra_compact: bool) -> Result<()> { +fn run_pr(args: &[String], verbose: u8, ultra_compact: bool) -> Result { if args.is_empty() { return run_passthrough("gh", "pr", args); } @@ -189,7 +207,7 @@ fn run_pr(args: &[String], verbose: u8, ultra_compact: bool) -> Result<()> { "list" => list_prs(&args[1..], verbose, ultra_compact), "view" => view_pr(&args[1..], verbose, ultra_compact), "checks" => pr_checks(&args[1..], verbose, ultra_compact), - "status" => pr_status(verbose, ultra_compact), + "status" => pr_status(&args[1..], verbose, ultra_compact), "create" => pr_create(&args[1..], verbose), "merge" => pr_merge(&args[1..], verbose), "diff" => pr_diff(&args[1..], verbose), @@ -199,9 +217,7 @@ fn run_pr(args: &[String], verbose: u8, ultra_compact: bool) -> Result<()> { } } -fn list_prs(args: &[String], _verbose: u8, ultra_compact: bool) -> Result<()> { - let timer = tracking::TimedExecution::start(); - +fn list_prs(args: &[String], _verbose: u8, ultra_compact: bool) -> Result { let mut cmd = resolved_command("gh"); cmd.args([ "pr", @@ -209,78 +225,69 @@ fn list_prs(args: &[String], _verbose: u8, ultra_compact: bool) -> Result<()> { "--json", "number,title,state,author,updatedAt", ]); - - // Pass through additional flags for arg in args { cmd.arg(arg); } + run_gh_json(cmd, "pr list", |json| format_pr_list(json, ultra_compact)) +} - let output = cmd.output().context("Failed to run gh pr list")?; - let raw = String::from_utf8_lossy(&output.stdout).to_string(); - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr).to_string(); - timer.track("gh pr list", "rtk gh pr list", &stderr, &stderr); - eprintln!("{}", stderr.trim()); - std::process::exit(output.status.code().unwrap_or(1)); - } - - let json: Value = - serde_json::from_slice(&output.stdout).context("Failed to parse gh pr list output")?; - - let mut filtered = String::new(); - - if let Some(prs) = json.as_array() { - if ultra_compact { - filtered.push_str("PRs\n"); - println!("PRs"); +fn format_pr_list(json: &Value, ultra_compact: bool) -> String { + let prs = match json.as_array() { + Some(prs) => prs, + None => return String::new(), + }; + if prs.is_empty() { + return if ultra_compact { + "No PRs\n".to_string() } else { - filtered.push_str("Pull Requests\n"); - println!("Pull Requests"); - } - - for pr in prs.iter().take(20) { - let number = pr["number"].as_i64().unwrap_or(0); - let title = pr["title"].as_str().unwrap_or("???"); - let state = pr["state"].as_str().unwrap_or("???"); - let author = pr["author"]["login"].as_str().unwrap_or("???"); - - let state_icon = if ultra_compact { - match state { - "OPEN" => "O", - "MERGED" => "M", - "CLOSED" => "C", - _ => "?", - } - } else { - match state { - "OPEN" => "[open]", - "MERGED" => "[merged]", - "CLOSED" => "[closed]", - _ => "[unknown]", - } - }; + "No Pull Requests\n".to_string() + }; + } + let mut out = String::new(); + out.push_str(if ultra_compact { + "PRs\n" + } else { + "Pull Requests\n" + }); + for pr in prs.iter().take(20) { + let number = pr["number"].as_i64().unwrap_or(0); + let title = pr["title"].as_str().unwrap_or("???"); + let state = pr["state"].as_str().unwrap_or("???"); + let author = pr["author"]["login"].as_str().unwrap_or("???"); + let icon = state_icon(state, ultra_compact); + out.push_str(&format!( + " {} #{} {} ({})\n", + icon, + number, + truncate(title, 60), + author + )); + } + if prs.len() > 20 { + out.push_str(&format!( + " ... {} more (use gh pr list for all)\n", + prs.len() - 20 + )); + } + out +} - let line = format!( - " {} #{} {} ({})\n", - state_icon, - number, - truncate(title, 60), - author - ); - filtered.push_str(&line); - print!("{}", line); +fn state_icon(state: &str, ultra_compact: bool) -> &'static str { + if ultra_compact { + match state { + "OPEN" => "O", + "MERGED" => "M", + "CLOSED" => "C", + _ => "?", } - - if prs.len() > 20 { - let more_line = format!(" ... {} more (use gh pr list for all)\n", prs.len() - 20); - filtered.push_str(&more_line); - print!("{}", more_line); + } else { + match state { + "OPEN" => "[open]", + "MERGED" => "[merged]", + "CLOSED" => "[closed]", + _ => "[unknown]", } } - - timer.track("gh pr list", "rtk gh pr list", &raw, &filtered); - Ok(()) } fn should_passthrough_pr_view(extra_args: &[String]) -> bool { @@ -295,20 +302,27 @@ fn should_passthrough_issue_view(extra_args: &[String]) -> bool { .any(|a| a == "--json" || a == "--jq" || a == "--web" || a == "--comments") } -fn view_pr(args: &[String], _verbose: u8, ultra_compact: bool) -> Result<()> { - let timer = tracking::TimedExecution::start(); +fn should_passthrough_pr_status(args: &[String]) -> bool { + args.iter().any(|a| { + matches!( + a.as_str(), + "--help" | "-h" | "--web" | "--jq" | "--template" + ) + }) +} + +fn pr_status_json_fields() -> &'static str { + "number,title,reviewDecision,statusCheckRollup" +} +fn view_pr(args: &[String], _verbose: u8, ultra_compact: bool) -> Result { let (pr_number, extra_args) = match extract_identifier_and_extra_args(args) { Some(result) => result, None => return Err(anyhow::anyhow!("PR number required")), }; - - // If the user provides --jq or --web, pass through directly. - // Note: --json is already handled globally by run() via has_json_flag. if should_passthrough_pr_view(&extra_args) { return run_passthrough_with_extra("gh", &["pr", "view", &pr_number], &extra_args); } - let mut cmd = resolved_command("gh"); cmd.args([ "pr", @@ -320,28 +334,13 @@ fn view_pr(args: &[String], _verbose: u8, ultra_compact: bool) -> Result<()> { for arg in &extra_args { cmd.arg(arg); } + run_gh_json(cmd, &format!("pr view {}", pr_number), |json| { + format_pr_view(json, ultra_compact) + }) +} - let output = cmd.output().context("Failed to run gh pr view")?; - let raw = String::from_utf8_lossy(&output.stdout).to_string(); - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr).to_string(); - timer.track( - &format!("gh pr view {}", pr_number), - &format!("rtk gh pr view {}", pr_number), - &stderr, - &stderr, - ); - eprintln!("{}", stderr.trim()); - std::process::exit(output.status.code().unwrap_or(1)); - } - - let json: Value = - serde_json::from_slice(&output.stdout).context("Failed to parse gh pr view output")?; - - let mut filtered = String::new(); - - // Extract essential info +fn format_pr_view(json: &Value, ultra_compact: bool) -> String { + let mut out = String::new(); let number = json["number"].as_i64().unwrap_or(0); let title = json["title"].as_str().unwrap_or("???"); let state = json["state"].as_str().unwrap_or("???"); @@ -349,40 +348,17 @@ fn view_pr(args: &[String], _verbose: u8, ultra_compact: bool) -> Result<()> { let url = json["url"].as_str().unwrap_or(""); let mergeable = json["mergeable"].as_str().unwrap_or("UNKNOWN"); - let state_icon = if ultra_compact { - match state { - "OPEN" => "O", - "MERGED" => "M", - "CLOSED" => "C", - _ => "?", - } - } else { - match state { - "OPEN" => "[open]", - "MERGED" => "[merged]", - "CLOSED" => "[closed]", - _ => "[unknown]", - } - }; - - let line = format!("{} PR #{}: {}\n", state_icon, number, title); - filtered.push_str(&line); - print!("{}", line); - - let line = format!(" {}\n", author); - filtered.push_str(&line); - print!("{}", line); + let icon = state_icon(state, ultra_compact); + out.push_str(&format!("{} PR #{}: {}\n", icon, number, title)); + out.push_str(&format!(" {}\n", author)); let mergeable_str = match mergeable { "MERGEABLE" => "[ok]", "CONFLICTING" => "[x]", _ => "?", }; - let line = format!(" {} | {}\n", state, mergeable_str); - filtered.push_str(&line); - print!("{}", line); + out.push_str(&format!(" {} | {}\n", state, mergeable_str)); - // Show reviews summary if let Some(reviews) = json["reviews"]["nodes"].as_array() { let approved = reviews .iter() @@ -392,18 +368,14 @@ fn view_pr(args: &[String], _verbose: u8, ultra_compact: bool) -> Result<()> { .iter() .filter(|r| r["state"].as_str() == Some("CHANGES_REQUESTED")) .count(); - if approved > 0 || changes > 0 { - let line = format!( + out.push_str(&format!( " Reviews: {} approved, {} changes requested\n", approved, changes - ); - filtered.push_str(&line); - print!("{}", line); + )); } } - // Show checks summary if let Some(checks) = json["statusCheckRollup"].as_array() { let total = checks.len(); let passed = checks @@ -420,90 +392,59 @@ fn view_pr(args: &[String], _verbose: u8, ultra_compact: bool) -> Result<()> { || c["state"].as_str() == Some("FAILURE") }) .count(); - if ultra_compact { if failed > 0 { - let line = format!(" [x]{}/{} {} fail\n", passed, total, failed); - filtered.push_str(&line); - print!("{}", line); + out.push_str(&format!(" [x]{}/{} {} fail\n", passed, total, failed)); } else { - let line = format!(" {}/{}\n", passed, total); - filtered.push_str(&line); - print!("{}", line); + out.push_str(&format!(" {}/{}\n", passed, total)); } } else { - let line = format!(" Checks: {}/{} passed\n", passed, total); - filtered.push_str(&line); - print!("{}", line); + out.push_str(&format!(" Checks: {}/{} passed\n", passed, total)); if failed > 0 { - let line = format!(" [warn] {} checks failed\n", failed); - filtered.push_str(&line); - print!("{}", line); + out.push_str(&format!(" [warn] {} checks failed\n", failed)); } } } - let line = format!(" {}\n", url); - filtered.push_str(&line); - print!("{}", line); + out.push_str(&format!(" {}\n", url)); - // Show filtered body if let Some(body) = json["body"].as_str() { if !body.is_empty() { let body_filtered = filter_markdown_body(body); if !body_filtered.is_empty() { - filtered.push('\n'); - println!(); + out.push('\n'); for line in body_filtered.lines() { - let formatted = format!(" {}\n", line); - filtered.push_str(&formatted); - print!("{}", formatted); + out.push_str(&format!(" {}\n", line)); } } } } - timer.track( - &format!("gh pr view {}", pr_number), - &format!("rtk gh pr view {}", pr_number), - &raw, - &filtered, - ); - Ok(()) + out } -fn pr_checks(args: &[String], _verbose: u8, _ultra_compact: bool) -> Result<()> { - let timer = tracking::TimedExecution::start(); - +fn pr_checks(args: &[String], _verbose: u8, _ultra_compact: bool) -> Result { let (pr_number, extra_args) = match extract_identifier_and_extra_args(args) { Some(result) => result, None => return Err(anyhow::anyhow!("PR number required")), }; - let mut cmd = resolved_command("gh"); cmd.args(["pr", "checks", &pr_number]); for arg in &extra_args { cmd.arg(arg); } + runner::run_filtered( + cmd, + "gh", + &format!("pr checks {}", pr_number), + format_pr_checks, + RunOptions::stdout_only() + .early_exit_on_failure() + .no_trailing_newline(), + ) +} - let output = cmd.output().context("Failed to run gh pr checks")?; - let raw = String::from_utf8_lossy(&output.stdout).to_string(); - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr).to_string(); - timer.track( - &format!("gh pr checks {}", pr_number), - &format!("rtk gh pr checks {}", pr_number), - &stderr, - &stderr, - ); - eprintln!("{}", stderr.trim()); - std::process::exit(output.status.code().unwrap_or(1)); - } - - let stdout = String::from_utf8_lossy(&output.stdout); - - // Parse and compress checks output +fn format_pr_checks(stdout: &str) -> String { let mut passed = 0; let mut failed = 0; let mut pending = 0; @@ -520,91 +461,102 @@ fn pr_checks(args: &[String], _verbose: u8, _ultra_compact: bool) -> Result<()> } } - let mut filtered = String::new(); - - let line = "CI Checks Summary:\n"; - filtered.push_str(line); - print!("{}", line); - - let line = format!(" [ok] Passed: {}\n", passed); - filtered.push_str(&line); - print!("{}", line); - - let line = format!(" [FAIL] Failed: {}\n", failed); - filtered.push_str(&line); - print!("{}", line); - + let mut out = String::new(); + out.push_str("CI Checks Summary:\n"); + out.push_str(&format!(" [ok] Passed: {}\n", passed)); + out.push_str(&format!(" [FAIL] Failed: {}\n", failed)); if pending > 0 { - let line = format!(" [pending] Pending: {}\n", pending); - filtered.push_str(&line); - print!("{}", line); + out.push_str(&format!(" [pending] Pending: {}\n", pending)); } - if !failed_checks.is_empty() { - let line = "\n Failed checks:\n"; - filtered.push_str(line); - print!("{}", line); + out.push_str("\n Failed checks:\n"); for check in failed_checks { - let line = format!(" {}\n", check); - filtered.push_str(&line); - print!("{}", line); + out.push_str(&format!(" {}\n", check)); } } - - timer.track( - &format!("gh pr checks {}", pr_number), - &format!("rtk gh pr checks {}", pr_number), - &raw, - &filtered, - ); - Ok(()) + out } -fn pr_status(_verbose: u8, _ultra_compact: bool) -> Result<()> { - let timer = tracking::TimedExecution::start(); +fn pr_status(args: &[String], _verbose: u8, _ultra_compact: bool) -> Result { + if should_passthrough_pr_status(args) { + let mut passthrough_args = Vec::with_capacity(args.len() + 1); + passthrough_args.push("status".to_string()); + passthrough_args.extend(args.iter().cloned()); + return run_passthrough("gh", "pr", &passthrough_args); + } let mut cmd = resolved_command("gh"); - cmd.args([ - "pr", - "status", - "--json", - "currentBranch,createdBy,reviewDecision,statusCheckRollup", - ]); - - let output = cmd.output().context("Failed to run gh pr status")?; - let raw = String::from_utf8_lossy(&output.stdout).to_string(); - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr).to_string(); - timer.track("gh pr status", "rtk gh pr status", &stderr, &stderr); - eprintln!("{}", stderr.trim()); - std::process::exit(output.status.code().unwrap_or(1)); + cmd.args(["pr", "status", "--json", pr_status_json_fields()]); + for arg in args { + cmd.arg(arg); } + run_gh_json(cmd, "pr status", format_pr_status) +} - let json: Value = - serde_json::from_slice(&output.stdout).context("Failed to parse gh pr status output")?; +fn format_pr_status(json: &Value) -> String { + let mut out = String::new(); - let mut filtered = String::new(); + if !json["currentBranch"].is_null() { + let current_branch = format_pr_status_entry(&json["currentBranch"]); + if !current_branch.is_empty() { + out.push_str("Current Branch\n"); + out.push_str(¤t_branch); + out.push('\n'); + } + } if let Some(created_by) = json["createdBy"].as_array() { - let line = format!("Your PRs ({}):\n", created_by.len()); - filtered.push_str(&line); - print!("{}", line); + out.push_str(&format!("Your PRs ({}):\n", created_by.len())); for pr in created_by.iter().take(5) { - let number = pr["number"].as_i64().unwrap_or(0); - let title = pr["title"].as_str().unwrap_or("???"); - let reviews = pr["reviewDecision"].as_str().unwrap_or("PENDING"); - let line = format!(" #{} {} [{}]\n", number, truncate(title, 50), reviews); - filtered.push_str(&line); - print!("{}", line); + let entry = format_pr_status_entry(pr); + if !entry.is_empty() { + out.push_str(&entry); + } } } + out +} + +fn format_pr_status_entry(pr: &Value) -> String { + if pr.is_null() { + return String::new(); + } - timer.track("gh pr status", "rtk gh pr status", &raw, &filtered); - Ok(()) + let number = pr["number"].as_i64().unwrap_or(0); + let title = pr["title"].as_str().unwrap_or("???"); + let reviews = pr["reviewDecision"].as_str().unwrap_or("PENDING"); + let mut out = format!(" #{} {} [{}]", number, truncate(title, 50), reviews); + + if let Some(checks) = pr["statusCheckRollup"].as_array() { + let total = checks.len(); + if total > 0 { + let passed = checks + .iter() + .filter(|c| { + c["conclusion"].as_str() == Some("SUCCESS") + || c["state"].as_str() == Some("SUCCESS") + }) + .count(); + let failed = checks + .iter() + .filter(|c| { + c["conclusion"].as_str() == Some("FAILURE") + || c["state"].as_str() == Some("FAILURE") + }) + .count(); + + out.push_str(&format!(" checks {}/{}", passed, total)); + if failed > 0 { + out.push_str(&format!(" fail {}", failed)); + } + } + } + + out.push('\n'); + out } -fn run_issue(args: &[String], verbose: u8, ultra_compact: bool) -> Result<()> { +fn run_issue(args: &[String], verbose: u8, ultra_compact: bool) -> Result { if args.is_empty() { return run_passthrough("gh", "issue", args); } @@ -616,83 +568,58 @@ fn run_issue(args: &[String], verbose: u8, ultra_compact: bool) -> Result<()> { } } -fn list_issues(args: &[String], _verbose: u8, ultra_compact: bool) -> Result<()> { - let timer = tracking::TimedExecution::start(); - +fn list_issues(args: &[String], _verbose: u8, ultra_compact: bool) -> Result { let mut cmd = resolved_command("gh"); cmd.args(["issue", "list", "--json", "number,title,state,author"]); - for arg in args { cmd.arg(arg); } + run_gh_json(cmd, "issue list", |json| { + format_issue_list(json, ultra_compact) + }) +} - let output = cmd.output().context("Failed to run gh issue list")?; - let raw = String::from_utf8_lossy(&output.stdout).to_string(); - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr).to_string(); - timer.track("gh issue list", "rtk gh issue list", &stderr, &stderr); - eprintln!("{}", stderr.trim()); - std::process::exit(output.status.code().unwrap_or(1)); - } - - let json: Value = - serde_json::from_slice(&output.stdout).context("Failed to parse gh issue list output")?; - - let mut filtered = String::new(); - - if let Some(issues) = json.as_array() { - filtered.push_str("Issues\n"); - println!("Issues"); - for issue in issues.iter().take(20) { - let number = issue["number"].as_i64().unwrap_or(0); - let title = issue["title"].as_str().unwrap_or("???"); - let state = issue["state"].as_str().unwrap_or("???"); - - let icon = if ultra_compact { - if state == "OPEN" { - "O" - } else { - "C" - } +fn format_issue_list(json: &Value, ultra_compact: bool) -> String { + let issues = match json.as_array() { + Some(issues) => issues, + None => return String::new(), + }; + if issues.is_empty() { + return "No Issues\n".to_string(); + } + let mut out = String::new(); + out.push_str("Issues\n"); + for issue in issues.iter().take(20) { + let number = issue["number"].as_i64().unwrap_or(0); + let title = issue["title"].as_str().unwrap_or("???"); + let state = issue["state"].as_str().unwrap_or("???"); + let icon = if ultra_compact { + if state == "OPEN" { + "O" } else { - if state == "OPEN" { - "[open]" - } else { - "[closed]" - } - }; - let line = format!(" {} #{} {}\n", icon, number, truncate(title, 60)); - filtered.push_str(&line); - print!("{}", line); - } - - if issues.len() > 20 { - let line = format!(" ... {} more\n", issues.len() - 20); - filtered.push_str(&line); - print!("{}", line); - } + "C" + } + } else if state == "OPEN" { + "[open]" + } else { + "[closed]" + }; + out.push_str(&format!(" {} #{} {}\n", icon, number, truncate(title, 60))); } - - timer.track("gh issue list", "rtk gh issue list", &raw, &filtered); - Ok(()) + if issues.len() > 20 { + out.push_str(&format!(" ... {} more\n", issues.len() - 20)); + } + out } -fn view_issue(args: &[String], _verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - +fn view_issue(args: &[String], _verbose: u8) -> Result { let (issue_number, extra_args) = match extract_identifier_and_extra_args(args) { Some(result) => result, None => return Err(anyhow::anyhow!("Issue number required")), }; - - // Passthrough when --comments, --json, --jq, or --web is present. - // --comments changes the output to include comments which our JSON - // field list doesn't request, causing silent data loss. if should_passthrough_issue_view(&extra_args) { return run_passthrough_with_extra("gh", &["issue", "view", &issue_number], &extra_args); } - let mut cmd = resolved_command("gh"); cmd.args([ "issue", @@ -704,25 +631,13 @@ fn view_issue(args: &[String], _verbose: u8) -> Result<()> { for arg in &extra_args { cmd.arg(arg); } + run_gh_json(cmd, &format!("issue view {}", issue_number), |json| { + format_issue_view(json) + }) +} - let output = cmd.output().context("Failed to run gh issue view")?; - let raw = String::from_utf8_lossy(&output.stdout).to_string(); - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr).to_string(); - timer.track( - &format!("gh issue view {}", issue_number), - &format!("rtk gh issue view {}", issue_number), - &stderr, - &stderr, - ); - eprintln!("{}", stderr.trim()); - std::process::exit(output.status.code().unwrap_or(1)); - } - - let json: Value = - serde_json::from_slice(&output.stdout).context("Failed to parse gh issue view output")?; - +fn format_issue_view(json: &Value) -> String { + let mut out = String::new(); let number = json["number"].as_i64().unwrap_or(0); let title = json["title"].as_str().unwrap_or("???"); let state = json["state"].as_str().unwrap_or("???"); @@ -734,51 +649,26 @@ fn view_issue(args: &[String], _verbose: u8) -> Result<()> { } else { "[closed]" }; - - let mut filtered = String::new(); - - let line = format!("{} Issue #{}: {}\n", icon, number, title); - filtered.push_str(&line); - print!("{}", line); - - let line = format!(" Author: @{}\n", author); - filtered.push_str(&line); - print!("{}", line); - - let line = format!(" Status: {}\n", state); - filtered.push_str(&line); - print!("{}", line); - - let line = format!(" URL: {}\n", url); - filtered.push_str(&line); - print!("{}", line); + out.push_str(&format!("{} Issue #{}: {}\n", icon, number, title)); + out.push_str(&format!(" Author: @{}\n", author)); + out.push_str(&format!(" Status: {}\n", state)); + out.push_str(&format!(" URL: {}\n", url)); if let Some(body) = json["body"].as_str() { if !body.is_empty() { let body_filtered = filter_markdown_body(body); if !body_filtered.is_empty() { - let line = "\n Description:\n"; - filtered.push_str(line); - print!("{}", line); + out.push_str("\n Description:\n"); for line in body_filtered.lines() { - let formatted = format!(" {}\n", line); - filtered.push_str(&formatted); - print!("{}", formatted); + out.push_str(&format!(" {}\n", line)); } } } } - - timer.track( - &format!("gh issue view {}", issue_number), - &format!("rtk gh issue view {}", issue_number), - &raw, - &filtered, - ); - Ok(()) + out } -fn run_workflow(args: &[String], verbose: u8, ultra_compact: bool) -> Result<()> { +fn run_workflow(args: &[String], verbose: u8, ultra_compact: bool) -> Result { if args.is_empty() { return run_passthrough("gh", "run", args); } @@ -790,9 +680,7 @@ fn run_workflow(args: &[String], verbose: u8, ultra_compact: bool) -> Result<()> } } -fn list_runs(args: &[String], _verbose: u8, ultra_compact: bool) -> Result<()> { - let timer = tracking::TimedExecution::start(); - +fn list_runs(args: &[String], _verbose: u8, ultra_compact: bool) -> Result { let mut cmd = resolved_command("gh"); cmd.args([ "run", @@ -801,76 +689,48 @@ fn list_runs(args: &[String], _verbose: u8, ultra_compact: bool) -> Result<()> { "databaseId,name,status,conclusion,createdAt", ]); cmd.arg("--limit").arg("10"); - for arg in args { cmd.arg(arg); } + run_gh_json(cmd, "run list", |json| format_run_list(json, ultra_compact)) +} - let output = cmd.output().context("Failed to run gh run list")?; - let raw = String::from_utf8_lossy(&output.stdout).to_string(); - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr).to_string(); - timer.track("gh run list", "rtk gh run list", &stderr, &stderr); - eprintln!("{}", stderr.trim()); - std::process::exit(output.status.code().unwrap_or(1)); - } - - let json: Value = - serde_json::from_slice(&output.stdout).context("Failed to parse gh run list output")?; - - let mut filtered = String::new(); - - if let Some(runs) = json.as_array() { - if ultra_compact { - filtered.push_str("Runs\n"); - println!("Runs"); +fn format_run_list(json: &Value, ultra_compact: bool) -> String { + let runs = match json.as_array() { + Some(runs) => runs, + None => return String::new(), + }; + let mut out = String::new(); + out.push_str(if ultra_compact { + "Runs\n" + } else { + "Workflow Runs\n" + }); + for run in runs { + let id = run["databaseId"].as_i64().unwrap_or(0); + let name = run["name"].as_str().unwrap_or("???"); + let status = run["status"].as_str().unwrap_or("???"); + let conclusion = run["conclusion"].as_str().unwrap_or(""); + let icon = if ultra_compact { + match conclusion { + "success" => "[ok]", + "failure" => "[x]", + "cancelled" => "X", + _ if status == "in_progress" => "~", + _ => "?", + } } else { - filtered.push_str("Workflow Runs\n"); - println!("Workflow Runs"); - } - for run in runs { - let id = run["databaseId"].as_i64().unwrap_or(0); - let name = run["name"].as_str().unwrap_or("???"); - let status = run["status"].as_str().unwrap_or("???"); - let conclusion = run["conclusion"].as_str().unwrap_or(""); - - let icon = if ultra_compact { - match conclusion { - "success" => "[ok]", - "failure" => "[x]", - "cancelled" => "X", - _ => { - if status == "in_progress" { - "~" - } else { - "?" - } - } - } - } else { - match conclusion { - "success" => "[ok]", - "failure" => "[FAIL]", - "cancelled" => "[X]", - _ => { - if status == "in_progress" { - "[time]" - } else { - "[pending]" - } - } - } - }; - - let line = format!(" {} {} [{}]\n", icon, truncate(name, 50), id); - filtered.push_str(&line); - print!("{}", line); - } + match conclusion { + "success" => "[ok]", + "failure" => "[FAIL]", + "cancelled" => "[X]", + _ if status == "in_progress" => "[time]", + _ => "[pending]", + } + }; + out.push_str(&format!(" {} {} [{}]\n", icon, truncate(name, 50), id)); } - - timer.track("gh run list", "rtk gh run list", &raw, &filtered); - Ok(()) + out } /// Check if run view args should bypass filtering and pass through directly. @@ -882,120 +742,77 @@ fn should_passthrough_run_view(extra_args: &[String]) -> bool { .any(|a| a == "--log-failed" || a == "--log" || a == "--json") } -fn view_run(args: &[String], _verbose: u8) -> Result<()> { +fn view_run(args: &[String], _verbose: u8) -> Result { let (run_id, extra_args) = match extract_identifier_and_extra_args(args) { Some(result) => result, None => return Err(anyhow::anyhow!("Run ID required")), }; - - // Pass through when user requests logs or JSON — the filter would strip them if should_passthrough_run_view(&extra_args) { return run_passthrough_with_extra("gh", &["run", "view", &run_id], &extra_args); } - - let timer = tracking::TimedExecution::start(); - let mut cmd = resolved_command("gh"); cmd.args(["run", "view", &run_id]); for arg in &extra_args { cmd.arg(arg); } + let run_id_owned = run_id.clone(); + runner::run_filtered( + cmd, + "gh", + &format!("run view {}", run_id), + move |stdout| format_run_view(stdout, &run_id_owned), + RunOptions::stdout_only() + .early_exit_on_failure() + .no_trailing_newline(), + ) +} - let output = cmd.output().context("Failed to run gh run view")?; - let raw = String::from_utf8_lossy(&output.stdout).to_string(); - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr).to_string(); - timer.track( - &format!("gh run view {}", run_id), - &format!("rtk gh run view {}", run_id), - &stderr, - &stderr, - ); - eprintln!("{}", stderr.trim()); - std::process::exit(output.status.code().unwrap_or(1)); - } - - // Parse output and show only failures - let stdout = String::from_utf8_lossy(&output.stdout); +fn format_run_view(stdout: &str, run_id: &str) -> String { + let mut out = String::new(); let mut in_jobs = false; - let mut filtered = String::new(); - - let line = format!("Workflow Run #{}\n", run_id); - filtered.push_str(&line); - print!("{}", line); - + out.push_str(&format!("Workflow Run #{}\n", run_id)); for line in stdout.lines() { if line.contains("JOBS") { in_jobs = true; } - if in_jobs { if line.contains('✓') || line.contains("success") { - // Skip successful jobs in compact mode continue; } if line.contains("[x]") || line.contains("fail") { - let formatted = format!(" [FAIL] {}\n", line.trim()); - filtered.push_str(&formatted); - print!("{}", formatted); + out.push_str(&format!(" [FAIL] {}\n", line.trim())); } } else if line.contains("Status:") || line.contains("Conclusion:") { - let formatted = format!(" {}\n", line.trim()); - filtered.push_str(&formatted); - print!("{}", formatted); + out.push_str(&format!(" {}\n", line.trim())); } } - - timer.track( - &format!("gh run view {}", run_id), - &format!("rtk gh run view {}", run_id), - &raw, - &filtered, - ); - Ok(()) + out } -fn run_repo(args: &[String], _verbose: u8, _ultra_compact: bool) -> Result<()> { - // Parse subcommand (default to "view") +fn run_repo(args: &[String], _verbose: u8, _ultra_compact: bool) -> Result { let (subcommand, rest_args) = if args.is_empty() { ("view", args) } else { (args[0].as_str(), &args[1..]) }; - if subcommand != "view" { return run_passthrough("gh", "repo", args); } - - let timer = tracking::TimedExecution::start(); - let mut cmd = resolved_command("gh"); cmd.arg("repo").arg("view"); - for arg in rest_args { cmd.arg(arg); } - cmd.args([ "--json", "name,owner,description,url,stargazerCount,forkCount,isPrivate", ]); + run_gh_json(cmd, "repo view", format_repo_view) +} - let output = cmd.output().context("Failed to run gh repo view")?; - let raw = String::from_utf8_lossy(&output.stdout).to_string(); - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr).to_string(); - timer.track("gh repo view", "rtk gh repo view", &stderr, &stderr); - eprintln!("{}", stderr.trim()); - std::process::exit(output.status.code().unwrap_or(1)); - } - - let json: Value = - serde_json::from_slice(&output.stdout).context("Failed to parse gh repo view output")?; - +fn format_repo_view(json: &Value) -> String { + let mut out = String::new(); let name = json["name"].as_str().unwrap_or("???"); let owner = json["owner"]["login"].as_str().unwrap_or("???"); let description = json["description"].as_str().unwrap_or(""); @@ -1003,119 +820,50 @@ fn run_repo(args: &[String], _verbose: u8, _ultra_compact: bool) -> Result<()> { let stars = json["stargazerCount"].as_i64().unwrap_or(0); let forks = json["forkCount"].as_i64().unwrap_or(0); let private = json["isPrivate"].as_bool().unwrap_or(false); - let visibility = if private { "[private]" } else { "[public]" }; - let mut filtered = String::new(); - - let line = format!("{}/{}\n", owner, name); - filtered.push_str(&line); - print!("{}", line); - - let line = format!(" {}\n", visibility); - filtered.push_str(&line); - print!("{}", line); - + out.push_str(&format!("{}/{}\n", owner, name)); + out.push_str(&format!(" {}\n", visibility)); if !description.is_empty() { - let line = format!(" {}\n", truncate(description, 80)); - filtered.push_str(&line); - print!("{}", line); + out.push_str(&format!(" {}\n", truncate(description, 80))); } - - let line = format!(" {} stars | {} forks\n", stars, forks); - filtered.push_str(&line); - print!("{}", line); - - let line = format!(" {}\n", url); - filtered.push_str(&line); - print!("{}", line); - - timer.track("gh repo view", "rtk gh repo view", &raw, &filtered); - Ok(()) + out.push_str(&format!(" {} stars | {} forks\n", stars, forks)); + out.push_str(&format!(" {}\n", url)); + out } -fn pr_create(args: &[String], _verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - +fn pr_create(args: &[String], _verbose: u8) -> Result { let mut cmd = resolved_command("gh"); cmd.args(["pr", "create"]); for arg in args { cmd.arg(arg); } - - let output = cmd.output().context("Failed to run gh pr create")?; - let stdout = String::from_utf8_lossy(&output.stdout).to_string(); - let stderr = String::from_utf8_lossy(&output.stderr).to_string(); - - if !output.status.success() { - timer.track("gh pr create", "rtk gh pr create", &stderr, &stderr); - eprintln!("{}", stderr.trim()); - std::process::exit(output.status.code().unwrap_or(1)); - } - - // gh pr create outputs the URL on success - let url = stdout.trim(); - - // Try to extract PR number from URL (e.g., https://github.com/owner/repo/pull/42) - let pr_num = url.rsplit('/').next().unwrap_or(""); - - let detail = if !pr_num.is_empty() && pr_num.chars().all(|c| c.is_ascii_digit()) { - format!("#{} {}", pr_num, url) - } else { - url.to_string() - }; - - let filtered = ok_confirmation("created", &detail); - println!("{}", filtered); - - timer.track("gh pr create", "rtk gh pr create", &stdout, &filtered); - Ok(()) + runner::run_filtered( + cmd, + "gh", + "pr create", + |stdout| { + let url = stdout.trim(); + let pr_num = url.rsplit('/').next().unwrap_or(""); + let detail = if !pr_num.is_empty() && pr_num.chars().all(|c| c.is_ascii_digit()) { + format!("#{} {}", pr_num, url) + } else { + url.to_string() + }; + ok_confirmation("created", &detail) + }, + RunOptions::stdout_only().early_exit_on_failure(), + ) } -fn pr_merge(args: &[String], _verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - - let mut cmd = resolved_command("gh"); - cmd.args(["pr", "merge"]); - for arg in args { - cmd.arg(arg); - } - - let output = cmd.output().context("Failed to run gh pr merge")?; - let stdout = String::from_utf8_lossy(&output.stdout).to_string(); - let stderr = String::from_utf8_lossy(&output.stderr).to_string(); - - if !output.status.success() { - timer.track("gh pr merge", "rtk gh pr merge", &stderr, &stderr); - eprintln!("{}", stderr.trim()); - std::process::exit(output.status.code().unwrap_or(1)); - } - - // Extract PR number from args (first non-flag arg) - let pr_num = args - .iter() - .find(|a| !a.starts_with('-')) - .map(|s| s.as_str()) - .unwrap_or(""); - - let detail = if !pr_num.is_empty() { - format!("#{}", pr_num) - } else { - String::new() - }; - - let filtered = ok_confirmation("merged", &detail); - println!("{}", filtered); - - // Use stdout or detail as raw input (gh pr merge doesn't output much) - let raw = if !stdout.trim().is_empty() { - stdout - } else { - detail.clone() - }; - - timer.track("gh pr merge", "rtk gh pr merge", &raw, &filtered); - Ok(()) +fn pr_merge(args: &[String], _verbose: u8) -> Result { + // gh pr merge is a destructive action — pass through the real output + // so the user (or AI agent) sees exactly what happened. + run_passthrough("gh", "pr", &{ + let mut a = vec!["merge".to_string()]; + a.extend_from_slice(args); + a + }) } /// Flags that change `gh pr diff` output from unified diff to a different format. @@ -1130,170 +878,77 @@ fn has_non_diff_format_flag(args: &[String]) -> bool { }) } -fn pr_diff(args: &[String], _verbose: u8) -> Result<()> { - // --no-compact: pass full diff through (gh CLI doesn't know this flag, strip it) +fn pr_diff(args: &[String], _verbose: u8) -> Result { let no_compact = args.iter().any(|a| a == "--no-compact"); let gh_args: Vec = args .iter() .filter(|a| *a != "--no-compact") .cloned() .collect(); - - // Passthrough when --no-compact or when a format flag changes output away from - // unified diff (e.g. --name-only produces a filename list, not diff hunks). if no_compact || has_non_diff_format_flag(&gh_args) { return run_passthrough_with_extra("gh", &["pr", "diff"], &gh_args); } - - let timer = tracking::TimedExecution::start(); - let mut cmd = resolved_command("gh"); cmd.args(["pr", "diff"]); for arg in gh_args.iter() { cmd.arg(arg); } - - let output = cmd.output().context("Failed to run gh pr diff")?; - let raw = String::from_utf8_lossy(&output.stdout).to_string(); - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr).to_string(); - timer.track("gh pr diff", "rtk gh pr diff", &stderr, &stderr); - eprintln!("{}", stderr.trim()); - std::process::exit(output.status.code().unwrap_or(1)); - } - - let filtered = if raw.trim().is_empty() { - let msg = "No diff\n"; - print!("{}", msg); - msg.to_string() - } else { - let compacted = git::compact_diff(&raw, 500); - println!("{}", compacted); - compacted - }; - - timer.track("gh pr diff", "rtk gh pr diff", &raw, &filtered); - Ok(()) + runner::run_filtered( + cmd, + "gh", + "pr diff", + |raw| { + if raw.trim().is_empty() { + "No diff".to_string() + } else { + git::compact_diff(raw, 500) + } + }, + RunOptions::stdout_only().early_exit_on_failure(), + ) } -/// Generic PR action handler for comment/edit -fn pr_action(action: &str, args: &[String], _verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); +fn pr_action(action: &str, args: &[String], _verbose: u8) -> Result { let subcmd = &args[0]; - - let mut cmd = resolved_command("gh"); - cmd.arg("pr"); - for arg in args { - cmd.arg(arg); - } - - let output = cmd - .output() - .context(format!("Failed to run gh pr {}", subcmd))?; - let stdout = String::from_utf8_lossy(&output.stdout).to_string(); - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr).to_string(); - timer.track( - &format!("gh pr {}", subcmd), - &format!("rtk gh pr {}", subcmd), - &stderr, - &stderr, - ); - eprintln!("{}", stderr.trim()); - std::process::exit(output.status.code().unwrap_or(1)); - } - - // Extract PR number from args (skip args[0] which is the subcommand) let pr_num = args[1..] .iter() .find(|a| !a.starts_with('-')) .map(|s| format!("#{}", s)) .unwrap_or_default(); - - let filtered = ok_confirmation(action, &pr_num); - println!("{}", filtered); - - // Use stdout or pr_num as raw input - let raw = if !stdout.trim().is_empty() { - stdout - } else { - pr_num.clone() - }; - - timer.track( - &format!("gh pr {}", subcmd), - &format!("rtk gh pr {}", subcmd), - &raw, - &filtered, - ); - Ok(()) + let mut cmd = resolved_command("gh"); + cmd.arg("pr"); + for arg in args { + cmd.arg(arg); + } + let action = action.to_string(); + runner::run_filtered( + cmd, + "gh", + &format!("pr {}", subcmd), + move |_stdout| ok_confirmation(&action, &pr_num), + RunOptions::stdout_only().early_exit_on_failure(), + ) } -fn run_api(args: &[String], _verbose: u8) -> Result<()> { +fn run_api(args: &[String], _verbose: u8) -> Result { // gh api is an explicit/advanced command — the user knows what they asked for. // Converting JSON to a schema destroys all values and forces Claude to re-fetch. // Passthrough preserves the full response and tracks metrics at 0% savings. run_passthrough("gh", "api", args) } -/// Pass through a command with base args + extra args, tracking as passthrough. -fn run_passthrough_with_extra(cmd: &str, base_args: &[&str], extra_args: &[String]) -> Result<()> { - let timer = tracking::TimedExecution::start(); - - let mut command = resolved_command(cmd); - for arg in base_args { - command.arg(arg); - } - for arg in extra_args { - command.arg(arg); - } - - let status = - command - .status() - .context(format!("Failed to run {} {}", cmd, base_args.join(" ")))?; - - let full_cmd = format!( - "{} {} {}", - cmd, - base_args.join(" "), - tracking::args_display(&extra_args.iter().map(|s| s.into()).collect::>()) - ); - timer.track_passthrough(&full_cmd, &format!("rtk {} (passthrough)", full_cmd)); - - if !status.success() { - std::process::exit(status.code().unwrap_or(1)); - } - - Ok(()) +// Edge case: error context is now "Failed to run {cmd}" (loses subcommand detail) +fn run_passthrough_with_extra(cmd: &str, base_args: &[&str], extra_args: &[String]) -> Result { + let mut os_args: Vec = + base_args.iter().map(std::ffi::OsString::from).collect(); + os_args.extend(extra_args.iter().map(std::ffi::OsString::from)); + crate::core::runner::run_passthrough(cmd, &os_args, 0) } -fn run_passthrough(cmd: &str, subcommand: &str, args: &[String]) -> Result<()> { - let timer = tracking::TimedExecution::start(); - - let mut command = resolved_command(cmd); - command.arg(subcommand); - for arg in args { - command.arg(arg); - } - - let status = command - .status() - .context(format!("Failed to run {} {}", cmd, subcommand))?; - - let args_str = tracking::args_display(&args.iter().map(|s| s.into()).collect::>()); - timer.track_passthrough( - &format!("{} {} {}", cmd, subcommand, args_str), - &format!("rtk {} {} {} (passthrough)", cmd, subcommand, args_str), - ); - - if !status.success() { - std::process::exit(status.code().unwrap_or(1)); - } - - Ok(()) +fn run_passthrough(cmd: &str, subcommand: &str, args: &[String]) -> Result { + let mut os_args: Vec = vec![std::ffi::OsString::from(subcommand)]; + os_args.extend(args.iter().map(std::ffi::OsString::from)); + crate::core::runner::run_passthrough(cmd, &os_args, 0) } #[cfg(test)] @@ -1519,6 +1174,67 @@ mod tests { assert!(should_passthrough_pr_view(&["--comments".into()])); } + #[test] + fn test_should_passthrough_pr_status_help() { + assert!(should_passthrough_pr_status(&["--help".into()])); + assert!(should_passthrough_pr_status(&["-h".into()])); + } + + #[test] + fn test_should_passthrough_pr_status_output_transform_flags() { + assert!(should_passthrough_pr_status(&["--web".into()])); + assert!(should_passthrough_pr_status(&[ + "--jq".into(), + ".currentBranch".into() + ])); + assert!(should_passthrough_pr_status(&[ + "--template".into(), + "{{.currentBranch.title}}".into() + ])); + } + + #[test] + fn test_should_passthrough_pr_status_repo_flag_stays_filtered() { + assert!(!should_passthrough_pr_status(&[ + "-R".into(), + "owner/repo".into() + ])); + } + + #[test] + fn test_pr_status_json_fields_excludes_current_branch() { + let fields = pr_status_json_fields(); + assert!(!fields.contains("currentBranch")); + assert!(fields.contains("number")); + assert!(fields.contains("title")); + assert!(fields.contains("reviewDecision")); + assert!(fields.contains("statusCheckRollup")); + } + + #[test] + fn test_format_pr_status_includes_current_branch_summary() { + let json = serde_json::json!({ + "currentBranch": { + "number": 934, + "title": "fix wrappers for standardization and exit codes", + "reviewDecision": "CHANGES_REQUESTED", + "statusCheckRollup": [ + {"conclusion": "SUCCESS"}, + {"state": "SUCCESS"}, + {"conclusion": "FAILURE"} + ] + }, + "createdBy": [] + }); + + let result = format_pr_status(&json); + assert!(result.contains("Current Branch")); + assert!(result.contains("#934")); + assert!(result.contains("CHANGES_REQUESTED")); + assert!(result.contains("checks 2/3")); + assert!(result.contains("fail 1")); + } + // --- should_passthrough_issue_view tests --- #[test] diff --git a/src/cmds/git/git.rs b/src/cmds/git/git.rs index 1ed848d63..f32c825b4 100644 --- a/src/cmds/git/git.rs +++ b/src/cmds/git/git.rs @@ -1,8 +1,10 @@ //! Filters git output — log, status, diff, and more — keeping just the essential info. use crate::core::config; +use crate::core::stream::exec_capture; use crate::core::tracking; -use crate::core::utils::resolved_command; +use crate::core::utils::{exit_code_from_output, exit_code_from_status, resolved_command}; +use std::process::Stdio; use anyhow::{Context, Result}; use std::ffi::OsString; use std::process::Command; @@ -39,7 +41,7 @@ pub fn run( max_lines: Option, verbose: u8, global_args: &[String], -) -> Result<()> { +) -> Result { match cmd { GitCommand::Diff => run_diff(args, max_lines, verbose, global_args), GitCommand::Log => run_log(args, max_lines, verbose, global_args), @@ -58,14 +60,76 @@ pub fn run( } } +/// Re-insert `--` before the first path-like argument when clap has consumed it. +/// +/// clap's `trailing_var_arg = true` silently drops `--` when it appears as the +/// first positional argument (before any other positional). This means: +/// `rtk git diff -- file` → args = ["file"] (clap ate `--`) +/// `rtk git diff HEAD -- file` → args = ["HEAD", "--", "file"] (preserved) +/// +/// Without the `--` separator git may treat an unambiguous path as a revision and +/// emit "fatal: ambiguous argument". We re-insert `--` before the first path-like +/// argument; see `normalize_diff_args_impl` for the detection rules. +fn normalize_diff_args(args: &[String]) -> Vec { + normalize_diff_args_impl(args, |p| std::path::Path::new(p).exists()) +} + +/// Testable core of `normalize_diff_args` — accepts an injectable filesystem existence checker. +/// +/// The path-detection logic is: +/// 1. Explicit path prefixes (`.`, `~`) → always a path, no filesystem check needed. +/// 2. Contains path separator (`/`, `\`) → use `path_exists` to distinguish branch names +/// (e.g. `feature/auth`) from real paths (e.g. `src/main.rs`). +/// 3. Bare word with no separator → never a path (avoids injecting `--` when a file +/// happens to share a name with a branch or ref, e.g. a file named `main`). +fn normalize_diff_args_impl(args: &[String], path_exists: F) -> Vec +where + F: Fn(&str) -> bool, +{ + // Already has `--` — nothing to do + if args.iter().any(|a| a == "--") { + return args.to_vec(); + } + let path_start = args.iter().position(|arg| { + if arg.starts_with('-') { + return false; + } + // Explicit path prefixes — always treat as path regardless of existence + if arg.starts_with('.') || arg.starts_with('~') { + return true; + } + // Contains path separator — use filesystem check to distinguish + // branch names (feature/auth) from real paths (src/main.rs) + if arg.contains('/') || arg.contains('\\') { + return path_exists(arg); + } + // Bare word (no separator, no special prefix) — never inject `--` + // This avoids misidentifying a ref/branch as a path even if a same-named + // file happens to exist on disk. + false + }); + match path_start { + Some(idx) => { + let mut out = args[..idx].to_vec(); + out.push("--".to_string()); + out.extend_from_slice(&args[idx..]); + out + } + None => args.to_vec(), + } +} + fn run_diff( args: &[String], max_lines: Option, verbose: u8, global_args: &[String], -) -> Result<()> { +) -> Result { let timer = tracking::TimedExecution::start(); + // Re-insert `--` when clap's trailing_var_arg consumed it (issue #1215) + let args = &normalize_diff_args(args); + // Check if user wants stat output let wants_stat = args .iter() @@ -85,25 +149,23 @@ fn run_diff( cmd.arg(arg); } - let output = cmd.output().context("Failed to run git diff")?; + let result = exec_capture(&mut cmd).context("Failed to run git diff")?; - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr); - eprintln!("{}", stderr); - std::process::exit(output.status.code().unwrap_or(1)); + if !result.success() { + eprintln!("{}", result.stderr); + return Ok(result.exit_code); } - let stdout = String::from_utf8_lossy(&output.stdout); - println!("{}", stdout.trim()); + println!("{}", result.stdout.trim()); timer.track( &format!("git diff {}", args.join(" ")), &format!("rtk git diff {} (passthrough)", args.join(" ")), - &stdout, - &stdout, + &result.stdout, + &result.stdout, ); - return Ok(()); + return Ok(0); } // Default RTK behavior: stat first, then compacted diff @@ -114,22 +176,19 @@ fn run_diff( cmd.arg(arg); } - let output = cmd.output().context("Failed to run git diff")?; - let stat_stdout = String::from_utf8_lossy(&output.stdout); + let result = exec_capture(&mut cmd).context("Failed to run git diff")?; - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr); - if !stderr.trim().is_empty() { - eprint!("{}", stderr); + if !result.success() { + if !result.stderr.trim().is_empty() { + eprint!("{}", result.stderr); } - let raw = stat_stdout.to_string(); timer.track( &format!("git diff {}", args.join(" ")), &format!("rtk git diff {}", args.join(" ")), - &raw, - &raw, + &result.stdout, + &result.stdout, ); - std::process::exit(output.status.code().unwrap_or(1)); + return Ok(result.exit_code); } if verbose > 0 { @@ -137,7 +196,7 @@ fn run_diff( } // Print stat summary first - println!("{}", stat_stdout.trim()); + println!("{}", result.stdout.trim()); // Now get actual diff but compact it let mut diff_cmd = git_cmd(global_args); @@ -146,13 +205,12 @@ fn run_diff( diff_cmd.arg(arg); } - let diff_output = diff_cmd.output().context("Failed to run git diff")?; - let diff_stdout = String::from_utf8_lossy(&diff_output.stdout); + let diff_result = exec_capture(&mut diff_cmd).context("Failed to run git diff")?; - let mut final_output = stat_stdout.to_string(); - if !diff_stdout.is_empty() { + let mut final_output = result.stdout.clone(); + if !diff_result.stdout.is_empty() { println!("\n--- Changes ---"); - let compacted = compact_diff(&diff_stdout, max_lines.unwrap_or(500)); + let compacted = compact_diff(&diff_result.stdout, max_lines.unwrap_or(500)); println!("{}", compacted); final_output.push_str("\n--- Changes ---\n"); final_output.push_str(&compacted); @@ -161,11 +219,11 @@ fn run_diff( timer.track( &format!("git diff {}", args.join(" ")), &format!("rtk git diff {}", args.join(" ")), - &format!("{}\n{}", stat_stdout, diff_stdout), + &format!("{}\n{}", result.stdout, diff_result.stdout), &final_output, ); - Ok(()) + Ok(0) } fn run_show( @@ -173,7 +231,7 @@ fn run_show( max_lines: Option, verbose: u8, global_args: &[String], -) -> Result<()> { +) -> Result { let timer = tracking::TimedExecution::start(); // If user wants --stat or --format only, pass through @@ -195,27 +253,25 @@ fn run_show( for arg in args { cmd.arg(arg); } - let output = cmd.output().context("Failed to run git show")?; - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr); - eprintln!("{}", stderr); - std::process::exit(output.status.code().unwrap_or(1)); + let result = exec_capture(&mut cmd).context("Failed to run git show")?; + if !result.success() { + eprintln!("{}", result.stderr); + return Ok(result.exit_code); } - let stdout = String::from_utf8_lossy(&output.stdout); if wants_blob_show { - print!("{}", stdout); + print!("{}", result.stdout); } else { - println!("{}", stdout.trim()); + println!("{}", result.stdout.trim()); } timer.track( &format!("git show {}", args.join(" ")), &format!("rtk git show {} (passthrough)", args.join(" ")), - &stdout, - &stdout, + &result.stdout, + &result.stdout, ); - return Ok(()); + return Ok(0); } // Get raw output for tracking @@ -224,9 +280,8 @@ fn run_show( for arg in args { raw_cmd.arg(arg); } - let raw_output = raw_cmd - .output() - .map(|o| String::from_utf8_lossy(&o.stdout).to_string()) + let raw_output = exec_capture(&mut raw_cmd) + .map(|r| r.stdout) .unwrap_or_default(); // Step 1: one-line commit summary @@ -235,14 +290,12 @@ fn run_show( for arg in args { summary_cmd.arg(arg); } - let summary_output = summary_cmd.output().context("Failed to run git show")?; - if !summary_output.status.success() { - let stderr = String::from_utf8_lossy(&summary_output.stderr); - eprintln!("{}", stderr); - std::process::exit(summary_output.status.code().unwrap_or(1)); + let summary_result = exec_capture(&mut summary_cmd).context("Failed to run git show")?; + if !summary_result.success() { + eprintln!("{}", summary_result.stderr); + return Ok(summary_result.exit_code); } - let summary = String::from_utf8_lossy(&summary_output.stdout); - println!("{}", summary.trim()); + println!("{}", summary_result.stdout.trim()); // Step 2: --stat summary let mut stat_cmd = git_cmd(global_args); @@ -250,9 +303,8 @@ fn run_show( for arg in args { stat_cmd.arg(arg); } - let stat_output = stat_cmd.output().context("Failed to run git show --stat")?; - let stat_stdout = String::from_utf8_lossy(&stat_output.stdout); - let stat_text = stat_stdout.trim(); + let stat_result = exec_capture(&mut stat_cmd).context("Failed to run git show --stat")?; + let stat_text = stat_result.stdout.trim(); if !stat_text.is_empty() { println!("{}", stat_text); } @@ -263,11 +315,10 @@ fn run_show( for arg in args { diff_cmd.arg(arg); } - let diff_output = diff_cmd.output().context("Failed to run git show (diff)")?; - let diff_stdout = String::from_utf8_lossy(&diff_output.stdout); - let diff_text = diff_stdout.trim(); + let diff_result = exec_capture(&mut diff_cmd).context("Failed to run git show (diff)")?; + let diff_text = diff_result.stdout.trim(); - let mut final_output = summary.to_string(); + let mut final_output = summary_result.stdout.clone(); if !diff_text.is_empty() { if verbose > 0 { println!("\n--- Changes ---"); @@ -284,7 +335,7 @@ fn run_show( &final_output, ); - Ok(()) + Ok(0) } fn is_blob_show_arg(arg: &str) -> bool { @@ -329,8 +380,9 @@ pub(crate) fn compact_diff(diff: &str, max_lines: usize) -> String { } in_hunk = true; hunk_shown = 0; - let hunk_info = line.split("@@").nth(1).unwrap_or("").trim(); - result.push(format!(" @@ {} @@", hunk_info)); + // Preserve the full unified diff hunk header, including trailing + // function / symbol context after the second @@ marker. + result.push(format!(" {}", line)); } else if in_hunk { if line.starts_with('+') && !line.starts_with("+++") { added += 1; @@ -386,7 +438,7 @@ fn run_log( _max_lines: Option, verbose: u8, global_args: &[String], -) -> Result<()> { +) -> Result { let timer = tracking::TimedExecution::start(); let mut cmd = git_cmd(global_args); @@ -439,33 +491,29 @@ fn run_log( cmd.arg(arg); } - let output = cmd.output().context("Failed to run git log")?; + let result = exec_capture(&mut cmd).context("Failed to run git log")?; - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr); - eprintln!("{}", stderr); - // Propagate git's exit code - std::process::exit(output.status.code().unwrap_or(1)); + if !result.success() { + eprintln!("{}", result.stderr); + return Ok(result.exit_code); } - let stdout = String::from_utf8_lossy(&output.stdout); - if verbose > 0 { eprintln!("Git log output:"); } // Post-process: truncate long messages, cap lines only if RTK set the default - let filtered = filter_log_output(&stdout, limit, user_set_limit, has_format_flag); + let filtered = filter_log_output(&result.stdout, limit, user_set_limit, has_format_flag); println!("{}", filtered); timer.track( &format!("git log {}", args.join(" ")), &format!("rtk git log {}", args.join(" ")), - &stdout, + &result.stdout, &filtered, ); - Ok(()) + Ok(0) } /// Filter git log output: truncate long messages, cap lines @@ -513,7 +561,7 @@ fn parse_user_limit(args: &[String]) -> Option { /// so we skip line capping (git already returns exactly N commits) and use a /// wider truncation threshold (120 chars) to preserve commit context that LLMs /// need for rebase/squash operations. -fn filter_log_output( +pub(crate) fn filter_log_output( output: &str, limit: usize, user_set_limit: bool, @@ -589,8 +637,7 @@ fn truncate_line(line: &str, width: usize) -> String { } } -/// Format porcelain output into compact RTK status display -fn format_status_output(porcelain: &str) -> String { +pub(crate) fn format_status_output(porcelain: &str) -> String { let lines: Vec<&str> = porcelain.lines().collect(); if lines.is_empty() { @@ -740,86 +787,76 @@ fn filter_status_with_args(output: &str) -> String { } } -fn run_status(args: &[String], verbose: u8, global_args: &[String]) -> Result<()> { +fn run_status(args: &[String], verbose: u8, global_args: &[String]) -> Result { let timer = tracking::TimedExecution::start(); // If user provided flags, apply minimal filtering if !args.is_empty() { - let output = git_cmd(global_args) - .arg("status") - .args(args) - .output() - .context("Failed to run git status")?; - - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); + let mut cmd = git_cmd(global_args); + cmd.arg("status").args(args); + let result = exec_capture(&mut cmd).context("Failed to run git status")?; - if !output.status.success() { - if !stderr.trim().is_empty() { - eprint!("{}", stderr); + if !result.success() { + if !result.stderr.trim().is_empty() { + eprint!("{}", result.stderr); } - let raw = stdout.to_string(); timer.track( &format!("git status {}", args.join(" ")), &format!("rtk git status {}", args.join(" ")), - &raw, - &raw, + &result.stdout, + &result.stdout, ); - std::process::exit(output.status.code().unwrap_or(1)); + return Ok(result.exit_code); } - if verbose > 0 || !stderr.is_empty() { - eprint!("{}", stderr); + if verbose > 0 || !result.stderr.is_empty() { + eprint!("{}", result.stderr); } // Apply minimal filtering: strip ANSI, remove hints, empty lines - let filtered = filter_status_with_args(&stdout); + let filtered = filter_status_with_args(&result.stdout); print!("{}", filtered); timer.track( &format!("git status {}", args.join(" ")), &format!("rtk git status {}", args.join(" ")), - &stdout, + &result.stdout, &filtered, ); - return Ok(()); + return Ok(0); } // Default RTK compact mode (no args provided) // Get raw git status for tracking - let raw_output = git_cmd(global_args) - .args(["status"]) - .output() - .map(|o| String::from_utf8_lossy(&o.stdout).to_string()) + let mut raw_cmd = git_cmd(global_args); + raw_cmd.args(["status"]); + let raw_output = exec_capture(&mut raw_cmd) + .map(|r| r.stdout) .unwrap_or_default(); - let output = git_cmd(global_args) - .args(["status", "--porcelain", "-b"]) - .output() - .context("Failed to run git status")?; - - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); + let mut cmd = git_cmd(global_args); + cmd.args(["status", "--porcelain", "-b"]); + let result = exec_capture(&mut cmd).context("Failed to run git status")?; - if !stderr.is_empty() && stderr.contains("not a git repository") { + if !result.stderr.is_empty() && result.stderr.contains("not a git repository") { let message = "Not a git repository".to_string(); eprintln!("{}", message); timer.track("git status", "rtk git status", &raw_output, &message); - std::process::exit(output.status.code().unwrap_or(128)); + return Ok(result.exit_code); } - let formatted = format_status_output(&stdout); + let formatted = format_status_output(&result.stdout); println!("{}", formatted); // Track for statistics timer.track("git status", "rtk git status", &raw_output, &formatted); - Ok(()) + Ok(0) } -fn run_add(args: &[String], verbose: u8, global_args: &[String]) -> Result<()> { +fn run_add(args: &[String], verbose: u8, global_args: &[String]) -> Result { let timer = tracking::TimedExecution::start(); let mut cmd = git_cmd(global_args); @@ -834,31 +871,26 @@ fn run_add(args: &[String], verbose: u8, global_args: &[String]) -> Result<()> { } } - let output = cmd.output().context("Failed to run git add")?; + let result = exec_capture(&mut cmd).context("Failed to run git add")?; if verbose > 0 { eprintln!("git add executed"); } - let raw_output = format!( - "{}\n{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); + let raw_output = format!("{}\n{}", result.stdout, result.stderr); - if output.status.success() { + if result.success() { // Count what was added - let status_output = git_cmd(global_args) - .args(["diff", "--cached", "--stat", "--shortstat"]) - .output() - .context("Failed to check staged files")?; + let mut stat_cmd = git_cmd(global_args); + stat_cmd.args(["diff", "--cached", "--stat", "--shortstat"]); + let stat_result = + exec_capture(&mut stat_cmd).context("Failed to check staged files")?; - let stat = String::from_utf8_lossy(&status_output.stdout); - let compact = if stat.trim().is_empty() { + let compact = if stat_result.stdout.trim().is_empty() { "ok (nothing to add)".to_string() } else { // Parse "1 file changed, 5 insertions(+)" format - let short = stat.lines().last().unwrap_or("").trim(); + let short = stat_result.stdout.lines().last().unwrap_or("").trim(); if short.is_empty() { "ok".to_string() } else { @@ -875,20 +907,17 @@ fn run_add(args: &[String], verbose: u8, global_args: &[String]) -> Result<()> { &compact, ); } else { - let stderr = String::from_utf8_lossy(&output.stderr); - let stdout = String::from_utf8_lossy(&output.stdout); eprintln!("FAILED: git add"); - if !stderr.trim().is_empty() { - eprintln!("{}", stderr); + if !result.stderr.trim().is_empty() { + eprintln!("{}", result.stderr); } - if !stdout.trim().is_empty() { - eprintln!("{}", stdout); + if !result.stdout.trim().is_empty() { + eprintln!("{}", result.stdout); } - // Propagate git's exit code - std::process::exit(output.status.code().unwrap_or(1)); + return Ok(result.exit_code); } - Ok(()) + Ok(0) } fn build_commit_command(args: &[String], global_args: &[String]) -> Command { @@ -900,7 +929,7 @@ fn build_commit_command(args: &[String], global_args: &[String]) -> Command { cmd } -fn run_commit(args: &[String], verbose: u8, global_args: &[String]) -> Result<()> { +fn run_commit(args: &[String], verbose: u8, global_args: &[String]) -> Result { let timer = tracking::TimedExecution::start(); let original_cmd = format!("git commit {}", args.join(" ")); @@ -910,11 +939,13 @@ fn run_commit(args: &[String], verbose: u8, global_args: &[String]) -> Result<() } let output = build_commit_command(args, global_args) + .stdin(Stdio::inherit()) .output() .context("Failed to run git commit")?; let stdout = String::from_utf8_lossy(&output.stdout); let stderr = String::from_utf8_lossy(&output.stderr); + let exit_code = exit_code_from_output(&output, "git commit"); let raw_output = format!("{}\n{}", stdout, stderr); if output.status.success() { @@ -937,31 +968,29 @@ fn run_commit(args: &[String], verbose: u8, global_args: &[String]) -> Result<() println!("{}", compact); timer.track(&original_cmd, "rtk git commit", &raw_output, &compact); + } else if stderr.contains("nothing to commit") || stdout.contains("nothing to commit") { + println!("ok (nothing to commit)"); + timer.track( + &original_cmd, + "rtk git commit", + &raw_output, + "ok (nothing to commit)", + ); } else { - if stderr.contains("nothing to commit") || stdout.contains("nothing to commit") { - println!("ok (nothing to commit)"); - timer.track( - &original_cmd, - "rtk git commit", - &raw_output, - "ok (nothing to commit)", - ); - } else { - if !stderr.trim().is_empty() { - eprint!("{}", stderr); - } - if !stdout.trim().is_empty() { - eprint!("{}", stdout); - } - timer.track(&original_cmd, "rtk git commit", &raw_output, &raw_output); - std::process::exit(output.status.code().unwrap_or(1)); + if !stderr.trim().is_empty() { + eprint!("{}", stderr); + } + if !stdout.trim().is_empty() { + eprint!("{}", stdout); } + timer.track(&original_cmd, "rtk git commit", &raw_output, &raw_output); + return Ok(exit_code); } - Ok(()) + Ok(0) } -fn run_push(args: &[String], verbose: u8, global_args: &[String]) -> Result<()> { +fn run_push(args: &[String], verbose: u8, global_args: &[String]) -> Result { let timer = tracking::TimedExecution::start(); if verbose > 0 { @@ -974,7 +1003,7 @@ fn run_push(args: &[String], verbose: u8, global_args: &[String]) -> Result<()> cmd.arg(arg); } - let output = cmd.output().context("Failed to run git push")?; + let output = cmd.stdin(Stdio::inherit()).output().context("Failed to run git push")?; let stderr = String::from_utf8_lossy(&output.stderr); let stdout = String::from_utf8_lossy(&output.stdout); @@ -984,18 +1013,18 @@ fn run_push(args: &[String], verbose: u8, global_args: &[String]) -> Result<()> let compact = if stderr.contains("Everything up-to-date") { "ok (up-to-date)".to_string() } else { - let mut result = String::new(); + let mut push_info = String::new(); for line in stderr.lines() { if line.contains("->") { let parts: Vec<&str> = line.split_whitespace().collect(); if parts.len() >= 3 { - result = format!("ok {}", parts[parts.len() - 1]); + push_info = format!("ok {}", parts[parts.len() - 1]); break; } } } - if !result.is_empty() { - result + if !push_info.is_empty() { + push_info } else { "ok".to_string() } @@ -1017,13 +1046,13 @@ fn run_push(args: &[String], verbose: u8, global_args: &[String]) -> Result<()> if !stdout.trim().is_empty() { eprintln!("{}", stdout); } - std::process::exit(output.status.code().unwrap_or(1)); + return Ok(exit_code_from_output(&output, "git push")); } - Ok(()) + Ok(0) } -fn run_pull(args: &[String], verbose: u8, global_args: &[String]) -> Result<()> { +fn run_pull(args: &[String], verbose: u8, global_args: &[String]) -> Result { let timer = tracking::TimedExecution::start(); if verbose > 0 { @@ -1036,56 +1065,55 @@ fn run_pull(args: &[String], verbose: u8, global_args: &[String]) -> Result<()> cmd.arg(arg); } - let output = cmd.output().context("Failed to run git pull")?; + let result = exec_capture(&mut cmd).context("Failed to run git pull")?; - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let raw_output = format!("{}\n{}", stdout, stderr); + let raw_output = format!("{}\n{}", result.stdout, result.stderr); - if output.status.success() { - let compact = - if stdout.contains("Already up to date") || stdout.contains("Already up-to-date") { - "ok (up-to-date)".to_string() - } else { - // Count files changed - let mut files = 0; - let mut insertions = 0; - let mut deletions = 0; - - for line in stdout.lines() { - if line.contains("file") && line.contains("changed") { - // Parse "3 files changed, 10 insertions(+), 2 deletions(-)" - for part in line.split(',') { - let part = part.trim(); - if part.contains("file") { - files = part - .split_whitespace() - .next() - .and_then(|n| n.parse().ok()) - .unwrap_or(0); - } else if part.contains("insertion") { - insertions = part - .split_whitespace() - .next() - .and_then(|n| n.parse().ok()) - .unwrap_or(0); - } else if part.contains("deletion") { - deletions = part - .split_whitespace() - .next() - .and_then(|n| n.parse().ok()) - .unwrap_or(0); - } + if result.success() { + let compact = if result.stdout.contains("Already up to date") + || result.stdout.contains("Already up-to-date") + { + "ok (up-to-date)".to_string() + } else { + // Count files changed + let mut files = 0; + let mut insertions = 0; + let mut deletions = 0; + + for line in result.stdout.lines() { + if line.contains("file") && line.contains("changed") { + // Parse "3 files changed, 10 insertions(+), 2 deletions(-)" + for part in line.split(',') { + let part = part.trim(); + if part.contains("file") { + files = part + .split_whitespace() + .next() + .and_then(|n| n.parse().ok()) + .unwrap_or(0); + } else if part.contains("insertion") { + insertions = part + .split_whitespace() + .next() + .and_then(|n| n.parse().ok()) + .unwrap_or(0); + } else if part.contains("deletion") { + deletions = part + .split_whitespace() + .next() + .and_then(|n| n.parse().ok()) + .unwrap_or(0); } } } + } - if files > 0 { - format!("ok {} files +{} -{}", files, insertions, deletions) - } else { - "ok".to_string() - } - }; + if files > 0 { + format!("ok {} files +{} -{}", files, insertions, deletions) + } else { + "ok".to_string() + } + }; println!("{}", compact); @@ -1097,19 +1125,19 @@ fn run_pull(args: &[String], verbose: u8, global_args: &[String]) -> Result<()> ); } else { eprintln!("FAILED: git pull"); - if !stderr.trim().is_empty() { - eprintln!("{}", stderr); + if !result.stderr.trim().is_empty() { + eprintln!("{}", result.stderr); } - if !stdout.trim().is_empty() { - eprintln!("{}", stdout); + if !result.stdout.trim().is_empty() { + eprintln!("{}", result.stdout); } - std::process::exit(output.status.code().unwrap_or(1)); + return Ok(result.exit_code); } - Ok(()) + Ok(0) } -fn run_branch(args: &[String], verbose: u8, global_args: &[String]) -> Result<()> { +fn run_branch(args: &[String], verbose: u8, global_args: &[String]) -> Result { let timer = tracking::TimedExecution::start(); if verbose > 0 { @@ -1156,19 +1184,17 @@ fn run_branch(args: &[String], verbose: u8, global_args: &[String]) -> Result<() // Detect positional arguments (not flags) — indicates branch creation let has_positional_arg = args.iter().any(|a| !a.starts_with('-')); - // --show-current: passthrough with raw stdout (not "ok ✓") + // --show-current: passthrough with raw stdout (not "ok") if has_show_flag { let mut cmd = git_cmd(global_args); cmd.arg("branch"); for arg in args { cmd.arg(arg); } - let output = cmd.output().context("Failed to run git branch")?; - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let combined = format!("{}{}", stdout, stderr); + let result = exec_capture(&mut cmd).context("Failed to run git branch")?; + let combined = result.combined(); - let trimmed = stdout.trim(); + let trimmed = result.stdout.trim(); timer.track( &format!("git branch {}", args.join(" ")), &format!("rtk git branch {}", args.join(" ")), @@ -1176,16 +1202,16 @@ fn run_branch(args: &[String], verbose: u8, global_args: &[String]) -> Result<() trimmed, ); - if output.status.success() { + if result.success() { println!("{}", trimmed); } else { eprintln!("FAILED: git branch {}", args.join(" ")); - if !stderr.trim().is_empty() { - eprintln!("{}", stderr); + if !result.stderr.trim().is_empty() { + eprintln!("{}", result.stderr); } - std::process::exit(output.status.code().unwrap_or(1)); + return Ok(result.exit_code); } - return Ok(()); + return Ok(0); } // Write operation: action flags, or positional args without list flags (= branch creation) @@ -1195,12 +1221,10 @@ fn run_branch(args: &[String], verbose: u8, global_args: &[String]) -> Result<() for arg in args { cmd.arg(arg); } - let output = cmd.output().context("Failed to run git branch")?; - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let combined = format!("{}{}", stdout, stderr); + let result = exec_capture(&mut cmd).context("Failed to run git branch")?; + let combined = result.combined(); - let msg = if output.status.success() { + let msg = if result.success() { "ok" } else { &combined @@ -1213,19 +1237,19 @@ fn run_branch(args: &[String], verbose: u8, global_args: &[String]) -> Result<() msg, ); - if output.status.success() { + if result.success() { println!("ok"); } else { eprintln!("FAILED: git branch {}", args.join(" ")); - if !stderr.trim().is_empty() { - eprintln!("{}", stderr); + if !result.stderr.trim().is_empty() { + eprintln!("{}", result.stderr); } - if !stdout.trim().is_empty() { - eprintln!("{}", stdout); + if !result.stdout.trim().is_empty() { + eprintln!("{}", result.stdout); } - std::process::exit(output.status.code().unwrap_or(1)); + return Ok(result.exit_code); } - return Ok(()); + return Ok(0); } // List mode: show compact branch list @@ -1239,41 +1263,39 @@ fn run_branch(args: &[String], verbose: u8, global_args: &[String]) -> Result<() cmd.arg(arg); } - let output = cmd.output().context("Failed to run git branch")?; - let stdout = String::from_utf8_lossy(&output.stdout); - let raw = stdout.to_string(); + let result = exec_capture(&mut cmd).context("Failed to run git branch")?; - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr); - if !stderr.trim().is_empty() { - eprint!("{}", stderr); + if !result.success() { + if !result.stderr.trim().is_empty() { + eprint!("{}", result.stderr); } timer.track( &format!("git branch {}", args.join(" ")), &format!("rtk git branch {}", args.join(" ")), - &raw, - &raw, + &result.stdout, + &result.stdout, ); - std::process::exit(output.status.code().unwrap_or(1)); + return Ok(result.exit_code); } - let filtered = filter_branch_output(&stdout); + let filtered = filter_branch_output(&result.stdout); println!("{}", filtered); timer.track( &format!("git branch {}", args.join(" ")), &format!("rtk git branch {}", args.join(" ")), - &raw, + &result.stdout, &filtered, ); - Ok(()) + Ok(0) } fn filter_branch_output(output: &str) -> String { let mut current = String::new(); let mut local: Vec = Vec::new(); let mut remote: Vec = Vec::new(); + let mut seen_remote: std::collections::HashSet = std::collections::HashSet::new(); for line in output.lines() { let line = line.trim(); @@ -1283,13 +1305,16 @@ fn filter_branch_output(output: &str) -> String { if let Some(branch) = line.strip_prefix("* ") { current = branch.to_string(); - } else if line.starts_with("remotes/origin/") { - let branch = line.strip_prefix("remotes/origin/").unwrap_or(line); - // Skip HEAD pointer - if branch.starts_with("HEAD ") { - continue; + } else if let Some(rest) = line.strip_prefix("remotes/") { + if let Some(slash_pos) = rest.find('/') { + let branch = &rest[slash_pos + 1..]; + if branch.starts_with("HEAD ") { + continue; + } + if seen_remote.insert(branch.to_string()) { + remote.push(branch.to_string()); + } } - remote.push(branch.to_string()); } else { local.push(line.to_string()); } @@ -1305,7 +1330,6 @@ fn filter_branch_output(output: &str) -> String { } if !remote.is_empty() { - // Filter out remotes that already exist locally let remote_only: Vec<&String> = remote .iter() .filter(|r| *r != ¤t && !local.contains(r)) @@ -1324,7 +1348,7 @@ fn filter_branch_output(output: &str) -> String { result.join("\n") } -fn run_fetch(args: &[String], verbose: u8, global_args: &[String]) -> Result<()> { +fn run_fetch(args: &[String], verbose: u8, global_args: &[String]) -> Result { let timer = tracking::TimedExecution::start(); if verbose > 0 { @@ -1337,21 +1361,20 @@ fn run_fetch(args: &[String], verbose: u8, global_args: &[String]) -> Result<()> cmd.arg(arg); } - let output = cmd.output().context("Failed to run git fetch")?; - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let raw = format!("{}{}", stdout, stderr); + let result = exec_capture(&mut cmd).context("Failed to run git fetch")?; + let raw = result.combined(); - if !output.status.success() { + if !result.success() { eprintln!("FAILED: git fetch"); - if !stderr.trim().is_empty() { - eprintln!("{}", stderr); + if !result.stderr.trim().is_empty() { + eprintln!("{}", result.stderr); } - std::process::exit(output.status.code().unwrap_or(1)); + return Ok(result.exit_code); } // Count new refs from stderr (git fetch outputs to stderr) - let new_refs: usize = stderr + let new_refs: usize = result + .stderr .lines() .filter(|l| l.contains("->") || l.contains("[new")) .count(); @@ -1365,7 +1388,7 @@ fn run_fetch(args: &[String], verbose: u8, global_args: &[String]) -> Result<()> println!("{}", msg); timer.track("git fetch", "rtk git fetch", &raw, &msg); - Ok(()) + Ok(0) } fn run_stash( @@ -1373,7 +1396,7 @@ fn run_stash( args: &[String], verbose: u8, global_args: &[String], -) -> Result<()> { +) -> Result { let timer = tracking::TimedExecution::start(); if verbose > 0 { @@ -1382,23 +1405,26 @@ fn run_stash( match subcommand { Some("list") => { - let output = git_cmd(global_args) - .args(["stash", "list"]) - .output() - .context("Failed to run git stash list")?; - let stdout = String::from_utf8_lossy(&output.stdout); - let raw = stdout.to_string(); - - if stdout.trim().is_empty() { + let mut cmd = git_cmd(global_args); + cmd.args(["stash", "list"]); + let result = + exec_capture(&mut cmd).context("Failed to run git stash list")?; + + if result.stdout.trim().is_empty() { let msg = "No stashes"; println!("{}", msg); - timer.track("git stash list", "rtk git stash list", &raw, msg); - return Ok(()); + timer.track("git stash list", "rtk git stash list", &result.stdout, msg); + return Ok(0); } - let filtered = filter_stash_list(&stdout); + let filtered = filter_stash_list(&result.stdout); println!("{}", filtered); - timer.track("git stash list", "rtk git stash list", &raw, &filtered); + timer.track( + "git stash list", + "rtk git stash list", + &result.stdout, + &filtered, + ); } Some("show") => { let mut cmd = git_cmd(global_args); @@ -1406,21 +1432,25 @@ fn run_stash( for arg in args { cmd.arg(arg); } - let output = cmd.output().context("Failed to run git stash show")?; - let stdout = String::from_utf8_lossy(&output.stdout); - let raw = stdout.to_string(); + let result = + exec_capture(&mut cmd).context("Failed to run git stash show")?; - let filtered = if stdout.trim().is_empty() { + let filtered = if result.stdout.trim().is_empty() { let msg = "Empty stash"; println!("{}", msg); msg.to_string() } else { - let compacted = compact_diff(&stdout, 100); + let compacted = compact_diff(&result.stdout, 100); println!("{}", compacted); compacted }; - timer.track("git stash show", "rtk git stash show", &raw, &filtered); + timer.track( + "git stash show", + "rtk git stash show", + &result.stdout, + &filtered, + ); } Some("pop") | Some("apply") | Some("drop") | Some("push") => { let sub = subcommand.unwrap(); @@ -1429,19 +1459,17 @@ fn run_stash( for arg in args { cmd.arg(arg); } - let output = cmd.output().context("Failed to run git stash")?; - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let combined = format!("{}{}", stdout, stderr); + let result = exec_capture(&mut cmd).context("Failed to run git stash")?; + let combined = result.combined(); - let msg = if output.status.success() { + let msg = if result.success() { let msg = format!("ok stash {}", sub); println!("{}", msg); msg } else { eprintln!("FAILED: git stash {}", sub); - if !stderr.trim().is_empty() { - eprintln!("{}", stderr); + if !result.stderr.trim().is_empty() { + eprintln!("{}", result.stderr); } combined.clone() }; @@ -1453,8 +1481,8 @@ fn run_stash( &msg, ); - if !output.status.success() { - std::process::exit(output.status.code().unwrap_or(1)); + if !result.success() { + return Ok(result.exit_code); } } Some(sub) => { @@ -1464,19 +1492,17 @@ fn run_stash( for arg in args { cmd.arg(arg); } - let output = cmd.output().context("Failed to run git stash")?; - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let combined = format!("{}{}", stdout, stderr); + let result = exec_capture(&mut cmd).context("Failed to run git stash")?; + let combined = result.combined(); - let msg = if output.status.success() { + let msg = if result.success() { let msg = format!("ok stash {}", sub); println!("{}", msg); msg } else { eprintln!("FAILED: git stash {}", sub); - if !stderr.trim().is_empty() { - eprintln!("{}", stderr); + if !result.stderr.trim().is_empty() { + eprintln!("{}", result.stderr); } combined.clone() }; @@ -1488,8 +1514,8 @@ fn run_stash( &msg, ); - if !output.status.success() { - std::process::exit(output.status.code().unwrap_or(1)); + if !result.success() { + return Ok(result.exit_code); } } None => { @@ -1499,13 +1525,11 @@ fn run_stash( for arg in args { cmd.arg(arg); } - let output = cmd.output().context("Failed to run git stash")?; - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let combined = format!("{}{}", stdout, stderr); + let result = exec_capture(&mut cmd).context("Failed to run git stash")?; + let combined = result.combined(); - let msg = if output.status.success() { - if stdout.contains("No local changes") { + let msg = if result.success() { + if result.stdout.contains("No local changes") { let msg = "ok (nothing to stash)"; println!("{}", msg); msg.to_string() @@ -1516,21 +1540,21 @@ fn run_stash( } } else { eprintln!("FAILED: git stash"); - if !stderr.trim().is_empty() { - eprintln!("{}", stderr); + if !result.stderr.trim().is_empty() { + eprintln!("{}", result.stderr); } combined.clone() }; timer.track("git stash", "rtk git stash", &combined, &msg); - if !output.status.success() { - std::process::exit(output.status.code().unwrap_or(1)); + if !result.success() { + return Ok(result.exit_code); } } } - Ok(()) + Ok(0) } fn filter_stash_list(output: &str) -> String { @@ -1554,7 +1578,7 @@ fn filter_stash_list(output: &str) -> String { result.join("\n") } -fn run_worktree(args: &[String], verbose: u8, global_args: &[String]) -> Result<()> { +fn run_worktree(args: &[String], verbose: u8, global_args: &[String]) -> Result { let timer = tracking::TimedExecution::start(); if verbose > 0 { @@ -1572,12 +1596,10 @@ fn run_worktree(args: &[String], verbose: u8, global_args: &[String]) -> Result< for arg in args { cmd.arg(arg); } - let output = cmd.output().context("Failed to run git worktree")?; - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let combined = format!("{}{}", stdout, stderr); + let result = exec_capture(&mut cmd).context("Failed to run git worktree")?; + let combined = result.combined(); - let msg = if output.status.success() { + let msg = if result.success() { "ok" } else { &combined @@ -1590,32 +1612,34 @@ fn run_worktree(args: &[String], verbose: u8, global_args: &[String]) -> Result< msg, ); - if output.status.success() { + if result.success() { println!("ok"); } else { eprintln!("FAILED: git worktree {}", args.join(" ")); - if !stderr.trim().is_empty() { - eprintln!("{}", stderr); + if !result.stderr.trim().is_empty() { + eprintln!("{}", result.stderr); } - std::process::exit(output.status.code().unwrap_or(1)); + return Ok(result.exit_code); } - return Ok(()); + return Ok(0); } // Default: list mode - let output = git_cmd(global_args) - .args(["worktree", "list"]) - .output() - .context("Failed to run git worktree list")?; - - let stdout = String::from_utf8_lossy(&output.stdout); - let raw = stdout.to_string(); + let mut cmd = git_cmd(global_args); + cmd.args(["worktree", "list"]); + let result = + exec_capture(&mut cmd).context("Failed to run git worktree list")?; - let filtered = filter_worktree_list(&stdout); + let filtered = filter_worktree_list(&result.stdout); println!("{}", filtered); - timer.track("git worktree list", "rtk git worktree", &raw, &filtered); + timer.track( + "git worktree list", + "rtk git worktree", + &result.stdout, + &filtered, + ); - Ok(()) + Ok(0) } fn filter_worktree_list(output: &str) -> String { @@ -1646,7 +1670,7 @@ fn filter_worktree_list(output: &str) -> String { } /// Runs an unsupported git subcommand by passing it through directly -pub fn run_passthrough(args: &[OsString], global_args: &[String], verbose: u8) -> Result<()> { +pub fn run_passthrough(args: &[OsString], global_args: &[String], verbose: u8) -> Result { let timer = tracking::TimedExecution::start(); if verbose > 0 { @@ -1664,9 +1688,9 @@ pub fn run_passthrough(args: &[OsString], global_args: &[String], verbose: u8) - ); if !status.success() { - std::process::exit(status.code().unwrap_or(1)); + return Ok(exit_code_from_status(&status, "git")); } - Ok(()) + Ok(0) } #[cfg(test)] @@ -1744,6 +1768,24 @@ mod tests { assert!(result.contains("+")); } + #[test] + fn test_compact_diff_preserves_full_hunk_header_context() { + let diff = r#"diff --git a/foo.rs b/foo.rs +--- a/foo.rs ++++ b/foo.rs +@@ -10,3 +10,4 @@ fn important_context() { + fn main() { ++ println!("hello"); + } +"#; + let result = compact_diff(diff, 100); + assert!( + result.contains("@@ -10,3 +10,4 @@ fn important_context() {"), + "Expected full hunk header with trailing context, got:\n{}", + result + ); + } + #[test] fn test_compact_diff_increased_hunk_limit() { // Build a hunk with 25 changed lines — should NOT be truncated with limit 30 @@ -1778,6 +1820,129 @@ mod tests { ); } + // ----- normalize_diff_args (issue #1215 + branch-name fix #1431) ----- + // + // Tests use normalize_diff_args_impl with a mock path-existence checker so + // they don't depend on the real filesystem. + + fn exists_mock<'a>(existing: &'a [&'a str]) -> impl Fn(&str) -> bool + 'a { + move |p| existing.contains(&p) + } + + /// Baseline: `--` already present → no-op, args unchanged. + #[test] + fn test_normalize_diff_args_noop_when_separator_present() { + let args = vec![ + "HEAD".to_string(), + "--".to_string(), + "src/main.rs".to_string(), + ]; + assert_eq!(normalize_diff_args_impl(&args, exists_mock(&[])), args); + } + + /// Core regression (issue #1215): clap ate `--` before a real file path. + /// When the path exists on disk, `--` must be re-inserted. + #[test] + fn test_normalize_diff_args_reinserts_separator_before_existing_path() { + let args = vec!["apps/client/frontend/src/MyComponent.tsx".to_string()]; + let normalized = normalize_diff_args_impl( + &args, + exists_mock(&["apps/client/frontend/src/MyComponent.tsx"]), + ); + assert_eq!( + normalized, + vec![ + "--".to_string(), + "apps/client/frontend/src/MyComponent.tsx".to_string() + ], + "-- must be injected before an existing path" + ); + } + + /// Ref before path: ["HEAD", "src/foo.rs"] where src/foo.rs exists → inject after HEAD. + #[test] + fn test_normalize_diff_args_reinserts_separator_after_ref() { + let args = vec!["HEAD".to_string(), "src/foo.rs".to_string()]; + let normalized = normalize_diff_args_impl(&args, exists_mock(&["src/foo.rs"])); + assert_eq!( + normalized, + vec!["HEAD".to_string(), "--".to_string(), "src/foo.rs".to_string()] + ); + } + + /// Flags before path: ["--cached", "src/foo.rs"] where src/foo.rs exists. + #[test] + fn test_normalize_diff_args_reinserts_separator_after_flag() { + let args = vec!["--cached".to_string(), "src/foo.rs".to_string()]; + let normalized = normalize_diff_args_impl(&args, exists_mock(&["src/foo.rs"])); + assert_eq!( + normalized, + vec!["--cached".to_string(), "--".to_string(), "src/foo.rs".to_string()] + ); + } + + /// Pure flags (no paths) → no injection. + #[test] + fn test_normalize_diff_args_no_injection_for_pure_flags() { + let args = vec!["--stat".to_string(), "--cached".to_string()]; + assert_eq!(normalize_diff_args_impl(&args, exists_mock(&[])), args); + } + + /// Dotfile that exists on disk → inject `--`. + #[test] + fn test_normalize_diff_args_dotfile_is_path() { + let args = vec![".gitignore".to_string()]; + let normalized = normalize_diff_args_impl(&args, exists_mock(&[".gitignore"])); + assert_eq!( + normalized, + vec!["--".to_string(), ".gitignore".to_string()] + ); + } + + /// A bare ref (HEAD) that doesn't exist as a file → no injection. + #[test] + fn test_normalize_diff_args_no_injection_for_bare_ref() { + let args = vec!["HEAD".to_string()]; + assert_eq!(normalize_diff_args_impl(&args, exists_mock(&[])), args); + } + + /// Branch name with `/` that does NOT exist as a file → no injection. + /// Regression for issue #1431: `rtk git diff feature/user-auth` must not inject `--`. + #[test] + fn test_normalize_diff_args_no_injection_for_branch_with_slash() { + let args = vec!["feature/user-auth".to_string()]; + assert_eq!( + normalize_diff_args_impl(&args, exists_mock(&[])), + args, + "branch names containing '/' must not trigger -- injection" + ); + } + + /// Range syntax with `/` → no injection. + /// Regression: `rtk git diff main...feature/user-auth` produced no output. + #[test] + fn test_normalize_diff_args_no_injection_for_range_with_slash() { + let args = vec!["main...feature/user-auth".to_string()]; + assert_eq!( + normalize_diff_args_impl(&args, exists_mock(&[])), + args, + "revision ranges like main...feature/user-auth must not trigger -- injection" + ); + } + + /// Bare word that happens to exist as a file on disk → still no injection. + /// A file named "main" must not cause `--` to be injected when the user + /// intends `rtk git diff main` as a branch comparison. + #[test] + fn test_normalize_diff_args_no_injection_for_bare_word_even_if_file_exists() { + let args = vec!["main".to_string()]; + assert_eq!( + normalize_diff_args_impl(&args, exists_mock(&["main"])), + args, + "bare words must never trigger -- injection even when a same-named file exists" + ); + } + #[test] fn test_is_blob_show_arg() { assert!(is_blob_show_arg("develop:modules/pairs_backtest.py")); @@ -1808,6 +1973,37 @@ mod tests { assert!(!result.contains("remote-only")); } + #[test] + fn test_filter_branch_multi_remote() { + let output = "* main\n develop\n remotes/origin/HEAD -> origin/main\n remotes/origin/main\n remotes/origin/feature-x\n remotes/upstream/main\n remotes/upstream/release-v3\n remotes/fork/main\n remotes/fork/experiment\n"; + let result = filter_branch_output(output); + assert!(result.contains("* main")); + assert!(result.contains("develop")); + assert!(result.contains("feature-x"), "origin branch shown: {}", result); + assert!( + result.contains("release-v3"), + "upstream branch shown: {}", + result + ); + assert!( + result.contains("experiment"), + "fork branch shown: {}", + result + ); + assert!( + !result.contains("remotes/"), + "remote prefix stripped: {}", + result + ); + let main_count = result.matches("main").count(); + assert!( + main_count <= 2, + "main deduplicated across remotes (found {} occurrences): {}", + main_count, + result + ); + } + #[test] fn test_filter_stash_list() { let output = diff --git a/src/cmds/git/glab_cmd.rs b/src/cmds/git/glab_cmd.rs new file mode 100644 index 000000000..139ed5898 --- /dev/null +++ b/src/cmds/git/glab_cmd.rs @@ -0,0 +1,1535 @@ +//! GitLab CLI (glab) command output compression. +//! +//! Provides token-optimized alternatives to verbose `glab` commands. +//! Mirrors gh_cmd.rs patterns, adapted for glab-specific differences: +//! - MR notation: `!42` (not `#42`) +//! - States: `opened`/`merged`/`closed` (lowercase, not UPPER) +//! - Author: `author.username` (not `author.login`) +//! - URL: `web_url` (not `url`) +//! - Description: `description` (not `body`) +//! - Merge status: `merge_status` ("can_be_merged") (not `mergeable`) +//! - Pipeline: `head_pipeline.status` (not `statusCheckRollup`) + +use super::git; +use crate::core::runner::{self, RunOptions}; +use crate::core::utils::{ok_confirmation, resolved_command, strip_ansi, truncate}; +use anyhow::Result; +use lazy_static::lazy_static; +use regex::Regex; +use serde_json::Value; +use std::process::Command; + +lazy_static! { + static ref HTML_COMMENT_RE: Regex = Regex::new(r"(?s)").unwrap(); + static ref BADGE_LINE_RE: Regex = + Regex::new(r"(?m)^\s*\[!\[[^\]]*\]\([^)]*\)\]\([^)]*\)\s*$").unwrap(); + static ref IMAGE_ONLY_LINE_RE: Regex = Regex::new(r"(?m)^\s*!\[[^\]]*\]\([^)]*\)\s*$").unwrap(); + static ref HORIZONTAL_RULE_RE: Regex = + Regex::new(r"(?m)^\s*(?:---+|\*\*\*+|___+)\s*$").unwrap(); + static ref MULTI_BLANK_RE: Regex = Regex::new(r"\n{3,}").unwrap(); + static ref MR_URL_RE: Regex = Regex::new(r"/-/merge_requests/(\d+)").unwrap(); + /// Match GitLab CI section markers: section_start/end:timestamp:name[0K + static ref SECTION_MARKER_RE: Regex = + Regex::new(r"section_(?:start|end):\d+:[a-z0-9_]+(?:\x1b\[0K|\[0K)*").unwrap(); + /// Match bare bracket ANSI-like codes without ESC prefix: [0K, [0;m, [36;1m, etc. + static ref BARE_ANSI_RE: Regex = Regex::new(r"\[[\d;]+[A-Za-z]").unwrap(); +} + +/// Filter markdown body to remove noise while preserving meaningful content. +/// Removes HTML comments, badge lines, image-only lines, horizontal rules, +/// and collapses excessive blank lines. Preserves code blocks untouched. +fn filter_markdown_body(body: &str) -> String { + if body.is_empty() { + return String::new(); + } + + let mut result = String::new(); + let mut remaining = body; + + loop { + let fence_pos = remaining + .find("```") + .or_else(|| remaining.find("~~~")) + .map(|pos| { + let fence = if remaining[pos..].starts_with("```") { + "```" + } else { + "~~~" + }; + (pos, fence) + }); + + match fence_pos { + Some((start, fence)) => { + let before = &remaining[..start]; + result.push_str(&filter_markdown_segment(before)); + + let after_open = start + fence.len(); + let code_start = remaining[after_open..] + .find('\n') + .map(|p| after_open + p + 1) + .unwrap_or(remaining.len()); + + let close_pos = remaining[code_start..] + .find(fence) + .map(|p| code_start + p + fence.len()); + + match close_pos { + Some(end) => { + result.push_str(&remaining[start..end]); + let after_close = remaining[end..] + .find('\n') + .map(|p| end + p + 1) + .unwrap_or(remaining.len()); + result.push_str(&remaining[end..after_close]); + remaining = &remaining[after_close..]; + } + None => { + result.push_str(&remaining[start..]); + remaining = ""; + } + } + } + None => { + result.push_str(&filter_markdown_segment(remaining)); + break; + } + } + } + + result.trim().to_string() +} + +/// Filter a markdown segment that is NOT inside a code block. +fn filter_markdown_segment(text: &str) -> String { + let mut s = HTML_COMMENT_RE.replace_all(text, "").to_string(); + s = BADGE_LINE_RE.replace_all(&s, "").to_string(); + s = IMAGE_ONLY_LINE_RE.replace_all(&s, "").to_string(); + s = HORIZONTAL_RULE_RE.replace_all(&s, "").to_string(); + s = MULTI_BLANK_RE.replace_all(&s, "\n\n").to_string(); + s +} + +/// State icon for MR/issue states (glab uses lowercase). +fn state_icon(state: &str, ultra_compact: bool) -> &'static str { + if ultra_compact { + match state { + "opened" => "O", + "merged" => "M", + "closed" => "C", + _ => "?", + } + } else { + match state { + "opened" => "[open]", + "merged" => "[merged]", + "closed" => "[closed]", + _ => "?", + } + } +} + +/// Pipeline status icon. Non-compact mode uses text tags for parity with +/// `gh_cmd.rs` (avoids multi-byte terminal rendering quirks; aligns with the +/// rest of the codebase). Ultra-compact keeps single-char density. +fn pipeline_icon(status: &str, ultra_compact: bool) -> &'static str { + if ultra_compact { + match status { + "success" => "+", + "failed" => "x", + "canceled" | "cancelled" => "X", + "running" | "pending" => "~", + "skipped" => "-", + _ => "?", + } + } else { + match status { + "success" => "[ok]", + "failed" => "[fail]", + "canceled" | "cancelled" => "[cancel]", + "running" => "[run]", + "pending" => "[pend]", + "skipped" => "[skip]", + _ => "?", + } + } +} + +/// Extract MR number from glab output URL or text. +fn extract_mr_number(text: &str) -> Option { + MR_URL_RE + .captures(text) + .and_then(|c| c.get(1)) + .map(|m| m.as_str().to_string()) +} + +/// Extract the first positional identifier (MR/issue number or URL) from args, +/// skipping glab flags that take a value. Returns the identifier and remaining args. +fn extract_identifier_and_extra_args(args: &[String]) -> Option<(String, Vec)> { + if args.is_empty() { + return None; + } + + // Known glab flags that take a value — skip these and their values + let flags_with_value = [ + "-R", + "--repo", + "-g", + "--group", + "-F", + "--output", + "-m", + "--message", + ]; + let mut identifier = None; + let mut extra = Vec::new(); + let mut skip_next = false; + + for arg in args { + if skip_next { + extra.push(arg.clone()); + skip_next = false; + continue; + } + if flags_with_value.contains(&arg.as_str()) { + extra.push(arg.clone()); + skip_next = true; + continue; + } + if arg.starts_with('-') { + extra.push(arg.clone()); + continue; + } + // First non-flag arg is the identifier (number/URL) + if identifier.is_none() { + identifier = Some(arg.clone()); + } else { + extra.push(arg.clone()); + } + } + + identifier.map(|id| (id, extra)) +} + +/// Check if user explicitly requested JSON/custom output format. +/// When present, passthrough to avoid double JSON injection. +fn has_output_flag(args: &[String]) -> bool { + args.iter() + .any(|a| a == "--output" || a == "-F" || a == "--json") +} + +/// Check if view subcommand should passthrough (--web, --comments, etc.). +fn should_passthrough_view(extra_args: &[String]) -> bool { + extra_args + .iter() + .any(|a| a == "--web" || a == "--comments" || a == "--output" || a == "-F") +} + +/// Run a glab command that emits JSON and filter through `filter_fn`. +/// On JSON parse failure (glab returns plain text for empty results), +/// fall back to the raw stdout. +fn run_glab_json(cmd: Command, label: &str, filter_fn: F) -> Result +where + F: Fn(&Value) -> String, +{ + runner::run_filtered( + cmd, + "glab", + label, + |stdout| match serde_json::from_str::(stdout) { + Ok(json) => filter_fn(&json), + Err(_) => stdout.to_string(), + }, + RunOptions::stdout_only() + .early_exit_on_failure() + .no_trailing_newline(), + ) +} + +/// Run a glab command with token-optimized output. +pub fn run(subcommand: &str, args: &[String], verbose: u8, ultra_compact: bool) -> Result { + // If the user explicitly requests a specific output format, passthrough unchanged. + if has_output_flag(args) { + return run_passthrough("glab", subcommand, args); + } + + match subcommand { + "mr" => run_mr(args, verbose, ultra_compact), + "issue" => run_issue(args, verbose, ultra_compact), + "ci" | "pipeline" => run_ci(args, verbose, ultra_compact), + "release" => run_release(args, verbose, ultra_compact), + "api" => run_api(args, verbose), + _ => run_passthrough("glab", subcommand, args), + } +} + +// ── MR subcommands ────────────────────────────────────────────────────── + +fn run_mr(args: &[String], verbose: u8, ultra_compact: bool) -> Result { + if args.is_empty() { + return run_passthrough("glab", "mr", args); + } + + match args[0].as_str() { + "list" => mr_list(&args[1..], verbose, ultra_compact), + "view" => mr_view(&args[1..], verbose, ultra_compact), + "create" => mr_create(&args[1..], verbose), + "merge" => mr_action("merge", "merged", &args[1..], verbose), + "approve" => mr_action("approve", "approved", &args[1..], verbose), + "diff" => mr_diff(&args[1..], verbose), + "note" => mr_action("note", "noted", &args[1..], verbose), + "update" => mr_action("update", "updated", &args[1..], verbose), + _ => run_passthrough("glab", "mr", args), + } +} + +/// Format MR list JSON into compact output (pure function, testable). +fn format_mr_list(json: &Value, ultra_compact: bool) -> String { + let mrs = match json.as_array() { + Some(arr) => arr, + None => return String::new(), + }; + if mrs.is_empty() { + return if ultra_compact { + "No MRs\n".to_string() + } else { + "No Merge Requests\n".to_string() + }; + } + + let mut filtered = String::new(); + filtered.push_str(if ultra_compact { + "MRs\n" + } else { + "Merge Requests\n" + }); + + for mr in mrs.iter().take(20) { + let iid = mr["iid"].as_i64().unwrap_or(0); + let title = mr["title"].as_str().unwrap_or("???"); + let state = mr["state"].as_str().unwrap_or("???"); + let author = mr["author"]["username"].as_str().unwrap_or("???"); + + let icon = state_icon(state, ultra_compact); + filtered.push_str(&format!( + " {} !{} {} ({})\n", + icon, + iid, + truncate(title, 60), + author + )); + } + + if mrs.len() > 20 { + filtered.push_str(&format!( + " ... {} more (use glab mr list for all)\n", + mrs.len() - 20 + )); + } + + filtered +} + +fn mr_list(args: &[String], _verbose: u8, ultra_compact: bool) -> Result { + let mut cmd = resolved_command("glab"); + cmd.args(["mr", "list", "-F", "json"]); + for arg in args { + cmd.arg(arg); + } + run_glab_json(cmd, "mr list", |json| format_mr_list(json, ultra_compact)) +} + +/// Format MR view JSON into compact output (pure function, testable). +fn format_mr_view(json: &Value, ultra_compact: bool) -> String { + let iid = json["iid"].as_i64().unwrap_or(0); + let title = json["title"].as_str().unwrap_or("???"); + let state = json["state"].as_str().unwrap_or("???"); + let author = json["author"]["username"].as_str().unwrap_or("???"); + let web_url = json["web_url"].as_str().unwrap_or(""); + let merge_status = json["merge_status"].as_str().unwrap_or("unknown"); + let source_branch = json["source_branch"].as_str().unwrap_or("???"); + let target_branch = json["target_branch"].as_str().unwrap_or("???"); + + let icon = state_icon(state, ultra_compact); + + let mut filtered = String::new(); + filtered.push_str(&format!("{} MR !{}: {}\n", icon, iid, title)); + filtered.push_str(&format!(" {}\n", author)); + + let mergeable_str = match merge_status { + "can_be_merged" => "[ok]", + "cannot_be_merged" => "[conflict]", + _ => "[?]", + }; + filtered.push_str(&format!(" {} | {}\n", state, mergeable_str)); + filtered.push_str(&format!(" {} -> {}\n", source_branch, target_branch)); + + if let Some(labels) = json["labels"].as_array() { + let joined: Vec<&str> = labels.iter().filter_map(|v| v.as_str()).collect(); + if !joined.is_empty() { + filtered.push_str(&format!(" Labels: {}\n", joined.join(", "))); + } + } + + if let Some(reviewers) = json["reviewers"].as_array() { + let names: Vec = reviewers + .iter() + .filter_map(|r| r["username"].as_str()) + .map(|u| format!("@{}", u)) + .collect(); + if !names.is_empty() { + filtered.push_str(&format!(" Reviewers: {}\n", names.join(", "))); + } + } + + if let Some(pipeline) = json.get("head_pipeline") { + if !pipeline.is_null() { + let pipeline_status = pipeline["status"].as_str().unwrap_or("unknown"); + let p_icon = pipeline_icon(pipeline_status, ultra_compact); + filtered.push_str(&format!(" Pipeline: {} {}\n", p_icon, pipeline_status)); + } + } + + filtered.push_str(&format!(" {}\n", web_url)); + + if let Some(desc) = json["description"].as_str() { + if !desc.is_empty() { + let desc_filtered = filter_markdown_body(desc); + if !desc_filtered.is_empty() { + filtered.push('\n'); + for line in desc_filtered.lines() { + filtered.push_str(&format!(" {}\n", line)); + } + } + } + } + + filtered +} + +fn mr_view(args: &[String], _verbose: u8, ultra_compact: bool) -> Result { + let (mr_number, extra_args) = match extract_identifier_and_extra_args(args) { + Some(pair) => pair, + None => return Err(anyhow::anyhow!("MR number required")), + }; + + // Passthrough for --web, --comments, or explicit output format + if should_passthrough_view(&extra_args) { + return run_passthrough_with_extra("glab", &["mr", "view", &mr_number], &extra_args); + } + + let mut cmd = resolved_command("glab"); + cmd.args(["mr", "view", &mr_number, "-F", "json"]); + for arg in &extra_args { + cmd.arg(arg); + } + run_glab_json(cmd, &format!("mr view {}", mr_number), |json| { + format_mr_view(json, ultra_compact) + }) +} + +fn mr_create(args: &[String], _verbose: u8) -> Result { + let mut cmd = resolved_command("glab"); + cmd.args(["mr", "create"]); + for arg in args { + cmd.arg(arg); + } + runner::run_filtered( + cmd, + "glab", + "mr create", + |stdout| { + // glab mr create outputs the URL on success + let url = stdout.trim(); + let mr_num = extract_mr_number(url).unwrap_or_default(); + let detail = if !mr_num.is_empty() { + format!("!{} {}", mr_num, url) + } else { + url.to_string() + }; + ok_confirmation("created", &detail) + }, + RunOptions::stdout_only().early_exit_on_failure(), + ) +} + +fn mr_diff(args: &[String], _verbose: u8) -> Result { + let mut cmd = resolved_command("glab"); + cmd.args(["mr", "diff"]); + for arg in args { + cmd.arg(arg); + } + runner::run_filtered( + cmd, + "glab", + "mr diff", + |stdout| { + if stdout.trim().is_empty() { + "No diff\n".to_string() + } else { + git::compact_diff(stdout, 500) + } + }, + RunOptions::stdout_only().early_exit_on_failure(), + ) +} + +/// Generic MR action handler for merge/approve/note/update. +/// Uses extract_identifier_and_extra_args to correctly find the MR number +/// even when it appears after flags (e.g. `glab mr note -m "msg" 42`). +fn mr_action(subcmd: &str, label: &str, args: &[String], _verbose: u8) -> Result { + let mut cmd = resolved_command("glab"); + cmd.args(["mr", subcmd]); + for arg in args { + cmd.arg(arg); + } + + let mr_num = extract_identifier_and_extra_args(args) + .map(|(id, _)| format!("!{}", id)) + .unwrap_or_default(); + let label = label.to_string(); + runner::run_filtered( + cmd, + "glab", + &format!("mr {}", subcmd), + move |_stdout| ok_confirmation(&label, &mr_num), + RunOptions::stdout_only().early_exit_on_failure(), + ) +} + +// ── Issue subcommands ─────────────────────────────────────────────────── + +fn run_issue(args: &[String], verbose: u8, ultra_compact: bool) -> Result { + if args.is_empty() { + return run_passthrough("glab", "issue", args); + } + + match args[0].as_str() { + "list" => issue_list(&args[1..], verbose, ultra_compact), + "view" => issue_view(&args[1..], verbose), + _ => run_passthrough("glab", "issue", args), + } +} + +/// Format issue list JSON into compact output (pure function, testable). +fn format_issue_list(json: &Value, ultra_compact: bool) -> String { + let issues = match json.as_array() { + Some(arr) => arr, + None => return String::new(), + }; + if issues.is_empty() { + return "No Issues\n".to_string(); + } + + let mut filtered = String::new(); + filtered.push_str("Issues\n"); + + for issue in issues.iter().take(20) { + let iid = issue["iid"].as_i64().unwrap_or(0); + let title = issue["title"].as_str().unwrap_or("???"); + let state = issue["state"].as_str().unwrap_or("???"); + + let icon = if ultra_compact { + if state == "opened" { + "O" + } else { + "C" + } + } else if state == "opened" { + "[open]" + } else { + "[closed]" + }; + filtered.push_str(&format!(" {} #{} {}\n", icon, iid, truncate(title, 60))); + } + + if issues.len() > 20 { + filtered.push_str(&format!(" ... {} more\n", issues.len() - 20)); + } + + filtered +} + +fn issue_list(args: &[String], _verbose: u8, ultra_compact: bool) -> Result { + let mut cmd = resolved_command("glab"); + cmd.args(["issue", "list", "-F", "json"]); + for arg in args { + cmd.arg(arg); + } + run_glab_json(cmd, "issue list", |json| { + format_issue_list(json, ultra_compact) + }) +} + +/// Format issue view JSON into compact output (pure function, testable). +fn format_issue_view(json: &Value) -> String { + let iid = json["iid"].as_i64().unwrap_or(0); + let title = json["title"].as_str().unwrap_or("???"); + let state = json["state"].as_str().unwrap_or("???"); + let author = json["author"]["username"].as_str().unwrap_or("???"); + let web_url = json["web_url"].as_str().unwrap_or(""); + + let icon = if state == "opened" { + "[open]" + } else { + "[closed]" + }; + + let mut filtered = String::new(); + filtered.push_str(&format!("{} Issue #{}: {}\n", icon, iid, title)); + filtered.push_str(&format!(" Author: @{}\n", author)); + filtered.push_str(&format!(" Status: {}\n", state)); + filtered.push_str(&format!(" URL: {}\n", web_url)); + + if let Some(desc) = json["description"].as_str() { + if !desc.is_empty() { + let desc_filtered = filter_markdown_body(desc); + if !desc_filtered.is_empty() { + filtered.push_str("\n Description:\n"); + for line in desc_filtered.lines() { + filtered.push_str(&format!(" {}\n", line)); + } + } + } + } + + filtered +} + +fn issue_view(args: &[String], _verbose: u8) -> Result { + let (issue_number, extra_args) = match extract_identifier_and_extra_args(args) { + Some(pair) => pair, + None => return Err(anyhow::anyhow!("Issue number required")), + }; + + if should_passthrough_view(&extra_args) { + return run_passthrough_with_extra("glab", &["issue", "view", &issue_number], &extra_args); + } + + let mut cmd = resolved_command("glab"); + cmd.args(["issue", "view", &issue_number, "-F", "json"]); + for arg in &extra_args { + cmd.arg(arg); + } + run_glab_json( + cmd, + &format!("issue view {}", issue_number), + format_issue_view, + ) +} + +// ── CI/Pipeline subcommands ───────────────────────────────────────────── + +fn run_ci(args: &[String], verbose: u8, ultra_compact: bool) -> Result { + if args.is_empty() { + return run_passthrough("glab", "ci", args); + } + + match args[0].as_str() { + "list" => ci_list(&args[1..], verbose, ultra_compact), + "status" => ci_status(&args[1..], verbose, ultra_compact), + "trace" => ci_trace(&args[1..]), + // "ci view" is an interactive TUI (tcell) — must run with inherited stdio + _ => run_passthrough("glab", "ci", args), + } +} + +/// Format CI list JSON into compact output (pure function, testable). +fn format_ci_list(json: &Value, ultra_compact: bool) -> String { + let pipelines = match json.as_array() { + Some(arr) => arr, + None => return String::new(), + }; + if pipelines.is_empty() { + return "No Pipelines\n".to_string(); + } + + let mut filtered = String::new(); + filtered.push_str("Pipelines\n"); + for pipeline in pipelines.iter().take(10) { + let id = pipeline["id"].as_i64().unwrap_or(0); + let status = pipeline["status"].as_str().unwrap_or("???"); + let ref_name = pipeline["ref"].as_str().unwrap_or("???"); + + let icon = pipeline_icon(status, ultra_compact); + filtered.push_str(&format!(" {} #{} {} ({})\n", icon, id, status, ref_name)); + } + filtered +} + +fn ci_list(args: &[String], _verbose: u8, ultra_compact: bool) -> Result { + let mut cmd = resolved_command("glab"); + cmd.args(["ci", "list", "-F", "json"]); + for arg in args { + cmd.arg(arg); + } + run_glab_json(cmd, "ci list", |json| format_ci_list(json, ultra_compact)) +} + +/// Format `glab ci status` text output (English keyword parsing, raw fallback). +/// Returns the raw input when no status keyword is recognized on any line +/// (e.g. non-English locale). +fn format_ci_status(raw: &str, ultra_compact: bool) -> String { + let mut filtered = String::new(); + let mut any_keyword_matched = false; + for line in raw.lines() { + let trimmed = line.trim(); + if trimmed.is_empty() { + continue; + } + let icon = if trimmed.contains("passed") || trimmed.contains("success") { + pipeline_icon("success", ultra_compact) + } else if trimmed.contains("failed") { + pipeline_icon("failed", ultra_compact) + } else if trimmed.contains("running") { + pipeline_icon("running", ultra_compact) + } else if trimmed.contains("pending") { + pipeline_icon("pending", ultra_compact) + } else if trimmed.contains("canceled") || trimmed.contains("cancelled") { + pipeline_icon("canceled", ultra_compact) + } else { + "" + }; + + if !icon.is_empty() { + any_keyword_matched = true; + filtered.push_str(&format!("{} {}\n", icon, trimmed)); + } else { + filtered.push_str(&format!(" {}\n", trimmed)); + } + } + + if !any_keyword_matched { + // Non-English locale or unrecognized format — preserve raw output verbatim. + raw.to_string() + } else { + filtered + } +} + +fn ci_status(args: &[String], _verbose: u8, ultra_compact: bool) -> Result { + // glab ci status does not support -F json — text parsing with raw fallback + let mut cmd = resolved_command("glab"); + cmd.args(["ci", "status"]); + for arg in args { + cmd.arg(arg); + } + runner::run_filtered( + cmd, + "glab", + "ci status", + |stdout| format_ci_status(stdout, ultra_compact), + RunOptions::stdout_only().early_exit_on_failure(), + ) +} + +fn ci_trace(args: &[String]) -> Result { + let mut cmd = resolved_command("glab"); + cmd.args(["ci", "trace"]); + for arg in args { + cmd.arg(arg); + } + runner::run_filtered( + cmd, + "glab", + "ci trace", + filter_ci_trace, + RunOptions::stdout_only().early_exit_on_failure(), + ) +} + +/// Filter CI job trace output: strip ANSI codes, section markers, and runner +/// boilerplate. Keep warnings, errors, and build output. +fn filter_ci_trace(raw: &str) -> String { + let cleaned = strip_ansi(raw); + let cleaned = BARE_ANSI_RE.replace_all(&cleaned, ""); + let cleaned = SECTION_MARKER_RE.replace_all(&cleaned, ""); + + let mut filtered = String::new(); + + for line in cleaned.lines() { + let trimmed = line.trim(); + + if trimmed.is_empty() { + continue; + } + + // Skip runner boilerplate + if trimmed.starts_with("Running with gitlab-runner") + || (trimmed.starts_with("on ") && trimmed.contains("system ID:")) + || trimmed.starts_with("Using Docker executor") + || trimmed.starts_with("Using Shell") + || trimmed.starts_with("Running on runner-") + || trimmed.starts_with("Running on ") + || trimmed.starts_with("Preparing the") + || trimmed.starts_with("Preparing environment") + || trimmed.starts_with("Getting source from") + || trimmed.starts_with("Resolving secrets") + || trimmed.starts_with("Cleaning up") + || trimmed.starts_with("Uploading artifacts") + || trimmed.starts_with("Downloading artifacts") + || trimmed.starts_with("Runtime platform") + { + continue; + } + + // Skip git fetch / checkout boilerplate + if trimmed.starts_with("Fetching changes with git") + || trimmed.starts_with("Initialized empty Git") + || trimmed.starts_with("Created fresh repository") + || trimmed.starts_with("Checking out ") + || trimmed.starts_with("Skipping Git submodules") + { + continue; + } + + filtered.push_str(trimmed); + filtered.push('\n'); + } + + filtered +} + +// ── Release subcommands ────────────────────────────────────────────────── + +fn run_release(args: &[String], _verbose: u8, _ultra_compact: bool) -> Result { + if args.is_empty() { + return run_passthrough("glab", "release", args); + } + + match args[0].as_str() { + "list" => release_list(&args[1..]), + "view" => release_view(&args[1..]), + _ => run_passthrough("glab", "release", args), + } +} + +/// Format `glab release list` tab-separated output into compact form. +/// Input format: "Name\tTag\tCreated\n" header + data rows. +fn format_release_list(raw: &str) -> Option { + let mut lines = raw.lines().peekable(); + let mut filtered = String::new(); + + // Skip "Showing N releases..." preamble and blank lines + while let Some(line) = lines.peek() { + let trimmed = line.trim(); + if trimmed.starts_with("Name\t") || trimmed.starts_with("NAME\t") { + lines.next(); // consume header + break; + } + lines.next(); + } + + filtered.push_str("Releases\n"); + + let mut count = 0; + for line in lines { + let trimmed = line.trim(); + if trimmed.is_empty() { + continue; + } + + let parts: Vec<&str> = trimmed.split('\t').collect(); + if parts.len() < 3 { + continue; + } + + let name = parts[0].trim(); + let tag = parts[1].trim(); + let created = parts[2].trim(); + + if name == tag { + filtered.push_str(&format!(" {} ({})\n", name, created)); + } else { + filtered.push_str(&format!(" {} [{}] ({})\n", name, tag, created)); + } + + count += 1; + if count >= 20 { + break; + } + } + + if count == 0 { + return None; + } + + Some(filtered) +} + +fn release_list(args: &[String]) -> Result { + let mut cmd = resolved_command("glab"); + cmd.args(["release", "list"]); + for arg in args { + cmd.arg(arg); + } + runner::run_filtered( + cmd, + "glab", + "release list", + |stdout| format_release_list(stdout).unwrap_or_else(|| stdout.to_string()), + RunOptions::stdout_only().early_exit_on_failure(), + ) +} + +fn release_view(args: &[String]) -> Result { + let mut cmd = resolved_command("glab"); + cmd.args(["release", "view"]); + for arg in args { + cmd.arg(arg); + } + runner::run_filtered( + cmd, + "glab", + "release view", + filter_release_view, + RunOptions::stdout_only().early_exit_on_failure(), + ) +} + +/// Filter release view output: strip SOURCES block, image lines, HTML comments, +/// horizontal rules, and collapse blank lines. +fn filter_release_view(raw: &str) -> String { + let mut filtered = String::new(); + let mut in_sources = false; + + for line in raw.lines() { + let trimmed = line.trim(); + + // Skip SOURCES section (archive download URLs) + if trimmed == "SOURCES" { + in_sources = true; + continue; + } + if in_sources { + if trimmed.starts_with("http://") || trimmed.starts_with("https://") { + continue; + } + in_sources = false; + } + + // Strip image-only lines + if trimmed.starts_with("![") && trimmed.ends_with(')') && trimmed.contains("](") { + continue; + } + // Strip glab's "Image: name → url" rendering + if trimmed.starts_with("Image:") && trimmed.contains('→') { + continue; + } + + // Strip HTML comments + if trimmed.starts_with("") { + continue; + } + + // Strip horizontal rules (--- rendered as --------) + if trimmed.chars().all(|c| c == '-') && trimmed.len() >= 3 { + continue; + } + + filtered.push_str(line); + filtered.push('\n'); + } + + // Collapse multiple blank lines + MULTI_BLANK_RE.replace_all(&filtered, "\n\n").to_string() +} + +// ── API subcommand ────────────────────────────────────────────────────── + +fn run_api(args: &[String], _verbose: u8) -> Result { + // glab api is an explicit/advanced command — the user knows what they asked for. + // Converting JSON to a schema destroys all values and forces Claude to re-fetch. + // Passthrough preserves the full response and tracks metrics at 0% savings. + run_passthrough("glab", "api", args) +} + +// ── Passthrough ───────────────────────────────────────────────────────── + +fn run_passthrough(cmd: &str, subcommand: &str, args: &[String]) -> Result { + let mut os_args: Vec = vec![std::ffi::OsString::from(subcommand)]; + os_args.extend(args.iter().map(std::ffi::OsString::from)); + runner::run_passthrough(cmd, &os_args, 0) +} + +fn run_passthrough_with_extra(cmd: &str, base_args: &[&str], extra_args: &[String]) -> Result { + let mut os_args: Vec = + base_args.iter().map(std::ffi::OsString::from).collect(); + os_args.extend(extra_args.iter().map(std::ffi::OsString::from)); + runner::run_passthrough(cmd, &os_args, 0) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn test_state_icon_opened() { + assert_eq!(state_icon("opened", false), "[open]"); + assert_eq!(state_icon("opened", true), "O"); + } + + #[test] + fn test_state_icon_merged() { + assert_eq!(state_icon("merged", false), "[merged]"); + assert_eq!(state_icon("merged", true), "M"); + } + + #[test] + fn test_state_icon_closed() { + assert_eq!(state_icon("closed", false), "[closed]"); + assert_eq!(state_icon("closed", true), "C"); + } + + #[test] + fn test_pipeline_icon_success() { + assert_eq!(pipeline_icon("success", false), "[ok]"); + assert_eq!(pipeline_icon("success", true), "+"); + } + + #[test] + fn test_pipeline_icon_failed() { + assert_eq!(pipeline_icon("failed", false), "[fail]"); + assert_eq!(pipeline_icon("failed", true), "x"); + } + + #[test] + fn test_pipeline_icon_running() { + assert_eq!(pipeline_icon("running", false), "[run]"); + assert_eq!(pipeline_icon("running", true), "~"); + } + + #[test] + fn test_extract_mr_number_from_url() { + let url = "https://gitlab.example.com/group/project/-/merge_requests/42"; + assert_eq!(extract_mr_number(url), Some("42".to_string())); + } + + #[test] + fn test_extract_mr_number_no_match() { + assert_eq!(extract_mr_number("not a url"), None); + } + + #[test] + fn test_filter_markdown_body_empty() { + assert_eq!(filter_markdown_body(""), ""); + } + + #[test] + fn test_filter_markdown_body_html_comments() { + let input = "Hello\n\nWorld"; + let result = filter_markdown_body(input); + assert!(!result.contains("\n```\nAfter"; + let result = filter_markdown_body(input); + assert!(result.contains("")); + assert!(result.contains("Text")); + assert!(result.contains("After")); + } + + #[test] + fn test_filter_markdown_body_blank_lines_collapse() { + let input = "Line 1\n\n\n\n\nLine 2"; + let result = filter_markdown_body(input); + assert!(!result.contains("\n\n\n")); + assert!(result.contains("Line 1")); + assert!(result.contains("Line 2")); + } + + #[test] + fn test_filter_markdown_body_badges_removed() { + let input = + "# Title\n[![CI](https://img.shields.io/badge.svg)](https://github.com/actions)\nText"; + let result = filter_markdown_body(input); + assert!(!result.contains("shields.io")); + assert!(result.contains("# Title")); + assert!(result.contains("Text")); + } + + #[test] + fn test_filter_markdown_body_meaningful_content_preserved() { + let input = "## Summary\n- Item 1\n- Item 2\n\n[Link](https://example.com)"; + let result = filter_markdown_body(input); + assert!(result.contains("## Summary")); + assert!(result.contains("- Item 1")); + assert!(result.contains("[Link](https://example.com)")); + } + + #[test] + fn test_ok_confirmation_mr_create() { + let result = ok_confirmation( + "created", + "!42 https://gitlab.example.com/-/merge_requests/42", + ); + assert!(result.contains("ok created")); + assert!(result.contains("!42")); + } + + #[test] + fn test_ok_confirmation_mr_merge() { + let result = ok_confirmation("merged", "!42"); + assert_eq!(result, "ok merged !42"); + } + + #[test] + fn test_ok_confirmation_mr_approve() { + let result = ok_confirmation("approved", "!42"); + assert_eq!(result, "ok approved !42"); + } + + fn count_tokens(text: &str) -> usize { + text.split_whitespace().count() + } + + fn parse_fixture(raw: &str) -> Value { + serde_json::from_str(raw).expect("valid JSON fixture") + } + + #[test] + fn test_mr_list_token_savings() { + let input = include_str!("../../../tests/fixtures/glab_mr_list_raw.json"); + let output = format_mr_list(&parse_fixture(input), false); + let input_tokens = count_tokens(input); + let output_tokens = count_tokens(&output); + let savings = 100.0 - (output_tokens as f64 / input_tokens as f64 * 100.0); + assert!( + savings >= 60.0, + "MR list: expected >=60% savings, got {:.1}% ({} -> {} tokens)", + savings, + input_tokens, + output_tokens + ); + } + + #[test] + fn test_mr_list_format() { + let input = include_str!("../../../tests/fixtures/glab_mr_list_raw.json"); + let output = format_mr_list(&parse_fixture(input), false); + assert!(output.contains("Merge Requests")); + assert!(output.contains("!314")); + assert!(output.contains("[open]")); // opened + assert!(output.contains("[merged]")); // merged + assert!(output.contains("[closed]")); // closed + } + + #[test] + fn test_mr_list_ultra_compact() { + let input = include_str!("../../../tests/fixtures/glab_mr_list_raw.json"); + let output = format_mr_list(&parse_fixture(input), true); + assert!(output.starts_with("MRs\n")); + assert!(output.contains("O ")); // opened + assert!(output.contains("M ")); // merged + assert!(output.contains("C ")); // closed + } + + #[test] + fn test_issue_list_token_savings() { + let input = include_str!("../../../tests/fixtures/glab_issue_list_raw.json"); + let output = format_issue_list(&parse_fixture(input), false); + let input_tokens = count_tokens(input); + let output_tokens = count_tokens(&output); + let savings = 100.0 - (output_tokens as f64 / input_tokens as f64 * 100.0); + assert!( + savings >= 60.0, + "Issue list: expected >=60% savings, got {:.1}% ({} -> {} tokens)", + savings, + input_tokens, + output_tokens + ); + } + + #[test] + fn test_issue_list_format() { + let input = include_str!("../../../tests/fixtures/glab_issue_list_raw.json"); + let output = format_issue_list(&parse_fixture(input), false); + assert!(output.contains("Issues")); + assert!(output.contains("#156")); + assert!(output.contains("[open]")); // opened + assert!(output.contains("[closed]")); // closed + } + + #[test] + fn test_format_mr_list_non_array_returns_empty() { + // Non-array JSON (e.g. error object) returns empty — run_glab_json then + // falls back to raw stdout through its JSON parse branch. + let output = format_mr_list(&Value::Object(Default::default()), false); + assert!(output.is_empty()); + } + + #[test] + fn test_format_issue_list_non_array_returns_empty() { + let output = format_issue_list(&Value::Object(Default::default()), false); + assert!(output.is_empty()); + } + + #[test] + fn test_extract_identifier_simple() { + let args: Vec = vec!["42".into()]; + let (id, extra) = extract_identifier_and_extra_args(&args).unwrap(); + assert_eq!(id, "42"); + assert!(extra.is_empty()); + } + + #[test] + fn test_extract_identifier_with_repo_flag_before() { + // glab mr view -R group/project 42 + let args: Vec = vec!["-R".into(), "group/project".into(), "42".into()]; + let (id, extra) = extract_identifier_and_extra_args(&args).unwrap(); + assert_eq!(id, "42"); + assert_eq!(extra, vec!["-R", "group/project"]); + } + + #[test] + fn test_extract_identifier_with_repo_flag_after() { + // glab mr view 42 -R group/project + let args: Vec = vec!["42".into(), "-R".into(), "group/project".into()]; + let (id, extra) = extract_identifier_and_extra_args(&args).unwrap(); + assert_eq!(id, "42"); + assert_eq!(extra, vec!["-R", "group/project"]); + } + + #[test] + fn test_extract_identifier_with_group_flag() { + let args: Vec = vec!["-g".into(), "mygroup".into(), "7".into()]; + let (id, extra) = extract_identifier_and_extra_args(&args).unwrap(); + assert_eq!(id, "7"); + assert_eq!(extra, vec!["-g", "mygroup"]); + } + + #[test] + fn test_extract_identifier_empty() { + let args: Vec = vec![]; + assert!(extract_identifier_and_extra_args(&args).is_none()); + } + + #[test] + fn test_extract_identifier_only_flags() { + let args: Vec = vec!["-R".into(), "group/project".into()]; + assert!(extract_identifier_and_extra_args(&args).is_none()); + } + + // ── has_output_flag tests ─────────────────────────────────────────── + + #[test] + fn test_has_output_flag_json() { + assert!(has_output_flag(&["--json".into()])); + } + + #[test] + fn test_has_output_flag_format() { + assert!(has_output_flag(&["-F".into(), "json".into()])); + assert!(has_output_flag(&["--output".into(), "text".into()])); + } + + #[test] + fn test_has_output_flag_none() { + assert!(!has_output_flag(&["mr".into(), "list".into()])); + } + + // ── should_passthrough_view tests ─────────────────────────────────── + + #[test] + fn test_should_passthrough_view_web() { + assert!(should_passthrough_view(&["--web".into()])); + } + + #[test] + fn test_should_passthrough_view_comments() { + assert!(should_passthrough_view(&["--comments".into()])); + } + + #[test] + fn test_should_passthrough_view_output() { + assert!(should_passthrough_view(&["-F".into(), "json".into()])); + } + + #[test] + fn test_should_passthrough_view_default() { + assert!(!should_passthrough_view(&[])); + } + + // ── mr_action identifier extraction ───────────────────────────────── + + #[test] + fn test_extract_identifier_with_message_flag() { + // glab mr note -m "comment" 42 — number should be 42, not "comment" + let args: Vec = vec!["-m".into(), "comment".into(), "42".into()]; + let (id, extra) = extract_identifier_and_extra_args(&args).unwrap(); + assert_eq!(id, "42"); + assert_eq!(extra, vec!["-m", "comment"]); + } + + // ── release list tests ────────────────────────────────────────────── + + #[test] + fn test_format_release_list() { + let input = include_str!("../../../tests/fixtures/glab_release_list_raw.txt"); + let output = format_release_list(input).expect("should parse release list"); + assert!(output.starts_with("Releases\n")); + assert!(output.contains("v3.2.1")); + assert!(output.contains("about 2 days ago")); + } + + #[test] + fn test_format_release_list_token_savings() { + let input = include_str!("../../../tests/fixtures/glab_release_list_raw.txt"); + let output = format_release_list(input).expect("should parse release list"); + let input_tokens = count_tokens(input); + let output_tokens = count_tokens(&output); + let savings = 100.0 - (output_tokens as f64 / input_tokens as f64 * 100.0); + // Release list text is already compact (tab-separated); savings are modest. + assert!( + savings >= 20.0, + "Release list: expected >=20% savings, got {:.1}% ({} -> {} tokens)", + savings, + input_tokens, + output_tokens + ); + } + + #[test] + fn test_format_release_list_empty() { + let input = "No releases available on owner/repo.\nName\tTag\tCreated\n"; + assert!(format_release_list(input).is_none()); + } + + #[test] + fn test_format_release_list_name_differs_from_tag() { + let input = "Showing 1 releases\n\nName\tTag\tCreated\nMy Release\tv1.0.0\t2 days ago\n"; + let output = format_release_list(input).expect("should parse"); + assert!(output.contains("My Release [v1.0.0]")); + } + + // ── ci trace tests ────────────────────────────────────────────────── + + #[test] + fn test_filter_ci_trace_strips_boilerplate() { + let input = include_str!("../../../tests/fixtures/glab_ci_trace_raw.txt"); + let output = filter_ci_trace(input); + // Runner boilerplate stripped + assert!(!output.contains("Running with gitlab-runner")); + assert!(!output.contains("Using Docker executor")); + assert!(!output.contains("Fetching changes with git")); + assert!(!output.contains("Checking out")); + assert!(!output.contains("Uploading artifacts")); + // Build output preserved + assert!(output.contains("npm ci")); + assert!(output.contains("npm run build")); + assert!(output.contains("npm test")); + // Test results preserved + assert!(output.contains("FAIL")); + assert!(output.contains("AssertionError")); + // Final error line preserved + assert!(output.contains("Job failed")); + } + + #[test] + fn test_filter_ci_trace_token_savings() { + let input = include_str!("../../../tests/fixtures/glab_ci_trace_raw.txt"); + let output = filter_ci_trace(input); + let input_tokens = count_tokens(input); + let output_tokens = count_tokens(&output); + let savings = 100.0 - (output_tokens as f64 / input_tokens as f64 * 100.0); + // CI trace preserves build output; savings come from stripping boilerplate. + assert!( + savings >= 30.0, + "CI trace: expected >=30% savings, got {:.1}% ({} -> {} tokens)", + savings, + input_tokens, + output_tokens + ); + } + + // ── release view tests ────────────────────────────────────────────── + + #[test] + fn test_filter_release_view_strips_sources() { + let input = include_str!("../../../tests/fixtures/glab_release_view_raw.txt"); + let output = filter_release_view(input); + // SOURCES section stripped + assert!(!output.contains("SOURCES")); + assert!(!output.contains("toolkit-v2.0.0.zip")); + assert!(!output.contains("toolkit-v2.0.0.tar.gz")); + // Content preserved + assert!(output.contains("Test Release v2.0")); + assert!(output.contains("Added widget support")); + assert!(output.contains("@alice_dev @bob_dev")); + // Noise stripped + assert!(!output.contains("--------")); + assert!(!output.contains("Image:")); + assert!(!output.contains(" file:line:col\n |\n | code\n" let mut current_rule = String::new(); + let mut in_error = false; + let mut current_block: Vec = Vec::new(); for line in output.lines() { - // Skip compilation lines + // Skip compilation progress lines if line.trim_start().starts_with("Compiling") || line.trim_start().starts_with("Checking") || line.trim_start().starts_with("Downloading") || line.trim_start().starts_with("Downloaded") || line.trim_start().starts_with("Finished") { + if in_error && !current_block.is_empty() { + error_blocks.push(current_block.clone()); + current_block.clear(); + in_error = false; + } continue; } - // "warning: unused variable [unused_variables]" or "warning: description [clippy::rule_name]" - if (line.starts_with("warning:") || line.starts_with("warning[")) - || (line.starts_with("error:") || line.starts_with("error[")) + // Skip noise: summary counts and abort lines + if (line.contains("generated") && line.contains("warning")) + || line.contains("aborting due to") + || line.contains("could not compile") { - // Skip summary lines: "warning: `rtk` (bin) generated 5 warnings" - if line.contains("generated") && line.contains("warning") { - continue; - } - // Skip "error: aborting" / "error: could not compile" - if line.contains("aborting due to") || line.contains("could not compile") { - continue; + continue; + } + + let is_error_line = line.starts_with("error:") || line.starts_with("error["); + let is_warning_line = line.starts_with("warning:") || line.starts_with("warning["); + + if is_error_line || is_warning_line { + // Flush any in-progress error block before starting a new diagnostic + if in_error && !current_block.is_empty() { + error_blocks.push(current_block.clone()); + current_block.clear(); } + in_error = false; - let is_error = line.starts_with("error"); - if is_error { + if is_error_line { error_count += 1; - error_details.push(truncate(line.trim(), 160)); + in_error = true; + current_block.push(line.to_string()); } else { warning_count += 1; } - // Extract rule name from brackets + // Extract rule/error-code from brackets for warning grouping current_rule = if let Some(bracket_start) = line.rfind('[') { if let Some(bracket_end) = line.rfind(']') { line[bracket_start + 1..bracket_end].to_string() @@ -935,8 +1133,7 @@ fn filter_cargo_clippy(output: &str) -> String { line.to_string() } } else { - // No bracket: use the message itself as the rule - let prefix = if is_error { "error: " } else { "warning: " }; + let prefix = if is_error_line { "error: " } else { "warning: " }; line.strip_prefix(prefix).unwrap_or(line).to_string() }; } else if line.trim_start().starts_with("--> ") { @@ -947,9 +1144,29 @@ fn filter_cargo_clippy(output: &str) -> String { .or_default() .push(location); } + if in_error { + current_block.push(line.to_string()); + } + } else if in_error { + if line.trim().is_empty() { + // Blank line terminates the error block + if !current_block.is_empty() { + error_blocks.push(current_block.clone()); + current_block.clear(); + } + in_error = false; + } else if current_block.len() < 15 { + // Collect code-context lines (|, ^, = note:, help:, etc.) + current_block.push(line.to_string()); + } } } + // Flush final error block + if in_error && !current_block.is_empty() { + error_blocks.push(current_block); + } + if error_count == 0 && warning_count == 0 { return "cargo clippy: No issues found".to_string(); } @@ -961,18 +1178,21 @@ fn filter_cargo_clippy(output: &str) -> String { )); result.push_str("═══════════════════════════════════════\n"); - if !error_details.is_empty() { - result.push_str("\nError details:\n"); - for (idx, detail) in error_details.iter().take(5).enumerate() { - result.push_str(&format!(" {}. {}\n", idx + 1, detail)); + // Show full error blocks so developers can see what needs fixing + if !error_blocks.is_empty() { + result.push_str("\nErrors:\n"); + for block in error_blocks.iter().take(10) { + for block_line in block { + result.push_str(&format!(" {}\n", truncate(block_line, 160))); + } + result.push('\n'); } - if error_details.len() > 5 { - result.push_str(&format!(" ... +{} more errors\n", error_details.len() - 5)); + if error_blocks.len() > 10 { + result.push_str(&format!(" ... +{} more errors\n", error_blocks.len() - 10)); } - result.push('\n'); } - // Sort rules by frequency + // Sort warning rules by frequency let mut rule_counts: Vec<_> = by_rule.iter().collect(); rule_counts.sort_by(|a, b| b.1.len().cmp(&a.1.len())); @@ -993,28 +1213,8 @@ fn filter_cargo_clippy(output: &str) -> String { result.trim().to_string() } -/// Runs an unsupported cargo subcommand by passing it through directly -pub fn run_passthrough(args: &[OsString], verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - - if verbose > 0 { - eprintln!("cargo passthrough: {:?}", args); - } - let status = resolved_command("cargo") - .args(args) - .status() - .context("Failed to run cargo")?; - - let args_str = tracking::args_display(args); - timer.track_passthrough( - &format!("cargo {}", args_str), - &format!("rtk cargo {} (passthrough)", args_str), - ); - - if !status.success() { - std::process::exit(status.code().unwrap_or(1)); - } - Ok(()) +pub fn run_passthrough(args: &[OsString], verbose: u8) -> Result { + crate::core::runner::run_passthrough("cargo", args, verbose) } #[cfg(test)] @@ -1413,10 +1613,47 @@ warning: unused variable: `x` [unused_variables] "#; let result = filter_cargo_clippy(output); assert!(result.contains("cargo clippy: 1 errors, 1 warnings")); - assert!(result.contains("Error details:")); + assert!(result.contains("Errors:")); assert!(result.contains("struct literals are not allowed here")); } + #[test] + fn test_filter_cargo_clippy_shows_full_error_block() { + // Full multi-line error block must be shown so the developer can debug + let output = r#" Checking rtk v0.5.0 +error[E0308]: mismatched types + --> src/main.rs:10:5 + | +9 | fn foo() -> i32 { + | --- expected `i32` because of return type +10| "hello" + | ^^^^^^^ expected `i32`, found `&str` + +error: aborting due to 1 previous error +"#; + let result = filter_cargo_clippy(output); + assert!(result.contains("cargo clippy: 1 errors, 0 warnings"), "got: {}", result); + assert!(result.contains("error[E0308]: mismatched types"), "got: {}", result); + assert!(result.contains("src/main.rs:10:5"), "got: {}", result); + assert!(result.contains("expected `i32`, found `&str`"), "got: {}", result); + } + + #[test] + fn test_filter_cargo_clippy_multiple_errors_show_all_blocks() { + let output = r#"error[E0308]: mismatched types + --> src/foo.rs:5:3 + +error[E0425]: cannot find value `x` + --> src/bar.rs:12:9 + +error: aborting due to 2 previous errors +"#; + let result = filter_cargo_clippy(output); + assert!(result.contains("2 errors"), "got: {}", result); + assert!(result.contains("src/foo.rs:5:3"), "got: {}", result); + assert!(result.contains("src/bar.rs:12:9"), "got: {}", result); + } + #[test] fn test_filter_cargo_install_success() { let output = r#" Installing rtk v0.11.0 @@ -1831,4 +2068,122 @@ error: test run failed result ); } + + // --- Streaming handler tests --- + + use crate::core::stream::tests::run_block_filter; + + #[test] + fn test_cargo_build_stream_success() { + let input = " Compiling libc v0.2.153\n Compiling cfg-if v1.0.0\n Compiling rtk v0.5.0\n Finished dev [unoptimized + debuginfo] target(s) in 15.23s\n"; + let mut f = BlockStreamFilter::new(CargoBuildHandler::new()); + let result = run_block_filter(&mut f, input, 0); + assert!(result.contains("3 crates compiled"), "got: {}", result); + assert!(result.contains("Finished"), "got: {}", result); + assert!(!result.contains("Compiling"), "got: {}", result); + } + + #[test] + fn test_cargo_build_stream_errors() { + let input = r#" Compiling rtk v0.5.0 +error[E0308]: mismatched types + --> src/main.rs:10:5 + | +10| "hello" + | ^^^^^^^ expected `i32`, found `&str` + +error: aborting due to 1 previous error +"#; + let mut f = BlockStreamFilter::new(CargoBuildHandler::new()); + let result = run_block_filter(&mut f, input, 1); + assert!(result.contains("E0308"), "got: {}", result); + assert!(result.contains("mismatched types"), "got: {}", result); + assert!(result.contains("1 errors"), "got: {}", result); + assert!(!result.contains("aborting"), "got: {}", result); + } + + #[test] + fn test_cargo_test_stream_all_pass() { + let input = r#" Compiling rtk v0.5.0 + Finished test [unoptimized + debuginfo] target(s) in 2.53s + Running target/debug/deps/rtk-abc123 + +running 15 tests +test utils::tests::test_truncate_short_string ... ok +test utils::tests::test_truncate_long_string ... ok +test utils::tests::test_strip_ansi_simple ... ok + +test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s +"#; + let mut f = BlockStreamFilter::new(CargoTestHandler::new()); + let result = run_block_filter(&mut f, input, 0); + assert!( + result.contains("cargo test: 15 passed (1 suite, 0.01s)"), + "got: {}", + result + ); + assert!(!result.contains("Compiling"), "got: {}", result); + } + + #[test] + fn test_cargo_test_stream_failures() { + let input = r#"running 5 tests +test foo::test_a ... ok +test foo::test_b ... FAILED +test foo::test_c ... ok + +failures: + +---- foo::test_b stdout ---- +thread 'foo::test_b' panicked at 'assert_eq!(1, 2)' + +failures: + foo::test_b + +test result: FAILED. 4 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out +"#; + let mut f = BlockStreamFilter::new(CargoTestHandler::new()); + let result = run_block_filter(&mut f, input, 1); + assert!(result.contains("test_b"), "got: {}", result); + assert!(result.contains("panicked"), "got: {}", result); + } + + #[test] + fn test_cargo_test_stream_multi_suite() { + let input = r#" Running unittests src/lib.rs (target/debug/deps/rtk-abc123) + +running 50 tests +test result: ok. 50 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.45s + + Running unittests src/main.rs (target/debug/deps/rtk-def456) + +running 30 tests +test result: ok. 30 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.30s +"#; + let mut f = BlockStreamFilter::new(CargoTestHandler::new()); + let result = run_block_filter(&mut f, input, 0); + assert!( + result.contains("cargo test: 80 passed (2 suites, 0.75s)"), + "got: {}", + result + ); + } + + #[test] + fn test_cargo_test_stream_compile_error() { + let input = r#" Compiling rtk v0.31.0 (/workspace/projects/rtk) +error[E0425]: cannot find value `missing_symbol` in this scope + --> tests/repro_compile_fail.rs:3:13 + | +3 | let _ = missing_symbol; + | ^^^^^^^^^^^^^^ not found in this scope + +For more information about this error, try `rustc --explain E0425`. +error: could not compile `rtk` (test "repro_compile_fail") due to 1 previous error +"#; + let mut f = BlockStreamFilter::new(CargoTestHandler::new()); + let result = run_block_filter(&mut f, input, 1); + assert!(result.contains("cargo test:"), "got: {}", result); + assert!(result.contains("1 errors"), "got: {}", result); + } } diff --git a/src/cmds/rust/mod.rs b/src/cmds/rust/mod.rs index c58ffbadf..bcfcfd851 100644 --- a/src/cmds/rust/mod.rs +++ b/src/cmds/rust/mod.rs @@ -1,4 +1 @@ -//! Rust ecosystem filters. - -pub mod cargo_cmd; -pub mod runner; +automod::dir!(pub "src/cmds/rust"); diff --git a/src/cmds/rust/runner.rs b/src/cmds/rust/runner.rs index 8c3f35270..476f90671 100644 --- a/src/cmds/rust/runner.rs +++ b/src/cmds/rust/runner.rs @@ -1,135 +1,148 @@ //! Runs arbitrary commands and captures only stderr or test failures. -use crate::core::tracking; -use anyhow::{Context, Result}; +use crate::core::stream::StreamFilter; +use anyhow::Result; +use lazy_static::lazy_static; use regex::Regex; -use std::process::{Command, Stdio}; +use std::process::Command; -/// Run a command and filter output to show only errors/warnings -pub fn run_err(command: &str, verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); +lazy_static! { + static ref ERROR_PATTERNS: Vec = vec![ + // Generic errors + Regex::new(r"(?i)^.*error[\s:\[].*$").unwrap(), + Regex::new(r"(?i)^.*\berr\b.*$").unwrap(), + Regex::new(r"(?i)^.*warning[\s:\[].*$").unwrap(), + Regex::new(r"(?i)^.*\bwarn\b.*$").unwrap(), + Regex::new(r"(?i)^.*failed.*$").unwrap(), + Regex::new(r"(?i)^.*failure.*$").unwrap(), + Regex::new(r"(?i)^.*exception.*$").unwrap(), + Regex::new(r"(?i)^.*panic.*$").unwrap(), + // Rust specific + Regex::new(r"^error\[E\d+\]:.*$").unwrap(), + Regex::new(r"^\s*--> .*:\d+:\d+$").unwrap(), + // Python + Regex::new(r"^Traceback.*$").unwrap(), + Regex::new(r#"^\s*File ".*", line \d+.*$"#).unwrap(), + // JavaScript/TypeScript + Regex::new(r"^\s*at .*:\d+:\d+.*$").unwrap(), + // Go + Regex::new(r"^.*\.go:\d+:.*$").unwrap(), + ]; +} - if verbose > 0 { - eprintln!("Running: {}", command); +struct ErrorStreamFilter { + in_error_block: bool, + blank_count: usize, + emitted_any: bool, +} + +impl ErrorStreamFilter { + fn new() -> Self { + Self { + in_error_block: false, + blank_count: 0, + emitted_any: false, + } } +} - let output = if cfg!(target_os = "windows") { - Command::new("cmd") - .args(["/C", command]) - .stdout(Stdio::piped()) - .stderr(Stdio::piped()) - .output() - } else { - Command::new("sh") - .args(["-c", command]) - .stdout(Stdio::piped()) - .stderr(Stdio::piped()) - .output() +impl StreamFilter for ErrorStreamFilter { + fn feed_line(&mut self, line: &str) -> Option { + let is_error = ERROR_PATTERNS.iter().any(|p| p.is_match(line)); + if is_error { + self.in_error_block = true; + self.blank_count = 0; + self.emitted_any = true; + Some(format!("{}\n", line)) + } else if self.in_error_block { + if line.trim().is_empty() { + self.blank_count += 1; + if self.blank_count >= 2 { + self.in_error_block = false; + None + } else { + self.emitted_any = true; + Some(format!("{}\n", line)) + } + } else if line.starts_with(' ') || line.starts_with('\t') { + self.blank_count = 0; + self.emitted_any = true; + Some(format!("{}\n", line)) + } else { + self.in_error_block = false; + None + } + } else { + None + } } - .context("Failed to execute command")?; - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let raw = format!("{}\n{}", stdout, stderr); - let filtered = filter_errors(&raw); - let mut rtk = String::new(); + fn flush(&mut self) -> String { + String::new() + } - if filtered.is_empty() { - if output.status.success() { - rtk.push_str("[ok] Command completed successfully (no errors)"); + fn on_exit(&mut self, exit_code: i32, raw: &str) -> Option { + if self.emitted_any { + return None; + } + if exit_code == 0 { + Some("[ok] Command completed successfully (no errors)".to_string()) } else { - rtk.push_str(&format!( - "[FAIL] Command failed (exit code: {:?})\n", - output.status.code() - )); + let mut msg = format!("[FAIL] Command failed (exit code: {})\n", exit_code); let lines: Vec<&str> = raw.lines().collect(); for line in lines.iter().rev().take(10).rev() { - rtk.push_str(&format!(" {}\n", line)); + msg.push_str(&format!(" {}\n", line)); } + Some(msg) } - } else { - rtk.push_str(&filtered); } +} - let exit_code = output - .status - .code() - .unwrap_or(if output.status.success() { 0 } else { 1 }); - if let Some(hint) = crate::core::tee::tee_and_hint(&raw, "err", exit_code) { - println!("{}\n{}", rtk, hint); +fn build_shell_command(command: &str) -> Command { + if cfg!(target_os = "windows") { + let mut c = Command::new("cmd"); + c.args(["/C", command]); + c } else { - println!("{}", rtk); + let mut c = Command::new("sh"); + c.args(["-c", command]); + c } - timer.track(command, "rtk run-err", &raw, &rtk); - Ok(()) } -/// Run tests and show only failures -pub fn run_test(command: &str, verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - +/// Run a command and filter output to show only errors/warnings +pub fn run_err(command: &str, verbose: u8) -> Result { if verbose > 0 { - eprintln!("Running tests: {}", command); - } - - let output = if cfg!(target_os = "windows") { - Command::new("cmd") - .args(["/C", command]) - .stdout(Stdio::piped()) - .stderr(Stdio::piped()) - .output() - } else { - Command::new("sh") - .args(["-c", command]) - .stdout(Stdio::piped()) - .stderr(Stdio::piped()) - .output() + eprintln!("Running: {}", command); } - .context("Failed to execute test command")?; - - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let raw = format!("{}\n{}", stdout, stderr); + let cmd = build_shell_command(command); + crate::core::runner::run_streamed( + cmd, + "err", + command, + Box::new(ErrorStreamFilter::new()), + crate::core::runner::RunOptions::with_tee("err"), + ) +} - let exit_code = output - .status - .code() - .unwrap_or(if output.status.success() { 0 } else { 1 }); - let summary = extract_test_summary(&raw, command); - if let Some(hint) = crate::core::tee::tee_and_hint(&raw, "test", exit_code) { - println!("{}\n{}", summary, hint); - } else { - println!("{}", summary); +/// Run tests and show only failures +pub fn run_test(command: &str, verbose: u8) -> Result { + if verbose > 0 { + eprintln!("Running tests: {}", command); } - timer.track(command, "rtk run-test", &raw, &summary); - Ok(()) + let cmd = build_shell_command(command); + let command_owned = command.to_string(); + crate::core::runner::run_filtered( + cmd, + "test", + command, + move |raw| extract_test_summary(raw, &command_owned), + crate::core::runner::RunOptions::with_tee("test"), + ) } +#[cfg(test)] fn filter_errors(output: &str) -> String { - lazy_static::lazy_static! { - static ref ERROR_PATTERNS: Vec = vec![ - // Generic errors - Regex::new(r"(?i)^.*error[\s:\[].*$").unwrap(), - Regex::new(r"(?i)^.*\berr\b.*$").unwrap(), - Regex::new(r"(?i)^.*warning[\s:\[].*$").unwrap(), - Regex::new(r"(?i)^.*\bwarn\b.*$").unwrap(), - Regex::new(r"(?i)^.*failed.*$").unwrap(), - Regex::new(r"(?i)^.*failure.*$").unwrap(), - Regex::new(r"(?i)^.*exception.*$").unwrap(), - Regex::new(r"(?i)^.*panic.*$").unwrap(), - // Rust specific - Regex::new(r"^error\[E\d+\]:.*$").unwrap(), - Regex::new(r"^\s*--> .*:\d+:\d+$").unwrap(), - // Python - Regex::new(r"^Traceback.*$").unwrap(), - Regex::new(r#"^\s*File ".*", line \d+.*$"#).unwrap(), - // JavaScript/TypeScript - Regex::new(r"^\s*at .*:\d+:\d+.*$").unwrap(), - // Go - Regex::new(r"^.*\.go:\d+:.*$").unwrap(), - ]; - } - let mut result = Vec::new(); let mut in_error_block = false; let mut blank_count = 0; @@ -150,7 +163,6 @@ fn filter_errors(output: &str) -> String { result.push(line.to_string()); } } else if line.starts_with(' ') || line.starts_with('\t') { - // Continuation of error result.push(line.to_string()); blank_count = 0; } else { @@ -166,20 +178,17 @@ fn extract_test_summary(output: &str, command: &str) -> String { let mut result = Vec::new(); let lines: Vec<&str> = output.lines().collect(); - // Detect test framework let is_cargo = command.contains("cargo test"); let is_pytest = command.contains("pytest"); let is_jest = command.contains("jest") || command.contains("npm test") || command.contains("yarn test"); let is_go = command.contains("go test"); - // Collect failures let mut failures = Vec::new(); let mut in_failure = false; let mut failure_lines = Vec::new(); for line in lines.iter() { - // Cargo test if is_cargo { if line.contains("test result:") { result.push(line.to_string()); @@ -195,7 +204,6 @@ fn extract_test_summary(output: &str, command: &str) -> String { } } - // Pytest if is_pytest { if line.contains(" passed") || line.contains(" failed") || line.contains(" error") { result.push(line.to_string()); @@ -205,7 +213,6 @@ fn extract_test_summary(output: &str, command: &str) -> String { } } - // Jest if is_jest { if line.contains("Tests:") || line.contains("Test Suites:") { result.push(line.to_string()); @@ -215,7 +222,6 @@ fn extract_test_summary(output: &str, command: &str) -> String { } } - // Go test if is_go { if line.starts_with("ok") || line.starts_with("FAIL") || line.starts_with("---") { result.push(line.to_string()); @@ -226,7 +232,6 @@ fn extract_test_summary(output: &str, command: &str) -> String { } } - // Build output let mut output = String::new(); if !failures.is_empty() { @@ -237,6 +242,12 @@ fn extract_test_summary(output: &str, command: &str) -> String { if failures.len() > 10 { output.push_str(&format!(" ... +{} more failures\n", failures.len() - 10)); } + for f in failure_lines.iter().take(20) { + output.push_str(&format!(" {}\n", f.trim())); + } + if failure_lines.len() > 20 { + output.push_str(&format!(" ... +{} more\n", failure_lines.len() - 20)); + } output.push('\n'); } @@ -246,7 +257,6 @@ fn extract_test_summary(output: &str, command: &str) -> String { output.push_str(&format!(" {}\n", r)); } } else { - // Fallback: show last few lines output.push_str("OUTPUT (last 5 lines):\n"); let start = lines.len().saturating_sub(5); for line in &lines[start..] { diff --git a/src/cmds/system/README.md b/src/cmds/system/README.md index ec3b327b3..1fdf13971 100644 --- a/src/cmds/system/README.md +++ b/src/cmds/system/README.md @@ -1,6 +1,6 @@ # System and Generic Utilities -> Part of [`src/cmds/`](../README.md) — see also [docs/TECHNICAL.md](../../../docs/TECHNICAL.md) +> Part of [`src/cmds/`](../README.md) — see also [docs/contributing/TECHNICAL.md](../../../docs/contributing/TECHNICAL.md) ## Specifics diff --git a/src/cmds/system/constants.rs b/src/cmds/system/constants.rs new file mode 100644 index 000000000..2454f5299 --- /dev/null +++ b/src/cmds/system/constants.rs @@ -0,0 +1,28 @@ +pub const NOISE_DIRS: &[&str] = &[ + "node_modules", + ".git", + "target", + "__pycache__", + ".next", + "dist", + "build", + ".cache", + ".turbo", + ".vercel", + ".pytest_cache", + ".mypy_cache", + ".tox", + ".venv", + "venv", + "env", + ".env", + "coverage", + ".nyc_output", + ".DS_Store", + "Thumbs.db", + ".idea", + ".vscode", + ".vs", + "*.egg-info", + ".eggs", +]; diff --git a/src/cmds/system/env_cmd.rs b/src/cmds/system/env_cmd.rs index 3b830fe42..72b20166a 100644 --- a/src/cmds/system/env_cmd.rs +++ b/src/cmds/system/env_cmd.rs @@ -4,6 +4,7 @@ use crate::core::tracking; use anyhow::Result; use std::collections::HashSet; use std::env; +use std::fmt::Write; /// Show filtered environment variables (hide sensitive data) pub fn run(filter: Option<&str>, show_all: bool, verbose: u8) -> Result<()> { @@ -123,7 +124,10 @@ pub fn run(filter: Option<&str>, show_all: bool, verbose: u8) -> Result<()> { println!("\nTotal: {} vars (showing {} relevant)", total, shown); } - let raw: String = vars.iter().map(|(k, v)| format!("{}={}\n", k, v)).collect(); + let raw: String = vars.iter().fold(String::new(), |mut output, (k, v)| { + let _ = writeln!(output, "{}={}", k, v); + output + }); let rtk = format!("{} vars -> {} shown", total, shown); timer.track("env", "rtk env", &raw, &rtk); Ok(()) @@ -204,3 +208,95 @@ fn is_interesting_var(key: &str) -> bool { let patterns = ["HOME", "USER", "LANG", "LC_", "TZ", "PWD", "OLDPWD"]; patterns.iter().any(|p| key.to_uppercase().starts_with(p)) } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn test_mask_value_short() { + assert_eq!(mask_value("abc"), "****"); + assert_eq!(mask_value(""), "****"); + } + + #[test] + fn test_mask_value_long() { + let result = mask_value("supersecrettoken"); + assert!(result.contains("****"), "Masked value should contain ****"); + assert!(result.starts_with("su"), "Should preserve 2-char prefix"); + assert!(result.ends_with("en"), "Should preserve 2-char suffix"); + } + + #[test] + fn test_mask_value_exactly_four() { + assert_eq!(mask_value("abcd"), "****"); + } + + #[test] + fn test_mask_value_five_chars() { + let result = mask_value("abcde"); + assert!(result.starts_with("ab")); + assert!(result.ends_with("de")); + } + + #[test] + fn test_is_lang_var_rust() { + assert!(is_lang_var("RUST_LOG")); + assert!(is_lang_var("CARGO_HOME")); + assert!(is_lang_var("GOPATH")); + assert!(is_lang_var("NODE_ENV")); + } + + #[test] + fn test_is_lang_var_negative() { + assert!(!is_lang_var("HOME")); + assert!(!is_lang_var("PATH")); + assert!(!is_lang_var("USER")); + } + + #[test] + fn test_is_cloud_var() { + assert!(is_cloud_var("AWS_ACCESS_KEY_ID")); + assert!(is_cloud_var("AZURE_CLIENT_ID")); + assert!(is_cloud_var("DOCKER_HOST")); + assert!(is_cloud_var("KUBERNETES_SERVICE_HOST")); + } + + #[test] + fn test_is_cloud_var_negative() { + assert!(!is_cloud_var("HOME")); + assert!(!is_cloud_var("RUST_LOG")); + } + + #[test] + fn test_is_tool_var() { + assert!(is_tool_var("EDITOR")); + assert!(is_tool_var("GIT_AUTHOR_NAME")); + assert!(is_tool_var("SSH_AUTH_SOCK")); + assert!(is_tool_var("CLAUDE_API_KEY")); + } + + #[test] + fn test_is_interesting_var() { + assert!(is_interesting_var("HOME")); + assert!(is_interesting_var("USER")); + assert!(is_interesting_var("LANG")); + assert!(is_interesting_var("TZ")); + assert!(is_interesting_var("PWD")); + } + + #[test] + fn test_is_interesting_var_negative() { + assert!(!is_interesting_var("RANDOM_VAR")); + assert!(!is_interesting_var("MY_CUSTOM_VAR")); + } + + #[test] + fn test_sensitive_patterns_contains_keys() { + let patterns = get_sensitive_patterns(); + assert!(patterns.contains("key")); + assert!(patterns.contains("secret")); + assert!(patterns.contains("password")); + assert!(patterns.contains("token")); + } +} diff --git a/src/cmds/system/find_cmd.rs b/src/cmds/system/find_cmd.rs index 942843fba..490619e2f 100644 --- a/src/cmds/system/find_cmd.rs +++ b/src/cmds/system/find_cmd.rs @@ -210,9 +210,13 @@ pub fn run( let want_dirs = file_type == "d"; + // When the pattern targets dotfiles (e.g. -name ".claude.json"), we must walk hidden + // entries; otherwise skip them to keep results tidy (#1101). + let search_hidden = effective_pattern.starts_with('.'); + let mut builder = WalkBuilder::new(path); builder - .hidden(true) // skip hidden files/dirs + .hidden(!search_hidden) // skip hidden files/dirs unless pattern targets dotfiles .git_ignore(true) // respect .gitignore .git_global(true) .git_exclude(true); @@ -560,6 +564,22 @@ mod tests { assert!(result.is_ok()); } + // --- #1101: dotfile pattern should not skip hidden files --- + + #[test] + fn find_dotfile_pattern_includes_hidden() { + // .gitignore exists at the repo root — must be found when using a dotfile pattern + let result = run(".gitignore", ".", 50, Some(1), "f", false, 0); + assert!(result.is_ok(), "run with dotfile pattern should not error"); + } + + #[test] + fn find_regular_pattern_skips_hidden() { + // Non-dot pattern should not error (hidden dirs remain skipped) + let result = run("*.rs", "src", 5, None, "f", false, 0); + assert!(result.is_ok()); + } + // --- integration: run on this repo --- #[test] diff --git a/src/cmds/system/format_cmd.rs b/src/cmds/system/format_cmd.rs index 4c4a31f4b..e147640ea 100644 --- a/src/cmds/system/format_cmd.rs +++ b/src/cmds/system/format_cmd.rs @@ -1,5 +1,6 @@ //! Runs code formatters (Prettier, Ruff) and shows only files that changed. +use crate::core::stream::exec_capture; use crate::core::tracking; use crate::core::utils::{package_manager_exec, resolved_command}; use crate::prettier_cmd; @@ -52,7 +53,7 @@ fn detect_formatter_in_dir(args: &[String], dir: &Path) -> String { "ruff".to_string() } -pub fn run(args: &[String], verbose: u8) -> Result<()> { +pub fn run(args: &[String], verbose: u8) -> Result { let timer = tracking::TimedExecution::start(); // Detect formatter @@ -111,14 +112,12 @@ pub fn run(args: &[String], verbose: u8) -> Result<()> { eprintln!("Running: {} {}", formatter, user_args.join(" ")); } - let output = cmd.output().context(format!( + let result = exec_capture(&mut cmd).context(format!( "Failed to run {}. Is it installed? Try: pip install {} (or npm/pnpm for JS formatters)", formatter, formatter ))?; - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let raw = format!("{}\n{}", stdout, stderr); + let raw = format!("{}\n{}", result.stdout, result.stderr); // Dispatch to appropriate filter based on formatter let filtered = match formatter.as_str() { @@ -137,12 +136,7 @@ pub fn run(args: &[String], verbose: u8) -> Result<()> { &filtered, ); - // Preserve exit code for CI/CD - if !output.status.success() { - std::process::exit(output.status.code().unwrap_or(1)); - } - - Ok(()) + Ok(result.exit_code) } /// Filter black output - show files that need formatting diff --git a/src/cmds/system/grep_cmd.rs b/src/cmds/system/grep_cmd.rs index 4550e8774..83b1886db 100644 --- a/src/cmds/system/grep_cmd.rs +++ b/src/cmds/system/grep_cmd.rs @@ -1,6 +1,7 @@ //! Filters grep output by grouping matches by file. use crate::core::config; +use crate::core::stream::exec_capture; use crate::core::tracking; use crate::core::utils::resolved_command; use anyhow::{Context, Result}; @@ -17,7 +18,7 @@ pub fn run( file_type: Option<&str>, extra_args: &[String], verbose: u8, -) -> Result<()> { +) -> Result { let timer = tracking::TimedExecution::start(); if verbose > 0 { @@ -28,7 +29,11 @@ pub fn run( let rg_pattern = pattern.replace(r"\|", "|"); let mut rg_cmd = resolved_command("rg"); - rg_cmd.args(["-n", "--no-heading", &rg_pattern, path]); + // --no-ignore-vcs: match grep -r behavior (don't skip .gitignore'd files). + // Without this, rg returns 0 matches for files in .gitignore, causing + // false negatives that make AI agents draw wrong conclusions. + // Using --no-ignore-vcs (not --no-ignore) so .ignore/.rgignore are still respected. + rg_cmd.args(["-n", "--no-heading", "--no-ignore-vcs", &rg_pattern, path]); if let Some(ft) = file_type { rg_cmd.arg("--type").arg(ft); @@ -42,26 +47,22 @@ pub fn run( rg_cmd.arg(arg); } - let output = rg_cmd - .output() + let result = exec_capture(&mut rg_cmd) .or_else(|_| { - resolved_command("grep") - .args(["-rn", pattern, path]) - .output() + let mut grep_cmd = resolved_command("grep"); + grep_cmd.args(["-rn", pattern, path]); + exec_capture(&mut grep_cmd) }) .context("grep/rg failed")?; - let stdout = String::from_utf8_lossy(&output.stdout); - let exit_code = output.status.code().unwrap_or(1); + let exit_code = result.exit_code; + let raw_output = result.stdout.clone(); - let raw_output = stdout.to_string(); - - if stdout.trim().is_empty() { + if result.stdout.trim().is_empty() { // Show stderr for errors (bad regex, missing file, etc.) if exit_code == 2 { - let stderr = String::from_utf8_lossy(&output.stderr); - if !stderr.trim().is_empty() { - eprintln!("{}", stderr.trim()); + if !result.stderr.trim().is_empty() { + eprintln!("{}", result.stderr.trim()); } } let msg = format!("0 matches for '{}'", pattern); @@ -72,23 +73,22 @@ pub fn run( &raw_output, &msg, ); - if exit_code != 0 { - std::process::exit(exit_code); - } - return Ok(()); + return Ok(exit_code); } - let mut by_file: HashMap> = HashMap::new(); - let mut total = 0; + // Always filter: truncate long lines, apply per-file and global caps. + // Output in standard file:line:content format that AI agents can parse. + // (A passthrough approach yields 0% savings — no reason for RTK to exist on that path.) + let total_matches = result.stdout.lines().count(); - // Compile context regex once (instead of per-line in clean_line) let context_re = if context_only { Regex::new(&format!("(?i).{{0,20}}{}.*", regex::escape(pattern))).ok() } else { None }; - for line in stdout.lines() { + let mut by_file: HashMap> = HashMap::new(); + for line in result.stdout.lines() { let parts: Vec<&str> = line.splitn(3, ':').collect(); let (file, line_num, content) = if parts.len() == 3 { @@ -101,43 +101,39 @@ pub fn run( continue; }; - total += 1; let cleaned = clean_line(content, max_line_len, context_re.as_ref(), pattern); by_file.entry(file).or_default().push((line_num, cleaned)); } let mut rtk_output = String::new(); - rtk_output.push_str(&format!("{} matches in {}F:\n\n", total, by_file.len())); + rtk_output.push_str(&format!( + "{} matches in {} files:\n\n", + total_matches, + by_file.len() + )); let mut shown = 0; let mut files: Vec<_> = by_file.iter().collect(); files.sort_by_key(|(f, _)| *f); + let per_file = config::limits().grep_max_per_file; for (file, matches) in files { if shown >= max_results { break; } let file_display = compact_path(file); - rtk_output.push_str(&format!("[file] {} ({}):\n", file_display, matches.len())); - - let per_file = config::limits().grep_max_per_file; for (line_num, content) in matches.iter().take(per_file) { - rtk_output.push_str(&format!(" {:>4}: {}\n", line_num, content)); - shown += 1; if shown >= max_results { break; } + rtk_output.push_str(&format!("{}:{}:{}\n", file_display, line_num, content)); + shown += 1; } - - if matches.len() > per_file { - rtk_output.push_str(&format!(" +{}\n", matches.len() - per_file)); - } - rtk_output.push('\n'); } - if total > shown { - rtk_output.push_str(&format!("... +{}\n", total - shown)); + if total_matches > shown { + rtk_output.push_str(&format!("[+{} more]\n", total_matches - shown)); } print!("{}", rtk_output); @@ -148,11 +144,7 @@ pub fn run( &rtk_output, ); - if exit_code != 0 { - std::process::exit(exit_code); - } - - Ok(()) + Ok(exit_code) } fn clean_line(line: &str, max_len: usize, context_re: Option<&Regex>, pattern: &str) -> String { @@ -320,4 +312,24 @@ mod tests { } // If rg is not installed, skip gracefully (test still passes) } + + #[test] + fn test_rg_no_ignore_vcs_flag_accepted() { + // Verify rg accepts --no-ignore-vcs (used to match grep -r behavior for .gitignore) + let mut cmd = resolved_command("rg"); + cmd.args([ + "-n", + "--no-heading", + "--no-ignore-vcs", + "NONEXISTENT_PATTERN_12345", + ".", + ]); + if let Ok(output) = cmd.output() { + assert!( + output.status.code() == Some(1) || output.status.success(), + "rg --no-ignore-vcs should be accepted" + ); + } + // If rg is not installed, skip gracefully (test still passes) + } } diff --git a/src/cmds/system/json_cmd.rs b/src/cmds/system/json_cmd.rs index 4e887417e..176b6e568 100644 --- a/src/cmds/system/json_cmd.rs +++ b/src/cmds/system/json_cmd.rs @@ -35,7 +35,7 @@ fn validate_json_extension(file: &Path) -> Result<()> { Ok(()) } -/// Show JSON (compact with values, or schema-only with --schema) +/// Show JSON (compact with values by default, or keys-only with --keys-only) pub fn run(file: &Path, max_depth: usize, schema_only: bool, verbose: u8) -> Result<()> { validate_json_extension(file)?; let timer = tracking::TimedExecution::start(); diff --git a/src/cmds/system/ls.rs b/src/cmds/system/ls.rs index 190d7c197..ac40ae328 100644 --- a/src/cmds/system/ls.rs +++ b/src/cmds/system/ls.rs @@ -1,42 +1,24 @@ //! Filters directory listings into a compact tree format. -use crate::core::tracking; +use super::constants::NOISE_DIRS; +use crate::core::runner::{self, RunOptions}; use crate::core::utils::resolved_command; -use anyhow::{Context, Result}; +use anyhow::Result; +use lazy_static::lazy_static; +use regex::Regex; use std::io::IsTerminal; -/// Noise directories commonly excluded from LLM context -const NOISE_DIRS: &[&str] = &[ - "node_modules", - ".git", - "target", - "__pycache__", - ".next", - "dist", - "build", - ".cache", - ".turbo", - ".vercel", - ".pytest_cache", - ".mypy_cache", - ".tox", - ".venv", - "venv", - "coverage", - ".nyc_output", - ".DS_Store", - "Thumbs.db", - ".idea", - ".vscode", - ".vs", - "*.egg-info", - ".eggs", -]; - -pub fn run(args: &[String], verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - - // Separate flags from paths +lazy_static! { + /// Matches the date+time portion in `ls -la` output, which serves as a + /// stable anchor regardless of owner/group column width. + /// E.g.: " Mar 31 16:18 " or " Dec 25 2024 " + static ref LS_DATE_RE: Regex = Regex::new( + r"\s+(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)\s+\d{1,2}\s+(?:\d{4}|\d{2}:\d{2})\s+" + ) + .unwrap(); +} + +pub fn run(args: &[String], verbose: u8) -> Result { let show_all = args .iter() .any(|a| (a.starts_with('-') && !a.starts_with("--") && a.contains('a')) || a == "--all"); @@ -52,13 +34,10 @@ pub fn run(args: &[String], verbose: u8) -> Result<()> { .map(|s| s.as_str()) .collect(); - // Build ls -la + any extra flags the user passed (e.g. -R) - // Strip -l, -a, -h (we handle all of these ourselves) let mut cmd = resolved_command("ls"); cmd.arg("-la"); for flag in &flags { if flag.starts_with("--") { - // Long flags: skip --all (already handled) if *flag != "--all" { cmd.arg(flag); } @@ -74,7 +53,6 @@ pub fn run(args: &[String], verbose: u8) -> Result<()> { } } - // Add paths (default to "." if none) if paths.is_empty() { cmd.arg("."); } else { @@ -83,52 +61,45 @@ pub fn run(args: &[String], verbose: u8) -> Result<()> { } } - let output = cmd.output().context("Failed to run ls")?; - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr); - eprint!("{}", stderr); - std::process::exit(output.status.code().unwrap_or(1)); - } - - let raw = String::from_utf8_lossy(&output.stdout).to_string(); - let (entries, summary) = compact_ls(&raw, show_all); - - // Only show summary in interactive mode (not when piped) - let is_tty = std::io::stdout().is_terminal(); - let filtered = if is_tty { - format!("{}{}", entries, summary) - } else { - entries - }; - - if verbose > 0 { - eprintln!( - "Chars: {} → {} ({}% reduction)", - raw.len(), - filtered.len(), - if !raw.is_empty() { - 100 - (filtered.len() * 100 / raw.len()) - } else { - 0 - } - ); - } - let target_display = if paths.is_empty() { ".".to_string() } else { paths.join(" ") }; - print!("{}", filtered); - timer.track( - &format!("ls -la {}", target_display), - "rtk ls", - &raw, - &filtered, - ); - - Ok(()) + + runner::run_filtered( + cmd, + "ls", + &format!("-la {}", target_display), + |raw| { + let (entries, summary) = compact_ls(raw, show_all); + + // Only show summary in interactive mode (not when piped) + let is_tty = std::io::stdout().is_terminal(); + let filtered = if is_tty { + format!("{}{}", entries, summary) + } else { + entries + }; + + if verbose > 0 { + eprintln!( + "Chars: {} → {} ({}% reduction)", + raw.len(), + filtered.len(), + if !raw.is_empty() { + 100 - (filtered.len() * 100 / raw.len()) + } else { + 0 + } + ); + } + filtered + }, + RunOptions::stdout_only() + .early_exit_on_failure() + .no_trailing_newline(), + ) } /// Format bytes into human-readable size @@ -142,6 +113,40 @@ fn human_size(bytes: u64) -> String { } } +/// Parse a single `ls -la` line, returning `(file_type_char, size, name)`. +/// +/// Uses the date field as a stable anchor — the date format in `ls -la` is +/// always three tokens (`Mon DD HH:MM` or `Mon DD YYYY`), so we locate it +/// with a regex, then extract size (rightmost number before the date) and +/// filename (everything after the date). This handles owner/group names that +/// contain spaces, which break the old fixed-column approach. +fn parse_ls_line(line: &str) -> Option<(char, u64, String)> { + let date_match = LS_DATE_RE.find(line)?; + let name = line[date_match.end()..].to_string(); + + let before_date = &line[..date_match.start()]; + let before_parts: Vec<&str> = before_date.split_whitespace().collect(); + if before_parts.len() < 4 { + return None; + } + + let perms = before_parts[0]; + let file_type = perms.chars().next()?; + + // Size is the rightmost parseable number before the date. + // nlinks is also numeric but appears earlier; scanning from the end + // guarantees we hit the size field first. + let mut size: u64 = 0; + for part in before_parts.iter().rev() { + if let Ok(s) = part.parse::() { + size = s; + break; + } + } + + Some((file_type, size, name)) +} + /// Parse ls -la output into compact format: /// name/ (dirs) /// name size (files) @@ -154,18 +159,13 @@ fn compact_ls(raw: &str, show_all: bool) -> (String, String) { let mut by_ext: HashMap = HashMap::new(); for line in raw.lines() { - // Skip total, empty, . and .. if line.starts_with("total ") || line.is_empty() { continue; } - let parts: Vec<&str> = line.split_whitespace().collect(); - if parts.len() < 9 { + let Some((file_type, size, name)) = parse_ls_line(line) else { continue; - } - - // Filename is everything from column 9 onward (handles spaces) - let name = parts[8..].join(" "); + }; // Skip . and .. if name == "." || name == ".." { @@ -177,12 +177,9 @@ fn compact_ls(raw: &str, show_all: bool) -> (String, String) { continue; } - let is_dir = parts[0].starts_with('d'); - - if is_dir { + if file_type == 'd' { dirs.push(name); - } else if parts[0].starts_with('-') || parts[0].starts_with('l') { - let size: u64 = parts[4].parse().unwrap_or(0); + } else if file_type == '-' || file_type == 'l' { let ext = if let Some(pos) = name.rfind('.') { name[pos..].to_string() } else { @@ -366,4 +363,109 @@ mod tests { line_count ); } + + // Regression test for #948: owner/group with spaces breaks fixed-column parsing + #[test] + fn test_compact_multiline_group() { + let input = "total 8\n\ + -rw-r--r-- 1 fjeanne utilisa. du domaine 0 Mar 31 16:18 empty.txt\n\ + -rw-r--r-- 1 fjeanne utilisa. du domaine 1234 Mar 31 16:18 data.json\n"; + let (entries, _summary) = compact_ls(input, false); + assert!( + entries.contains("empty.txt"), + "should contain 'empty.txt', got: {entries}" + ); + assert!( + entries.contains("data.json"), + "should contain 'data.json', got: {entries}" + ); + assert!( + !entries.contains("16:18"), + "time should not leak into filename, got: {entries}" + ); + assert!( + entries.contains("0B"), + "empty.txt should show 0B, got: {entries}" + ); + assert!( + entries.contains("1.2K"), + "data.json should show 1.2K (1234 bytes), got: {entries}" + ); + } + + #[test] + fn test_compact_year_format_date() { + // Some systems show year instead of time for old files + let input = "total 8\n\ + -rw-r--r-- 1 user staff 5678 Dec 25 2024 archive.tar\n"; + let (entries, _summary) = compact_ls(input, false); + assert!( + entries.contains("archive.tar"), + "should contain filename, got: {entries}" + ); + assert!( + entries.contains("5.5K"), + "should show 5.5K, got: {entries}" + ); + } + + #[test] + fn test_parse_ls_line_basic() { + let (ft, size, name) = parse_ls_line( + "-rw-r--r-- 1 user staff 1234 Jan 1 12:00 file.txt", + ) + .unwrap(); + assert_eq!(ft, '-'); + assert_eq!(size, 1234); + assert_eq!(name, "file.txt"); + } + + #[test] + fn test_parse_ls_line_multiline_group() { + let (ft, size, name) = parse_ls_line( + "-rw-r--r-- 1 fjeanne utilisa. du domaine 0 Mar 31 16:18 empty.txt", + ) + .unwrap(); + assert_eq!(ft, '-'); + assert_eq!(size, 0); + assert_eq!(name, "empty.txt"); + } + + #[test] + fn test_parse_ls_line_dir_with_space_in_group() { + let (ft, size, name) = parse_ls_line( + "drwxr-xr-x 2 fjeanne utilisa. du domaine 64 Mar 31 16:18 my dir", + ) + .unwrap(); + assert_eq!(ft, 'd'); + assert_eq!(size, 64); + assert_eq!(name, "my dir"); + } + + #[test] + fn test_parse_ls_line_symlink() { + let (ft, size, name) = parse_ls_line( + "lrwxr-xr-x 1 user staff 10 Jan 1 12:00 link -> target", + ) + .unwrap(); + assert_eq!(ft, 'l'); + assert_eq!(size, 10); + assert_eq!(name, "link -> target"); + } + + #[test] + fn test_parse_ls_line_returns_none_for_total() { + assert!(parse_ls_line("total 48").is_none()); + } + + #[test] + fn test_parse_ls_line_year_format() { + let (ft, size, name) = parse_ls_line( + "-rw-r--r-- 1 user staff 5678 Dec 25 2024 old.tar.gz", + ) + .unwrap(); + assert_eq!(ft, '-'); + assert_eq!(size, 5678); + assert_eq!(name, "old.tar.gz"); + } } diff --git a/src/cmds/system/mod.rs b/src/cmds/system/mod.rs index a7686922b..a4404bc1c 100644 --- a/src/cmds/system/mod.rs +++ b/src/cmds/system/mod.rs @@ -1,15 +1 @@ -//! General-purpose system command filters. - -pub mod deps; -pub mod env_cmd; -pub mod find_cmd; -pub mod format_cmd; -pub mod grep_cmd; -pub mod json_cmd; -pub mod local_llm; -pub mod log_cmd; -pub mod ls; -pub mod read; -pub mod summary; -pub mod tree; -pub mod wc_cmd; +automod::dir!(pub "src/cmds/system"); diff --git a/src/cmds/system/pipe_cmd.rs b/src/cmds/system/pipe_cmd.rs new file mode 100644 index 000000000..fe569a597 --- /dev/null +++ b/src/cmds/system/pipe_cmd.rs @@ -0,0 +1,534 @@ +use anyhow::Result; +use std::io::Read; + +use crate::core::stream::RAW_CAP; + +pub fn resolve_filter(name: &str) -> Option String> { + match name { + "cargo-test" | "cargo" => Some(crate::cmds::rust::cargo_cmd::filter_cargo_test), + "pytest" => Some(crate::cmds::python::pytest_cmd::filter_pytest_output), + "go-test" => Some(go_test_wrapper), + "go-build" => Some(crate::cmds::go::go_cmd::filter_go_build), + "tsc" => Some(crate::cmds::js::tsc_cmd::filter_tsc_output), + "vitest" => Some(vitest_wrapper), + "grep" | "rg" => Some(grep_wrapper), + "find" | "fd" => Some(find_wrapper), + "git-log" => Some(git_log_wrapper), + "git-diff" => Some(git_diff_wrapper), + "git-status" => Some(crate::cmds::git::git::format_status_output), + "mypy" => Some(crate::cmds::python::mypy_cmd::filter_mypy_output), + "ruff-check" => Some(crate::cmds::python::ruff_cmd::filter_ruff_check_json), + "ruff-format" => Some(crate::cmds::python::ruff_cmd::filter_ruff_format), + "prettier" => Some(crate::cmds::js::prettier_cmd::filter_prettier_output), + _ => None, + } +} + +fn go_test_wrapper(input: &str) -> String { + crate::cmds::go::go_cmd::filter_go_test_json(input) +} + +fn git_log_wrapper(input: &str) -> String { + crate::cmds::git::git::filter_log_output(input, 50, false, false) +} + +fn git_diff_wrapper(input: &str) -> String { + crate::cmds::git::git::compact_diff(input, 200) +} + +fn vitest_wrapper(input: &str) -> String { + use crate::cmds::js::vitest_cmd::VitestParser; + use crate::parser::{FormatMode, OutputParser, TokenFormatter}; + let result = VitestParser::parse(input); + match result { + crate::parser::ParseResult::Full(data) => data.format(FormatMode::Compact), + crate::parser::ParseResult::Degraded(data, _) => data.format(FormatMode::Compact), + crate::parser::ParseResult::Passthrough(raw) => raw, + } +} + +fn grep_wrapper(input: &str) -> String { + use std::collections::HashMap; + + let mut by_file: HashMap<&str, Vec<(&str, &str)>> = HashMap::new(); + let mut total = 0; + + for line in input.lines() { + let parts: Vec<&str> = line.splitn(3, ':').collect(); + if parts.len() == 3 { + if let Ok(_line_num) = parts[1].parse::() { + total += 1; + by_file.entry(parts[0]).or_default().push((parts[1], parts[2])); + } + } + } + + if total == 0 { + return input.to_string(); + } + + let mut out = format!("{} matches in {}F:\n\n", total, by_file.len()); + let mut files: Vec<_> = by_file.iter().collect(); + files.sort_by_key(|(f, _)| *f); + + for (file, matches) in files { + out.push_str(&format!("[file] {} ({}):\n", file, matches.len())); + for (line_num, content) in matches.iter().take(10) { + out.push_str(&format!(" {:>4}: {}\n", line_num, content.trim())); + } + if matches.len() > 10 { + out.push_str(&format!(" +{}\n", matches.len() - 10)); + } + out.push('\n'); + } + + out +} + +fn find_wrapper(input: &str) -> String { + use std::collections::HashMap; + + let paths: Vec<&str> = input.lines().filter(|l| !l.trim().is_empty()).collect(); + + if paths.is_empty() { + return input.to_string(); + } + + let mut by_dir: HashMap<&str, Vec<&str>> = HashMap::new(); + + for path in &paths { + let dir = match path.rfind('/') { + Some(pos) => &path[..pos], + None => ".", + }; + let name = match path.rfind('/') { + Some(pos) => &path[pos + 1..], + None => path, + }; + by_dir.entry(dir).or_default().push(name); + } + + let mut out = format!("{} files in {} dirs:\n\n", paths.len(), by_dir.len()); + let mut dirs: Vec<_> = by_dir.iter().collect(); + dirs.sort_by_key(|(d, _)| *d); + + for (dir, files) in dirs.iter().take(20) { + out.push_str(&format!("{}/ ({})\n", dir, files.len())); + for f in files.iter().take(10) { + out.push_str(&format!(" {}\n", f)); + } + if files.len() > 10 { + out.push_str(&format!(" +{}\n", files.len() - 10)); + } + } + + if dirs.len() > 20 { + out.push_str(&format!("\n+{} more dirs\n", dirs.len() - 20)); + } + + out +} + +pub fn auto_detect_filter(input: &str) -> fn(&str) -> String { + let end = input.len().min(1024); + // Avoid panic: byte 1024 may fall inside a multi-byte UTF-8 char + let end = input.floor_char_boundary(end); + let first_1k = &input[..end]; + + if first_1k.contains("test result:") && first_1k.contains("passed;") { + return crate::cmds::rust::cargo_cmd::filter_cargo_test; + } + + if first_1k.contains("=== test session starts") { + return crate::cmds::python::pytest_cmd::filter_pytest_output; + } + + let first_trimmed = first_1k.trim_start(); + if first_trimmed.starts_with('{') && first_1k.contains("\"Action\"") { + return go_test_wrapper; + } + + if first_1k.contains(": error:") && first_1k.contains(".py:") { + return crate::cmds::python::mypy_cmd::filter_mypy_output; + } + + // grep/rg: lines matching file:number:content + if first_1k + .lines() + .take(5) + .filter(|l| !l.trim().is_empty()) + .any(|l| { + let parts: Vec<_> = l.splitn(3, ':').collect(); + parts.len() == 3 && parts[1].parse::().is_ok() + }) + { + return grep_wrapper; + } + + if first_1k.contains("\"testResults\"") || first_1k.contains("\"numTotalTests\"") { + return vitest_wrapper; + } + + // find/fd: all non-empty lines look like file paths, minimum 3 lines + let path_like_lines: usize = first_1k + .lines() + .filter(|l| { + let t = l.trim(); + !t.is_empty() + && !t.contains(':') + && (t.starts_with('.') || t.starts_with('/') || t.contains('/')) + }) + .count(); + let nonempty_lines: usize = first_1k.lines().filter(|l| !l.trim().is_empty()).count(); + if nonempty_lines >= 3 && path_like_lines == nonempty_lines { + return find_wrapper; + } + + identity_filter +} + +fn identity_filter(input: &str) -> String { + input.to_string() +} + +fn apply_filter(filter_fn: fn(&str) -> String, input: &str) -> String { + std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| filter_fn(input))) + .unwrap_or_else(|_| { + eprintln!("[rtk] warning: filter panicked — passing through raw output"); + input.to_string() + }) +} + +pub fn run(filter_name: Option<&str>, passthrough: bool) -> Result<()> { + if passthrough { + std::io::copy(&mut std::io::stdin(), &mut std::io::stdout()) + .map_err(|e| anyhow::anyhow!("Failed to relay stdin: {}", e))?; + return Ok(()); + } + + let mut buf = String::new(); + std::io::stdin() + .take((RAW_CAP + 1) as u64) + .read_to_string(&mut buf) + .map_err(|e| anyhow::anyhow!("Failed to read stdin: {}", e))?; + if buf.len() > RAW_CAP { + anyhow::bail!("stdin exceeds {} byte limit", RAW_CAP); + } + + let filter_fn = match filter_name { + Some(name) => resolve_filter(name).ok_or_else(|| { + anyhow::anyhow!( + "Unknown filter '{}'. Available: cargo-test, pytest, go-test, go-build, \ + tsc, vitest, grep, rg, find, fd, git-log, git-diff, git-status, \ + mypy, ruff-check, ruff-format, prettier", + name + ) + })?, + None => auto_detect_filter(&buf), + }; + + let output = apply_filter(filter_fn, &buf); + print!("{}", output); + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn test_resolve_filter_cargo_test() { + let f = resolve_filter("cargo-test").expect("cargo-test filter must exist"); + let out = f("test result: ok. 5 passed; 0 failed"); + assert!(out.contains("passed") || out.contains("PASS"), "out={}", out); + } + + #[test] + fn test_resolve_filter_cargo_alias() { + assert!(resolve_filter("cargo").is_some()); + } + + #[test] + fn test_resolve_filter_grep() { + let f = resolve_filter("grep").expect("grep filter must exist"); + let input = "src/main.rs:42:fn main() {\nsrc/lib.rs:10:pub fn helper() {}\n"; + let out = f(input); + assert!( + out.contains("main.rs") || out.contains("matches"), + "out={}", + out + ); + } + + #[test] + fn test_resolve_filter_rg_alias() { + assert!(resolve_filter("rg").is_some()); + } + + #[test] + fn test_resolve_filter_pytest() { + assert!(resolve_filter("pytest").is_some()); + } + + #[test] + fn test_resolve_filter_go_test() { + assert!(resolve_filter("go-test").is_some()); + } + + #[test] + fn test_resolve_filter_tsc() { + assert!(resolve_filter("tsc").is_some()); + } + + #[test] + fn test_resolve_filter_vitest() { + assert!(resolve_filter("vitest").is_some()); + } + + #[test] + fn test_resolve_filter_git_log() { + assert!(resolve_filter("git-log").is_some()); + } + + #[test] + fn test_resolve_filter_git_diff() { + assert!(resolve_filter("git-diff").is_some()); + } + + #[test] + fn test_resolve_filter_git_status() { + assert!(resolve_filter("git-status").is_some()); + } + + #[test] + fn test_resolve_filter_unknown_returns_none() { + assert!(resolve_filter("nonexistent-filter").is_none()); + } + + #[test] + fn test_auto_detect_cargo_test() { + let input = "test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured\n"; + let f = auto_detect_filter(input); + let out = f(input); + assert!(!out.is_empty()); + } + + #[test] + fn test_auto_detect_pytest() { + let input = "=== test session starts ===\ncollected 3 items\n"; + let f = auto_detect_filter(input); + let out = f(input); + assert!(!out.is_empty()); + } + + #[test] + fn test_auto_detect_grep_format() { + let input = "src/main.rs:42:fn main() {\nsrc/lib.rs:10:pub fn helper() {}\n"; + let f = auto_detect_filter(input); + let out = f(input); + assert!(!out.is_empty()); + } + + #[test] + fn test_auto_detect_go_test_ndjson() { + let input = r#"{"Time":"2024-01-01T00:00:00Z","Action":"run","Package":"example/pkg"} +{"Time":"2024-01-01T00:00:01Z","Action":"pass","Package":"example/pkg","Elapsed":0.5} +"#; + let f = auto_detect_filter(input); + let out = f(input); + assert!(!out.is_empty()); + } + + #[test] + fn test_auto_detect_unknown_returns_identity() { + let input = "some random text that doesn't match any filter pattern\n"; + let f = auto_detect_filter(input); + let out = f(input); + assert_eq!(out, input); + } + + #[test] + fn test_git_log_wrapper() { + let input = "abc1234 Fix bug in parser (2 days ago) \n\ + def5678 Add new feature (3 days ago) \n"; + let out = git_log_wrapper(input); + assert!(!out.is_empty()); + } + + #[test] + fn test_git_diff_wrapper() { + let input = "diff --git a/src/main.rs b/src/main.rs\n\ + --- a/src/main.rs\n\ + +++ b/src/main.rs\n\ + @@ -1,3 +1,4 @@\n\ + +// new comment\n\ + fn main() {}\n"; + let out = git_diff_wrapper(input); + assert!(!out.is_empty()); + } + + #[test] + fn test_resolve_filter_find() { + let f = resolve_filter("find").expect("find filter must exist"); + let input = "./src/main.rs\n./src/lib.rs\n./tests/foo.rs\n"; + let out = f(input); + assert!(out.contains("3 files"), "out={}", out); + } + + #[test] + fn test_resolve_filter_fd_alias() { + assert!(resolve_filter("fd").is_some()); + } + + #[test] + fn test_auto_detect_find_paths() { + let input = "./src/main.rs\n./src/lib.rs\n./src/cmd/mod.rs\n./tests/foo.rs\n"; + let f = auto_detect_filter(input); + let out = f(input); + assert!(out.contains("4 files"), "out={}", out); + } + + #[test] + fn test_auto_detect_find_absolute_paths() { + let input = "/home/user/src/main.rs\n/home/user/src/lib.rs\n/home/user/tests/foo.rs\n"; + let f = auto_detect_filter(input); + let out = f(input); + assert!(out.contains("3 files"), "out={}", out); + } + + #[test] + fn test_auto_detect_find_not_triggered_for_few_lines() { + let input = "./src/main.rs\n./src/lib.rs\n"; + let f = auto_detect_filter(input); + let out = f(input); + assert_eq!(out, input); + } + + #[test] + fn test_auto_detect_find_not_triggered_for_grep_output() { + let input = "src/main.rs:42:fn main() {\nsrc/lib.rs:10:pub fn helper() {}\nsrc/a.rs:1:x\n"; + let f = auto_detect_filter(input); + let out = f(input); + assert!( + !out.contains("files"), + "should not trigger find filter: out={}", + out + ); + } + + #[test] + fn test_auto_detect_empty_input_is_identity() { + let f = auto_detect_filter(""); + let out = f(""); + assert_eq!(out, ""); + } + + #[test] + fn test_auto_detect_multibyte_at_1024_boundary() { + // Build input where byte 1024 falls inside a multi-byte char (é = 2 bytes) + let mut input = "a".repeat(1023); + input.push('é'); // 2-byte char starting at byte 1023, ends at 1025 + let f = auto_detect_filter(&input); + let out = f(&input); + assert_eq!(out, input); + } + + #[test] + fn test_auto_detect_single_line_unknown() { + let input = "hello world\n"; + let f = auto_detect_filter(input); + let out = f(input); + assert_eq!(out, input); + } + + #[test] + fn test_resolve_filter_go_build() { + assert!(resolve_filter("go-build").is_some()); + } + + #[test] + fn test_resolve_filter_mypy() { + assert!(resolve_filter("mypy").is_some()); + } + + #[test] + fn test_resolve_filter_ruff_check() { + assert!(resolve_filter("ruff-check").is_some()); + } + + #[test] + fn test_resolve_filter_ruff_format() { + assert!(resolve_filter("ruff-format").is_some()); + } + + #[test] + fn test_resolve_filter_prettier() { + assert!(resolve_filter("prettier").is_some()); + } + + #[test] + fn test_panicking_filter_returns_passthrough() { + fn panicking_filter(_input: &str) -> String { + panic!("filter bug"); + } + let input = "some output\n"; + let result = super::apply_filter(panicking_filter, input); + assert_eq!(result, input); + } + + fn count_tokens(s: &str) -> usize { + s.split_whitespace().count() + } + + #[test] + fn test_grep_wrapper_token_savings() { + // Realistic rg output: 200 matches across 10 files (20 per file → 10 shown + truncation) + let mut input = String::new(); + for file_idx in 1..=10 { + for line in 1..=20 { + input.push_str(&format!( + "src/cmds/module{}/handler.rs:{}: let result = process_request(ctx, &payload).await?;\n", + file_idx, line * 10 + )); + } + } + let output = grep_wrapper(&input); + let savings = 100.0 - (count_tokens(&output) as f64 / count_tokens(&input) as f64 * 100.0); + assert!( + savings >= 40.0, // TODO: grep pipe filter below 60% target — improve grouping + "grep filter: expected ≥40% savings, got {:.1}% (in={}, out={})", + savings, count_tokens(&input), count_tokens(&output) + ); + } + + #[test] + fn test_find_wrapper_token_savings() { + // Realistic find output: 500 files across 30 dirs (20-dir cap + 10-file cap both trigger) + let mut input = String::new(); + for dir in 1..=30 { + for file in 1..=17 { + input.push_str(&format!( + "./src/components/feature{}/sub_{}/component_{}.tsx\n", + dir, dir, file + )); + } + } + let output = find_wrapper(&input); + let savings = 100.0 - (count_tokens(&output) as f64 / count_tokens(&input) as f64 * 100.0); + assert!( + savings >= 40.0, // TODO: find pipe filter below 60% target — improve grouping + "find filter: expected ≥40% savings, got {:.1}% (in={}, out={})", + savings, count_tokens(&input), count_tokens(&output) + ); + } + + #[test] + fn test_auto_detect_mypy_output() { + let input = "src/app.py:42: error: Argument 1 has incompatible type [arg-type]\n\ + src/utils.py:10: error: Missing return statement [return]\n\ + Found 2 errors in 2 files\n"; + let f = auto_detect_filter(input); + let out = f(input); + assert!(!out.is_empty()); + } +} diff --git a/src/cmds/system/read.rs b/src/cmds/system/read.rs index 1b53d11ff..2f56687ee 100644 --- a/src/cmds/system/read.rs +++ b/src/cmds/system/read.rs @@ -70,7 +70,7 @@ pub fn run( } else { filtered.clone() }; - println!("{}", rtk_output); + print!("{}", rtk_output); timer.track( &format!("cat {}", file.display()), "rtk read", @@ -134,7 +134,7 @@ pub fn run_stdin( } else { filtered.clone() }; - println!("{}", rtk_output); + print!("{}", rtk_output); timer.track("cat - (stdin)", "rtk read -", &content, &rtk_output); Ok(()) @@ -226,4 +226,74 @@ fn main() {{ assert!(output.starts_with("a\n")); assert!(output.contains("more lines")); } + + fn rtk_bin() -> std::path::PathBuf { + std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("target") + .join("debug") + .join("rtk") + } + + #[test] + #[ignore] + fn test_read_two_valid_files_concatenated() { + let bin = rtk_bin(); + assert!(bin.exists(), "Run `cargo build` first"); + + let mut f1 = NamedTempFile::with_suffix(".txt").unwrap(); + let mut f2 = NamedTempFile::with_suffix(".txt").unwrap(); + writeln!(f1, "alpha\nbravo").unwrap(); + writeln!(f2, "charlie\ndelta").unwrap(); + + let output = std::process::Command::new(&bin) + .args(["read", &f1.path().to_string_lossy(), &f2.path().to_string_lossy()]) + .output() + .expect("failed to run rtk read"); + + assert!(output.status.success()); + let stdout = String::from_utf8_lossy(&output.stdout); + assert!(stdout.contains("alpha"), "first file content missing"); + assert!(stdout.contains("charlie"), "second file content missing"); + } + + #[test] + #[ignore] + fn test_read_valid_and_nonexistent() { + let bin = rtk_bin(); + assert!(bin.exists(), "Run `cargo build` first"); + + let mut f1 = NamedTempFile::with_suffix(".txt").unwrap(); + writeln!(f1, "valid content").unwrap(); + + let output = std::process::Command::new(&bin) + .args(["read", &f1.path().to_string_lossy(), "/tmp/rtk_nonexistent_file.txt"]) + .output() + .expect("failed to run rtk read"); + + assert!(!output.status.success(), "should exit non-zero on missing file"); + let stdout = String::from_utf8_lossy(&output.stdout); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!(stdout.contains("valid content"), "valid file should still be printed"); + assert!(stderr.contains("rtk_nonexistent_file"), "should report missing file on stderr"); + } + + #[test] + #[ignore] + fn test_read_stdin_dedup_warning() { + let bin = rtk_bin(); + assert!(bin.exists(), "Run `cargo build` first"); + + let output = std::process::Command::new(&bin) + .args(["read", "-", "-"]) + .stdin(std::process::Stdio::piped()) + .output() + .expect("failed to run rtk read"); + + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("stdin specified more than once"), + "should warn about duplicate stdin, got stderr: {}", + stderr + ); + } } diff --git a/src/cmds/system/summary.rs b/src/cmds/system/summary.rs index be44f883c..c6e2ec051 100644 --- a/src/cmds/system/summary.rs +++ b/src/cmds/system/summary.rs @@ -1,42 +1,37 @@ //! Runs a command and produces a heuristic summary of its output. +use crate::core::stream::exec_capture; use crate::core::tracking; use crate::core::utils::truncate; use anyhow::{Context, Result}; use regex::Regex; -use std::process::{Command, Stdio}; +use std::process::Command; /// Run a command and provide a heuristic summary -pub fn run(command: &str, verbose: u8) -> Result<()> { +pub fn run(command: &str, verbose: u8) -> Result { let timer = tracking::TimedExecution::start(); if verbose > 0 { eprintln!("Running and summarizing: {}", command); } - let output = if cfg!(target_os = "windows") { - Command::new("cmd") - .args(["/C", command]) - .stdout(Stdio::piped()) - .stderr(Stdio::piped()) - .output() + let mut cmd = if cfg!(target_os = "windows") { + let mut c = Command::new("cmd"); + c.args(["/C", command]); + c } else { - Command::new("sh") - .args(["-c", command]) - .stdout(Stdio::piped()) - .stderr(Stdio::piped()) - .output() - } - .context("Failed to execute command")?; + let mut c = Command::new("sh"); + c.args(["-c", command]); + c + }; + let result = exec_capture(&mut cmd).context("Failed to execute command")?; - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let raw = format!("{}\n{}", stdout, stderr); + let raw = format!("{}\n{}", result.stdout, result.stderr); - let summary = summarize_output(&raw, command, output.status.success()); + let summary = summarize_output(&raw, command, result.success()); println!("{}", summary); timer.track(command, "rtk summary", &raw, &summary); - Ok(()) + Ok(result.exit_code) } fn summarize_output(output: &str, command: &str, success: bool) -> String { diff --git a/src/cmds/system/tree.rs b/src/cmds/system/tree.rs index 57706a6a2..576e6c800 100644 --- a/src/cmds/system/tree.rs +++ b/src/cmds/system/tree.rs @@ -6,44 +6,12 @@ //! Token optimization: automatically excludes noise directories via -I pattern //! unless -a flag is present (respecting user intent). -use crate::core::tracking; +use super::constants::NOISE_DIRS; +use crate::core::runner::{self, RunOptions}; use crate::core::utils::{resolved_command, tool_exists}; -use anyhow::{Context, Result}; - -/// Noise directories commonly excluded from LLM context -const NOISE_DIRS: &[&str] = &[ - "node_modules", - ".git", - "target", - "__pycache__", - ".next", - "dist", - "build", - ".cache", - ".turbo", - ".vercel", - ".pytest_cache", - ".mypy_cache", - ".tox", - ".venv", - "venv", - "env", - ".env", - "coverage", - ".nyc_output", - ".DS_Store", - "Thumbs.db", - ".idea", - ".vscode", - ".vs", - "*.egg-info", - ".eggs", -]; - -pub fn run(args: &[String], verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); - - // Check if tree is installed +use anyhow::Result; + +pub fn run(args: &[String], verbose: u8) -> Result { if !tool_exists("tree") { anyhow::bail!( "tree command not found. Install it first:\n\ @@ -56,49 +24,42 @@ pub fn run(args: &[String], verbose: u8) -> Result<()> { let mut cmd = resolved_command("tree"); - // Determine if user wants all files or default behavior let show_all = args.iter().any(|a| a == "-a" || a == "--all"); let has_ignore = args.iter().any(|a| a == "-I" || a.starts_with("--ignore=")); - // Auto-inject -I pattern unless user wants all or already specified -I if !show_all && !has_ignore { let ignore_pattern = NOISE_DIRS.join("|"); cmd.arg("-I").arg(&ignore_pattern); } - // Pass all user args for arg in args { cmd.arg(arg); } - let output = cmd.output().context("Failed to run tree")?; - - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr); - eprint!("{}", stderr); - std::process::exit(output.status.code().unwrap_or(1)); - } - - let raw = String::from_utf8_lossy(&output.stdout).to_string(); - let filtered = filter_tree_output(&raw); - - if verbose > 0 { - eprintln!( - "Lines: {} → {} ({}% reduction)", - raw.lines().count(), - filtered.lines().count(), - if raw.lines().count() > 0 { - 100 - (filtered.lines().count() * 100 / raw.lines().count()) - } else { - 0 + runner::run_filtered( + cmd, + "tree", + &args.join(" "), + |raw| { + let filtered = filter_tree_output(raw); + if verbose > 0 { + eprintln!( + "Lines: {} → {} ({}% reduction)", + raw.lines().count(), + filtered.lines().count(), + if raw.lines().count() > 0 { + 100 - (filtered.lines().count() * 100 / raw.lines().count()) + } else { + 0 + } + ); } - ); - } - - print!("{}", filtered); - timer.track("tree", "rtk tree", &raw, &filtered); - - Ok(()) + filtered + }, + RunOptions::stdout_only() + .early_exit_on_failure() + .no_trailing_newline(), + ) } fn filter_tree_output(raw: &str) -> String { diff --git a/src/cmds/system/wc_cmd.rs b/src/cmds/system/wc_cmd.rs index 14e4f69f1..dcfe98b39 100644 --- a/src/cmds/system/wc_cmd.rs +++ b/src/cmds/system/wc_cmd.rs @@ -6,13 +6,11 @@ /// - `wc -w file.py` → `96` /// - `wc -c file.py` → `978` /// - `wc -l *.py` → table with common path prefix stripped -use crate::core::tracking; +use crate::core::runner::{self, RunOptions}; use crate::core::utils::resolved_command; -use anyhow::{Context, Result}; - -pub fn run(args: &[String], verbose: u8) -> Result<()> { - let timer = tracking::TimedExecution::start(); +use anyhow::Result; +pub fn run(args: &[String], verbose: u8) -> Result { let mut cmd = resolved_command("wc"); for arg in args { cmd.arg(arg); @@ -22,35 +20,14 @@ pub fn run(args: &[String], verbose: u8) -> Result<()> { eprintln!("Running: wc {}", args.join(" ")); } - let output = cmd.output().context("Failed to run wc")?; - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - - if !output.status.success() { - let msg = if stderr.trim().is_empty() { - stdout.trim().to_string() - } else { - stderr.trim().to_string() - }; - eprintln!("FAILED: wc {}", msg); - std::process::exit(output.status.code().unwrap_or(1)); - } - - let raw = stdout.to_string(); - - // Detect which columns the user requested let mode = detect_mode(args); - let filtered = filter_wc_output(&raw, &mode); - println!("{}", filtered); - - timer.track( - &format!("wc {}", args.join(" ")), - &format!("rtk wc {}", args.join(" ")), - &raw, - &filtered, - ); - - Ok(()) + runner::run_filtered( + cmd, + "wc", + &args.join(" "), + |stdout| filter_wc_output(stdout, &mode), + RunOptions::stdout_only(), + ) } /// Which columns the user requested diff --git a/src/core/README.md b/src/core/README.md index 29befe0c3..81b4b0de7 100644 --- a/src/core/README.md +++ b/src/core/README.md @@ -1,17 +1,17 @@ # Core Infrastructure -> See also [docs/TECHNICAL.md](../../docs/TECHNICAL.md) for the full architecture overview +> See also [docs/contributing/TECHNICAL.md](../../docs/contributing/TECHNICAL.md) for the full architecture overview ## Scope Domain-agnostic building blocks with **no knowledge of any specific command, hook, or agent**. If a module references "git", "cargo", "claude", or any external tool by name, it does not belong here. Core is a leaf in the dependency graph — it is consumed by all other components but imports from none of them. -Owns: configuration loading, token tracking persistence, TOML filter engine, tee output recovery, display formatting, telemetry, and shared utilities. +Owns: configuration loading, token tracking persistence, TOML filter engine, tee output recovery, display formatting, and shared utilities. Does **not** own: command-specific filtering logic (that's `cmds/`), hook lifecycle management (that's `src/hooks/`), or analytics dashboards (that's `analytics/`). ## Purpose -Core infrastructure shared by all RTK command modules. These are the foundational building blocks that every filter, tracker, and command handler depends on. This module group has no inward dependencies — it is a leaf in the dependency graph, ensuring clean layering across the codebase. +Core infrastructure shared by all RTK command modules. Every filter, tracker, and command handler depends on these modules. No inward dependencies — leaf in the dependency graph (no circular imports possible). ## TOML Filter Pipeline @@ -78,9 +78,6 @@ max_files = 20 max_file_size = 1048576 directory = "/custom/tee/dir" -[telemetry] -enabled = true - [hooks] exclude_commands = ["curl", "playwright"] # Never auto-rewrite these @@ -119,11 +116,7 @@ Consumers must call `timer.track()` on **all** code paths — success, failure, Consumers that parse structured output (JSON, NDJSON, state machines) should call `tee::tee_and_hint()` to save raw output for LLM recovery on failure. Tee must be called before `std::process::exit()`. -### Gaps (to be fixed) - -- `ls.rs`, `tree.rs` — exit before `track()` on error path (lost metrics) -- `container.rs` — inconsistent tracking across subcommands -- Many command modules still missing tee integration — see `src/cmds/README.md` for the full list +For truncation recovery on **success** (e.g., list truncated at 20 items), use `tee::force_tee_hint()` which bypasses the tee mode check and writes regardless of exit code. This ensures LLMs always have a `[full output: ...]` recovery path instead of burning tokens working around missing data. ## Adding New Functionality Place new infrastructure code here if it meets **all** of these criteria: (1) it has no dependencies on command modules or hooks, (2) it is used by two or more other modules, and (3) it provides a general-purpose utility rather than command-specific logic. Follow the existing pattern of lazy-initialized resources (`lazy_static!` for regex, on-demand config loading) to preserve the <10ms startup target. Add `#[cfg(test)] mod tests` with unit tests in the same file. diff --git a/src/core/config.rs b/src/core/config.rs index 99e28c213..2d110229e 100644 --- a/src/core/config.rs +++ b/src/core/config.rs @@ -1,5 +1,6 @@ //! Reads user settings from config.toml. +use super::constants::{CONFIG_TOML, DEFAULT_HISTORY_DAYS, RTK_DATA_DIR}; use anyhow::Result; use serde::{Deserialize, Serialize}; use std::path::PathBuf; @@ -15,8 +16,6 @@ pub struct Config { #[serde(default)] pub tee: crate::core::tee::TeeConfig, #[serde(default)] - pub telemetry: TelemetryConfig, - #[serde(default)] pub hooks: HooksConfig, #[serde(default)] pub limits: LimitsConfig, @@ -42,7 +41,7 @@ impl Default for TrackingConfig { fn default() -> Self { Self { enabled: true, - history_days: 90, + history_days: DEFAULT_HISTORY_DAYS as u32, database_path: None, } } @@ -87,17 +86,6 @@ impl Default for FilterConfig { } } -#[derive(Debug, Serialize, Deserialize)] -pub struct TelemetryConfig { - pub enabled: bool, -} - -impl Default for TelemetryConfig { - fn default() -> Self { - Self { enabled: true } - } -} - #[derive(Debug, Serialize, Deserialize)] pub struct LimitsConfig { /// Max total grep results to show (default: 200) @@ -129,11 +117,6 @@ pub fn limits() -> LimitsConfig { Config::load().map(|c| c.limits).unwrap_or_default() } -/// Check if telemetry is enabled in config. Returns None if config can't be loaded. -pub fn telemetry_enabled() -> Option { - Config::load().ok().map(|c| c.telemetry.enabled) -} - impl Config { pub fn load() -> Result { let path = get_config_path()?; @@ -168,7 +151,7 @@ impl Config { fn get_config_path() -> Result { let config_dir = dirs::config_dir().unwrap_or_else(|| PathBuf::from(".")); - Ok(config_dir.join("rtk").join("config.toml")) + Ok(config_dir.join(RTK_DATA_DIR).join(CONFIG_TOML)) } pub fn show_config() -> Result<()> { diff --git a/src/core/constants.rs b/src/core/constants.rs new file mode 100644 index 000000000..a5ecd3846 --- /dev/null +++ b/src/core/constants.rs @@ -0,0 +1,6 @@ +pub const RTK_DATA_DIR: &str = "rtk"; +pub const HISTORY_DB: &str = "history.db"; +pub const CONFIG_TOML: &str = "config.toml"; +pub const FILTERS_TOML: &str = "filters.toml"; +pub const TRUSTED_FILTERS_JSON: &str = "trusted_filters.json"; +pub const DEFAULT_HISTORY_DAYS: i64 = 90; diff --git a/src/core/filter.rs b/src/core/filter.rs index 1fc5a23fb..90f89ade6 100644 --- a/src/core/filter.rs +++ b/src/core/filter.rs @@ -36,8 +36,6 @@ impl std::fmt::Display for FilterLevel { pub trait FilterStrategy { fn filter(&self, content: &str, lang: &Language) -> String; - #[allow(dead_code)] - fn name(&self) -> &'static str; } #[derive(Debug, Clone, Copy, PartialEq, Eq)] @@ -153,10 +151,6 @@ impl FilterStrategy for NoFilter { fn filter(&self, content: &str, _lang: &Language) -> String { content.to_string() } - - fn name(&self) -> &'static str { - "none" - } } pub struct MinimalFilter; @@ -234,10 +228,6 @@ impl FilterStrategy for MinimalFilter { let result = MULTIPLE_BLANK_LINES.replace_all(&result, "\n\n"); result.trim().to_string() } - - fn name(&self) -> &'static str { - "minimal" - } } pub struct AggressiveFilter; @@ -320,10 +310,6 @@ impl FilterStrategy for AggressiveFilter { result.trim().to_string() } - - fn name(&self) -> &'static str { - "aggressive" - } } pub fn get_filter(level: FilterLevel) -> Box { @@ -340,14 +326,15 @@ pub fn smart_truncate(content: &str, max_lines: usize, _lang: &Language) -> Stri return content.to_string(); } - let mut result = Vec::with_capacity(max_lines); + let mut result = Vec::with_capacity(max_lines + 1); let mut kept_lines = 0; - let mut skipped_section = false; for line in &lines { let trimmed = line.trim(); - // Always keep signatures and important structural elements + // Prioritize structurally important lines so the visible window stays useful. + // The old approach interleaved "// ... N lines omitted" markers which AI agents + // treated as code, causing parsing confusion and extra retry loops. let is_important = FUNC_SIGNATURE.is_match(trimmed) || IMPORT_PATTERN.is_match(trimmed) || trimmed.starts_with("pub ") @@ -356,31 +343,20 @@ pub fn smart_truncate(content: &str, max_lines: usize, _lang: &Language) -> Stri || trimmed == "{"; if is_important || kept_lines < max_lines / 2 { - if skipped_section { - result.push(format!( - " // ... {} lines omitted", - lines.len() - kept_lines - )); - skipped_section = false; - } result.push((*line).to_string()); kept_lines += 1; - } else { - skipped_section = true; } + // Non-important lines beyond max_lines/2 are silently skipped — + // no inline markers that could be mistaken for file content. if kept_lines >= max_lines - 1 { break; } } - if skipped_section || kept_lines < lines.len() { - result.push(format!( - "// ... {} more lines (total: {})", - lines.len() - kept_lines, - lines.len() - )); - } + // Single end-of-output marker: not code syntax, unambiguous to AI agents. + // Invariant: kept_lines + N == lines.len() (N = lines not shown) + result.push(format!("[{} more lines]", lines.len() - kept_lines)); result.join("\n") } @@ -498,10 +474,10 @@ fn main() { #[test] fn test_smart_truncate_overflow_count_exact() { - // 200 plain-text lines with max_lines=20. - // smart_truncate keeps the first max_lines/2=10 lines, then skips the rest. - // The overflow message "// ... N more lines (total: T)" must satisfy: - // kept_count + N == T + // 200 plain-text lines (no function signatures/imports) with max_lines=20. + // Smart selection keeps up to max_lines/2=10 non-important lines then stops. + // The overflow message "[N more lines]" must satisfy: + // kept_count + N == total_lines let total_lines = 200usize; let max_lines = 20usize; let content: String = (0..total_lines) @@ -517,11 +493,12 @@ fn main() { .find(|l| l.contains("more lines")) .unwrap_or_else(|| panic!("No overflow message found in:\n{}", output)); - // Parse "// ... N more lines (total: T)" + // Parse "[N more lines]" let reported_more: usize = overflow_line - .split_whitespace() - .find(|w| w.parse::().is_ok()) - .and_then(|w| w.parse().ok()) + .trim() + .strip_prefix('[') + .and_then(|s| s.split_whitespace().next()) + .and_then(|n| n.parse().ok()) .unwrap_or_else(|| panic!("Could not parse overflow count from: {}", overflow_line)); let kept_count = output @@ -538,4 +515,36 @@ fn main() { total_lines ); } + + #[test] + fn test_smart_truncate_no_annotations() { + // 10 plain-text lines, max_lines=3: smart logic keeps first max_lines/2=1 line. + // (None of the lines match FUNC_SIGNATURE or IMPORT_PATTERN patterns.) + let input = "line1\nline2\nline3\nline4\nline5\nline6\nline7\nline8\nline9\nline10\n"; + let output = smart_truncate(input, 3, &Language::Unknown); + // Must NOT contain old-style "// ... N lines omitted" annotations + assert!( + !output.contains("// ..."), + "smart_truncate must not insert synthetic comment annotations" + ); + // Must contain clean end-of-output marker (1 kept + 9 omitted = 10 total) + assert!(output.contains("[9 more lines]")); + // Only the first line is kept (plain-text, no important signatures) + assert!(output.starts_with("line1\n")); + } + + #[test] + fn test_smart_truncate_no_truncation_when_under_limit() { + let input = "a\nb\nc\n"; + let output = smart_truncate(input, 10, &Language::Unknown); + assert_eq!(output, input); + assert!(!output.contains("more lines")); + } + + #[test] + fn test_smart_truncate_exact_limit() { + let input = "a\nb\nc"; + let output = smart_truncate(input, 3, &Language::Unknown); + assert_eq!(output, input); + } } diff --git a/src/core/mod.rs b/src/core/mod.rs index 0a490aad3..454a7f8a4 100644 --- a/src/core/mod.rs +++ b/src/core/mod.rs @@ -1,10 +1,12 @@ //! Building blocks shared across all RTK modules. pub mod config; +pub mod constants; pub mod display_helpers; pub mod filter; +pub mod runner; +pub mod stream; pub mod tee; -pub mod telemetry; pub mod toml_filter; pub mod tracking; pub mod utils; diff --git a/src/core/runner.rs b/src/core/runner.rs new file mode 100644 index 000000000..f127a6081 --- /dev/null +++ b/src/core/runner.rs @@ -0,0 +1,201 @@ +//! Shared command execution skeleton for filter modules. + +use anyhow::{Context, Result}; +use std::process::Command; + +use crate::core::stream::{self, FilterMode, StdinMode, StreamFilter}; +use crate::core::tracking; + +pub fn print_with_hint(filtered: &str, raw: &str, tee_label: &str, exit_code: i32) { + if let Some(hint) = crate::core::tee::tee_and_hint(raw, tee_label, exit_code) { + println!("{}\n{}", filtered, hint); + } else { + println!("{}", filtered); + } +} + +#[derive(Default)] +pub struct RunOptions<'a> { + pub tee_label: Option<&'a str>, + pub filter_stdout_only: bool, + pub skip_filter_on_failure: bool, + pub no_trailing_newline: bool, +} + +impl<'a> RunOptions<'a> { + pub fn with_tee(label: &'a str) -> Self { + Self { + tee_label: Some(label), + ..Default::default() + } + } + + pub fn stdout_only() -> Self { + Self { + filter_stdout_only: true, + ..Default::default() + } + } + + pub fn tee(mut self, label: &'a str) -> Self { + self.tee_label = Some(label); + self + } + + pub fn early_exit_on_failure(mut self) -> Self { + self.skip_filter_on_failure = true; + self + } + + pub fn no_trailing_newline(mut self) -> Self { + self.no_trailing_newline = true; + self + } +} + +pub enum RunMode<'a> { + Filtered(Box String + 'a>), + Streamed(Box), + Passthrough, +} + +pub fn run( + mut cmd: Command, + tool_name: &str, + args_display: &str, + mode: RunMode<'_>, + opts: RunOptions<'_>, +) -> Result { + let timer = tracking::TimedExecution::start(); + let cmd_label = format!("{} {}", tool_name, args_display); + + match mode { + RunMode::Filtered(filter_fn) => { + let result = stream::run_streaming(&mut cmd, StdinMode::Null, FilterMode::CaptureOnly) + .with_context(|| format!("Failed to run {}", tool_name))?; + + let exit_code = result.exit_code; + let raw = &result.raw; + let raw_stdout = &result.raw_stdout; + + if opts.skip_filter_on_failure && exit_code != 0 { + if !result.raw_stdout.trim().is_empty() { + print!("{}", result.raw_stdout); + } + if !result.raw_stderr.trim().is_empty() { + eprint!("{}", result.raw_stderr); + } + timer.track(&cmd_label, &format!("rtk {}", cmd_label), raw, raw); + return Ok(exit_code); + } + + let text_to_filter = if opts.filter_stdout_only { + raw_stdout + } else { + raw + }; + let filtered = filter_fn(text_to_filter); + + if let Some(label) = opts.tee_label { + print_with_hint(&filtered, raw, label, exit_code); + } else if opts.no_trailing_newline { + print!("{}", filtered); + } else { + println!("{}", filtered); + } + + let raw_for_tracking = if opts.filter_stdout_only { + raw_stdout + } else { + raw + }; + timer.track( + &cmd_label, + &format!("rtk {}", cmd_label), + raw_for_tracking, + &filtered, + ); + Ok(exit_code) + } + RunMode::Streamed(filter) => { + let result = + stream::run_streaming(&mut cmd, StdinMode::Null, FilterMode::Streaming(filter)) + .with_context(|| format!("Failed to run {}", tool_name))?; + + if let Some(label) = opts.tee_label { + if let Some(hint) = + crate::core::tee::tee_and_hint(&result.raw, label, result.exit_code) + { + println!("{}", hint); + } + } + + timer.track( + &cmd_label, + &format!("rtk {}", cmd_label), + &result.raw, + &result.filtered, + ); + Ok(result.exit_code) + } + RunMode::Passthrough => { + let result = + stream::run_streaming(&mut cmd, StdinMode::Inherit, FilterMode::Passthrough) + .with_context(|| format!("Failed to run {}", tool_name))?; + + timer.track_passthrough(&cmd_label, &format!("rtk {} (passthrough)", cmd_label)); + Ok(result.exit_code) + } + } +} + +pub fn run_filtered( + cmd: Command, + tool_name: &str, + args_display: &str, + filter_fn: F, + opts: RunOptions<'_>, +) -> Result +where + F: Fn(&str) -> String, +{ + run( + cmd, + tool_name, + args_display, + RunMode::Filtered(Box::new(filter_fn)), + opts, + ) +} + +pub fn run_passthrough(tool: &str, args: &[std::ffi::OsString], verbose: u8) -> Result { + if verbose > 0 { + eprintln!("{} passthrough: {:?}", tool, args); + } + let mut cmd = crate::core::utils::resolved_command(tool); + cmd.args(args); + let args_str = tracking::args_display(args); + run( + cmd, + tool, + &args_str, + RunMode::Passthrough, + RunOptions::default(), + ) +} + +pub fn run_streamed( + cmd: Command, + tool_name: &str, + args_display: &str, + filter: Box, + opts: RunOptions<'_>, +) -> Result { + run( + cmd, + tool_name, + args_display, + RunMode::Streamed(filter), + opts, + ) +} diff --git a/src/core/stream.rs b/src/core/stream.rs new file mode 100644 index 000000000..0ef451468 --- /dev/null +++ b/src/core/stream.rs @@ -0,0 +1,983 @@ +use anyhow::{Context, Result}; +use regex::Regex; +use std::io::{self, BufRead, BufReader, BufWriter, Write}; +use std::process::{Command, Stdio}; +use std::sync::mpsc; + +pub trait StreamFilter { + fn feed_line(&mut self, line: &str) -> Option; + fn flush(&mut self) -> String; + fn on_exit(&mut self, _exit_code: i32, _raw: &str) -> Option { + None + } +} + +pub trait BlockHandler { + fn should_skip(&mut self, line: &str) -> bool; + fn is_block_start(&mut self, line: &str) -> bool; + fn is_block_continuation(&mut self, line: &str, block: &[String]) -> bool; + fn format_summary(&self, exit_code: i32, raw: &str) -> Option; +} + +pub struct BlockStreamFilter { + handler: H, + in_block: bool, + current_block: Vec, + blocks_emitted: usize, +} + +impl BlockStreamFilter { + pub fn new(handler: H) -> Self { + Self { + handler, + in_block: false, + current_block: Vec::new(), + blocks_emitted: 0, + } + } + + fn emit_block(&mut self) -> Option { + if self.current_block.is_empty() { + return None; + } + let block = self.current_block.join("\n"); + self.current_block.clear(); + self.blocks_emitted += 1; + Some(format!("{}\n", block)) + } +} + +impl StreamFilter for BlockStreamFilter { + fn feed_line(&mut self, line: &str) -> Option { + if self.handler.should_skip(line) { + return None; + } + + if self.handler.is_block_start(line) { + let prev = self.emit_block(); + self.current_block.push(line.to_string()); + self.in_block = true; + prev + } else if self.in_block { + if self + .handler + .is_block_continuation(line, &self.current_block) + { + self.current_block.push(line.to_string()); + None + } else { + self.in_block = false; + self.emit_block() + } + } else { + None + } + } + + fn flush(&mut self) -> String { + self.emit_block().unwrap_or_default() + } + + fn on_exit(&mut self, exit_code: i32, raw: &str) -> Option { + self.handler.format_summary(exit_code, raw) + } +} + +#[allow(dead_code)] // available for command modules; currently used in tests only +pub struct RegexBlockFilter { + start_re: Regex, + skip_prefixes: Vec, + tool_name: String, + block_count: usize, +} + +impl RegexBlockFilter { + pub fn new(tool_name: &str, start_pattern: &str) -> Self { + Self { + start_re: Regex::new(start_pattern).unwrap_or_else(|e| { + panic!("RegexBlockFilter: bad pattern '{}': {}", start_pattern, e) + }), + skip_prefixes: Vec::new(), + tool_name: tool_name.to_string(), + block_count: 0, + } + } + + #[allow(dead_code)] + pub fn skip_prefix(mut self, prefix: &str) -> Self { + self.skip_prefixes.push(prefix.to_string()); + self + } + + #[allow(dead_code)] + pub fn skip_prefixes(mut self, prefixes: &[&str]) -> Self { + self.skip_prefixes + .extend(prefixes.iter().map(|s| s.to_string())); + self + } +} + +impl BlockHandler for RegexBlockFilter { + fn should_skip(&mut self, line: &str) -> bool { + self.skip_prefixes.iter().any(|p| line.starts_with(p)) + } + + fn is_block_start(&mut self, line: &str) -> bool { + if self.start_re.is_match(line) { + self.block_count += 1; + true + } else { + false + } + } + + fn is_block_continuation(&mut self, line: &str, _block: &[String]) -> bool { + line.starts_with(' ') || line.starts_with('\t') + } + + fn format_summary(&self, _exit_code: i32, _raw: &str) -> Option { + if self.block_count == 0 { + Some(format!("{}: no errors found\n", self.tool_name)) + } else { + Some(format!( + "{}: {} blocks in output\n", + self.tool_name, self.block_count + )) + } + } +} + +pub trait StdinFilter: Send { + fn feed_line(&mut self, line: &str) -> Option; + fn flush(&mut self) -> String; +} + +#[allow(dead_code)] // test utility: wraps closures as StreamFilter +pub struct LineFilter Option> { + f: F, +} + +#[allow(dead_code)] +impl Option> LineFilter { + pub fn new(f: F) -> Self { + Self { f } + } +} + +impl Option> StreamFilter for LineFilter { + fn feed_line(&mut self, line: &str) -> Option { + (self.f)(line) + } + + fn flush(&mut self) -> String { + String::new() + } +} + +pub enum FilterMode<'a> { + Streaming(Box), + Buffered(Box String + 'a>), + CaptureOnly, + Passthrough, +} + +pub enum StdinMode { + Inherit, + #[allow(dead_code)] // future API: stdin filtering for interactive commands + Filter(Box), + Null, +} + +pub struct StreamResult { + pub exit_code: i32, + pub raw: String, + pub raw_stdout: String, + pub raw_stderr: String, + pub filtered: String, +} + +impl StreamResult { + #[allow(dead_code)] + pub fn success(&self) -> bool { + self.exit_code == 0 + } +} + +pub fn status_to_exit_code(status: std::process::ExitStatus) -> i32 { + if let Some(code) = status.code() { + return code; + } + #[cfg(unix)] + { + use std::os::unix::process::ExitStatusExt; + if let Some(sig) = status.signal() { + return 128 + sig; + } + } + 1 +} + +// ISSUE #897: ChildGuard RAII prevents zombie processes that caused kernel panic +pub const RAW_CAP: usize = 10_485_760; // 10 MiB + +pub fn run_streaming( + cmd: &mut Command, + stdin_mode: StdinMode, + stdout_mode: FilterMode<'_>, +) -> Result { + if matches!(stdout_mode, FilterMode::Passthrough) { + match &stdin_mode { + StdinMode::Inherit => { + cmd.stdin(Stdio::inherit()); + } + _ => { + cmd.stdin(Stdio::null()); + } + }; + cmd.stdout(Stdio::inherit()); + cmd.stderr(Stdio::inherit()); + let status = cmd.status().context("Failed to spawn process")?; + return Ok(StreamResult { + exit_code: status_to_exit_code(status), + raw: String::new(), + raw_stdout: String::new(), + raw_stderr: String::new(), + filtered: String::new(), + }); + } + + match &stdin_mode { + StdinMode::Inherit => { + cmd.stdin(Stdio::inherit()); + } + StdinMode::Filter(_) | StdinMode::Null => { + cmd.stdin(Stdio::piped()); + } + } + cmd.stdout(Stdio::piped()); + cmd.stderr(Stdio::piped()); + + struct ChildGuard(std::process::Child); + impl Drop for ChildGuard { + fn drop(&mut self) { + self.0.wait().ok(); + } + } + + let is_streaming = matches!(stdout_mode, FilterMode::Streaming(_)); + + let mut child = ChildGuard(cmd.spawn().context("Failed to spawn process")?); + + let stdin_thread: Option> = match stdin_mode { + StdinMode::Filter(mut filter) => { + let child_stdin = child.0.stdin.take().context("No child stdin handle")?; + Some(std::thread::spawn(move || { + let mut writer = BufWriter::new(child_stdin); + let stdin_handle = io::stdin(); + for line in BufReader::new(stdin_handle.lock()) + .lines() + .map_while(Result::ok) + { + if let Some(out) = filter.feed_line(&line) { + if writeln!(writer, "{}", out).is_err() { + break; + } + } + } + let tail = filter.flush(); + if !tail.is_empty() { + write!(writer, "{}", tail).ok(); + } + })) + } + StdinMode::Null => { + child.0.stdin.take(); + None + } + StdinMode::Inherit => None, + }; + + let stdout = child.0.stdout.take().context("No child stdout handle")?; + let stderr = child.0.stderr.take().context("No child stderr handle")?; + let mut raw_stdout = String::new(); + let mut raw_stderr = String::new(); + let mut filtered = String::new(); + let mut capped_out = false; + let mut capped_err = false; + let mut saved_filter: Option> = None; + let mut filter_fd_is_stderr = false; + + if is_streaming { + enum StreamLine { + Stdout(String), + Stderr(String), + } + + let (tx, rx) = mpsc::channel(); + let tx_out = tx.clone(); + let stdout_thread = std::thread::spawn(move || { + for line in BufReader::new(stdout).lines().map_while(Result::ok) { + if tx_out.send(StreamLine::Stdout(line)).is_err() { + break; + } + } + }); + let tx_err = tx; + let stderr_thread = std::thread::spawn(move || { + for line in BufReader::new(stderr).lines().map_while(Result::ok) { + if tx_err.send(StreamLine::Stderr(line)).is_err() { + break; + } + } + }); + + if let FilterMode::Streaming(mut filter) = stdout_mode { + let stdout_handle = io::stdout(); + let mut out = stdout_handle.lock(); + let stderr_handle = io::stderr(); + let mut err_out = stderr_handle.lock(); + + for msg in rx { + let (line, is_stderr) = match msg { + StreamLine::Stderr(l) => (l, true), + StreamLine::Stdout(l) => (l, false), + }; + if is_stderr { + if !capped_err { + if raw_stderr.len() + line.len() + 1 <= RAW_CAP { + raw_stderr.push_str(&line); + raw_stderr.push('\n'); + } else { + capped_err = true; + eprintln!("[rtk] warning: stderr exceeds 10 MiB — capture truncated"); + } + } + } else if !capped_out { + if raw_stdout.len() + line.len() + 1 <= RAW_CAP { + raw_stdout.push_str(&line); + raw_stdout.push('\n'); + } else { + capped_out = true; + eprintln!("[rtk] warning: stdout exceeds 10 MiB — filter input truncated"); + } + } + filter_fd_is_stderr = is_stderr; + if let Some(output) = filter.feed_line(&line) { + filtered.push_str(&output); + let dest: &mut dyn Write = if is_stderr { &mut err_out } else { &mut out }; + match write!(dest, "{}", output) { + Err(e) if e.kind() == io::ErrorKind::BrokenPipe => break, + Err(e) => return Err(e.into()), + Ok(_) => {} + } + } + } + let tail = filter.flush(); + filtered.push_str(&tail); + let flush_dest: &mut dyn Write = if filter_fd_is_stderr { + &mut err_out + } else { + &mut out + }; + match write!(flush_dest, "{}", tail) { + Err(e) if e.kind() == io::ErrorKind::BrokenPipe => {} + Err(e) => return Err(e.into()), + Ok(_) => {} + } + saved_filter = Some(filter); + } + + stdout_thread.join().ok(); + stderr_thread.join().ok(); + } else { + let stderr_thread = std::thread::spawn(move || -> String { + let mut raw_err = String::new(); + let mut capped = false; + for line in BufReader::new(stderr).lines().map_while(Result::ok) { + if raw_err.len() + line.len() + 1 <= RAW_CAP { + raw_err.push_str(&line); + raw_err.push('\n'); + } else if !capped { + capped = true; + } + } + raw_err + }); + + { + let stdout_handle = io::stdout(); + let mut out = stdout_handle.lock(); + + match stdout_mode { + FilterMode::Passthrough => unreachable!("handled by early-return above"), + FilterMode::Streaming(_) => unreachable!("handled by is_streaming branch"), + FilterMode::Buffered(filter_fn) => { + for line in BufReader::new(stdout).lines().map_while(Result::ok) { + if raw_stdout.len() + line.len() + 1 <= RAW_CAP { + raw_stdout.push_str(&line); + raw_stdout.push('\n'); + } else if !capped_out { + capped_out = true; + eprintln!( + "[rtk] warning: output exceeds 10 MiB — filter input truncated" + ); + } + } + filtered = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { + filter_fn(&raw_stdout) + })) + .unwrap_or_else(|_| { + eprintln!("[rtk] warning: filter panicked — passing through raw output"); + raw_stdout.clone() + }); + match write!(out, "{}", filtered) { + Err(e) if e.kind() == io::ErrorKind::BrokenPipe => {} + Err(e) => return Err(e.into()), + Ok(_) => {} + } + } + FilterMode::CaptureOnly => { + for line in BufReader::new(stdout).lines().map_while(Result::ok) { + if raw_stdout.len() + line.len() + 1 <= RAW_CAP { + raw_stdout.push_str(&line); + raw_stdout.push('\n'); + } else if !capped_out { + capped_out = true; + eprintln!( + "[rtk] warning: output exceeds 10 MiB — filter input truncated" + ); + } + } + filtered = raw_stdout.clone(); + } + } + } + + raw_stderr = stderr_thread.join().unwrap_or_else(|e| { + eprintln!("[rtk] warning: stderr reader thread panicked: {:?}", e); + String::new() + }); + } + if let Some(t) = stdin_thread { + t.join().ok(); + } + + let status = child.0.wait().context("Failed to wait for child")?; + let exit_code = status_to_exit_code(status); + let raw = format!("{}{}", raw_stdout, raw_stderr); + + if let Some(mut f) = saved_filter { + if let Some(post) = f.on_exit(exit_code, &raw) { + filtered.push_str(&post); + let mut dest: Box = if filter_fd_is_stderr { + Box::new(io::stderr().lock()) + } else { + Box::new(io::stdout().lock()) + }; + match write!(dest, "{}", post) { + Err(e) if e.kind() == io::ErrorKind::BrokenPipe => {} + Err(e) => return Err(e.into()), + Ok(_) => {} + } + } + } + + Ok(StreamResult { + exit_code, + raw, + raw_stdout, + raw_stderr, + filtered, + }) +} + +pub struct CaptureResult { + pub stdout: String, + pub stderr: String, + pub exit_code: i32, +} + +impl CaptureResult { + pub fn success(&self) -> bool { + self.exit_code == 0 + } + + pub fn combined(&self) -> String { + format!("{}{}", self.stdout, self.stderr) + } +} + +pub fn exec_capture(cmd: &mut Command) -> Result { + cmd.stdin(Stdio::null()); + let output = cmd.output().context("Failed to execute command")?; + Ok(CaptureResult { + stdout: String::from_utf8_lossy(&output.stdout).into_owned(), + stderr: String::from_utf8_lossy(&output.stderr).into_owned(), + exit_code: status_to_exit_code(output.status), + }) +} + +#[cfg(test)] +pub(crate) mod tests { + use super::*; + use std::process::Command; + + #[test] + fn test_exit_code_zero() { + let status = Command::new("true").status().unwrap(); + assert_eq!(status_to_exit_code(status), 0); + } + + #[test] + fn test_exit_code_nonzero() { + let status = Command::new("false").status().unwrap(); + assert_eq!(status_to_exit_code(status), 1); + } + + #[cfg(unix)] + #[test] + fn test_exit_code_signal_kill() { + let mut child = Command::new("sleep").arg("60").spawn().unwrap(); + child.kill().unwrap(); + let status = child.wait().unwrap(); + assert_eq!(status_to_exit_code(status), 137); + } + + #[test] + fn test_line_filter_passes_lines() { + let mut f = LineFilter::new(|l| Some(format!("{}\n", l.to_uppercase()))); + assert_eq!(f.feed_line("hello"), Some("HELLO\n".to_string())); + } + + #[test] + fn test_line_filter_drops_lines() { + let mut f = LineFilter::new(|l| { + if l.starts_with('#') { + None + } else { + Some(l.to_string()) + } + }); + assert_eq!(f.feed_line("# comment"), None); + assert_eq!(f.feed_line("code"), Some("code".to_string())); + } + + #[test] + fn test_line_filter_flush_empty() { + let mut f = LineFilter::new(|l| Some(l.to_string())); + assert_eq!(f.flush(), String::new()); + } + + #[test] + fn test_stream_result_success() { + let r = StreamResult { + exit_code: 0, + raw: String::new(), + raw_stdout: String::new(), + raw_stderr: String::new(), + filtered: String::new(), + }; + assert!(r.success()); + } + + #[test] + fn test_stream_result_failure() { + let r = StreamResult { + exit_code: 1, + raw: String::new(), + raw_stdout: String::new(), + raw_stderr: String::new(), + filtered: String::new(), + }; + assert!(!r.success()); + } + + #[test] + fn test_stream_result_signal_not_success() { + let r = StreamResult { + exit_code: 137, + raw: String::new(), + raw_stdout: String::new(), + raw_stderr: String::new(), + filtered: String::new(), + }; + assert!(!r.success()); + } + + #[test] + fn test_run_streaming_passthrough_echo() { + let mut cmd = Command::new("echo"); + cmd.arg("hello"); + let result = run_streaming(&mut cmd, StdinMode::Null, FilterMode::Passthrough).unwrap(); + assert_eq!(result.exit_code, 0); + // Passthrough inherits TTY — raw/filtered are empty + assert!(result.raw.is_empty()); + } + + #[test] + fn test_run_streaming_exit_code_preserved() { + // nosemgrep: interpreter-execution + let mut cmd = Command::new("sh"); + cmd.args(["-c", "exit 42"]); + let result = run_streaming(&mut cmd, StdinMode::Null, FilterMode::Passthrough).unwrap(); + assert_eq!(result.exit_code, 42); + } + + #[test] + fn test_run_streaming_exit_code_zero() { + let mut cmd = Command::new("true"); + let result = run_streaming(&mut cmd, StdinMode::Null, FilterMode::Passthrough).unwrap(); + assert_eq!(result.exit_code, 0); + assert!(result.success()); + } + + #[test] + fn test_run_streaming_exit_code_one() { + let mut cmd = Command::new("false"); + let result = run_streaming(&mut cmd, StdinMode::Null, FilterMode::Passthrough).unwrap(); + assert_eq!(result.exit_code, 1); + assert!(!result.success()); + } + + #[cfg(not(windows))] + #[test] + fn test_run_streaming_streaming_filter_drops_lines() { + let mut cmd = Command::new("printf"); + cmd.arg("a\nb\nc\n"); + let filter = LineFilter::new(|l| { + if l == "b" { + None + } else { + Some(format!("{}\n", l)) + } + }); + let result = run_streaming( + &mut cmd, + StdinMode::Null, + FilterMode::Streaming(Box::new(filter)), + ) + .unwrap(); + assert!(result.filtered.contains('a')); + assert!(!result.filtered.contains('b')); + assert!(result.filtered.contains('c')); + assert_eq!(result.exit_code, 0); + } + + #[cfg(not(windows))] + #[test] + fn test_run_streaming_buffered_filter() { + let mut cmd = Command::new("printf"); + cmd.arg("line1\nline2\nline3\n"); + let result = run_streaming( + &mut cmd, + StdinMode::Null, + FilterMode::Buffered(Box::new(|s: &str| s.to_uppercase())), + ) + .unwrap(); + assert!(result.filtered.contains("LINE1")); + assert!(result.filtered.contains("LINE2")); + assert_eq!(result.exit_code, 0); + } + + #[test] + fn test_run_streaming_raw_cap_at_10mb() { + // nosemgrep: interpreter-execution + let mut cmd = Command::new("sh"); + // ~11 MiB of 80-char lines (fast: fewer lines than `yes | head -6M`) + cmd.args([ + "-c", + "dd if=/dev/zero bs=1024 count=11264 2>/dev/null | tr '\\0' 'a' | fold -w 80", + ]); + let result = run_streaming(&mut cmd, StdinMode::Null, FilterMode::CaptureOnly).unwrap(); + assert!( + result.raw.len() <= 10_485_760 + 100, + "raw should be capped at ~10 MiB, got {} bytes", + result.raw.len() + ); + assert!( + result.raw.len() > 1_000_000, + "Should have captured significant data" + ); + } + + #[test] + fn test_run_streaming_stderr_cap_at_10mb() { + // nosemgrep: interpreter-execution + let mut cmd = Command::new("sh"); + // ~11 MiB on stderr, nothing on stdout + cmd.args([ + "-c", + "dd if=/dev/zero bs=1024 count=11264 2>/dev/null | tr '\\0' 'a' | fold -w 80 1>&2", + ]); + let result = run_streaming(&mut cmd, StdinMode::Null, FilterMode::CaptureOnly).unwrap(); + // raw = raw_stdout + raw_stderr; stdout is empty so raw ≈ stderr size + assert!( + result.raw.len() <= RAW_CAP + 200, + "stderr in raw should be capped at ~10 MiB, got {} bytes", + result.raw.len() + ); + } + + #[test] + fn test_child_guard_prevents_zombie() { + let mut cmd = Command::new("true"); + let result = run_streaming(&mut cmd, StdinMode::Null, FilterMode::CaptureOnly); + assert!(result.is_ok()); + assert_eq!(result.unwrap().exit_code, 0); + } + + #[test] + fn test_run_streaming_null_stdin_cat() { + let mut cmd = Command::new("cat"); + let result = run_streaming(&mut cmd, StdinMode::Null, FilterMode::Passthrough).unwrap(); + assert_eq!(result.exit_code, 0); + } + + #[test] + fn test_run_streaming_raw_contains_stdout() { + let mut cmd = Command::new("echo"); + cmd.arg("test_output_xyz"); + let result = run_streaming(&mut cmd, StdinMode::Null, FilterMode::CaptureOnly).unwrap(); + assert!(result.raw.contains("test_output_xyz")); + } + + #[test] + fn test_run_streaming_capture_only_filtered_equals_raw() { + let mut cmd = Command::new("echo"); + cmd.arg("check_equality"); + let result = run_streaming(&mut cmd, StdinMode::Null, FilterMode::CaptureOnly).unwrap(); + assert_eq!(result.filtered.trim(), result.raw_stdout.trim()); + } + + #[test] + fn test_exec_capture_success() { + let mut cmd = Command::new("echo"); + cmd.arg("hello_capture"); + let result = exec_capture(&mut cmd).unwrap(); + assert!(result.success()); + assert_eq!(result.exit_code, 0); + assert!(result.stdout.contains("hello_capture")); + } + + #[test] + fn test_exec_capture_failure() { + let mut cmd = Command::new("false"); + let result = exec_capture(&mut cmd).unwrap(); + assert!(!result.success()); + assert_eq!(result.exit_code, 1); + } + + #[test] + fn test_exec_capture_stderr() { + // nosemgrep: interpreter-execution + let mut cmd = Command::new("sh"); + cmd.args(["-c", "echo err_msg >&2"]); + let result = exec_capture(&mut cmd).unwrap(); + assert!(result.stderr.contains("err_msg")); + } + + #[test] + fn test_exec_capture_combined() { + // nosemgrep: interpreter-execution + let mut cmd = Command::new("sh"); + cmd.args(["-c", "echo out_msg; echo err_msg >&2"]); + let result = exec_capture(&mut cmd).unwrap(); + let combined = result.combined(); + assert!(combined.contains("out_msg")); + assert!(combined.contains("err_msg")); + } + + #[test] + fn test_capture_result_combined_empty() { + let r = CaptureResult { + stdout: String::new(), + stderr: String::new(), + exit_code: 0, + }; + assert_eq!(r.combined(), ""); + } + + pub fn run_block_filter(filter: &mut dyn StreamFilter, input: &str, exit_code: i32) -> String { + let mut output = String::new(); + for line in input.lines() { + if let Some(s) = filter.feed_line(line) { + output.push_str(&s); + } + } + output.push_str(&filter.flush()); + if let Some(post) = filter.on_exit(exit_code, input) { + output.push_str(&post); + } + output + } + + struct TestHandler; + + impl BlockHandler for TestHandler { + fn should_skip(&mut self, line: &str) -> bool { + line.starts_with("SKIP") + } + fn is_block_start(&mut self, line: &str) -> bool { + line.starts_with("ERROR") + } + fn is_block_continuation(&mut self, line: &str, _block: &[String]) -> bool { + line.starts_with(" ") + } + fn format_summary(&self, _exit_code: i32, _raw: &str) -> Option { + Some("DONE\n".to_string()) + } + } + + #[test] + fn test_block_filter_emits_blocks() { + let mut f = BlockStreamFilter::new(TestHandler); + let input = "SKIP noise\nERROR first\n detail1\nnon-block\nERROR second\n detail2\n"; + let result = run_block_filter(&mut f, input, 0); + assert!(result.contains("ERROR first\n detail1"), "got: {}", result); + assert!( + result.contains("ERROR second\n detail2"), + "got: {}", + result + ); + assert!(!result.contains("SKIP"), "got: {}", result); + assert!(result.ends_with("DONE\n"), "got: {}", result); + } + + #[test] + fn test_block_filter_no_blocks() { + let mut f = BlockStreamFilter::new(TestHandler); + let result = run_block_filter(&mut f, "nothing here\njust text\n", 0); + assert_eq!(result, "DONE\n"); + } + + #[test] + fn test_regex_block_filter_emits_blocks() { + let handler = RegexBlockFilter::new("test", r"^error\["); + let mut f = BlockStreamFilter::new(handler); + let input = "ok line\nerror[E0308]: mismatched types\n expected `u32`\nok again\nerror[E0599]: no method\n help: try\n"; + let result = run_block_filter(&mut f, input, 1); + assert!( + result.contains("error[E0308]: mismatched types\n expected `u32`"), + "got: {}", + result + ); + assert!( + result.contains("error[E0599]: no method\n help: try"), + "got: {}", + result + ); + assert!( + result.contains("test: 2 blocks in output"), + "got: {}", + result + ); + } + + #[test] + fn test_regex_block_filter_skip_prefix() { + let handler = RegexBlockFilter::new("test", r"^error").skip_prefix("warning:"); + let mut f = BlockStreamFilter::new(handler); + let input = "warning: unused var\nerror: bad type\n detail\nwarning: dead code\n"; + let result = run_block_filter(&mut f, input, 1); + assert!(result.contains("error: bad type"), "got: {}", result); + assert!(!result.contains("warning:"), "got: {}", result); + } + + #[test] + fn test_regex_block_filter_no_blocks() { + let handler = RegexBlockFilter::new("mytest", r"^FAIL"); + let mut f = BlockStreamFilter::new(handler); + let result = run_block_filter(&mut f, "all passed\nok\n", 0); + assert_eq!(result, "mytest: no errors found\n"); + } + + #[test] + fn test_regex_block_filter_indent_continuation() { + let handler = RegexBlockFilter::new("test", r"^ERR"); + let mut f = BlockStreamFilter::new(handler); + let input = "ERR space indent\n two spaces\n\ttab indent\nnon-indent\n"; + let result = run_block_filter(&mut f, input, 1); + assert!( + result.contains("ERR space indent\n two spaces\n\ttab indent"), + "got: {}", + result + ); + assert!(!result.contains("non-indent"), "got: {}", result); + } + + #[test] + fn test_regex_block_filter_multiple_skip_prefixes() { + let handler = + RegexBlockFilter::new("test", r"^error").skip_prefixes(&["note:", "warning:", "help:"]); + let mut f = BlockStreamFilter::new(handler); + let input = "note: see docs\nwarning: unused\nhelp: try this\nerror: fatal\n details\n"; + let result = run_block_filter(&mut f, input, 1); + assert!(!result.contains("note:"), "got: {}", result); + assert!(!result.contains("warning:"), "got: {}", result); + assert!(!result.contains("help:"), "got: {}", result); + assert!( + result.contains("error: fatal\n details"), + "got: {}", + result + ); + } + + #[cfg(not(windows))] + #[test] + fn test_streaming_filters_both_fds_and_routes_to_correct_fd() { + // nosemgrep: interpreter-execution + let mut cmd = Command::new("sh"); + cmd.args(["-c", "echo 'error[E0308]: type mismatch'; echo ' Compiling foo v1.0' >&2; echo ' Downloading bar v2.0' >&2; echo ' Finished dev' >&2; echo 'real error on stderr' >&2"]); + + struct CargoLikeHandler; + impl BlockHandler for CargoLikeHandler { + fn should_skip(&mut self, line: &str) -> bool { + let trimmed = line.trim_start(); + trimmed.starts_with("Compiling") + || trimmed.starts_with("Downloading") + || trimmed.starts_with("Finished") + } + fn is_block_start(&mut self, line: &str) -> bool { + line.starts_with("error") + } + fn is_block_continuation(&mut self, line: &str, _block: &[String]) -> bool { + line.starts_with(' ') + } + fn format_summary(&self, _: i32, _: &str) -> Option { + None + } + } + + let filter = BlockStreamFilter::new(CargoLikeHandler); + let result = run_streaming( + &mut cmd, + StdinMode::Null, + FilterMode::Streaming(Box::new(filter)), + ) + .unwrap(); + + assert!( + result.filtered.contains("error[E0308]"), + "filtered should contain stdout errors, got: {}", + result.filtered + ); + assert!( + !result.filtered.contains("Compiling"), + "cargo noise should be filtered out, got: {}", + result.filtered + ); + assert!( + !result.filtered.contains("Downloading"), + "cargo noise should be filtered out, got: {}", + result.filtered + ); + assert!( + result.raw_stderr.contains("Compiling"), + "raw_stderr should capture all stderr lines" + ); + assert!( + result.raw_stderr.contains("real error on stderr"), + "raw_stderr should capture all stderr lines" + ); + } +} diff --git a/src/core/tee.rs b/src/core/tee.rs index a4cf3cccb..c67ea8349 100644 --- a/src/core/tee.rs +++ b/src/core/tee.rs @@ -1,5 +1,6 @@ //! Raw output recovery -- saves unfiltered output to disk on command failure. +use super::constants::RTK_DATA_DIR; use crate::core::config::Config; use std::path::PathBuf; @@ -46,7 +47,7 @@ fn get_tee_dir(config: &Config) -> Option { } // Default: ~/.local/share/rtk/tee/ - dirs::data_local_dir().map(|d| d.join("rtk").join("tee")) + dirs::data_local_dir().map(|d| d.join(RTK_DATA_DIR).join("tee")) } /// Rotate old tee files: keep only the last `max_files`, delete oldest. @@ -120,11 +121,17 @@ fn write_tee_file( let filename = format!("{}_{}.log", epoch, slug); let filepath = tee_dir.join(filename); - // Truncate at max_file_size + // Truncate at max_file_size (find a safe UTF-8 char boundary) let content = if raw.len() > max_file_size { + let boundary = raw + .char_indices() + .take_while(|(i, _)| *i < max_file_size) + .last() + .map(|(i, c)| i + c.len_utf8()) + .unwrap_or(0); format!( "{}\n\n--- truncated at {} bytes ---", - &raw[..max_file_size], + &raw[..boundary], max_file_size ) } else { @@ -183,6 +190,44 @@ pub fn tee_and_hint(raw: &str, command_slug: &str, exit_code: i32) -> Option= MIN_TEE_SIZE and tee is enabled. +/// Returns hint string if file was written, None if skipped/disabled. +/// +/// Used by AWS filters when FilterResult.truncated = true, ensuring +/// the LLM has access to full untruncated output via the hint path. +pub fn force_tee_hint(raw: &str, command_slug: &str) -> Option { + // Check RTK_TEE=0 env override (disable) + if std::env::var("RTK_TEE").ok().as_deref() == Some("0") { + return None; + } + + // Skip if output too small + if raw.len() < MIN_TEE_SIZE { + return None; + } + + let config = Config::load().ok()?; + + // Respect enabled flag but ignore mode (force tee) + if !config.tee.enabled { + return None; + } + + let tee_dir = get_tee_dir(&config)?; + let tee_dir = std::fs::create_dir_all(&tee_dir).ok().and(Some(tee_dir))?; + + let path = write_tee_file( + raw, + command_slug, + &tee_dir, + config.tee.max_file_size, + config.tee.max_files, + )?; + + Some(format_hint(&path)) +} + /// TeeMode controls when tee writes files. #[derive(Debug, Clone, serde::Serialize, serde::Deserialize, PartialEq, Default)] #[serde(rename_all = "lowercase")] @@ -317,6 +362,47 @@ mod tests { assert!(content.len() < 2000); } + #[test] + fn test_write_tee_file_truncation_utf8_boundary() { + let tmpdir = tempfile::tempdir().unwrap(); + // Create a string where the truncation point falls inside a multi-byte char. + // Japanese chars are 3 bytes each in UTF-8. + // 332 chars * 3 bytes = 996 bytes, then one more = 999 bytes. + // With max_file_size=998, the cut falls mid-character. + let japanese = "\u{6F22}".repeat(333); // 999 bytes of 3-byte chars + assert_eq!(japanese.len(), 999); + + // Truncate at 998 — falls in the middle of the 333rd character + let result = write_tee_file(&japanese, "test_utf8", tmpdir.path(), 998, 20); + assert!(result.is_some()); + + let path = result.unwrap(); + let content = fs::read_to_string(&path).unwrap(); + assert!(content.contains("--- truncated at 998 bytes ---")); + // Should contain 332 full characters (996 bytes), not panic + assert!(content.starts_with(&"\u{6F22}".repeat(332))); + } + + #[test] + fn test_write_tee_file_truncation_emoji() { + let tmpdir = tempfile::tempdir().unwrap(); + // Emoji are 4 bytes each in UTF-8 + let emojis = "\u{1F600}".repeat(100); // 400 bytes + assert_eq!(emojis.len(), 400); + + // Truncate at 201 — falls mid-emoji (4-byte boundary is at 200, 204) + let result = write_tee_file(&emojis, "test_emoji", tmpdir.path(), 201, 20); + assert!(result.is_some()); + + let path = result.unwrap(); + let content = fs::read_to_string(&path).unwrap(); + assert!(content.contains("--- truncated at 201 bytes ---")); + // The emoji portion should be exactly 200 bytes (50 emojis), + // rounded down from 201 to the nearest char boundary + let target = "\u{1F600}".repeat(50); + assert!(content.starts_with(&target)); + } + #[test] fn test_cleanup_old_files() { let tmpdir = tempfile::tempdir().unwrap(); @@ -399,4 +485,22 @@ directory = "/tmp/rtk-tee" let mode: TeeMode = serde_json::from_str(r#""never""#).unwrap(); assert_eq!(mode, TeeMode::Never); } + + #[test] + fn test_force_tee_hint_skip_small_output() { + // force_tee_hint should respect MIN_TEE_SIZE + let small_output = "short error"; + let hint = force_tee_hint(small_output, "test_cmd"); + assert!(hint.is_none(), "Should skip output < MIN_TEE_SIZE"); + } + + #[test] + fn test_force_tee_hint_respects_env_disable() { + // When RTK_TEE=0, force_tee_hint should return None + std::env::set_var("RTK_TEE", "0"); + let large_output = "x".repeat(1000); + let hint = force_tee_hint(&large_output, "test_cmd"); + std::env::remove_var("RTK_TEE"); + assert!(hint.is_none(), "Should respect RTK_TEE=0"); + } } diff --git a/src/core/telemetry.rs b/src/core/telemetry.rs deleted file mode 100644 index 3b9729555..000000000 --- a/src/core/telemetry.rs +++ /dev/null @@ -1,338 +0,0 @@ -//! Optional usage ping so we know which commands people run most. - -use crate::core::config; -use crate::core::tracking; -use sha2::{Digest, Sha256}; -use std::io::Write; -use std::path::PathBuf; -use std::sync::OnceLock; - -static CACHED_SALT: OnceLock = OnceLock::new(); - -const TELEMETRY_URL: Option<&str> = option_env!("RTK_TELEMETRY_URL"); -const TELEMETRY_TOKEN: Option<&str> = option_env!("RTK_TELEMETRY_TOKEN"); -const PING_INTERVAL_SECS: u64 = 23 * 3600; // 23 hours - -/// Send a telemetry ping if enabled and not already sent today. -/// Fire-and-forget: errors are silently ignored. -pub fn maybe_ping() { - // No URL compiled in → telemetry disabled - if TELEMETRY_URL.is_none() { - return; - } - - // Check opt-out: env var - if std::env::var("RTK_TELEMETRY_DISABLED").unwrap_or_default() == "1" { - return; - } - - // Check opt-out: config.toml - if let Some(false) = config::telemetry_enabled() { - return; - } - - // Check last ping time - let marker = telemetry_marker_path(); - if let Ok(metadata) = std::fs::metadata(&marker) { - if let Ok(modified) = metadata.modified() { - if let Ok(elapsed) = modified.elapsed() { - if elapsed.as_secs() < PING_INTERVAL_SECS { - return; - } - } - } - } - - // Touch marker file immediately (before sending) to avoid double-ping - touch_marker(&marker); - - // Spawn thread so we never block the CLI - std::thread::spawn(|| { - let _ = send_ping(); - }); -} - -fn send_ping() -> Result<(), Box> { - let url = TELEMETRY_URL.ok_or("no telemetry URL")?; - let device_hash = generate_device_hash(); - let version = env!("CARGO_PKG_VERSION").to_string(); - let os = std::env::consts::OS.to_string(); - let arch = std::env::consts::ARCH.to_string(); - let install_method = detect_install_method(); - - // Get stats from tracking DB - let (commands_24h, top_commands, savings_pct, tokens_saved_24h, tokens_saved_total) = - get_stats(); - - let payload = serde_json::json!({ - "device_hash": device_hash, - "version": version, - "os": os, - "arch": arch, - "install_method": install_method, - "commands_24h": commands_24h, - "top_commands": top_commands, - "savings_pct": savings_pct, - "tokens_saved_24h": tokens_saved_24h, - "tokens_saved_total": tokens_saved_total, - }); - - let mut req = ureq::post(url).set("Content-Type", "application/json"); - - if let Some(token) = TELEMETRY_TOKEN { - req = req.set("X-RTK-Token", token); - } - - // 2 second timeout — if server is down, we move on - req.timeout(std::time::Duration::from_secs(2)) - .send_string(&payload.to_string())?; - - Ok(()) -} - -fn generate_device_hash() -> String { - let salt = get_or_create_salt(); - let hostname = hostname::get() - .map(|h| h.to_string_lossy().to_string()) - .unwrap_or_default(); - let username = std::env::var("USER") - .or_else(|_| std::env::var("USERNAME")) - .unwrap_or_default(); - - let mut hasher = Sha256::new(); - hasher.update(salt.as_bytes()); - hasher.update(b":"); - hasher.update(hostname.as_bytes()); - hasher.update(b":"); - hasher.update(username.as_bytes()); - format!("{:x}", hasher.finalize()) -} - -fn get_or_create_salt() -> String { - CACHED_SALT - .get_or_init(|| { - let salt_path = salt_file_path(); - - if let Ok(contents) = std::fs::read_to_string(&salt_path) { - let trimmed = contents.trim().to_string(); - if trimmed.len() == 64 && trimmed.chars().all(|c| c.is_ascii_hexdigit()) { - return trimmed; - } - } - - let salt = random_salt(); - if let Some(parent) = salt_path.parent() { - let _ = std::fs::create_dir_all(parent); - } - if let Ok(mut f) = std::fs::File::create(&salt_path) { - let _ = f.write_all(salt.as_bytes()); - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - let _ = std::fs::set_permissions( - &salt_path, - std::fs::Permissions::from_mode(0o600), - ); - } - } - salt - }) - .clone() -} - -fn random_salt() -> String { - let mut buf = [0u8; 32]; - if getrandom::fill(&mut buf).is_err() { - let fallback = format!("{:?}:{}", std::time::SystemTime::now(), std::process::id()); - let mut hasher = Sha256::new(); - hasher.update(fallback.as_bytes()); - return format!("{:x}", hasher.finalize()); - } - buf.iter().map(|b| format!("{:02x}", b)).collect() -} - -fn salt_file_path() -> PathBuf { - dirs::data_local_dir() - .unwrap_or_else(|| PathBuf::from("/tmp")) - .join("rtk") - .join(".device_salt") -} - -fn get_stats() -> (i64, Vec, Option, i64, i64) { - let tracker = match tracking::Tracker::new() { - Ok(t) => t, - Err(_) => return (0, vec![], None, 0, 0), - }; - - let since_24h = chrono::Utc::now() - chrono::Duration::hours(24); - - // Get 24h command count and top commands from tracking DB - let commands_24h = tracker.count_commands_since(since_24h).unwrap_or(0); - - let top_commands = tracker.top_commands(5).unwrap_or_default(); - - let savings_pct = tracker.overall_savings_pct().ok(); - - let tokens_saved_24h = tracker.tokens_saved_24h(since_24h).unwrap_or(0); - - let tokens_saved_total = tracker.total_tokens_saved().unwrap_or(0); - - ( - commands_24h, - top_commands, - savings_pct, - tokens_saved_24h, - tokens_saved_total, - ) -} - -fn detect_install_method() -> &'static str { - let exe = match std::env::current_exe() { - Ok(p) => p, - Err(_) => return "unknown", - }; - let real_path = std::fs::canonicalize(&exe) - .unwrap_or(exe) - .to_string_lossy() - .to_string(); - install_method_from_path(&real_path) -} - -fn install_method_from_path(path: &str) -> &'static str { - if path.contains("/Cellar/rtk/") || path.contains("/homebrew/") { - "homebrew" - } else if path.contains("/.cargo/bin/") || path.contains("\\.cargo\\bin\\") { - "cargo" - } else if path.contains("/.local/bin/") || path.contains("\\.local\\bin\\") { - "script" - } else if path.contains("/nix/store/") { - "nix" - } else { - "other" - } -} - -fn telemetry_marker_path() -> PathBuf { - let data_dir = dirs::data_local_dir() - .unwrap_or_else(|| PathBuf::from("/tmp")) - .join("rtk"); - let _ = std::fs::create_dir_all(&data_dir); - data_dir.join(".telemetry_last_ping") -} - -fn touch_marker(path: &PathBuf) { - let _ = std::fs::write(path, b""); -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn test_device_hash_is_stable() { - let h1 = generate_device_hash(); - let h2 = generate_device_hash(); - assert_eq!(h1, h2); - assert_eq!(h1.len(), 64); - } - - #[test] - fn test_device_hash_is_valid_hex() { - let hash = generate_device_hash(); - assert!(hash.chars().all(|c| c.is_ascii_hexdigit())); - } - - #[test] - fn test_salt_is_persisted() { - let s1 = get_or_create_salt(); - let s2 = get_or_create_salt(); - assert_eq!(s1, s2); - assert_eq!(s1.len(), 64); - assert!(s1.chars().all(|c| c.is_ascii_hexdigit())); - } - - #[test] - fn test_random_salt_uniqueness() { - let s1 = random_salt(); - let s2 = random_salt(); - assert_ne!(s1, s2); - assert_eq!(s1.len(), 64); - assert_eq!(s2.len(), 64); - } - - #[test] - fn test_salt_file_path_is_in_rtk_dir() { - let path = salt_file_path(); - assert!(path.to_string_lossy().contains("rtk")); - assert!(path.to_string_lossy().contains(".device_salt")); - } - - #[test] - fn test_marker_path_exists() { - let path = telemetry_marker_path(); - assert!(path.to_string_lossy().contains("rtk")); - } - - #[test] - fn test_install_method_unix_paths() { - assert_eq!( - install_method_from_path("/opt/homebrew/Cellar/rtk/0.28.0/bin/rtk"), - "homebrew" - ); - assert_eq!( - install_method_from_path("/usr/local/homebrew/bin/rtk"), - "homebrew" - ); - assert_eq!( - install_method_from_path("/home/user/.cargo/bin/rtk"), - "cargo" - ); - assert_eq!( - install_method_from_path("/home/user/.local/bin/rtk"), - "script" - ); - assert_eq!( - install_method_from_path("/nix/store/abc123-rtk/bin/rtk"), - "nix" - ); - assert_eq!(install_method_from_path("/usr/bin/rtk"), "other"); - } - - #[test] - fn test_install_method_windows_paths() { - assert_eq!( - install_method_from_path("C:\\Users\\user\\.cargo\\bin\\rtk.exe"), - "cargo" - ); - assert_eq!( - install_method_from_path("C:\\Users\\user\\.local\\bin\\rtk.exe"), - "script" - ); - assert_eq!( - install_method_from_path("C:\\Program Files\\rtk\\rtk.exe"), - "other" - ); - } - - #[test] - fn test_detect_install_method_returns_known_value() { - let method = detect_install_method(); - assert!( - ["homebrew", "cargo", "script", "nix", "other", "unknown"].contains(&method), - "Unexpected install method: {}", - method - ); - } - - #[test] - fn test_get_stats_returns_tuple() { - let (cmds, top, pct, saved_24h, saved_total) = get_stats(); - assert!(cmds >= 0); - assert!(top.len() <= 5); - assert!(saved_24h >= 0); - assert!(saved_total >= 0); - if let Some(p) = pct { - assert!((0.0..=100.0).contains(&p)); - } - } -} diff --git a/src/core/toml_filter.rs b/src/core/toml_filter.rs index 625977179..06060d22d 100644 --- a/src/core/toml_filter.rs +++ b/src/core/toml_filter.rs @@ -22,6 +22,7 @@ /// 6. head/tail_lines — keep first/last N lines /// 7. max_lines — absolute line cap /// 8. on_empty — message if result is empty +use super::constants::{FILTERS_TOML, RTK_DATA_DIR}; use lazy_static::lazy_static; use regex::{Regex, RegexSet}; use serde::Deserialize; @@ -101,6 +102,10 @@ struct TomlFilterDef { tail_lines: Option, max_lines: Option, on_empty: Option, + /// When true, stderr is captured and merged with stdout before filtering. + /// Use for tools like liquibase that emit banners/logs to stderr. + #[serde(default)] + filter_stderr: bool, } // --------------------------------------------------------------------------- @@ -144,6 +149,8 @@ pub struct CompiledFilter { tail_lines: Option, pub max_lines: Option, on_empty: Option, + /// When true, the runner should capture stderr and merge it with stdout. + pub filter_stderr: bool, } // --------------------------------------------------------------------------- @@ -210,7 +217,7 @@ impl TomlFilterRegistry { // Priority 2: user-global ~/.config/rtk/filters.toml if let Some(config_dir) = dirs::config_dir() { - let global_path = config_dir.join("rtk").join("filters.toml"); + let global_path = config_dir.join(RTK_DATA_DIR).join(FILTERS_TOML); if let Ok(content) = std::fs::read_to_string(&global_path) { match Self::parse_and_compile(&content, "user-global") { Ok(f) => filters.extend(f), @@ -390,6 +397,7 @@ fn compile_filter(name: String, def: TomlFilterDef) -> Result) -> (Option, Option< } } -/// Number of days to retain tracking history before automatic cleanup. -const HISTORY_DAYS: i64 = 90; +use super::constants::{DEFAULT_HISTORY_DAYS, HISTORY_DB, RTK_DATA_DIR}; /// Main tracking interface for recording and querying command history. /// @@ -327,6 +326,56 @@ impl Tracker { Ok(Self { conn }) } + /// Create an isolated in-memory tracker for tests. + #[cfg(test)] + pub fn new_in_memory() -> Result { + let conn = Connection::open_in_memory().context("Failed to open in-memory DB")?; + let tracker = Self { conn }; + tracker.init_schema()?; + Ok(tracker) + } + + fn init_schema(&self) -> Result<()> { + self.conn.execute( + "CREATE TABLE IF NOT EXISTS commands ( + id INTEGER PRIMARY KEY, + timestamp TEXT NOT NULL, + original_cmd TEXT NOT NULL, + rtk_cmd TEXT NOT NULL, + input_tokens INTEGER NOT NULL, + output_tokens INTEGER NOT NULL, + saved_tokens INTEGER NOT NULL, + savings_pct REAL NOT NULL, + exec_time_ms INTEGER DEFAULT 0, + project_path TEXT DEFAULT '' + )", + [], + )?; + self.conn.execute( + "CREATE INDEX IF NOT EXISTS idx_timestamp ON commands(timestamp)", + [], + )?; + self.conn.execute( + "CREATE INDEX IF NOT EXISTS idx_project_path_timestamp ON commands(project_path, timestamp)", + [], + )?; + self.conn.execute( + "CREATE TABLE IF NOT EXISTS parse_failures ( + id INTEGER PRIMARY KEY, + timestamp TEXT NOT NULL, + raw_command TEXT NOT NULL, + error_message TEXT NOT NULL, + fallback_succeeded INTEGER NOT NULL DEFAULT 0 + )", + [], + )?; + self.conn.execute( + "CREATE INDEX IF NOT EXISTS idx_pf_timestamp ON parse_failures(timestamp)", + [], + )?; + Ok(()) + } + /// Record a command execution with token counts and timing. /// /// Calculates savings metrics and stores the record in the database. @@ -387,7 +436,7 @@ impl Tracker { } fn cleanup_old(&self) -> Result<()> { - let cutoff = Utc::now() - chrono::Duration::days(HISTORY_DAYS); + let cutoff = Utc::now() - chrono::Duration::days(DEFAULT_HISTORY_DAYS); self.conn.execute( "DELETE FROM commands WHERE timestamp < ?1", params![cutoff.to_rfc3339()], @@ -399,6 +448,19 @@ impl Tracker { Ok(()) } + /// Delete all tracked data (commands + parse_failures), resetting all stats to zero. + pub fn reset_all(&self) -> Result<()> { + self.conn + .execute_batch( + "BEGIN; + DELETE FROM commands; + DELETE FROM parse_failures; + COMMIT;", + ) + .context("Failed to reset tracking database")?; + Ok(()) + } + /// Record a parse failure for analytics. pub fn record_parse_failure( &self, @@ -899,6 +961,7 @@ impl Tracker { } /// Count commands since a given timestamp (for telemetry). + #[allow(dead_code)] pub fn count_commands_since(&self, since: chrono::DateTime) -> Result { let ts = since.format("%Y-%m-%dT%H:%M:%S").to_string(); let count: i64 = self.conn.query_row( @@ -910,6 +973,7 @@ impl Tracker { } /// Get top N commands by frequency (for telemetry). + #[allow(dead_code)] pub fn top_commands(&self, limit: usize) -> Result> { let mut stmt = self.conn.prepare( "SELECT rtk_cmd, COUNT(*) as cnt FROM commands @@ -924,6 +988,7 @@ impl Tracker { } /// Get overall savings percentage (for telemetry). + #[allow(dead_code)] pub fn overall_savings_pct(&self) -> Result { let (total_input, total_saved): (i64, i64) = self.conn.query_row( "SELECT COALESCE(SUM(input_tokens), 0), COALESCE(SUM(saved_tokens), 0) FROM commands", @@ -938,6 +1003,7 @@ impl Tracker { } /// Get total tokens saved across all tracked commands (for telemetry). + #[allow(dead_code)] pub fn total_tokens_saved(&self) -> Result { let saved: i64 = self.conn.query_row( "SELECT COALESCE(SUM(saved_tokens), 0) FROM commands", @@ -948,6 +1014,7 @@ impl Tracker { } /// Get tokens saved in the last 24 hours (for telemetry). + #[allow(dead_code)] pub fn tokens_saved_24h(&self, since: chrono::DateTime) -> Result { let ts = since.format("%Y-%m-%dT%H:%M:%S").to_string(); let saved: i64 = self.conn.query_row( @@ -957,6 +1024,213 @@ impl Tracker { )?; Ok(saved) } + + /// Top N passthrough commands (0% savings) — commands missing a filter. + /// Groups by first word only to avoid leaking arguments into telemetry. + #[allow(dead_code)] + pub fn top_passthrough(&self, limit: usize) -> Result> { + let mut stmt = self.conn.prepare( + "SELECT TRIM(SUBSTR(original_cmd, 1, INSTR(original_cmd || ' ', ' ') - 1)) as tool, + COUNT(*) as cnt FROM commands + WHERE input_tokens = 0 AND output_tokens = 0 + GROUP BY tool ORDER BY cnt DESC LIMIT ?1", + )?; + let rows = stmt.query_map(params![limit as i64], |row| { + let cmd: String = row.get(0)?; + let count: i64 = row.get(1)?; + Ok((cmd, count)) + })?; + Ok(rows.filter_map(|r| r.ok()).collect()) + } + + /// Count parse failures in the last 24 hours. + #[allow(dead_code)] + pub fn parse_failures_since(&self, since: chrono::DateTime) -> Result { + let ts = since.format("%Y-%m-%dT%H:%M:%S").to_string(); + let count: i64 = self.conn.query_row( + "SELECT COUNT(*) FROM parse_failures WHERE timestamp >= ?1", + params![ts], + |row| row.get(0), + )?; + Ok(count) + } + + /// Count commands with low savings (<30%) — filters that need improvement. + #[allow(dead_code)] + pub fn low_savings_commands(&self, limit: usize) -> Result> { + let mut stmt = self.conn.prepare( + "SELECT rtk_cmd, AVG(savings_pct) as avg_sav FROM commands + WHERE input_tokens > 0 + GROUP BY rtk_cmd + HAVING avg_sav < 30.0 AND avg_sav > 0.0 + ORDER BY COUNT(*) DESC LIMIT ?1", + )?; + let rows = stmt.query_map(params![limit as i64], |row| { + let cmd: String = row.get(0)?; + let sav: f64 = row.get(1)?; + let short = cmd.split_whitespace().take(3).collect::>().join(" "); + Ok((short, sav)) + })?; + Ok(rows.filter_map(|r| r.ok()).collect()) + } + + /// Average savings percentage per command (unweighted — each command name counts once). + #[allow(dead_code)] + pub fn avg_savings_per_command(&self) -> Result { + let avg: f64 = self.conn.query_row( + "SELECT COALESCE(AVG(avg_sav), 0.0) FROM ( + SELECT rtk_cmd, AVG(savings_pct) as avg_sav + FROM commands WHERE input_tokens > 0 + GROUP BY rtk_cmd + )", + [], + |row| row.get(0), + )?; + Ok(avg) + } + + /// Count invocations of a specific meta-command (by rtk_cmd suffix). + #[allow(dead_code)] + pub fn count_meta_command(&self, name: &str) -> Result { + let pattern = format!("rtk {}", name); + let count: i64 = self.conn.query_row( + "SELECT COUNT(*) FROM commands WHERE rtk_cmd LIKE ?1 || '%'", + params![pattern], + |row| row.get(0), + )?; + Ok(count) + } + + /// Days since first recorded command (installation age). + #[allow(dead_code)] + pub fn first_seen_days(&self) -> Result { + let oldest: Option = + match self + .conn + .query_row("SELECT MIN(timestamp) FROM commands", [], |row| row.get(0)) + { + Ok(v) => v, + Err(rusqlite::Error::QueryReturnedNoRows) => None, + Err(e) => return Err(anyhow::anyhow!("Failed to query first seen timestamp: {e}")), + }; + match oldest { + Some(ts) => { + let first = chrono::NaiveDateTime::parse_from_str(&ts, "%Y-%m-%dT%H:%M:%S") + .or_else(|_| chrono::NaiveDateTime::parse_from_str(&ts, "%Y-%m-%d %H:%M:%S")) + .map(|dt| dt.and_utc()) + .unwrap_or_else(|_| chrono::Utc::now()); + let days = (chrono::Utc::now() - first).num_days(); + Ok(days.max(0)) + } + None => Ok(0), + } + } + + /// Number of distinct active days in the last 30 days. + #[allow(dead_code)] + pub fn active_days_30d(&self) -> Result { + let since = (chrono::Utc::now() - chrono::Duration::days(30)) + .format("%Y-%m-%dT%H:%M:%S") + .to_string(); + let count: i64 = self.conn.query_row( + "SELECT COUNT(DISTINCT DATE(timestamp)) FROM commands WHERE timestamp >= ?1", + params![since], + |row| row.get(0), + )?; + Ok(count) + } + + /// Total number of recorded commands. + #[allow(dead_code)] + pub fn commands_total(&self) -> Result { + let count: i64 = self + .conn + .query_row("SELECT COUNT(*) FROM commands", [], |row| row.get(0))?; + Ok(count) + } + + /// Ecosystem distribution as percentages (top categories by command prefix). + #[allow(dead_code)] + pub fn ecosystem_mix(&self) -> Result> { + let total: f64 = self.conn.query_row( + "SELECT COUNT(*) FROM commands WHERE input_tokens > 0 AND timestamp >= datetime('now', '-90 days')", + [], + |row| row.get(0), + )?; + if total == 0.0 { + return Ok(vec![]); + } + let mut stmt = self.conn.prepare( + "SELECT rtk_cmd, COUNT(*) as cnt FROM commands + WHERE input_tokens > 0 AND timestamp >= datetime('now', '-90 days') + GROUP BY rtk_cmd ORDER BY cnt DESC", + )?; + let mut categories: std::collections::HashMap = + std::collections::HashMap::new(); + let rows = stmt.query_map([], |row| { + let cmd: String = row.get(0)?; + let cnt: f64 = row.get(1)?; + Ok((cmd, cnt)) + })?; + for row in rows.flatten() { + let cat = categorize_command(&row.0); + *categories.entry(cat).or_default() += row.1; + } + let mut result: Vec<(String, f64)> = categories + .into_iter() + .map(|(cat, cnt)| (cat, (cnt / total * 100.0).round())) + .collect(); + result.sort_by(|a, b| b.1.partial_cmp(&a.1).unwrap_or(std::cmp::Ordering::Equal)); + result.truncate(8); + Ok(result) + } + + /// Tokens saved in the last 30 days. + #[allow(dead_code)] + pub fn tokens_saved_30d(&self) -> Result { + let since = (chrono::Utc::now() - chrono::Duration::days(30)) + .format("%Y-%m-%dT%H:%M:%S") + .to_string(); + let saved: i64 = self.conn.query_row( + "SELECT COALESCE(SUM(saved_tokens), 0) FROM commands WHERE timestamp >= ?1", + params![since], + |row| row.get(0), + )?; + Ok(saved) + } + + /// Number of distinct project paths. + #[allow(dead_code)] + pub fn projects_count(&self) -> Result { + let count: i64 = self.conn.query_row( + "SELECT COUNT(DISTINCT project_path) FROM commands WHERE project_path != ''", + [], + |row| row.get(0), + )?; + Ok(count) + } +} + +/// Map an rtk_cmd to an ecosystem category for telemetry. +#[allow(dead_code)] +fn categorize_command(rtk_cmd: &str) -> String { + let parts: Vec<&str> = rtk_cmd.split_whitespace().collect(); + let tool = parts.get(1).copied().unwrap_or("other"); + match tool { + "git" | "gh" | "gt" => "git", + "cargo" => "cargo", + "npm" | "npx" | "pnpm" | "vitest" | "tsc" | "lint" | "prettier" | "next" | "playwright" + | "prisma" => "js", + "pytest" | "ruff" | "mypy" | "pip" => "python", + "go" | "golangci-lint" => "go", + "docker" | "kubectl" => "cloud", + "rspec" | "rubocop" | "rake" => "ruby", + "dotnet" => "dotnet", + "ls" | "tree" | "grep" | "find" | "wc" | "read" | "env" | "json" | "log" | "smart" + | "diff" | "deps" | "summary" | "format" => "system", + _ => "other", + } + .to_string() } fn get_db_path() -> Result { @@ -974,7 +1248,7 @@ fn get_db_path() -> Result { // Priority 3: Default platform-specific location let data_dir = dirs::data_local_dir().unwrap_or_else(|| PathBuf::from(".")); - Ok(data_dir.join("rtk").join("history.db")) + Ok(data_dir.join(RTK_DATA_DIR).join(HISTORY_DB)) } /// Individual parse failure record. @@ -1161,42 +1435,6 @@ pub fn args_display(args: &[OsString]) -> String { .join(" ") } -/// Track a command execution (legacy function, use [`TimedExecution`] for new code). -/// -/// # Deprecation Notice -/// -/// This function is deprecated. Use [`TimedExecution`] instead for automatic -/// timing and cleaner API. -/// -/// # Arguments -/// -/// - `original_cmd`: Standard command (e.g., "ls -la") -/// - `rtk_cmd`: RTK command used (e.g., "rtk ls") -/// - `input`: Standard command output (for token estimation) -/// - `output`: RTK command output (for token estimation) -/// -/// # Migration -/// -/// ```no_run -/// # use rtk::tracking::{track, TimedExecution}; -/// // Old (deprecated) -/// track("ls -la", "rtk ls", "input", "output"); -/// -/// // New (preferred) -/// let timer = TimedExecution::start(); -/// timer.track("ls -la", "rtk ls", "input", "output"); -/// ``` -#[deprecated(note = "Use TimedExecution instead")] -#[allow(dead_code)] -pub fn track(original_cmd: &str, rtk_cmd: &str, input: &str, output: &str) { - let input_tokens = estimate_tokens(input); - let output_tokens = estimate_tokens(output); - - if let Ok(tracker) = Tracker::new() { - let _ = tracker.record(original_cmd, rtk_cmd, input_tokens, output_tokens, 0); - } -} - #[cfg(test)] mod tests { use super::*; @@ -1323,29 +1561,27 @@ mod tests { } // 7. get_db_path respects environment variable RTK_DB_PATH + // 8. get_db_path falls back to default when no custom config + // Combined into one test to avoid env var race between parallel tests #[test] - fn test_custom_db_path_env() { + fn test_db_path_env_and_default() { use std::env; + use std::sync::Mutex; + static ENV_LOCK: Mutex<()> = Mutex::new(()); + let _guard = ENV_LOCK.lock().unwrap(); - let custom_path = "/tmp/rtk_test_custom.db"; - env::set_var("RTK_DB_PATH", custom_path); - + let custom_path = env::temp_dir().join("rtk_test_custom.db"); + env::set_var("RTK_DB_PATH", &custom_path); let db_path = get_db_path().expect("Failed to get db path"); - assert_eq!(db_path, PathBuf::from(custom_path)); - - env::remove_var("RTK_DB_PATH"); - } - - // 8. get_db_path falls back to default when no custom config - #[test] - fn test_default_db_path() { - use std::env; + assert_eq!(db_path, custom_path); - // Ensure no env var is set env::remove_var("RTK_DB_PATH"); - let db_path = get_db_path().expect("Failed to get db path"); - assert!(db_path.ends_with("rtk/history.db")); + assert!( + db_path.ends_with("rtk/history.db"), + "expected default path ending with rtk/history.db, got: {}", + db_path.display() + ); } // 9. project_filter_params uses GLOB pattern with * wildcard // added @@ -1426,4 +1662,44 @@ mod tests { // but we can verify recovery_rate is between 0 and 100 assert!(summary.recovery_rate >= 0.0 && summary.recovery_rate <= 100.0); } + + #[test] + fn test_reset_all_clears_both_tables() { + let tracker = Tracker::new_in_memory().expect("Failed to create in-memory tracker"); + let pid = std::process::id(); + + // Insert into commands + tracker + .record( + "git status", + &format!("rtk git status reset_test_{}", pid), + 100, + 20, + 50, + ) + .expect("Failed to record command"); + + // Insert into parse_failures + tracker + .record_parse_failure(&format!("bad_cmd_reset_test_{}", pid), "parse error", false) + .expect("Failed to record parse failure"); + + // Reset everything + tracker.reset_all().expect("Failed to reset"); + + // Both tables should be empty + let summary = tracker.get_summary().expect("Failed to get summary"); + assert_eq!( + summary.total_commands, 0, + "commands table should be empty after reset" + ); + + let failures = tracker + .get_parse_failure_summary() + .expect("Failed to get failure summary"); + assert_eq!( + failures.total, 0, + "parse_failures table should be empty after reset" + ); + } } diff --git a/src/core/utils.rs b/src/core/utils.rs index c1882fa81..feae1fb6d 100644 --- a/src/core/utils.rs +++ b/src/core/utils.rs @@ -60,27 +60,6 @@ pub fn strip_ansi(text: &str) -> String { /// /// # Returns /// `(stdout: String, stderr: String, exit_code: i32)` -/// -/// # Examples -/// ```no_run -/// use rtk::utils::execute_command; -/// let (stdout, stderr, code) = execute_command("echo", &["test"]).unwrap(); -/// assert_eq!(code, 0); -/// ``` -#[allow(dead_code)] -pub fn execute_command(cmd: &str, args: &[&str]) -> Result<(String, String, i32)> { - let output = resolved_command(cmd) - .args(args) - .output() - .context(format!("Failed to execute {}", cmd))?; - - let stdout = String::from_utf8_lossy(&output.stdout).to_string(); - let stderr = String::from_utf8_lossy(&output.stderr).to_string(); - let exit_code = output.status.code().unwrap_or(-1); - - Ok((stdout, stderr, exit_code)) -} - /// Formats a token count with K/M suffixes for readability. /// /// # Arguments @@ -228,6 +207,27 @@ pub fn exit_code_from_output(output: &std::process::Output, label: &str) -> i32 } } +/// Extract exit code from an ExitStatus (for `.status()` calls, not `.output()`). +/// Returns the actual exit code, or `128 + signal` per Unix convention when +/// terminated by a signal. Falls back to 1 on non-Unix platforms. +pub fn exit_code_from_status(status: &std::process::ExitStatus, label: &str) -> i32 { + match status.code() { + Some(code) => code, + None => { + #[cfg(unix)] + { + use std::os::unix::process::ExitStatusExt; + if let Some(sig) = status.signal() { + eprintln!("[rtk] {}: process terminated by signal {}", label, sig); + return 128 + sig; + } + } + eprintln!("[rtk] {}: process terminated by signal", label); + 1 + } + } +} + /// Return the last `n` lines of output with a label, for use as a fallback /// when filter parsing fails. Logs a diagnostic to stderr. pub fn fallback_tail(output: &str, label: &str, n: usize) -> String { @@ -368,6 +368,42 @@ pub fn tool_exists(name: &str) -> bool { which::which(name).is_ok() } +/// Extract short name from AWS ARN. +/// Example: `arn:aws:ecs:region:acct:service/cluster/name` -> `name` +/// For simple ARNs like `arn:aws:iam::123:user/alice`, returns `alice`. +pub fn shorten_arn(arn: &str) -> &str { + // ARNs use "/" or ":" as separators. Try "/" first (service/cluster/name pattern), + // then fall back to ":" for Lambda/IAM ARNs. + let slash_result = arn.rsplit('/').next().unwrap_or(arn); + // If rsplit('/') returned the whole string (no '/' found), try ':' + if slash_result == arn { + arn.rsplit(':').next().unwrap_or(arn) + } else { + slash_result + } +} + +/// Convert bytes to human-readable format (KB, MB, GB, TB). +/// Used for S3 object sizes. +pub fn human_bytes(bytes: u64) -> String { + const KB: u64 = 1024; + const MB: u64 = KB * 1024; + const GB: u64 = MB * 1024; + const TB: u64 = GB * 1024; + + if bytes >= TB { + format!("{:.1} TB", bytes as f64 / TB as f64) + } else if bytes >= GB { + format!("{:.1} GB", bytes as f64 / GB as f64) + } else if bytes >= MB { + format!("{:.1} MB", bytes as f64 / MB as f64) + } else if bytes >= KB { + format!("{:.1} KB", bytes as f64 / KB as f64) + } else { + format!("{} B", bytes) + } +} + #[cfg(test)] mod tests { use super::*; @@ -421,21 +457,6 @@ mod tests { assert_eq!(strip_ansi(input), "Green normal Red"); } - #[test] - fn test_execute_command_success() { - let result = execute_command("echo", &["test"]); - assert!(result.is_ok()); - let (stdout, _, code) = result.unwrap(); - assert_eq!(code, 0); - assert!(stdout.contains("test")); - } - - #[test] - fn test_execute_command_failure() { - let result = execute_command("nonexistent_command_xyz_12345", &[]); - assert!(result.is_err()); - } - #[test] fn test_format_tokens_millions() { assert_eq!(format_tokens(1_234_567), "1.2M"); @@ -763,4 +784,82 @@ mod tests { ); } } + + // ===== AWS helper function tests ===== + + #[test] + fn test_shorten_arn_ecs_service() { + assert_eq!( + shorten_arn("arn:aws:ecs:us-east-1:123:service/cluster/api-service"), + "api-service" + ); + } + + #[test] + fn test_shorten_arn_iam_user() { + assert_eq!(shorten_arn("arn:aws:iam::123456789012:user/alice"), "alice"); + } + + #[test] + fn test_shorten_arn_lambda() { + assert_eq!( + shorten_arn("arn:aws:lambda:us-west-2:123:function:my-function"), + "my-function" + ); + } + + #[test] + fn test_shorten_arn_fallback() { + // Non-ARN string - return as-is + assert_eq!(shorten_arn("simple-name"), "simple-name"); + } + + #[test] + fn test_human_bytes_bytes() { + assert_eq!(human_bytes(0), "0 B"); + assert_eq!(human_bytes(512), "512 B"); + assert_eq!(human_bytes(1023), "1023 B"); + } + + #[test] + fn test_human_bytes_kb() { + assert_eq!(human_bytes(1024), "1.0 KB"); + assert_eq!(human_bytes(2048), "2.0 KB"); + assert_eq!(human_bytes(1536), "1.5 KB"); + } + + #[test] + fn test_human_bytes_mb() { + assert_eq!(human_bytes(1_048_576), "1.0 MB"); + assert_eq!(human_bytes(5_242_880), "5.0 MB"); + } + + #[test] + fn test_human_bytes_gb() { + assert_eq!(human_bytes(1_073_741_824), "1.0 GB"); + assert_eq!(human_bytes(2_147_483_648), "2.0 GB"); + } + + #[test] + fn test_human_bytes_tb() { + assert_eq!(human_bytes(1_099_511_627_776), "1.0 TB"); + } + + #[test] + fn test_count_tokens_basic() { + assert_eq!(count_tokens("hello world"), 2); + assert_eq!(count_tokens("one two three four"), 4); + } + + #[test] + fn test_count_tokens_empty() { + assert_eq!(count_tokens(""), 0); + assert_eq!(count_tokens(" "), 0); + } + + #[test] + fn test_count_tokens_multiple_spaces() { + assert_eq!(count_tokens("hello world"), 2); + assert_eq!(count_tokens(" hello world "), 2); + } } diff --git a/src/discover/README.md b/src/discover/README.md index cb523f4dc..4897fe870 100644 --- a/src/discover/README.md +++ b/src/discover/README.md @@ -1,32 +1,73 @@ -# Discover — Claude Code History Analysis +# Discover — History Analysis & Command Rewrite -> See also [docs/TECHNICAL.md](../../docs/TECHNICAL.md) for the full architecture overview +> Full rewrite pipeline diagram: [docs/contributing/TECHNICAL.md](../../docs/contributing/TECHNICAL.md#32-hook-interception-command-rewriting) -## Purpose +## What This Module Does -Scans Claude Code JSONL session files to identify commands that could benefit from RTK filtering. Powers the `rtk discover` command, which reports missed savings opportunities and adoption metrics. +This module has two jobs: -Also provides the **command rewrite registry** — the single source of truth for all rewrite patterns used by every LLM agent hook to decide which commands to rewrite. +1. **Rewrite commands** — Every LLM agent hook calls `rtk rewrite "git status"`. This module decides whether to rewrite it (`rtk git status`) or pass it through unchanged. This is the hot path — every command the LLM runs goes through here. -## Key Types +2. **Analyze history** — `rtk discover` scans past LLM sessions to find commands that *could have been* rewritten but weren't. Same classification logic, different consumer. -- **`Classification`** — Result of `classify_command()`: `Supported { rtk_equivalent, category, savings_pct, status }`, `Unsupported { base_command }`, or `Ignored` -- **`RtkStatus`** — `Existing` (dedicated handler), `Passthrough` (external_subcommand), `NotSupported` -- **`SessionProvider`** trait — abstraction for session file discovery (currently only `ClaudeProvider`) -- **`ExtractedCommand`** — command string + output length + error flag extracted from JSONL +## How Command Rewriting Works -## Dependencies +When a hook sends `cargo fmt --all && cargo test 2>&1 | tail -20`: -- **Uses**: `walkdir` (session file discovery), `lazy_static`/`regex` (pattern matching), `serde_json` (JSONL parsing) -- **Used by**: `src/hooks/rewrite_cmd.rs` (imports `registry::classify_command` for `rtk rewrite`), `src/learn/` (imports `provider::ClaudeProvider` for session extraction), `src/main.rs` (routes `rtk discover` command) +**Tokenization** — The lexer (`lexer.rs`) turns the raw string into typed tokens. It's a single-pass state machine that understands shell quoting, escapes, redirects, and operators. This is critical because naive string splitting breaks on quoted content like `git commit -m "fix && update"`. -## Registry Architecture +``` +"cargo test 2>&1 && git status" +→ [Arg("cargo"), Arg("test"), Redirect("2>&1"), Operator("&&"), Arg("git"), Arg("status")] +``` -`registry.rs` is the largest file in the project. It contains: +**Compound splitting** — The rewrite engine walks the tokens, splitting on `Operator` (`&&`, `||`, `;`) and `Pipe` (`|`). Each segment is rewritten independently. For pipes, only the left side is rewritten (the pipe consumer like `grep` or `head` runs raw). `find`/`fd` before a pipe is never rewritten because rtk's grouped output format breaks pipe consumers like `xargs`. -1. **Pattern matching** — Compiled regexes in `lazy_static!` matching command prefixes (e.g., `^git\s+(status|log|diff|...)`) -2. **Compound splitting** — `split_command_chain()` handles `&&`, `||`, `;`, `|`, `&` operators with shell quoting awareness -3. **RTK_DISABLED detection** — `has_rtk_disabled_prefix()` / `strip_disabled_prefix()` for per-command override -4. **Category averages** — `category_avg_tokens()` estimates output tokens when real data unavailable +**Per-segment rewriting** — Each segment goes through: -The registry is used by both `rtk discover` (analysis) and `rtk rewrite` (live rewriting). Same patterns, different consumers. +1. Strip trailing redirects (`2>&1`, `>/dev/null`) — matched via lexer tokens, set aside, re-appended after rewriting +2. Short-circuit special cases — `head -20 file` → `rtk read file --max-lines 20`, `tail -n 5 file` → `rtk read file --tail-lines 5`. These can't go through generic prefix replacement because it would produce `rtk read -20 file` (wrong flag position) +3. Classify the command — strip env prefixes (`sudo`, `FOO="bar baz"`), normalize paths (`/usr/bin/grep` → `grep`), strip git global opts (`git -C /tmp` → `git`), then match against 60+ regex patterns from `rules.rs` +4. Apply the rewrite — find the matching rule, replace the command prefix with `rtk `, re-prepend the env prefix, re-append the redirect suffix + +**Guards along the way:** +- `RTK_DISABLED=1` in the env prefix → skip rewrite +- `gh` with `--json`/`--jq`/`--template` → skip (structured output, rtk would corrupt it) +- `cat` with flags other than `-n` → skip (different semantics than `rtk read`) +- `cat`/`head`/`tail` with `>` or `>>` → skip (write operation, not a read) +- Command in `hooks.exclude_commands` config → skip + +**Result**: `rtk cargo fmt --all && rtk cargo test 2>&1 | tail -20`. Bash handles the `&&` and `|` at execution time — each `rtk` invocation is a separate process. + +## How History Analysis Works + +`rtk discover` reads Claude Code JSONL session files. Each file contains `tool_use`/`tool_result` pairs for every command the LLM ran. The module: + +1. Extracts commands from the JSONL (via `SessionProvider` trait — currently only Claude Code) +2. Splits compound commands using the same lexer-based tokenization +3. Classifies each command against the same rules used for live rewriting +4. Aggregates results: which commands could have been rewritten, estimated token savings, adoption rate + +The classification logic is shared between discover and rewrite — same patterns, same rules, different consumers. + +## Env Prefix Handling + +The `ENV_PREFIX` regex strips env variable assignments, `sudo`, and `env` from the front of commands. It handles: +- Unquoted: `FOO=bar` +- Double-quoted with spaces: `FOO="bar baz"` +- Single-quoted: `FOO='bar baz'` +- Escaped quotes: `FOO="he said \"hello\""` +- Chained: `A="x y" B=1 sudo git status` + +The prefix is stripped twice: once in `classify_command()` to match the underlying command against rules, and again in `rewrite_segment()` to extract it for re-prepending to the rewritten command. + +## Adding a New Rewrite Rule + +Add an entry to `rules.rs`. Each rule has: +- `pattern` — regex that matches the command (used by `RegexSet` for fast matching) +- `rtk_cmd` — the RTK command it maps to (e.g., `"rtk cargo"`) +- `rewrite_prefixes` — command prefixes to replace (e.g., `&["cargo"]`) +- `category`, `savings_pct` — metadata for discover reports +- `subcmd_savings`, `subcmd_status` — per-subcommand overrides + +No other files need to change. The registry compiles the patterns at first use via `lazy_static`. diff --git a/src/discover/lexer.rs b/src/discover/lexer.rs new file mode 100644 index 000000000..8a126530a --- /dev/null +++ b/src/discover/lexer.rs @@ -0,0 +1,1032 @@ +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum TokenKind { + Arg, + Operator, + Pipe, + Redirect, + Shellism, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ParsedToken { + pub kind: TokenKind, + pub value: String, + pub offset: usize, +} + +pub fn tokenize(input: &str) -> Vec { + let mut tokens = Vec::new(); + let mut current = String::new(); + let mut current_start: usize = 0; + let mut byte_pos: usize = 0; + let mut chars = input.chars().peekable(); + let mut quote: Option = None; + let mut escaped = false; + + while let Some(c) = chars.next() { + let char_len = c.len_utf8(); + + if escaped { + current.push('\\'); + current.push(c); + byte_pos += char_len; + escaped = false; + continue; + } + if c == '\\' && quote != Some('\'') { + escaped = true; + if current.is_empty() { + current_start = byte_pos; + } + byte_pos += char_len; + continue; + } + + if let Some(q) = quote { + if c == q { + quote = None; + } + current.push(c); + byte_pos += char_len; + continue; + } + if c == '\'' || c == '"' { + quote = Some(c); + if current.is_empty() { + current_start = byte_pos; + } + current.push(c); + byte_pos += char_len; + continue; + } + + match c { + '$' => { + flush_arg(&mut tokens, &mut current, current_start); + let start = byte_pos; + byte_pos += char_len; + if chars + .peek() + .is_some_and(|&nc| nc.is_ascii_alphabetic() || nc == '_') + { + let mut name = String::from("$"); + while let Some(&nc) = chars.peek() { + if !nc.is_ascii_alphanumeric() && nc != '_' { + break; + } + chars.next(); + byte_pos += nc.len_utf8(); + name.push(nc); + } + tokens.push(ParsedToken { + kind: TokenKind::Arg, + value: name, + offset: start, + }); + } else { + tokens.push(ParsedToken { + kind: TokenKind::Shellism, + value: "$".into(), + offset: start, + }); + } + current_start = byte_pos; + } + '*' | '?' | '`' | '(' | ')' | '{' | '}' | '!' => { + flush_arg(&mut tokens, &mut current, current_start); + tokens.push(ParsedToken { + kind: TokenKind::Shellism, + value: c.to_string(), + offset: byte_pos, + }); + byte_pos += char_len; + current_start = byte_pos; + } + '|' => { + flush_arg(&mut tokens, &mut current, current_start); + let start = byte_pos; + byte_pos += char_len; + if chars.peek() == Some(&'|') { + chars.next(); + byte_pos += 1; + tokens.push(ParsedToken { + kind: TokenKind::Operator, + value: "||".into(), + offset: start, + }); + } else { + tokens.push(ParsedToken { + kind: TokenKind::Pipe, + value: "|".into(), + offset: start, + }); + } + current_start = byte_pos; + } + ';' => { + flush_arg(&mut tokens, &mut current, current_start); + tokens.push(ParsedToken { + kind: TokenKind::Operator, + value: ";".into(), + offset: byte_pos, + }); + byte_pos += char_len; + current_start = byte_pos; + } + '&' => { + flush_arg(&mut tokens, &mut current, current_start); + let start = byte_pos; + byte_pos += char_len; + if chars.peek() == Some(&'&') { + chars.next(); + byte_pos += 1; + tokens.push(ParsedToken { + kind: TokenKind::Operator, + value: "&&".into(), + offset: start, + }); + } else if chars.peek() == Some(&'>') { + chars.next(); + byte_pos += 1; + let mut val = String::from("&>"); + if chars.peek() == Some(&'>') { + chars.next(); + byte_pos += 1; + val.push('>'); + } + tokens.push(ParsedToken { + kind: TokenKind::Redirect, + value: val, + offset: start, + }); + } else { + tokens.push(ParsedToken { + kind: TokenKind::Shellism, + value: "&".into(), + offset: start, + }); + } + current_start = byte_pos; + } + '>' => { + let fd_prefix = + if !current.is_empty() && current.chars().all(|ch| ch.is_ascii_digit()) { + Some(std::mem::take(&mut current)) + } else { + flush_arg(&mut tokens, &mut current, current_start); + None + }; + let redir_start = if fd_prefix.is_some() { + current_start + } else { + byte_pos + }; + let mut val = fd_prefix.unwrap_or_default(); + val.push('>'); + byte_pos += char_len; + if chars.peek() == Some(&'>') { + chars.next(); + byte_pos += 1; + val.push('>'); + } + if chars.peek() == Some(&'&') { + chars.next(); + byte_pos += 1; + val.push('&'); + while let Some(&nc) = chars.peek() { + if !nc.is_ascii_digit() && nc != '-' { + break; + } + chars.next(); + val.push(nc); + byte_pos += nc.len_utf8(); + } + } + tokens.push(ParsedToken { + kind: TokenKind::Redirect, + value: val, + offset: redir_start, + }); + current_start = byte_pos; + } + '<' => { + flush_arg(&mut tokens, &mut current, current_start); + let start = byte_pos; + let mut val = String::from("<"); + byte_pos += char_len; + if chars.peek() == Some(&'<') { + chars.next(); + byte_pos += 1; + val.push('<'); + } + tokens.push(ParsedToken { + kind: TokenKind::Redirect, + value: val, + offset: start, + }); + current_start = byte_pos; + } + c if c.is_whitespace() => { + flush_arg(&mut tokens, &mut current, current_start); + byte_pos += c.len_utf8(); + current_start = byte_pos; + } + _ => { + if current.is_empty() { + current_start = byte_pos; + } + current.push(c); + byte_pos += char_len; + } + } + } + + if escaped { + current.push('\\'); + } + flush_arg(&mut tokens, &mut current, current_start); + tokens +} + +fn flush_arg(tokens: &mut Vec, current: &mut String, offset: usize) { + if !current.is_empty() { + tokens.push(ParsedToken { + kind: TokenKind::Arg, + value: std::mem::take(current), + offset, + }); + } +} + +/// Split a shell command on operators (`&&`, `||`, `;`) and optionally pipes (`|`), +/// respecting quoted strings via the lexer. +/// +/// When `stop_at_pipe` is true, returns only segments before the first `|` +/// (used by command rewriting — only the left side of a pipe gets rewritten). +/// When false, splits through pipes too (used by permission checking — +/// every segment must be validated). +pub fn split_on_operators(cmd: &str, stop_at_pipe: bool) -> Vec<&str> { + let trimmed = cmd.trim(); + if trimmed.is_empty() { + return vec![]; + } + + let tokens = tokenize(trimmed); + let mut results = Vec::new(); + let mut seg_start: usize = 0; + + for tok in &tokens { + match tok.kind { + TokenKind::Operator => { + let segment = trimmed[seg_start..tok.offset].trim(); + if !segment.is_empty() { + results.push(segment); + } + seg_start = tok.offset + tok.value.len(); + } + TokenKind::Pipe => { + let segment = trimmed[seg_start..tok.offset].trim(); + if !segment.is_empty() { + results.push(segment); + } + if stop_at_pipe { + return results; + } + seg_start = tok.offset + tok.value.len(); + } + _ => {} + } + } + + let tail = trimmed[seg_start..].trim(); + if !tail.is_empty() { + results.push(tail); + } + + results +} + +pub fn strip_quotes(s: &str) -> String { + let chars: Vec = s.chars().collect(); + if chars.len() >= 2 + && ((chars[0] == '"' && chars[chars.len() - 1] == '"') + || (chars[0] == '\'' && chars[chars.len() - 1] == '\'')) + { + return chars[1..chars.len() - 1].iter().collect(); + } + s.to_string() +} + +pub fn shell_split(input: &str) -> Vec { + let mut tokens = Vec::new(); + let mut current = String::new(); + let mut chars = input.chars().peekable(); + let mut in_single = false; + let mut in_double = false; + + while let Some(c) = chars.next() { + match c { + '\\' if !in_single => { + if let Some(next) = chars.next() { + current.push(next); + } + } + '\'' if !in_double => { + in_single = !in_single; + } + '"' if !in_single => { + in_double = !in_double; + } + ' ' | '\t' if !in_single && !in_double => { + if !current.is_empty() { + tokens.push(std::mem::take(&mut current)); + } + } + _ => { + current.push(c); + } + } + } + + if !current.is_empty() { + tokens.push(current); + } + + tokens +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn test_simple_command() { + let tokens = tokenize("git status"); + assert_eq!(tokens.len(), 2); + assert_eq!(tokens[0].kind, TokenKind::Arg); + assert_eq!(tokens[0].value, "git"); + assert_eq!(tokens[1].value, "status"); + } + + #[test] + fn test_command_with_args() { + let tokens = tokenize("git commit -m message"); + assert_eq!(tokens.len(), 4); + assert_eq!(tokens[0].value, "git"); + assert_eq!(tokens[1].value, "commit"); + assert_eq!(tokens[2].value, "-m"); + assert_eq!(tokens[3].value, "message"); + } + + #[test] + fn test_quoted_operator_not_split() { + let tokens = tokenize(r#"git commit -m "Fix && Bug""#); + assert!(!tokens + .iter() + .any(|t| matches!(t.kind, TokenKind::Operator) && t.value == "&&")); + assert!(tokens.iter().any(|t| t.value.contains("Fix && Bug"))); + } + + #[test] + fn test_single_quoted_string() { + let tokens = tokenize("echo 'hello world'"); + assert!(tokens.iter().any(|t| t.value == "'hello world'")); + } + + #[test] + fn test_double_quoted_string() { + let tokens = tokenize(r#"echo "hello world""#); + assert!(tokens.iter().any(|t| t.value == "\"hello world\"")); + } + + #[test] + fn test_empty_quoted_string() { + let tokens = tokenize("echo \"\""); + assert!(tokens.iter().any(|t| t.value == "\"\"")); + } + + #[test] + fn test_nested_quotes() { + let tokens = tokenize(r#"echo "outer 'inner' outer""#); + assert!(tokens.iter().any(|t| t.value.contains("'inner'"))); + } + + #[test] + fn test_escaped_space() { + let tokens = tokenize("echo hello\\ world"); + assert!(tokens.iter().any(|t| t.value.contains("hello"))); + } + + #[test] + fn test_backslash_in_single_quotes() { + let tokens = tokenize(r#"echo 'hello\nworld'"#); + assert!(tokens.iter().any(|t| t.value.contains(r"\n"))); + } + + #[test] + fn test_escaped_quote_in_double() { + let tokens = tokenize(r#"echo "hello\"world""#); + assert!(tokens.iter().any(|t| t.value.contains("hello"))); + } + + #[test] + fn test_empty_input() { + assert!(tokenize("").is_empty()); + } + + #[test] + fn test_whitespace_only() { + assert!(tokenize(" ").is_empty()); + } + + #[test] + fn test_unclosed_single_quote() { + let tokens = tokenize("'unclosed"); + assert!(!tokens.is_empty()); + } + + #[test] + fn test_unclosed_double_quote() { + let tokens = tokenize("\"unclosed"); + assert!(!tokens.is_empty()); + } + + #[test] + fn test_unicode_preservation() { + let tokens = tokenize("echo \"héllo wörld\""); + assert!(tokens.iter().any(|t| t.value.contains("héllo"))); + } + + #[test] + fn test_multiple_spaces() { + let tokens = tokenize("git status"); + assert_eq!(tokens.len(), 2); + } + + #[test] + fn test_leading_trailing_spaces() { + let tokens = tokenize(" git status "); + assert_eq!(tokens.len(), 2); + } + + #[test] + fn test_and_operator() { + let tokens = tokenize("cmd1 && cmd2"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Operator && t.value == "&&")); + } + + #[test] + fn test_or_operator() { + let tokens = tokenize("cmd1 || cmd2"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Operator && t.value == "||")); + } + + #[test] + fn test_semicolon() { + let tokens = tokenize("cmd1 ; cmd2"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Operator && t.value == ";")); + } + + #[test] + fn test_multiple_and() { + let tokens = tokenize("a && b && c"); + let ops: Vec<_> = tokens + .iter() + .filter(|t| t.kind == TokenKind::Operator) + .collect(); + assert_eq!(ops.len(), 2); + } + + #[test] + fn test_mixed_operators() { + let tokens = tokenize("a && b || c"); + let ops: Vec<_> = tokens + .iter() + .filter(|t| t.kind == TokenKind::Operator) + .collect(); + assert_eq!(ops.len(), 2); + } + + #[test] + fn test_operator_at_start() { + let tokens = tokenize("&& cmd"); + assert!(tokens.iter().any(|t| t.value == "&&")); + } + + #[test] + fn test_operator_at_end() { + let tokens = tokenize("cmd &&"); + assert!(tokens.iter().any(|t| t.value == "&&")); + } + + #[test] + fn test_pipe_detection() { + let tokens = tokenize("cat file | grep pattern"); + assert!(tokens.iter().any(|t| t.kind == TokenKind::Pipe)); + } + + #[test] + fn test_quoted_pipe_not_pipe() { + let tokens = tokenize("\"a|b\""); + assert!(!tokens.iter().any(|t| t.kind == TokenKind::Pipe)); + } + + #[test] + fn test_multiple_pipes() { + let tokens = tokenize("a | b | c"); + let pipes: Vec<_> = tokens + .iter() + .filter(|t| t.kind == TokenKind::Pipe) + .collect(); + assert_eq!(pipes.len(), 2); + } + + #[test] + fn test_glob_detection() { + let tokens = tokenize("ls *.rs"); + assert!(tokens.iter().any(|t| t.kind == TokenKind::Shellism)); + } + + #[test] + fn test_quoted_glob_not_shellism() { + let tokens = tokenize("echo \"*.txt\""); + assert!(!tokens.iter().any(|t| t.kind == TokenKind::Shellism)); + } + + #[test] + fn test_simple_var_is_arg() { + let tokens = tokenize("echo $HOME"); + assert!( + tokens + .iter() + .any(|t| t.kind == TokenKind::Arg && t.value == "$HOME"), + "Simple $VAR must be Arg — shell expands at execution time" + ); + assert!( + !tokens.iter().any(|t| t.kind == TokenKind::Shellism), + "No Shellism expected for simple $VAR" + ); + } + + #[test] + fn test_simple_var_enables_native_routing() { + let tokens = tokenize("git log $BRANCH"); + assert!( + !tokens.iter().any(|t| t.kind == TokenKind::Shellism), + "git log $BRANCH must have no Shellism" + ); + } + + #[test] + fn test_dollar_subshell_stays_shellism() { + let tokens = tokenize("echo $(date)"); + assert!(tokens.iter().any(|t| t.kind == TokenKind::Shellism)); + } + + #[test] + fn test_dollar_brace_stays_shellism() { + let tokens = tokenize("echo ${HOME}"); + assert!(tokens.iter().any(|t| t.kind == TokenKind::Shellism)); + } + + #[test] + fn test_dollar_special_vars_stay_shellism() { + for s in &["echo $?", "echo $$", "echo $!"] { + let tokens = tokenize(s); + assert!( + tokens.iter().any(|t| t.kind == TokenKind::Shellism), + "{} should produce Shellism", + s + ); + } + } + + #[test] + fn test_dollar_digit_stays_shellism() { + let tokens = tokenize("echo $1"); + assert!(tokens.iter().any(|t| t.kind == TokenKind::Shellism)); + } + + #[test] + fn test_quoted_variable_not_shellism() { + let tokens = tokenize("echo \"$HOME\""); + assert!(!tokens.iter().any(|t| t.kind == TokenKind::Shellism)); + } + + #[test] + fn test_backtick_substitution() { + let tokens = tokenize("echo `date`"); + assert!(tokens.iter().any(|t| t.kind == TokenKind::Shellism)); + } + + #[test] + fn test_subshell_detection() { + let tokens = tokenize("echo $(date)"); + let shellisms: Vec<_> = tokens + .iter() + .filter(|t| t.kind == TokenKind::Shellism) + .collect(); + assert!(!shellisms.is_empty()); + } + + #[test] + fn test_brace_expansion() { + let tokens = tokenize("echo {a,b}.txt"); + assert!(tokens.iter().any(|t| t.kind == TokenKind::Shellism)); + } + + #[test] + fn test_escaped_glob() { + let tokens = tokenize("echo \\*.txt"); + assert!(!tokens + .iter() + .any(|t| t.kind == TokenKind::Shellism && t.value == "*")); + } + + #[test] + fn test_redirect_out() { + let tokens = tokenize("cmd > file"); + assert!(tokens.iter().any(|t| t.kind == TokenKind::Redirect)); + } + + #[test] + fn test_redirect_append() { + let tokens = tokenize("cmd >> file"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Redirect && t.value == ">>")); + } + + #[test] + fn test_redirect_in() { + let tokens = tokenize("cmd < file"); + assert!(tokens.iter().any(|t| t.kind == TokenKind::Redirect)); + } + + #[test] + fn test_redirect_stderr() { + let tokens = tokenize("cmd 2> file"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Redirect && t.value.starts_with("2>"))); + } + + #[test] + fn test_redirect_stderr_no_space() { + let tokens = tokenize("cmd 2>/dev/null"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Redirect && t.value == "2>")); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Arg && t.value == "/dev/null")); + } + + #[test] + fn test_redirect_dev_null() { + let tokens = tokenize("cmd > /dev/null"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Redirect && t.value == ">")); + } + + #[test] + fn test_redirect_2_to_1_single_token() { + let tokens = tokenize("cmd 2>&1"); + assert_eq!(tokens.len(), 2); + assert_eq!(tokens[1].kind, TokenKind::Redirect); + assert_eq!(tokens[1].value, "2>&1"); + assert!(!tokens + .iter() + .any(|t| t.kind == TokenKind::Shellism && t.value == "&")); + } + + #[test] + fn test_redirect_1_to_2_single_token() { + let tokens = tokenize("cmd 1>&2"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Redirect && t.value == "1>&2")); + } + + #[test] + fn test_redirect_fd_close() { + let tokens = tokenize("cmd 2>&-"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Redirect && t.value == "2>&-")); + } + + #[test] + fn test_redirect_shorthand_dup() { + let tokens = tokenize("cmd >&2"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Redirect && t.value == ">&2")); + } + + #[test] + fn test_redirect_amp_gt() { + let tokens = tokenize("cmd &>/dev/null"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Redirect && t.value == "&>")); + } + + #[test] + fn test_redirect_amp_gt_gt() { + let tokens = tokenize("cmd &>>/dev/null"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Redirect && t.value == "&>>")); + } + + #[test] + fn test_combined_redirect_chain() { + let tokens = tokenize("cmd > /dev/null 2>&1"); + let redirects: Vec<_> = tokens + .iter() + .filter(|t| t.kind == TokenKind::Redirect) + .collect(); + assert_eq!(redirects.len(), 2); + assert_eq!(redirects[0].value, ">"); + assert_eq!(redirects[1].value, "2>&1"); + } + + #[test] + fn test_redirect_append_to_file() { + let tokens = tokenize("echo hello >> /tmp/output.txt"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Redirect && t.value == ">>")); + } + + #[test] + fn test_redirect_heredoc_marker() { + let tokens = tokenize("cat <&1 | head"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Redirect && t.value == "2>&1")); + assert!(tokens.iter().any(|t| t.kind == TokenKind::Pipe)); + } + + #[test] + fn test_redirect_2_to_1_with_and() { + let tokens = tokenize("cargo test 2>&1 && echo done"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Redirect && t.value == "2>&1")); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Operator && t.value == "&&")); + } + + #[test] + fn test_exclamation_is_shellism() { + let tokens = tokenize("if ! grep -q pattern file; then echo missing; fi"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Shellism && t.value == "!")); + } + + #[test] + fn test_background_job_is_shellism() { + let tokens = tokenize("sleep 10 &"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Shellism && t.value == "&")); + } + + #[test] + fn test_background_not_confused_with_amp_redirect() { + let tokens = tokenize("cargo test &>/dev/null"); + assert!(!tokens + .iter() + .any(|t| t.kind == TokenKind::Shellism && t.value == "&")); + assert!(tokens.iter().any(|t| t.kind == TokenKind::Redirect)); + } + + #[test] + fn test_semicolon_no_space() { + let tokens = tokenize("git status;cargo test"); + assert_eq!( + tokens + .iter() + .filter(|t| t.kind == TokenKind::Operator) + .count(), + 1 + ); + assert_eq!( + tokens.iter().filter(|t| t.kind == TokenKind::Arg).count(), + 4 + ); + } + + #[test] + fn test_offset_tracking() { + let tokens = tokenize("a && b"); + assert_eq!(tokens[0].offset, 0); + assert_eq!(tokens[1].offset, 2); + assert_eq!(tokens[2].offset, 5); + } + + #[test] + fn test_offset_segment_extraction() { + let cmd = "git add . && cargo test"; + let tokens = tokenize(cmd); + let op = tokens + .iter() + .find(|t| t.kind == TokenKind::Operator) + .unwrap(); + let left = cmd[..op.offset].trim(); + let right_start = op.offset + op.value.len(); + let right = cmd[right_start..].trim(); + assert_eq!(left, "git add ."); + assert_eq!(right, "cargo test"); + } + + #[test] + fn test_env_prefix_is_arg() { + let tokens = tokenize("GIT_SSH_COMMAND=ssh git push"); + assert_eq!(tokens[0].kind, TokenKind::Arg); + assert_eq!(tokens[0].value, "GIT_SSH_COMMAND=ssh"); + } + + #[test] + fn test_complex_compound() { + let tokens = tokenize("cargo fmt --all && cargo clippy --all-targets && cargo test"); + let operators: Vec<_> = tokens + .iter() + .filter(|t| t.kind == TokenKind::Operator) + .collect(); + assert_eq!(operators.len(), 2); + assert!(operators.iter().all(|t| t.value == "&&")); + } + + #[test] + fn test_find_pipe_xargs() { + let tokens = tokenize("find . -name '*.rs' | xargs grep 'fn run'"); + let pipe_idx = tokens + .iter() + .position(|t| t.kind == TokenKind::Pipe) + .unwrap(); + assert!(pipe_idx > 0); + let before_pipe: Vec<_> = tokens[..pipe_idx] + .iter() + .filter(|t| t.kind == TokenKind::Arg) + .collect(); + assert!(before_pipe.iter().any(|t| t.value == "find")); + } + + #[test] + fn test_fd_redirect_needs_adjacent_digit() { + let tokens = tokenize("echo 2 > file"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Arg && t.value == "2")); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Redirect && t.value == ">")); + } + + #[test] + fn test_fd_redirect_no_space() { + let tokens = tokenize("echo 2>file"); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Redirect && t.value == "2>")); + assert!(tokens + .iter() + .any(|t| t.kind == TokenKind::Arg && t.value == "file")); + } + + #[test] + fn test_shell_split_simple() { + assert_eq!( + shell_split("head -50 file.php"), + vec!["head", "-50", "file.php"] + ); + } + + #[test] + fn test_shell_split_double_quotes() { + assert_eq!( + shell_split(r#"git log --format="%H %s""#), + vec!["git", "log", "--format=%H %s"] + ); + } + + #[test] + fn test_shell_split_single_quotes() { + assert_eq!( + shell_split("grep -r 'hello world' ."), + vec!["grep", "-r", "hello world", "."] + ); + } + + #[test] + fn test_shell_split_single_word() { + assert_eq!(shell_split("ls"), vec!["ls"]); + } + + #[test] + fn test_shell_split_empty() { + let result: Vec = shell_split(""); + assert!(result.is_empty()); + } + + #[test] + fn test_shell_split_backslash_escape() { + assert_eq!( + shell_split(r"echo hello\ world"), + vec!["echo", "hello world"] + ); + } + + #[test] + fn test_shell_split_unclosed_quote() { + let result = shell_split("echo 'hello"); + assert_eq!(result, vec!["echo", "hello"]); + } + + #[test] + fn test_shell_split_mixed_quotes() { + assert_eq!( + shell_split(r#"echo "it's" 'a "test"'"#), + vec!["echo", "it's", "a \"test\""] + ); + } + + #[test] + fn test_shell_split_tabs() { + assert_eq!(shell_split("a\tb\tc"), vec!["a", "b", "c"]); + } + + #[test] + fn test_shell_split_multiple_spaces() { + assert_eq!(shell_split("a b c"), vec!["a", "b", "c"]); + } + + #[test] + fn test_strip_quotes_double() { + assert_eq!(strip_quotes("\"hello\""), "hello"); + } + + #[test] + fn test_strip_quotes_single() { + assert_eq!(strip_quotes("'hello'"), "hello"); + } + + #[test] + fn test_strip_quotes_none() { + assert_eq!(strip_quotes("hello"), "hello"); + } + + #[test] + fn test_strip_quotes_mismatched() { + assert_eq!(strip_quotes("\"hello'"), "\"hello'"); + } + + #[test] + fn test_split_on_operators_stop_at_pipe() { + assert_eq!(split_on_operators("a | b | c", true), vec!["a"]); + assert_eq!(split_on_operators("a && b | c", true), vec!["a", "b"]); + } + + #[test] + fn test_split_on_operators_through_pipes() { + assert_eq!(split_on_operators("a | b | c", false), vec!["a", "b", "c"]); + assert_eq!( + split_on_operators("a && b | c ; d", false), + vec!["a", "b", "c", "d"] + ); + } + + #[test] + fn test_split_on_operators_quoted() { + assert_eq!( + split_on_operators(r#"echo "a && b" && cargo test"#, false), + vec![r#"echo "a && b""#, "cargo test"] + ); + } + + #[test] + fn test_split_on_operators_empty() { + assert!(split_on_operators("", false).is_empty()); + assert!(split_on_operators(" ", true).is_empty()); + } +} diff --git a/src/discover/mod.rs b/src/discover/mod.rs index 4be67a505..e5b4a87b8 100644 --- a/src/discover/mod.rs +++ b/src/discover/mod.rs @@ -1,5 +1,6 @@ //! Scans AI coding sessions to find commands that could benefit from RTK filtering. +pub mod lexer; pub mod provider; pub mod registry; mod report; @@ -20,8 +21,13 @@ struct SupportedBucket { rtk_equivalent: &'static str, category: &'static str, count: usize, + /// Total estimated tokens *saved* (post-filter). Used for the "Est. Savings" column. total_output_tokens: usize, - savings_pct: f64, + /// Total estimated tokens *before* filtering (raw output). Accumulated alongside + /// `total_output_tokens` so the bucket's effective savings rate can be derived as + /// `total_output_tokens / total_raw_output_tokens` — a weighted average across + /// all sub-commands, regardless of which sub-command was seen first. + total_raw_output_tokens: usize, // For display: the most common raw command command_counts: HashMap, } @@ -119,7 +125,7 @@ pub fn run( category, count: 0, total_output_tokens: 0, - savings_pct: estimated_savings_pct, + total_raw_output_tokens: 0, command_counts: HashMap::new(), } }); @@ -139,6 +145,9 @@ pub fn run( let savings = (output_tokens as f64 * estimated_savings_pct / 100.0) as usize; bucket.total_output_tokens += savings; + // Accumulate pre-savings tokens so we can compute a weighted effective + // savings rate across all sub-commands in this bucket later. + bucket.total_raw_output_tokens += output_tokens; // Track the display name with status let display_name = truncate_command(part); @@ -195,13 +204,22 @@ pub fn run( }) .unwrap_or_else(|| (String::new(), report::RtkStatus::Existing)); + // Derive the effective savings rate from accumulated totals rather than + // using the first-seen sub-command's rate. This gives a weighted average + // across all sub-commands that fell in this bucket. + let effective_savings_pct = if bucket.total_raw_output_tokens > 0 { + bucket.total_output_tokens as f64 * 100.0 / bucket.total_raw_output_tokens as f64 + } else { + 0.0 + }; + SupportedEntry { command: command_with_status, count: bucket.count, rtk_equivalent: bucket.rtk_equivalent, category: bucket.category, estimated_savings_tokens: bucket.total_output_tokens, - estimated_savings_pct: bucket.savings_pct, + estimated_savings_pct: effective_savings_pct, rtk_status: status, } }) diff --git a/src/discover/provider.rs b/src/discover/provider.rs index 286c38916..08b4ddc8f 100644 --- a/src/discover/provider.rs +++ b/src/discover/provider.rs @@ -1,5 +1,6 @@ //! Reads Claude Code session logs from disk and streams their command history. +use crate::hooks::constants::CLAUDE_DIR; use anyhow::{Context, Result}; use std::collections::HashMap; use std::fs; @@ -44,7 +45,7 @@ impl ClaudeProvider { /// Get the base directory for Claude Code projects. fn projects_dir() -> Result { let home = dirs::home_dir().context("could not determine home directory")?; - let dir = home.join(".claude").join("projects"); + let dir = home.join(CLAUDE_DIR).join("projects"); if !dir.exists() { anyhow::bail!( "Claude Code projects directory not found: {}\nMake sure Claude Code has been used at least once.", @@ -55,9 +56,26 @@ impl ClaudeProvider { } /// Encode a filesystem path to Claude Code's directory name format. - /// `/Users/foo/bar` → `-Users-foo-bar` + /// + /// Claude Code replaces `/`, `.`, `_`, `\`, and any non-ASCII character + /// with `-` when computing the project directory slug under `~/.claude/projects/`. + /// + /// `/Users/foo/bar` → `-Users-foo-bar` + /// `/Users/first.last/bar` → `-Users-first-last-bar` + /// `/home/chris/2_project` → `-home-chris-2-project` + /// `C:\Users\foo\bar` → `C:-Users-foo-bar` pub fn encode_project_path(path: &str) -> String { - path.replace('/', "-") + const SANITIZED_CHARS: &[char] = &['/', '.', '_', '\\']; + + path.chars() + .map(|c| { + if !c.is_ascii() || SANITIZED_CHARS.contains(&c) { + '-' + } else { + c + } + }) + .collect() } } @@ -336,6 +354,54 @@ mod tests { ); } + #[test] + fn test_encode_project_path_dot_in_username() { + // Claude Code replaces both '/' and '.' with '-'. + // A cwd like /Users/first.last must produce the same slug as + // Claude's projects directory (-Users-first-last), otherwise + // `rtk discover` finds zero sessions for that project. + assert_eq!( + ClaudeProvider::encode_project_path("/Users/first.last/my-project"), + "-Users-first-last-my-project" + ); + } + + #[test] + fn test_encode_project_path_multiple_dots() { + assert_eq!( + ClaudeProvider::encode_project_path("/Users/a.b.c/proj"), + "-Users-a-b-c-proj" + ); + } + + #[test] + fn test_encode_project_path_underscore() { + // Claude Code also replaces '_' with '-' (https://github.com/anthropics/claude-code/issues/24067) + assert_eq!( + ClaudeProvider::encode_project_path("/home/chris/2_project-files/proj"), + "-home-chris-2-project-files-proj" + ); + } + + #[test] + fn test_encode_project_path_non_ascii() { + // Non-ASCII characters are each replaced with '-' (https://github.com/anthropics/claude-code/issues/40946) + // '/home/user/' + '外' + '主' + '/app' -> '-home-user' + '-' + '-' + '-' + '-' + 'app' + assert_eq!( + ClaudeProvider::encode_project_path("/home/user/\u{5916}\u{4e3b}/app"), + "-home-user----app" + ); + } + + #[test] + fn test_encode_project_path_windows() { + // Windows backslashes are also replaced with '-' + assert_eq!( + ClaudeProvider::encode_project_path(r"C:\Users\foo\bar"), + "C:-Users-foo-bar" + ); + } + #[test] fn test_match_project_filter() { let encoded = ClaudeProvider::encode_project_path("/Users/foo/Sites/rtk"); diff --git a/src/discover/registry.rs b/src/discover/registry.rs index 95d397339..8b7ad326d 100644 --- a/src/discover/registry.rs +++ b/src/discover/registry.rs @@ -3,7 +3,8 @@ use lazy_static::lazy_static; use regex::{Regex, RegexSet}; -use super::rules::{IGNORED_EXACT, IGNORED_PREFIXES, PATTERNS, RULES}; +use super::lexer::{split_on_operators, tokenize, TokenKind}; +use super::rules::{IGNORED_EXACT, IGNORED_PREFIXES, RULES}; /// Result of classifying a command. #[derive(Debug, PartialEq)] @@ -37,23 +38,52 @@ pub fn category_avg_tokens(category: &str, subcmd: &str) -> usize { "Infra" => 120, "Network" => 150, "GitHub" => 200, + "GitLab" => 200, "PackageManager" => 150, _ => 150, } } lazy_static! { - static ref REGEX_SET: RegexSet = RegexSet::new(PATTERNS).expect("invalid regex patterns"); - static ref COMPILED: Vec = PATTERNS + static ref REGEX_SET: RegexSet = + RegexSet::new(RULES.iter().map(|r| r.pattern)).expect("invalid regex patterns"); + static ref COMPILED: Vec = RULES .iter() - .map(|p| Regex::new(p).expect("invalid regex")) + .map(|r| Regex::new(r.pattern).expect("invalid regex")) .collect(); - static ref ENV_PREFIX: Regex = - Regex::new(r"^(?:sudo\s+|env\s+|[A-Z_][A-Z0-9_]*=[^\s]*\s+)+").unwrap(); + static ref ENV_PREFIX: Regex = { + let double_quoted = r#""(?:[^"\\]|\\.)*""#; + let single_quoted = r#"'(?:[^'\\]|\\.)*'"#; + let unquoted = r#"[^\s]*"#; + let env_value = format!("(?:{}|{}|{})", double_quoted, single_quoted, unquoted); + let env_assign = format!(r#"[A-Z_][A-Z0-9_]*={}"#, env_value); + Regex::new(&format!(r#"^(?:sudo\s+|env\s+|{}\s+)+"#, env_assign)).unwrap() + }; // Git global options that appear before the subcommand: -C , -c , // --git-dir , --work-tree , and flag-only options (#163) static ref GIT_GLOBAL_OPT: Regex = Regex::new(r"^(?:(?:-C\s+\S+|-c\s+\S+|--git-dir(?:=\S+|\s+\S+)|--work-tree(?:=\S+|\s+\S+)|--no-pager|--no-optional-locks|--bare|--literal-pathspecs)\s+)+").unwrap(); + static ref HEAD_N: Regex = Regex::new(r"^head\s+-(\d+)\s+(.+)$").unwrap(); + static ref HEAD_LINES: Regex = Regex::new(r"^head\s+--lines=(\d+)\s+(.+)$").unwrap(); + static ref TAIL_N: Regex = Regex::new(r"^tail\s+-(\d+)\s+(.+)$").unwrap(); + static ref TAIL_N_SPACE: Regex = Regex::new(r"^tail\s+-n\s+(\d+)\s+(.+)$").unwrap(); + static ref TAIL_LINES_EQ: Regex = Regex::new(r"^tail\s+--lines=(\d+)\s+(.+)$").unwrap(); + static ref TAIL_LINES_SPACE: Regex = Regex::new(r"^tail\s+--lines\s+(\d+)\s+(.+)$").unwrap(); +} + +const GOLANGCI_GLOBAL_OPT_WITH_VALUE: &[&str] = &[ + "-c", + "--color", + "--config", + "--cpu-profile-path", + "--mem-profile-path", + "--trace-path", +]; + +#[derive(Debug, Clone, Copy)] +struct GolangciRunParts<'a> { + global_segment: &'a str, + run_segment: &'a str, } /// Classify a single (already-split) command. @@ -86,6 +116,9 @@ pub fn classify_command(cmd: &str) -> Classification { let cmd_normalized = strip_absolute_path(cmd_clean); // Strip git global options: git -C /tmp status → git status (#163) let cmd_normalized = strip_git_global_opts(&cmd_normalized); + // Strip golangci-lint global options before `run` so classify/rewrite stays + // aligned with the runtime wrapper behavior. + let cmd_normalized = strip_golangci_global_opts(&cmd_normalized); let cmd_clean = cmd_normalized.as_str(); // Exclude cat/head/tail with redirect operators — these are writes, not reads (#315) @@ -189,89 +222,25 @@ fn extract_base_command(cmd: &str) -> &str { } } -/// Split a command chain on `&&`, `||`, `;` outside quotes. -/// For pipes `|`, only keep the first command. -/// Lines with `<<` (heredoc) or `$((` are returned whole. +/// Quote-aware heredoc detection — `<<` inside quotes is not a heredoc. +pub fn has_heredoc(cmd: &str) -> bool { + tokenize(cmd) + .iter() + .any(|t| t.kind == TokenKind::Redirect && t.value.starts_with("<<")) +} + pub fn split_command_chain(cmd: &str) -> Vec<&str> { let trimmed = cmd.trim(); if trimmed.is_empty() { return vec![]; } - // Heredoc or arithmetic expansion: treat as single command - if trimmed.contains("<<") || trimmed.contains("$((") { + // Lexer-based for `<<`; string-based for `$((` (lexer splits it across tokens). + if has_heredoc(trimmed) || trimmed.contains("$((") { return vec![trimmed]; } - let mut results = Vec::new(); - let mut start = 0; - let bytes = trimmed.as_bytes(); - let len = bytes.len(); - let mut i = 0; - let mut in_single = false; - let mut in_double = false; - let mut pipe_seen = false; - - while i < len { - let b = bytes[i]; - match b { - b'\'' if !in_double => { - in_single = !in_single; - i += 1; - } - b'"' if !in_single => { - in_double = !in_double; - i += 1; - } - b'|' if !in_single && !in_double => { - if i + 1 < len && bytes[i + 1] == b'|' { - // || - let segment = trimmed[start..i].trim(); - if !segment.is_empty() { - results.push(segment); - } - i += 2; - start = i; - } else { - // pipe: keep only first command - let segment = trimmed[start..i].trim(); - if !segment.is_empty() { - results.push(segment); - } - pipe_seen = true; - break; - } - } - b'&' if !in_single && !in_double && i + 1 < len && bytes[i + 1] == b'&' => { - let segment = trimmed[start..i].trim(); - if !segment.is_empty() { - results.push(segment); - } - i += 2; - start = i; - } - b';' if !in_single && !in_double => { - let segment = trimmed[start..i].trim(); - if !segment.is_empty() { - results.push(segment); - } - i += 1; - start = i; - } - _ => { - i += 1; - } - } - } - - if !pipe_seen && start < len { - let segment = trimmed[start..].trim(); - if !segment.is_empty() { - results.push(segment); - } - } - - results + split_on_operators(trimmed, true) } /// Strip git global options before the subcommand (#163). @@ -287,6 +256,105 @@ fn strip_git_global_opts(cmd: &str) -> String { format!("git {}", stripped.trim()) } +/// Strip golangci-lint global options before the `run` subcommand. +/// `golangci-lint --color never run ./...` → `golangci-lint run ./...` +/// Returns the original string unchanged if this is not a supported compact `run` invocation. +fn strip_golangci_global_opts(cmd: &str) -> String { + match parse_golangci_run_parts(cmd) { + Some(parts) => format!("golangci-lint {}", parts.run_segment), + None => cmd.to_string(), + } +} + +/// Parse supported golangci-lint invocations with optional global flags before `run`. +fn parse_golangci_run_parts(cmd: &str) -> Option> { + let tokens = split_token_spans(cmd); + let first = tokens.first()?; + if first.0 != "golangci-lint" && first.0 != "golangci" { + return None; + } + + let mut i = 1; + while i < tokens.len() { + let token = tokens[i].0; + + if token == "--" { + return None; + } + + if !token.starts_with('-') { + if token == "run" { + let global_segment = if i > 1 { + cmd[tokens[1].1..tokens[i].1].trim() + } else { + "" + }; + let run_segment = cmd[tokens[i].1..].trim(); + return Some(GolangciRunParts { + global_segment, + run_segment, + }); + } + return None; + } + + if let Some(flag) = split_golangci_flag_name(token) { + if golangci_flag_takes_separate_value(token, flag) { + i += 1; + } + } + + i += 1; + } + + None +} + +fn split_golangci_flag_name(arg: &str) -> Option<&str> { + if arg.starts_with("--") { + return Some(arg.split_once('=').map(|(flag, _)| flag).unwrap_or(arg)); + } + + if arg.starts_with('-') { + return Some(arg); + } + + None +} + +fn golangci_flag_takes_separate_value(arg: &str, flag: &str) -> bool { + if !GOLANGCI_GLOBAL_OPT_WITH_VALUE.contains(&flag) { + return false; + } + + if arg.starts_with("--") && arg.contains('=') { + return false; + } + + true +} + +fn split_token_spans(cmd: &str) -> Vec<(&str, usize, usize)> { + let mut tokens = Vec::new(); + let mut start = None; + + for (idx, ch) in cmd.char_indices() { + if ch.is_whitespace() { + if let Some(token_start) = start.take() { + tokens.push((&cmd[token_start..idx], token_start, idx)); + } + } else if start.is_none() { + start = Some(idx); + } + } + + if let Some(token_start) = start { + tokens.push((&cmd[token_start..], token_start, cmd.len())); + } + + tokens +} + /// Normalize absolute binary paths: `/usr/bin/grep -rn foo` → `grep -rn foo` (#485) /// Only strips if the first word contains a `/` (Unix path). fn strip_absolute_path(cmd: &str) -> String { @@ -329,49 +397,66 @@ pub fn strip_disabled_prefix(cmd: &str) -> &str { trimmed[prefix_len..].trim_start() } -lazy_static! { - // Match trailing shell redirections: - // Alt 1: N>&M or N>&- (fd redirect/close): 2>&1, 1>&2, 2>&- - // Alt 2: &>file or &>>file (bash redirect both): &>/dev/null - // Alt 3: N>file or N>>file (fd to file): 2>/dev/null, >/tmp/out, 1>>log - // Note: [^(\\s] excludes process substitutions like >(tee) from false-positive matching - static ref TRAILING_REDIRECT: Regex = - Regex::new(r"\s+(?:[0-9]?>&[0-9-]|&>>?\S+|[0-9]?>>?\s*[^(\s]\S*)\s*$").unwrap(); -} - -/// Strip trailing stderr/stdout redirects from a command segment (#530). -/// Returns (command_without_redirects, redirect_suffix). fn strip_trailing_redirects(cmd: &str) -> (&str, &str) { - if let Some(m) = TRAILING_REDIRECT.find(cmd) { - // Verify redirect is not inside quotes (single-pass count) - let before = &cmd[..m.start()]; - let (sq, dq) = before.chars().fold((0u32, 0u32), |(s, d), c| match c { - '\'' => (s + 1, d), - '"' => (s, d + 1), - _ => (s, d), - }); - if sq % 2 == 0 && dq % 2 == 0 { - return (&cmd[..m.start()], &cmd[m.start()..]); + let tokens = tokenize(cmd); + if tokens.is_empty() { + return (cmd, ""); + } + + let mut redir_boundary = tokens.len(); + let mut i = tokens.len(); + while i > 0 { + i -= 1; + match tokens[i].kind { + TokenKind::Redirect => { + redir_boundary = i; + } + TokenKind::Arg => { + if i > 0 && tokens[i - 1].kind == TokenKind::Redirect { + redir_boundary = i - 1; + i -= 1; + } else { + break; + } + } + _ => break, } } - (cmd, "") + + if redir_boundary >= tokens.len() { + return (cmd, ""); + } + + let cut = tokens[redir_boundary].offset; + let cmd_part = cmd[..cut].trim_end(); + let redir_part = &cmd[cmd_part.len()..]; + (cmd_part, redir_part) } /// Returns `None` if the command is unsupported or ignored (hook should pass through). /// /// Handles compound commands (`&&`, `||`, `;`) by rewriting each segment independently. -/// For pipes (`|`), only rewrites the first command (the filter stays raw). +/// For pipes (`|`), only rewrites the left-hand command (pipe targets stay raw), +/// but continues rewriting segments after subsequent `&&`/`||`/`;` operators. pub fn rewrite_command(cmd: &str, excluded: &[String]) -> Option { let trimmed = cmd.trim(); if trimmed.is_empty() { return None; } - // Heredoc or arithmetic expansion — unsafe to split/rewrite - if trimmed.contains("<<") || trimmed.contains("$((") { + if has_heredoc(trimmed) || trimmed.contains("$((") { + return None; + } + + // Shell function definitions: rewriting commands inside function bodies + // produces invalid references when the function is later called. + // Detect patterns like `name() {`, `function name {`, `name ()`. + if trimmed.contains("() {") || trimmed.contains("() \n") || trimmed.starts_with("function ") { return None; } + let compiled = compile_exclude_patterns(excluded); + // Simple (non-compound) already-RTK command — return as-is. // For compound commands that start with "rtk" (e.g. "rtk git add . && cargo test"), // fall through to rewrite_compound so the remaining segments get rewritten. @@ -384,136 +469,103 @@ pub fn rewrite_command(cmd: &str, excluded: &[String]) -> Option { return Some(trimmed.to_string()); } - rewrite_compound(trimmed, excluded) + rewrite_compound(trimmed, &compiled) } /// Rewrite a compound command (with `&&`, `||`, `;`, `|`) by rewriting each segment. -fn rewrite_compound(cmd: &str, excluded: &[String]) -> Option { - let bytes = cmd.as_bytes(); - let len = bytes.len(); - let mut result = String::with_capacity(len + 32); +fn rewrite_compound(cmd: &str, excluded: &[ExcludePattern]) -> Option { + let tokens = tokenize(cmd); + let mut result = String::with_capacity(cmd.len() + 32); let mut any_changed = false; - let mut seg_start = 0; - let mut i = 0; - let mut in_single = false; - let mut in_double = false; - - while i < len { - let b = bytes[i]; - match b { - b'\'' if !in_double => { - in_single = !in_single; - i += 1; - } - b'"' if !in_single => { - in_double = !in_double; - i += 1; - } - b'|' if !in_single && !in_double => { - if i + 1 < len && bytes[i + 1] == b'|' { - // `||` operator — rewrite left, continue - let seg = cmd[seg_start..i].trim(); - let rewritten = - rewrite_segment(seg, excluded).unwrap_or_else(|| seg.to_string()); - if rewritten != seg { - any_changed = true; - } - result.push_str(&rewritten); - result.push_str(" || "); - i += 2; - while i < len && bytes[i] == b' ' { - i += 1; + let mut seg_start: usize = 0; + + for tok in &tokens { + if tok.offset < seg_start { + continue; + } + match tok.kind { + TokenKind::Operator => { + let seg = cmd[seg_start..tok.offset].trim(); + let rewritten = rewrite_segment(seg, excluded).unwrap_or_else(|| seg.to_string()); + if rewritten != seg { + any_changed = true; + } + result.push_str(&rewritten); + if tok.value == ";" { + result.push(';'); + let after = tok.offset + tok.value.len(); + if after < cmd.len() { + result.push(' '); } - seg_start = i; } else { - // `|` pipe — rewrite first segment only, pass through the rest unchanged - let seg = cmd[seg_start..i].trim(); - // Skip rewriting `find`/`fd` in pipes — rtk find outputs a grouped - // format that is incompatible with pipe consumers like xargs, grep, - // wc, sort, etc. which expect one path per line (#439). - let is_pipe_incompatible = seg.starts_with("find ") - || seg == "find" - || seg.starts_with("fd ") - || seg == "fd"; - let rewritten = if is_pipe_incompatible { - seg.to_string() - } else { - rewrite_segment(seg, excluded).unwrap_or_else(|| seg.to_string()) - }; - if rewritten != seg { - any_changed = true; - } - result.push_str(&rewritten); - // Preserve the space before the pipe that was lost by trim() result.push(' '); - result.push_str(cmd[i..].trim_start()); - return if any_changed { Some(result) } else { None }; + result.push_str(&tok.value); + result.push(' '); + } + seg_start = tok.offset + tok.value.len(); + while seg_start < cmd.len() && cmd.as_bytes().get(seg_start) == Some(&b' ') { + seg_start += 1; } } - b'&' if !in_single && !in_double && i + 1 < len && bytes[i + 1] == b'&' => { - // `&&` operator — rewrite left, continue - let seg = cmd[seg_start..i].trim(); - let rewritten = rewrite_segment(seg, excluded).unwrap_or_else(|| seg.to_string()); + TokenKind::Pipe => { + let seg = cmd[seg_start..tok.offset].trim(); + // curl/wget piped to jq/python/grep produces JSON or HTML that + // downstream consumers parse — rtk filtering would break them. + let is_pipe_incompatible = seg.starts_with("find ") + || seg == "find" + || seg.starts_with("fd ") + || seg == "fd" + || seg.starts_with("curl ") + || seg == "curl" + || seg.starts_with("wget ") + || seg == "wget"; + let rewritten = if is_pipe_incompatible { + seg.to_string() + } else { + rewrite_segment(seg, excluded).unwrap_or_else(|| seg.to_string()) + }; if rewritten != seg { any_changed = true; } result.push_str(&rewritten); - result.push_str(" && "); - i += 2; - while i < len && bytes[i] == b' ' { - i += 1; - } - seg_start = i; - } - b'&' if !in_single && !in_double => { - // #346: redirect detection — 2>&1 / >&2 (> before &) or &>file / &>>file (> after &) - let is_redirect = - (i > 0 && bytes[i - 1] == b'>') || (i + 1 < len && bytes[i + 1] == b'>'); - if is_redirect { - i += 1; - } else { - // single `&` background execution operator - let seg = cmd[seg_start..i].trim(); - let rewritten = - rewrite_segment(seg, excluded).unwrap_or_else(|| seg.to_string()); - if rewritten != seg { - any_changed = true; + + let pipe_group_end = tokens.iter().find(|t| { + t.offset > tok.offset + && (t.kind == TokenKind::Operator + || (t.kind == TokenKind::Shellism && t.value == "&")) + }); + + match pipe_group_end { + Some(next_op) => { + result.push(' '); + result.push_str(cmd[tok.offset..next_op.offset].trim()); + seg_start = next_op.offset; } - result.push_str(&rewritten); - result.push_str(" & "); - i += 1; - while i < len && bytes[i] == b' ' { - i += 1; + None => { + result.push(' '); + result.push_str(cmd[tok.offset..].trim_start()); + return if any_changed { Some(result) } else { None }; } - seg_start = i; } } - b';' if !in_single && !in_double => { - // `;` separator - let seg = cmd[seg_start..i].trim(); + TokenKind::Shellism if tok.value == "&" => { + let seg = cmd[seg_start..tok.offset].trim(); let rewritten = rewrite_segment(seg, excluded).unwrap_or_else(|| seg.to_string()); if rewritten != seg { any_changed = true; } result.push_str(&rewritten); - result.push(';'); - i += 1; - while i < len && bytes[i] == b' ' { - i += 1; - } - if i < len { - result.push(' '); + result.push_str(" & "); + seg_start = tok.offset + tok.value.len(); + while seg_start < cmd.len() && cmd.as_bytes().get(seg_start) == Some(&b' ') { + seg_start += 1; } - seg_start = i; - } - _ => { - i += 1; } + _ => {} } } - // Last (or only) segment - let seg = cmd[seg_start..len].trim(); + let seg = cmd[seg_start..].trim(); let rewritten = rewrite_segment(seg, excluded).unwrap_or_else(|| seg.to_string()); if rewritten != seg { any_changed = true; @@ -527,45 +579,17 @@ fn rewrite_compound(cmd: &str, excluded: &[String]) -> Option { } } -/// Rewrite `head -N file` → `rtk read file --max-lines N`. -/// Returns `None` if the command doesn't match this pattern (fall through to generic logic). -fn rewrite_head_numeric(cmd: &str) -> Option { - // Match: head - (with optional env prefix) - lazy_static! { - static ref HEAD_N: Regex = Regex::new(r"^head\s+-(\d+)\s+(.+)$").expect("valid regex"); - static ref HEAD_LINES: Regex = - Regex::new(r"^head\s+--lines=(\d+)\s+(.+)$").expect("valid regex"); - } - if let Some(caps) = HEAD_N.captures(cmd) { - let n = caps.get(1)?.as_str(); - let file = caps.get(2)?.as_str(); - return Some(format!("rtk read {} --max-lines {}", file, n)); - } - if let Some(caps) = HEAD_LINES.captures(cmd) { - let n = caps.get(1)?.as_str(); - let file = caps.get(2)?.as_str(); - return Some(format!("rtk read {} --max-lines {}", file, n)); - } - // head with any other flag (e.g. -c, -q): skip rewriting to avoid clap errors +fn rewrite_line_range(cmd: &str) -> Option { + for re in [&*HEAD_N, &*HEAD_LINES] { + if let Some(caps) = re.captures(cmd) { + let n = caps.get(1)?.as_str(); + let file = caps.get(2)?.as_str(); + return Some(format!("rtk read {} --max-lines {}", file, n)); + } + } if cmd.starts_with("head -") { return None; } - None -} - -/// Rewrite `tail` numeric line forms to `rtk read ... --tail-lines N`. -/// Returns `None` when the pattern is unsupported (caller falls through / skips rewrite). -fn rewrite_tail_lines(cmd: &str) -> Option { - lazy_static! { - static ref TAIL_N: Regex = Regex::new(r"^tail\s+-(\d+)\s+(.+)$").expect("valid regex"); - static ref TAIL_N_SPACE: Regex = - Regex::new(r"^tail\s+-n\s+(\d+)\s+(.+)$").expect("valid regex"); - static ref TAIL_LINES_EQ: Regex = - Regex::new(r"^tail\s+--lines=(\d+)\s+(.+)$").expect("valid regex"); - static ref TAIL_LINES_SPACE: Regex = - Regex::new(r"^tail\s+--lines\s+(\d+)\s+(.+)$").expect("valid regex"); - } - for re in [ &*TAIL_N, &*TAIL_N_SPACE, @@ -578,20 +602,84 @@ fn rewrite_tail_lines(cmd: &str) -> Option { return Some(format!("rtk read {} --tail-lines {}", file, n)); } } - - // Unknown tail form: skip rewrite to preserve native behavior. None } -/// Rewrite a single (non-compound) command segment. -/// Returns `Some(rewritten)` if matched (including already-RTK pass-through). -/// Returns `None` if no match (caller uses original segment). -fn rewrite_segment(seg: &str, excluded: &[String]) -> Option { +/// Shell prefix builtins that modify how the shell runs a command +/// but don't change which command runs. Strip before routing, re-prepend after. +const SHELL_PREFIX_BUILTINS: &[&str] = &["noglob", "command", "builtin", "exec", "nocorrect"]; + +const MAX_PREFIX_DEPTH: usize = 10; + +enum ExcludePattern { + Regex(Regex), + Prefix(String), +} + +fn compile_exclude_patterns(patterns: &[String]) -> Vec { + patterns + .iter() + .filter_map(|pattern| { + let trimmed = pattern.trim(); + if trimmed.is_empty() || trimmed == "^" { + eprintln!( + "rtk: warning: ignoring trivial exclude_commands pattern '{}'", + pattern + ); + return None; + } + let anchored = if trimmed.starts_with('^') { + trimmed.to_string() + } else { + format!(r"^{}($|\s)", regex::escape(trimmed)) + }; + Some(match Regex::new(&anchored) { + Ok(re) => ExcludePattern::Regex(re), + Err(e) => { + eprintln!( + "rtk: warning: invalid exclude_commands pattern '{}': {}", + pattern, e + ); + ExcludePattern::Prefix(pattern.clone()) + } + }) + }) + .collect() +} + +fn rewrite_segment(seg: &str, excluded: &[ExcludePattern]) -> Option { + rewrite_segment_inner(seg, excluded, 0) +} + +fn is_excluded(cmd: &str, excluded: &[ExcludePattern]) -> bool { + excluded.iter().any(|pat| match pat { + ExcludePattern::Regex(re) => re.is_match(cmd), + ExcludePattern::Prefix(prefix) => cmd.starts_with(prefix.as_str()), + }) +} + +fn rewrite_segment_inner(seg: &str, excluded: &[ExcludePattern], depth: usize) -> Option { let trimmed = seg.trim(); if trimmed.is_empty() { return None; } + if depth >= MAX_PREFIX_DEPTH { + return None; + } + + for &prefix in SHELL_PREFIX_BUILTINS { + if let Some(rest) = strip_word_prefix(trimmed, prefix) { + if rest.is_empty() { + return None; + } + return match rewrite_segment_inner(rest, excluded, depth + 1) { + Some(rewritten) => Some(format!("{} {}", prefix, rewritten)), + None => None, + }; + } + } + // Strip trailing stderr/stdout redirects before matching (#530) // e.g. "git status 2>&1" → match "git status", re-append " 2>&1" let (cmd_part, redirect_suffix) = strip_trailing_redirects(trimmed); @@ -601,25 +689,15 @@ fn rewrite_segment(seg: &str, excluded: &[String]) -> Option { return Some(trimmed.to_string()); } - // Special case: `head -N file` / `head --lines=N file` → `rtk read file --max-lines N` - // Must intercept before generic prefix replacement, which would produce `rtk read -20 file`. - // Only intercept when head has a flag (-N, --lines=N, -c, etc.); plain `head file` falls - // through to the generic rewrite below and produces `rtk read file` as expected. - if cmd_part.starts_with("head -") { - return rewrite_head_numeric(cmd_part).map(|r| format!("{}{}", r, redirect_suffix)); - } - - // tail has several forms that are not compatible with generic prefix replacement. - // Only rewrite recognized numeric line forms; otherwise skip rewrite. - if cmd_part.starts_with("tail ") { - return rewrite_tail_lines(cmd_part).map(|r| format!("{}{}", r, redirect_suffix)); + if cmd_part.starts_with("head -") || cmd_part.starts_with("tail ") { + return rewrite_line_range(cmd_part).map(|r| format!("{}{}", r, redirect_suffix)); } // Most cat flags (-v, -A, -e, -t, -s, -b, --show-all, etc.) have different // semantics than rtk read or no equivalent at all. Only `-n` (line numbers) // maps correctly to `rtk read -n`. Skip rewrite for any other flag. - if cmd_part.starts_with("cat ") { - let args = cmd_part["cat ".len()..].trim_start(); + if let Some(cmd_args) = cmd_part.strip_prefix("cat ") { + let args = cmd_args.trim_start(); if args.starts_with('-') && !args.starts_with("-n ") && !args.starts_with("-n\t") { return None; } @@ -628,9 +706,9 @@ fn rewrite_segment(seg: &str, excluded: &[String]) -> Option { // Use classify_command for correct ignore/prefix handling let rtk_equivalent = match classify_command(cmd_part) { Classification::Supported { rtk_equivalent, .. } => { - // Check if the base command is excluded from rewriting (#243) - let base = cmd_part.split_whitespace().next().unwrap_or(""); - if excluded.iter().any(|e| e == base) { + let stripped = ENV_PREFIX.replace(cmd_part, ""); + let cmd_clean = stripped.trim(); + if is_excluded(cmd_clean, excluded) { return None; } rtk_equivalent @@ -648,10 +726,27 @@ fn rewrite_segment(seg: &str, excluded: &[String]) -> Option { let cmd_clean = stripped_cow.trim(); // #345: RTK_DISABLED=1 in env prefix → skip rewrite entirely + // #508: warn on stderr so agents learn to stop overusing it if has_rtk_disabled_prefix(cmd_part) { + eprintln!( + "[rtk] RTK_DISABLED=1 detected — skipping filter for this command. \ + Remove RTK_DISABLED=1 to restore token savings." + ); return None; } + if let Some(parts) = parse_golangci_run_parts(cmd_clean) { + let rewritten = if parts.global_segment.is_empty() { + format!("{}rtk golangci-lint {}", env_prefix, parts.run_segment) + } else { + format!( + "{}rtk golangci-lint {} {}", + env_prefix, parts.global_segment, parts.run_segment + ) + }; + return Some(rewritten); + } + // #196: gh with --json/--jq/--template produces structured output that // rtk gh would corrupt — skip rewrite so the caller gets raw JSON. if rule.rtk_cmd == "rtk gh" { @@ -712,6 +807,40 @@ mod tests { ); } + #[test] + fn test_classify_yadm_status() { + assert_eq!( + classify_command("yadm status"), + Classification::Supported { + rtk_equivalent: "rtk git", + category: "Git", + estimated_savings_pct: 70.0, + status: RtkStatus::Existing, + } + ); + } + + #[test] + fn test_classify_yadm_diff() { + assert_eq!( + classify_command("yadm diff"), + Classification::Supported { + rtk_equivalent: "rtk git", + category: "Git", + estimated_savings_pct: 80.0, + status: RtkStatus::Existing, + } + ); + } + + #[test] + fn test_rewrite_yadm_status() { + assert_eq!( + rewrite_command("yadm status", &[]), + Some("rtk git status".to_string()) + ); + } + #[test] fn test_classify_git_diff_cached() { assert_eq!( @@ -889,15 +1018,6 @@ mod tests { ); } - #[test] - fn test_patterns_rules_length_match() { - assert_eq!( - PATTERNS.len(), - RULES.len(), - "PATTERNS and RULES must be aligned" - ); - } - #[test] fn test_registry_covers_all_cargo_subcommands() { // Verify that every CargoCommand variant (Build, Test, Clippy, Check, Fmt) @@ -1152,19 +1272,32 @@ mod tests { } #[test] - fn test_rewrite_npx_tsc() { - assert_eq!( - rewrite_command("npx tsc --noEmit", &[]), - Some("rtk tsc --noEmit".into()) - ); - } - - #[test] - fn test_rewrite_pnpm_tsc() { - assert_eq!( - rewrite_command("pnpm tsc --noEmit", &[]), - Some("rtk tsc --noEmit".into()) - ); + fn test_rewrite_tsc() { + let commands = vec![ + "npm exec tsc", + "npm rum tsc", + "npm run tsc", + "npm run-script tsc", + "npm urn tsc", + "npm x tsc", + "pnpm dlx tsc", + "pnpm exec tsc", + "pnpm run tsc", + "pnpm run-script tsc", + "npm tsc", + "npx tsc", + "pnpm tsc", + "pnpx tsc", + "tsc", + ]; + for command in commands { + assert_eq!( + rewrite_command(&format!("{command} --noEmit"), &[]), + Some("rtk tsc --noEmit".into()), + "Failed for command: {}", + command + ); + } } #[test] @@ -1204,19 +1337,61 @@ mod tests { } #[test] - fn test_rewrite_npx_playwright() { - assert_eq!( - rewrite_command("npx playwright test", &[]), - Some("rtk playwright test".into()) - ); + fn test_rewrite_playwright() { + let commands = vec![ + "npm exec playwright", + "npm rum playwright", + "npm run playwright", + "npm run-script playwright", + "npm urn playwright", + "npm x playwright", + "pnpm dlx playwright", + "pnpm exec playwright", + "pnpm run playwright", + "pnpm run-script playwright", + "npm playwright", + "npx playwright", + "pnpm playwright", + "pnpx playwright", + "playwright", + ]; + for command in commands { + assert_eq!( + rewrite_command(&format!("{command} test"), &[]), + Some("rtk playwright test".into()), + "Failed for command: {}", + command + ); + } } #[test] fn test_rewrite_next_build() { - assert_eq!( - rewrite_command("next build --turbo", &[]), - Some("rtk next --turbo".into()) - ); + let commands = vec![ + "npm exec next build", + "npm rum next build", + "npm run next build", + "npm run-script next build", + "npm urn next build", + "npm x next build", + "pnpm dlx next build", + "pnpm exec next build", + "pnpm run next build", + "pnpm run-script next build", + "npm next build", + "npx next build", + "pnpm next build", + "pnpx next build", + "next build", + ]; + for command in commands { + assert_eq!( + rewrite_command(&format!("{command} --turbo"), &[]), + Some("rtk next --turbo".into()), + "Failed for command: {}", + command + ); + } } #[test] @@ -1296,70 +1471,162 @@ mod tests { } #[test] - fn test_rewrite_non_rtk_disabled_env_still_rewrites() { - assert_eq!( - rewrite_command("SOME_VAR=1 git status", &[]), - Some("SOME_VAR=1 rtk git status".into()) - ); + fn test_rewrite_rtk_disabled_warns_on_stderr() { + assert_eq!(rewrite_command("RTK_DISABLED=1 git status", &[]), None); } - // --- #346: 2>&1 and &> redirect detection --- - #[test] - fn test_rewrite_redirect_2_gt_amp_1_with_pipe() { - assert_eq!( - rewrite_command("cargo test 2>&1 | head", &[]), - Some("rtk cargo test 2>&1 | head".into()) + fn test_rewrite_rtk_disabled_subprocess_warns() { + let rtk_bin = std::path::Path::new(env!("CARGO_MANIFEST_DIR")) + .join("target") + .join("debug") + .join("rtk"); + if !rtk_bin.exists() { + return; + } + let rtk_mtime = std::fs::metadata(&rtk_bin) + .ok() + .and_then(|m| m.modified().ok()); + let test_mtime = std::env::current_exe() + .ok() + .and_then(|p| std::fs::metadata(p).ok()) + .and_then(|m| m.modified().ok()); + if let (Some(rtk_t), Some(test_t)) = (rtk_mtime, test_mtime) { + if rtk_t < test_t { + return; + } + } + + let output = std::process::Command::new(&rtk_bin) + .args(["rewrite", "RTK_DISABLED=1 git status"]) + .output() + .expect("Failed to run rtk"); + + assert!( + !output.status.success(), + "Should exit non-zero (no rewrite)" + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("RTK_DISABLED=1 detected"), + "Should warn on stderr, got: {}", + stderr ); } #[test] - fn test_rewrite_redirect_2_gt_amp_1_trailing() { + fn test_rewrite_non_rtk_disabled_env_still_rewrites() { assert_eq!( - rewrite_command("cargo test 2>&1", &[]), - Some("rtk cargo test 2>&1".into()) + rewrite_command("SOME_VAR=1 git status", &[]), + Some("SOME_VAR=1 rtk git status".into()) ); } #[test] - fn test_rewrite_redirect_plain_2_devnull() { - // 2>/dev/null has no `&`, never broken — non-regression + fn test_rewrite_env_quoted_value_with_spaces() { assert_eq!( - rewrite_command("git status 2>/dev/null", &[]), - Some("rtk git status 2>/dev/null".into()) + rewrite_command( + r#"GIT_SSH_COMMAND="ssh -o StrictHostKeyChecking=no" git push"#, + &[] + ), + Some(r#"GIT_SSH_COMMAND="ssh -o StrictHostKeyChecking=no" rtk git push"#.into()) ); } #[test] - fn test_rewrite_redirect_2_gt_amp_1_with_and() { + fn test_rewrite_env_single_quoted_value_with_spaces() { assert_eq!( - rewrite_command("cargo test 2>&1 && echo done", &[]), - Some("rtk cargo test 2>&1 && echo done".into()) + rewrite_command("EDITOR='vim -u NONE' git commit", &[]), + Some("EDITOR='vim -u NONE' rtk git commit".into()) ); } #[test] - fn test_rewrite_redirect_amp_gt_devnull() { + fn test_rewrite_env_quoted_plus_unquoted() { assert_eq!( - rewrite_command("cargo test &>/dev/null", &[]), - Some("rtk cargo test &>/dev/null".into()) + rewrite_command(r#"FOO="bar baz" BAR=1 git status"#, &[]), + Some(r#"FOO="bar baz" BAR=1 rtk git status"#.into()) ); } #[test] - fn test_rewrite_redirect_double() { - // Double redirect: only last one stripped, but full command rewrites correctly + fn test_rewrite_env_escaped_quotes_in_value() { assert_eq!( - rewrite_command("git status 2>&1 >/dev/null", &[]), - Some("rtk git status 2>&1 >/dev/null".into()) + rewrite_command(r#"FOO="he said \"hello\"" git status"#, &[]), + Some(r#"FOO="he said \"hello\"" rtk git status"#.into()) ); } #[test] - fn test_rewrite_redirect_fd_close() { - // 2>&- (close stderr fd) + fn test_classify_env_quoted_value_stripped() { assert_eq!( - rewrite_command("git status 2>&-", &[]), + classify_command(r#"GIT_SSH_COMMAND="ssh -o StrictHostKeyChecking=no" git push"#), + Classification::Supported { + rtk_equivalent: "rtk git", + category: "Git", + estimated_savings_pct: 70.0, + status: RtkStatus::Existing, + } + ); + } + + // --- #346: 2>&1 and &> redirect detection --- + + #[test] + fn test_rewrite_redirect_2_gt_amp_1_with_pipe() { + assert_eq!( + rewrite_command("cargo test 2>&1 | head", &[]), + Some("rtk cargo test 2>&1 | head".into()) + ); + } + + #[test] + fn test_rewrite_redirect_2_gt_amp_1_trailing() { + assert_eq!( + rewrite_command("cargo test 2>&1", &[]), + Some("rtk cargo test 2>&1".into()) + ); + } + + #[test] + fn test_rewrite_redirect_plain_2_devnull() { + // 2>/dev/null has no `&`, never broken — non-regression + assert_eq!( + rewrite_command("git status 2>/dev/null", &[]), + Some("rtk git status 2>/dev/null".into()) + ); + } + + #[test] + fn test_rewrite_redirect_2_gt_amp_1_with_and() { + assert_eq!( + rewrite_command("cargo test 2>&1 && echo done", &[]), + Some("rtk cargo test 2>&1 && echo done".into()) + ); + } + + #[test] + fn test_rewrite_redirect_amp_gt_devnull() { + assert_eq!( + rewrite_command("cargo test &>/dev/null", &[]), + Some("rtk cargo test &>/dev/null".into()) + ); + } + + #[test] + fn test_rewrite_redirect_double() { + // Double redirect: only last one stripped, but full command rewrites correctly + assert_eq!( + rewrite_command("git status 2>&1 >/dev/null", &[]), + Some("rtk git status 2>&1 >/dev/null".into()) + ); + } + + #[test] + fn test_rewrite_redirect_fd_close() { + // 2>&- (close stderr fd) + assert_eq!( + rewrite_command("git status 2>&-", &[]), Some("rtk git status 2>&-".into()) ); } @@ -1473,6 +1740,55 @@ mod tests { )); } + #[test] + fn test_classify_glab_mr() { + assert!(matches!( + classify_command("glab mr list"), + Classification::Supported { + rtk_equivalent: "rtk glab", + .. + } + )); + } + + #[test] + fn test_classify_glab_ci() { + assert!(matches!( + classify_command("glab ci list"), + Classification::Supported { + rtk_equivalent: "rtk glab", + .. + } + )); + } + + #[test] + fn test_classify_glab_release() { + assert!(matches!( + classify_command("glab release list"), + Classification::Supported { + rtk_equivalent: "rtk glab", + .. + } + )); + } + + #[test] + fn test_rewrite_glab_mr_list() { + assert_eq!( + rewrite_command("glab mr list", &[]), + Some("rtk glab mr list".into()) + ); + } + + #[test] + fn test_rewrite_glab_ci_status() { + assert_eq!( + rewrite_command("glab ci status", &[]), + Some("rtk glab ci status".into()) + ); + } + #[test] fn test_classify_cargo_install() { assert!(matches!( @@ -1908,7 +2224,73 @@ mod tests { assert!(matches!( classify_command("golangci-lint run"), Classification::Supported { - rtk_equivalent: "rtk golangci-lint", + rtk_equivalent: "rtk golangci-lint run", + .. + } + )); + } + + #[test] + fn test_classify_golangci_lint_with_flag_before_run() { + assert!(matches!( + classify_command("golangci-lint -v run ./..."), + Classification::Supported { + rtk_equivalent: "rtk golangci-lint run", + .. + } + )); + } + + #[test] + fn test_classify_golangci_lint_with_value_flag_before_run() { + assert!(matches!( + classify_command("golangci-lint --color never run ./..."), + Classification::Supported { + rtk_equivalent: "rtk golangci-lint run", + .. + } + )); + } + + #[test] + fn test_classify_golangci_lint_with_inline_value_flag_before_run() { + assert!(matches!( + classify_command("golangci-lint --color=never run ./..."), + Classification::Supported { + rtk_equivalent: "rtk golangci-lint run", + .. + } + )); + } + + #[test] + fn test_classify_golangci_lint_with_inline_config_flag_before_run() { + assert!(matches!( + classify_command("golangci-lint --config=foo.yml run ./..."), + Classification::Supported { + rtk_equivalent: "rtk golangci-lint run", + .. + } + )); + } + + #[test] + fn test_classify_golangci_lint_bare_is_not_compact_wrapper() { + assert!(!matches!( + classify_command("golangci-lint"), + Classification::Supported { + rtk_equivalent: "rtk golangci-lint run", + .. + } + )); + } + + #[test] + fn test_classify_golangci_lint_other_subcommand_is_not_compact_wrapper() { + assert!(!matches!( + classify_command("golangci-lint version"), + Classification::Supported { + rtk_equivalent: "rtk golangci-lint run", .. } )); @@ -1946,70 +2328,513 @@ mod tests { ); } - // --- JS/TS tooling --- + #[test] + fn test_rewrite_golangci_lint_with_flag_before_run() { + assert_eq!( + rewrite_command("golangci-lint -v run ./...", &[]), + Some("rtk golangci-lint -v run ./...".into()) + ); + } #[test] - fn test_classify_vitest() { - assert!(matches!( - classify_command("vitest run"), - Classification::Supported { - rtk_equivalent: "rtk vitest", - .. - } - )); + fn test_rewrite_golangci_lint_with_value_flag_before_run() { + assert_eq!( + rewrite_command("golangci-lint --color never run ./...", &[]), + Some("rtk golangci-lint --color never run ./...".into()) + ); } #[test] - fn test_rewrite_vitest() { + fn test_rewrite_golangci_lint_with_inline_value_flag_before_run() { assert_eq!( - rewrite_command("vitest run", &[]), - Some("rtk vitest run".into()) + rewrite_command("golangci-lint --color=never run ./...", &[]), + Some("rtk golangci-lint --color=never run ./...".into()) ); } #[test] - fn test_rewrite_pnpm_vitest() { + fn test_rewrite_golangci_lint_with_inline_config_flag_before_run() { assert_eq!( - rewrite_command("pnpm vitest run", &[]), - Some("rtk vitest run".into()) + rewrite_command("golangci-lint --config=foo.yml run ./...", &[]), + Some("rtk golangci-lint --config=foo.yml run ./...".into()) ); } #[test] - fn test_classify_prisma() { - assert!(matches!( - classify_command("npx prisma migrate dev"), - Classification::Supported { - rtk_equivalent: "rtk prisma", - .. - } - )); + fn test_rewrite_env_prefixed_golangci_lint_with_value_flag_before_run() { + assert_eq!( + rewrite_command("FOO=1 golangci-lint --color never run ./...", &[]), + Some("FOO=1 rtk golangci-lint --color never run ./...".into()) + ); } #[test] - fn test_rewrite_prisma() { + fn test_rewrite_env_prefixed_golangci_lint_with_inline_value_flag_before_run() { assert_eq!( - rewrite_command("npx prisma migrate dev", &[]), - Some("rtk prisma migrate dev".into()) + rewrite_command("FOO=1 golangci-lint --color=never run ./...", &[]), + Some("FOO=1 rtk golangci-lint --color=never run ./...".into()) ); } + #[test] + fn test_rewrite_bare_golangci_lint_skips_compact_wrapper() { + assert_eq!(rewrite_command("golangci-lint", &[]), None); + } + + #[test] + fn test_rewrite_other_golangci_lint_subcommand_skips_compact_wrapper() { + assert_eq!(rewrite_command("golangci-lint version", &[]), None); + } + + // --- JS/TS tooling --- + + #[test] + fn test_classify_lint() { + let commands = vec![ + "npm exec biome", + "npm exec eslint", + "npm rum biome", + "npm rum eslint", + "npm rum lint", + "npm run biome", + "npm run eslint", + "npm run lint", + "npm run-script biome", + "npm run-script eslint", + "npm run-script lint", + "npm urn biome", + "npm urn eslint", + "npm urn lint", + "npm x biome", + "npm x eslint", + "pnpm dlx biome", + "pnpm dlx eslint", + "pnpm exec biome", + "pnpm exec eslint", + "pnpm run biome", + "pnpm run eslint", + "pnpm run lint", + "pnpm run-script biome", + "pnpm run-script eslint", + "pnpm run-script lint", + "npm biome", + "npm eslint", + "npm lint", + "npx biome", + "npx eslint", + "npx lint", + "pnpm biome", + "pnpm eslint", + "pnpm lint", + "pnpx biome", + "pnpx eslint", + "pnpx lint", + "biome", + "eslint", + "lint", + ]; + for command in commands { + assert!( + matches!( + classify_command(command), + Classification::Supported { + rtk_equivalent: "rtk lint", + .. + } + ), + "Failed for command: {}", + command + ); + } + } + + #[test] + fn test_rewrite_lint() { + let commands = vec![ + "npm exec biome", + "npm exec eslint", + "npm rum biome", + "npm rum eslint", + "npm rum lint", + "npm run biome", + "npm run eslint", + "npm run lint", + "npm run-script biome", + "npm run-script eslint", + "npm run-script lint", + "npm urn biome", + "npm urn eslint", + "npm urn lint", + "npm x biome", + "npm x eslint", + "pnpm dlx biome", + "pnpm dlx eslint", + "pnpm exec biome", + "pnpm exec eslint", + "pnpm run biome", + "pnpm run eslint", + "pnpm run lint", + "pnpm run-script biome", + "pnpm run-script eslint", + "pnpm run-script lint", + "npm biome", + "npm eslint", + "npm lint", + "npx biome", + "npx eslint", + "npx lint", + "pnpm biome", + "pnpm eslint", + "pnpm lint", + "pnpx biome", + "pnpx eslint", + "pnpx lint", + "biome", + "eslint", + "lint", + ]; + for command in commands { + assert_eq!( + rewrite_command(command, &[]), + Some("rtk lint".into()), + "Failed for command: {}", + command + ); + } + } + + #[test] + fn test_classify_jest() { + let commands = vec![ + "jest run", + "jest", + "npm exec jest run", + "npm exec jest", + "npm jest run", + "npm jest", + "npm rum jest run", + "npm rum jest", + "npm run jest run", + "npm run jest", + "npm run-script jest run", + "npm run-script jest", + "npm urn jest run", + "npm urn jest", + "npm x jest run", + "npm x jest", + "npx jest run", + "npx jest", + "pnpm dlx jest run", + "pnpm dlx jest", + "pnpm exec jest run", + "pnpm exec jest", + "pnpm jest run", + "pnpm jest", + "pnpm run jest run", + "pnpm run jest", + "pnpm run-script jest run", + "pnpm run-script jest", + "pnpx jest run", + "pnpx jest", + ]; + for command in commands { + assert!( + matches!( + classify_command(command), + Classification::Supported { + rtk_equivalent: "rtk jest", + .. + } + ), + "Failed for command: {}", + command + ); + } + } + + #[test] + fn test_rewrite_jest() { + let commands = vec![ + "jest run", + "jest", + "npm exec jest run", + "npm exec jest", + "npm jest run", + "npm jest", + "npm rum jest run", + "npm rum jest", + "npm run jest run", + "npm run jest", + "npm run-script jest run", + "npm run-script jest", + "npm urn jest run", + "npm urn jest", + "npm x jest run", + "npm x jest", + "npx jest run", + "npx jest", + "pnpm dlx jest run", + "pnpm dlx jest", + "pnpm exec jest run", + "pnpm exec jest", + "pnpm jest run", + "pnpm jest", + "pnpm run jest run", + "pnpm run jest", + "pnpm run-script jest run", + "pnpm run-script jest", + "pnpx jest run", + "pnpx jest", + ]; + for command in commands { + assert_eq!( + rewrite_command(command, &[]), + Some("rtk jest".into()), + "Failed for command: {}", + command + ); + } + } + + #[test] + fn test_classify_vitest() { + let commands = vec![ + "npm exec vitest run", + "npm exec vitest", + "npm rum vitest run", + "npm rum vitest", + "npm run vitest run", + "npm run vitest", + "npm run-script vitest run", + "npm run-script vitest", + "npm urn vitest run", + "npm urn vitest", + "npm vitest run", + "npm vitest", + "npm x vitest run", + "npm x vitest", + "npx vitest run", + "npx vitest", + "pnpm dlx vitest run", + "pnpm dlx vitest", + "pnpm exec vitest run", + "pnpm exec vitest", + "pnpm run vitest run", + "pnpm run vitest", + "pnpm run-script vitest run", + "pnpm run-script vitest", + "pnpm vitest run", + "pnpm vitest", + "pnpx vitest run", + "pnpx vitest", + "vitest run", + "vitest", + ]; + for command in commands { + assert!( + matches!( + classify_command(command), + Classification::Supported { + rtk_equivalent: "rtk vitest", + .. + } + ), + "Failed for command: {}", + command + ); + } + } + + #[test] + fn test_rewrite_vitest() { + let commands = vec![ + "npm exec vitest run", + "npm exec vitest", + "npm rum vitest run", + "npm rum vitest", + "npm run vitest run", + "npm run vitest", + "npm run-script vitest run", + "npm run-script vitest", + "npm urn vitest run", + "npm urn vitest", + "npm vitest run", + "npm vitest", + "npm x vitest run", + "npm x vitest", + "npx vitest run", + "npx vitest", + "pnpm dlx vitest run", + "pnpm dlx vitest", + "pnpm exec vitest run", + "pnpm exec vitest", + "pnpm run vitest run", + "pnpm run vitest", + "pnpm run-script vitest run", + "pnpm run-script vitest", + "pnpm vitest run", + "pnpm vitest", + "pnpx vitest run", + "pnpx vitest", + "vitest run", + "vitest", + ]; + for command in commands { + assert_eq!( + rewrite_command(command, &[]), + Some("rtk vitest".into()), + "Failed for command: {}", + command + ); + } + } + + #[test] + fn test_classify_prisma() { + let commands = vec![ + "npm exec prisma", + "npm rum prisma", + "npm run prisma", + "npm run-script prisma", + "npm urn prisma", + "npm x prisma", + "pnpm dlx prisma", + "pnpm exec prisma", + "pnpm run prisma", + "pnpm run-script prisma", + "npm prisma", + "npx prisma", + "pnpm prisma", + "pnpx prisma", + "prisma", + ]; + for command in commands { + assert!( + matches!( + classify_command(format!("{command} migrate dev").as_str()), + Classification::Supported { + rtk_equivalent: "rtk prisma", + .. + } + ), + "Failed for command: {}", + command + ); + } + } + + #[test] + fn test_rewrite_prisma() { + let commands = vec![ + "npm exec prisma", + "npm rum prisma", + "npm run prisma", + "npm run-script prisma", + "npm urn prisma", + "npm x prisma", + "pnpm dlx prisma", + "pnpm exec prisma", + "pnpm run prisma", + "pnpm run-script prisma", + "npm prisma", + "npx prisma", + "pnpm prisma", + "pnpx prisma", + "prisma", + ]; + for command in commands { + assert_eq!( + rewrite_command(format!("{command} migrate dev").as_str(), &[]), + Some("rtk prisma migrate dev".into()), + "Failed for command: {}", + command + ); + } + } + #[test] fn test_rewrite_prettier() { + let commands = vec![ + "npm exec prettier", + "npm rum prettier", + "npm run prettier", + "npm run-script prettier", + "npm urn prettier", + "npm x prettier", + "pnpm dlx prettier", + "pnpm exec prettier", + "pnpm run prettier", + "pnpm run-script prettier", + "npm prettier", + "npx prettier", + "pnpm prettier", + "pnpx prettier", + "prettier", + ]; + for command in commands { + assert_eq!( + rewrite_command(format!("{command} --check src/").as_str(), &[]), + Some("rtk prettier --check src/".into()), + "Failed for command: {}", + command + ); + } + } + + #[test] + fn test_rewrite_pnpm_command() { + let commands = vec![ + "exec", + "i", + "install", + "list", + "ls", + "outdated", + "run", + "run-script", + ]; + for command in commands { + assert_eq!( + rewrite_command(format!("pnpm {command}").as_str(), &[]), + Some(format!("rtk pnpm {command}")), + "Failed for command: pnpm {}", + command + ); + } + } + + #[test] + fn test_rewrite_npm_bare_subcommand() { + let commands = vec!["exec", "run", "run-script", "x"]; + for command in commands { + assert_eq!( + rewrite_command(format!("npm {command}").as_str(), &[]), + Some(format!("rtk npm {command}")), + "Failed for bare command: npm {}", + command + ); + } + } + + #[test] + fn test_rewrite_npm_with_args() { assert_eq!( - rewrite_command("npx prettier --check src/", &[]), - Some("rtk prettier --check src/".into()) + rewrite_command("npm run test", &[]), + Some("rtk npm run test".to_string()), + ); + assert_eq!( + rewrite_command("npm exec vitest", &[]), + Some("rtk vitest".to_string()), ); } #[test] - fn test_rewrite_pnpm_list() { + fn test_rewrite_npx() { assert_eq!( - rewrite_command("pnpm list", &[]), - Some("rtk pnpm list".into()) + rewrite_command("npx svgo", &[]), + Some("rtk npx svgo".to_string()), ); } - // --- Compound operator edge cases --- #[test] @@ -2103,25 +2928,14 @@ mod tests { ); } - // --- Ensure PATTERNS and RULES stay aligned after modifications --- - - #[test] - fn test_patterns_rules_aligned_after_aws_psql() { - // If this fails, someone added a PATTERN without a matching RULE (or vice versa) - assert_eq!( - PATTERNS.len(), - RULES.len(), - "PATTERNS[{}] != RULES[{}] — they must stay 1:1", - PATTERNS.len(), - RULES.len() - ); - } - - // --- All RULES have non-empty rtk_cmd and at least one rewrite_prefix --- - #[test] - fn test_all_rules_have_valid_rtk_cmd() { + fn test_all_rules_are_complete() { for rule in RULES { + assert!( + !rule.pattern.is_empty(), + "Rule '{}' has empty pattern", + rule.rtk_cmd + ); assert!(!rule.rtk_cmd.is_empty(), "Rule with empty rtk_cmd found"); assert!( rule.rtk_cmd.starts_with("rtk "), @@ -2172,15 +2986,66 @@ mod tests { ); } - // --- Every PATTERN compiles to a valid Regex --- + #[test] + fn test_exclude_env_prefixed_command() { + let excluded = vec!["psql".to_string()]; + assert_eq!( + rewrite_command("PGPASSWORD=postgres psql -h localhost", &excluded), + None + ); + } + + #[test] + fn test_exclude_subcommand_pattern() { + let excluded = vec!["git push".to_string()]; + assert_eq!(rewrite_command("git push origin main", &excluded), None); + } + + #[test] + fn test_exclude_regex_pattern() { + let excluded = vec!["^curl".to_string()]; + assert_eq!(rewrite_command("curl http://example.com", &excluded), None); + } + + #[test] + fn test_exclude_invalid_regex_fallback() { + let excluded = vec!["curl[".to_string()]; + assert!(rewrite_command("curl http://example.com", &excluded).is_some()); + } + + #[test] + fn test_exclude_does_not_substring_match() { + let excluded = vec!["go".to_string()]; + assert!(rewrite_command("golangci-lint run ./...", &excluded).is_some()); + } + + #[test] + fn test_exclude_does_not_match_hyphenated_command() { + let excluded = vec!["golangci".to_string()]; + assert!(rewrite_command("golangci-lint run ./...", &excluded).is_some()); + } + + #[test] + fn test_exclude_empty_pattern_ignored() { + let excluded = vec!["".to_string()]; + assert!(rewrite_command("git status", &excluded).is_some()); + } + + #[test] + fn test_exclude_bare_anchor_ignored() { + let excluded = vec!["^".to_string()]; + assert!(rewrite_command("git status", &excluded).is_some()); + } #[test] fn test_all_patterns_are_valid_regex() { use regex::Regex; - for (i, pattern) in PATTERNS.iter().enumerate() { + for (i, rule) in RULES.iter().enumerate() { assert!( - Regex::new(pattern).is_ok(), - "PATTERNS[{i}] = '{pattern}' is not a valid regex" + Regex::new(rule.pattern).is_ok(), + "RULES[{i}] ({}) has invalid pattern '{}'", + rule.rtk_cmd, + rule.pattern ); } } @@ -2379,6 +3244,31 @@ mod tests { assert_eq!(strip_git_global_opts("cargo test"), "cargo test"); } + #[test] + fn test_strip_golangci_global_opts_helper() { + assert_eq!( + strip_golangci_global_opts("golangci-lint -v run ./..."), + "golangci-lint run ./..." + ); + assert_eq!( + strip_golangci_global_opts("golangci-lint --color never run ./..."), + "golangci-lint run ./..." + ); + assert_eq!( + strip_golangci_global_opts("golangci-lint --color=never run ./..."), + "golangci-lint run ./..." + ); + assert_eq!( + strip_golangci_global_opts("golangci-lint --config=foo.yml run ./..."), + "golangci-lint run ./..." + ); + assert_eq!( + strip_golangci_global_opts("golangci-lint version"), + "golangci-lint version" + ); + assert_eq!(strip_golangci_global_opts("cargo test"), "cargo test"); + } + // --- #wc: wc filter was silently ignored by the hook --- #[test] @@ -2424,4 +3314,217 @@ mod tests { Some("rtk wc src/*.rs".into()) ); } + + #[test] + fn test_classify_command_substitution_passthrough() { + assert_eq!( + classify_command("git log $(git rev-parse HEAD~1)"), + Classification::Supported { + rtk_equivalent: "rtk git", + category: "Git", + estimated_savings_pct: 70.0, + status: RtkStatus::Existing, + } + ); + } + + #[test] + fn test_rewrite_command_substitution_passthrough() { + assert_eq!( + rewrite_command("git log $(git rev-parse HEAD~1)", &[]), + Some("rtk git log $(git rev-parse HEAD~1)".into()) + ); + } + + #[test] + fn test_split_command_substitution_no_split() { + assert_eq!( + split_command_chain("git log $(git rev-parse HEAD~1)"), + vec!["git log $(git rev-parse HEAD~1)"] + ); + } + + #[test] + fn test_shell_prefix_noglob() { + assert_eq!( + rewrite_command("noglob git status", &[]), + Some("noglob rtk git status".into()) + ); + } + + #[test] + fn test_shell_prefix_command() { + assert_eq!( + rewrite_command("command git status", &[]), + Some("command rtk git status".into()) + ); + } + + #[test] + fn test_shell_prefix_builtin_exec_nocorrect() { + assert_eq!( + rewrite_command("builtin git status", &[]), + Some("builtin rtk git status".into()) + ); + assert_eq!( + rewrite_command("exec git status", &[]), + Some("exec rtk git status".into()) + ); + assert_eq!( + rewrite_command("nocorrect git status", &[]), + Some("nocorrect rtk git status".into()) + ); + } + + #[test] + fn test_shell_prefix_unknown_inner() { + assert_eq!(rewrite_command("noglob unknown_cmd --flag", &[]), None); + } + + #[test] + fn test_python3_m_pytest() { + assert_eq!( + rewrite_command("python3 -m pytest tests/", &[]), + Some("rtk pytest tests/".into()) + ); + } + + #[test] + fn test_pip_show() { + assert_eq!( + rewrite_command("pip show flask", &[]), + Some("rtk pip show flask".into()) + ); + } + + #[test] + fn test_gt_graphite() { + assert_eq!(rewrite_command("gt log", &[]), Some("rtk gt log".into())); + } + + #[test] + fn test_command_no_longer_ignored() { + assert_ne!( + classify_command("command git status"), + Classification::Ignored + ); + } + + // --- Pipe + operator rewrite --- + + #[test] + fn test_rewrite_pipe_then_and() { + assert_eq!( + rewrite_command("git log | head -5 && git stash", &[]), + Some("rtk git log | head -5 && rtk git stash".into()) + ); + } + + #[test] + fn test_rewrite_pipe_then_semicolon() { + assert_eq!( + rewrite_command("cargo test | head; git status", &[]), + Some("rtk cargo test | head; rtk git status".into()) + ); + } + + #[test] + fn test_rewrite_pipe_then_or() { + assert_eq!( + rewrite_command("cargo test | grep FAIL || git stash", &[]), + Some("rtk cargo test | grep FAIL || rtk git stash".into()) + ); + } + + #[test] + fn test_rewrite_env_pipe_then_and() { + assert_eq!( + rewrite_command( + "RUST_BACKTRACE=1 cargo test 2>&1 | grep FAILED && git stash", + &[] + ), + Some("RUST_BACKTRACE=1 rtk cargo test 2>&1 | grep FAILED && rtk git stash".into()) + ); + } + + #[test] + fn test_rewrite_and_then_pipe() { + assert_eq!( + rewrite_command("git status && cargo test | grep FAIL", &[]), + Some("rtk git status && rtk cargo test | grep FAIL".into()) + ); + } + + #[test] + fn test_rewrite_multi_pipe_then_and() { + assert_eq!( + rewrite_command("git log | head | tail && git status", &[]), + Some("rtk git log | head | tail && rtk git status".into()) + ); + } + + // Pipe-incompatible commands: curl/wget must not be rewritten when piped + + // Shell function definitions must not be rewritten + + #[test] + fn test_rewrite_skip_shell_function_definition() { + assert_eq!( + rewrite_command("create_link() { curl -s https://api.example.com; }", &[]), + None + ); + } + + #[test] + fn test_rewrite_skip_multiline_function_definition() { + let cmd = "create_link() {\n curl -s https://api.example.com\n}"; + assert_eq!(rewrite_command(cmd, &[]), None); + } + + #[test] + fn test_rewrite_skip_function_keyword() { + assert_eq!( + rewrite_command("function fetch_data { curl -s https://example.com; }", &[]), + None + ); + } + + #[test] + fn test_rewrite_skip_function_with_invocation() { + let cmd = r#"create_link() { curl -s "https://api.short.io/links" | python3 -c "import sys,json; print(json.load(sys.stdin))"; }; create_link "https://example.com""#; + assert_eq!(rewrite_command(cmd, &[]), None); + } + + #[test] + fn test_rewrite_curl_pipe_skipped() { + assert_eq!( + rewrite_command("curl -s https://api.example.com | jq .name", &[]), + None + ); + } + + #[test] + fn test_rewrite_curl_no_pipe_still_rewritten() { + assert_eq!( + rewrite_command("curl -s https://api.example.com", &[]), + Some("rtk curl -s https://api.example.com".into()) + ); + } + + #[test] + fn test_rewrite_wget_pipe_skipped() { + assert_eq!( + rewrite_command("wget -qO- https://example.com | grep title", &[]), + None + ); + } + + #[test] + fn test_rewrite_curl_compound_and_pipe() { + // curl in && chain: rewrite. curl before pipe: don't rewrite. + assert_eq!( + rewrite_command("git status && curl -s https://api.example.com | jq .", &[]), + Some("rtk git status && curl -s https://api.example.com | jq .".into()) + ); + } } diff --git a/src/discover/report.rs b/src/discover/report.rs index 1fffa3278..128ecb45e 100644 --- a/src/discover/report.rs +++ b/src/discover/report.rs @@ -1,5 +1,6 @@ //! Data types for reporting which commands RTK can and cannot optimize. +use crate::hooks::constants::{HOOKS_SUBDIR, REWRITE_HOOK_FILE}; use serde::Serialize; /// RTK support status for a command. @@ -82,12 +83,12 @@ pub fn format_text(report: &DiscoverReport, limit: usize, verbose: bool) -> Stri report.sessions_scanned, report.since_days, report.total_commands )); out.push_str(&format!( - "Already using RTK: {} commands ({}%)\n", + "Already using RTK: {} commands ({:.1}%)\n", report.already_rtk, if report.total_commands > 0 { - report.already_rtk * 100 / report.total_commands + report.already_rtk as f64 * 100.0 / report.total_commands as f64 } else { - 0 + 0.0 } )); @@ -169,7 +170,10 @@ pub fn format_text(report: &DiscoverReport, limit: usize, verbose: bool) -> Stri // Cursor note: check if Cursor hooks are installed if let Some(home) = dirs::home_dir() { - let cursor_hook = home.join(".cursor").join("hooks").join("rtk-rewrite.sh"); + let cursor_hook = home + .join(".cursor") + .join(HOOKS_SUBDIR) + .join(REWRITE_HOOK_FILE); if cursor_hook.exists() { out.push_str("\nNote: Cursor sessions are tracked via `rtk gain` (discover scans Claude Code only)\n"); } @@ -210,3 +214,57 @@ fn truncate_str(s: &str, max: usize) -> String { format!("{}..", truncated) } } + +#[cfg(test)] +mod tests { + use super::*; + + fn make_report(total_commands: usize, already_rtk: usize) -> DiscoverReport { + DiscoverReport { + sessions_scanned: 1, + total_commands, + already_rtk, + since_days: 30, + supported: vec![], + unsupported: vec![], + parse_errors: 0, + rtk_disabled_count: 0, + rtk_disabled_examples: vec![], + } + } + + // B6 regression: integer division truncated small percentages to 0%. + // Example: 3/1000 = 0% (old bug), should be "0.3%". + #[test] + fn test_already_rtk_percent_shows_decimal() { + let report = make_report(1000, 3); + let output = format_text(&report, 10, false); + // "0.3%" must appear; old code would print "0%" + assert!( + output.contains("0.3%"), + "Expected '0.3%' in output but got:\n{}", + output + ); + assert!( + !output.contains("(0%)"), + "Output must not contain '(0%)' — integer division bug still present:\n{}", + output + ); + } + + // Edge case: 0/0 must not divide-by-zero. + #[test] + fn test_already_rtk_percent_zero_total() { + let report = make_report(0, 0); + let output = format_text(&report, 10, false); + assert!(output.contains("0 commands (0.0%)")); + } + + // Full percent: 1000/1000 = 100.0% + #[test] + fn test_already_rtk_percent_full() { + let report = make_report(1000, 1000); + let output = format_text(&report, 10, false); + assert!(output.contains("100.0%")); + } +} diff --git a/src/discover/rules.rs b/src/discover/rules.rs index 98fe288d2..74c876a1e 100644 --- a/src/discover/rules.rs +++ b/src/discover/rules.rs @@ -1,11 +1,8 @@ -//! The master list of shell commands RTK knows how to rewrite. - use super::report::RtkStatus; -/// A rule mapping a shell command pattern to its RTK equivalent. pub struct RtkRule { + pub pattern: &'static str, pub rtk_cmd: &'static str, - /// Original command prefixes to replace with rtk_cmd (longest first for correct matching). pub rewrite_prefixes: &'static [&'static str], pub category: &'static str, pub savings_pct: f64, @@ -13,88 +10,11 @@ pub struct RtkRule { pub subcmd_status: &'static [(&'static str, RtkStatus)], } -// Patterns ordered to match RULES indices exactly. -pub const PATTERNS: &[&str] = &[ - r"^git\s+(?:-[Cc]\s+\S+\s+)*(status|log|diff|show|add|commit|push|pull|branch|fetch|stash|worktree)", - r"^gh\s+(pr|issue|run|repo|api|release)", - r"^cargo\s+(build|test|clippy|check|fmt|install)", - r"^pnpm\s+(list|ls|outdated|install)", - r"^npm\s+(run|exec)", - r"^npx\s+", - r"^(cat|head|tail)\s+", - r"^(rg|grep)\s+", - r"^ls(\s|$)", - r"^find\s+", - r"^(npx\s+|pnpm\s+)?tsc(\s|$)", - r"^(npx\s+|pnpm\s+)?(eslint|biome|lint)(\s|$)", - r"^(npx\s+|pnpm\s+)?prettier", - r"^(npx\s+|pnpm\s+)?next\s+build", - r"^(pnpm\s+|npx\s+)?(vitest|jest|test)(\s|$)", - r"^(npx\s+|pnpm\s+)?playwright", - r"^(npx\s+|pnpm\s+)?prisma", - r"^docker\s+(ps|images|logs|run|exec|build|compose\s+(ps|logs|build))", - r"^kubectl\s+(get|logs|describe|apply)", - r"^tree(\s|$)", - r"^diff\s+", - r"^curl\s+", - r"^wget\s+", - r"^(python3?\s+-m\s+)?mypy(\s|$)", - // Python tooling - r"^ruff\s+(check|format)", - r"^(python\s+-m\s+)?pytest(\s|$)", - r"^(pip3?|uv\s+pip)\s+(list|outdated|install)", - // Go tooling - r"^go\s+(test|build|vet)", - r"^golangci-lint(\s|$)", - // Ruby tooling - r"^bundle\s+(install|update)\b", - r"^(?:bundle\s+exec\s+)?(?:bin/)?(?:rake|rails)\s+test", - r"^(?:bundle\s+exec\s+)?rspec(?:\s|$)", - r"^(?:bundle\s+exec\s+)?rubocop(?:\s|$)", - // AWS CLI - r"^aws\s+", - // PostgreSQL - r"^psql(\s|$)", - // TOML-filtered commands - r"^ansible-playbook\b", - r"^brew\s+(install|upgrade)\b", - r"^composer\s+(install|update|require)\b", - r"^df(\s|$)", - r"^dotnet\s+build\b", - r"^du\b", - r"^fail2ban-client\b", - r"^gcloud\b", - r"^hadolint\b", - r"^helm\b", - r"^iptables\b", - r"^make\b", - r"^markdownlint\b", - r"^mix\s+(compile|format)(\s|$)", - r"^mvn\s+(compile|package|clean|install)\b", - r"^ping\b", - r"^pio\s+run", - r"^poetry\s+(install|lock|update)\b", - r"^pre-commit\b", - r"^ps(\s|$)", - r"^quarto\s+render", - r"^rsync\b", - r"^shellcheck\b", - r"^shopify\s+theme\s+(push|pull)", - r"^sops\b", - r"^swift\s+(build|test)\b", - r"^systemctl\s+status\b", - r"^terraform\s+plan", - r"^tofu\s+(fmt|init|plan|validate)(\s|$)", - r"^trunk\s+build", - r"^uv\s+(sync|pip\s+install)\b", - r"^yamllint\b", - r"^wc(\s|$)", -]; - pub const RULES: &[RtkRule] = &[ RtkRule { + pattern: r"^(?:git|yadm)\s+(?:-[Cc]\s+\S+\s+)*(status|log|diff|show|add|commit|push|pull|branch|fetch|stash|worktree)", rtk_cmd: "rtk git", - rewrite_prefixes: &["git"], + rewrite_prefixes: &["git", "yadm"], category: "Git", savings_pct: 70.0, subcmd_savings: &[ @@ -106,6 +26,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^gh\s+(pr|issue|run|repo|api|release)", rtk_cmd: "rtk gh", rewrite_prefixes: &["gh"], category: "GitHub", @@ -114,6 +35,16 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^glab\s+(mr|issue|ci|pipeline|api|release)", + rtk_cmd: "rtk glab", + rewrite_prefixes: &["glab"], + category: "GitLab", + savings_pct: 82.0, + subcmd_savings: &[("mr", 87.0), ("ci", 82.0), ("issue", 80.0)], + subcmd_status: &[], + }, + RtkRule { + pattern: r"^cargo\s+(build|test|clippy|check|fmt|install)", rtk_cmd: "rtk cargo", rewrite_prefixes: &["cargo"], category: "Cargo", @@ -122,6 +53,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[("fmt", RtkStatus::Passthrough)], }, RtkRule { + pattern: r"^pnpm\s+(exec|i|install|list|ls|outdated|run|run-script)", rtk_cmd: "rtk pnpm", rewrite_prefixes: &["pnpm"], category: "PackageManager", @@ -130,6 +62,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^npm\s+(exec|run|run-script|rum|urn|x)(\s|$)", rtk_cmd: "rtk npm", rewrite_prefixes: &["npm"], category: "PackageManager", @@ -138,6 +71,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^npx\s+", rtk_cmd: "rtk npx", rewrite_prefixes: &["npx"], category: "PackageManager", @@ -146,6 +80,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^(cat|head|tail)\s+", rtk_cmd: "rtk read", rewrite_prefixes: &["cat", "head", "tail"], category: "Files", @@ -154,6 +89,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^(rg|grep)\s+", rtk_cmd: "rtk grep", rewrite_prefixes: &["rg", "grep"], category: "Files", @@ -162,6 +98,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^ls(\s|$)", rtk_cmd: "rtk ls", rewrite_prefixes: &["ls"], category: "Files", @@ -170,6 +107,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^find\s+", rtk_cmd: "rtk find", rewrite_prefixes: &["find"], category: "Files", @@ -178,23 +116,75 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { - // Longest prefixes first for correct matching + pattern: r"^((p?np(m|x)|p?npm\s+(exec|run|run-script)|npm\s+(rum|urn|x)|pnpm\s+dlx)\s+)?tsc(\s|$)", rtk_cmd: "rtk tsc", - rewrite_prefixes: &["pnpm tsc", "npx tsc", "tsc"], + rewrite_prefixes: &[ + "npm exec tsc", + "npm rum tsc", + "npm run tsc", + "npm run-script tsc", + "npm tsc", + "npm urn tsc", + "npm x tsc", + "npx tsc", + "pnpm dlx tsc", + "pnpm exec tsc", + "pnpm run tsc", + "pnpm run-script tsc", + "pnpm tsc", + "pnpx tsc", + "tsc", + ], category: "Build", savings_pct: 83.0, subcmd_savings: &[], subcmd_status: &[], }, RtkRule { + pattern: r"^((p?np(m|x)|p?npm\s+(exec|run|run-script)|npm\s+(rum|urn|x)|pnpm\s+dlx)\s+)?(biome|eslint|lint)(\s|$)", rtk_cmd: "rtk lint", rewrite_prefixes: &[ - "npx eslint", - "pnpm lint", - "npx biome", - "eslint", "biome", + "eslint", "lint", + "npm biome", + "npm eslint", + "npm exec biome", + "npm exec eslint", + "npm lint", + "npm rum biome", + "npm rum eslint", + "npm rum lint", + "npm run biome", + "npm run eslint", + "npm run lint", + "npm run-script biome", + "npm run-script eslint", + "npm run-script lint", + "npm urn biome", + "npm urn eslint", + "npm urn lint", + "npm x biome", + "npm x eslint", + "npx biome", + "npx eslint", + "npx lint", + "pnpm biome", + "pnpm dlx biome", + "pnpm dlx eslint", + "pnpm eslint", + "pnpm exec biome", + "pnpm exec eslint", + "pnpm lint", + "pnpm run biome", + "pnpm run eslint", + "pnpm run lint", + "pnpm run-script biome", + "pnpm run-script eslint", + "pnpm run-script lint", + "pnpx biome", + "pnpx eslint", + "pnpx lint", ], category: "Build", savings_pct: 84.0, @@ -202,47 +192,187 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^((p?np(m|x)|p?npm\s+(exec|run|run-script)|npm\s+(rum|urn|x)|pnpm\s+dlx)\s+)?prettier", rtk_cmd: "rtk prettier", - rewrite_prefixes: &["npx prettier", "pnpm prettier", "prettier"], + rewrite_prefixes: &[ + "npm exec prettier", + "npm prettier", + "npm rum prettier", + "npm run prettier", + "npm run-script prettier", + "npm urn prettier", + "npm x prettier", + "npx prettier", + "pnpm dlx prettier", + "pnpm exec prettier", + "pnpm prettier", + "pnpm run prettier", + "pnpm run-script prettier", + "pnpx prettier", + "prettier", + ], category: "Build", savings_pct: 70.0, subcmd_savings: &[], subcmd_status: &[], }, RtkRule { - // "next build" is stripped to "rtk next" — the build subcommand is internal + pattern: r"^((p?np(m|x)|p?npm\s+(exec|run|run-script)|npm\s+(rum|urn|x)|pnpm\s+dlx)\s+)?next\s+build", rtk_cmd: "rtk next", - rewrite_prefixes: &["npx next build", "pnpm next build", "next build"], + rewrite_prefixes: &[ + "next build", + "npm exec next build", + "npm next build", + "npm rum next build", + "npm run next build", + "npm run-script next build", + "npm urn next build", + "npm x next build", + "npx next build", + "pnpm dlx next build", + "pnpm exec next build", + "pnpm next build", + "pnpm run next build", + "pnpm run-script next build", + "pnpx next build", + ], category: "Build", savings_pct: 87.0, subcmd_savings: &[], subcmd_status: &[], }, RtkRule { + pattern: r"^((p?np(m|x)|p?npm\s+(exec|run|run-script)|npm\s+(rum|urn|x)|pnpm\s+dlx)\s+)?jest(\s+run)?(\s|$)", + rtk_cmd: "rtk jest", + rewrite_prefixes: &[ + "jest run", + "jest", + "npm exec jest run", + "npm exec jest", + "npm jest run", + "npm jest", + "npm rum jest run", + "npm rum jest", + "npm run jest run", + "npm run jest", + "npm run-script jest run", + "npm run-script jest", + "npm urn jest run", + "npm urn jest", + "npm x jest run", + "npm x jest", + "npx jest run", + "npx jest", + "pnpm dlx jest run", + "pnpm dlx jest", + "pnpm exec jest run", + "pnpm exec jest", + "pnpm jest run", + "pnpm jest", + "pnpm run jest run", + "pnpm run jest", + "pnpm run-script jest run", + "pnpm run-script jest", + "pnpx jest run", + "pnpx jest", + ], + category: "Tests", + savings_pct: 99.0, + subcmd_savings: &[], + subcmd_status: &[], + }, + RtkRule { + pattern: r"^((p?np(m|x)|p?npm\s+(exec|run|run-script)|npm\s+(rum|urn|x)|pnpm\s+dlx)\s+)?vitest(\s+run)?(\s|$)", rtk_cmd: "rtk vitest", - rewrite_prefixes: &["pnpm vitest", "npx vitest", "vitest", "jest"], + rewrite_prefixes: &[ + "npm exec vitest run", + "npm exec vitest", + "npm rum vitest run", + "npm rum vitest", + "npm run vitest run", + "npm run vitest", + "npm run-script vitest run", + "npm run-script vitest", + "npm urn vitest run", + "npm urn vitest", + "npm vitest run", + "npm vitest", + "npm x vitest run", + "npm x vitest", + "npx vitest run", + "npx vitest", + "pnpm dlx vitest run", + "pnpm dlx vitest", + "pnpm exec vitest run", + "pnpm exec vitest", + "pnpm run vitest run", + "pnpm run vitest", + "pnpm run-script vitest run", + "pnpm run-script vitest", + "pnpm vitest run", + "pnpm vitest", + "pnpx vitest run", + "pnpx vitest", + "vitest run", + "vitest", + ], category: "Tests", savings_pct: 99.0, subcmd_savings: &[], subcmd_status: &[], }, RtkRule { + pattern: r"^((p?np(m|x)|p?npm\s+(exec|run|run-script)|npm\s+(rum|urn|x)|pnpm\s+dlx)\s+)?playwright", rtk_cmd: "rtk playwright", - rewrite_prefixes: &["npx playwright", "pnpm playwright", "playwright"], + rewrite_prefixes: &[ + "npm exec playwright", + "npm playwright", + "npm rum playwright", + "npm run playwright", + "npm run-script playwright", + "npm urn playwright", + "npm x playwright", + "npx playwright", + "playwright", + "pnpm dlx playwright", + "pnpm exec playwright", + "pnpm playwright", + "pnpm run playwright", + "pnpm run-script playwright", + "pnpx playwright", + ], category: "Tests", savings_pct: 94.0, subcmd_savings: &[], subcmd_status: &[], }, RtkRule { + pattern: r"^((p?np(m|x)|p?npm\s+(exec|run|run-script)|npm\s+(rum|urn|x)|pnpm\s+dlx)\s+)?prisma", rtk_cmd: "rtk prisma", - rewrite_prefixes: &["npx prisma", "pnpm prisma", "prisma"], + rewrite_prefixes: &[ + "npm exec prisma", + "npm prisma", + "npm rum prisma", + "npm run prisma", + "npm run-script prisma", + "npm urn prisma", + "npm x prisma", + "npx prisma", + "pnpm dlx prisma", + "pnpm exec prisma", + "pnpm prisma", + "pnpm run prisma", + "pnpm run-script prisma", + "pnpx prisma", + "prisma", + ], category: "Build", savings_pct: 88.0, subcmd_savings: &[], subcmd_status: &[], }, RtkRule { + pattern: r"^docker\s+(ps|images|logs|run|exec|build|compose\s+(ps|logs|build))", rtk_cmd: "rtk docker", rewrite_prefixes: &["docker"], category: "Infra", @@ -251,6 +381,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^kubectl\s+(get|logs|describe|apply)", rtk_cmd: "rtk kubectl", rewrite_prefixes: &["kubectl"], category: "Infra", @@ -259,6 +390,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^tree(\s|$)", rtk_cmd: "rtk tree", rewrite_prefixes: &["tree"], category: "Files", @@ -267,6 +399,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^diff\s+", rtk_cmd: "rtk diff", rewrite_prefixes: &["diff"], category: "Files", @@ -275,6 +408,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^curl\s+", rtk_cmd: "rtk curl", rewrite_prefixes: &["curl"], category: "Network", @@ -283,6 +417,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^wget\s+", rtk_cmd: "rtk wget", rewrite_prefixes: &["wget"], category: "Network", @@ -291,6 +426,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^(python3?\s+-m\s+)?mypy(\s|$)", rtk_cmd: "rtk mypy", rewrite_prefixes: &["python3 -m mypy", "python -m mypy", "mypy"], category: "Build", @@ -298,8 +434,8 @@ pub const RULES: &[RtkRule] = &[ subcmd_savings: &[], subcmd_status: &[], }, - // Python tooling RtkRule { + pattern: r"^ruff\s+(check|format)", rtk_cmd: "rtk ruff", rewrite_prefixes: &["ruff"], category: "Python", @@ -308,14 +444,16 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^(python[0-9.]*\s+-m\s+)?pytest(\s|$)", rtk_cmd: "rtk pytest", - rewrite_prefixes: &["python -m pytest", "pytest"], + rewrite_prefixes: &["python3 -m pytest", "python -m pytest", "pytest"], category: "Python", savings_pct: 90.0, subcmd_savings: &[], subcmd_status: &[], }, RtkRule { + pattern: r"^(pip3?|uv\s+pip)\s+(list|outdated|install|show)", rtk_cmd: "rtk pip", rewrite_prefixes: &["pip3", "pip", "uv pip"], category: "Python", @@ -323,8 +461,8 @@ pub const RULES: &[RtkRule] = &[ subcmd_savings: &[("list", 75.0), ("outdated", 80.0)], subcmd_status: &[], }, - // Go tooling RtkRule { + pattern: r"^go\s+(test|build|vet)", rtk_cmd: "rtk go", rewrite_prefixes: &["go"], category: "Go", @@ -333,15 +471,16 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { - rtk_cmd: "rtk golangci-lint", - rewrite_prefixes: &["golangci-lint", "golangci"], + pattern: r"^(?:golangci-lint|golangci)\s+(run)(?:\s|$)", + rtk_cmd: "rtk golangci-lint run", + rewrite_prefixes: &["golangci-lint run", "golangci run"], category: "Go", savings_pct: 85.0, subcmd_savings: &[], subcmd_status: &[], }, - // Ruby tooling RtkRule { + pattern: r"^bundle\s+(install|update)\b", rtk_cmd: "rtk bundle", rewrite_prefixes: &["bundle"], category: "Ruby", @@ -350,6 +489,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^(?:bundle\s+exec\s+)?(?:bin/)?(?:rake|rails)\s+test", rtk_cmd: "rtk rake", rewrite_prefixes: &[ "bundle exec rails", @@ -364,6 +504,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^(?:bundle\s+exec\s+)?rspec(?:\s|$)", rtk_cmd: "rtk rspec", rewrite_prefixes: &["bundle exec rspec", "bin/rspec", "rspec"], category: "Tests", @@ -372,6 +513,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^(?:bundle\s+exec\s+)?rubocop(?:\s|$)", rtk_cmd: "rtk rubocop", rewrite_prefixes: &["bundle exec rubocop", "rubocop"], category: "Build", @@ -379,17 +521,32 @@ pub const RULES: &[RtkRule] = &[ subcmd_savings: &[], subcmd_status: &[], }, - // AWS CLI RtkRule { + pattern: r"^aws\s+", rtk_cmd: "rtk aws", rewrite_prefixes: &["aws"], category: "Infra", savings_pct: 80.0, - subcmd_savings: &[], + subcmd_savings: &[ + ("sts", 80.0), + ("s3", 60.0), + ("ec2", 85.0), + ("ecs", 90.0), + ("rds", 80.0), + ("cloudformation", 90.0), + ("logs", 88.0), + ("lambda", 90.0), + ("iam", 85.0), + ("dynamodb", 70.0), + ("s3api", 75.0), + ("eks", 87.0), + ("sqs", 78.0), + ("secretsmanager", 75.0), + ], subcmd_status: &[], }, - // PostgreSQL RtkRule { + pattern: r"^psql(\s|$)", rtk_cmd: "rtk psql", rewrite_prefixes: &["psql"], category: "Infra", @@ -397,8 +554,8 @@ pub const RULES: &[RtkRule] = &[ subcmd_savings: &[], subcmd_status: &[], }, - // TOML-filtered commands RtkRule { + pattern: r"^ansible-playbook\b", rtk_cmd: "rtk ansible-playbook", rewrite_prefixes: &["ansible-playbook"], category: "Infra", @@ -407,6 +564,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^brew\s+(install|upgrade)\b", rtk_cmd: "rtk brew", rewrite_prefixes: &["brew"], category: "PackageManager", @@ -415,6 +573,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^composer\s+(install|update|require)\b", rtk_cmd: "rtk composer", rewrite_prefixes: &["composer"], category: "PackageManager", @@ -423,6 +582,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^df(\s|$)", rtk_cmd: "rtk df", rewrite_prefixes: &["df"], category: "System", @@ -431,6 +591,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^dotnet\s+build\b", rtk_cmd: "rtk dotnet", rewrite_prefixes: &["dotnet"], category: "Build", @@ -439,6 +600,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^du\b", rtk_cmd: "rtk du", rewrite_prefixes: &["du"], category: "System", @@ -447,6 +609,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^fail2ban-client\b", rtk_cmd: "rtk fail2ban-client", rewrite_prefixes: &["fail2ban-client"], category: "Infra", @@ -455,6 +618,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^gcloud\b", rtk_cmd: "rtk gcloud", rewrite_prefixes: &["gcloud"], category: "Infra", @@ -463,6 +627,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^hadolint\b", rtk_cmd: "rtk hadolint", rewrite_prefixes: &["hadolint"], category: "Build", @@ -471,6 +636,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^helm\b", rtk_cmd: "rtk helm", rewrite_prefixes: &["helm"], category: "Infra", @@ -479,6 +645,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^iptables\b", rtk_cmd: "rtk iptables", rewrite_prefixes: &["iptables"], category: "Infra", @@ -487,6 +654,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^make\b", rtk_cmd: "rtk make", rewrite_prefixes: &["make"], category: "Build", @@ -495,6 +663,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^markdownlint\b", rtk_cmd: "rtk markdownlint", rewrite_prefixes: &["markdownlint"], category: "Build", @@ -503,6 +672,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^mix\s+(compile|format)(\s|$)", rtk_cmd: "rtk mix", rewrite_prefixes: &["mix"], category: "Build", @@ -511,6 +681,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^mvn\s+(compile|package|clean|install)\b", rtk_cmd: "rtk mvn", rewrite_prefixes: &["mvn"], category: "Build", @@ -519,6 +690,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^ping\b", rtk_cmd: "rtk ping", rewrite_prefixes: &["ping"], category: "Network", @@ -527,6 +699,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^pio\s+run", rtk_cmd: "rtk pio", rewrite_prefixes: &["pio"], category: "Build", @@ -535,6 +708,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^poetry\s+(install|lock|update)\b", rtk_cmd: "rtk poetry", rewrite_prefixes: &["poetry"], category: "Python", @@ -543,6 +717,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^pre-commit\b", rtk_cmd: "rtk pre-commit", rewrite_prefixes: &["pre-commit"], category: "Build", @@ -551,6 +726,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^ps(\s|$)", rtk_cmd: "rtk ps", rewrite_prefixes: &["ps"], category: "System", @@ -559,6 +735,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^quarto\s+render", rtk_cmd: "rtk quarto", rewrite_prefixes: &["quarto"], category: "Build", @@ -567,6 +744,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^rsync\b", rtk_cmd: "rtk rsync", rewrite_prefixes: &["rsync"], category: "Network", @@ -575,6 +753,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^shellcheck\b", rtk_cmd: "rtk shellcheck", rewrite_prefixes: &["shellcheck"], category: "Build", @@ -583,6 +762,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^shopify\s+theme\s+(push|pull)", rtk_cmd: "rtk shopify", rewrite_prefixes: &["shopify"], category: "Build", @@ -591,6 +771,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^sops\b", rtk_cmd: "rtk sops", rewrite_prefixes: &["sops"], category: "Infra", @@ -599,6 +780,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^swift\s+(build|test)\b", rtk_cmd: "rtk swift", rewrite_prefixes: &["swift"], category: "Build", @@ -607,6 +789,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^systemctl\s+status\b", rtk_cmd: "rtk systemctl", rewrite_prefixes: &["systemctl"], category: "System", @@ -615,6 +798,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^terraform\s+plan", rtk_cmd: "rtk terraform", rewrite_prefixes: &["terraform"], category: "Infra", @@ -623,6 +807,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^tofu\s+(fmt|init|plan|validate)(\s|$)", rtk_cmd: "rtk tofu", rewrite_prefixes: &["tofu"], category: "Infra", @@ -631,6 +816,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^trunk\s+build", rtk_cmd: "rtk trunk", rewrite_prefixes: &["trunk"], category: "Build", @@ -639,6 +825,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^uv\s+(sync|pip\s+install)\b", rtk_cmd: "rtk uv", rewrite_prefixes: &["uv"], category: "Python", @@ -647,6 +834,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^yamllint\b", rtk_cmd: "rtk yamllint", rewrite_prefixes: &["yamllint"], category: "Build", @@ -655,6 +843,7 @@ pub const RULES: &[RtkRule] = &[ subcmd_status: &[], }, RtkRule { + pattern: r"^wc(\s|$)", rtk_cmd: "rtk wc", rewrite_prefixes: &["wc"], category: "Files", @@ -662,9 +851,26 @@ pub const RULES: &[RtkRule] = &[ subcmd_savings: &[], subcmd_status: &[], }, + RtkRule { + pattern: r"^gt\s+", + rtk_cmd: "rtk gt", + rewrite_prefixes: &["gt"], + category: "Git", + savings_pct: 70.0, + subcmd_savings: &[], + subcmd_status: &[], + }, + RtkRule { + pattern: r"^liquibase(?:\s|$)", + rtk_cmd: "rtk liquibase", + rewrite_prefixes: &["liquibase"], + category: "Infra", + savings_pct: 65.0, + subcmd_savings: &[], + subcmd_status: &[], + }, ]; -/// Commands to ignore (shell builtins, trivial, already rtk). pub const IGNORED_PREFIXES: &[&str] = &[ "cd ", "cd\t", @@ -681,7 +887,6 @@ pub const IGNORED_PREFIXES: &[&str] = &[ "touch ", "which ", "type ", - "command ", "test ", "true", "false", diff --git a/src/filters/README.md b/src/filters/README.md index 1fabae7a7..226899cc2 100644 --- a/src/filters/README.md +++ b/src/filters/README.md @@ -1,6 +1,6 @@ # Built-in Filters -> See also [docs/TECHNICAL.md](../../docs/TECHNICAL.md) for the full architecture overview +> See also [docs/contributing/TECHNICAL.md](../../docs/contributing/TECHNICAL.md) for the full architecture overview Each `.toml` file in this directory defines one filter and its inline tests. Files are concatenated alphabetically by `build.rs` into a single TOML blob embedded in the binary. @@ -51,6 +51,7 @@ expected = "expected filtered output" | `description` | string | Human-readable description | | `match_command` | regex | Matches the command string (e.g. `"^docker\\s+inspect"`) | | `strip_ansi` | bool | Strip ANSI escape codes before processing | +| `filter_stderr` | bool | Capture and merge stderr into stdout before filtering (use for tools like liquibase that emit banners to stderr) | | `strip_lines_matching` | regex[] | Drop lines matching any regex | | `keep_lines_matching` | regex[] | Keep only lines matching at least one regex | | `replace` | array | Regex substitutions (`{ pattern, replacement }`) | @@ -64,3 +65,58 @@ expected = "expected filtered output" Use the command name as the filename: `terraform-plan.toml`, `docker-inspect.toml`, `mix-compile.toml`. For commands with subcommands, prefer `-.toml` over grouping multiple filters in one file. + +## Build and runtime pipeline + +How a `.toml` file goes from contributor → binary → filtered output. + +```mermaid +flowchart TD + A[["src/filters/my-tool.toml\n(new file)"]] --> B + + subgraph BUILD ["cargo build"] + B["build.rs\n1. ls src/filters/*.toml\n2. sort alphabetically\n3. concat → BUILTIN_TOML"] --> C + C{"TOML valid?\nDuplicate names?"} -->|"fail"| D[["Build fails\nerror points to bad file"]] + C -->|"ok"| E[["OUT_DIR/builtin_filters.toml\n(generated)"]] + E --> F["rustc embeds via include_str!"] + F --> G[["rtk binary\nBUILTIN_TOML embedded"]] + end + + subgraph TESTS ["cargo test"] + H["test_builtin_filter_count\nassert_eq!(filters.len(), N)"] -->|"wrong count"| I[["FAIL"]] + J["test_builtin_all_filters_present\nassert!(names.contains('my-tool'))"] -->|"name missing"| K[["FAIL"]] + L["test_builtin_all_filters_have_inline_tests\nassert!(tested.contains(name))"] -->|"no tests"| M[["FAIL"]] + end + + subgraph RUNTIME ["rtk my-tool args"] + R["TomlFilterRegistry::load()\n1. .rtk/filters.toml\n2. ~/.config/rtk/filters.toml\n3. BUILTIN_TOML\n4. passthrough"] --> S + S{"match_command\nmatches?"} -->|"no match"| T[["exec raw (passthrough)"]] + S -->|"match"| U["exec command\ncapture stdout"] + U --> V["8-stage pipeline\nstrip_ansi → replace → match_output\n→ strip/keep_lines → truncate\n→ tail_lines → max_lines → on_empty"] + V --> W[["print filtered output + exit code"]] + end + + G --> H & J & L & R +``` + +## Filter lookup priority + +```mermaid +flowchart LR + CMD["rtk my-tool args"] --> P1 + P1{"1. .rtk/filters.toml\n(project-local)"} + P1 -->|"match"| WIN["apply filter"] + P1 -->|"no match"| P2 + P2{"2. ~/.config/rtk/filters.toml\n(user-global)"} + P2 -->|"match"| WIN + P2 -->|"no match"| P3 + P3{"3. BUILTIN_TOML\n(binary)"} + P3 -->|"match"| WIN + P3 -->|"no match"| P4[["exec raw (passthrough)"]] +``` + +First match wins. A project filter with the same name as a built-in shadows the built-in and triggers a warning: + +``` +[rtk] warning: filter 'make' is shadowing a built-in filter +``` diff --git a/src/filters/liquibase.toml b/src/filters/liquibase.toml new file mode 100644 index 000000000..f3d20f167 --- /dev/null +++ b/src/filters/liquibase.toml @@ -0,0 +1,84 @@ +[filters.liquibase] +description = "Compact liquibase output — strip headers and generic info" +match_command = "(?:^|/)liquibase(?:\\s|$)" +strip_ansi = true +filter_stderr = true +strip_lines_matching = [ + "^\\s*$", + "^Starting Liquibase at", + "^Liquibase (?:Community|Open Source)", + "^Liquibase Home:", + "^Java Home", + "^Libraries:", + "^\\s*-\\s+\\S+\\.jar", + "^INFO \\[liquibase\\.integration\\]", + "^INFO \\[liquibase\\.core\\] Reading resource", + "^INFO \\[liquibase\\.core\\] Parsing", + "^(?:\\[?INFO\\]?\\s*)?#+$", + "^\\s*##" +] +on_empty = "liquibase: ok" +max_lines = 200 + +[[tests.liquibase]] +name = "strip ascii banner and info logs from subcommand" +input = ''' +#################################################### +## _ _ _ _ ## +## | | (_) (_) | ## +#################################################### +Starting Liquibase at 10:12:11 (version 4.29.1) +Liquibase Version: 4.29.1 +Liquibase Open Source 4.29.1 by Liquibase +INFO [liquibase.integration] Starting command +INFO [liquibase.core] Reading resource db/changelog.xml +INFO [liquibase.core] Parsing db/changelog.xml +Running Changeset: filepath::id::author +Changeset filepath::id::author ran successfully +''' +expected = ''' +Liquibase Version: 4.29.1 +Running Changeset: filepath::id::author +Changeset filepath::id::author ran successfully''' + +[[tests.liquibase]] +name = "strip --version noise, keep only version line" +input = ''' +#################################################### +## _ _ _ _ ## +#################################################### +Starting Liquibase at 13:45:24 using Java 17.0.15 (version 4.30.0 #4943 built at 2024-10-31 17:00+0000) +Liquibase Home: D:\mcp\bash\lbr\third-party +Java Home C:\Program Files\Java\jdk-17.0.15 (Version 17.0.15) +Libraries: + - internal\lib\commons-io.jar: Apache Commons IO 2.17.0 By The Apache Software Foundation + - internal\lib\picocli.jar: picocli 4.7.6 By Remko Popma + - lib\ojdbc10-19.30.0.0.jar: JDBC 19.30.0.0.0 By Oracle Corporation + +Liquibase Version: 4.30.0 +Liquibase Open Source 4.30.0 by Liquibase +''' +expected = ''' +Liquibase Version: 4.30.0''' + +[[tests.liquibase]] +name = "keep status and error lines" +input = ''' +#################################################### +## _ _ _ _ ## +#################################################### +Starting Liquibase at 10:00:00 (version 4.30.0) +Liquibase Version: 4.30.0 +Liquibase Open Source 4.30.0 by Liquibase +HR@jdbc:oracle:thin:@localhost:1523:XE is up to date +Liquibase command 'status' was executed successfully. +''' +expected = ''' +Liquibase Version: 4.30.0 +HR@jdbc:oracle:thin:@localhost:1523:XE is up to date +Liquibase command 'status' was executed successfully.''' + +[[tests.liquibase]] +name = "empty input" +input = "" +expected = "liquibase: ok" diff --git a/src/hooks/README.md b/src/hooks/README.md index 65e05f03d..bf947a0f1 100644 --- a/src/hooks/README.md +++ b/src/hooks/README.md @@ -1,6 +1,6 @@ # Hook System -> See also [docs/TECHNICAL.md](../../docs/TECHNICAL.md) for the full architecture overview | [hooks/](../../hooks/README.md) for deployed hook artifacts +> See also [docs/contributing/TECHNICAL.md](../../docs/contributing/TECHNICAL.md) for the full architecture overview | [hooks/](../../hooks/README.md) for deployed hook artifacts ## Scope @@ -28,7 +28,7 @@ LLM agent integration layer that installs, validates, and executes command-rewri | Claude-MD (legacy) | `rtk init --claude-md` | 134-line RTK block | CLAUDE.md | | Windsurf | `rtk init -g --agent windsurf` | `.windsurfrules` | -- | | Cline | `rtk init --agent cline` | `.clinerules` | -- | -| Codex | `rtk init --codex` | RTK.md | AGENTS.md | +| Codex | `rtk init --codex` | RTK.md in `$CODEX_HOME` or `~/.codex` | AGENTS.md | | Cursor | `rtk init -g --agent cursor` | Cursor hook | hooks.json | @@ -61,13 +61,42 @@ Controls how `rtk init` modifies agent settings files: All file operations use atomic writes (tempfile + rename) to prevent corruption on crash. Settings files are backed up to `.bak` before modification. All operations are idempotent -- running `rtk init` multiple times is safe. -## Exit Code Contract +## Permission Model -Hook processors in `hook_cmd.rs` must return `Ok(())` on every path — success, no-match, parse error, and unexpected input. Returning `Err` propagates to `main()` and exits non-zero, which blocks the agent's command from executing. This violates the non-blocking guarantee documented in `hooks/README.md`. +RTK enforces a permission precedence that matches Claude Code's least-privilege default: + +``` +Deny > Ask > Allow (explicit) > Default (ask) +``` + +Rules are loaded from all Claude Code `settings.json` files (project + global, including `.local` variants). Only `Bash(...)` rules are extracted; other scopes (Read, Write) are ignored. + +| Verdict | Trigger | rewrite_cmd exit | Hook behavior | +|---------|---------|-----------------|---------------| +| Deny | `permissions.deny` rule matched | 2 | Passthrough — host tool handles denial | +| Ask | `permissions.ask` rule matched | 3 | Rewrite + let host tool prompt user | +| Allow | `permissions.allow` rule matched | 0 | Rewrite + auto-allow | +| Default | No rule matched | 3 | Rewrite + let host tool prompt user | + +### Per-tool support -### Gaps (to be fixed) +| Tool | ask support | Behavior on Default | +|------|------------|-------------------| +| Claude Code (rtk-rewrite.sh) | Yes | `permissionDecision: "ask"` — user prompted | +| Copilot VS Code (rtk hook copilot) | Yes | `permissionDecision: "ask"` — user prompted | +| Gemini CLI (rtk hook gemini) | No (allow/deny only) | allow (limitation — no ask mode in Gemini) | +| Copilot CLI (rtk hook copilot) | No updatedInput | deny-with-suggestion (unchanged) | +| Codex | ask parsed but no-op | allow (limitation — fails open) | -- `hook_cmd.rs::run_gemini()` — uses `.context()?` on JSON parse, which returns `Err` on malformed input +### Implementation + +- `permissions.rs` — loads deny/ask/allow rules, evaluates precedence, returns `PermissionVerdict` +- `rewrite_cmd.rs` — maps verdict to exit code (consumed by shell hook) +- `hook_cmd.rs` — maps verdict to JSON `permissionDecision` field (Copilot/Gemini) + +## Exit Code Contract + +Hook processors in `hook_cmd.rs` must return `Ok(())` on every path — success, no-match, parse error, and unexpected input. Returning `Err` propagates to `main()` and exits non-zero, which blocks the agent's command from executing. This violates the non-blocking guarantee documented in `hooks/README.md`. ## Adding New Functionality To add support for a new AI coding agent: (1) add the hook installation logic to `init.rs` following the existing agent patterns, (2) if the agent requires a custom hook protocol (like Gemini's `BeforeTool`), add a processor function in `hook_cmd.rs`, (3) add the agent's hook file path to `hook_check.rs` for validation, and (4) update `integrity.rs` with the expected hash for the new hook file. Test by running `rtk init` in a fresh environment and verifying the hook rewrites commands correctly in the target agent. diff --git a/src/hooks/constants.rs b/src/hooks/constants.rs new file mode 100644 index 000000000..fcacdb3e5 --- /dev/null +++ b/src/hooks/constants.rs @@ -0,0 +1,19 @@ +pub const REWRITE_HOOK_FILE: &str = "rtk-rewrite.sh"; +pub const GEMINI_HOOK_FILE: &str = "rtk-hook-gemini.sh"; +pub const CLAUDE_DIR: &str = ".claude"; +pub const HOOKS_SUBDIR: &str = "hooks"; +pub const SETTINGS_JSON: &str = "settings.json"; +pub const SETTINGS_LOCAL_JSON: &str = "settings.local.json"; +pub const HOOKS_JSON: &str = "hooks.json"; +pub const PRE_TOOL_USE_KEY: &str = "PreToolUse"; +pub const BEFORE_TOOL_KEY: &str = "BeforeTool"; + +/// Native Rust hook command for Claude Code (replaces rtk-rewrite.sh). +pub const CLAUDE_HOOK_COMMAND: &str = "rtk hook claude"; +/// Native Rust hook command for Cursor (replaces rtk-rewrite.sh). +pub const CURSOR_HOOK_COMMAND: &str = "rtk hook cursor"; + +pub const OPENCODE_PLUGIN_PATH: &str = ".config/opencode/plugins/rtk.ts"; +pub const CURSOR_DIR: &str = ".cursor"; +pub const CODEX_DIR: &str = ".codex"; +pub const GEMINI_DIR: &str = ".gemini"; diff --git a/src/hooks/hook_check.rs b/src/hooks/hook_check.rs index 9fb0b3379..12f49755e 100644 --- a/src/hooks/hook_check.rs +++ b/src/hooks/hook_check.rs @@ -1,5 +1,12 @@ //! Detects whether RTK hooks are installed and warns if they are outdated. +use super::constants::{ + CLAUDE_DIR, CLAUDE_HOOK_COMMAND, HOOKS_SUBDIR, PRE_TOOL_USE_KEY, REWRITE_HOOK_FILE, + SETTINGS_JSON, +}; +#[cfg(test)] +use super::constants::{CODEX_DIR, CURSOR_DIR, GEMINI_DIR, GEMINI_HOOK_FILE, OPENCODE_PLUGIN_PATH}; +use crate::core::constants::RTK_DATA_DIR; use std::path::PathBuf; const CURRENT_HOOK_VERSION: u8 = 3; @@ -24,10 +31,23 @@ pub fn status() -> HookStatus { Some(h) => h, None => return HookStatus::Ok, }; - if !home.join(".claude").exists() { + let claude_dir = home.join(CLAUDE_DIR); + if !claude_dir.exists() { + return HookStatus::Ok; + } + + // Check for new binary command in settings.json first + if binary_hook_registered(&claude_dir) { + // If old script file still exists alongside new command, report Outdated + // (migration not complete — user should run `rtk init -g` to clean up) + let old_hook = claude_dir.join(HOOKS_SUBDIR).join(REWRITE_HOOK_FILE); + if old_hook.exists() { + return HookStatus::Outdated; + } return HookStatus::Ok; } + // Fall back to legacy script file check let Some(hook_path) = hook_installed_path() else { return HookStatus::Missing; }; @@ -41,6 +61,33 @@ pub fn status() -> HookStatus { } } +/// Check if the native binary command is registered in settings.json +fn binary_hook_registered(claude_dir: &std::path::Path) -> bool { + let settings_path = claude_dir.join(SETTINGS_JSON); + let content = match std::fs::read_to_string(&settings_path) { + Ok(c) if !c.trim().is_empty() => c, + _ => return false, + }; + let root: serde_json::Value = match serde_json::from_str(&content) { + Ok(v) => v, + Err(_) => return false, + }; + let pre_tool_use = match root + .get("hooks") + .and_then(|h| h.get(PRE_TOOL_USE_KEY)) + .and_then(|p| p.as_array()) + { + Some(arr) => arr, + None => return false, + }; + pre_tool_use + .iter() + .filter_map(|entry| entry.get("hooks")?.as_array()) + .flatten() + .filter_map(|hook| hook.get("command")?.as_str()) + .any(|cmd| cmd == CLAUDE_HOOK_COMMAND) +} + /// Check if the installed hook is missing or outdated, warn once per day. pub fn maybe_warn() { // Don't block startup — fail silently on any error @@ -88,9 +135,27 @@ pub fn parse_hook_version(content: &str) -> u8 { 0 // No version tag = version 0 (outdated) } +#[cfg(test)] +fn other_integration_installed(home: &std::path::Path) -> bool { + let paths = [ + home.join(OPENCODE_PLUGIN_PATH), + home.join(CURSOR_DIR) + .join(HOOKS_SUBDIR) + .join(REWRITE_HOOK_FILE), + home.join(CODEX_DIR).join("AGENTS.md"), + home.join(GEMINI_DIR) + .join(HOOKS_SUBDIR) + .join(GEMINI_HOOK_FILE), + ]; + paths.iter().any(|p| p.exists()) +} + fn hook_installed_path() -> Option { let home = dirs::home_dir()?; - let path = home.join(".claude").join("hooks").join("rtk-rewrite.sh"); + let path = home + .join(CLAUDE_DIR) + .join(HOOKS_SUBDIR) + .join(REWRITE_HOOK_FILE); if path.exists() { Some(path) } else { @@ -99,7 +164,7 @@ fn hook_installed_path() -> Option { } fn warn_marker_path() -> Option { - let data_dir = dirs::data_local_dir()?.join("rtk"); + let data_dir = dirs::data_local_dir()?.join(RTK_DATA_DIR); Some(data_dir.join(".hook_warn_last")) } @@ -141,32 +206,82 @@ mod tests { assert_eq!(s.clone(), HookStatus::Missing); } + #[test] + fn test_other_integration_none() { + let tmp = tempfile::tempdir().expect("tempdir"); + assert!(!other_integration_installed(tmp.path())); + } + + #[test] + fn test_other_integration_opencode() { + let tmp = tempfile::tempdir().expect("tempdir"); + let path = tmp.path().join(OPENCODE_PLUGIN_PATH); + std::fs::create_dir_all(path.parent().unwrap()).unwrap(); + std::fs::write(&path, b"plugin").unwrap(); + assert!(other_integration_installed(tmp.path())); + } + + #[test] + fn test_other_integration_cursor() { + let tmp = tempfile::tempdir().expect("tempdir"); + let path = tmp + .path() + .join(CURSOR_DIR) + .join(HOOKS_SUBDIR) + .join(REWRITE_HOOK_FILE); + std::fs::create_dir_all(path.parent().unwrap()).unwrap(); + std::fs::write(&path, b"hook").unwrap(); + assert!(other_integration_installed(tmp.path())); + } + + #[test] + fn test_other_integration_codex() { + let tmp = tempfile::tempdir().expect("tempdir"); + let path = tmp.path().join(CODEX_DIR).join("AGENTS.md"); + std::fs::create_dir_all(path.parent().unwrap()).unwrap(); + std::fs::write(&path, b"agents").unwrap(); + assert!(other_integration_installed(tmp.path())); + } + + #[test] + fn test_other_integration_gemini() { + let tmp = tempfile::tempdir().expect("tempdir"); + let path = tmp + .path() + .join(GEMINI_DIR) + .join(HOOKS_SUBDIR) + .join(GEMINI_HOOK_FILE); + std::fs::create_dir_all(path.parent().unwrap()).unwrap(); + std::fs::write(&path, b"hook").unwrap(); + assert!(other_integration_installed(tmp.path())); + } + + #[test] + fn test_other_integration_empty_dirs_not_enough() { + let tmp = tempfile::tempdir().expect("tempdir"); + std::fs::create_dir_all(tmp.path().join(CURSOR_DIR).join(HOOKS_SUBDIR)).unwrap(); + std::fs::create_dir_all(tmp.path().join(CODEX_DIR)).unwrap(); + std::fs::create_dir_all(tmp.path().join(GEMINI_DIR)).unwrap(); + assert!(!other_integration_installed(tmp.path())); + } + #[test] fn test_status_returns_valid_variant() { - // Skip on machines without Claude Code or without hook + // Skip on machines without Claude Code let home = match dirs::home_dir() { Some(h) => h, None => return, }; - if !home - .join(".claude") - .join("hooks") - .join("rtk-rewrite.sh") - .exists() - { - // No hook — status should be Missing (if .claude exists) or Ok (if not) - let s = status(); - if home.join(".claude").exists() { - assert_eq!(s, HookStatus::Missing); - } else { - assert_eq!(s, HookStatus::Ok); - } + let claude_dir = home.join(".claude"); + if !claude_dir.exists() { + assert_eq!(status(), HookStatus::Ok); return; } + // With .claude dir present, status must be one of the valid variants let s = status(); assert!( - s == HookStatus::Ok || s == HookStatus::Outdated, - "Expected Ok or Outdated when hook exists, got {:?}", + s == HookStatus::Ok || s == HookStatus::Outdated || s == HookStatus::Missing, + "Expected valid HookStatus variant, got {:?}", s ); } diff --git a/src/hooks/hook_cmd.rs b/src/hooks/hook_cmd.rs index 8eb4e2fa4..cd3c82d1e 100644 --- a/src/hooks/hook_cmd.rs +++ b/src/hooks/hook_cmd.rs @@ -1,10 +1,29 @@ //! Processes incoming hook calls from AI agents and rewrites commands on the fly. +//! +//! Uses `writeln!(stdout, ...)` instead of `println!` — accidental stdout/stderr +//! corrupts the JSON protocol (Claude Code bug #4669 silently disables the hook). +use super::constants::PRE_TOOL_USE_KEY; +use super::permissions::{self, PermissionVerdict}; use anyhow::{Context, Result}; use serde_json::{json, Value}; -use std::io::{self, Read}; +use std::io::{self, Read, Write}; -use crate::discover::registry::rewrite_command; +use crate::discover::registry::{has_heredoc, rewrite_command}; + +const STDIN_CAP: usize = 1_048_576; // 1 MiB + +fn read_stdin_limited() -> Result { + let mut input = String::new(); + io::stdin() + .take((STDIN_CAP + 1) as u64) + .read_to_string(&mut input) + .context("Failed to read stdin")?; + if input.len() > STDIN_CAP { + anyhow::bail!("hook stdin exceeds {} byte limit", STDIN_CAP); + } + Ok(input) +} // ── Copilot hook (VS Code + Copilot CLI) ────────────────────── @@ -21,10 +40,7 @@ enum HookFormat { /// Run the Copilot preToolUse hook. /// Auto-detects VS Code Copilot Chat vs Copilot CLI format. pub fn run_copilot() -> Result<()> { - let mut input = String::new(); - io::stdin() - .read_to_string(&mut input) - .context("Failed to read stdin")?; + let input = read_stdin_limited()?; let input = input.trim(); if input.is_empty() { @@ -34,7 +50,7 @@ pub fn run_copilot() -> Result<()> { let v: Value = match serde_json::from_str(input) { Ok(v) => v, Err(e) => { - eprintln!("[rtk hook] Failed to parse JSON input: {e}"); + let _ = writeln!(io::stderr(), "[rtk hook] Failed to parse JSON input: {e}"); return Ok(()); } }; @@ -87,7 +103,7 @@ fn detect_format(v: &Value) -> HookFormat { } fn get_rewritten(cmd: &str) -> Option { - if cmd.contains("<<") { + if has_heredoc(cmd) { return None; } @@ -105,29 +121,51 @@ fn get_rewritten(cmd: &str) -> Option { } fn handle_vscode(cmd: &str) -> Result<()> { + let verdict = permissions::check_command(cmd); + if verdict == PermissionVerdict::Deny { + audit_log("deny", cmd, ""); + return Ok(()); + } + let rewritten = match get_rewritten(cmd) { Some(r) => r, None => return Ok(()), }; + // Allow (explicit rule matched): auto-allow the rewritten command. + // Ask/Default (no allow rule matched): rewrite but let the host tool prompt. + let decision = match verdict { + PermissionVerdict::Allow => "allow", + _ => "ask", + }; + + audit_log("rewrite", cmd, &rewritten); + let output = json!({ "hookSpecificOutput": { - "hookEventName": "PreToolUse", - "permissionDecision": "allow", + "hookEventName": PRE_TOOL_USE_KEY, + "permissionDecision": decision, "permissionDecisionReason": "RTK auto-rewrite", "updatedInput": { "command": rewritten } } }); - println!("{output}"); + let _ = writeln!(io::stdout(), "{output}"); Ok(()) } fn handle_copilot_cli(cmd: &str) -> Result<()> { + if permissions::check_command(cmd) == PermissionVerdict::Deny { + audit_log("deny", cmd, ""); + return Ok(()); + } + let rewritten = match get_rewritten(cmd) { Some(r) => r, None => return Ok(()), }; + audit_log("rewrite", cmd, &rewritten); + let output = json!({ "permissionDecision": "deny", "permissionDecisionReason": format!( @@ -135,20 +173,15 @@ fn handle_copilot_cli(cmd: &str) -> Result<()> { rewritten ) }); - println!("{output}"); + let _ = writeln!(io::stdout(), "{output}"); Ok(()) } // ── Gemini hook ─────────────────────────────────────────────── /// Run the Gemini CLI BeforeTool hook. -/// Reads JSON from stdin, rewrites shell commands to rtk equivalents, -/// outputs JSON to stdout in Gemini CLI format. pub fn run_gemini() -> Result<()> { - let mut input = String::new(); - io::stdin() - .read_to_string(&mut input) - .context("Failed to read hook input from stdin")?; + let input = read_stdin_limited()?; let json: Value = serde_json::from_str(&input).context("Failed to parse hook input as JSON")?; @@ -169,9 +202,24 @@ pub fn run_gemini() -> Result<()> { return Ok(()); } - // Delegate to the single source of truth for command rewriting - match rewrite_command(cmd, &[]) { - Some(rewritten) => print_rewrite(&rewritten), + // Check deny rules — Gemini CLI only supports allow/deny (no ask mode). + if permissions::check_command(cmd) == PermissionVerdict::Deny { + let _ = writeln!( + io::stdout(), + r#"{{"decision":"deny","reason":"Blocked by RTK permission rule"}}"# + ); + return Ok(()); + } + + let excluded = crate::core::config::Config::load() + .map(|c| c.hooks.exclude_commands) + .unwrap_or_default(); + + match rewrite_command(cmd, &excluded) { + Some(ref rewritten) => { + audit_log("rewrite", cmd, rewritten); + print_rewrite(rewritten); + } None => print_allow(), } @@ -179,7 +227,7 @@ pub fn run_gemini() -> Result<()> { } fn print_allow() { - println!(r#"{{"decision":"allow"}}"#); + let _ = writeln!(io::stdout(), r#"{{"decision":"allow"}}"#); } fn print_rewrite(cmd: &str) { @@ -191,7 +239,271 @@ fn print_rewrite(cmd: &str) { } } }); - println!("{}", output); + let _ = writeln!(io::stdout(), "{}", output); +} + +// ── Audit logging ───────────────────────────────────────────── + +/// Best-effort audit log when RTK_HOOK_AUDIT=1. +fn audit_log(action: &str, original: &str, rewritten: &str) { + if std::env::var("RTK_HOOK_AUDIT").as_deref() != Ok("1") { + return; + } + let _ = audit_log_inner(action, original, rewritten); +} + +/// Escape newlines to prevent log-line injection in the pipe-delimited audit log. +fn sanitize_log_field(s: &str) -> String { + s.replace('\\', "\\\\") + .replace('|', "\\|") + .replace('\n', "\\n") + .replace('\r', "\\r") +} + +fn audit_log_inner(action: &str, original: &str, rewritten: &str) -> Option<()> { + let home = dirs::home_dir()?; + let dir = home.join(".local").join("share").join("rtk"); + std::fs::create_dir_all(&dir).ok()?; + let path = dir.join("hook-audit.log"); + let mut file = std::fs::OpenOptions::new() + .create(true) + .append(true) + .open(path) + .ok()?; + let ts = chrono::Local::now().format("%Y-%m-%dT%H:%M:%S"); + writeln!( + file, + "{} | {} | {} | {}", + ts, + action, + sanitize_log_field(original), + sanitize_log_field(rewritten) + ) + .ok() +} + +// ── Claude Code native hook ──────────────────────────────────── + +enum PayloadAction { + Rewrite { + cmd: String, + rewritten: String, + output: Value, + }, + Skip { + reason: &'static str, + cmd: String, + }, + Ignore, +} + +fn process_claude_payload(v: &Value) -> PayloadAction { + let cmd = match v + .pointer("/tool_input/command") + .and_then(|c| c.as_str()) + .filter(|c| !c.is_empty()) + { + Some(c) => c, + None => return PayloadAction::Ignore, + }; + + let verdict = permissions::check_command(cmd); + if verdict == PermissionVerdict::Deny { + return PayloadAction::Skip { + reason: "skip:deny_rule", + cmd: cmd.to_string(), + }; + } + + let rewritten = match get_rewritten(cmd) { + Some(r) => r, + None => { + return PayloadAction::Skip { + reason: "skip:no_match", + cmd: cmd.to_string(), + } + } + }; + + let updated_input = { + let mut ti = v.get("tool_input").cloned().unwrap_or_else(|| json!({})); + if let Some(obj) = ti.as_object_mut() { + obj.insert("command".into(), Value::String(rewritten.clone())); + } + ti + }; + + let mut hook_output = json!({ + "hookEventName": PRE_TOOL_USE_KEY, + "permissionDecisionReason": "RTK auto-rewrite", + "updatedInput": updated_input + }); + + if verdict == PermissionVerdict::Allow { + hook_output + .as_object_mut() + .unwrap() + .insert("permissionDecision".into(), json!("allow")); + } + + PayloadAction::Rewrite { + cmd: cmd.to_string(), + rewritten, + output: json!({ "hookSpecificOutput": hook_output }), + } +} + +/// Run the Claude Code PreToolUse hook natively. +pub fn run_claude() -> Result<()> { + let input = read_stdin_limited()?; + + let input = input.trim(); + if input.is_empty() { + return Ok(()); + } + + let v: Value = match serde_json::from_str(input) { + Ok(v) => v, + Err(e) => { + let _ = writeln!(io::stderr(), "[rtk hook] Failed to parse JSON input: {e}"); + return Ok(()); + } + }; + + match process_claude_payload(&v) { + PayloadAction::Rewrite { + cmd, + rewritten, + output, + } => { + audit_log("rewrite", &cmd, &rewritten); + let _ = writeln!(io::stdout(), "{output}"); + } + PayloadAction::Skip { reason, cmd } => { + audit_log(reason, &cmd, ""); + } + PayloadAction::Ignore => {} + } + + Ok(()) +} + +#[cfg(test)] +fn run_claude_inner(input: &str) -> Option { + let v: Value = serde_json::from_str(input).ok()?; + match process_claude_payload(&v) { + PayloadAction::Rewrite { output, .. } => Some(output.to_string()), + _ => None, + } +} + +// ── Cursor native hook ───────────────────────────────────────── + +/// Run the Cursor Agent hook natively. +pub fn run_cursor() -> Result<()> { + let input = read_stdin_limited()?; + + let input = input.trim(); + if input.is_empty() { + let _ = writeln!(io::stdout(), "{{}}"); + return Ok(()); + } + + let v: Value = match serde_json::from_str(input) { + Ok(v) => v, + Err(_) => { + let _ = writeln!(io::stdout(), "{{}}"); + return Ok(()); + } + }; + + let cmd = match v + .pointer("/tool_input/command") + .and_then(|c| c.as_str()) + .filter(|c| !c.is_empty()) + { + Some(c) => c.to_string(), + None => { + let _ = writeln!(io::stdout(), "{{}}"); + return Ok(()); + } + }; + + let verdict = permissions::check_command(&cmd); + if verdict == PermissionVerdict::Deny { + audit_log("deny", &cmd, ""); + let _ = writeln!(io::stdout(), "{{}}"); + return Ok(()); + } + + let rewritten = match get_rewritten(&cmd) { + Some(r) => r, + None => { + let _ = writeln!(io::stdout(), "{{}}"); + return Ok(()); + } + }; + + let decision = match verdict { + PermissionVerdict::Allow => "allow", + _ => "ask", + }; + + audit_log("rewrite", &cmd, &rewritten); + + let output = json!({ + "permission": decision, + "updated_input": { "command": rewritten } + }); + let _ = writeln!(io::stdout(), "{output}"); + Ok(()) +} + +#[cfg(test)] +fn run_cursor_inner(input: &str) -> String { + run_cursor_inner_with_rules(input, &[], &[], &[]) +} + +#[cfg(test)] +fn run_cursor_inner_with_rules( + input: &str, + deny_rules: &[String], + ask_rules: &[String], + allow_rules: &[String], +) -> String { + let v: Value = match serde_json::from_str(input) { + Ok(v) => v, + Err(_) => return "{}".to_string(), + }; + + let cmd = match v + .pointer("/tool_input/command") + .and_then(|c| c.as_str()) + .filter(|c| !c.is_empty()) + { + Some(c) => c.to_string(), + None => return "{}".to_string(), + }; + + let verdict = permissions::check_command_with_rules(&cmd, deny_rules, ask_rules, allow_rules); + if verdict == PermissionVerdict::Deny { + return "{}".to_string(); + } + + match get_rewritten(&cmd) { + Some(rewritten) => { + let decision = match verdict { + PermissionVerdict::Allow => "allow", + _ => "ask", + }; + let output = json!({ + "permission": decision, + "updated_input": { "command": rewritten } + }); + output.to_string() + } + None => "{}".to_string(), + } } #[cfg(test)] @@ -271,7 +583,6 @@ mod tests { #[test] fn test_print_allow_format() { - // Verify the allow JSON format matches Gemini CLI expectations let expected = r#"{"decision":"allow"}"#; assert_eq!(expected, r#"{"decision":"allow"}"#); } @@ -296,7 +607,6 @@ mod tests { #[test] fn test_gemini_hook_uses_rewrite_command() { - // Verify that rewrite_command handles the cases we need for Gemini assert_eq!( rewrite_command("git status", &[]), Some("rtk git status".into()) @@ -305,12 +615,10 @@ mod tests { rewrite_command("cargo test", &[]), Some("rtk cargo test".into()) ); - // Already rtk → returned as-is (idempotent) assert_eq!( rewrite_command("rtk git status", &[]), Some("rtk git status".into()) ); - // Heredoc → no rewrite assert_eq!(rewrite_command("cat < String { + json!({ + "tool_name": "Bash", + "tool_input": { "command": cmd } + }) + .to_string() + } + + fn claude_input_with_fields(cmd: &str, timeout: u64, description: &str) -> String { + json!({ + "tool_name": "Bash", + "tool_input": { + "command": cmd, + "timeout": timeout, + "description": description + } + }) + .to_string() + } + + #[test] + fn test_claude_rewrite_git_status() { + let result = run_claude_inner(&claude_input("git status")).unwrap(); + let v: Value = serde_json::from_str(&result).unwrap(); + let cmd = v + .pointer("/hookSpecificOutput/updatedInput/command") + .and_then(|c| c.as_str()) + .unwrap(); + assert_eq!(cmd, "rtk git status"); + } + + #[test] + fn test_claude_rewrite_preserves_tool_input_fields() { + let input = claude_input_with_fields("git status", 30000, "Check repo status"); + let result = run_claude_inner(&input).unwrap(); + let v: Value = serde_json::from_str(&result).unwrap(); + let updated = &v["hookSpecificOutput"]["updatedInput"]; + assert_eq!(updated["command"], "rtk git status"); + assert_eq!(updated["timeout"], 30000); + assert_eq!(updated["description"], "Check repo status"); + } + + #[test] + fn test_claude_passthrough_no_output() { + assert!(run_claude_inner(&claude_input("htop")).is_none()); + } + + #[test] + fn test_claude_heredoc_passthrough() { + assert!(run_claude_inner(&claude_input("cat < String { + json!({ + "tool_name": "Bash", + "tool_input": { "command": cmd } + }) + .to_string() + } + + #[test] + fn test_cursor_rewrite_flat_format() { + let result = run_cursor_inner(&cursor_input("git status")); + let v: Value = serde_json::from_str(&result).unwrap(); + // Default permission (no explicit allow rule) → "ask" + assert_eq!(v["permission"], "ask"); + assert_eq!(v["updated_input"]["command"], "rtk git status"); + assert!(v.get("hookSpecificOutput").is_none()); + } + + #[test] + fn test_cursor_passthrough_empty_json() { + let result = run_cursor_inner(&cursor_input("htop")); + assert_eq!(result, "{}"); + } + + #[test] + fn test_cursor_empty_input_empty_json() { + let result = run_cursor_inner(""); + assert_eq!(result, "{}"); + } + + #[test] + fn test_cursor_heredoc_passthrough() { + let result = run_cursor_inner(&cursor_input("cat < = content.trim().split(" | ").collect(); + assert_eq!( + parts.len(), + 4, + "Expected 4 pipe-delimited fields, got: {:?}", + parts + ); + assert_eq!(parts[1], "rewrite"); + assert_eq!(parts[2], "git status"); + assert_eq!(parts[3], "rtk git status"); + + let _ = std::fs::remove_dir_all(&tmp); + } + + // --- Adversarial tests --- + + #[test] + fn test_audit_log_sanitizes_newlines() { + let sanitized = sanitize_log_field("git status\nfake | inject | evil"); + assert!(!sanitized.contains('\n')); + assert!(sanitized.contains("\\n")); + } + + #[test] + fn test_audit_log_sanitizes_pipe_delimiter() { + let sanitized = sanitize_log_field("git log | head"); + assert!( + !sanitized.contains(" | "), + "unescaped ' | ' breaks field parsing: {}", + sanitized + ); + assert!(sanitized.contains("\\|")); + } + + #[test] + fn test_claude_unicode_null_passthrough() { + let input = claude_input("git status \u{0000}\u{FEFF}"); + let _ = run_claude_inner(&input); + } + + #[test] + fn test_claude_extremely_long_command() { + let long_cmd = format!("git status {}", "A".repeat(100_000)); + let input = claude_input(&long_cmd); + let _ = run_claude_inner(&input); + } + + #[test] + fn test_cursor_deny_blocks_rewrite() { + use super::permissions::check_command_with_rules; + let deny = vec!["git status".to_string()]; + assert_eq!( + check_command_with_rules("git status", &deny, &[], &[]), + PermissionVerdict::Deny + ); + } + + #[test] + fn test_gemini_deny_blocks_rewrite() { + use super::permissions::check_command_with_rules; + let deny = vec!["cargo test".to_string()]; + assert_eq!( + check_command_with_rules("cargo test", &deny, &[], &[]), + PermissionVerdict::Deny + ); + // Denied commands must not be rewritten — Gemini handler checks deny before rewrite + assert!( + get_rewritten("cargo test").is_some(), + "cargo test should be rewritable when not denied" + ); + } } diff --git a/src/hooks/init.rs b/src/hooks/init.rs index 438aca7a0..87face931 100644 --- a/src/hooks/init.rs +++ b/src/hooks/init.rs @@ -6,14 +6,11 @@ use std::io::Write; use std::path::{Path, PathBuf}; use tempfile::NamedTempFile; +use super::constants::{ + BEFORE_TOOL_KEY, CLAUDE_DIR, CLAUDE_HOOK_COMMAND, CODEX_DIR, CURSOR_HOOK_COMMAND, + GEMINI_HOOK_FILE, HOOKS_JSON, HOOKS_SUBDIR, PRE_TOOL_USE_KEY, REWRITE_HOOK_FILE, SETTINGS_JSON, +}; use super::integrity; -use crate::core::config; - -// Embedded hook script (guards before set -euo pipefail) -const REWRITE_HOOK: &str = include_str!("../../hooks/claude/rtk-rewrite.sh"); - -// Embedded Cursor hook script (preToolUse format) -const CURSOR_REWRITE_HOOK: &str = include_str!("../../hooks/cursor/rtk-rewrite.sh"); // Embedded OpenCode plugin (auto-rewrite) const OPENCODE_PLUGIN: &str = include_str!("../../hooks/opencode/rtk.ts"); @@ -53,6 +50,12 @@ schema_version = 1 # max_lines = 40 "#; +const RTK_MD: &str = "RTK.md"; +const CLAUDE_MD: &str = "CLAUDE.md"; +const AGENTS_MD: &str = "AGENTS.md"; +const RTK_MD_REF: &str = "@RTK.md"; +const GEMINI_MD: &str = "GEMINI.md"; + /// Control flow for settings.json patching #[derive(Debug, Clone, Copy, PartialEq)] pub enum PatchMode { @@ -100,11 +103,16 @@ rtk prettier --check # Files needing format only (70%) rtk next build # Next.js build with route metrics (87%) ``` -### Test (90-99% savings) +### Test (60-99% savings) ```bash rtk cargo test # Cargo test failures only (90%) -rtk vitest run # Vitest failures only (99.5%) +rtk go test # Go test failures only (90%) +rtk jest # Jest failures only (99.5%) +rtk vitest # Vitest failures only (99.5%) rtk playwright test # Playwright failures only (94%) +rtk pytest # Python test failures only (90%) +rtk rake test # Ruby test failures only (90%) +rtk rspec # RSpec test failures only (60%) rtk test # Generic test wrapper - failures only ``` @@ -282,75 +290,9 @@ pub fn run( install_cursor_hooks(verbose)?; } - println!(); - let env_disabled = std::env::var("RTK_TELEMETRY_DISABLED").unwrap_or_default() == "1"; - let config_disabled = matches!(config::telemetry_enabled(), Some(false)); - if env_disabled || config_disabled { - println!(" [info] Anonymous telemetry is disabled"); - } else { - println!(" [info] Anonymous telemetry is enabled by default (opt-out: RTK_TELEMETRY_DISABLED=1)"); - } - println!(" [info] See: https://github.com/rtk-ai/rtk#privacy--telemetry"); - Ok(()) } -/// Prepare hook directory and return paths (hook_dir, hook_path) -fn prepare_hook_paths() -> Result<(PathBuf, PathBuf)> { - let claude_dir = resolve_claude_dir()?; - let hook_dir = claude_dir.join("hooks"); - fs::create_dir_all(&hook_dir) - .with_context(|| format!("Failed to create hook directory: {}", hook_dir.display()))?; - let hook_path = hook_dir.join("rtk-rewrite.sh"); - Ok((hook_dir, hook_path)) -} - -/// Write hook file if missing or outdated, return true if changed -#[cfg(unix)] -fn ensure_hook_installed(hook_path: &Path, verbose: u8) -> Result { - let changed = if hook_path.exists() { - let existing = fs::read_to_string(hook_path) - .with_context(|| format!("Failed to read existing hook: {}", hook_path.display()))?; - - if existing == REWRITE_HOOK { - if verbose > 0 { - eprintln!("Hook already up to date: {}", hook_path.display()); - } - false - } else { - fs::write(hook_path, REWRITE_HOOK) - .with_context(|| format!("Failed to write hook to {}", hook_path.display()))?; - if verbose > 0 { - eprintln!("Updated hook: {}", hook_path.display()); - } - true - } - } else { - fs::write(hook_path, REWRITE_HOOK) - .with_context(|| format!("Failed to write hook to {}", hook_path.display()))?; - if verbose > 0 { - eprintln!("Created hook: {}", hook_path.display()); - } - true - }; - - // Set executable permissions - use std::os::unix::fs::PermissionsExt; - fs::set_permissions(hook_path, fs::Permissions::from_mode(0o755)) - .with_context(|| format!("Failed to set hook permissions: {}", hook_path.display()))?; - - // Store SHA-256 hash for runtime integrity verification. - // Always store (idempotent) to ensure baseline exists even for - // hooks installed before integrity checks were added. - integrity::store_hash(hook_path) - .with_context(|| format!("Failed to store integrity hash for {}", hook_path.display()))?; - if verbose > 0 && changed { - eprintln!("Stored integrity hash for hook"); - } - - Ok(changed) -} - /// Idempotent file write: create or update if content differs fn write_if_changed(path: &Path, content: &str, name: &str, verbose: u8) -> Result { if path.exists() { @@ -363,7 +305,7 @@ fn write_if_changed(path: &Path, content: &str, name: &str, verbose: u8) -> Resu } Ok(false) } else { - fs::write(path, content) + atomic_write(path, content) .with_context(|| format!("Failed to write {}: {}", name, path.display()))?; if verbose > 0 { eprintln!("Updated {}: {}", name, path.display()); @@ -371,7 +313,7 @@ fn write_if_changed(path: &Path, content: &str, name: &str, verbose: u8) -> Resu Ok(true) } } else { - fs::write(path, content) + atomic_write(path, content) .with_context(|| format!("Failed to write {}: {}", name, path.display()))?; if verbose > 0 { eprintln!("Created {}: {}", name, path.display()); @@ -435,14 +377,13 @@ fn prompt_user_consent(settings_path: &Path) -> Result { Ok(response == "y" || response == "yes") } -/// Print manual instructions for settings.json patching -fn print_manual_instructions(hook_path: &Path, include_opencode: bool) { +fn print_manual_instructions(hook_command: &str, include_opencode: bool) { println!("\n MANUAL STEP: Add this to ~/.claude/settings.json:"); println!(" {{"); println!(" \"hooks\": {{ \"PreToolUse\": [{{"); println!(" \"matcher\": \"Bash\","); println!(" \"hooks\": [{{ \"type\": \"command\","); - println!(" \"command\": \"{}\"", hook_path.display()); + println!(" \"command\": \"{}\"", hook_command); println!(" }}]"); println!(" }}]}}"); println!(" }}"); @@ -453,10 +394,11 @@ fn print_manual_instructions(hook_path: &Path, include_opencode: bool) { } } -/// Remove RTK hook entry from settings.json -/// Returns true if hook was found and removed fn remove_hook_from_json(root: &mut serde_json::Value) -> bool { - let hooks = match root.get_mut("hooks").and_then(|h| h.get_mut("PreToolUse")) { + let hooks = match root + .get_mut("hooks") + .and_then(|h| h.get_mut(PRE_TOOL_USE_KEY)) + { Some(pre_tool_use) => pre_tool_use, None => return false, }; @@ -466,19 +408,19 @@ fn remove_hook_from_json(root: &mut serde_json::Value) -> bool { None => return false, }; - // Find and remove RTK entry let original_len = pre_tool_use_array.len(); pre_tool_use_array.retain(|entry| { if let Some(hooks_array) = entry.get("hooks").and_then(|h| h.as_array()) { for hook in hooks_array { if let Some(command) = hook.get("command").and_then(|c| c.as_str()) { - if command.contains("rtk-rewrite.sh") { - return false; // Remove this entry + // Match both legacy script path and new binary command + if command.contains(REWRITE_HOOK_FILE) || command == CLAUDE_HOOK_COMMAND { + return false; } } } } - true // Keep this entry + true }); pre_tool_use_array.len() < original_len @@ -488,7 +430,7 @@ fn remove_hook_from_json(root: &mut serde_json::Value) -> bool { /// Backs up before modification, returns true if hook was found and removed fn remove_hook_from_settings(verbose: u8) -> Result { let claude_dir = resolve_claude_dir()?; - let settings_path = claude_dir.join("settings.json"); + let settings_path = claude_dir.join(SETTINGS_JSON); if !settings_path.exists() { if verbose > 0 { @@ -575,12 +517,12 @@ pub fn uninstall(global: bool, gemini: bool, codex: bool, cursor: bool, verbose: return Ok(()); } - // 1. Remove hook file - let hook_path = claude_dir.join("hooks").join("rtk-rewrite.sh"); + // 1. Remove legacy hook file (if exists from old installation) + let hook_path = claude_dir.join(HOOKS_SUBDIR).join(REWRITE_HOOK_FILE); if hook_path.exists() { fs::remove_file(&hook_path) .with_context(|| format!("Failed to remove hook: {}", hook_path.display()))?; - removed.push(format!("Hook: {}", hook_path.display())); + removed.push(format!("Hook script: {}", hook_path.display())); } // 1b. Remove integrity hash file @@ -589,7 +531,7 @@ pub fn uninstall(global: bool, gemini: bool, codex: bool, cursor: bool, verbose: } // 2. Remove RTK.md - let rtk_md_path = claude_dir.join("RTK.md"); + let rtk_md_path = claude_dir.join(RTK_MD); if rtk_md_path.exists() { fs::remove_file(&rtk_md_path) .with_context(|| format!("Failed to remove RTK.md: {}", rtk_md_path.display()))?; @@ -597,15 +539,15 @@ pub fn uninstall(global: bool, gemini: bool, codex: bool, cursor: bool, verbose: } // 3. Remove @RTK.md reference from CLAUDE.md - let claude_md_path = claude_dir.join("CLAUDE.md"); + let claude_md_path = claude_dir.join(CLAUDE_MD); if claude_md_path.exists() { let content = fs::read_to_string(&claude_md_path) .with_context(|| format!("Failed to read CLAUDE.md: {}", claude_md_path.display()))?; - if content.contains("@RTK.md") { + if content.contains(RTK_MD_REF) { let new_content = content .lines() - .filter(|line| !line.trim().starts_with("@RTK.md")) + .filter(|line| !line.trim().starts_with(RTK_MD_REF)) .collect::>() .join("\n"); @@ -672,8 +614,9 @@ fn uninstall_codex(global: bool, verbose: u8) -> Result<()> { fn uninstall_codex_at(codex_dir: &Path, verbose: u8) -> Result> { let mut removed = Vec::new(); + let absolute_rtk_md_ref = codex_rtk_md_ref(codex_dir); - let rtk_md_path = codex_dir.join("RTK.md"); + let rtk_md_path = codex_dir.join(RTK_MD); if rtk_md_path.exists() { fs::remove_file(&rtk_md_path) .with_context(|| format!("Failed to remove RTK.md: {}", rtk_md_path.display()))?; @@ -683,27 +626,28 @@ fn uninstall_codex_at(codex_dir: &Path, verbose: u8) -> Result> { removed.push(format!("RTK.md: {}", rtk_md_path.display())); } - let agents_md_path = codex_dir.join("AGENTS.md"); - if remove_rtk_reference_from_agents(&agents_md_path, verbose)? { + let agents_md_path = codex_dir.join(AGENTS_MD); + if remove_rtk_reference_from_agents( + &agents_md_path, + &[RTK_MD_REF, absolute_rtk_md_ref.as_str()], + verbose, + )? { removed.push("AGENTS.md: removed @RTK.md reference".to_string()); } Ok(removed) } -/// Orchestrator: patch settings.json with RTK hook +/// Orchestrator: patch settings.json with RTK hook (binary command variant) /// Handles reading, checking, prompting, merging, backing up, and atomic writing -fn patch_settings_json( - hook_path: &Path, +fn patch_settings_json_command( + hook_command: &str, mode: PatchMode, verbose: u8, include_opencode: bool, ) -> Result { let claude_dir = resolve_claude_dir()?; - let settings_path = claude_dir.join("settings.json"); - let hook_command = hook_path - .to_str() - .context("Hook path contains invalid UTF-8")?; + let settings_path = claude_dir.join(SETTINGS_JSON); // Read or create settings.json let mut root = if settings_path.exists() { @@ -731,12 +675,12 @@ fn patch_settings_json( // Handle mode match mode { PatchMode::Skip => { - print_manual_instructions(hook_path, include_opencode); + print_manual_instructions(hook_command, include_opencode); return Ok(PatchResult::Skipped); } PatchMode::Ask => { if !prompt_user_consent(&settings_path)? { - print_manual_instructions(hook_path, include_opencode); + print_manual_instructions(hook_command, include_opencode); return Ok(PatchResult::Declined); } } @@ -745,8 +689,7 @@ fn patch_settings_json( } } - // Deep-merge hook - insert_hook_entry(&mut root, hook_command); + insert_hook_entry(&mut root, hook_command)?; // Backup original if settings_path.exists() { @@ -811,31 +754,27 @@ fn clean_double_blanks(content: &str) -> String { /// Deep-merge RTK hook entry into settings.json /// Creates hooks.PreToolUse structure if missing, preserves existing hooks -fn insert_hook_entry(root: &mut serde_json::Value, hook_command: &str) { - // Ensure root is an object +fn insert_hook_entry(root: &mut serde_json::Value, hook_command: &str) -> Result<()> { let root_obj = match root.as_object_mut() { Some(obj) => obj, None => { *root = serde_json::json!({}); - root.as_object_mut() - .expect("Just created object, must succeed") + root.as_object_mut().expect("just-created json object") } }; - // Use entry() API for idiomatic insertion let hooks = root_obj .entry("hooks") .or_insert_with(|| serde_json::json!({})) .as_object_mut() - .expect("hooks must be an object"); + .context("hooks value is not an object")?; let pre_tool_use = hooks - .entry("PreToolUse") + .entry(PRE_TOOL_USE_KEY) .or_insert_with(|| serde_json::json!([])) .as_array_mut() - .expect("PreToolUse must be an array"); + .context("PreToolUse value is not an array")?; - // Append RTK hook entry pre_tool_use.push(serde_json::json!({ "matcher": "Bash", "hooks": [{ @@ -843,14 +782,15 @@ fn insert_hook_entry(root: &mut serde_json::Value, hook_command: &str) { "command": hook_command }] })); + Ok(()) } /// Check if RTK hook is already present in settings.json -/// Matches on rtk-rewrite.sh substring to handle different path formats +/// Matches on legacy rtk-rewrite.sh path OR new `rtk hook claude` command fn hook_already_present(root: &serde_json::Value, hook_command: &str) -> bool { let pre_tool_use_array = match root .get("hooks") - .and_then(|h| h.get("PreToolUse")) + .and_then(|h| h.get(PRE_TOOL_USE_KEY)) .and_then(|p| p.as_array()) { Some(arr) => arr, @@ -863,27 +803,11 @@ fn hook_already_present(root: &serde_json::Value, hook_command: &str) -> bool { .flatten() .filter_map(|hook| hook.get("command")?.as_str()) .any(|cmd| { - // Exact match OR both contain rtk-rewrite.sh - cmd == hook_command - || (cmd.contains("rtk-rewrite.sh") && hook_command.contains("rtk-rewrite.sh")) + cmd == hook_command || cmd == CLAUDE_HOOK_COMMAND || cmd.contains(REWRITE_HOOK_FILE) }) } /// Default mode: hook + slim RTK.md + @RTK.md reference -#[cfg(not(unix))] -fn run_default_mode( - _global: bool, - _patch_mode: PatchMode, - _verbose: u8, - _install_opencode: bool, -) -> Result<()> { - eprintln!("[warn] Hook-based mode requires Unix (macOS/Linux)."); - eprintln!(" Windows: use --claude-md mode for full injection."); - eprintln!(" Falling back to --claude-md mode."); - run_claude_md_mode(_global, _verbose, _install_opencode) -} - -#[cfg(unix)] fn run_default_mode( global: bool, patch_mode: PatchMode, @@ -898,15 +822,14 @@ fn run_default_mode( } let claude_dir = resolve_claude_dir()?; - let rtk_md_path = claude_dir.join("RTK.md"); - let claude_md_path = claude_dir.join("CLAUDE.md"); + let rtk_md_path = claude_dir.join(RTK_MD); + let claude_md_path = claude_dir.join(CLAUDE_MD); - // 1. Prepare hook directory and install hook - let (_hook_dir, hook_path) = prepare_hook_paths()?; - let hook_changed = ensure_hook_installed(&hook_path, verbose)?; + // 1. Migrate old hook script if present + migrate_old_hook_script(verbose); // 2. Write RTK.md - write_if_changed(&rtk_md_path, RTK_SLIM, "RTK.md", verbose)?; + write_if_changed(&rtk_md_path, RTK_SLIM, RTK_MD, verbose)?; let opencode_plugin_path = if install_opencode { let path = prepare_opencode_plugin_path()?; @@ -920,13 +843,8 @@ fn run_default_mode( let migrated = patch_claude_md(&claude_md_path, verbose)?; // 4. Print success message - let hook_status = if hook_changed { - "installed/updated" - } else { - "already up to date" - }; - println!("\nRTK hook {} (global).\n", hook_status); - println!(" Hook: {}", hook_path.display()); + println!("\nRTK hook registered (global).\n"); + println!(" Command: {}", CLAUDE_HOOK_COMMAND); println!(" RTK.md: {} (10 lines)", rtk_md_path.display()); if let Some(path) = &opencode_plugin_path { println!(" OpenCode: {}", path.display()); @@ -938,13 +856,14 @@ fn run_default_mode( println!(" replaced with @RTK.md (10 lines)"); } - // 5. Patch settings.json - let patch_result = patch_settings_json(&hook_path, patch_mode, verbose, install_opencode)?; + // 5. Patch settings.json with binary command + let patch_result = + patch_settings_json_command(CLAUDE_HOOK_COMMAND, patch_mode, verbose, install_opencode)?; // Report result match patch_result { PatchResult::Patched => { - // Already printed by patch_settings_json + // Already printed by patch_settings_json_command } PatchResult::AlreadyPresent => { println!("\n settings.json: hook already present"); @@ -955,7 +874,7 @@ fn run_default_mode( } } PatchResult::Declined | PatchResult::Skipped => { - // Manual instructions already printed by patch_settings_json + // Manual instructions already printed } } @@ -967,6 +886,119 @@ fn run_default_mode( Ok(()) } +/// Migrate old hook script to new binary command. +/// Deletes `~/.claude/hooks/rtk-rewrite.sh` and `.rtk-hook.sha256` if present, +/// and removes the stale settings.json entry so the new `rtk hook claude` entry +/// can be registered. +fn migrate_old_hook_script(verbose: u8) { + if let Some(home) = dirs::home_dir() { + let old_hook = home + .join(CLAUDE_DIR) + .join(HOOKS_SUBDIR) + .join(REWRITE_HOOK_FILE); + if old_hook.exists() { + if let Err(e) = std::fs::remove_file(&old_hook) { + if verbose > 0 { + eprintln!(" [warn] Failed to remove old hook script: {e}"); + } + } else { + if verbose > 0 { + eprintln!(" [ok] Removed old hook script: {}", old_hook.display()); + } + // Clean up the stale settings.json entry that pointed to the deleted script + if let Err(e) = remove_legacy_settings_entries(verbose) { + if verbose > 0 { + eprintln!(" [warn] Failed to clean legacy settings.json entry: {e}"); + } + } + } + } + // Remove legacy hash file + let hash_file = home + .join(CLAUDE_DIR) + .join(HOOKS_SUBDIR) + .join(".rtk-hook.sha256"); + if hash_file.exists() { + let _ = std::fs::remove_file(&hash_file); + } + // Remove Cursor legacy hook + let cursor_hook = home.join(".cursor").join("hooks").join(REWRITE_HOOK_FILE); + if cursor_hook.exists() { + let _ = std::fs::remove_file(&cursor_hook); + } + } +} + +/// Remove only legacy `rtk-rewrite.sh` entries from settings.json. +/// Preserves any existing `rtk hook claude` entries (new format). +fn remove_legacy_settings_entries(verbose: u8) -> Result<()> { + let claude_dir = resolve_claude_dir()?; + let settings_path = claude_dir.join(SETTINGS_JSON); + + if !settings_path.exists() { + return Ok(()); + } + + let content = fs::read_to_string(&settings_path) + .with_context(|| format!("Failed to read {}", settings_path.display()))?; + if content.trim().is_empty() { + return Ok(()); + } + + let mut root: serde_json::Value = serde_json::from_str(&content) + .with_context(|| format!("Failed to parse {}", settings_path.display()))?; + + if !remove_legacy_hook_entries_from_json(&mut root) { + return Ok(()); + } + + // Backup before modifying + let backup_path = settings_path.with_extension("json.bak"); + fs::copy(&settings_path, &backup_path) + .with_context(|| format!("Failed to backup to {}", backup_path.display()))?; + + let serialized = + serde_json::to_string_pretty(&root).context("Failed to serialize settings.json")?; + atomic_write(&settings_path, &serialized)?; + + if verbose > 0 { + eprintln!(" [ok] Removed legacy rtk-rewrite.sh entry from settings.json"); + } + Ok(()) +} + +/// Remove only legacy `rtk-rewrite.sh` hook entries from a parsed settings.json. +/// Returns true if any entries were removed. +/// Does NOT remove `rtk hook claude` entries — those are the new format. +fn remove_legacy_hook_entries_from_json(root: &mut serde_json::Value) -> bool { + let pre_tool_use_array = match root + .get_mut("hooks") + .and_then(|h| h.get_mut(PRE_TOOL_USE_KEY)) + .and_then(|p| p.as_array_mut()) + { + Some(arr) => arr, + None => return false, + }; + + let original_len = pre_tool_use_array.len(); + pre_tool_use_array.retain(|entry| { + let dominated_by_legacy = entry + .get("hooks") + .and_then(|h| h.as_array()) + .map(|hooks| { + hooks.iter().all(|hook| { + hook.get("command") + .and_then(|c| c.as_str()) + .is_some_and(|cmd| cmd.contains(REWRITE_HOOK_FILE)) + }) + }) + .unwrap_or(false); + !dominated_by_legacy + }); + + pre_tool_use_array.len() < original_len +} + /// Generate .rtk/filters.toml template in the current directory if not present. fn generate_project_filters_template(verbose: u8) -> Result<()> { let rtk_dir = std::path::Path::new(".rtk"); @@ -994,7 +1026,7 @@ fn generate_project_filters_template(verbose: u8) -> Result<()> { /// Generate ~/.config/rtk/filters.toml template if not present. fn generate_global_filters_template(verbose: u8) -> Result<()> { let config_dir = dirs::config_dir().unwrap_or_else(|| std::path::PathBuf::from(".config")); - let rtk_dir = config_dir.join("rtk"); + let rtk_dir = config_dir.join(crate::core::constants::RTK_DATA_DIR); let path = rtk_dir.join("filters.toml"); if path.exists() { @@ -1017,17 +1049,6 @@ fn generate_global_filters_template(verbose: u8) -> Result<()> { } /// Hook-only mode: just the hook, no RTK.md -#[cfg(not(unix))] -fn run_hook_only_mode( - _global: bool, - _patch_mode: PatchMode, - _verbose: u8, - _install_opencode: bool, -) -> Result<()> { - anyhow::bail!("Hook install requires Unix (macOS/Linux). Use WSL or --claude-md mode.") -} - -#[cfg(unix)] fn run_hook_only_mode( global: bool, patch_mode: PatchMode, @@ -1040,9 +1061,8 @@ fn run_hook_only_mode( return Ok(()); } - // Prepare and install hook - let (_hook_dir, hook_path) = prepare_hook_paths()?; - let hook_changed = ensure_hook_installed(&hook_path, verbose)?; + // Migrate old hook script if present + migrate_old_hook_script(verbose); let opencode_plugin_path = if install_opencode { let path = prepare_opencode_plugin_path()?; @@ -1052,13 +1072,8 @@ fn run_hook_only_mode( None }; - let hook_status = if hook_changed { - "installed/updated" - } else { - "already up to date" - }; - println!("\nRTK hook {} (hook-only mode).\n", hook_status); - println!(" Hook: {}", hook_path.display()); + println!("\nRTK hook registered (hook-only mode).\n"); + println!(" Command: {}", CLAUDE_HOOK_COMMAND); if let Some(path) = &opencode_plugin_path { println!(" OpenCode: {}", path.display()); } @@ -1066,13 +1081,14 @@ fn run_hook_only_mode( " Note: No RTK.md created. Claude won't know about meta commands (gain, discover, proxy)." ); - // Patch settings.json - let patch_result = patch_settings_json(&hook_path, patch_mode, verbose, install_opencode)?; + // Patch settings.json with binary command + let patch_result = + patch_settings_json_command(CLAUDE_HOOK_COMMAND, patch_mode, verbose, install_opencode)?; // Report result match patch_result { PatchResult::Patched => { - // Already printed by patch_settings_json + // Already printed by patch_settings_json_command } PatchResult::AlreadyPresent => { println!("\n settings.json: hook already present"); @@ -1083,7 +1099,7 @@ fn run_hook_only_mode( } } PatchResult::Declined | PatchResult::Skipped => { - // Manual instructions already printed by patch_settings_json + // Manual instructions already printed } } @@ -1095,9 +1111,9 @@ fn run_hook_only_mode( /// Legacy mode: full 137-line injection into CLAUDE.md fn run_claude_md_mode(global: bool, verbose: u8, install_opencode: bool) -> Result<()> { let path = if global { - resolve_claude_dir()?.join("CLAUDE.md") + resolve_claude_dir()?.join(CLAUDE_MD) } else { - PathBuf::from("CLAUDE.md") + PathBuf::from(CLAUDE_MD) }; if global { @@ -1245,14 +1261,103 @@ fn run_windsurf_mode(verbose: u8) -> Result<()> { Ok(()) } +// ─── Kilo Code support ──────────────────────────────────────── + +const KILOCODE_RULES: &str = include_str!("../../hooks/kilocode/rules.md"); + +pub fn run_kilocode_mode(verbose: u8) -> Result<()> { + run_kilocode_mode_at(&std::env::current_dir()?, verbose) +} + +fn run_kilocode_mode_at(base_dir: &Path, verbose: u8) -> Result<()> { + // Kilo Code reads .kilocode/rules/ from the project root (workspace-scoped) + let target_dir = base_dir.join(".kilocode/rules"); + let rules_path = target_dir.join("rtk-rules.md"); + + let existing = fs::read_to_string(&rules_path).unwrap_or_default(); + if existing.contains("RTK") || existing.contains("rtk") { + println!("\nRTK already configured for Kilo Code in this project.\n"); + println!(" Rules: .kilocode/rules/rtk-rules.md (already present)"); + } else { + fs::create_dir_all(&target_dir).context("Failed to create .kilocode/rules directory")?; + let new_content = if existing.trim().is_empty() { + KILOCODE_RULES.to_string() + } else { + format!("{}\n\n{}", existing.trim(), KILOCODE_RULES) + }; + fs::write(&rules_path, &new_content) + .context("Failed to write .kilocode/rules/rtk-rules.md")?; + + if verbose > 0 { + eprintln!("Wrote .kilocode/rules/rtk-rules.md"); + } + + println!("\nRTK configured for Kilo Code.\n"); + println!(" Rules: .kilocode/rules/rtk-rules.md (installed)"); + } + println!(" Kilo Code will now use rtk commands for token savings."); + println!(" Test with: git status\n"); + + Ok(()) +} + +// ─── Google Antigravity support ─────────────────────────────── + +const ANTIGRAVITY_RULES: &str = include_str!("../../hooks/antigravity/rules.md"); + +pub fn run_antigravity_mode(verbose: u8) -> Result<()> { + run_antigravity_mode_at(&std::env::current_dir()?, verbose) +} + +fn run_antigravity_mode_at(base_dir: &Path, verbose: u8) -> Result<()> { + // Antigravity reads .agents/rules/ from the project root (workspace-scoped) + let target_dir = base_dir.join(".agents/rules"); + let rules_path = target_dir.join("antigravity-rtk-rules.md"); + + let existing = fs::read_to_string(&rules_path).unwrap_or_default(); + if existing.contains("RTK") || existing.contains("rtk") { + println!("\nRTK already configured for Antigravity in this project.\n"); + println!(" Rules: .agents/rules/antigravity-rtk-rules.md (already present)"); + } else { + fs::create_dir_all(&target_dir).context("Failed to create .agents/rules directory")?; + let new_content = if existing.trim().is_empty() { + ANTIGRAVITY_RULES.to_string() + } else { + format!("{}\n\n{}", existing.trim(), ANTIGRAVITY_RULES) + }; + fs::write(&rules_path, &new_content) + .context("Failed to write .agents/rules/antigravity-rtk-rules.md")?; + + if verbose > 0 { + eprintln!("Wrote .agents/rules/antigravity-rtk-rules.md"); + } + + println!("\nRTK configured for Google Antigravity.\n"); + println!(" Rules: .agents/rules/antigravity-rtk-rules.md (installed)"); + } + println!(" Antigravity will now use rtk commands for token savings."); + println!(" Test with: git status\n"); + + Ok(()) +} + fn run_codex_mode(global: bool, verbose: u8) -> Result<()> { let (agents_md_path, rtk_md_path) = if global { let codex_dir = resolve_codex_dir()?; - (codex_dir.join("AGENTS.md"), codex_dir.join("RTK.md")) + (codex_dir.join(AGENTS_MD), codex_dir.join(RTK_MD)) } else { - (PathBuf::from("AGENTS.md"), PathBuf::from("RTK.md")) + (PathBuf::from(AGENTS_MD), PathBuf::from(RTK_MD)) }; + run_codex_mode_with_paths(agents_md_path, rtk_md_path, global, verbose) +} + +fn run_codex_mode_with_paths( + agents_md_path: PathBuf, + rtk_md_path: PathBuf, + global: bool, + verbose: u8, +) -> Result<()> { if global { if let Some(parent) = agents_md_path.parent() { fs::create_dir_all(parent).with_context(|| { @@ -1264,15 +1369,28 @@ fn run_codex_mode(global: bool, verbose: u8) -> Result<()> { } } - write_if_changed(&rtk_md_path, RTK_SLIM_CODEX, "RTK.md", verbose)?; - let added_ref = patch_agents_md(&agents_md_path, verbose)?; + // ISSUE #892: In global mode, use absolute path so @RTK.md resolves + // from any CWD (worktrees, nested projects). Codex resolves @ references + // relative to CWD, not the AGENTS.md file location. + let rtk_md_ref = if global { + codex_rtk_md_ref( + rtk_md_path + .parent() + .context("RTK.md path missing parent directory")?, + ) + } else { + RTK_MD_REF.to_string() + }; + + write_if_changed(&rtk_md_path, RTK_SLIM_CODEX, RTK_MD, verbose)?; + let added_ref = patch_agents_md(&agents_md_path, &rtk_md_ref, verbose)?; println!("\nRTK configured for Codex CLI.\n"); println!(" RTK.md: {}", rtk_md_path.display()); if added_ref { - println!(" AGENTS.md: @RTK.md reference added"); + println!(" AGENTS.md: {} reference added", rtk_md_ref); } else { - println!(" AGENTS.md: @RTK.md reference already present"); + println!(" AGENTS.md: {} reference already present", rtk_md_ref); } if global { println!( @@ -1375,7 +1493,7 @@ fn patch_claude_md(path: &Path, verbose: u8) -> Result { } // Check if @RTK.md already present - if content.contains("@RTK.md") { + if content.contains(RTK_MD_REF) { if verbose > 0 { eprintln!("@RTK.md reference already present in CLAUDE.md"); } @@ -1401,8 +1519,8 @@ fn patch_claude_md(path: &Path, verbose: u8) -> Result { Ok(migrated) } -/// Patch AGENTS.md: add @RTK.md, migrate old inline block if present -fn patch_agents_md(path: &Path, verbose: u8) -> Result { +/// Patch AGENTS.md: add @RTK.md (or absolute path), migrate old inline block if present +fn patch_agents_md(path: &Path, rtk_md_ref: &str, verbose: u8) -> Result { let mut content = if path.exists() { fs::read_to_string(path) .with_context(|| format!("Failed to read AGENTS.md: {}", path.display()))? @@ -1422,9 +1540,21 @@ fn patch_agents_md(path: &Path, verbose: u8) -> Result { } } - if content.contains("@RTK.md") { + // ISSUE #892: Check for both relative and absolute @RTK.md references + if content.contains(RTK_MD_REF) || content.contains(rtk_md_ref) { if verbose > 0 { - eprintln!("@RTK.md reference already present in AGENTS.md"); + eprintln!("{} reference already present in AGENTS.md", rtk_md_ref); + } + // ISSUE #892: Migrate old relative @RTK.md to absolute path if needed + if rtk_md_ref != RTK_MD_REF && content.contains(RTK_MD_REF) && !content.contains(rtk_md_ref) + { + content = content.replace(RTK_MD_REF, rtk_md_ref); + atomic_write(path, &content) + .with_context(|| format!("Failed to write AGENTS.md: {}", path.display()))?; + if verbose > 0 { + eprintln!("Migrated {} to {}", RTK_MD_REF, rtk_md_ref); + } + return Ok(true); } if migrated { atomic_write(path, &content) @@ -1434,34 +1564,44 @@ fn patch_agents_md(path: &Path, verbose: u8) -> Result { } let new_content = if content.is_empty() { - "@RTK.md\n".to_string() + format!("{}\n", rtk_md_ref) } else { - format!("{}\n\n@RTK.md\n", content.trim()) + format!("{}\n\n{}\n", content.trim(), rtk_md_ref) }; atomic_write(path, &new_content) .with_context(|| format!("Failed to write AGENTS.md: {}", path.display()))?; if verbose > 0 { - eprintln!("Added @RTK.md reference to AGENTS.md"); + eprintln!("Added {} reference to AGENTS.md", rtk_md_ref); } Ok(true) } -fn remove_rtk_reference_from_agents(path: &Path, verbose: u8) -> Result { +fn has_rtk_reference(content: &str, refs: &[&str]) -> bool { + content + .lines() + .map(str::trim) + .any(|line| refs.contains(&line)) +} + +fn remove_rtk_reference_from_agents(path: &Path, refs: &[&str], verbose: u8) -> Result { if !path.exists() { return Ok(false); } let content = fs::read_to_string(path) .with_context(|| format!("Failed to read AGENTS.md: {}", path.display()))?; - if !content.contains("@RTK.md") { + if !has_rtk_reference(&content, refs) { return Ok(false); } let new_content = content .lines() - .filter(|line| !line.trim().starts_with("@RTK.md")) + .filter(|line| { + let trimmed = line.trim(); + !refs.contains(&trimmed) + }) .collect::>() .join("\n"); let cleaned = clean_double_blanks(&new_content); @@ -1470,7 +1610,7 @@ fn remove_rtk_reference_from_agents(path: &Path, verbose: u8) -> Result { if verbose > 0 { eprintln!( - "Removed @RTK.md reference from AGENTS.md: {}", + "Removed RTK.md reference from AGENTS.md: {}", path.display() ); } @@ -1516,26 +1656,45 @@ fn remove_rtk_block(content: &str) -> (String, bool) { } } -/// Resolve ~/.claude directory with proper home expansion -fn resolve_claude_dir() -> Result { +fn resolve_home_subdir(subdir: &str) -> Result { dirs::home_dir() - .map(|h| h.join(".claude")) + .map(|h| h.join(subdir)) .context("Cannot determine home directory. Is $HOME set?") } -/// Resolve ~/.codex directory with proper home expansion +fn resolve_claude_dir() -> Result { + if let Ok(dir) = std::env::var("RTK_CLAUDE_DIR") { + return Ok(PathBuf::from(dir)); + } + resolve_home_subdir(CLAUDE_DIR) +} + fn resolve_codex_dir() -> Result { - dirs::home_dir() - .map(|h| h.join(".codex")) - .context("Cannot determine home directory. Is $HOME set?") + resolve_codex_dir_from( + std::env::var_os("CODEX_HOME").map(PathBuf::from), + dirs::home_dir(), + ) +} + +fn resolve_codex_dir_from( + codex_home: Option, + home_dir: Option, +) -> Result { + if let Some(path) = codex_home.filter(|path| !path.as_os_str().is_empty()) { + return Ok(path); + } + + home_dir + .map(|home| home.join(CODEX_DIR)) + .context("Cannot determine Codex config directory. Set $CODEX_HOME or $HOME.") +} + +fn codex_rtk_md_ref(codex_dir: &Path) -> String { + format!("@{}", codex_dir.join(RTK_MD).display()) } -/// Resolve OpenCode config directory (~/.config/opencode) -/// OpenCode uses ~/.config/opencode on all platforms (XDG convention), -/// NOT the macOS-native ~/Library/Application Support/. + fn resolve_opencode_dir() -> Result { - dirs::home_dir() - .map(|h| h.join(".config").join("opencode")) - .context("Cannot determine home directory. Is $HOME set?") + resolve_home_subdir(".config/opencode") } /// Return OpenCode plugin path: ~/.config/opencode/plugins/rtk.ts @@ -1583,51 +1742,40 @@ fn remove_opencode_plugin(verbose: u8) -> Result> { // ─── Cursor Agent support ───────────────────────────────────────────── -/// Resolve ~/.cursor directory fn resolve_cursor_dir() -> Result { - dirs::home_dir() - .map(|h| h.join(".cursor")) - .context("Cannot determine home directory. Is $HOME set?") + resolve_home_subdir(".cursor") } -/// Install Cursor hooks: hook script + hooks.json +/// Install Cursor hooks: register binary command in hooks.json fn install_cursor_hooks(verbose: u8) -> Result<()> { let cursor_dir = resolve_cursor_dir()?; - let hooks_dir = cursor_dir.join("hooks"); - fs::create_dir_all(&hooks_dir).with_context(|| { - format!( - "Failed to create Cursor hooks directory: {}", - hooks_dir.display() - ) - })?; - - // 1. Write hook script - let hook_path = hooks_dir.join("rtk-rewrite.sh"); - let hook_changed = write_if_changed(&hook_path, CURSOR_REWRITE_HOOK, "Cursor hook", verbose)?; - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - fs::set_permissions(&hook_path, fs::Permissions::from_mode(0o755)).with_context(|| { - format!( - "Failed to set Cursor hook permissions: {}", - hook_path.display() - ) - })?; + // Migrate old hook script if present + let old_hook = cursor_dir.join("hooks").join(REWRITE_HOOK_FILE); + if old_hook.exists() { + let _ = fs::remove_file(&old_hook); + if verbose > 0 { + eprintln!( + " [ok] Removed old Cursor hook script: {}", + old_hook.display() + ); + } + // Clean stale hooks.json entry pointing to the deleted script + let hooks_json_path = cursor_dir.join(HOOKS_JSON); + if let Err(e) = remove_legacy_cursor_hooks_json_entries(&hooks_json_path, verbose) { + if verbose > 0 { + eprintln!(" [warn] Failed to clean legacy Cursor hooks.json entry: {e}"); + } + } } - // 2. Create or patch hooks.json - let hooks_json_path = cursor_dir.join("hooks.json"); + // Create or patch hooks.json with binary command + let hooks_json_path = cursor_dir.join(HOOKS_JSON); let patched = patch_cursor_hooks_json(&hooks_json_path, verbose)?; // Report - let hook_status = if hook_changed { - "installed/updated" - } else { - "already up to date" - }; - println!("\nCursor hook {} (global).\n", hook_status); - println!(" Hook: {}", hook_path.display()); + println!("\nCursor hook registered (global).\n"); + println!(" Command: {}", CURSOR_HOOK_COMMAND); println!(" hooks.json: {}", hooks_json_path.display()); if patched { @@ -1665,8 +1813,7 @@ fn patch_cursor_hooks_json(path: &Path, verbose: u8) -> Result { return Ok(false); } - // Insert the RTK preToolUse entry - insert_cursor_hook_entry(&mut root); + insert_cursor_hook_entry(&mut root)?; // Backup if exists if path.exists() { @@ -1687,6 +1834,7 @@ fn patch_cursor_hooks_json(path: &Path, verbose: u8) -> Result { } /// Check if RTK preToolUse hook is already present in Cursor hooks.json +/// Matches on legacy rtk-rewrite.sh path OR new `rtk hook cursor` command fn cursor_hook_already_present(root: &serde_json::Value) -> bool { let hooks = match root .get("hooks") @@ -1701,40 +1849,93 @@ fn cursor_hook_already_present(root: &serde_json::Value) -> bool { entry .get("command") .and_then(|c| c.as_str()) - .is_some_and(|cmd| cmd.contains("rtk-rewrite.sh")) + .is_some_and(|cmd| cmd.contains(REWRITE_HOOK_FILE) || cmd == CURSOR_HOOK_COMMAND) }) } /// Insert RTK preToolUse entry into Cursor hooks.json -fn insert_cursor_hook_entry(root: &mut serde_json::Value) { +fn insert_cursor_hook_entry(root: &mut serde_json::Value) -> Result<()> { let root_obj = match root.as_object_mut() { Some(obj) => obj, None => { *root = serde_json::json!({ "version": 1 }); - root.as_object_mut() - .expect("Just created object, must succeed") + root.as_object_mut().expect("just-created json object") } }; - // Ensure version key root_obj.entry("version").or_insert(serde_json::json!(1)); let hooks = root_obj .entry("hooks") .or_insert_with(|| serde_json::json!({})) .as_object_mut() - .expect("hooks must be an object"); + .context("hooks value is not an object")?; let pre_tool_use = hooks .entry("preToolUse") .or_insert_with(|| serde_json::json!([])) .as_array_mut() - .expect("preToolUse must be an array"); + .context("preToolUse value is not an array")?; pre_tool_use.push(serde_json::json!({ - "command": "./hooks/rtk-rewrite.sh", + "command": CURSOR_HOOK_COMMAND, "matcher": "Shell" })); + Ok(()) +} + +/// Remove only legacy `rtk-rewrite.sh` entries from Cursor hooks.json. +/// Preserves any existing `rtk hook cursor` entries (new format). +fn remove_legacy_cursor_hooks_json_entries(path: &Path, verbose: u8) -> Result<()> { + if !path.exists() { + return Ok(()); + } + + let content = + fs::read_to_string(path).with_context(|| format!("Failed to read {}", path.display()))?; + if content.trim().is_empty() { + return Ok(()); + } + + let mut root: serde_json::Value = serde_json::from_str(&content) + .with_context(|| format!("Failed to parse {}", path.display()))?; + + if !remove_legacy_cursor_hook_entries_from_json(&mut root) { + return Ok(()); + } + + let serialized = + serde_json::to_string_pretty(&root).context("Failed to serialize hooks.json")?; + atomic_write(path, &serialized)?; + + if verbose > 0 { + eprintln!(" [ok] Removed legacy rtk-rewrite.sh entry from Cursor hooks.json"); + } + Ok(()) +} + +/// Remove only legacy `rtk-rewrite.sh` entries from parsed Cursor hooks.json. +/// Returns true if any entries were removed. +/// Does NOT remove `rtk hook cursor` entries — those are the new format. +fn remove_legacy_cursor_hook_entries_from_json(root: &mut serde_json::Value) -> bool { + let pre_tool_use = match root + .get_mut("hooks") + .and_then(|h| h.get_mut("preToolUse")) + .and_then(|p| p.as_array_mut()) + { + Some(arr) => arr, + None => return false, + }; + + let original_len = pre_tool_use.len(); + pre_tool_use.retain(|entry| { + !entry + .get("command") + .and_then(|c| c.as_str()) + .is_some_and(|cmd| cmd.contains(REWRITE_HOOK_FILE)) + }); + + pre_tool_use.len() < original_len } /// Remove Cursor RTK artifacts: hook script + hooks.json entry @@ -1743,7 +1944,7 @@ fn remove_cursor_hooks(verbose: u8) -> Result> { let mut removed = Vec::new(); // 1. Remove hook script - let hook_path = cursor_dir.join("hooks").join("rtk-rewrite.sh"); + let hook_path = cursor_dir.join(HOOKS_SUBDIR).join(REWRITE_HOOK_FILE); if hook_path.exists() { fs::remove_file(&hook_path) .with_context(|| format!("Failed to remove Cursor hook: {}", hook_path.display()))?; @@ -1751,7 +1952,7 @@ fn remove_cursor_hooks(verbose: u8) -> Result> { } // 2. Remove RTK entry from hooks.json - let hooks_json_path = cursor_dir.join("hooks.json"); + let hooks_json_path = cursor_dir.join(HOOKS_JSON); if hooks_json_path.exists() { let content = fs::read_to_string(&hooks_json_path) .with_context(|| format!("Failed to read {}", hooks_json_path.display()))?; @@ -1781,6 +1982,7 @@ fn remove_cursor_hooks(verbose: u8) -> Result> { /// Remove RTK preToolUse entry from Cursor hooks.json /// Returns true if entry was found and removed +/// Matches both legacy script path and new binary command fn remove_cursor_hook_from_json(root: &mut serde_json::Value) -> bool { let pre_tool_use = match root .get_mut("hooks") @@ -1796,7 +1998,7 @@ fn remove_cursor_hook_from_json(root: &mut serde_json::Value) -> bool { !entry .get("command") .and_then(|c| c.as_str()) - .is_some_and(|cmd| cmd.contains("rtk-rewrite.sh")) + .is_some_and(|cmd| cmd.contains(REWRITE_HOOK_FILE) || cmd == CURSOR_HOOK_COMMAND) }); pre_tool_use.len() < original_len @@ -1813,19 +2015,33 @@ pub fn show_config(codex: bool) -> Result<()> { fn show_claude_config() -> Result<()> { let claude_dir = resolve_claude_dir()?; - let hook_path = claude_dir.join("hooks").join("rtk-rewrite.sh"); - let rtk_md_path = claude_dir.join("RTK.md"); - let global_claude_md = claude_dir.join("CLAUDE.md"); - let local_claude_md = PathBuf::from("CLAUDE.md"); + let hook_path = claude_dir.join(HOOKS_SUBDIR).join(REWRITE_HOOK_FILE); + let rtk_md_path = claude_dir.join(RTK_MD); + let global_claude_md = claude_dir.join(CLAUDE_MD); + let local_claude_md = PathBuf::from(CLAUDE_MD); println!("rtk Configuration:\n"); - // Check hook - if hook_path.exists() { - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - let metadata = fs::metadata(&hook_path)?; + // Check hook: prefer binary command detection, fall back to script file + let settings_path = claude_dir.join(SETTINGS_JSON); + let binary_hook_registered = if settings_path.exists() { + let content = fs::read_to_string(&settings_path).unwrap_or_default(); + if let Ok(root) = serde_json::from_str::(&content) { + hook_already_present(&root, CLAUDE_HOOK_COMMAND) + } else { + false + } + } else { + false + }; + + if binary_hook_registered { + println!("[ok] Hook: {} (native binary command)", CLAUDE_HOOK_COMMAND); + } else if hook_path.exists() { + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let metadata = fs::metadata(&hook_path)?; let perms = metadata.permissions(); let is_executable = perms.mode() & 0o111 != 0; @@ -1842,15 +2058,12 @@ fn show_claude_config() -> Result<()> { ); } else if !is_thin_delegator { println!( - "[warn] Hook: {} (outdated — inline logic, not thin delegator)", + "[warn] Hook: {} (outdated — run `rtk init -g` to upgrade to native binary)", hook_path.display() ); - println!( - " → Run `rtk init --global` to upgrade to the single source of truth hook" - ); } else if is_executable && has_guards { println!( - "[ok] Hook: {} (thin delegator, version {})", + "[warn] Hook: {} (legacy script v{} — run `rtk init -g` to upgrade)", hook_path.display(), hook_version ); @@ -1864,7 +2077,10 @@ fn show_claude_config() -> Result<()> { #[cfg(not(unix))] { - println!("[ok] Hook: {} (exists)", hook_path.display()); + println!( + "[warn] Hook: {} (legacy script — run `rtk init -g` to upgrade)", + hook_path.display() + ); } } else { println!("[--] Hook: not found"); @@ -1877,30 +2093,32 @@ fn show_claude_config() -> Result<()> { println!("[--] RTK.md: not found"); } - // Check hook integrity - match integrity::verify_hook_at(&hook_path) { - Ok(integrity::IntegrityStatus::Verified) => { - println!("[ok] Integrity: hook hash verified"); - } - Ok(integrity::IntegrityStatus::Tampered { .. }) => { - println!("[FAIL] Integrity: hook modified outside rtk init (run: rtk verify)"); - } - Ok(integrity::IntegrityStatus::NoBaseline) => { - println!("[warn] Integrity: no baseline hash (run: rtk init -g to establish)"); - } - Ok(integrity::IntegrityStatus::NotInstalled) - | Ok(integrity::IntegrityStatus::OrphanedHash) => { - // Don't show integrity line if hook isn't installed - } - Err(_) => { - println!("[warn] Integrity: check failed"); + // Check hook integrity (only relevant for legacy script hooks) + if hook_path.exists() && !binary_hook_registered { + match integrity::verify_hook_at(&hook_path) { + Ok(integrity::IntegrityStatus::Verified) => { + println!("[ok] Integrity: hook hash verified"); + } + Ok(integrity::IntegrityStatus::Tampered { .. }) => { + println!("[FAIL] Integrity: hook modified outside rtk init (run: rtk verify)"); + } + Ok(integrity::IntegrityStatus::NoBaseline) => { + println!("[warn] Integrity: no baseline hash (run: rtk init -g to establish)"); + } + Ok(integrity::IntegrityStatus::NotInstalled) + | Ok(integrity::IntegrityStatus::OrphanedHash) => { + // Don't show integrity line if hook isn't installed + } + Err(_) => { + println!("[warn] Integrity: check failed"); + } } } // Check global CLAUDE.md if global_claude_md.exists() { let content = fs::read_to_string(&global_claude_md)?; - if content.contains("@RTK.md") { + if content.contains(RTK_MD_REF) { println!("[ok] Global (~/.claude/CLAUDE.md): @RTK.md reference"); } else if content.contains("\n\n### Acceptance Criteria\n- [ ] `rtk glab ci list` shows compact pipeline summary\n- [ ] `rtk glab ci status` shows current pipeline status\n- [ ] Token savings >= 80%\n\n---\n\n[![status](https://img.shields.io/badge/status-in_progress-yellow)](https://example.com)\n" + }, + { + "iid": 150, + "title": "rtk cargo test shows full output when no failures", + "state": "opened", + "author": {"username": "bob_report", "name": "Bob Reporter", "id": 100}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/issues/150", + "created_at": "2026-02-28T08:00:00Z", + "updated_at": "2026-03-02T16:00:00Z", + "labels": ["bug", "cargo"], + "assignees": [{"username": "dave_fix"}], + "description": "When all tests pass, `rtk cargo test` still shows verbose compilation output instead of just the summary line.\n\n### Steps to Reproduce\n1. Run `rtk cargo test` in a project with all passing tests\n2. Observe that compiler output is included\n\n### Expected\nOnly show test summary when all tests pass.\n\n### Actual\nFull compiler warnings and test output shown." + }, + { + "iid": 145, + "title": "Add Helm CLI support", + "state": "opened", + "author": {"username": "carol_infra", "name": "Carol Infra", "id": 200}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/issues/145", + "created_at": "2026-02-25T12:00:00Z", + "updated_at": "2026-03-04T09:00:00Z", + "labels": ["enhancement", "infra"], + "assignees": [], + "description": "Helm CLI outputs are verbose. Would be great to have RTK support for:\n- `helm list` (compact table)\n- `helm status` (summary only)\n- `helm install/upgrade` (ok confirmation)\n\nSimilar to how `rtk kubectl` works." + }, + { + "iid": 140, + "title": "Binary size increased 30% after Python/Go modules", + "state": "opened", + "author": {"username": "eve_perf", "name": "Eve Performance", "id": 300}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/issues/140", + "created_at": "2026-02-20T15:00:00Z", + "updated_at": "2026-02-22T10:00:00Z", + "labels": ["performance", "build"], + "assignees": [{"username": "frank_contrib"}], + "description": "After merging Python and Go support, stripped release binary went from 3.2MB to 4.1MB.\n\nInvestigate if we can:\n- Use feature flags to make modules optional\n- Reduce regex count (share patterns across modules)\n- Review serde usage (maybe avoid full JSON parsing for simple cases)" + }, + { + "iid": 135, + "title": "rtk gain --history shows wrong dates on macOS", + "state": "closed", + "author": {"username": "george_mac", "name": "George Mac", "id": 400}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/issues/135", + "created_at": "2026-02-15T09:00:00Z", + "updated_at": "2026-02-18T11:00:00Z", + "labels": ["bug", "macos"], + "assignees": [{"username": "alice_dev"}], + "description": "On macOS, `rtk gain --history` shows dates in UTC instead of local timezone.\n\nFixed in v0.23.1." + }, + { + "iid": 130, + "title": "Support TOML-based filter DSL", + "state": "opened", + "author": {"username": "heidi_arch", "name": "Heidi Architect", "id": 500}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/issues/130", + "created_at": "2026-02-10T08:00:00Z", + "updated_at": "2026-02-12T16:00:00Z", + "labels": ["enhancement", "architecture"], + "assignees": [], + "description": "Instead of writing Rust code for each new filter, allow users to define filters in TOML.\n\n```toml\n[[filter]]\ncommand = \"terraform plan\"\npattern = \"^(Plan|Apply|Error):\"\nformat = \"compact\"\n```\n\nThis would make RTK extensible without recompilation." + }, + { + "iid": 125, + "title": "Improve error messages for missing commands", + "state": "closed", + "author": {"username": "ivan_docs", "name": "Ivan Writer", "id": 600}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/issues/125", + "created_at": "2026-02-05T14:00:00Z", + "updated_at": "2026-02-06T09:00:00Z", + "labels": ["enhancement", "ux"], + "assignees": [{"username": "ivan_docs"}], + "description": "When the underlying command is not installed (e.g., `rtk glab mr list` without glab), the error message is confusing:\n\n```\nError: Failed to run glab mr list\n```\n\nShould say something like:\n```\nError: glab not found. Install it: https://gitlab.com/gitlab-org/cli\n```" + }, + { + "iid": 120, + "title": "Add rtk completion command for shell completions", + "state": "opened", + "author": {"username": "judy_shell", "name": "Judy Shell", "id": 700}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/issues/120", + "created_at": "2026-02-01T11:00:00Z", + "updated_at": "2026-02-03T15:00:00Z", + "labels": ["enhancement", "shell"], + "assignees": [], + "description": "Clap supports generating shell completions via `clap_complete`. Add a `rtk completion bash/zsh/fish` command.\n\nThis would help discoverability of available commands." + }, + { + "iid": 115, + "title": "rtk read crashes on binary files", + "state": "closed", + "author": {"username": "karl_refactor", "name": "Karl Refactorer", "id": 800}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/issues/115", + "created_at": "2026-01-28T10:00:00Z", + "updated_at": "2026-01-30T12:00:00Z", + "labels": ["bug", "crash"], + "assignees": [{"username": "dave_fix"}], + "description": "Running `rtk read /path/to/binary.exe` panics with:\n```\nthread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Utf8Error'\n```\n\nShould detect binary files and skip filtering." + }, + { + "iid": 110, + "title": "Track savings per project directory", + "state": "opened", + "author": {"username": "lisa_feat", "name": "Lisa Feature", "id": 900}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/issues/110", + "created_at": "2026-01-25T09:00:00Z", + "updated_at": "2026-01-27T14:00:00Z", + "labels": ["enhancement", "analytics"], + "assignees": [], + "description": "Currently `rtk gain` shows global stats. It would be useful to see savings broken down by project directory.\n\nProposal: store `cwd` in the tracking database and add `rtk gain --by-project` flag." + } +] diff --git a/tests/fixtures/glab_mr_list_raw.json b/tests/fixtures/glab_mr_list_raw.json new file mode 100644 index 000000000..c502b62a7 --- /dev/null +++ b/tests/fixtures/glab_mr_list_raw.json @@ -0,0 +1,182 @@ +[ + { + "iid": 314, + "title": "feat(glab): add GitLab CLI (glab) command support", + "state": "opened", + "author": {"username": "alice_dev", "name": "Alice Developer", "id": 42}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/merge_requests/314", + "created_at": "2026-03-01T10:00:00Z", + "updated_at": "2026-03-05T14:30:00Z", + "source_branch": "feat/glab-support", + "target_branch": "master", + "merge_status": "can_be_merged", + "draft": false, + "labels": ["enhancement", "cli"], + "assignees": [{"username": "alice_dev", "name": "Alice Developer"}], + "reviewers": [{"username": "bob_review"}, {"username": "carol_review"}], + "description": "## Summary\n\nAdd GitLab CLI support.\n\n\n\n## Changes\n- New module\n- MR/issue/CI filtering\n- Token savings 80-87%\n\n---\n\n[![CI](https://img.shields.io/badge/CI-passing-green)](https://ci.example.com)\n", + "head_pipeline": {"id": 98765, "status": "success", "ref": "feat/glab-support"} + }, + { + "iid": 310, + "title": "fix(git): handle merge commits in compact diff", + "state": "merged", + "author": {"username": "dave_fix", "name": "Dave Fixer", "id": 100}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/merge_requests/310", + "created_at": "2026-02-28T08:00:00Z", + "updated_at": "2026-03-02T16:00:00Z", + "source_branch": "fix/merge-commits", + "target_branch": "master", + "merge_status": "can_be_merged", + "draft": false, + "labels": ["bug", "git"], + "assignees": [{"username": "dave_fix"}], + "reviewers": [{"username": "eve_review"}], + "description": "Fix handling of merge commits in `compact_diff`. Previously, merge commits were being skipped entirely which lost context.\n\n### Test Plan\n- [x] Unit tests added\n- [x] Manual verification with merge-heavy repos\n", + "head_pipeline": {"id": 98700, "status": "success", "ref": "fix/merge-commits"} + }, + { + "iid": 305, + "title": "feat(aws): add AWS CLI module with token-optimized output", + "state": "opened", + "author": {"username": "frank_contrib", "name": "Frank Contributor", "id": 200}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/merge_requests/305", + "created_at": "2026-02-25T12:00:00Z", + "updated_at": "2026-03-04T09:00:00Z", + "source_branch": "feat/aws-cli", + "target_branch": "master", + "merge_status": "cannot_be_merged", + "draft": true, + "labels": ["enhancement", "infra"], + "assignees": [], + "reviewers": [{"username": "grace_review"}, {"username": "heidi_review"}], + "description": "Add AWS CLI support.\n\n![architecture](https://example.com/arch.png)\n\n## Commands\n- `rtk aws s3 ls`\n- `rtk aws ec2 describe-instances`\n- `rtk aws ecs list-services`\n\n## Token Savings\n| Command | Savings |\n|---------|--------|\n| s3 ls | 75% |\n| ec2 describe | 85% |\n| ecs list | 80% |\n", + "head_pipeline": {"id": 98650, "status": "failed", "ref": "feat/aws-cli"} + }, + { + "iid": 302, + "title": "chore(master): release 0.24.0", + "state": "merged", + "author": {"username": "release-bot", "name": "Release Bot", "id": 1}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/merge_requests/302", + "created_at": "2026-02-20T00:00:00Z", + "updated_at": "2026-02-20T01:00:00Z", + "source_branch": "release-please--branches--master", + "target_branch": "master", + "merge_status": "can_be_merged", + "draft": false, + "labels": ["release"], + "assignees": [], + "reviewers": [], + "description": "## [0.24.0](https://example.com/compare/v0.23.0...v0.24.0)\n\n### Features\n* feat(aws): add AWS CLI module\n* feat(psql): add PostgreSQL module\n\n### Bug Fixes\n* fix(playwright): fix JSON parser\n", + "head_pipeline": {"id": 98600, "status": "success", "ref": "release-please--branches--master"} + }, + { + "iid": 298, + "title": "docs: update README with Python and Go command examples", + "state": "merged", + "author": {"username": "ivan_docs", "name": "Ivan Writer", "id": 300}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/merge_requests/298", + "created_at": "2026-02-18T15:00:00Z", + "updated_at": "2026-02-19T10:00:00Z", + "source_branch": "docs/python-go-examples", + "target_branch": "master", + "merge_status": "can_be_merged", + "draft": false, + "labels": ["documentation"], + "assignees": [{"username": "ivan_docs"}], + "reviewers": [{"username": "judy_review"}], + "description": "Update README.md with comprehensive examples for:\n- Python commands (ruff, pytest, pip)\n- Go commands (go test, go build, golangci-lint)\n\nAll examples tested manually.", + "head_pipeline": null + }, + { + "iid": 295, + "title": "refactor: extract parser module from runner.rs", + "state": "closed", + "author": {"username": "karl_refactor", "name": "Karl Refactorer", "id": 400}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/merge_requests/295", + "created_at": "2026-02-15T09:00:00Z", + "updated_at": "2026-02-16T11:00:00Z", + "source_branch": "refactor/parser-module", + "target_branch": "master", + "merge_status": "can_be_merged", + "draft": false, + "labels": ["refactor"], + "assignees": [{"username": "karl_refactor"}], + "reviewers": [], + "description": "Extract parser logic from runner.rs into dedicated parser/ module.\n\n---\n\nThis was superseded by #300 which took a different approach.\n\n***\n", + "head_pipeline": {"id": 98500, "status": "canceled", "ref": "refactor/parser-module"} + }, + { + "iid": 290, + "title": "feat(tee): save raw output on failure for LLM re-read", + "state": "merged", + "author": {"username": "lisa_feat", "name": "Lisa Feature", "id": 500}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/merge_requests/290", + "created_at": "2026-02-10T08:00:00Z", + "updated_at": "2026-02-12T16:00:00Z", + "source_branch": "feat/tee-output", + "target_branch": "master", + "merge_status": "can_be_merged", + "draft": false, + "labels": ["enhancement"], + "assignees": [{"username": "lisa_feat"}], + "reviewers": [{"username": "mike_review"}], + "description": "## Tee Output Recovery\n\nSave raw unfiltered output on command failure.\nPrint one-line hint so LLMs can re-read instead of re-run.\n\n### Configuration\n```toml\n[tee]\nenabled = true\ndir = \"~/.local/share/rtk/tee\"\nmax_files = 20\nmax_size = 1048576\n```\n", + "head_pipeline": {"id": 98400, "status": "success", "ref": "feat/tee-output"} + }, + { + "iid": 285, + "title": "ci: add ARM64 Linux build to release workflow", + "state": "merged", + "author": {"username": "nancy_ci", "name": "Nancy CI", "id": 600}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/merge_requests/285", + "created_at": "2026-02-05T14:00:00Z", + "updated_at": "2026-02-06T09:00:00Z", + "source_branch": "ci/arm64-build", + "target_branch": "master", + "merge_status": "can_be_merged", + "draft": false, + "labels": ["ci"], + "assignees": [{"username": "nancy_ci"}], + "reviewers": [{"username": "oscar_review"}], + "description": "Add ARM64 Linux target to the release workflow.\n\n- Uses `cross` for cross-compilation\n- Generates `.deb` and `.rpm` packages\n- Tested on Raspberry Pi 4 and AWS Graviton", + "head_pipeline": {"id": 98300, "status": "success", "ref": "ci/arm64-build"} + }, + { + "iid": 280, + "title": "fix(vitest): handle watch mode output gracefully", + "state": "opened", + "author": {"username": "peter_bugfix", "name": "Peter Bugfix", "id": 700}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/merge_requests/280", + "created_at": "2026-02-01T11:00:00Z", + "updated_at": "2026-02-03T15:00:00Z", + "source_branch": "fix/vitest-watch", + "target_branch": "master", + "merge_status": "unchecked", + "draft": false, + "labels": ["bug", "vitest"], + "assignees": [{"username": "peter_bugfix"}], + "reviewers": [], + "description": "When vitest runs in watch mode, output is continuous and doesn't have a clear end marker. This fix detects watch mode and falls back to passthrough.\n\n\n", + "head_pipeline": {"id": 98200, "status": "running", "ref": "fix/vitest-watch"} + }, + { + "iid": 275, + "title": "feat(discover): add rtk discover command for missed savings analysis", + "state": "merged", + "author": {"username": "quinn_dev", "name": "Quinn Developer", "id": 800}, + "web_url": "https://gitlab.example.com/acme/toolkit/-/merge_requests/275", + "created_at": "2026-01-28T10:00:00Z", + "updated_at": "2026-01-30T12:00:00Z", + "source_branch": "feat/discover", + "target_branch": "master", + "merge_status": "can_be_merged", + "draft": false, + "labels": ["enhancement", "analytics"], + "assignees": [{"username": "quinn_dev"}], + "reviewers": [{"username": "rachel_review"}, {"username": "sam_review"}], + "description": "Add `rtk discover` command that scans Claude Code JSONL sessions and reports missed savings opportunities.\n\n## Features\n- Classifies commands as Supported/Unsupported/Ignored\n- Groups by category with estimated token savings\n- Reports top missed commands\n\n## Example\n```\n$ rtk discover\nAnalyzed 1,234 commands across 45 sessions\n\nMissed savings by category:\n Git: 234 commands, ~16,800 tokens\n Cargo: 89 commands, ~7,120 tokens\n```\n", + "head_pipeline": {"id": 98100, "status": "success", "ref": "feat/discover"} + } +] diff --git a/tests/fixtures/glab_release_list_raw.txt b/tests/fixtures/glab_release_list_raw.txt new file mode 100644 index 000000000..a919148b1 --- /dev/null +++ b/tests/fixtures/glab_release_list_raw.txt @@ -0,0 +1,13 @@ +Showing 10 releases on acme/toolkit. + +Name Tag Created +v3.2.1 v3.2.1 about 2 days ago +v3.2.0 v3.2.0 about 1 week ago +v3.1.0 v3.1.0 about 3 weeks ago +v3.0.0 v3.0.0 about 1 month ago +v2.5.0 v2.5.0 about 3 months ago +v2.4.1 v2.4.1 about 5 months ago +v2.4.0 v2.4.0 about 6 months ago +v2.3.0 v2.3.0 about 9 months ago +v2.2.0 v2.2.0 about 1 year ago +v2.1.0 v2.1.0 about 2 years ago diff --git a/tests/fixtures/glab_release_view_raw.txt b/tests/fixtures/glab_release_view_raw.txt new file mode 100644 index 000000000..88893ee37 --- /dev/null +++ b/tests/fixtures/glab_release_view_raw.txt @@ -0,0 +1,30 @@ +Test Release v2.0 +alice_dev released this 3 days ago +abc1234 - v2.0.0 + + ## What's Changed + + - Added widget support + - Fixed authentication bug + + ### Contributors + + @alice_dev @bob_dev + + -------- + + Image: logo → https://example.com/logo.png + + + + +ASSETS +There are no assets for this release +SOURCES +https://gitlab.example.com/acme/toolkit/-/archive/v2.0.0/toolkit-v2.0.0.zip +https://gitlab.example.com/acme/toolkit/-/archive/v2.0.0/toolkit-v2.0.0.tar.gz +https://gitlab.example.com/acme/toolkit/-/archive/v2.0.0/toolkit-v2.0.0.tar.bz2 +https://gitlab.example.com/acme/toolkit/-/archive/v2.0.0/toolkit-v2.0.0.tar + + +View this release on GitLab at https://gitlab.example.com/acme/toolkit/-/releases/v2.0.0