diff --git a/CHANGELOG.md b/CHANGELOG.md index 5d21ccfdd..11bdfb93a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -**Highlights:** container infrastructure refactor (ADR-T-009), +**Highlights:** global command-line output contract (ADR-T-010), +container infrastructure refactor (ADR-T-009), native role-based authorization replacing Casbin (ADR-T-008), RSA-signed JWTs with revocation support (ADR-T-007), domain-scoped error system (ADR-T-006), MSRV raised to 1.88. @@ -15,6 +16,23 @@ error system (ADR-T-006), MSRV raised to 1.88. ### Breaking changes - MSRV raised from 1.85 to 1.88. +- First-party command-line entrypoints are now governed by ADR-T-010's + JSON-only output contract. Stdout is reserved for machine-readable result + data, stderr is reserved for machine-readable diagnostics/control records, and + stdout-producing commands refuse direct terminal stdout. `parse_torrent` now + emits JSON result data on stdout. `create_test_torrent`, + `import_tracker_statistics`, `seeder`, and `upgrade` now keep stdout empty + while reporting status and diagnostics as JSON on stderr. The container entry + script also reports validation failures, status records, utility failures, and + debug phase records as JSON/NDJSON on stderr instead of plain text or shell + trace output. Scripts that scraped previous plain-text command or startup + output must switch to exit codes and JSON/NDJSON stderr parsing. +- The `torrust-index` server's application logs now use JSON records on stderr + instead of the previous human-formatted tracing output. Log consumers should + parse stderr as NDJSON or pipe it through a JSON viewer. +- `torrust-index-auth-keypair` and `torrust-index-health-check` stdout JSON now + includes a top-level `schema` field. Scripts that expected an exact object + shape must tolerate or consume the schema field. - `database.connect_url` and `tracker.token` are now mandatory schema fields with no defaults. Supply them via env-var override (`TORRUST_INDEX_CONFIG_OVERRIDE_DATABASE__CONNECT_URL`, @@ -57,6 +75,56 @@ error system (ADR-T-006), MSRV raised to 1.88. domain-scoped enums: `AuthError`, `UserError`, `TorrentError`, `CategoryTagError`, with a thin `ApiError` wrapper (ADR-T-006). +### ADR-T-010 — Global command-line output contract + +#### Added + +- ADR-T-010 establishes a repository-wide JSON-only output contract for + first-party command-line entrypoints. Stdout is reserved for result data; + stderr carries diagnostics and control records; commands that emit stdout + result data refuse direct terminal stdout. +- `torrust-index-cli-common` provides the shared implementation for that + contract: JSON `clap` help/version/usage handling, JSON panic diagnostics, + JSON stderr tracing, TTY refusal, stdout JSON emission, command runners, + baseline exit-code classes, control-plane record types, and redaction + helpers. +- Regression coverage now protects the contract with CLI behavior tests, + binary-boundary checks, and workspace lint rules denying accidental raw + stream output or direct process exits outside the shared CLI boundary. + +#### Changed + +- Helper binaries (`torrust-index-auth-keypair`, + `torrust-index-config-probe`, and `torrust-index-health-check`) share the + JSON CLI boundary. Their successful stdout payloads remain single JSON + objects, now explicitly versioned with a top-level `schema` field, while + help, version, argv errors, TTY refusal, panic diagnostics, and tracing are + emitted as JSON records on stderr. +- The `torrust-index` server and root Rust binaries return explicit + `ExitCode` values at their `main` boundaries and install the shared JSON + panic hook. Central application logging uses JSON tracing on stderr, with a + non-empty `RUST_LOG` taking precedence over the configured default filter. +- `parse_torrent` is a stdout-result command. It emits one JSON object with + `schema`, `torrent`, `original_v1_info_hash`, and `input_byte_length`, leaves + stdout empty on failure, and refuses direct terminal stdout with a JSON + diagnostic record. +- `create_test_torrent`, `import_tracker_statistics`, `seeder`, and `upgrade` + are no-stdout side-effect commands. They keep stdout empty, report status and + diagnostics as JSON/NDJSON on stderr, and propagate command failures instead + of printing plain text or relying on panic output. +- Command-reachable shared libraries use the command diagnostic path instead of + raw stream output. Shutdown notices are structured tracing records, mail + template failures are returned to callers, terminal color formatting is + removed from command paths, and parsing helpers leave reporting decisions to + their command callers. +- The container entry script follows the JSON stderr contract during startup: + it captures helper stdout internally, keeps its own stdout empty before + `su-exec`, checks for `jq` before JSON-dependent helpers run, emits explicit + `DEBUG=1` phase records instead of `set -x`, and wraps controlled utility + failures with captured stderr fields. +- Operator documentation and command examples describe the completed contract + across the README, container guide, upgrade notes, and command module docs. + ### ADR-T-009 — Container infrastructure refactor #### Added @@ -67,7 +135,7 @@ error system (ADR-T-006), MSRV raised to 1.88. configuration system. Leaf crate — no `tokio`, `reqwest`, `sqlx`, `hyper`, `rustls`, `native-tls`, or `openssl` in its dep closure. - Helper-binary crates split into leaves with no HTTP/TLS in their - dep closure: `torrust-index-cli-common` (shared P9 scaffolding — + dep closure: `torrust-index-cli-common` (shared ADR-T-010 scaffolding — `refuse_if_stdout_is_tty`, `init_json_tracing`, `emit`, `BaseArgs`), `torrust-index-health-check` (stdlib-only, Happy Eyeballs IPv6/IPv4 fallback), `torrust-index-auth-keypair` (RSA-2048 key generator), @@ -126,7 +194,7 @@ error system (ADR-T-006), MSRV raised to 1.88. - `EXPOSE ${IMPORTER_API_PORT}/tcp` in Containerfile; port 3002 mapped in compose. - `restart: unless-stopped` on index and tracker compose services. -- `DEBUG=1` env-var gate for entry-script shell tracing (`set -x`). +- `DEBUG=1` env-var gate for entry-script JSON phase diagnostics. - `#[doc(hidden)] pub mod test_helpers` in `torrust-index-config` exposing `PLACEHOLDER_TOML` and `placeholder_settings()` — single source of truth for the ~40 tests across both crates that @@ -391,8 +459,8 @@ error system (ADR-T-006), MSRV raised to 1.88. - Dev-only ports (MySQL 3306, tracker 6969/7070/1212, mailcatcher 1025/1080) no longer bind to `0.0.0.0`; bound to `127.0.0.1`. -- Entry script `set -x` gated behind `DEBUG=1` to avoid leaking - env vars into logs. +- Entry script debug mode now emits structured JSON phase records instead of + enabling `set -x`, avoiding shell-trace leakage of env vars into logs. - Compose credentials annotated as DEV-ONLY with TODO for Docker secrets migration (ADR-T-009 §S1). diff --git a/Cargo.lock b/Cargo.lock index 1e49e2594..5392b6fd9 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -591,16 +591,6 @@ version = "1.0.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570" -[[package]] -name = "colored" -version = "2.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "117725a109d387c937a1533ce01b450cbde6b88abceea8473c4d7a85853cda3c" -dependencies = [ - "lazy_static", - "windows-sys 0.59.0", -] - [[package]] name = "combine" version = "4.6.7" @@ -3901,15 +3891,6 @@ version = "0.5.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8f50febec83f5ee1df3015341d8bd429f2d1cc62bcba7ea2076759d315084683" -[[package]] -name = "text-colorizer" -version = "1.0.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b30f9b94bd367aacc3f62cd28668b10c7ae1784c7d27e223a1c21646221a9166" -dependencies = [ - "colored", -] - [[package]] name = "thiserror" version = "1.0.69" @@ -4219,16 +4200,15 @@ dependencies = [ "sqlx", "tempfile", "tera", - "text-colorizer", "thiserror 2.0.18", "tokio", "toml 1.1.2+spec-1.1.0", + "torrust-index-cli-common", "torrust-index-config", "torrust-index-render-text-as-image", "tower", "tower-http", "tracing", - "tracing-subscriber", "url", "urlencoding", "uuid", @@ -4256,6 +4236,7 @@ dependencies = [ "serde_json", "tracing", "tracing-subscriber", + "url", ] [[package]] @@ -4293,6 +4274,7 @@ dependencies = [ name = "torrust-index-entry-script" version = "4.0.0-develop" dependencies = [ + "serde_json", "tempfile", ] @@ -4913,15 +4895,6 @@ dependencies = [ "windows-targets 0.52.6", ] -[[package]] -name = "windows-sys" -version = "0.59.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1e38bc4d79ed67fd075bcc251a1c39b32a1776bbe92e5bef1f0bf1f8c531853b" -dependencies = [ - "windows-targets 0.52.6", -] - [[package]] name = "windows-sys" version = "0.60.2" diff --git a/Cargo.toml b/Cargo.toml index 907e715df..832d9d9d7 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -52,6 +52,7 @@ opt-level = 3 opt-level = 3 [dependencies] +torrust-index-cli-common = { version = "4.0.0-develop", path = "packages/index-cli-common" } torrust-index-config = { version = "4.0.0-develop", path = "packages/index-config" } torrust-index-render-text-as-image = { version = "0.1.0", path = "packages/render-text-as-image" } @@ -100,14 +101,12 @@ sha-1 = "0" sha2 = "0" sqlx = { version = "0", features = ["migrate", "mysql", "runtime-tokio-native-tls", "sqlite", "time"] } tera = { version = "1", default-features = false } -text-colorizer = "1" thiserror = "2" tokio = { version = "1", features = ["fs", "io-util", "macros", "net", "rt-multi-thread", "signal", "sync", "time"] } toml = "1" tower = { version = "0", features = ["timeout"] } tower-http = { version = "0", features = ["compression-full", "cors", "propagate-header", "request-id", "trace"] } tracing = "0" -tracing-subscriber = { version = "0", features = ["json"] } url = { version = "2", features = ["serde"] } urlencoding = "2" uuid = { version = "1", features = ["v4"] } @@ -139,9 +138,12 @@ warnings = { level = "deny", priority = -1 } all = { level = "deny", priority = -1 } complexity = { level = "deny", priority = -1 } correctness = { level = "deny", priority = -1 } +exit = { level = "deny", priority = 0 } nursery = { level = "warn", priority = -1 } pedantic = { level = "deny", priority = -1 } perf = { level = "deny", priority = -1 } +print_stderr = { level = "deny", priority = 0 } +print_stdout = { level = "deny", priority = 0 } style = { level = "deny", priority = -1 } suspicious = { level = "deny", priority = -1 } diff --git a/Containerfile b/Containerfile index 0856c4194..c481085c8 100644 --- a/Containerfile +++ b/Containerfile @@ -12,7 +12,7 @@ RUN cargo binstall --no-confirm --locked cargo-chef cargo-nextest FROM rust:slim-trixie AS tester WORKDIR /tmp -RUN apt-get update; apt-get install -y curl sqlite3; apt-get autoclean +RUN apt-get update; apt-get install -y curl jq sqlite3; apt-get autoclean RUN curl -L --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/cargo-bins/cargo-binstall/v1.18.1/install-from-binstall-release.sh | bash RUN cargo binstall --no-confirm --locked cargo-nextest imdl @@ -80,6 +80,12 @@ RUN cargo chef prepare --recipe-path /build/recipe.json ## Cook (debug) FROM chef AS dependencies_debug WORKDIR /build/src +# The debug archive stage holds both the Cargo target directory and the +# nextest archive in one layer. Full debuginfo makes that layer exceed +# common builder storage limits while not changing the test surface. +ENV CARGO_INCREMENTAL=0 \ + CARGO_PROFILE_DEV_DEBUG=0 \ + CARGO_PROFILE_TEST_DEBUG=0 COPY --from=recipe /build/recipe.json /build/recipe.json RUN cargo chef cook --workspace --all-targets --all-features --recipe-path /build/recipe.json RUN cargo nextest archive --workspace --all-targets --all-features --archive-file /build/temp.tar.zst ; rm -f /build/temp.tar.zst @@ -98,6 +104,13 @@ WORKDIR /build/src COPY . /build/src RUN cargo nextest archive --workspace --all-targets --all-features --archive-file /build/torrust-index-debug.tar.zst +## Build Runtime Binary (debug) +FROM dependencies_debug AS build_debug_runtime +WORKDIR /build/src +COPY . /build/src +RUN unset CARGO_PROFILE_DEV_DEBUG CARGO_PROFILE_TEST_DEBUG; \ + cargo build --package torrust-index --all-features --bin torrust-index + ## Build Archive (release) FROM dependencies AS build WORKDIR /build/src @@ -314,6 +327,8 @@ ENV TORRUST_INDEX_CONFIG_TOML_PATH=/etc/torrust/index/index.toml \ EXPOSE 3001/tcp 3002/tcp VOLUME ["/var/lib/torrust/index","/var/log/torrust/index","/etc/torrust/index"] COPY --from=test_debug /app/ /usr/ +COPY --from=build_debug_runtime --chmod=0755 --chown=0:0 \ + /build/src/target/debug/torrust-index /usr/bin/torrust-index # jq binary for entry-script JSON consumption (§2.2 step 4). # Root-only (0500) — same posture as busybox and su-exec. # The two shared libraries (libjq, libonig) are required at diff --git a/README.md b/README.md index 0205d0235..c10435856 100644 --- a/README.md +++ b/README.md @@ -162,6 +162,90 @@ The following services are provided by the default configuration: - API - `http://127.0.0.1:3001/`. +### Command-Line Output + +First-party Torrust Index command-line entrypoints are governed by +[ADR-T-010](./adr/010-global-command-line-output-contract.md): stdout is +reserved for machine-readable result data, stderr is reserved for +machine-readable diagnostics/control records, and commands that emit stdout +result data refuse to write it directly to a terminal. + +The `torrust-index` server binary is a no-stdout command. Application tracing is +emitted as JSON records on stderr; the configured `[logging].threshold` selects +the default filter, and a non-empty `RUST_LOG` environment variable overrides +that default. Panics that cross the binary boundary are reported as ADR-T-010 +JSON control-plane records on stderr. + +Command-reachable server libraries use the same diagnostic path. Shutdown +grace-period notices are structured tracing records, and mail-template +initialization or rendering failures are propagated to callers for JSON +diagnostic reporting instead of being printed or exiting from the mailer +library. + +The shared helper infrastructure now wraps `clap` help, version, and usage +errors as JSON control-plane records on stderr, installs a JSON-only panic hook, +and uses JSON tracing on stderr. The container helper binaries emit exactly one +JSON object on stdout when successful, include a top-level `schema` field, and +should be inspected through a pipe or redirect: + +```sh +torrust-index-auth-keypair | jq . +torrust-index-config-probe | jq . +torrust-index-health-check http://127.0.0.1:3001/health_check | jq . +``` + +For helper diagnostics, a non-empty `RUST_LOG` environment variable takes +precedence over `--debug`; otherwise `--debug` raises the default diagnostic +filter to debug. + +The container entry script is also a no-stdout orchestration command. It captures +helper stdout internally, keeps its own stdout empty before `su-exec`, and emits +startup validation failures, status records, utility failures, and `DEBUG=1` +phase diagnostics as JSON records on stderr. Use `docker logs ... 2>&1` or your +runtime's stderr capture and parse those lines as NDJSON when automation needs +startup diagnostics. + +Two root diagnostic commands have also been migrated. `parse_torrent` is a +stdout-result command: it emits one JSON object containing `schema`, `torrent`, +`original_v1_info_hash`, and `input_byte_length`, and it refuses direct terminal +stdout. Pipe or redirect it before inspection: + +```sh +fixture=./tests/fixtures/torrents/6c690018c5786dbbb00161f62b0712d69296df97_with_custom_info_dict_key.torrent +cargo run --quiet --bin parse_torrent -- "$fixture" | jq . +``` + +`create_test_torrent` is a no-stdout side-effect command. It writes the torrent +file into an existing destination directory, keeps stdout empty, and emits JSON +status or diagnostic records on stderr: + +```sh +mkdir -p ./output/test/torrents +cargo run --quiet --bin create_test_torrent -- ./output/test/torrents 2>create-test-torrent.ndjson +jq . create-test-torrent.ndjson +``` + +The root maintenance binaries `import_tracker_statistics`, `seeder`, and +`upgrade` are no-stdout side-effect commands. They keep stdout empty, use the +shared JSON `clap` wrapper for help, version, and argv errors, and emit status +or diagnostic records as JSON/NDJSON on stderr. Automation should branch on the +process exit code and parse stderr as JSON when it needs diagnostics: + +```sh +cargo run --quiet --bin import_tracker_statistics -- 2>import-tracker-statistics.ndjson + +cargo run --quiet --bin seeder -- \ + --api-base-url "http://localhost:3001" \ + --number-of-torrents 10 \ + --user admin \ + --password "$TORRUST_INDEX_ADMIN_PASSWORD" \ + --interval 0 \ + 2>seeder.ndjson + +cargo run --quiet --bin upgrade -- ./data.db ./data_v2.db ./uploads 2>upgrade.ndjson +jq . upgrade.ndjson +``` + ## Documentation - [API (Version 1)][api] @@ -177,6 +261,7 @@ The following services are provided by the default configuration: - [ADR-T-007: Refactor the JWT System](adr/007-jwt-system-refactor.md) — Centralise JWT handling into `src/jwt.rs`, redesign claims to RFC 7519, move to RS256 asymmetric signing, and consolidate session validation into a single code path. - [ADR-T-008: Refactor the Roles and Permissions System](adr/008-roles-and-permissions-refactor.md) — Replace Casbin with a native Rust permission system (`PermissionMatrix` + `RequirePermission` Axum extractors), migrate from `administrator: bool` to a `role` column, and add a `/me/permissions` discovery endpoint. - [ADR-T-009: Container Infrastructure Refactor](adr/009-container-infrastructure-refactor.md) — Split the runtime image into `release` (distroless, root-only toolset) and `debug` bases; extract three helper binaries (`torrust-index-health-check`, `torrust-index-auth-keypair`, `torrust-index-config-probe`) into their own workspace crates with no HTTP/TLS/async-runtime deps; strip credentials from shipped TOMLs and make `database.connect_url` / `tracker.token` mandatory schema fields; split Compose into a production-shaped `compose.yaml` baseline plus an auto-loaded `compose.override.yaml` dev sandbox; and add an internal audit record for vendored `su-exec`. +- [ADR-T-010: Global Command-Line Output Contract](adr/010-global-command-line-output-contract.md) — Apply the JSON-only stdout/stderr contract across first-party command-line entrypoints: stdout is result JSON, stderr is diagnostic JSON/NDJSON, and commands with stdout result data refuse direct TTY output. ## Contributing diff --git a/adr/007-jwt-system-refactor.md b/adr/007-jwt-system-refactor.md index fc6c9eb30..bdfca2214 100644 --- a/adr/007-jwt-system-refactor.md +++ b/adr/007-jwt-system-refactor.md @@ -225,11 +225,13 @@ longer exists. Design: - Refuses to run if stdout is a terminal (exit code 2). - Emits a single JSON object - `{"private_key_pem": "...", "public_key_pem": "..."}` - on stdout (P9 of ADR-T-009). The original raw-PEM + `{"schema": 1, "private_key_pem": "...", "public_key_pem": "..."}` + on stdout ([ADR-T-010](010-global-command-line-output-contract.md)). The original raw-PEM output was replaced in Phase 2. -- Diagnostics on stderr via `tracing` (NDJSON); `--debug` for verbose. -- Uses `clap` for CLI. +- Diagnostics on stderr via JSON `tracing` (NDJSON); `RUST_LOG` takes + precedence over `--debug` for filter selection. +- Uses the shared ADR-T-010 `clap` wrapper, so `--help`, `--version`, and argv + errors are JSON control-plane records on stderr. #### Container integration diff --git a/adr/009-container-infrastructure-refactor.md b/adr/009-container-infrastructure-refactor.md index ad585c575..3d76564ce 100644 --- a/adr/009-container-infrastructure-refactor.md +++ b/adr/009-container-infrastructure-refactor.md @@ -3,7 +3,7 @@ **Status:** Implemented **Date:** 2026-04-19 **Supersedes:** Earlier `ADR-T-009` draft ("Container Infrastructure Hardening") whose tactical S-N items were merged without a written ADR file. Those items are summarised in [Prior Work](#prior-work) and are not re-litigated here. -**Relates to:** [ADR-T-007](007-jwt-system-refactor.md) (auth key generation performed by the entry script). +**Relates to:** [ADR-T-007](007-jwt-system-refactor.md) (auth key generation performed by the entry script), [ADR-T-010](010-global-command-line-output-contract.md) (global command-line output contract). --- @@ -49,8 +49,8 @@ The decisions below follow from a small set of invariants the container subsyste - **P5.** Where two components must agree on a value (path, port, credential), exactly one of them owns it and tells the other; they do not independently maintain a shared constant. - **P6.** The compose baseline is production-shaped; dev affordances are an additive override layer, never a subtraction from the baseline. - **P7.** Vendored security-sensitive code is treated as code we own, with a current internal audit record. -- **P8.** No machine-readable stdout to a TTY. Every helper binary that emits structured output (JSON, PEM) on stdout refuses to run when stdout is a terminal. The check is unconditional — it does not depend on whether the specific output is sensitive. Operators who want to see the output interactively pipe to `jq`, `less`, or `cat`. -- **P9.** Universal helper conventions. Every helper binary links the same baseline crates without exception or per-crate justification: `clap` (argv), `tracing` + `tracing-subscriber` with `json` feature (stderr diagnostics), `serde` + `serde_json` (stdout wire format). These are not enumerated in per-crate allowlists. On success (exit 0), stdout is one JSON object followed by one trailing newline. On failure (exit ≠ 0), stdout is empty — the exit code is the sole branch signal for callers, and the diagnostic goes to stderr via `tracing`. Stderr is always NDJSON `tracing` events regardless of exit code. A shared `torrust-index-cli-common` library crate provides the scaffolding (`refuse_if_stdout_is_tty`, `init_json_tracing`, `emit`, and a common `BaseArgs` with `--debug`). +- **P8.** Helper binaries implement the TTY-refusal rule now defined globally by [ADR-T-010](010-global-command-line-output-contract.md): commands that emit stdout result data refuse to write it directly to a terminal. +- **P9.** Helper binaries implement the stdout/stderr contract now defined globally by [ADR-T-010](010-global-command-line-output-contract.md). This ADR keeps one helper-specific dependency consequence: every helper binary links the same baseline crates without exception or per-crate justification: `clap` (argv), `tracing` + `tracing-subscriber` with `env-filter` and `json` features (stderr diagnostics), `serde` + `serde_json` (stdout wire format). These are not enumerated in per-crate allowlists. A shared `torrust-index-cli-common` library crate provides the scaffolding: JSON `clap` wrapping, TTY refusal, JSON tracing, JSON panic diagnostics, stdout JSON emission, command runners, and a common `BaseArgs` with `--debug`. --- @@ -261,7 +261,7 @@ The script does not poll env vars to discover the configuration. A small `torrus A workspace crate `packages/index-config-probe/` (binary `torrust-index-config-probe`) loads the same `Settings` the application loads and emits the container-relevant resolved values as a JSON object on stdout. -**Dependencies.** `torrust-index-config` (path dependency) and `torrust-index-cli-common` (P9 scaffolding). The helper inherits the parsing surface (`figment`, `toml`, `serde`, `serde_with`, `url`, `camino`, `derive_more`, `thiserror`, `tracing`) via `torrust-index-config`; it adds direct `url` and `percent-encoding` deps for sqlite-URL path-extraction logic. `figment` is declared with `default-features = false` and an explicit feature allowlist (`toml`, `env`) in `torrust-index-config`'s `Cargo.toml` so a future feature flip cannot smuggle `tokio` in transitively. +**Dependencies.** `torrust-index-config` (path dependency) and `torrust-index-cli-common` (ADR-T-010 scaffolding). The helper inherits the parsing surface (`figment`, `toml`, `serde`, `serde_with`, `url`, `camino`, `derive_more`, `thiserror`, `tracing`) via `torrust-index-config`; it adds direct `url` and `percent-encoding` deps for sqlite-URL path-extraction logic. `figment` is declared with `default-features = false` and an explicit feature allowlist (`toml`, `env`) in `torrust-index-config`'s `Cargo.toml` so a future feature flip cannot smuggle `tokio` in transitively. **Contract.** @@ -275,7 +275,10 @@ env var. No CLI flags override the config-file path — callers set TORRUST_INDEX_CONFIG_TOML_PATH in the environment before invoking the probe, the same mechanism the application uses. -Refuses to run when stdout is a TTY (exit 2, per P8). +Refuses to run when stdout is a TTY (exit 2, per ADR-T-010). + +`--help`, `--version`, argv errors, TTY refusal, and panic diagnostics are JSON +control-plane records on stderr. They do not emit stdout result data. On success (exit 0), emits one JSON object + trailing newline on stdout: @@ -322,7 +325,7 @@ PEM material is *never* emitted, only its presence (`"pem_set": true`). The prob |------|---------| | 0 | Recognised, well-formed configuration. | | 1 | Unhandled panic or unexpected I/O on stdout. | -| 2 | Stdout is a TTY (P8), or clap argv-parse failure. | +| 2 | Stdout is a TTY (ADR-T-010), or clap argv-parse failure. | | 3 | Config-load failure (missing field, parse error, IO error). The underlying error message is forwarded verbatim to stderr via tracing. | | 4 | Security-critical field present but empty. Currently: `tracker.token`. | | 5 | Unrecognised database scheme. | @@ -733,16 +736,16 @@ The release-base symlink loop covers every applet the entry script invokes by ba ### D5 — Helper binaries as separate workspace crates -**Follows from:** P2, P8, P9. +**Follows from:** P2, P8, P9, and the global command-line output contract later extracted as ADR-T-010. **Addresses:** [R4](#r4--health_check-pulls-in-reqwest-for-a-localhost-get). -Every helper binary is extracted into its own workspace crate under `packages/index-*/` and follows P9's universal conventions. A shared `packages/index-cli-common/` library crate (`torrust-index-cli-common`) provides the scaffolding so each binary's `main` is only domain logic. +Every helper binary is extracted into its own workspace crate under `packages/index-*/` and follows the command-line output contract now defined globally by ADR-T-010. A shared `packages/index-cli-common/` library crate (`torrust-index-cli-common`) provides the scaffolding so each binary's `main` is only domain logic and command-boundary wiring. The crate boundary makes the "no HTTP/TLS deps" property a manifest-level invariant: a future contributor cannot accidentally re-introduce `reqwest` because the crate's `Cargo.toml` simply does not list it. `reqwest` remains in the workspace for the importer and tracker clients; the goal is to prune it from the *helper binaries'* dep closures, not from the workspace. #### Helper crate roster -| Crate | Path | Domain deps (beyond P9 baseline) | +| Crate | Path | Domain deps (beyond ADR-T-010 helper baseline) | |---|---|---| | `torrust-index-cli-common` | `packages/index-cli-common/` | *(library — no binary)* | | `torrust-index-health-check` | `packages/index-health-check/` | *(none — stdlib networking)* | @@ -757,16 +760,45 @@ The dep-closure exclusion check ([Acceptance Criterion #5](#5-helper-binary-dep- **Public API:** ```rust -/// Refuse to run if stdout is a terminal (P8). -/// Prints a diagnostic to stderr and exits with code 2. +/// Parse argv with clap, emitting JSON help/version/usage records on stderr. +pub fn parse_args_or_exit() -> T; + +/// Refuse to run if stdout is a terminal (ADR-T-010). +/// Emits a JSON control-plane record to stderr and exits with code 2. pub fn refuse_if_stdout_is_tty(binary_name: &str); -/// Initialise `tracing-subscriber` with JSON output on stderr. -pub fn init_json_tracing(level: tracing::Level); +/// Initialise JSON stderr tracing with RUST_LOG / --debug precedence. +pub fn init_json_tracing_with_debug(debug: bool, default_level: tracing::Level); + +/// Install the JSON-only panic hook. +pub fn install_json_panic_hook(command_name: &str); /// Serialise `value` as one JSON object + trailing newline to stdout. pub fn emit(value: &T) -> std::io::Result<()>; +/// Run a stdout-producing single-JSON-object command. +pub fn run_stdout_json_command( + command_name: &str, + debug: bool, + default_level: tracing::Level, + run: Run, +) -> std::process::ExitCode +where + Output: serde::Serialize, + CommandError: std::fmt::Display, + Run: FnOnce() -> Result; + +/// Run a side-effect command that does not emit stdout result data. +pub fn run_no_stdout_command( + command_name: &str, + debug: bool, + default_level: tracing::Level, + run: Run, +) -> std::process::ExitCode +where + CommandError: std::fmt::Display, + Run: FnOnce() -> Result<(), CommandError>; + /// Common `--debug` flag. Flatten into each binary's `Args` via `#[command(flatten)]`. #[derive(clap::Args)] pub struct BaseArgs { @@ -775,19 +807,16 @@ pub struct BaseArgs { } ``` -**Dependencies.** The P9 baseline and nothing else: `clap`, `tracing`, `tracing-subscriber` (with `json` feature), `serde`, `serde_json`. +**Dependencies.** The ADR-T-010 helper baseline and nothing else: `clap`, `tracing`, `tracing-subscriber` (with `env-filter` and `json` features), `serde`, `serde_json`. -Every binary's `main` reduces to: +For helpers whose errors map to ADR-T-010's baseline exit classes, `main` +reduces to: ```rust fn main() -> std::process::ExitCode { - let args = Args::parse(); - refuse_if_stdout_is_tty("torrust-index-"); - init_json_tracing(if args.base.debug { Level::DEBUG } else { Level::INFO }); - match run(&args) { - Ok(out) => { emit(&out).unwrap(); ExitCode::SUCCESS } - Err(e) => { error!(error = %e, "…"); ExitCode::from(e.exit_code()) } - } + install_json_panic_hook("torrust-index-"); + let args = parse_args_or_exit::(); + run_stdout_json_command("torrust-index-", args.base.debug, Level::INFO, || run(&args)) } ``` @@ -795,9 +824,9 @@ fn main() -> std::process::ExitCode { Moved from `src/bin/health_check.rs` to `packages/index-health-check/`. Rewritten with `std::net::TcpStream` + minimal HTTP/1.1 GET (~30 lines), with `set_read_timeout` / `set_write_timeout` for a short connect/read window. No async runtime. -JSON stdout on success: +Stdout result JSON on success: ```json -{"target": "http://localhost:3001/health_check", "status": 200, "elapsed_ms": 4} +{"schema": 1, "target": "http://localhost:3001/health_check", "status": 200, "elapsed_ms": 4} ``` On failure, stdout is empty; the exit code is the sole branch signal for callers (Docker, the entry script). Tests cover non-2xx response, connection refused, read timeout, and malformed status line using a `TcpListener` on an ephemeral port. @@ -806,12 +835,12 @@ On failure, stdout is empty; the exit code is the sole branch signal for callers Moved from `src/bin/generate_auth_keypair.rs` to `packages/index-auth-keypair/`. Domain dep is `rsa` (which re-exports `pkcs8`). -JSON stdout: +Stdout result JSON: ```json -{"private_key_pem": "-----BEGIN PRIVATE KEY-----\n...", "public_key_pem": "-----BEGIN PUBLIC KEY-----\n..."} +{"schema": 1, "private_key_pem": "-----BEGIN PRIVATE KEY-----\n...", "public_key_pem": "-----BEGIN PUBLIC KEY-----\n..."} ``` -This eliminated the `sed` post-processing in the previous documented usage. Consumers use `jq -r .private_key_pem` (shell) or `serde_json::from_reader::` (Rust). The existing TTY guard migrated to the shared `refuse_if_stdout_is_tty`, unifying on exit code 2 (was exit 1). +This eliminated the `sed` post-processing in the previous documented usage. Consumers use `jq -r .private_key_pem` (shell) or `serde_json::from_reader::` (Rust). The existing TTY guard migrated to the shared ADR-T-010 infrastructure, unifying on exit code 2 (was exit 1). The entry script's keygen invocation changed from `torrust-generate-auth-keypair` to `torrust-index-auth-keypair`, and the consumer migrated from `sed` PEM-block extraction to `jq` in the same change (`sed` cannot recover usable PEM from the new single-line JSON output). @@ -1119,9 +1148,9 @@ done exit 0 ``` -### 6. Helper JSON + TTY contract (P8, P9) +### 6. Helper JSON + TTY contract (ADR-T-010) -Every helper binary, when invoked with stdout attached to a TTY, exits with code 2 before producing any output. When invoked with stdout piped, every helper emits exactly one JSON object followed by one trailing newline on stdout, and `tracing` NDJSON events on stderr. +Every helper binary, when invoked with stdout attached to a TTY, exits with code 2 before producing any stdout. When invoked with stdout piped, every helper emits exactly one JSON object followed by one trailing newline on stdout, and JSON/NDJSON control-plane or tracing records on stderr. Help, version, argv errors, TTY refusal, and panic diagnostics are JSON control-plane records on stderr. This is the helper-binary acceptance slice of the global contract later extracted as ADR-T-010. ```sh set -eu @@ -1138,7 +1167,7 @@ for bin in torrust-index-health-check \ [ "$rc" -eq 2 ] || { echo "FAIL: $bin did not exit 2 on TTY (got $rc)" >&2; exit 1; } [ -z "$tty_out" ] || { echo "FAIL: $bin emitted output before TTY refusal" >&2; exit 1; } - # JSON stdout + # stdout result JSON case $bin in *health-check) out=$(docker run --rm --entrypoint="/usr/bin/$bin" \ @@ -1237,7 +1266,7 @@ Tracked for visibility; not part of this refactor: - `docker buildx` multi-platform builds (`linux/arm64`). - Image signing with `cosign`. - Pin base images (`gcr.io/distroless/cc-debian13` and `:debug`) by digest rather than tag for reproducible builds and supply-chain integrity. -- Reimplement the entry script's first-boot work as a small Rust binary (`torrust-index-entry`), eliminating vendored `su-exec` (privilege drop via direct `setgroups`/`setgid`/`setuid` syscalls), the shell-based IFS/heredoc parsing of probe output, and most of the curated busybox applet set. The `torrust-index-config` extraction, the P9 universal helper conventions, and the `torrust-index-config-probe` helper are deliberate stepping stones: they pull the parsing surface out of the root crate, establish the stderr-tracing / stdout-JSON contract all helpers share, and prove the script-↔-Rust integration shape before committing to the full rewrite. The entry binary would depend on `torrust-index-config` and `torrust-index-auth-keypair` directly, eliminating the serialisation boundary entirely. +- Reimplement the entry script's first-boot work as a small Rust binary (`torrust-index-entry`), eliminating vendored `su-exec` (privilege drop via direct `setgroups`/`setgid`/`setuid` syscalls), the shell-based IFS/heredoc parsing of probe output, and most of the curated busybox applet set. The `torrust-index-config` extraction, the ADR-T-010 helper conventions, and the `torrust-index-config-probe` helper are deliberate prerequisites: they pull the parsing surface out of the root crate, establish the stderr-tracing / stdout-JSON contract all helpers share, and prove the script-↔-Rust integration shape before committing to the full rewrite. The entry binary would depend on `torrust-index-config` and `torrust-index-auth-keypair` directly, eliminating the serialisation boundary entirely. - Promote `packages/render-text-as-image/` to a published crate and drop the root crate's `path = "packages/..."` override; once that lands, the directory can safely be added to `.containerignore`. --- diff --git a/adr/010-global-command-line-output-contract.md b/adr/010-global-command-line-output-contract.md new file mode 100644 index 000000000..fcc4de6cc --- /dev/null +++ b/adr/010-global-command-line-output-contract.md @@ -0,0 +1,454 @@ +# ADR-T-010: Global Command-Line Output Contract + +**Status:** Decided and implemented +**Date decided:** 2026-05-13 +**Date implemented:** 2026-05-13 +**Supersedes:** The output-stream rules from ADR-T-009 P8/P9. +**Relates to:** [ADR-T-009](009-container-infrastructure-refactor.md) (helper-binary extraction and dependency rules) + +--- + +## Context + +ADR-T-009 introduced a strict stdout/stderr contract for container helper +binaries: JSON results on stdout, JSON diagnostics on stderr, and no stdout +result data directly to a terminal. That decision was made inside the +container-infrastructure refactor because the entry script needed reliable JSON +from small Rust helpers. + +The contract is not container-specific. The application has several first-party +command-line entrypoints: the server binary, maintenance commands under +`src/bin/`, container helpers under `packages/index-*/`, and future operator +tools. If each command decides independently what stdout and stderr mean, shell +integration becomes brittle and diagnostics can corrupt data streams. + +## Decision + +Adopt one repository-wide JSON-only command-line output contract for every +first-party Torrust Index command-line entrypoint that is shipped, documented, +or intended for operators. + +--- + +## Scope + +### In scope + +Every shipped, documented, or operator-facing first-party command-line +entrypoint: + +- `src/main.rs` (`torrust-index` server binary). +- `src/bin/create_test_torrent.rs`. +- `src/bin/import_tracker_statistics.rs`. +- `src/bin/parse_torrent.rs`. +- `src/bin/seeder.rs`. +- `src/bin/upgrade.rs`. +- `packages/index-auth-keypair/src/bin/torrust-index-auth-keypair.rs`. +- `packages/index-config-probe/src/bin/torrust-index-config-probe.rs`. +- `packages/index-health-check/src/bin/torrust-index-health-check.rs`. +- `share/container/entry_script_sh` and `share/container/entry_script_lib_sh`. + +Command-reachable library paths that emit operator-facing diagnostics: + +- `src/bootstrap/logging.rs`. +- `src/console/commands/seeder/app.rs`. +- `src/console/commands/seeder/logging.rs`. +- `src/console/cronjobs/tracker_statistics_importer.rs`. +- `src/mailer.rs`. +- `src/tracker/statistics_importer.rs`. +- `src/upgrades/from_v1_0_0_to_v2_0_0/`. +- `src/utils/parse_torrent.rs`. +- `src/web/api/server/signals.rs`. + +Future entry, migration, maintenance, diagnostic, or operator tools are +governed by this ADR from creation. + +### Out of scope + +Tests, examples, benches, one-off developer fixtures, and library packages with +no shipped binary entrypoint are outside the normative scope unless they are +documented as application commands: + +- `build.rs` Cargo protocol output such as `cargo:rerun-if-changed=...`. +- Tests, benches, examples, and harnesses under `tests/`, `src/tests/`, and + package test directories. +- Developer-only scripts under `contrib/dev-tools/`. +- Library packages with no shipped binary entrypoint, such as Mudlark and + `render-text-as-image`. + +--- + +## Contract + +### Streams + +Stdout and stderr are both machine-readable streams. A command writes JSON +records to them or leaves them empty. Plain human-readable text is not a valid +application output format on either stream. + +Stdout is reserved for command result data intended for a caller to consume. +Diagnostics, logs, progress messages, warnings, help, usage, prompts, and +status updates go to stderr as JSON control-plane records. + +A command that has no stdout result data leaves stdout empty. A long-running +server process normally has no stdout result data; its diagnostics are logs and +therefore belong on stderr. + +### JSON output + +When a command emits stdout result data, the default wire format is exactly one +JSON object followed by one trailing newline. + +On success (exit 0), a command that has stdout result data emits its JSON object +on stdout and may emit JSON diagnostics on stderr. + +On failure (exit not 0), stdout is empty. The exit code is the branch signal for +callers, and JSON diagnostics go to stderr. + +Commands that need a different stdout JSON shape, such as streaming output, must +document that exception in the command's own contract and explain why the +single-object JSON contract does not fit. When stdout result data is streaming, +stdout uses NDJSON: one JSON object per line. Non-JSON stdout or stderr is +outside this contract and requires a new ADR. + +### TTY refusal + +A command that emits stdout result data refuses to run when stdout is attached +to a terminal. It exits before producing stdout and reports the diagnostic as +JSON on stderr. + +The refusal is unconditional for commands with stdout result data. It does not +depend on whether the payload is sensitive, and it is not caused by the JSON +encoding. JSON is the only output encoding for both streams; the TTY refusal +exists because stdout result data is intended for another process or file. +Operators who want to inspect output interactively can pipe it to another +program such as `jq`, `less`, or `cat`. + +Commands that do not emit stdout result data do not refuse merely because stdout +is attached to a terminal. They leave stdout empty and write any diagnostics to +stderr as JSON. + +### Exit codes + +The shared baseline exit-code classes are: + +| Class | Code | Meaning | +|---------|------|---------| +| success | 0 | Command succeeded. | +| failure | 1 | Runtime, startup, internal, or command execution failure. | +| usage | 2 | Command-line usage failure: clap argv errors, TTY refusal, or invalid arguments. | + +Exit code 2 is reserved for command-line usage failures, including TTY refusal +and argv parsing errors produced by `clap`. + +Command-specific non-usage exit codes may still be documented by the owning +command contract. For example, `torrust-index-config-probe` keeps its existing +configuration and probe failure codes. + +### Diagnostics + +Operator-facing and script-facing commands use `tracing` for diagnostics. The +diagnostic writer is stderr, configured with JSON output. + +Stderr is a JSON control and diagnostic stream. When stderr emits multiple +records over time, it uses NDJSON: one complete JSON object per line. Diagnostic +records are `tracing` events so scripts can consume diagnostics without +scraping text. Non-diagnostic control records, such as help and usage, also +write JSON objects to stderr. Plain-text diagnostic formatting is not an output +mode for first-party application binaries; operators can pipe JSON diagnostics +to a viewer when they want a friendlier presentation. + +Command-specific diagnostics do not use `println!` or `eprintln!` for progress, +status, or errors; those would put raw text on stdout or stderr. + +### Help and usage output + +Help and usage information is command output and follows the same JSON-only +stream contract, but it is not stdout result data. + +A help request writes a JSON control-plane record to stderr and exits with +code 0. It does not trigger stdout TTY refusal, because it does not emit stdout +result data. + +A usage or argv-parse error writes a JSON diagnostic/control-plane record to +stderr and exits with code 2. + +Rust commands use `clap` for argv parsing. Raw `clap` help or error text is +wrapped in the JSON contract by the shared CLI infrastructure. + +--- + +## Shared Control-Plane Record Schema + +Shared stderr control-plane records use this top-level JSON shape: + +| Field | Type | Description | +|-----------|--------|-------------| +| `schema` | number | Shared control-plane record schema version. Initial value: `1`. | +| `command` | string | Binary or entrypoint name. | +| `kind` | string | One of `help`, `version`, `usage_error`, `tty_refusal`, `panic`, `status`, or `diagnostic`. | +| `message` | string | Short human-readable message carried inside the JSON record. | +| `fields` | object | Optional kind-specific object tagged with `type`. | + +### Structured field variants + +| Kind | Fields | +|---------------|--------| +| `help` | `text` | +| `version` | `version` | +| `usage_error` | `exit_code`, `clap_error_kind` | +| `tty_refusal` | `exit_code`, `stream` | +| `panic` | `exit_code`, `thread`, `location` | + +Panic payloads are not part of the shared record because they may contain +secrets. + +--- + +## Command Output Classification + +### Commands with stdout result data + +These commands emit one JSON object on stdout on success, refuse when stdout is +attached to a terminal, and leave stdout empty on failure: + +| Command | Stdout result schema | +|--------------------------------|----------------------| +| `torrust-index-auth-keypair` | `schema`, `private_key_pem`, `public_key_pem` | +| `torrust-index-config-probe` | `schema`, `database`, `auth` | +| `torrust-index-health-check` | `schema`, `target`, `status`, `elapsed_ms` | +| `parse_torrent` | `schema`, decoded torrent, original v1 info hash, input byte length | + +All use the default single-object stdout contract. None use the documented +streaming NDJSON exception. + +### Commands with no stdout result data + +These commands leave stdout empty and write diagnostics to stderr as JSON: + +| Command | Description | +|--------------------------------|-------------| +| `torrust-index` | Long-running server; logs go to stderr. | +| `create_test_torrent` | Side-effect command; writes a torrent file. | +| `import_tracker_statistics` | Side-effect maintenance command. | +| `seeder` | Side-effect load/seeding command. | +| `upgrade` | Side-effect migration command. | +| `share/container/entry_script_sh` | Orchestration entrypoint; stdout from helpers is captured inside command substitutions. | + +--- + +## Redaction Policy + +JSON diagnostics make accidental secret exposure easier to automate. The +following redaction rules are applied before JSON stderr becomes the output +path: + +- Never log raw database URLs that contain credentials. Log a redacted form + with password, token, and query-secret components removed. +- Never log JWT secrets, private keys, admin tokens, session secrets, API keys, + SMTP passwords, or mailer credentials. +- Avoid putting secrets in error `Display` strings. Prefer typed error fields + that can be redacted before logging. +- Keep raw external utility stderr out of top-level diagnostic messages unless + it has been reviewed or wrapped as a field that can be redacted. + +Shared redaction helpers replace secret-like field names with `[redacted]` and +strip userinfo plus secret-bearing query parameters from database URLs before +they are logged. + +--- + +## Shared Rust CLI Infrastructure + +`packages/index-cli-common` (`torrust-index-cli-common`) provides the shared +scaffolding for the global contract so every Rust binary shares the same +implementation: + +- **JSON clap handling:** `parse_args_or_exit::()` wraps + `clap::Parser::try_parse()`. Help and version requests emit JSON control + records to stderr and exit 0; argv errors emit JSON diagnostic/control + records to stderr and exit 2; stdout remains empty. +- **JSON stderr control-plane emission:** A direct record helper for + control-plane output that does not depend on a tracing subscriber. Used for + clap help, clap parse errors, early startup failures, and panic hooks. +- **JSON panic hook:** `install_json_panic_hook(command_name)` emits one JSON + diagnostic record to stderr and terminates with exit code 1. It does not + call Rust's default panic hook. It is safe for non-main-thread panics: emit + a best-effort JSON diagnostic once, avoid waiting on other application + threads, and terminate the process without returning to the default panic + path. +- **JSON tracing on stderr:** Idempotent initialization (`try_init` or an + equivalent guard) so early startup and later application setup cannot + double-install a subscriber. Each tracing event is emitted as one complete + JSON line on stderr using a non-interleaving locked writer per event, even + when multiple tasks or threads log concurrently. +- **TTY refusal:** For commands with stdout result data. Exit code 2 with a + JSON stderr diagnostic. +- **Stdout JSON emission:** `emit()` writes exactly one JSON object plus one + trailing newline to stdout, called only after TTY refusal. +- **Command runners:** Small runner helpers for the two command classes: + stdout-producing single-object commands and no-stdout side-effect commands. +- **Tracing filter precedence:** A non-empty `RUST_LOG` environment variable + wins, otherwise `--debug` selects debug-level diagnostics, otherwise the + command's default level is used. +- **`--debug` flag:** Shared across all commands. +- **`ExitCode` centralization:** Direct `std::process::exit` usage is + centralized in this shared infrastructure. Binaries return `ExitCode` from + their own `main` function. + +--- + +## Implementation Summary + +### Rust helper binaries (`packages/index-*`) + +`torrust-index-auth-keypair`, `torrust-index-config-probe`, and +`torrust-index-health-check` use the shared JSON clap parser, install the +shared JSON panic hook, expose `--version` through clap metadata, keep their +stdout result schemas unchanged, and preserve TTY refusal for stdout result +data. `torrust-index-config-probe` no longer preserves Rust's default +plain-text panic output. + +### Central application logging + +Central application logging uses the shared JSON stderr tracing setup. The root +package depends on `torrust-index-cli-common`. Root binaries return explicit +`ExitCode` values at their `main` boundaries. + +### `parse_torrent` and `create_test_torrent` + +`parse_torrent` uses the shared JSON clap parser, JSON panic hook, and JSON +stderr tracing runner. It emits one JSON stdout result object and refuses +terminal stdout. `create_test_torrent` uses the same shared infrastructure and +remains a no-stdout side-effect command with JSON diagnostics on stderr. + +### `import_tracker_statistics`, `seeder`, and `upgrade` + +All three use the shared JSON clap parser, JSON panic hook, JSON stderr tracing +runner, empty stdout side-effect contract, structured tracing diagnostics, and +propagated command errors. The command-reachable tracker statistics and upgrade +modules no longer emit raw stream output or terminal color formatting. + +### Command-reachable shared libraries + +Server shutdown notices use structured tracing diagnostics. Mail template +initialization errors are returned to callers for JSON diagnostic reporting +instead of printing or exiting from the mailer library. `text-colorizer` is +removed from root runtime code. Color formatting is removed from +command-reachable modules. Library parsing helpers return errors and let +command callers decide how to report them. + +### Container entry script + +The shell entrypoint checks for `jq`, emits JSON diagnostics and debug phase +records on stderr, wraps validation failures in shared control-plane records, +captures expected utility stderr where the script controls the utility +invocation, and keeps helper stdout inside command substitutions. `set -x` +under `DEBUG=1` is replaced with explicit JSON debug records at phase +boundaries. Shell JSON diagnostics use `jq -cn --arg ...` for string escaping. +If `jq` is missing, one fixed, minimal JSON diagnostic is emitted to stderr +without interpolating untrusted values. File writes to `/etc/motd` and +`/etc/profile` remain plain text because they are not stream output. + +### Documentation and changelog + +Operator documentation and changelog entries are updated for the command-output +migrations. `README.md` command examples, `docs/containers.md`, +`upgrades/from_v1_0_0_to_v2_0_0/README.md`, and command module docs are +aligned with the migrated command behavior. `CHANGELOG.md` entries are marked +as breaking when command output changes can affect scripts that consumed +previous plain-text output. + +--- + +## Tests And Guards + +### Shared infrastructure tests (`packages/index-cli-common`) + +- JSON help records, JSON version records, JSON argv-error records, exit code + mapping, TTY-refusal records, stdout JSON emission, and the panic hook's JSON + shape. +- Schema/version fields on control-plane records. +- Redaction of common secret-bearing fields. +- `RUST_LOG`/`--debug` precedence. +- Concurrent JSON logging: records emitted from multiple threads or tasks are + each captured as one complete JSON object per stderr line. + +### Helper binary contract tests + +Success stdout shape, empty stdout on failure, JSON stderr diagnostics, clap +help JSON, clap version JSON, clap error JSON, and exit code 2 for usage +failures. + +### Root binary contract tests + +`parse_torrent` success/failure stdout shape. No-stdout commands keeping stdout +empty while logging JSON stderr. + +### Container entry-script tests + +`packages/index-entry-script` tests parse and assert JSON stderr records for +validation failures and status branches. + +### Workspace Clippy guards + +Configured through workspace lint levels in `Cargo.toml` (no separate +`clippy.toml`): + +- `clippy::print_stdout` and `clippy::print_stderr` are denied. Local + `#[allow]` annotations exist for Cargo build-script protocol output, + developer examples, out-of-scope test diagnostics, and the shared CLI + infrastructure's controlled stdout emitter. +- `clippy::exit` is denied. A local exception exists for the shared CLI + infrastructure's `exit_with` helper. + +### Binary-boundary regression test + +The root `cli_contract` integration test scans all in-scope binaries and fails +if a `main` boundary stops returning `ExitCode` or regresses to `Result` +termination, because Rust's default `Result` termination writes raw +`Error: ...` text to stderr. + +### TTY-refusal smoke tests + +Stdout-producing commands are tested for TTY refusal using a pseudo-terminal +library or tool such as `rexpect` or `portable-pty`. + +### Suggested verification commands + +```sh +cargo fmt --all +cargo check --workspace --all-targets --all-features 2>&1 | tee /tmp/adr010-cargo-check.log +cargo clippy --workspace --all-targets --all-features -- -D warnings 2>&1 | tee /tmp/adr010-cargo-clippy.log +cargo test --workspace --all-targets --all-features 2>&1 | tee /tmp/adr010-cargo-test.log +cargo test --workspace --all-targets --all-features --release 2>&1 | tee /tmp/adr010-cargo-test-release.log +cargo check --workspace --all-targets --no-default-features 2>&1 | tee /tmp/adr010-cargo-check-no-default-features.log +cargo test --workspace --all-targets --no-default-features 2>&1 | tee /tmp/adr010-cargo-test-no-default-features.log +cargo test --doc --workspace --all-features 2>&1 | tee /tmp/adr010-cargo-test-doc.log +cargo doc --workspace --all-features --no-deps 2>&1 | tee /tmp/adr010-cargo-doc.log +``` + +--- + +## Consequences + +ADR-T-009 remains the historical record for why the helper binaries were +extracted and why the first implementation exists. This ADR is the canonical +application-wide output contract. + +New command-line entrypoints must state whether they emit stdout result data. If +they do, they must either use the default single-object JSON contract or +document a justified JSON exception. + +The main server and maintenance commands are governed by the same stdout/stderr +separation as the helper binaries. The difference is only whether they have +stdout result data. + +All existing first-party command-line entrypoints now conform to this contract. +Commands that predate this ADR have been migrated; no non-conforming legacy +commands remain as precedent. + +The exact exit-code taxonomy for root maintenance commands beyond the baseline +`success`, `failure`, and `usage` classes is left to future command-specific +contracts. Existing helper-specific exit codes remain stable unless a +command-specific contract says otherwise. \ No newline at end of file diff --git a/build.rs b/build.rs index d5068697c..ec3a95d90 100644 --- a/build.rs +++ b/build.rs @@ -1,3 +1,5 @@ +#![allow(clippy::print_stdout)] + // generated by `sqlx migrate build-script` fn main() { // trigger recompilation when a new migration is added diff --git a/docs/containers.md b/docs/containers.md index cadefa542..c035f05c2 100644 --- a/docs/containers.md +++ b/docs/containers.md @@ -405,6 +405,58 @@ failure rather than silent misbehaviour. ## Runtime Image Notes +### Command-Line Output Contract + +The `torrust-index` server process is a no-stdout command. Once the Rust +application starts, its tracing diagnostics are JSON records on stderr. The +configured `[logging].threshold` selects the default filter, and a non-empty +`RUST_LOG` environment variable overrides that default. The server does not +refuse terminal stdout, because it does not emit stdout result data. + +Command-reachable server libraries follow that same stream contract. Shutdown +grace-period notices are structured tracing records, and mail-template +initialization or rendering failures are returned to callers for JSON diagnostic +reporting rather than printed or handled by exiting from the mailer library. + +Container helper binaries follow the ADR-T-010 stdout/stderr split. When they +emit result data, stdout is exactly one JSON object with a trailing newline and +a top-level `schema` field. Diagnostics, help, version output, argv errors, TTY +refusal, and panic reports are emitted on stderr as JSON records. + +The stdout-producing helpers are: + +- `torrust-index-auth-keypair`: `schema`, `private_key_pem`, `public_key_pem`. +- `torrust-index-config-probe`: `schema`, `database`, `auth`. +- `torrust-index-health-check`: `schema`, `target`, `status`, `elapsed_ms`. + +These helpers refuse to write stdout result data directly to a terminal. Pipe or +redirect the result instead: + +```sh +torrust-index-auth-keypair | jq . +torrust-index-config-probe | jq . +torrust-index-health-check http://127.0.0.1:3001/health_check | jq . +``` + +For helper diagnostics, a non-empty `RUST_LOG` environment variable takes +precedence over `--debug`; otherwise `--debug` raises the helper's default +diagnostic filter to debug. Help and version requests do not emit stdout result +data and therefore do not trigger TTY refusal; they write JSON control-plane +records to stderr and exit with code 0. + +The container entry script captures helper stdout internally and does not +forward it to the terminal. Before it execs the application, the entry script is +a no-stdout orchestration command: validation failures, status notices, expected +utility failures, `jq` parsing failures, unexpected shell exits, and `DEBUG=1` +phase records are emitted on stderr as one JSON object per line. The records use +the shared `schema`, `command`, `kind`, `message`, and `fields` shape; the +entry-script command name is `torrust-index-entry-script`. + +Expected utility stderr captured by the script is carried inside JSON fields +instead of being forwarded as top-level stream text. Helper stdout remains inside +command substitutions. File writes to `/etc/motd` and `/etc/profile` are not +stream output. + ### Healthcheck (both targets) Both `release` and `debug` ship the same two-probe `HEALTHCHECK` @@ -439,9 +491,9 @@ the entry script needs at first boot: `su-exec` is a separate root-only binary at `/bin/su-exec`, not a busybox applet. `jq` is a separate root-only binary at -`/usr/bin/jq` used by the entry script's auth-keypair -bootstrap. None of these are reachable by the unprivileged -`torrust` user. +`/usr/bin/jq` used by the entry script's JSON diagnostics, +config-probe parsing, and auth-keypair bootstrap. None of +these are reachable by the unprivileged `torrust` user. There is no `/busybox/` directory in the release image — the full busybox applet tree from the upstream `:debug` @@ -467,13 +519,18 @@ interactive shell as the application user. ### Entry Script Debugging The container entry script does not produce verbose output by default. -To enable shell tracing (`set -x`) for startup troubleshooting, set the -`DEBUG` environment variable: +To emit JSON phase records for startup troubleshooting, set the `DEBUG` +environment variable: ```sh --env DEBUG=1 ``` +Debug records are written to stderr as ADR-T-010 JSON status records with +`fields.level = "debug"`. The script no longer enables shell tracing with +`set -x`, so automation should parse stderr as NDJSON rather than scrape shell +trace lines. + The entry script also runs under `set -eu` (POSIX `errexit` + `nounset`): any unchecked command failure aborts startup immediately, and references to unset variables are treated as @@ -516,8 +573,8 @@ above. — invoked as root after the default TOML is in place. The probe is the same loader the application uses, so it sees the operator's full TOML + env-var stack. Its JSON - output is consumed by `jq` and drives the remaining - steps. The probe runs *before* the script exports any + output (`schema`, `database`, `auth`) is consumed by `jq` + and drives the remaining steps. The probe runs *before* the script exports any `TORRUST_INDEX_CONFIG_OVERRIDE_*` of its own, so its output reflects only operator-supplied values. 5. **`TORRUST_INDEX_CONFIG_OVERRIDE_AUTH__{PRIVATE,PUBLIC}_KEY_{PEM,PATH}`** @@ -576,23 +633,29 @@ in the repo wires both overrides for the local dev workflow Both runtime images ship a root-only `/usr/bin/jq` (mode `0500 root:root`, sourced from a pristine `rust:slim-trixie` `jq_donor` build stage in the Containerfile). It is invoked -only during the entry script's pre-`su-exec` phase to parse -the config probe's JSON output and the auth-keypair helper's -JSON output. The unprivileged `torrust` user has no access -to `/usr/bin/jq` after privilege drop. +only during the entry script's pre-`su-exec` phase to emit +properly escaped JSON diagnostics, parse the config probe's +JSON output, and split the auth-keypair helper's JSON output. +If `jq` is unavailable, the script emits a fixed minimal JSON +diagnostic before exiting. The unprivileged `torrust` user has +no access to `/usr/bin/jq` after privilege drop. + +Operators who run the helpers manually should use the same pattern: pipe stdout +result data to `jq`, redirect it to a file, or capture it from another process. +Direct terminal stdout is refused by design. #### Sourced Shell Library -The entry script's pure helper functions (`inst`, -`key_configured`, `validate_auth_keys`, `seed_sqlite`) live -in a separate POSIX `sh` library shipped at +The entry script's reusable POSIX `sh` helpers live in a +separate library shipped at `/usr/local/lib/torrust/entry_script_lib_sh` (mode `0444 root:root`, sourced — not exec'd). Splitting them out lets the workspace test crate [`packages/index-entry-script/`](../packages/index-entry-script/) -drive each helper through a host `sh` subprocess and assert -the exit-code / stderr contracts of every branch of -ADR-T-009 §7.1's auth-key invariants and §7.2's seeding -outcomes. The library has no top-level side effects, so -sourcing it from either the entry script or a test harness -is safe. +drive helpers such as `validate_auth_keys` and `seed_sqlite` +through a host `sh` subprocess and assert their exit-code and +JSON stderr contracts. The same library owns the entry script's +JSON control-plane emitters, checked utility wrappers, `jq` +read/write helpers, and unexpected-exit trap. It has no top-level +side effects, so sourcing it from either the entry script or a test +harness is safe. diff --git a/packages/index-auth-keypair/src/bin/torrust-index-auth-keypair.rs b/packages/index-auth-keypair/src/bin/torrust-index-auth-keypair.rs index 31b41ef57..08d937461 100644 --- a/packages/index-auth-keypair/src/bin/torrust-index-auth-keypair.rs +++ b/packages/index-auth-keypair/src/bin/torrust-index-auth-keypair.rs @@ -1,5 +1,8 @@ //! Generate an RSA-2048 key pair for JWT authentication. //! +//! Emits one JSON object with `schema`, `private_key_pem`, and `public_key_pem` +//! on stdout. Direct terminal stdout is refused; pipe or redirect the result. +//! //! # Usage //! //! ```sh @@ -10,13 +13,16 @@ use std::process::ExitCode; use clap::Parser; -use torrust_index_auth_keypair::generate_keypair; -use torrust_index_cli_common::{BaseArgs, emit, init_json_tracing, refuse_if_stdout_is_tty}; -use tracing::{error, info}; +use torrust_index_auth_keypair::{KeypairOutput, generate_keypair}; +use torrust_index_cli_common::{BaseArgs, install_json_panic_hook, parse_args_or_exit, run_stdout_json_command}; +use tracing::info; + +const COMMAND_NAME: &str = "torrust-index-auth-keypair"; #[derive(Parser)] #[command( name = "torrust-index-auth-keypair", + version, about = "Generate an RSA-2048 key pair for Torrust Index JWT authentication" )] struct Args { @@ -25,32 +31,82 @@ struct Args { } fn main() -> ExitCode { - let args = Args::parse(); - init_json_tracing(if args.base.debug { - tracing::Level::DEBUG - } else { - tracing::Level::INFO - }); - refuse_if_stdout_is_tty("torrust-index-auth-keypair"); - - info!("Generating RSA-2048 key pair..."); - - match generate_keypair() { - Ok(out) => { - info!("Key pair generated successfully."); - match emit(&out) { - Ok(()) => ExitCode::SUCCESS, - Err(e) => { - // Writing to stdout failed (e.g. broken pipe). Honour - // the helper's exit-code contract instead of panicking. - error!(error = %e, "failed to write key pair to stdout"); - ExitCode::FAILURE - } - } - } - Err(e) => { - error!(error = %e, "keypair generation failed"); - ExitCode::FAILURE - } + install_json_panic_hook(COMMAND_NAME); + + let args = parse_args_or_exit::(); + + run_stdout_json_command::(COMMAND_NAME, args.base.debug, tracing::Level::INFO, || { + info!("generating RSA-2048 key pair"); + generate_keypair() + }) +} + +#[cfg(test)] +mod tests { + //! # Auth-keypair binary CLI contract tests + //! + //! | Test | What it covers | + //! |---------------------------------------|----------------------------------------| + //! | `help_is_json_control_record` | `--help` is wrapped as JSON metadata | + //! | `version_is_json_control_record` | `--version` is wrapped as JSON metadata| + //! | `usage_error_is_json_control_record` | argv errors become JSON usage records | + + use torrust_index_cli_common::{CommandExit, ControlPlaneFields, ControlPlaneRecordKind, parse_args_from}; + + use super::{Args, COMMAND_NAME}; + + #[test] + fn help_is_json_control_record() { + let Err(exit) = parse_args_from::([COMMAND_NAME, "--help"]) else { + panic!("help should stop parsing"); + }; + + assert_eq!(exit.exit, CommandExit::Success); + assert_eq!(exit.record.command, COMMAND_NAME); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::Help); + + let Some(ControlPlaneFields::Help { text }) = exit.record.fields else { + panic!("help record should carry help text"); + }; + assert!(text.contains("Generate an RSA-2048 key pair")); + assert!(text.contains("--debug")); + } + + #[test] + fn version_is_json_control_record() { + let Err(exit) = parse_args_from::([COMMAND_NAME, "--version"]) else { + panic!("version should stop parsing"); + }; + + assert_eq!(exit.exit, CommandExit::Success); + assert_eq!(exit.record.command, COMMAND_NAME); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::Version); + + let Some(ControlPlaneFields::Version { version }) = exit.record.fields else { + panic!("version record should carry version text"); + }; + assert_eq!(version, format!("{COMMAND_NAME} {}", env!("CARGO_PKG_VERSION"))); + } + + #[test] + fn usage_error_is_json_control_record() { + let Err(exit) = parse_args_from::([COMMAND_NAME, "--no-such-flag"]) else { + panic!("unknown flags should stop parsing"); + }; + + assert_eq!(exit.exit, CommandExit::Usage); + assert_eq!(exit.record.command, COMMAND_NAME); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::UsageError); + assert!(exit.record.message.contains("--no-such-flag")); + + let Some(ControlPlaneFields::UsageError { + exit_code, + clap_error_kind, + }) = exit.record.fields + else { + panic!("usage record should carry usage fields"); + }; + assert_eq!(exit_code, CommandExit::Usage.code()); + assert_eq!(clap_error_kind, "unknown_argument"); } } diff --git a/packages/index-auth-keypair/src/lib.rs b/packages/index-auth-keypair/src/lib.rs index f5eda4d41..b42a22711 100644 --- a/packages/index-auth-keypair/src/lib.rs +++ b/packages/index-auth-keypair/src/lib.rs @@ -1,7 +1,7 @@ //! RSA-2048 key pair generator for Torrust Index JWT authentication. //! -//! Outputs a JSON object with `private_key_pem` and `public_key_pem` -//! fields to stdout (P9). Diagnostics go to stderr via `tracing`. +//! Outputs a JSON object with `schema`, `private_key_pem`, and `public_key_pem` +//! fields to stdout per ADR-T-010. Diagnostics go to stderr via JSON `tracing`. use rsa::RsaPrivateKey; use rsa::pkcs8::{EncodePrivateKey, EncodePublicKey, LineEnding}; @@ -10,8 +10,12 @@ use serde::{Deserialize, Serialize}; #[cfg(test)] mod tests; +/// Schema version of the keypair JSON output. +pub const SCHEMA: u32 = 1; + #[derive(Serialize, Deserialize)] pub struct KeypairOutput { + pub schema: u32, pub private_key_pem: String, pub public_key_pem: String, } @@ -33,6 +37,7 @@ pub fn generate_keypair() -> Result { .map_err(|e| format!("public key PEM export failed: {e}"))?; Ok(KeypairOutput { + schema: SCHEMA, private_key_pem: private_pem.to_string(), public_key_pem: public_pem, }) diff --git a/packages/index-auth-keypair/src/tests/mod.rs b/packages/index-auth-keypair/src/tests/mod.rs index 95bf43d55..262bf3677 100644 --- a/packages/index-auth-keypair/src/tests/mod.rs +++ b/packages/index-auth-keypair/src/tests/mod.rs @@ -3,6 +3,7 @@ //! | Test | What it covers | //! |---------------------------------------|-----------------------------------------| //! | `generated_json_round_trips` | JSON output deserialises back | +//! | `generated_output_carries_schema` | Output schema field is stable | //! | `private_pem_parses` | Private key PEM is valid PKCS#8 | //! | `public_pem_parses` | Public key PEM is valid SPKI | @@ -13,10 +14,17 @@ fn generated_json_round_trips() { let output = super::generate_keypair().unwrap(); let json = serde_json::to_string(&output).unwrap(); let parsed: super::KeypairOutput = serde_json::from_str(&json).unwrap(); + assert_eq!(parsed.schema, super::SCHEMA); assert!(!parsed.private_key_pem.is_empty()); assert!(!parsed.public_key_pem.is_empty()); } +#[test] +fn generated_output_carries_schema() { + let output = super::generate_keypair().unwrap(); + assert_eq!(output.schema, super::SCHEMA); +} + #[test] fn private_pem_parses() { let output = super::generate_keypair().unwrap(); diff --git a/packages/index-auth-keypair/tests/keypair_generation.rs b/packages/index-auth-keypair/tests/keypair_generation.rs index d1ddeaf4a..5f93bf7cc 100644 --- a/packages/index-auth-keypair/tests/keypair_generation.rs +++ b/packages/index-auth-keypair/tests/keypair_generation.rs @@ -3,21 +3,29 @@ //! | Test | What it covers | //! |-----------------------------------------|-------------------------------------------| //! | `generated_keypair_round_trips_as_json` | JSON output deserialises back correctly | +//! | `generated_keypair_carries_schema` | JSON output schema field is stable | //! | `output_contains_valid_pem_keys` | PEM keys parse as RSA PKCS#8 / SPKI | //! | `successive_calls_produce_distinct_keys` | No hardcoded/cached key material | use rsa::pkcs8::{DecodePrivateKey, DecodePublicKey}; -use torrust_index_auth_keypair::{KeypairOutput, generate_keypair}; +use torrust_index_auth_keypair::{KeypairOutput, SCHEMA, generate_keypair}; #[test] fn generated_keypair_round_trips_as_json() { let output = generate_keypair().unwrap(); let json = serde_json::to_string(&output).unwrap(); let parsed: KeypairOutput = serde_json::from_str(&json).unwrap(); + assert_eq!(parsed.schema, SCHEMA); assert!(!parsed.private_key_pem.is_empty()); assert!(!parsed.public_key_pem.is_empty()); } +#[test] +fn generated_keypair_carries_schema() { + let output = generate_keypair().unwrap(); + assert_eq!(output.schema, SCHEMA); +} + #[test] fn output_contains_valid_pem_keys() { let output = generate_keypair().unwrap(); diff --git a/packages/index-cli-common/Cargo.toml b/packages/index-cli-common/Cargo.toml index ebe64ad59..56d7b4120 100644 --- a/packages/index-cli-common/Cargo.toml +++ b/packages/index-cli-common/Cargo.toml @@ -15,7 +15,8 @@ clap = { version = "4", features = ["derive"] } serde = { version = "1", features = ["derive"] } serde_json = "1" tracing = "0" -tracing-subscriber = { version = "0", features = ["json"] } +tracing-subscriber = { version = "0", features = ["env-filter", "json"] } +url = "2" [lints] workspace = true diff --git a/packages/index-cli-common/src/lib.rs b/packages/index-cli-common/src/lib.rs index e02dbf6f1..0983c0e7d 100644 --- a/packages/index-cli-common/src/lib.rs +++ b/packages/index-cli-common/src/lib.rs @@ -1,37 +1,643 @@ -//! Shared CLI scaffolding for Torrust Index helper binaries (P9). +//! Shared CLI scaffolding for Torrust Index command-line tools (ADR-T-010). //! //! Every helper binary uses this crate for: -//! - TTY refusal (P8) -//! - JSON tracing initialisation on stderr +//! - JSON wrapping for `clap` help, version, and argv errors +//! - JSON control-plane records on stderr before tracing is installed +//! - TTY refusal for stdout result data +//! - JSON panic diagnostics without Rust's default text panic hook +//! - JSON tracing initialisation on stderr with `RUST_LOG` / `--debug` precedence //! - JSON output on stdout +//! - small runners for stdout-producing and no-stdout commands +use std::borrow::Cow; +use std::ffi::{OsStr, OsString}; +use std::future::Future; use std::io::{self, IsTerminal, Write}; +use std::process::ExitCode; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::{Mutex, MutexGuard, TryLockError}; + +use clap::{CommandFactory, Parser}; +use serde::{Deserialize, Serialize}; +use tracing_subscriber::fmt::MakeWriter; #[cfg(test)] mod tests; -/// Refuse to run if stdout is a terminal (P8). +/// Schema version for shared ADR-T-010 control-plane records. +pub const CONTROL_PLANE_SCHEMA: u32 = 1; + +/// Placeholder used when a diagnostic value is intentionally hidden. +pub const REDACTED: &str = "[redacted]"; + +static PANIC_REPORTED: AtomicBool = AtomicBool::new(false); +static PANIC_PAYLOAD_REPORTING_ENABLED: AtomicBool = AtomicBool::new(true); +static STDERR_WRITE_LOCK: Mutex<()> = Mutex::new(()); + +/// Baseline exit-code classes shared by ADR-T-010 command-line tools. +#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub enum CommandExit { + /// Successful completion. + Success, + /// Runtime, startup, internal, or command execution failure. + Failure, + /// Command-line usage failure, including clap errors and TTY refusal. + Usage, +} + +impl CommandExit { + /// Return the numeric process status for this exit class. + #[must_use] + pub const fn code(self) -> u8 { + match self { + Self::Success => 0, + Self::Failure => 1, + Self::Usage => 2, + } + } + + /// Return this class as a standard library [`ExitCode`]. + #[must_use] + pub fn exit_code(self) -> ExitCode { + ExitCode::from(self.code()) + } + + /// Map a numeric status back to a shared exit class. + #[must_use] + pub const fn from_code(code: u8) -> Option { + match code { + 0 => Some(Self::Success), + 1 => Some(Self::Failure), + 2 => Some(Self::Usage), + _ => None, + } + } +} + +/// JSON record kinds emitted on stderr as ADR-T-010 control-plane data. +#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub enum ControlPlaneRecordKind { + /// Help text requested by the caller. + Help, + /// Version information requested by the caller. + Version, + /// Command-line usage or argv parsing failure. + UsageError, + /// Refusal to write stdout result data directly to a terminal. + TtyRefusal, + /// Panic diagnostic emitted by the shared panic hook. + Panic, + /// Non-error status update. + Status, + /// Runtime diagnostic or error. + Diagnostic, +} + +/// Standard streams referenced by control-plane records. +#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub enum StandardStream { + /// Standard output. + Stdout, + /// Standard error. + Stderr, +} + +/// Structured fields attached to a control-plane record. +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +#[serde(tag = "type", rename_all = "snake_case")] +pub enum ControlPlaneFields { + /// Help text returned by clap or a command-specific renderer. + Help { text: String }, + /// Version string returned by clap or the command. + Version { version: String }, + /// Details for an argv parsing or usage error. + UsageError { exit_code: u8, clap_error_kind: String }, + /// Details for stdout TTY refusal. + TtyRefusal { exit_code: u8, stream: StandardStream }, + /// Details for a panic diagnostic. The panic payload is only exposed when debug diagnostics are enabled. + Panic { + exit_code: u8, + thread: Option, + location: Option, + #[serde(skip_serializing_if = "Option::is_none")] + payload: Option, + }, +} + +/// Shared JSON control-plane record written to stderr. +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +pub struct ControlPlaneRecord { + /// Record schema version. + pub schema: u32, + /// Binary or entrypoint name. + pub command: String, + /// Machine-readable record kind. + pub kind: ControlPlaneRecordKind, + /// Short human-readable message, still carried inside JSON. + pub message: String, + /// Kind-specific structured fields. + #[serde(skip_serializing_if = "Option::is_none")] + pub fields: Option, +} + +impl ControlPlaneRecord { + /// Build a generic control-plane record. + #[must_use] + pub fn new(command: &str, kind: ControlPlaneRecordKind, message: &str, fields: Option) -> Self { + Self { + schema: CONTROL_PLANE_SCHEMA, + command: command.to_string(), + kind, + message: message.to_string(), + fields, + } + } + + /// Build a help record. + #[must_use] + pub fn help(command: &str, text: &str) -> Self { + Self::new( + command, + ControlPlaneRecordKind::Help, + "help requested", + Some(ControlPlaneFields::Help { text: text.to_string() }), + ) + } + + /// Build a version record. + #[must_use] + pub fn version(command: &str, version: &str) -> Self { + Self::new( + command, + ControlPlaneRecordKind::Version, + "version requested", + Some(ControlPlaneFields::Version { + version: version.to_string(), + }), + ) + } + + /// Build an argv usage-error record. + #[must_use] + pub fn usage_error(command: &str, message: &str, clap_error_kind: &str) -> Self { + Self::new( + command, + ControlPlaneRecordKind::UsageError, + message, + Some(ControlPlaneFields::UsageError { + exit_code: CommandExit::Usage.code(), + clap_error_kind: clap_error_kind.to_string(), + }), + ) + } + + /// Build a stdout TTY-refusal record. + #[must_use] + pub fn tty_refusal(command: &str) -> Self { + Self::new( + command, + ControlPlaneRecordKind::TtyRefusal, + "stdout is a terminal; pipe to a file or another process", + Some(ControlPlaneFields::TtyRefusal { + exit_code: CommandExit::Usage.code(), + stream: StandardStream::Stdout, + }), + ) + } + + /// Build a panic diagnostic record. + #[must_use] + pub fn panic(command: &str, thread: Option<&str>, location: Option<&str>, payload: Option<&str>) -> Self { + Self::new( + command, + ControlPlaneRecordKind::Panic, + "unexpected panic", + Some(ControlPlaneFields::Panic { + exit_code: CommandExit::Failure.code(), + thread: thread.map(str::to_string), + location: location.map(str::to_string), + payload: payload.map(str::to_string), + }), + ) + } + + /// Build a non-error status record. + #[must_use] + pub fn status(command: &str, message: &str) -> Self { + Self::new(command, ControlPlaneRecordKind::Status, message, None) + } + + /// Build a runtime diagnostic record. + #[must_use] + pub fn diagnostic(command: &str, message: &str) -> Self { + Self::new(command, ControlPlaneRecordKind::Diagnostic, message, None) + } +} + +/// Control-plane record and exit class produced while parsing argv. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct CliExit { + /// Record to write to stderr. + pub record: ControlPlaneRecord, + /// Process exit class to use after writing the record. + pub exit: CommandExit, +} + +impl CliExit { + /// Build a new parse-control exit value. + #[must_use] + pub const fn new(record: ControlPlaneRecord, exit: CommandExit) -> Self { + Self { record, exit } + } + + /// Return this exit class as a standard library [`ExitCode`]. + #[must_use] + pub fn exit_code(&self) -> ExitCode { + self.exit.exit_code() + } +} + +/// Source used to choose the JSON tracing filter directive. +#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub enum TracingFilterSource { + /// The `RUST_LOG` environment variable supplied the directive. + RustLog, + /// The shared `--debug` flag selected debug logging. + DebugFlag, + /// The caller-supplied default level was used. + DefaultLevel, +} + +/// Resolved JSON tracing filter directive. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct TracingFilter { + /// Directive passed to `tracing-subscriber`'s environment filter. + pub directive: String, + /// Input source that selected the directive. + pub source: TracingFilterSource, +} + +impl TracingFilter { + /// Build a resolved tracing filter directive. + #[must_use] + pub fn new(directive: impl Into, source: TracingFilterSource) -> Self { + Self { + directive: directive.into(), + source, + } + } +} + +/// Return true when a diagnostic field name is likely to contain a secret. +#[must_use] +pub fn is_sensitive_field_name(field_name: &str) -> bool { + const SENSITIVE_MARKERS: &[&str] = &[ + "admin_token", + "api_key", + "apikey", + "credential", + "credentials", + "jwt_secret", + "mailer_password", + "passwd", + "password", + "private_key", + "pwd", + "secret", + "session_secret", + "smtp_password", + "token", + ]; + + let normalized = normalize_name(field_name); + SENSITIVE_MARKERS.iter().any(|marker| normalized.contains(marker)) +} + +/// Redact a diagnostic field value according to the ADR-T-010 policy. +#[must_use] +pub fn redact_field_value<'a>(field_name: &str, value: &'a str) -> Cow<'a, str> { + if is_sensitive_field_name(field_name) { + Cow::Borrowed(REDACTED) + } else { + redact_database_url(value) + } +} + +/// Redact credentials and secret query parameters from a database URL. +#[must_use] +pub fn redact_database_url(value: &str) -> Cow<'_, str> { + const DATABASE_SCHEMES: &[&str] = &["mariadb", "mysql", "postgres", "postgresql", "sqlite"]; + + let Ok(mut url) = url::Url::parse(value) else { + return Cow::Borrowed(value); + }; + + if !DATABASE_SCHEMES.contains(&url.scheme()) { + return Cow::Borrowed(value); + } + + let username_changed = !url.username().is_empty() && url.set_username("").is_ok(); + let password_changed = url.password().is_some() && url.set_password(None).is_ok(); + + let mut safe_query_pairs = Vec::new(); + let mut removed_query_pair = false; + for (key, query_value) in url.query_pairs() { + if is_sensitive_query_key(&key) { + removed_query_pair = true; + } else { + safe_query_pairs.push((key.into_owned(), query_value.into_owned())); + } + } + + if removed_query_pair { + url.set_query(None); + { + let mut query_pairs = url.query_pairs_mut(); + for (key, query_value) in safe_query_pairs { + query_pairs.append_pair(&key, &query_value); + } + } + } + + let changed = username_changed || password_changed || removed_query_pair; + + if changed { + Cow::Owned(url.to_string()) + } else { + Cow::Borrowed(value) + } +} + +fn normalize_name(name: &str) -> String { + let mut normalized = String::with_capacity(name.len()); + for ch in name.chars() { + if ch.is_ascii_alphanumeric() { + normalized.push(ch.to_ascii_lowercase()); + } else { + normalized.push('_'); + } + } + normalized +} + +fn is_sensitive_query_key(key: &str) -> bool { + let normalized = normalize_name(key); + normalized == "key" || normalized.ends_with("_key") || is_sensitive_field_name(key) +} + +/// Resolve the JSON tracing filter with ADR-T-010 precedence. /// -/// Emits a `tracing::error!` event (NDJSON on stderr, per P9) -/// and exits with code 2. Call this **after** [`init_json_tracing`] -/// so the diagnostic is structured rather than a bare `eprintln!`. -pub fn refuse_if_stdout_is_tty(binary_name: &str) { +/// `RUST_LOG` wins when set and non-empty. Otherwise `--debug` selects `debug`, +/// and the caller-supplied default level is used last. +#[must_use] +pub fn tracing_filter(debug: bool, default_level: tracing::Level) -> TracingFilter { + tracing_filter_from_rust_log(std::env::var_os("RUST_LOG").as_deref(), debug, default_level) +} + +fn tracing_filter_from_rust_log(rust_log: Option<&OsStr>, debug: bool, default_level: tracing::Level) -> TracingFilter { + if let Some(value) = rust_log { + let directive = value.to_string_lossy(); + let trimmed = directive.trim(); + if !trimmed.is_empty() { + return TracingFilter::new(trimmed, TracingFilterSource::RustLog); + } + } + + if debug { + TracingFilter::new("debug", TracingFilterSource::DebugFlag) + } else { + TracingFilter::new(level_directive(default_level), TracingFilterSource::DefaultLevel) + } +} + +fn level_directive(level: tracing::Level) -> String { + level.as_str().to_ascii_lowercase() +} + +/// Parse argv with `clap`, converting help, version, and usage errors into +/// ADR-T-010 JSON control-plane records. +/// +/// # Errors +/// +/// Returns [`CliExit`] when parsing should stop and the caller should write the +/// enclosed record to stderr before exiting with the enclosed exit class. +/// The error value is intentionally returned by value because this path is only +/// used when parsing stops, and boxing it would complicate the public helper API. +#[allow(clippy::result_large_err)] +pub fn parse_args_from(args: I) -> Result +where + T: Parser, + I: IntoIterator, + A: Into + Clone, +{ + T::try_parse_from(args).map_err(|clap_error| cli_exit_from_clap_error::(&clap_error)) +} + +/// Parse process argv with `clap`, writing ADR-T-010 control-plane records and +/// exiting when parsing is a help, version, or usage-control path. +#[must_use] +pub fn parse_args_or_exit() -> T +where + T: Parser, +{ + match parse_args_from::(std::env::args_os()) { + Ok(args) => args, + Err(exit) => { + let _ignored = emit_control_plane_record(&exit.record); + exit_with(exit.exit); + } + } +} + +fn cli_exit_from_clap_error(clap_error: &clap::Error) -> CliExit +where + T: CommandFactory, +{ + let command_name = T::command().get_name().to_string(); + let text = clap_error.to_string().trim_end().to_string(); + + match clap_error.kind() { + clap::error::ErrorKind::DisplayHelp => CliExit::new(ControlPlaneRecord::help(&command_name, &text), CommandExit::Success), + clap::error::ErrorKind::DisplayVersion => { + CliExit::new(ControlPlaneRecord::version(&command_name, &text), CommandExit::Success) + } + _ => CliExit::new( + ControlPlaneRecord::usage_error(&command_name, &text, &clap_error_kind_name(clap_error.kind())), + CommandExit::Usage, + ), + } +} + +fn clap_error_kind_name(kind: clap::error::ErrorKind) -> String { + debug_name_to_snake_case(&format!("{kind:?}")) +} + +fn debug_name_to_snake_case(name: &str) -> String { + let mut normalized = String::with_capacity(name.len()); + let mut previous_was_lower_or_digit = false; + + for character in name.chars() { + if character.is_ascii_uppercase() { + if !normalized.is_empty() && previous_was_lower_or_digit { + normalized.push('_'); + } + normalized.push(character.to_ascii_lowercase()); + previous_was_lower_or_digit = false; + } else { + normalized.push(character); + previous_was_lower_or_digit = character.is_ascii_lowercase() || character.is_ascii_digit(); + } + } + + normalized +} + +/// Write one ADR-T-010 control-plane JSON record to stderr. +/// +/// This helper does not depend on `tracing`, so it is safe for clap help, +/// clap parse errors, early startup failures, TTY refusal, and panic hooks. +/// +/// # Errors +/// +/// Returns an error if serialisation or writing to stderr fails. +pub fn emit_control_plane_record(record: &ControlPlaneRecord) -> io::Result<()> { + let mut writer = LockedStderrWriter::new(); + write_json_line(&mut writer, record) +} + +fn try_emit_control_plane_record(record: &ControlPlaneRecord) -> io::Result { + let Some(mut writer) = LockedStderrWriter::try_new() else { + return Ok(false); + }; + + write_json_line(&mut writer, record)?; + Ok(true) +} + +fn write_json_line(writer: &mut W, value: &T) -> io::Result<()> { + let json = serde_json::to_string(value)?; + writer.write_all(json.as_bytes())?; + writer.write_all(b"\n")?; + writer.flush() +} + +/// Install a JSON-only panic hook for ADR-T-010 command-line entrypoints. +/// +/// The hook emits one best-effort control-plane record on stderr and terminates +/// the process with exit code 1 without invoking Rust's default text panic hook. +pub fn install_json_panic_hook(command_name: &str) { + let command_name = command_name.to_string(); + std::panic::set_hook(Box::new(move |panic_info| { + if !PANIC_REPORTED.swap(true, Ordering::SeqCst) { + let current_thread = std::thread::current(); + let thread_name = current_thread.name(); + let location = panic_info + .location() + .map(|location| format!("{}:{}:{}", location.file(), location.line(), location.column())); + let payload = panic_payload_reporting_enabled() + .then(|| panic_payload_message(panic_info)) + .flatten(); + let record = ControlPlaneRecord::panic(&command_name, thread_name, location.as_deref(), payload); + let _ignored = try_emit_control_plane_record(&record); + } + + exit_with(CommandExit::Failure); + })); +} + +/// Enable or disable string panic payloads in JSON panic diagnostics. +/// +/// Payload reporting starts enabled so panics before argument parsing still +/// include their string payload. Call this with the parsed `--debug` value once +/// arguments are available. +pub fn set_panic_payload_reporting_enabled(enabled: bool) { + PANIC_PAYLOAD_REPORTING_ENABLED.store(enabled, Ordering::SeqCst); +} + +fn panic_payload_reporting_enabled() -> bool { + PANIC_PAYLOAD_REPORTING_ENABLED.load(Ordering::SeqCst) +} + +fn panic_payload_message<'a>(info: &'a std::panic::PanicHookInfo<'_>) -> Option<&'a str> { + panic_payload_message_from_payload(info.payload()) +} + +fn panic_payload_message_from_payload(payload: &(dyn std::any::Any + Send)) -> Option<&str> { + payload + .downcast_ref::<&str>() + .copied() + .or_else(|| payload.downcast_ref::().map(String::as_str)) +} + +/// Exit the current process with an ADR-T-010 exit class. +#[allow(clippy::exit)] +pub fn exit_with(exit: CommandExit) -> ! { + std::process::exit(i32::from(exit.code())); +} + +/// Return a TTY-refusal record when stdout is attached to a terminal. +#[must_use] +pub fn stdout_tty_refusal_record(command_name: &str) -> Option { if io::stdout().is_terminal() { - tracing::error!( - binary = binary_name, - "stdout is a terminal \u{2014} pipe to a file or another process" - ); - std::process::exit(2); + Some(ControlPlaneRecord::tty_refusal(command_name)) + } else { + None + } +} + +/// Refuse to run if stdout is a terminal (ADR-T-010). +/// +/// Emits a JSON control-plane record on stderr and exits with code 2. +pub fn refuse_if_stdout_is_tty(binary_name: &str) { + if let Some(record) = stdout_tty_refusal_record(binary_name) { + let _ignored = emit_control_plane_record(&record); + exit_with(CommandExit::Usage); } } /// Initialise `tracing-subscriber` with JSON output on stderr. pub fn init_json_tracing(level: tracing::Level) { + let _installed = try_init_json_tracing(level); +} + +/// Try to initialise `tracing-subscriber` with JSON output on stderr. +/// +/// Returns `false` if another subscriber is already installed. +#[must_use] +pub fn try_init_json_tracing(level: tracing::Level) -> bool { + let filter = tracing_filter(false, level); + try_init_json_tracing_with_directive(&filter.directive) +} + +/// Initialise JSON stderr tracing with `RUST_LOG` / `--debug` precedence. +pub fn init_json_tracing_with_debug(debug: bool, default_level: tracing::Level) { + let _installed = try_init_json_tracing_with_debug(debug, default_level); +} + +/// Try to initialise JSON stderr tracing with `RUST_LOG` / `--debug` precedence. +/// +/// Returns `false` if another subscriber is already installed. +#[must_use] +pub fn try_init_json_tracing_with_debug(debug: bool, default_level: tracing::Level) -> bool { + let filter = tracing_filter(debug, default_level); + try_init_json_tracing_with_directive(&filter.directive) +} + +fn try_init_json_tracing_with_directive(filter_directive: &str) -> bool { + let fallback_directive = level_directive(tracing::Level::INFO); + let filter = tracing_subscriber::EnvFilter::try_new(filter_directive) + .unwrap_or_else(|_error| tracing_subscriber::EnvFilter::new(fallback_directive)); + tracing_subscriber::fmt() .json() - .with_max_level(level) - .with_writer(io::stderr) - .init(); + .with_env_filter(filter) + .with_writer(LockedStderr) + .try_init() + .is_ok() } /// Serialise `value` as one JSON object + trailing newline to stdout. @@ -40,12 +646,154 @@ pub fn init_json_tracing(level: tracing::Level) { /// /// Returns an error if serialisation or writing to stdout fails. pub fn emit(value: &T) -> io::Result<()> { - let json = serde_json::to_string(value)?; let stdout = io::stdout(); let mut out = stdout.lock(); - out.write_all(json.as_bytes())?; - out.write_all(b"\n")?; - out.flush() + write_json_line(&mut out, value) +} + +/// Run a stdout-producing single-JSON-object command. +/// +/// The runner installs the JSON panic hook, initialises JSON stderr tracing, +/// refuses terminal stdout, writes successful result data to stdout, and maps +/// failures to ADR-T-010 baseline exit classes. +pub fn run_stdout_json_command( + command_name: &str, + debug: bool, + default_level: tracing::Level, + run: Run, +) -> ExitCode +where + Output: serde::Serialize, + CommandError: std::fmt::Display, + Run: FnOnce() -> Result, +{ + set_panic_payload_reporting_enabled(debug); + install_json_panic_hook(command_name); + init_json_tracing_with_debug(debug, default_level); + + if let Some(record) = stdout_tty_refusal_record(command_name) { + let _ignored = emit_control_plane_record(&record); + return CommandExit::Usage.exit_code(); + } + + match run() { + Ok(output) => match emit(&output) { + Ok(()) => CommandExit::Success.exit_code(), + Err(error) => { + tracing::error!(error = %error, "failed to write JSON to stdout"); + CommandExit::Failure.exit_code() + } + }, + Err(error) => { + tracing::error!(error = %error, "command failed"); + CommandExit::Failure.exit_code() + } + } +} + +/// Run a side-effect command that does not emit stdout result data. +/// +/// The runner installs the JSON panic hook, initialises JSON stderr tracing, and +/// maps failures to ADR-T-010 baseline exit classes. It deliberately does not +/// perform stdout TTY refusal because the command has no stdout result data. +pub fn run_no_stdout_command( + command_name: &str, + debug: bool, + default_level: tracing::Level, + run: Run, +) -> ExitCode +where + CommandError: std::fmt::Display, + Run: FnOnce() -> Result<(), CommandError>, +{ + set_panic_payload_reporting_enabled(debug); + install_json_panic_hook(command_name); + init_json_tracing_with_debug(debug, default_level); + + match run() { + Ok(()) => CommandExit::Success.exit_code(), + Err(error) => { + tracing::error!(error = %error, "command failed"); + CommandExit::Failure.exit_code() + } + } +} + +/// Run an async side-effect command that does not emit stdout result data. +/// +/// The runner installs the JSON panic hook, initialises JSON stderr tracing, and +/// maps failures to ADR-T-010 baseline exit classes. It deliberately does not +/// perform stdout TTY refusal because the command has no stdout result data. +pub async fn run_no_stdout_command_async( + command_name: &str, + debug: bool, + default_level: tracing::Level, + run: Run, +) -> ExitCode +where + CommandError: std::fmt::Display, + Run: FnOnce() -> RunFuture, + RunFuture: Future>, +{ + set_panic_payload_reporting_enabled(debug); + install_json_panic_hook(command_name); + init_json_tracing_with_debug(debug, default_level); + + match run().await { + Ok(()) => CommandExit::Success.exit_code(), + Err(error) => { + tracing::error!(error = %error, "command failed"); + CommandExit::Failure.exit_code() + } + } +} + +struct LockedStderr; + +impl<'writer> MakeWriter<'writer> for LockedStderr { + type Writer = LockedStderrWriter; + + fn make_writer(&'writer self) -> Self::Writer { + LockedStderrWriter::new() + } +} + +struct LockedStderrWriter { + _guard: MutexGuard<'static, ()>, + stderr: io::Stderr, +} + +impl LockedStderrWriter { + fn new() -> Self { + Self { + _guard: STDERR_WRITE_LOCK.lock().unwrap_or_else(std::sync::PoisonError::into_inner), + stderr: io::stderr(), + } + } + + fn try_new() -> Option { + match STDERR_WRITE_LOCK.try_lock() { + Ok(guard) => Some(Self { + _guard: guard, + stderr: io::stderr(), + }), + Err(TryLockError::Poisoned(poisoned)) => Some(Self { + _guard: poisoned.into_inner(), + stderr: io::stderr(), + }), + Err(TryLockError::WouldBlock) => None, + } + } +} + +impl Write for LockedStderrWriter { + fn write(&mut self, buf: &[u8]) -> io::Result { + self.stderr.write(buf) + } + + fn flush(&mut self) -> io::Result<()> { + self.stderr.flush() + } } /// Common `--debug` flag for all helpers. Flatten into each diff --git a/packages/index-cli-common/src/tests/mod.rs b/packages/index-cli-common/src/tests/mod.rs index a00740988..73029ecbc 100644 --- a/packages/index-cli-common/src/tests/mod.rs +++ b/packages/index-cli-common/src/tests/mod.rs @@ -8,6 +8,24 @@ //! | `emit_to_real_stdout_succeeds` | `emit` itself writes to the captured stdout. | //! | `base_args_parses_default` | `BaseArgs::debug` defaults to `false`. | //! | `base_args_parses_long_flag` | `--debug` flips `BaseArgs::debug` to `true`. | +//! | `command_exit_codes_match_contract` | Baseline process statuses are fixed. | +//! | `control_record_serialises_shape` | Shared stderr record shape is stable. | +//! | `json_line_writer_appends_newline` | JSON record helper writes one complete line. | +//! | `usage_error_record_carries_fields` | Usage records include exit code and clap kind. | +//! | `tty_refusal_record_carries_fields` | TTY refusal records identify stdout and code 2. | +//! | `panic_record_omits_payload_without_debug` | Panic records hide payloads without debug. | +//! | `panic_record_carries_debug_payload` | Panic records can expose string payloads. | +//! | `panic_payload_reporting_defaults_enabled_then_follows_debug_flag` | Startup payload gate behavior. | +//! | `panic_payload_message_extracts_string_payloads` | String panic payloads are downcast. | +//! | `parse_args_from_returns_help_record` | Clap help becomes JSON stderr control data. | +//! | `parse_args_from_returns_version_record` | Clap version becomes JSON stderr control data. | +//! | `parse_args_from_returns_usage_record` | Clap argv errors become JSON usage records. | +//! | `tracing_filter_prefers_rust_log` | `RUST_LOG` wins over `--debug`. | +//! | `tracing_filter_uses_debug_flag` | `--debug` selects debug without `RUST_LOG`. | +//! | `tracing_filter_uses_default_level` | Default level is used last. | +//! | `redacts_sensitive_field_values` | Secret-like field names are hidden. | +//! | `redacts_database_url_secrets` | DB credentials and query secrets are removed. | +//! | `keeps_public_key_fields_visible` | Public key metadata is not treated as secret. | //! //! `refuse_if_stdout_is_tty` and `init_json_tracing` mutate //! global process state (the `process::exit` path and the @@ -18,12 +36,18 @@ //! interferes with the rest of the test binary's output. use std::collections::BTreeMap; +use std::ffi::OsStr; use std::io::{self, Write}; use clap::Parser; use serde::Serialize; +use serde_json::json; -use crate::BaseArgs; +use crate::{ + BaseArgs, CONTROL_PLANE_SCHEMA, CommandExit, ControlPlaneFields, ControlPlaneRecord, ControlPlaneRecordKind, REDACTED, + StandardStream, TracingFilterSource, panic_payload_message_from_payload, panic_payload_reporting_enabled, parse_args_from, + redact_database_url, redact_field_value, set_panic_payload_reporting_enabled, tracing_filter_from_rust_log, write_json_line, +}; /// A `Write` that fails every call with `BrokenPipe`. /// @@ -41,6 +65,17 @@ impl Write for FailingWriter { } } +#[derive(Parser)] +#[command(name = "fixture-helper", version = "1.2.3", about = "Fixture helper")] +#[allow(dead_code)] +struct FixtureCli { + #[arg(long)] + name: Option, + + #[command(flatten)] + base: BaseArgs, +} + /// Pure helper that mirrors `emit` but writes to an arbitrary /// `Write`. Lets us assert the on-the-wire bytes without /// touching the real `stdout()` lock (which would interleave @@ -131,3 +166,224 @@ fn base_args_parses_long_flag() { let parsed = Cli::try_parse_from(["prog", "--debug"]).expect("--debug is valid"); assert!(parsed.base.debug); } + +#[test] +fn command_exit_codes_match_contract() { + assert_eq!(CommandExit::Success.code(), 0); + assert_eq!(CommandExit::Failure.code(), 1); + assert_eq!(CommandExit::Usage.code(), 2); + + assert_eq!(CommandExit::from_code(0), Some(CommandExit::Success)); + assert_eq!(CommandExit::from_code(1), Some(CommandExit::Failure)); + assert_eq!(CommandExit::from_code(2), Some(CommandExit::Usage)); + assert_eq!(CommandExit::from_code(3), None); +} + +#[test] +fn control_record_serialises_shape() { + let record = ControlPlaneRecord::new("fixture", ControlPlaneRecordKind::Status, "ready", None); + let value = serde_json::to_value(record).unwrap(); + + assert_eq!(value["schema"], json!(CONTROL_PLANE_SCHEMA)); + assert_eq!(value["command"], json!("fixture")); + assert_eq!(value["kind"], json!("status")); + assert_eq!(value["message"], json!("ready")); + assert!(value.get("fields").is_none()); +} + +#[test] +fn json_line_writer_appends_newline() { + let record = ControlPlaneRecord::status("fixture", "ready"); + let mut buf = Vec::new(); + + write_json_line(&mut buf, &record).expect("record should serialize"); + + assert!(buf.ends_with(b"\n")); + let line = std::str::from_utf8(&buf).expect("JSON must be UTF-8"); + let value: serde_json::Value = serde_json::from_str(line).expect("record must be a JSON object"); + assert_eq!(value["schema"], json!(CONTROL_PLANE_SCHEMA)); + assert_eq!(value["kind"], json!("status")); +} + +#[test] +fn usage_error_record_carries_fields() { + let record = ControlPlaneRecord::usage_error("fixture", "unknown argument", "unknown_argument"); + + assert_eq!(record.kind, ControlPlaneRecordKind::UsageError); + assert_eq!( + record.fields, + Some(ControlPlaneFields::UsageError { + exit_code: CommandExit::Usage.code(), + clap_error_kind: "unknown_argument".to_string(), + }) + ); + + let value = serde_json::to_value(record).unwrap(); + assert_eq!(value["fields"]["type"], json!("usage_error")); + assert_eq!(value["fields"]["exit_code"], json!(2)); +} + +#[test] +fn tty_refusal_record_carries_fields() { + let record = ControlPlaneRecord::tty_refusal("fixture"); + + assert_eq!(record.kind, ControlPlaneRecordKind::TtyRefusal); + assert_eq!( + record.fields, + Some(ControlPlaneFields::TtyRefusal { + exit_code: CommandExit::Usage.code(), + stream: StandardStream::Stdout, + }) + ); +} + +#[test] +fn panic_record_omits_payload_without_debug() { + let record = ControlPlaneRecord::panic("fixture", Some("main"), Some("src/main.rs:12:34"), None); + let value = serde_json::to_value(record).unwrap(); + + assert_eq!(value["kind"], json!("panic")); + assert_eq!(value["fields"]["type"], json!("panic")); + assert_eq!(value["fields"]["exit_code"], json!(1)); + assert_eq!(value["fields"]["thread"], json!("main")); + assert!(value["fields"].get("payload").is_none()); +} + +#[test] +fn panic_record_carries_debug_payload() { + let record = ControlPlaneRecord::panic( + "fixture", + Some("main"), + Some("src/main.rs:12:34"), + Some("panic with \"quoted\" detail"), + ); + let line = serde_json::to_string(&record).unwrap(); + + assert!(line.contains(r#""payload":"panic with \"quoted\" detail""#)); + + let value: serde_json::Value = serde_json::from_str(&line).unwrap(); + assert_eq!(value["fields"]["payload"], json!("panic with \"quoted\" detail")); +} + +#[test] +fn panic_payload_reporting_defaults_enabled_then_follows_debug_flag() { + assert!(panic_payload_reporting_enabled()); + + set_panic_payload_reporting_enabled(false); + assert!(!panic_payload_reporting_enabled()); + + set_panic_payload_reporting_enabled(true); + assert!(panic_payload_reporting_enabled()); +} + +#[test] +fn panic_payload_message_extracts_string_payloads() { + let borrowed_payload: &(dyn std::any::Any + Send) = &"borrowed panic"; + let owned_payload: &(dyn std::any::Any + Send) = &String::from("owned panic"); + let numeric_payload: &(dyn std::any::Any + Send) = &1_u8; + + assert_eq!(panic_payload_message_from_payload(borrowed_payload), Some("borrowed panic")); + assert_eq!(panic_payload_message_from_payload(owned_payload), Some("owned panic")); + assert_eq!(panic_payload_message_from_payload(numeric_payload), None); +} + +#[test] +fn parse_args_from_returns_help_record() { + let Err(exit) = parse_args_from::(["fixture-helper", "--help"]) else { + panic!("help should stop parsing"); + }; + + assert_eq!(exit.exit, CommandExit::Success); + assert_eq!(exit.record.command, "fixture-helper"); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::Help); + + let Some(ControlPlaneFields::Help { text }) = exit.record.fields else { + panic!("help record should carry help text"); + }; + assert!(text.contains("Fixture helper")); + assert!(text.contains("--debug")); +} + +#[test] +fn parse_args_from_returns_version_record() { + let Err(exit) = parse_args_from::(["fixture-helper", "--version"]) else { + panic!("version should stop parsing"); + }; + + assert_eq!(exit.exit, CommandExit::Success); + assert_eq!(exit.record.command, "fixture-helper"); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::Version); + + let Some(ControlPlaneFields::Version { version }) = exit.record.fields else { + panic!("version record should carry version text"); + }; + assert_eq!(version, "fixture-helper 1.2.3"); +} + +#[test] +fn parse_args_from_returns_usage_record() { + let Err(exit) = parse_args_from::(["fixture-helper", "--no-such-flag"]) else { + panic!("unknown flags should stop parsing"); + }; + + assert_eq!(exit.exit, CommandExit::Usage); + assert_eq!(exit.record.command, "fixture-helper"); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::UsageError); + assert!(exit.record.message.contains("--no-such-flag")); + + let Some(ControlPlaneFields::UsageError { + exit_code, + clap_error_kind, + }) = exit.record.fields + else { + panic!("usage record should carry usage fields"); + }; + assert_eq!(exit_code, CommandExit::Usage.code()); + assert_eq!(clap_error_kind, "unknown_argument"); +} + +#[test] +fn tracing_filter_prefers_rust_log() { + let filter = tracing_filter_from_rust_log(Some(OsStr::new("warn,tower_http=debug")), true, tracing::Level::INFO); + + assert_eq!(filter.directive, "warn,tower_http=debug"); + assert_eq!(filter.source, TracingFilterSource::RustLog); +} + +#[test] +fn tracing_filter_uses_debug_flag() { + let filter = tracing_filter_from_rust_log(Some(OsStr::new(" ")), true, tracing::Level::INFO); + + assert_eq!(filter.directive, "debug"); + assert_eq!(filter.source, TracingFilterSource::DebugFlag); +} + +#[test] +fn tracing_filter_uses_default_level() { + let filter = tracing_filter_from_rust_log(None, false, tracing::Level::WARN); + + assert_eq!(filter.directive, "warn"); + assert_eq!(filter.source, TracingFilterSource::DefaultLevel); +} + +#[test] +fn redacts_sensitive_field_values() { + assert_eq!(redact_field_value("tracker.token", "MyAccessToken"), REDACTED); + assert_eq!(redact_field_value("smtp_password", "secret"), REDACTED); + assert_eq!(redact_field_value("auth.private_key_pem", "PEM"), REDACTED); +} + +#[test] +fn redacts_database_url_secrets() { + let redacted = redact_database_url("mysql://user:pass@example.test/db?ssl-mode=required&token=abc&password=def"); + + assert_eq!(redacted, "mysql://example.test/db?ssl-mode=required"); +} + +#[test] +fn keeps_public_key_fields_visible() { + assert_eq!( + redact_field_value("auth.public_key_path", "/etc/torrust/public.pem"), + "/etc/torrust/public.pem" + ); +} diff --git a/packages/index-cli-common/tests/public_api.rs b/packages/index-cli-common/tests/public_api.rs index f7bae95fb..c308d5ff5 100644 --- a/packages/index-cli-common/tests/public_api.rs +++ b/packages/index-cli-common/tests/public_api.rs @@ -14,10 +14,11 @@ //! | `base_args_flattens_into_clap_parser` | `BaseArgs` composes via `#[command(flatten)]` | //! | `base_args_long_flag_toggles_debug` | The `--debug` long flag flips the field | //! | `base_args_rejects_unknown_short_flag` | clap rejects an unrelated flag at parse time | +//! | `parse_args_from_wraps_help_as_json_control_record` | shared parser returns JSON help metadata | use clap::Parser; use serde::Serialize; -use torrust_index_cli_common::{BaseArgs, emit}; +use torrust_index_cli_common::{BaseArgs, CommandExit, ControlPlaneFields, ControlPlaneRecordKind, emit, parse_args_from}; /// Minimal helper-binary-shaped CLI: every helper composes /// `BaseArgs` via `#[command(flatten)]`, so this fixture @@ -31,6 +32,14 @@ struct FixtureCli { base: BaseArgs, } +#[derive(Parser)] +#[command(name = "fixture-helper", version = "1.2.3", about = "Fixture helper")] +#[allow(dead_code)] +struct FixtureHelpCli { + #[command(flatten)] + base: BaseArgs, +} + #[test] fn emit_real_stdout_through_public_api() { // Under `cargo test`, stdout is captured by the harness, @@ -62,9 +71,24 @@ fn base_args_rejects_unknown_short_flag() { // Pins clap's contract that unknown flags surface as a // parse error rather than being silently dropped — every // helper binary depends on this for the `unknown-flag → - // exit 2` mapping documented in ADR-T-009 §6.1. + // exit 2` mapping documented in ADR-T-010. let Err(err) = FixtureCli::try_parse_from(["fixture-helper", "--no-such-flag"]) else { panic!("unknown flag must be rejected by clap"); }; assert_eq!(err.exit_code(), 2, "clap argv-parse failure exit code is 2"); } + +#[test] +fn parse_args_from_wraps_help_as_json_control_record() { + let Err(exit) = parse_args_from::(["fixture-helper", "--help"]) else { + panic!("help should return a control-plane exit record"); + }; + + assert_eq!(exit.exit, CommandExit::Success); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::Help); + + let Some(ControlPlaneFields::Help { text }) = exit.record.fields else { + panic!("help record should expose clap help text"); + }; + assert!(text.contains("Fixture helper")); +} diff --git a/packages/index-config-probe/src/bin/torrust-index-config-probe.rs b/packages/index-config-probe/src/bin/torrust-index-config-probe.rs index cfadecedb..05f3dd98c 100644 --- a/packages/index-config-probe/src/bin/torrust-index-config-probe.rs +++ b/packages/index-config-probe/src/bin/torrust-index-config-probe.rs @@ -1,19 +1,27 @@ //! Resolve the Torrust Index configuration and print the //! container-relevant subset as a single JSON object on stdout. +//! The JSON object contains `schema`, `database`, and `auth`. +//! Direct terminal stdout is refused; pipe or redirect the result. //! //! See ADR-T-009 §D3. use std::process::ExitCode; use clap::Parser; -use torrust_index_cli_common::{BaseArgs, emit, init_json_tracing, refuse_if_stdout_is_tty}; +use torrust_index_cli_common::{ + BaseArgs, emit, init_json_tracing_with_debug, install_json_panic_hook, parse_args_or_exit, refuse_if_stdout_is_tty, + set_panic_payload_reporting_enabled, +}; use torrust_index_config::{DEFAULT_CONFIG_TOML_PATH, Info, load_settings}; use torrust_index_config_probe::{ProbeError, probe}; use tracing::error; +const COMMAND_NAME: &str = "torrust-index-config-probe"; + #[derive(Parser)] #[command( name = "torrust-index-config-probe", + version, about = "Emit the container-relevant subset of the resolved Torrust Index configuration as JSON" )] struct Args { @@ -22,15 +30,12 @@ struct Args { } fn main() -> ExitCode { - install_panic_hook(); + install_json_panic_hook(COMMAND_NAME); - let args = Args::parse(); - init_json_tracing(if args.base.debug { - tracing::Level::DEBUG - } else { - tracing::Level::INFO - }); - refuse_if_stdout_is_tty("torrust-index-config-probe"); + let args = parse_args_or_exit::(); + set_panic_payload_reporting_enabled(args.base.debug); + init_json_tracing_with_debug(args.base.debug, tracing::Level::INFO); + refuse_if_stdout_is_tty(COMMAND_NAME); // `Info::from_env` is the JSON-safe sibling of `Info::new`: // it reads the same env vars but skips the diagnostic @@ -64,16 +69,72 @@ fn main() -> ExitCode { } } -/// Map any unhandled panic to exit code 1 so the contract in -/// ADR-T-009 §D3 ("1 — Internal error (unhandled -/// panic, unexpected I/O)") is honoured. Without this hook a -/// panic would exit with Rust's default 101. -fn install_panic_hook() { - let default = std::panic::take_hook(); - std::panic::set_hook(Box::new(move |info| { - // Preserve the default formatted backtrace on stderr, - // then exit with the documented code. - default(info); - std::process::exit(1); - })); +#[cfg(test)] +mod tests { + //! # Config-probe binary CLI contract tests + //! + //! | Test | What it covers | + //! |---------------------------------------|----------------------------------------| + //! | `help_is_json_control_record` | `--help` is wrapped as JSON metadata | + //! | `version_is_json_control_record` | `--version` is wrapped as JSON metadata| + //! | `usage_error_is_json_control_record` | argv errors become JSON usage records | + + use torrust_index_cli_common::{CommandExit, ControlPlaneFields, ControlPlaneRecordKind, parse_args_from}; + + use super::{Args, COMMAND_NAME}; + + #[test] + fn help_is_json_control_record() { + let Err(exit) = parse_args_from::([COMMAND_NAME, "--help"]) else { + panic!("help should stop parsing"); + }; + + assert_eq!(exit.exit, CommandExit::Success); + assert_eq!(exit.record.command, COMMAND_NAME); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::Help); + + let Some(ControlPlaneFields::Help { text }) = exit.record.fields else { + panic!("help record should carry help text"); + }; + assert!(text.contains("Emit the container-relevant subset")); + assert!(text.contains("--debug")); + } + + #[test] + fn version_is_json_control_record() { + let Err(exit) = parse_args_from::([COMMAND_NAME, "--version"]) else { + panic!("version should stop parsing"); + }; + + assert_eq!(exit.exit, CommandExit::Success); + assert_eq!(exit.record.command, COMMAND_NAME); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::Version); + + let Some(ControlPlaneFields::Version { version }) = exit.record.fields else { + panic!("version record should carry version text"); + }; + assert_eq!(version, format!("{COMMAND_NAME} {}", env!("CARGO_PKG_VERSION"))); + } + + #[test] + fn usage_error_is_json_control_record() { + let Err(exit) = parse_args_from::([COMMAND_NAME, "--no-such-flag"]) else { + panic!("unknown flags should stop parsing"); + }; + + assert_eq!(exit.exit, CommandExit::Usage); + assert_eq!(exit.record.command, COMMAND_NAME); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::UsageError); + assert!(exit.record.message.contains("--no-such-flag")); + + let Some(ControlPlaneFields::UsageError { + exit_code, + clap_error_kind, + }) = exit.record.fields + else { + panic!("usage record should carry usage fields"); + }; + assert_eq!(exit_code, CommandExit::Usage.code()); + assert_eq!(clap_error_kind, "unknown_argument"); + } } diff --git a/packages/index-config/src/lib.rs b/packages/index-config/src/lib.rs index ae7fbc49d..46e7b63eb 100644 --- a/packages/index-config/src/lib.rs +++ b/packages/index-config/src/lib.rs @@ -207,7 +207,7 @@ impl Info { // tokens, SMTP passwords, …) so log only the env-var name // — never its value — and route through `tracing` (stderr) // so we don't pollute the JSON-only stdout contract used - // by helper binaries (P9). + // by helper binaries (ADR-T-010). tracing::info!( env_var = ENV_VAR_CONFIG_TOML, "loading extra configuration from environment variable" @@ -229,7 +229,7 @@ impl Info { /// Build [`Info`] from the same env vars [`Self::new`] reads, /// without the diagnostic `println!`s. /// - /// Helper binaries that own a JSON-only stdout contract (P9) + /// Helper binaries that own a JSON-only stdout contract (ADR-T-010) /// must use this constructor instead of [`Self::new`] to avoid /// corrupting their output stream. #[must_use] diff --git a/packages/index-entry-script/Cargo.toml b/packages/index-entry-script/Cargo.toml index 1a1649eaf..adb34a79f 100644 --- a/packages/index-entry-script/Cargo.toml +++ b/packages/index-entry-script/Cargo.toml @@ -14,6 +14,7 @@ version.workspace = true path = "src/lib.rs" [dependencies] +serde_json = "1" tempfile = "3" [lints] diff --git a/packages/index-entry-script/src/lib.rs b/packages/index-entry-script/src/lib.rs index faa2455cb..51151dc31 100644 --- a/packages/index-entry-script/src/lib.rs +++ b/packages/index-entry-script/src/lib.rs @@ -7,7 +7,7 @@ //! The crate ships **no runtime code** of its own — it exists //! purely as a home for `tests/` that invoke `sh` as a //! subprocess against the shell library and assert exit -//! codes / stderr contents. This keeps the tests inside +//! codes / JSON stderr records. This keeps the tests inside //! `cargo test --workspace` (so CI runs them automatically) //! while the helpers themselves remain POSIX `sh`, since they //! must run inside the distroless busybox runtime where Rust @@ -36,12 +36,14 @@ //! `TORRUST_INDEX_CONFIG_OVERRIDE_AUTH__*_PATH` plus key //! materialisation against the real generator. //! -//! Both belong in the container e2e suite (Phase 8 / 9). +//! Both belong in the container e2e suite. use std::path::PathBuf; use std::process::{Command, Output, Stdio}; use std::sync::OnceLock; +use serde_json::Value; + /// Contents of the shell library, embedded at compile time. /// /// Embedding sidesteps the cargo-nextest archive → @@ -158,3 +160,64 @@ pub fn run_sh_with_args(snippet: &str, args: &[&str]) -> Output { } cmd.output().expect("failed to spawn sh; is /bin/sh available?") } + +/// Parse every stderr line as one JSON record. +/// +/// # Panics +/// +/// Panics when any non-empty stderr line is not valid JSON. +#[doc(hidden)] +#[must_use] +pub fn stderr_json_records(output: &Output) -> Vec { + let stderr = String::from_utf8_lossy(&output.stderr); + stderr + .lines() + .filter(|line| !line.is_empty()) + .map(|line| serde_json::from_str(line).unwrap_or_else(|error| panic!("stderr line is not JSON: {line}; error: {error}"))) + .collect() +} + +/// Return the single JSON record emitted on stderr. +/// +/// # Panics +/// +/// Panics when stderr does not contain exactly one JSON record. +#[doc(hidden)] +#[must_use] +pub fn single_stderr_json_record(output: &Output) -> Value { + let records = stderr_json_records(output); + assert_eq!( + records.len(), + 1, + "expected exactly one stderr JSON record; stderr={}", + String::from_utf8_lossy(&output.stderr), + ); + records.into_iter().next().expect("one record was asserted above") +} + +/// Assert the common entry-script JSON control-plane fields. +/// +/// # Panics +/// +/// Panics when the record does not match the entry-script contract or +/// the message does not contain `message_needle`. +#[doc(hidden)] +pub fn assert_entry_script_record(record: &Value, kind: &str, level: &str, message_needle: &str) { + assert_eq!(record.get("schema").and_then(Value::as_u64), Some(1)); + assert_eq!( + record.get("command").and_then(Value::as_str), + Some("torrust-index-entry-script"), + ); + assert_eq!(record.get("kind").and_then(Value::as_str), Some(kind)); + assert_eq!(record.pointer("/fields/type").and_then(Value::as_str), Some("entry_script"),); + assert_eq!(record.pointer("/fields/level").and_then(Value::as_str), Some(level)); + + let message = record + .get("message") + .and_then(Value::as_str) + .expect("record message must be a string"); + assert!( + message.contains(message_needle), + "expected message containing {message_needle:?}; got {message:?}", + ); +} diff --git a/packages/index-entry-script/tests/seed_sqlite.rs b/packages/index-entry-script/tests/seed_sqlite.rs index b4806103f..4faf64a2a 100644 --- a/packages/index-entry-script/tests/seed_sqlite.rs +++ b/packages/index-entry-script/tests/seed_sqlite.rs @@ -7,21 +7,21 @@ //! //! | Test | Outcome | //! |--------------------------------|----------------------------------| -//! | `empty_path_errors` | exit 1 with "database.path is empty" | -//! | `memory_path_skips` | exit 0 with INFO line | -//! | `relative_path_skips` | exit 0 with WARN line | +//! | `empty_path_errors` | exit 1 with JSON diagnostic | +//! | `memory_path_skips` | exit 0 with JSON info status | +//! | `relative_path_skips` | exit 0 with JSON warning status | //! | `nonempty_absolute_untouched` | exit 0, file bytes unchanged | -//! | `outside_volumes_errors` | exit 1 with "outside the … volumes" | +//! | `outside_volumes_errors` | exit 1 with JSON diagnostic | //! //! The "missing-under-volume seeded" outcome (mkdir + `inst()` //! into `/var/lib/torrust/index/`) requires root and the //! container's `torrust` user; it is exercised by the -//! container e2e suite (Phase 8/9). +//! container e2e suite. use std::fs; use tempfile::TempDir; -use torrust_index_entry_script::run_sh_with_args; +use torrust_index_entry_script::{assert_entry_script_record, run_sh_with_args, single_stderr_json_record}; const SNIPPET: &str = "seed_sqlite \"$1\""; @@ -29,10 +29,11 @@ const SNIPPET: &str = "seed_sqlite \"$1\""; fn empty_path_errors() { let out = run_sh_with_args(SNIPPET, &[""]); assert_eq!(out.status.code(), Some(1), "expected exit 1"); - let stderr = String::from_utf8_lossy(&out.stderr); - assert!( - stderr.contains("database.path is empty"), - "missing diagnostic; stderr={stderr}", + let record = single_stderr_json_record(&out); + assert_entry_script_record(&record, "diagnostic", "error", "database.path is empty"); + assert_eq!( + record.pointer("/fields/exit_code").and_then(serde_json::Value::as_u64), + Some(1) ); } @@ -45,11 +46,8 @@ fn memory_path_skips() { out.status.code(), String::from_utf8_lossy(&out.stderr), ); - let stderr = String::from_utf8_lossy(&out.stderr); - assert!( - stderr.contains("INFO") && stderr.contains(":memory:"), - "missing INFO/:memory: line; stderr={stderr}", - ); + let record = single_stderr_json_record(&out); + assert_entry_script_record(&record, "status", "info", ":memory:"); } #[test] @@ -61,11 +59,8 @@ fn relative_path_skips() { out.status.code(), String::from_utf8_lossy(&out.stderr), ); - let stderr = String::from_utf8_lossy(&out.stderr); - assert!( - stderr.contains("WARN") && stderr.contains("relative SQLite path"), - "missing WARN/relative line; stderr={stderr}", - ); + let record = single_stderr_json_record(&out); + assert_entry_script_record(&record, "status", "warn", "relative SQLite path"); } #[test] @@ -101,9 +96,10 @@ fn outside_volumes_errors() { let out = run_sh_with_args(SNIPPET, &[path_str]); assert_eq!(out.status.code(), Some(1), "expected exit 1"); - let stderr = String::from_utf8_lossy(&out.stderr); - assert!( - stderr.contains("outside the") && stderr.contains("volumes"), - "missing volumes-guard diagnostic; stderr={stderr}", + let record = single_stderr_json_record(&out); + assert_entry_script_record(&record, "diagnostic", "error", "outside the volumes"); + assert_eq!( + record.pointer("/fields/exit_code").and_then(serde_json::Value::as_u64), + Some(1) ); } diff --git a/packages/index-entry-script/tests/validate_auth_keys.rs b/packages/index-entry-script/tests/validate_auth_keys.rs index 2632fee2f..63cd65985 100644 --- a/packages/index-entry-script/tests/validate_auth_keys.rs +++ b/packages/index-entry-script/tests/validate_auth_keys.rs @@ -9,11 +9,11 @@ //! | `both_none_passes` | happy path — no config | //! | `both_path_passes` | happy path — both PATH | //! | `both_pem_passes` | happy path — both PEM | -//! | `private_pem_and_path_errors` | per-key mutual exclusion | -//! | `public_pem_and_path_errors` | per-key mutual exclusion | -//! | `private_only_errors` | pair completeness | -//! | `public_only_errors` | pair completeness | -//! | `mixed_pem_path_errors` | cross-pair source consistency | +//! | `private_pem_and_path_errors` | JSON diagnostic for per-key mutual exclusion | +//! | `public_pem_and_path_errors` | JSON diagnostic for per-key mutual exclusion | +//! | `private_only_errors` | JSON diagnostic for pair completeness | +//! | `public_only_errors` | JSON diagnostic for pair completeness | +//! | `mixed_pem_path_errors` | JSON diagnostic for cross-pair source consistency | //! //! Argument order to `validate_auth_keys`: //! @@ -22,7 +22,7 @@ //! pub_pem_set pub_path_set pub_source //! ``` -use torrust_index_entry_script::run_sh_with_args; +use torrust_index_entry_script::{assert_entry_script_record, run_sh_with_args, single_stderr_json_record}; const SNIPPET: &str = "validate_auth_keys \"$@\""; @@ -53,10 +53,11 @@ fn assert_fail(args: &[&str], needle: &str) { out.status.code(), String::from_utf8_lossy(&out.stderr), ); - let stderr = String::from_utf8_lossy(&out.stderr); - assert!( - stderr.contains(needle), - "expected diagnostic containing {needle:?} for args {args:?}; got: {stderr}", + let record = single_stderr_json_record(&out); + assert_entry_script_record(&record, "diagnostic", "error", needle); + assert_eq!( + record.pointer("/fields/exit_code").and_then(serde_json::Value::as_u64), + Some(1) ); } diff --git a/packages/index-health-check/src/bin/torrust-index-health-check.rs b/packages/index-health-check/src/bin/torrust-index-health-check.rs index fbaba35d2..1d9b53255 100644 --- a/packages/index-health-check/src/bin/torrust-index-health-check.rs +++ b/packages/index-health-check/src/bin/torrust-index-health-check.rs @@ -1,18 +1,22 @@ //! Minimal health-check binary for Torrust Index containers. //! -//! On success (exit 0), emits a JSON object to stdout per P9. -//! On failure (exit ≠ 0), stdout is empty; diagnostics go to stderr. +//! On success (exit 0), emits one JSON object with `schema`, `target`, `status`, +//! and `elapsed_ms` to stdout per ADR-T-010. +//! On failure (exit ≠ 0), stdout is empty; diagnostics go to stderr as JSON. +//! Direct terminal stdout is refused; pipe or redirect the result. use std::process::ExitCode; use clap::Parser; -use torrust_index_cli_common::{BaseArgs, emit, init_json_tracing, refuse_if_stdout_is_tty}; -use torrust_index_health_check::do_health_check; -use tracing::error; +use torrust_index_cli_common::{BaseArgs, install_json_panic_hook, parse_args_or_exit, run_stdout_json_command}; +use torrust_index_health_check::{HealthCheckError, HealthCheckOutput, do_health_check}; + +const COMMAND_NAME: &str = "torrust-index-health-check"; #[derive(Parser)] #[command( name = "torrust-index-health-check", + version, about = "Minimal health-check for Torrust Index containers" )] struct Args { @@ -24,29 +28,81 @@ struct Args { } fn main() -> ExitCode { - let args = Args::parse(); - init_json_tracing(if args.base.debug { - tracing::Level::DEBUG - } else { - tracing::Level::INFO - }); - refuse_if_stdout_is_tty("torrust-index-health-check"); - - match do_health_check(&args.url) { - Ok(out) => match emit(&out) { - Ok(()) => ExitCode::SUCCESS, - Err(e) => { - // Writing the JSON object to stdout failed (e.g. broken - // pipe when the consumer has already exited). Surface - // the failure through the documented exit-code contract - // rather than letting Rust's panic handler set its own. - error!(error = %e, "failed to write health-check output to stdout"); - ExitCode::FAILURE - } - }, - Err(e) => { - error!(error = %e, "health check failed"); - ExitCode::FAILURE - } + install_json_panic_hook(COMMAND_NAME); + + let args = parse_args_or_exit::(); + + run_stdout_json_command::(COMMAND_NAME, args.base.debug, tracing::Level::INFO, || { + do_health_check(&args.url) + }) +} + +#[cfg(test)] +mod tests { + //! # Health-check binary CLI contract tests + //! + //! | Test | What it covers | + //! |---------------------------------------|----------------------------------------| + //! | `help_is_json_control_record` | `--help` is wrapped as JSON metadata | + //! | `version_is_json_control_record` | `--version` is wrapped as JSON metadata| + //! | `usage_error_is_json_control_record` | argv errors become JSON usage records | + + use torrust_index_cli_common::{CommandExit, ControlPlaneFields, ControlPlaneRecordKind, parse_args_from}; + + use super::{Args, COMMAND_NAME}; + + #[test] + fn help_is_json_control_record() { + let Err(exit) = parse_args_from::([COMMAND_NAME, "--help"]) else { + panic!("help should stop parsing"); + }; + + assert_eq!(exit.exit, CommandExit::Success); + assert_eq!(exit.record.command, COMMAND_NAME); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::Help); + + let Some(ControlPlaneFields::Help { text }) = exit.record.fields else { + panic!("help record should carry help text"); + }; + assert!(text.contains("Minimal health-check")); + assert!(text.contains("--debug")); + } + + #[test] + fn version_is_json_control_record() { + let Err(exit) = parse_args_from::([COMMAND_NAME, "--version"]) else { + panic!("version should stop parsing"); + }; + + assert_eq!(exit.exit, CommandExit::Success); + assert_eq!(exit.record.command, COMMAND_NAME); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::Version); + + let Some(ControlPlaneFields::Version { version }) = exit.record.fields else { + panic!("version record should carry version text"); + }; + assert_eq!(version, format!("{COMMAND_NAME} {}", env!("CARGO_PKG_VERSION"))); + } + + #[test] + fn usage_error_is_json_control_record() { + let Err(exit) = parse_args_from::([COMMAND_NAME, "--no-such-flag"]) else { + panic!("unknown flags should stop parsing"); + }; + + assert_eq!(exit.exit, CommandExit::Usage); + assert_eq!(exit.record.command, COMMAND_NAME); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::UsageError); + assert!(exit.record.message.contains("--no-such-flag")); + + let Some(ControlPlaneFields::UsageError { + exit_code, + clap_error_kind, + }) = exit.record.fields + else { + panic!("usage record should carry usage fields"); + }; + assert_eq!(exit_code, CommandExit::Usage.code()); + assert_eq!(clap_error_kind, "unknown_argument"); } } diff --git a/packages/index-health-check/src/lib.rs b/packages/index-health-check/src/lib.rs index d5d20b625..58216bb4a 100644 --- a/packages/index-health-check/src/lib.rs +++ b/packages/index-health-check/src/lib.rs @@ -13,8 +13,12 @@ use serde::Serialize; #[cfg(test)] mod tests; +/// Schema version of the health-check JSON output. +pub const SCHEMA: u32 = 1; + #[derive(Serialize)] pub struct HealthCheckOutput { + pub schema: u32, pub target: String, pub status: u16, pub elapsed_ms: u64, @@ -105,6 +109,7 @@ pub fn do_health_check(url: &str) -> Result } Ok(HealthCheckOutput { + schema: SCHEMA, target: url.to_string(), status, elapsed_ms, diff --git a/packages/index-health-check/src/tests/mod.rs b/packages/index-health-check/src/tests/mod.rs index 65e18f70d..939d50f64 100644 --- a/packages/index-health-check/src/tests/mod.rs +++ b/packages/index-health-check/src/tests/mod.rs @@ -1,8 +1,11 @@ +#![allow(clippy::print_stderr)] + //! # Health-check tests //! //! | Test | What it covers | //! |------------------------------------|---------------------------------------| //! | `success_on_200` | Happy path: 200 OK response | +//! | `success_output_carries_schema` | Output schema field is stable | //! | `failure_on_non_2xx` | Non-success HTTP status code | //! | `failure_on_connection_refused` | Target not listening | //! | `failure_on_read_timeout` | Server accepts but never responds | @@ -42,9 +45,19 @@ fn success_on_200() { let result = handle.join().unwrap(); assert!(result.is_ok()); let output = result.unwrap(); + assert_eq!(output.schema, super::SCHEMA); assert_eq!(output.status, 200); } +#[test] +fn success_output_carries_schema() { + let (listener, url) = ephemeral_server(); + let handle = std::thread::spawn(move || super::do_health_check(&url)); + serve_once(&listener, b"HTTP/1.1 200 OK\r\nContent-Length: 0\r\n\r\n"); + let output = handle.join().unwrap().unwrap(); + assert_eq!(output.schema, super::SCHEMA); +} + #[test] fn failure_on_non_2xx() { let (listener, url) = ephemeral_server(); diff --git a/packages/index-health-check/tests/health_check.rs b/packages/index-health-check/tests/health_check.rs index 47dae86de..d30997886 100644 --- a/packages/index-health-check/tests/health_check.rs +++ b/packages/index-health-check/tests/health_check.rs @@ -11,7 +11,7 @@ use std::io::{Read, Write}; use std::net::TcpListener; -use torrust_index_health_check::do_health_check; +use torrust_index_health_check::{SCHEMA, do_health_check}; /// Bind an ephemeral-port listener and return (listener, url). fn ephemeral_server() -> (TcpListener, String) { @@ -77,6 +77,7 @@ fn output_contains_expected_fields() { let output = handle.join().unwrap().unwrap(); let json = serde_json::to_value(&output).unwrap(); + assert_eq!(json["schema"], SCHEMA); assert!(json.get("target").is_some(), "missing 'target' field"); assert!(json.get("status").is_some(), "missing 'status' field"); assert!(json.get("elapsed_ms").is_some(), "missing 'elapsed_ms' field"); diff --git a/packages/mudlark/src/testing/runner.rs b/packages/mudlark/src/testing/runner.rs index f83808b07..caac1e61f 100644 --- a/packages/mudlark/src/testing/runner.rs +++ b/packages/mudlark/src/testing/runner.rs @@ -7,6 +7,8 @@ // checking invariants or budget constraints at a configurable // interval. +#![allow(clippy::print_stderr)] + use std::fmt::Display; use super::plan::Plan; diff --git a/packages/mudlark/src/tests/decompose_basis.rs b/packages/mudlark/src/tests/decompose_basis.rs index cc5a1c549..d2fc2d950 100644 --- a/packages/mudlark/src/tests/decompose_basis.rs +++ b/packages/mudlark/src/tests/decompose_basis.rs @@ -1,6 +1,8 @@ // SPDX-License-Identifier: AGPL-3.0-only // SPDX-FileCopyrightText: 2026 Torrust project contributors +#![allow(clippy::print_stderr)] + //! Boundary tests for **basis decomposition** via `contour_range()` //! (ADR-M-039). //! diff --git a/packages/mudlark/src/tests/semi_internal_plateau.rs b/packages/mudlark/src/tests/semi_internal_plateau.rs index 4a935df1c..ba6a1e27a 100644 --- a/packages/mudlark/src/tests/semi_internal_plateau.rs +++ b/packages/mudlark/src/tests/semi_internal_plateau.rs @@ -1,6 +1,8 @@ // SPDX-License-Identifier: AGPL-3.0-only // SPDX-FileCopyrightText: 2026 Torrust project contributors +#![allow(clippy::print_stderr)] + //! Unit tests for degenerate `SemiInternal` plateau shapes. //! //! These tests manually construct G-tree topologies with specific diff --git a/packages/mudlark/tests/eviction_plateau.rs b/packages/mudlark/tests/eviction_plateau.rs index 06a20eba1..ba6cf94b0 100644 --- a/packages/mudlark/tests/eviction_plateau.rs +++ b/packages/mudlark/tests/eviction_plateau.rs @@ -2,6 +2,7 @@ // SPDX-FileCopyrightText: 2026 Torrust project contributors #![cfg(feature = "dynamic-contour-tracking")] +#![allow(clippy::print_stderr)] //! Integration tests for eviction ↔ plateau invariant maintenance. //! //! These tests exercise the plateau invariants (P-I2 basis diff --git a/packages/mudlark/tests/negative_f64.rs b/packages/mudlark/tests/negative_f64.rs index a8c8aa164..a4e4163cd 100644 --- a/packages/mudlark/tests/negative_f64.rs +++ b/packages/mudlark/tests/negative_f64.rs @@ -1,6 +1,8 @@ // SPDX-License-Identifier: AGPL-3.0-only // SPDX-FileCopyrightText: 2026 Torrust project contributors +#![allow(clippy::print_stderr)] + //! Integration tests for negative `f64` observations. //! //! `f64` implements `Accumulator` with `zero() = 0.0`, which diff --git a/packages/mudlark/tests/pedagogy.rs b/packages/mudlark/tests/pedagogy.rs index eb5a6a5d0..592687ef5 100644 --- a/packages/mudlark/tests/pedagogy.rs +++ b/packages/mudlark/tests/pedagogy.rs @@ -1,6 +1,8 @@ // SPDX-License-Identifier: AGPL-3.0-only // SPDX-FileCopyrightText: 2026 Torrust project contributors +#![allow(clippy::print_stdout)] + //! # End-to-End Pedagogy Test for the Worked Example //! //! This test walks through the construction, observation, rebalancing, diff --git a/packages/mudlark/tests/pedagogy_advanced.rs b/packages/mudlark/tests/pedagogy_advanced.rs index d9554b128..1449c9931 100644 --- a/packages/mudlark/tests/pedagogy_advanced.rs +++ b/packages/mudlark/tests/pedagogy_advanced.rs @@ -1,6 +1,8 @@ // SPDX-License-Identifier: AGPL-3.0-only // SPDX-FileCopyrightText: 2026 Torrust project contributors +#![allow(clippy::print_stdout)] + //! # Advanced Pedagogy Test — Companion to [`pedagogy.rs`] //! //! The basic pedagogy test walks through the **mutation surface** of the diff --git a/packages/render-text-as-image/examples/test_render.rs b/packages/render-text-as-image/examples/test_render.rs index 7396b2052..bad69891a 100644 --- a/packages/render-text-as-image/examples/test_render.rs +++ b/packages/render-text-as-image/examples/test_render.rs @@ -1,3 +1,5 @@ +#![allow(clippy::print_stdout)] + use std::fs; use torrust_index_render_text_as_image::{RenderParams, Rgba, render_text_to_png}; diff --git a/share/container/entry_script_lib_sh b/share/container/entry_script_lib_sh index d7afe7e04..45e6ceace 100644 --- a/share/container/entry_script_lib_sh +++ b/share/container/entry_script_lib_sh @@ -9,12 +9,234 @@ # Conventions: # - All functions are POSIX sh; no bashisms. # - Functions use only `[ ... ]` and `case` for tests. -# - Functions emit human-readable diagnostics on stderr -# and use `exit 1` for unrecoverable input violations -# (auth-key validation, seed_sqlite path errors). When -# sourced for tests, callers must invoke each function -# in a subshell to observe the exit status without -# terminating the test driver. +# - Functions emit JSON diagnostics on stderr and use +# `exit 1` for unrecoverable input violations (auth-key +# validation, seed_sqlite path errors). When sourced for +# tests, callers must invoke each function in a subshell +# to observe the exit status without terminating the test +# driver. + +ENTRY_SCRIPT_JSON_SCHEMA=1 + +json_command_name() { + printf '%s' "${ENTRY_SCRIPT_COMMAND_NAME:-torrust-index-entry-script}" +} + +json_emit_failed_exit() { + ENTRY_SCRIPT_JSON_EXIT_HANDLED=1 + printf '%s\n' '{"schema":1,"command":"torrust-index-entry-script","kind":"diagnostic","message":"failed to emit JSON diagnostic","fields":{"type":"entry_script","level":"error","exit_code":1}}' >&2 + exit 1 +} + +json_require_jq() { + if command -v jq >/dev/null 2>&1; then + return 0 + fi + + ENTRY_SCRIPT_JSON_EXIT_HANDLED=1 + printf '%s\n' '{"schema":1,"command":"torrust-index-entry-script","kind":"diagnostic","message":"jq is required for container entry-script JSON diagnostics","fields":{"type":"entry_script","level":"error","exit_code":1}}' >&2 + exit 1 +} + +json_log() { + _json_kind=$1 + _json_level=$2 + _json_message=$3 + + json_require_jq + jq -cn \ + --arg command "$(json_command_name)" \ + --arg kind "$_json_kind" \ + --arg message "$_json_message" \ + --arg level "$_json_level" \ + '{schema:1, command:$command, kind:$kind, message:$message, fields:{type:"entry_script", level:$level}}' >&2 \ + || json_emit_failed_exit +} + +json_error_exit() { + _json_message=$1 + _json_exit_code=${2:-1} + + json_require_jq + jq -cn \ + --arg command "$(json_command_name)" \ + --arg message "$_json_message" \ + --argjson exit_code "$_json_exit_code" \ + '{schema:1, command:$command, kind:"diagnostic", message:$message, fields:{type:"entry_script", level:"error", exit_code:$exit_code}}' >&2 \ + || json_emit_failed_exit + + ENTRY_SCRIPT_JSON_EXIT_HANDLED=1 + exit "$_json_exit_code" +} + +json_external_log() { + _json_kind=$1 + _json_level=$2 + _json_message=$3 + _json_external_command=$4 + _json_external_stderr=$5 + + json_require_jq + jq -cn \ + --arg command "$(json_command_name)" \ + --arg kind "$_json_kind" \ + --arg level "$_json_level" \ + --arg message "$_json_message" \ + --arg external_command "$_json_external_command" \ + --arg external_stderr "$_json_external_stderr" \ + '{schema:1, command:$command, kind:$kind, message:$message, fields:{type:"external_command", level:$level, external_command:$external_command, external_stderr:$external_stderr}}' >&2 \ + || json_emit_failed_exit +} + +json_external_error_exit() { + _json_message=$1 + _json_exit_code=$2 + _json_external_command=$3 + _json_external_stderr=$4 + + json_require_jq + jq -cn \ + --arg command "$(json_command_name)" \ + --arg message "$_json_message" \ + --argjson exit_code "$_json_exit_code" \ + --arg external_command "$_json_external_command" \ + --arg external_stderr "$_json_external_stderr" \ + '{schema:1, command:$command, kind:"diagnostic", message:$message, fields:{type:"external_command", level:"error", exit_code:$exit_code, external_command:$external_command, external_stderr:$external_stderr}}' >&2 \ + || json_emit_failed_exit + + ENTRY_SCRIPT_JSON_EXIT_HANDLED=1 + exit "$_json_exit_code" +} + +json_unexpected_exit() { + _json_status=$? + + if [ "$_json_status" -eq 0 ]; then + return 0 + fi + if [ "${ENTRY_SCRIPT_JSON_EXIT_HANDLED:-0}" = "1" ]; then + exit "$_json_status" + fi + + ENTRY_SCRIPT_JSON_EXIT_HANDLED=1 + json_require_jq + jq -cn \ + --arg command "$(json_command_name)" \ + --arg phase "${entry_script_phase:-unknown}" \ + --argjson exit_code "$_json_status" \ + '{schema:1, command:$command, kind:"diagnostic", message:"unexpected entry script failure", fields:{type:"entry_script", level:"error", exit_code:$exit_code, phase:$phase}}' >&2 \ + || json_emit_failed_exit + exit "$_json_status" +} + +entry_script_set_phase() { + entry_script_phase=$1 + if [ "${DEBUG:-}" = "1" ]; then + json_log status debug "entry script phase: $entry_script_phase" + fi +} + +run_checked() { + _run_phase=$1 + shift + _run_command=$1 + _run_stderr="${TMPDIR:-/tmp}/torrust-entry-script.$$.$_run_command.stderr" + rm -f "$_run_stderr" + + if "$@" 2>"$_run_stderr"; then + _run_captured_stderr=$(cat "$_run_stderr" 2>/dev/null || true) + rm -f "$_run_stderr" + if [ -n "$_run_captured_stderr" ]; then + json_external_log status warn "$_run_phase wrote to stderr" "$_run_command" "$_run_captured_stderr" + fi + return 0 + fi + + _run_status=$? + _run_captured_stderr=$(cat "$_run_stderr" 2>/dev/null || true) + rm -f "$_run_stderr" + json_external_error_exit "$_run_phase failed" "$_run_status" "$_run_command" "$_run_captured_stderr" +} + +jq_read() { + _jq_filter=$1 + _jq_input=$2 + _jq_stderr="${TMPDIR:-/tmp}/torrust-entry-script.$$.jq.stderr" + rm -f "$_jq_stderr" + + if _jq_output=$(printf '%s' "$_jq_input" | jq -r "$_jq_filter" 2>"$_jq_stderr"); then + rm -f "$_jq_stderr" + printf '%s' "$_jq_output" + return 0 + fi + + _jq_status=$? + _jq_captured_stderr=$(cat "$_jq_stderr" 2>/dev/null || true) + rm -f "$_jq_stderr" + json_external_error_exit "jq failed while reading JSON input" "$_jq_status" jq "$_jq_captured_stderr" +} + +jq_write_file() { + _jq_filter=$1 + _jq_input=$2 + _jq_output_path=$3 + _jq_stderr="${TMPDIR:-/tmp}/torrust-entry-script.$$.jq.stderr" + rm -f "$_jq_stderr" + + if printf '%s' "$_jq_input" | jq -r "$_jq_filter" 2>"$_jq_stderr" >"$_jq_output_path"; then + rm -f "$_jq_stderr" + return 0 + fi + + _jq_status=$? + _jq_captured_stderr=$(cat "$_jq_stderr" 2>/dev/null || true) + rm -f "$_jq_stderr" + json_external_error_exit "jq failed while writing JSON field" "$_jq_status" jq "$_jq_captured_stderr" +} + +append_line_checked() { + _append_phase=$1 + _append_line=$2 + _append_path=$3 + _append_stderr="${TMPDIR:-/tmp}/torrust-entry-script.$$.append.stderr" + rm -f "$_append_stderr" + + if printf '%s\n' "$_append_line" 2>"$_append_stderr" >>"$_append_path"; then + _append_captured_stderr=$(cat "$_append_stderr" 2>/dev/null || true) + rm -f "$_append_stderr" + if [ -n "$_append_captured_stderr" ]; then + json_external_log status warn "$_append_phase wrote to stderr" shell "$_append_captured_stderr" + fi + return 0 + fi + + _append_status=$? + _append_captured_stderr=$(cat "$_append_stderr" 2>/dev/null || true) + rm -f "$_append_stderr" + json_external_error_exit "$_append_phase failed" "$_append_status" shell "$_append_captured_stderr" +} + +append_file_checked() { + _append_phase=$1 + _append_source=$2 + _append_path=$3 + _append_stderr="${TMPDIR:-/tmp}/torrust-entry-script.$$.append.stderr" + rm -f "$_append_stderr" + + if cat "$_append_source" 2>"$_append_stderr" >>"$_append_path"; then + _append_captured_stderr=$(cat "$_append_stderr" 2>/dev/null || true) + rm -f "$_append_stderr" + if [ -n "$_append_captured_stderr" ]; then + json_external_log status warn "$_append_phase wrote to stderr" cat "$_append_captured_stderr" + fi + return 0 + fi + + _append_status=$? + _append_captured_stderr=$(cat "$_append_stderr" 2>/dev/null || true) + rm -f "$_append_stderr" + json_external_error_exit "$_append_phase failed" "$_append_status" cat "$_append_captured_stderr" +} # install -D wrapper: copy $1 to $2 only if $1 exists and $2 # does not yet exist. Mode 0640, owner torrust:torrust. @@ -22,7 +244,7 @@ # POSIX §2.9.4.1. inst() { if [ -n "$1" ] && [ -n "$2" ] && [ -e "$1" ] && [ ! -e "$2" ]; then - install -D -m 0640 -o torrust -g torrust "$1" "$2"; fi; } + run_checked "install $2" install -D -m 0640 -o torrust -g torrust "$1" "$2"; fi; } # Returns 0 if $1 is one of the closed-set source values # {pem,path}; 1 otherwise. Tests positively against the @@ -59,31 +281,21 @@ validate_auth_keys() { _pub_pem_set=$4; _pub_path_set=$5; _pub_src=$6 if [ "$_priv_pem_set" = true ] && [ "$_priv_path_set" = true ]; then - echo "ERROR: both PRIVATE_KEY_PEM and PRIVATE_KEY_PATH are set;" \ - "these are mutually exclusive — pick one." >&2 - exit 1 + json_error_exit "both PRIVATE_KEY_PEM and PRIVATE_KEY_PATH are set; these are mutually exclusive - pick one." fi if [ "$_pub_pem_set" = true ] && [ "$_pub_path_set" = true ]; then - echo "ERROR: both PUBLIC_KEY_PEM and PUBLIC_KEY_PATH are set;" \ - "these are mutually exclusive — pick one." >&2 - exit 1 + json_error_exit "both PUBLIC_KEY_PEM and PUBLIC_KEY_PATH are set; these are mutually exclusive - pick one." fi _priv_has=0; key_configured "$_priv_src" && _priv_has=1 _pub_has=0; key_configured "$_pub_src" && _pub_has=1 if [ "$_priv_has" -ne "$_pub_has" ]; then - echo "ERROR: auth keys must be configured as a complete pair;" \ - "one key is configured but the other is not." >&2 - exit 1 + json_error_exit "auth keys must be configured as a complete pair; one key is configured but the other is not." fi if [ "$_priv_has" -eq 1 ] && [ "$_pub_has" -eq 1 ] \ && [ "$_priv_src" != "$_pub_src" ]; then - echo "ERROR: private key source is '$_priv_src'" \ - "but public key source is '$_pub_src';" \ - "mixed PEM/PATH across the key pair is not supported" \ - "— use the same delivery mechanism for both keys." >&2 - exit 1 + json_error_exit "private key source is '$_priv_src' but public key source is '$_pub_src'; mixed PEM/PATH across the key pair is not supported - use the same delivery mechanism for both keys." fi } @@ -100,22 +312,18 @@ seed_sqlite() { _template=/usr/share/torrust/default/database/index.sqlite3.db if [ -z "$_path" ]; then - echo "ERROR: probe reported sqlite driver but" \ - "database.path is empty — possible probe bug" >&2 - exit 1 + json_error_exit "probe reported sqlite driver but database.path is empty - possible probe bug" fi if [ "$_path" = ":memory:" ]; then - echo "INFO: SQLite :memory: — no database file to seed" >&2 + json_log status info "SQLite :memory: - no database file to seed" return 0 fi case $_path in /*) ;; *) - echo "WARN: relative SQLite path '$_path';" \ - "not seeding — application will create on" \ - "first connect if mode=rwc" >&2 + json_log status warn "relative SQLite path '$_path'; not seeding - application will create on first connect if mode=rwc" return 0 ;; esac @@ -125,7 +333,7 @@ seed_sqlite() { fi if [ -e "$_path" ] && [ ! -s "$_path" ]; then - rm -f "$_path" + run_checked "remove empty SQLite database file" rm -f "$_path" fi _dir=$(dirname "$_path") @@ -134,17 +342,12 @@ seed_sqlite() { /etc/torrust/index|/etc/torrust/index/*|\ /var/lib/torrust/index|/var/lib/torrust/index/*|\ /var/log/torrust/index|/var/log/torrust/index/*) - mkdir -p "$_dir" - chown torrust:torrust "$_dir" - chmod 0750 "$_dir" + run_checked "create SQLite database directory" mkdir -p "$_dir" + run_checked "set SQLite database directory owner" chown torrust:torrust "$_dir" + run_checked "set SQLite database directory mode" chmod 0750 "$_dir" ;; *) - echo "ERROR: database path $_dir is outside the" \ - "volumes the entry script manages." >&2 - echo " Pre-create it with appropriate" \ - "ownership, or use a path under" \ - "/var/lib/torrust/index/." >&2 - exit 1 + json_error_exit "database path $_dir is outside the volumes the entry script manages. Pre-create it with appropriate ownership, or use a path under /var/lib/torrust/index/." ;; esac fi diff --git a/share/container/entry_script_sh b/share/container/entry_script_sh index f4022560a..9e9da162a 100755 --- a/share/container/entry_script_sh +++ b/share/container/entry_script_sh @@ -1,5 +1,4 @@ #!/bin/sh -[ "${DEBUG:-}" = "1" ] && set -x # ── Canonical env-var manifest (ADR-T-009 Acceptance Criterion #7) ── # Single source of truth for every environment variable the @@ -44,17 +43,21 @@ set -eu # shellcheck disable=SC1091 # runtime path; not resolvable at lint time . /usr/local/lib/torrust/entry_script_lib_sh +json_require_jq +entry_script_phase=startup +trap 'json_unexpected_exit' EXIT +entry_script_set_phase startup + # ── User account setup ──────────────────────────────────── +entry_script_set_phase "user account setup" # D7: validate that USER_ID is numeric and is not 0 (root). case ${USER_ID:-} in ''|*[!0-9]*) - echo "ERROR: USER_ID is unset or not numeric" >&2 - exit 1 + json_error_exit "USER_ID is unset or not numeric" ;; esac if [ "$USER_ID" -eq 0 ]; then - echo "ERROR: USER_ID is 0 (root) — refusing to run as root" >&2 - exit 1 + json_error_exit "USER_ID is 0 (root) - refusing to run as root" fi # Use the busybox `adduser` short-option form so the same @@ -76,19 +79,21 @@ fi # writable layer, so the `torrust` group/user already exist # on subsequent boots. Both steps are guarded so the restart # path is a no-op rather than a fatal exit under `set -e`. -if ! grep -q '^torrust:' /etc/group; then - addgroup -g "$USER_ID" torrust +if ! grep -q '^torrust:' /etc/group 2>/dev/null; then + run_checked "create torrust group" addgroup -g "$USER_ID" torrust fi -if ! grep -q '^torrust:' /etc/passwd; then - adduser -D -s /bin/sh -u "$USER_ID" -G torrust torrust +if ! grep -q '^torrust:' /etc/passwd 2>/dev/null; then + run_checked "create torrust user" adduser -D -s /bin/sh -u "$USER_ID" -G torrust torrust fi # ── Volume directory ownership ──────────────────────────── -mkdir -p /var/lib/torrust/index/database/ /var/log/torrust/index/ /etc/torrust/index/ -chown -R "${USER_ID}" /var/lib/torrust /var/log/torrust /etc/torrust -chmod -R 2770 /var/lib/torrust /var/log/torrust /etc/torrust +entry_script_set_phase "volume directory ownership" +run_checked "create managed volume directories" mkdir -p /var/lib/torrust/index/database/ /var/log/torrust/index/ /etc/torrust/index/ +run_checked "set managed volume owner" chown -R "${USER_ID}" /var/lib/torrust /var/log/torrust /etc/torrust +run_checked "set managed volume mode" chmod -R 2770 /var/lib/torrust /var/log/torrust /etc/torrust # ── Default TOML installation ───────────────────────────── +entry_script_set_phase "default TOML installation" # Phase 5 made `database.connect_url` mandatory and Phase 9 # consequently consolidated the two driver-suffixed shipped # samples into a single driver-agnostic @@ -105,14 +110,10 @@ case ${TORRUST_INDEX_DATABASE_DRIVER:-} in default_config="/usr/share/torrust/default/config/index.container.toml" ;; '') - echo "ERROR: \$TORRUST_INDEX_DATABASE_DRIVER was not set!" >&2 - exit 1 + json_error_exit "TORRUST_INDEX_DATABASE_DRIVER was not set" ;; *) - echo "ERROR: unsupported database driver:" \ - "\"$TORRUST_INDEX_DATABASE_DRIVER\"" >&2 - echo " Supported values: sqlite3, mysql." >&2 - exit 1 + json_error_exit "unsupported database driver: '$TORRUST_INDEX_DATABASE_DRIVER'. Supported values: sqlite3, mysql." ;; esac @@ -120,43 +121,54 @@ install_config="/etc/torrust/index/index.toml" inst "$default_config" "$install_config" # ── Message of the day ──────────────────────────────────── +entry_script_set_phase "message of the day" case ${RUNTIME:-} in - runtime) printf '\n in runtime \n' >> /etc/motd ;; - debug) printf '\n in debug mode \n' >> /etc/motd ;; - release) printf '\n in release mode \n' >> /etc/motd ;; + runtime) + append_line_checked "append runtime motd spacer" "" /etc/motd + append_line_checked "append runtime motd marker" " in runtime " /etc/motd + ;; + debug) + append_line_checked "append debug motd spacer" "" /etc/motd + append_line_checked "append debug motd marker" " in debug mode " /etc/motd + ;; + release) + append_line_checked "append release motd spacer" "" /etc/motd + append_line_checked "append release motd marker" " in release mode " /etc/motd + ;; *) - echo "ERROR: running in unknown mode: \"${RUNTIME:-}\"" >&2 - exit 1 + json_error_exit "running in unknown mode: '${RUNTIME:-}'" ;; esac if [ -e "/usr/share/torrust/container/message" ]; then - cat "/usr/share/torrust/container/message" >> /etc/motd - chmod 0644 /etc/motd + append_file_checked "append container motd message" "/usr/share/torrust/container/message" /etc/motd + run_checked "set motd mode" chmod 0644 /etc/motd fi # Load message of the day from Profile # shellcheck disable=SC2016 -echo '[ ! -z "$TERM" -a -r /etc/motd ] && cat /etc/motd' >> /etc/profile +append_line_checked "append motd profile hook" '[ ! -z "$TERM" -a -r /etc/motd ] && cat /etc/motd' /etc/profile -cd /home/torrust || exit 1 +if ! cd /home/torrust 2>/dev/null; then + json_error_exit "failed to switch to /home/torrust" +fi # ── Config probe (ADR-T-009 §7.2) ───────────────────────── +entry_script_set_phase "config probe" # Resolve the operator's true configuration (TOML + env var # overrides) via the same loader the application uses. The # probe runs *before* this script exports any # TORRUST_INDEX_CONFIG_OVERRIDE_* of its own, so its output # reflects only operator-supplied values. The probe emits # one JSON object on stdout (P9 contract). -probe_json=$(/usr/bin/torrust-index-config-probe) +if ! probe_json=$(/usr/bin/torrust-index-config-probe); then + json_error_exit "config probe failed" +fi # Schema version gate — fail fast on probe/script mismatch. -probe_schema=$(printf '%s' "$probe_json" | jq -r '.schema') +probe_schema=$(jq_read '.schema' "$probe_json") if [ "$probe_schema" != "1" ]; then - echo "ERROR: config probe emitted schema=$probe_schema" \ - "but this entry script expects schema=1" \ - "— possible probe/script version mismatch" >&2 - exit 1 + json_error_exit "config probe emitted schema=$probe_schema but this entry script expects schema=1 - possible probe/script version mismatch" fi # jq field extraction. Each variable is assigned directly @@ -167,22 +179,23 @@ fi # also dereferenced via `eval` in the dispatch and # materialisation loops below (computed variable names of # the form `auth_${pair}_source`). -database_driver=$(printf '%s' "$probe_json" | jq -r '.database.driver') -database_path=$(printf '%s' "$probe_json" | jq -r '.database.path // empty') +database_driver=$(jq_read '.database.driver' "$probe_json") +database_path=$(jq_read '.database.path // empty' "$probe_json") -auth_private_key_pem_set=$(printf '%s' "$probe_json" | jq -r '.auth.private_key.pem_set') -auth_private_key_path_set=$(printf '%s' "$probe_json" | jq -r '.auth.private_key.path_set') -auth_private_key_source=$(printf '%s' "$probe_json" | jq -r '.auth.private_key.source') +auth_private_key_pem_set=$(jq_read '.auth.private_key.pem_set' "$probe_json") +auth_private_key_path_set=$(jq_read '.auth.private_key.path_set' "$probe_json") +auth_private_key_source=$(jq_read '.auth.private_key.source' "$probe_json") # shellcheck disable=SC2034 # dereferenced via eval in auth-key loops -auth_private_key_path=$(printf '%s' "$probe_json" | jq -r '.auth.private_key.path // empty') +auth_private_key_path=$(jq_read '.auth.private_key.path // empty' "$probe_json") -auth_public_key_pem_set=$(printf '%s' "$probe_json" | jq -r '.auth.public_key.pem_set') -auth_public_key_path_set=$(printf '%s' "$probe_json" | jq -r '.auth.public_key.path_set') -auth_public_key_source=$(printf '%s' "$probe_json" | jq -r '.auth.public_key.source') +auth_public_key_pem_set=$(jq_read '.auth.public_key.pem_set' "$probe_json") +auth_public_key_path_set=$(jq_read '.auth.public_key.path_set' "$probe_json") +auth_public_key_source=$(jq_read '.auth.public_key.source' "$probe_json") # shellcheck disable=SC2034 # dereferenced via eval in auth-key loops -auth_public_key_path=$(printf '%s' "$probe_json" | jq -r '.auth.public_key.path // empty') +auth_public_key_path=$(jq_read '.auth.public_key.path // empty' "$probe_json") # ── Auth-key validation (post-probe) ────────────────────── +entry_script_set_phase "auth key validation" # Three invariants enforced together (see validate_auth_keys # in entry_script_lib_sh): mutual exclusion within each key, # pair-completeness across both keys, and cross-pair source @@ -192,6 +205,7 @@ validate_auth_keys \ "$auth_public_key_pem_set" "$auth_public_key_path_set" "$auth_public_key_source" # ── Three-way auth-key path resolution (cases 1/2/3) ────── +entry_script_set_phase "auth key path resolution" # After this loop, ${pair}_path is set for every non-PEM key. for pair in private_key public_key; do src_var="auth_${pair}_source" @@ -225,6 +239,7 @@ for pair in private_key public_key; do done # ── Volumes-only directory guard for auth keys ──────────── +entry_script_set_phase "auth key directory guard" # Applies to cases 2 and 3. The script auto-creates parent # directories only inside the volumes it owns; anything # else is the operator's responsibility. @@ -242,38 +257,37 @@ for pair in private_key public_key; do /etc/torrust/index|/etc/torrust/index/*|\ /var/lib/torrust/index|/var/lib/torrust/index/*|\ /var/log/torrust/index|/var/log/torrust/index/*) - mkdir -p "$d" - chown torrust:torrust "$d" - chmod 0700 "$d" + run_checked "create auth key directory" mkdir -p "$d" + run_checked "set auth key directory owner" chown torrust:torrust "$d" + run_checked "set auth key directory mode" chmod 0700 "$d" ;; *) - echo "ERROR: auth key path $d is outside the volumes" \ - "the entry script manages." >&2 - echo " Pre-create it with appropriate ownership," \ - "or place keys under /etc/torrust/index/ or" \ - "/var/lib/torrust/index/." >&2 - exit 1 + json_error_exit "auth key path $d is outside the volumes the entry script manages. Pre-create it with appropriate ownership, or place keys under /etc/torrust/index/ or /var/lib/torrust/index/." ;; esac done # ── Key materialisation (cases 2 and 3) ─────────────────── +entry_script_set_phase "auth key materialisation" # Both keys are generated together as a pair (the generator # emits a matched keypair in one invocation). If either file # is missing or empty, regenerate both — a half-pair is not # useful. if [ -n "${private_key_path:-}" ] && [ -n "${public_key_path:-}" ]; then if [ ! -s "$private_key_path" ] || [ ! -s "$public_key_path" ]; then - keypair_json=$(/usr/bin/torrust-index-auth-keypair) - printf '%s' "$keypair_json" | jq -r .private_key_pem > "$private_key_path" - printf '%s' "$keypair_json" | jq -r .public_key_pem > "$public_key_path" - chown torrust:torrust "$private_key_path" "$public_key_path" - chmod 0400 "$private_key_path" - chmod 0400 "$public_key_path" + if ! keypair_json=$(/usr/bin/torrust-index-auth-keypair); then + json_error_exit "auth keypair generator failed" + fi + jq_write_file .private_key_pem "$keypair_json" "$private_key_path" + jq_write_file .public_key_pem "$keypair_json" "$public_key_path" + run_checked "set auth key owner" chown torrust:torrust "$private_key_path" "$public_key_path" + run_checked "set private auth key mode" chmod 0400 "$private_key_path" + run_checked "set public auth key mode" chmod 0400 "$public_key_path" fi fi # ── Database seeding dispatch (probe-driven) ────────────── +entry_script_set_phase "database seeding" # `database_driver` comes from the probe's # `.database.driver` field, derived from `connect_url`'s URL # scheme — not from `TORRUST_INDEX_DATABASE_DRIVER`. @@ -292,11 +306,13 @@ case $database_driver in # The probe rejects unknown schemes before reaching # this point, so this branch indicates a probe-vs- # script version mismatch. - echo "ERROR: unexpected database.driver='$database_driver'" \ - "from config probe — possible probe/script version mismatch" >&2 - exit 1 + json_error_exit "unexpected database.driver='$database_driver' from config probe - possible probe/script version mismatch" ;; esac # ── Drop privileges and exec the application ────────────── +entry_script_set_phase "exec application" +if [ ! -x /bin/su-exec ]; then + json_error_exit "/bin/su-exec is not executable" +fi exec /bin/su-exec torrust "$@" diff --git a/src/bin/create_test_torrent.rs b/src/bin/create_test_torrent.rs index 726122823..41a032348 100644 --- a/src/bin/create_test_torrent.rs +++ b/src/bin/create_test_torrent.rs @@ -1,33 +1,73 @@ //! Command line tool to create a test torrent file. //! -//! It's only used for debugging purposes. -use std::env; +//! ADR-T-010 classifies this as a no-stdout side-effect command: stdout remains +//! empty, and status or diagnostic records are emitted as JSON/NDJSON on stderr. +//! Capture stderr when automation needs the generated path or failure details. +//! +//! ```text +//! cargo run --quiet --bin create_test_torrent -- ./output/test/torrents 2>create-test-torrent.ndjson +//! jq . create-test-torrent.ndjson +//! ``` + use std::fs::File; use std::io::Write; -use std::path::Path; +use std::path::{Path, PathBuf}; +use std::process::ExitCode; +use clap::Parser; +use thiserror::Error; use torrust_index::models::torrent_file::{Torrent, TorrentFile, TorrentInfoDictionary}; use torrust_index::services::hasher::sha1; // DevSkim: ignore DS126858 use torrust_index::utils::parse_torrent; +use torrust_index_cli_common::{BaseArgs, install_json_panic_hook, parse_args_or_exit, run_no_stdout_command}; +use tracing::info; use uuid::Uuid; -fn main() { - let args: Vec = env::args().collect(); +const COMMAND_NAME: &str = "create_test_torrent"; - if args.len() != 2 { - eprintln!("Usage: cargo run --bin create_test_torrent "); - eprintln!("Example: cargo run --bin create_test_torrent ./output/test/torrents"); - std::process::exit(1); - } +#[derive(Parser)] +#[command(name = "create_test_torrent", version, about = "Create a test torrent file")] +struct Args { + /// Directory where the generated torrent file will be written. + destination_folder: PathBuf, + + #[command(flatten)] + base: BaseArgs, +} + +#[derive(Debug, Error)] +enum CommandError { + #[error("generated file content is too large to fit in a torrent length field")] + ContentTooLarge, + + #[error("failed to encode torrent: {source}")] + EncodeTorrent { source: serde_bencode::Error }, + + #[error("failed to create torrent file: {source}")] + CreateFile { source: std::io::Error }, + + #[error("failed to write torrent file: {source}")] + WriteFile { source: std::io::Error }, +} - let destination_folder = &args[1]; +fn main() -> ExitCode { + install_json_panic_hook(COMMAND_NAME); + let args = parse_args_or_exit::(); + + run_no_stdout_command::(COMMAND_NAME, args.base.debug, tracing::Level::INFO, || { + create_test_torrent(&args.destination_folder) + }) +} + +fn create_test_torrent(destination_folder: &Path) -> Result<(), CommandError> { let id = Uuid::new_v4(); // Content of the file from which the torrent will be generated. // We use the UUID as the content of the file. let file_contents = format!("{id}\n"); let file_name = format!("file-{id}.txt"); + let file_length = i64::try_from(file_contents.len()).map_err(|_error| CommandError::ContentTooLarge)?; let torrent = Torrent { info: TorrentInfoDictionary::with( @@ -38,7 +78,7 @@ fn main() { &sha1(&file_contents), // DevSkim: ignore DS126858 &[TorrentFile { path: vec![file_name.clone()], // Adjusted to include the actual file name - length: i64::try_from(file_contents.len()).expect("file contents size in bytes cannot exceed i64::MAX"), + length: file_length, md5sum: None, // DevSkim: ignore DS126858 }], ), @@ -52,21 +92,117 @@ fn main() { created_by: None, }; - match parse_torrent::encode_torrent(&torrent) { - Ok(bytes) => { - // Construct the path where the torrent file will be saved - let file_path = Path::new(destination_folder).join(format!("{file_name}.torrent")); - - // Attempt to create and write to the file - let mut file = match File::create(&file_path) { - Ok(file) => file, - Err(e) => panic!("Failed to create file {}: {e}", file_path.display()), - }; - - if let Err(e) = file.write_all(&bytes) { - panic!("Failed to write to file {}: {e}", file_path.display()); - } - } - Err(e) => panic!("Error encoding torrent: {e}"), + let bytes = parse_torrent::encode_torrent(&torrent).map_err(|source| CommandError::EncodeTorrent { source })?; + let torrent_file_path = destination_folder.join(format!("{file_name}.torrent")); + let mut output_file = File::create(&torrent_file_path).map_err(|source| CommandError::CreateFile { source })?; + output_file + .write_all(&bytes) + .map_err(|source| CommandError::WriteFile { source })?; + + info!(path = %torrent_file_path.display(), "created test torrent file"); + + Ok(()) +} + +#[cfg(test)] +mod tests { + //! # Create-test-torrent binary CLI contract tests + //! + //! | Test | What it covers | + //! |---------------------------------------|----------------------------------------| + //! | `help_is_json_control_record` | `--help` is wrapped as JSON metadata | + //! | `version_is_json_control_record` | `--version` is wrapped as JSON metadata| + //! | `usage_error_is_json_control_record` | argv errors become JSON usage records | + //! | `command_writes_valid_torrent_file` | side effect succeeds without stdout data| + //! | `missing_directory_yields_error` | file creation errors are propagated | + + use tempfile::TempDir; + use torrust_index::utils::parse_torrent::decode_and_validate_torrent_file; + use torrust_index_cli_common::{CommandExit, ControlPlaneFields, ControlPlaneRecordKind, parse_args_from}; + + use super::{Args, COMMAND_NAME, create_test_torrent}; + + #[test] + fn help_is_json_control_record() { + let Err(exit) = parse_args_from::([COMMAND_NAME, "--help"]) else { + panic!("help should stop parsing"); + }; + + assert_eq!(exit.exit, CommandExit::Success); + assert_eq!(exit.record.command, COMMAND_NAME); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::Help); + + let Some(ControlPlaneFields::Help { text }) = exit.record.fields else { + panic!("help record should carry help text"); + }; + assert!(text.contains("Create a test torrent file")); + assert!(text.contains("--debug")); + } + + #[test] + fn version_is_json_control_record() { + let Err(exit) = parse_args_from::([COMMAND_NAME, "--version"]) else { + panic!("version should stop parsing"); + }; + + assert_eq!(exit.exit, CommandExit::Success); + assert_eq!(exit.record.command, COMMAND_NAME); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::Version); + + let Some(ControlPlaneFields::Version { version }) = exit.record.fields else { + panic!("version record should carry version text"); + }; + assert_eq!(version, format!("{COMMAND_NAME} {}", env!("CARGO_PKG_VERSION"))); + } + + #[test] + fn usage_error_is_json_control_record() { + let Err(exit) = parse_args_from::([COMMAND_NAME, "--no-such-flag"]) else { + panic!("unknown flags should stop parsing"); + }; + + assert_eq!(exit.exit, CommandExit::Usage); + assert_eq!(exit.record.command, COMMAND_NAME); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::UsageError); + assert!(exit.record.message.contains("--no-such-flag")); + + let Some(ControlPlaneFields::UsageError { + exit_code, + clap_error_kind, + }) = exit.record.fields + else { + panic!("usage record should carry usage fields"); + }; + assert_eq!(exit_code, CommandExit::Usage.code()); + assert_eq!(clap_error_kind, "unknown_argument"); + } + + #[test] + fn command_writes_valid_torrent_file() { + let directory = TempDir::new().expect("temporary directory must be created"); + + create_test_torrent(directory.path()).expect("torrent creation must succeed"); + + let mut entries = std::fs::read_dir(directory.path()).expect("temporary directory must be readable"); + let entry = entries + .next() + .expect("one torrent file must be written") + .expect("directory entry must be readable"); + assert!(entries.next().is_none(), "only one torrent file should be written"); + assert_eq!(entry.path().extension().and_then(std::ffi::OsStr::to_str), Some("torrent")); + + let bytes = std::fs::read(entry.path()).expect("torrent file must be readable"); + let (_torrent, original_info_hash) = decode_and_validate_torrent_file(&bytes).expect("written torrent must decode"); + assert_eq!(original_info_hash.to_hex_string().len(), 40); + } + + #[test] + fn missing_directory_yields_error() { + let directory = TempDir::new().expect("temporary directory must be created"); + let missing_directory = directory.path().join("missing"); + + let error = create_test_torrent(&missing_directory).expect_err("missing output directory must fail"); + + assert!(error.to_string().contains("failed to create torrent file")); } } diff --git a/src/bin/import_tracker_statistics.rs b/src/bin/import_tracker_statistics.rs index 07a0a0c61..68e007275 100644 --- a/src/bin/import_tracker_statistics.rs +++ b/src/bin/import_tracker_statistics.rs @@ -2,10 +2,40 @@ //! //! It imports the number of seeders and leechers for all torrents from the linked tracker. //! -//! You can execute it with: `cargo run --bin import_tracker_statistics` +//! You can execute it with: +//! +//! ```text +//! cargo run --quiet --bin import_tracker_statistics -- 2>import-tracker-statistics.ndjson +//! jq . import-tracker-statistics.ndjson +//! ``` +//! +//! ADR-T-010 classifies this as a side-effect command: stdout remains empty and +//! diagnostics are JSON records on stderr. Scripts should branch on the process +//! exit code and parse stderr as NDJSON when they need diagnostics. +use std::process::ExitCode; + +use clap::Parser; use torrust_index::console::commands::tracker_statistics_importer::app::run; +use torrust_index_cli_common::{BaseArgs, install_json_panic_hook, parse_args_or_exit, run_no_stdout_command_async}; + +const COMMAND_NAME: &str = "import_tracker_statistics"; + +#[derive(Parser)] +#[command( + name = "import_tracker_statistics", + version, + about = "Import torrent statistics from the linked tracker" +)] +struct Args { + #[command(flatten)] + base: BaseArgs, +} #[tokio::main] -async fn main() { - run().await; +async fn main() -> ExitCode { + install_json_panic_hook(COMMAND_NAME); + + let args = parse_args_or_exit::(); + + run_no_stdout_command_async(COMMAND_NAME, args.base.debug, tracing::Level::INFO, run).await } diff --git a/src/bin/parse_torrent.rs b/src/bin/parse_torrent.rs index 06c6423a3..936d43e86 100644 --- a/src/bin/parse_torrent.rs +++ b/src/bin/parse_torrent.rs @@ -1,40 +1,185 @@ -//! Command line tool to parse a torrent file and print the decoded torrent. +//! Command line tool to parse a torrent file and emit the decoded torrent. //! -//! It's only used for debugging purposes. -use std::env; -use std::fs::File; -use std::io::{self, Read}; - -use serde_bencode::de::from_bytes; -use serde_bencode::value::Value as BValue; -use torrust_index::utils::parse_torrent; - -fn main() -> io::Result<()> { - let args: Vec = env::args().collect(); - if args.len() != 2 { - eprintln!("Usage: cargo run --bin parse_torrent "); - eprintln!( - "Example: cargo run --bin parse_torrent ./tests/fixtures/torrents/MC_GRID.zip-3cd18ff2d3eec881207dcc5ca5a2c3a2a3afe462.torrent" - ); - std::process::exit(1); +//! ADR-T-010 classifies this as a stdout-result command: success emits one JSON +//! object with `schema`, `torrent`, `original_v1_info_hash`, and +//! `input_byte_length`; failures leave stdout empty and report JSON diagnostics +//! on stderr. Direct terminal stdout is refused, so pipe or redirect the result. +//! +//! ```text +//! cargo run --quiet --bin parse_torrent -- ./path/to/file.torrent | jq . +//! ``` + +use std::path::{Path, PathBuf}; +use std::process::ExitCode; + +use clap::Parser; +use serde::Serialize; +use thiserror::Error; +use torrust_index::models::torrent_file::Torrent; +use torrust_index::utils::parse_torrent::{DecodeTorrentFileError, decode_and_validate_torrent_file}; +use torrust_index_cli_common::{BaseArgs, install_json_panic_hook, parse_args_or_exit, run_stdout_json_command}; + +const COMMAND_NAME: &str = "parse_torrent"; +const OUTPUT_SCHEMA: u32 = 1; + +#[derive(Parser)] +#[command( + name = "parse_torrent", + version, + about = "Parse a torrent file and emit decoded torrent metadata as JSON" +)] +struct Args { + /// Path to the torrent file to parse. + torrent_file: PathBuf, + + #[command(flatten)] + base: BaseArgs, +} + +#[derive(Debug, Serialize)] +struct Output { + schema: u32, + torrent: Torrent, + original_v1_info_hash: String, + input_byte_length: usize, +} + +#[derive(Debug, Error)] +enum CommandError { + #[error("failed to read torrent file: {source}")] + ReadInput { source: std::io::Error }, + + #[error("failed to decode torrent file: {source}")] + DecodeInput { source: DecodeTorrentFileError }, +} + +fn main() -> ExitCode { + install_json_panic_hook(COMMAND_NAME); + + let args = parse_args_or_exit::(); + + run_stdout_json_command::(COMMAND_NAME, args.base.debug, tracing::Level::INFO, || { + parse_file(&args.torrent_file) + }) +} + +fn parse_file(path: &Path) -> Result { + let bytes = std::fs::read(path).map_err(|source| CommandError::ReadInput { source })?; + let input_byte_length = bytes.len(); + let (torrent, original_info_hash) = + decode_and_validate_torrent_file(&bytes).map_err(|source| CommandError::DecodeInput { source })?; + + Ok(Output { + schema: OUTPUT_SCHEMA, + torrent, + original_v1_info_hash: original_info_hash.to_hex_string(), + input_byte_length, + }) +} + +#[cfg(test)] +mod tests { + //! # Parse-torrent binary CLI contract tests + //! + //! | Test | What it covers | + //! |---------------------------------------|----------------------------------------| + //! | `help_is_json_control_record` | `--help` is wrapped as JSON metadata | + //! | `version_is_json_control_record` | `--version` is wrapped as JSON metadata| + //! | `usage_error_is_json_control_record` | argv errors become JSON usage records | + //! | `valid_torrent_yields_json_result` | stdout result schema and metadata | + //! | `invalid_torrent_yields_error` | invalid input fails before stdout data | + + use std::io::Write; + + use serde_json::Value; + use tempfile::NamedTempFile; + use torrust_index_cli_common::{CommandExit, ControlPlaneFields, ControlPlaneRecordKind, parse_args_from}; + + use super::{Args, COMMAND_NAME, OUTPUT_SCHEMA, parse_file}; + + const VALID_TORRENT_BYTES: &[u8] = include_bytes!( + "../../tests/fixtures/torrents/6c690018c5786dbbb00161f62b0712d69296df97_with_custom_info_dict_key.torrent" + ); + + #[test] + fn help_is_json_control_record() { + let Err(exit) = parse_args_from::([COMMAND_NAME, "--help"]) else { + panic!("help should stop parsing"); + }; + + assert_eq!(exit.exit, CommandExit::Success); + assert_eq!(exit.record.command, COMMAND_NAME); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::Help); + + let Some(ControlPlaneFields::Help { text }) = exit.record.fields else { + panic!("help record should carry help text"); + }; + assert!(text.contains("Parse a torrent file")); + assert!(text.contains("--debug")); } - println!("Reading the torrent file ..."); + #[test] + fn version_is_json_control_record() { + let Err(exit) = parse_args_from::([COMMAND_NAME, "--version"]) else { + panic!("version should stop parsing"); + }; + + assert_eq!(exit.exit, CommandExit::Success); + assert_eq!(exit.record.command, COMMAND_NAME); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::Version); + + let Some(ControlPlaneFields::Version { version }) = exit.record.fields else { + panic!("version record should carry version text"); + }; + assert_eq!(version, format!("{COMMAND_NAME} {}", env!("CARGO_PKG_VERSION"))); + } + + #[test] + fn usage_error_is_json_control_record() { + let Err(exit) = parse_args_from::([COMMAND_NAME, "--no-such-flag"]) else { + panic!("unknown flags should stop parsing"); + }; + + assert_eq!(exit.exit, CommandExit::Usage); + assert_eq!(exit.record.command, COMMAND_NAME); + assert_eq!(exit.record.kind, ControlPlaneRecordKind::UsageError); + assert!(exit.record.message.contains("--no-such-flag")); + + let Some(ControlPlaneFields::UsageError { + exit_code, + clap_error_kind, + }) = exit.record.fields + else { + panic!("usage record should carry usage fields"); + }; + assert_eq!(exit_code, CommandExit::Usage.code()); + assert_eq!(clap_error_kind, "unknown_argument"); + } + + #[test] + fn valid_torrent_yields_json_result() { + let mut file = NamedTempFile::new().expect("temporary file must be created"); + file.write_all(VALID_TORRENT_BYTES).expect("fixture torrent must be writable"); + + let output = parse_file(file.path()).expect("fixture torrent must parse"); + let json = serde_json::to_string(&output).expect("parse output must serialize"); + let value: Value = serde_json::from_str(&json).expect("parse output must be valid JSON"); + + assert_eq!(value["schema"], OUTPUT_SCHEMA); + assert_eq!(value["original_v1_info_hash"], "6c690018c5786dbbb00161f62b0712d69296df97"); + assert!(value["input_byte_length"].as_u64().is_some_and(|length| length > 0)); + assert!(value["torrent"].is_object()); + assert!(value.get("path").is_none(), "stdout schema must not expose the input path"); + } - let mut file = File::open(&args[1])?; - let mut bytes = Vec::new(); - file.read_to_end(&mut bytes)?; + #[test] + fn invalid_torrent_yields_error() { + let mut file = NamedTempFile::new().expect("temporary file must be created"); + file.write_all(b"not bencoded torrent data") + .expect("temporary file must be writable"); - println!("Decoding torrent with standard serde implementation ..."); + let error = parse_file(file.path()).expect_err("invalid torrent data must fail"); - match from_bytes::(&bytes) { - Ok(_value) => match parse_torrent::decode_torrent(&bytes) { - Ok(torrent) => { - println!("Parsed torrent: \n{torrent:#?}"); - Ok(()) - } - Err(e) => Err(io::Error::other(format!("Error: invalid torrent!. {e}"))), - }, - Err(e) => Err(io::Error::other(format!("Error: invalid bencode data!. {e}"))), + assert!(error.to_string().contains("failed to decode torrent file")); } } diff --git a/src/bin/seeder.rs b/src/bin/seeder.rs index 01fc566f6..38806843d 100644 --- a/src/bin/seeder.rs +++ b/src/bin/seeder.rs @@ -1,7 +1,21 @@ //! Program to upload random torrents to a live Index API. -use torrust_index::console::commands::seeder::app; +//! +//! ADR-T-010 classifies this as a side-effect command: stdout remains empty and +//! diagnostics are JSON records on stderr. Scripts should branch on the process +//! exit code and parse stderr as NDJSON when they need diagnostics. +use std::process::ExitCode; + +use torrust_index::console::commands::seeder::app::{self, Args}; +use torrust_index_cli_common::{install_json_panic_hook, parse_args_or_exit, run_no_stdout_command_async}; + +const COMMAND_NAME: &str = "seeder"; #[tokio::main] -async fn main() -> Result<(), Box> { - app::run().await +async fn main() -> ExitCode { + install_json_panic_hook(COMMAND_NAME); + + let args = parse_args_or_exit::(); + let debug = args.base.debug; + + run_no_stdout_command_async(COMMAND_NAME, debug, tracing::Level::INFO, || app::run(args)).await } diff --git a/src/bin/upgrade.rs b/src/bin/upgrade.rs index fd072f4fa..76a127407 100644 --- a/src/bin/upgrade.rs +++ b/src/bin/upgrade.rs @@ -1,10 +1,50 @@ //! Upgrade command. //! It updates the application from version v1.0.0 to v2.0.0. -//! You can execute it with: `cargo run --bin upgrade ./data.db ./data_v2.db ./uploads` +//! You can execute it with: +//! +//! ```text +//! cargo run --quiet --bin upgrade -- ./data.db ./data_v2.db ./uploads 2>upgrade.ndjson +//! jq . upgrade.ndjson +//! ``` +//! +//! ADR-T-010 classifies this as a side-effect command: stdout remains empty and +//! diagnostics are JSON records on stderr. Scripts should branch on the process +//! exit code and parse stderr as NDJSON when they need diagnostics. +use std::process::ExitCode; -use torrust_index::upgrades::from_v1_0_0_to_v2_0_0::upgrader::run; +use clap::Parser; +use torrust_index::upgrades::from_v1_0_0_to_v2_0_0::upgrader::{Arguments, run}; +use torrust_index_cli_common::{BaseArgs, install_json_panic_hook, parse_args_or_exit, run_no_stdout_command_async}; + +const COMMAND_NAME: &str = "upgrade"; + +#[derive(Parser)] +#[command(name = "upgrade", version, about = "Upgrade Torrust Index data from v1.0.0 to v2.0.0")] +struct Args { + /// Source database file in version v1.0.0. + source_database_file: String, + + /// Target database file for version v2.0.0 data. + target_database_file: String, + + /// Directory where v1.0.0 torrent files are stored. + upload_path: String, + + #[command(flatten)] + base: BaseArgs, +} #[tokio::main] -async fn main() { - run().await; +async fn main() -> ExitCode { + install_json_panic_hook(COMMAND_NAME); + + let args = parse_args_or_exit::(); + let debug = args.base.debug; + let command_args = Arguments { + source_database_file: args.source_database_file, + target_database_file: args.target_database_file, + upload_path: args.upload_path, + }; + + run_no_stdout_command_async(COMMAND_NAME, debug, tracing::Level::INFO, || run(command_args)).await } diff --git a/src/bootstrap/logging.rs b/src/bootstrap/logging.rs index 699b03803..13912f128 100644 --- a/src/bootstrap/logging.rs +++ b/src/bootstrap/logging.rs @@ -1,66 +1,32 @@ //! Setup for the application logging. -//! -//! - `Off` -//! - `Error` -//! - `Warn` -//! - `Info` -//! - `Debug` -//! - `Trace` -use std::sync::Once; - -use tracing::info; +use torrust_index_cli_common::init_json_tracing; use tracing::level_filters::LevelFilter; +use tracing::{Level, info}; use crate::config::Threshold; -static INIT: Once = Once::new(); - pub fn setup(threshold: &Threshold) { let tracing_level_filter: LevelFilter = threshold.clone().into(); - if tracing_level_filter == LevelFilter::OFF { - return; - } - - INIT.call_once(|| { - tracing_stdout_init(tracing_level_filter, &TraceStyle::Default); - }); + setup_level_filter(tracing_level_filter); } -fn tracing_stdout_init(filter: LevelFilter, style: &TraceStyle) { - let builder = tracing_subscriber::fmt().with_max_level(filter); - - let () = match style { - TraceStyle::Default => builder.init(), - TraceStyle::Pretty(display_filename) => builder.pretty().with_file(*display_filename).init(), - TraceStyle::Compact => builder.compact().init(), - TraceStyle::Json => builder.json().init(), +pub fn setup_level_filter(filter: LevelFilter) { + let Some(level) = level_from_filter(filter) else { + return; }; + init_json_tracing(level); info!("Logging initialized"); } -#[derive(Debug)] -pub enum TraceStyle { - Default, - Pretty(bool), - Compact, - Json, -} - -impl std::fmt::Display for TraceStyle { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - let style = match self { - Self::Default => "Default Style", - Self::Pretty(path) => match path { - true => "Pretty Style with File Paths", - false => "Pretty Style without File Paths", - }, - - Self::Compact => "Compact Style", - Self::Json => "Json Format", - }; - - f.write_str(style) +const fn level_from_filter(filter: LevelFilter) -> Option { + match filter { + LevelFilter::OFF => None, + LevelFilter::ERROR => Some(Level::ERROR), + LevelFilter::WARN => Some(Level::WARN), + LevelFilter::INFO => Some(Level::INFO), + LevelFilter::DEBUG => Some(Level::DEBUG), + LevelFilter::TRACE => Some(Level::TRACE), } } diff --git a/src/console/commands/seeder/api.rs b/src/console/commands/seeder/api.rs index 3bd0726ba..d4666b902 100644 --- a/src/console/commands/seeder/api.rs +++ b/src/console/commands/seeder/api.rs @@ -2,7 +2,7 @@ use thiserror::Error; use tracing::debug; -use crate::web::api::client::v1::client::Client; +use crate::web::api::client::v1::client::{Client, Error as ClientError}; use crate::web::api::client::v1::contexts::category::forms::AddCategoryForm; use crate::web::api::client::v1::contexts::category::responses::{ListItem, ListResponse}; use crate::web::api::client::v1::contexts::torrent::forms::UploadTorrentMultipartForm; @@ -17,31 +17,55 @@ pub enum Error { TorrentInfoHashAlreadyExists, #[error("Torrent with the same title already exist in the database")] TorrentTitleAlreadyExists, + #[error("failed to fetch categories: {error:?}")] + FetchCategories { error: ClientError }, + #[error("category list API returned an unexpected response: status {status}, content type {content_type:?}")] + UnexpectedCategoryListResponse { status: u16, content_type: Option }, + #[error("failed to parse category list response: {source}")] + ParseCategoryListResponse { source: serde_json::Error }, + #[error("failed to add category: {error:?}")] + AddCategory { error: ClientError }, + #[error("add category API returned an unexpected response: status {status}, content type {content_type:?}")] + UnexpectedAddCategoryResponse { status: u16, content_type: Option }, + #[error("failed to build upload multipart form: {source}")] + BuildUploadForm { source: reqwest::Error }, + #[error("failed to upload torrent: {error:?}")] + UploadTorrent { error: ClientError }, + #[error("upload torrent API returned an unexpected response: status {status}, content type {content_type:?}")] + UnexpectedUploadTorrentResponse { status: u16, content_type: Option }, + #[error("failed to parse upload torrent response: {source}")] + ParseUploadTorrentResponse { source: serde_json::Error }, + #[error("failed to login: {error:?}")] + Login { error: ClientError }, + #[error("login API returned an unexpected response: status {status}, content type {content_type:?}")] + UnexpectedLoginResponse { status: u16, content_type: Option }, + #[error("failed to parse login response: {source}")] + ParseLoginResponse { source: serde_json::Error }, } /// It uploads a torrent file to the Torrust Index. /// /// # Errors /// -/// It returns an error if the torrent already exists in the database. -/// -/// # Panics -/// -/// Panics if the response body is not a valid JSON. +/// It returns an error if the torrent already exists in the database or if an +/// API response cannot be handled. pub async fn upload_torrent(client: &Client, upload_torrent_form: UploadTorrentMultipartForm) -> Result { - let categories = get_categories(client).await; + let categories = get_categories(client).await?; if !contains_category_with_name(&categories, &upload_torrent_form.category) { - add_category(client, &upload_torrent_form.category).await; + add_category(client, &upload_torrent_form.category).await?; } // todo: if we receive timeout error we should retry later. Otherwise we // have to restart the seeder manually. + let form = upload_torrent_form + .try_into() + .map_err(|source| Error::BuildUploadForm { source })?; let response = client - .upload_torrent(upload_torrent_form.try_into().expect("multipart form should be valid")) + .upload_torrent(form) .await - .expect("API should return a response"); + .map_err(|error| Error::UploadTorrent { error })?; debug!(target:"seeder", "response: {}", response.status); @@ -55,64 +79,91 @@ pub async fn upload_torrent(client: &Client, upload_torrent_form: UploadTorrentM } } - assert!(response.is_json_and_ok(), "Error uploading torrent: {}", response.body); + if !response.is_json_and_ok() { + return Err(Error::UnexpectedUploadTorrentResponse { + status: response.status, + content_type: response.content_type, + }); + } let uploaded_torrent_response: UploadedTorrentResponse = - serde_json::from_str(&response.body).expect("a valid JSON response should be returned from the Torrust Index API"); + serde_json::from_str(&response.body).map_err(|source| Error::ParseUploadTorrentResponse { source })?; Ok(uploaded_torrent_response.data) } /// It logs in the user and returns the user data. /// -/// # Panics +/// # Errors /// -/// Panics if the response body is not a valid JSON. -pub async fn login(client: &Client, username: &str, password: &str) -> LoggedInUserData { +/// Returns an error if the API request fails or the response cannot be handled. +pub async fn login(client: &Client, username: &str, password: &str) -> Result { let response = client .login_user(LoginForm { login: username.to_owned(), password: password.to_owned(), }) .await - .expect("API should return a response"); + .map_err(|error| Error::Login { error })?; - let res: SuccessfulLoginResponse = serde_json::from_str(&response.body).unwrap_or_else(|_| { - panic!( - "a valid JSON response should be returned after login. Received: {}", - response.body - ) - }); + if !response.is_json_and_ok() { + return Err(Error::UnexpectedLoginResponse { + status: response.status, + content_type: response.content_type, + }); + } + + let res: SuccessfulLoginResponse = + serde_json::from_str(&response.body).map_err(|source| Error::ParseLoginResponse { source })?; - res.data + Ok(res.data) } /// It returns all the index categories. /// -/// # Panics +/// # Errors /// -/// Panics if the response body is not a valid JSON. -pub async fn get_categories(client: &Client) -> Vec { - let response = client.get_categories().await.expect("API should return a response"); +/// Returns an error if the API request fails or the response cannot be handled. +pub async fn get_categories(client: &Client) -> Result, Error> { + let response = client + .get_categories() + .await + .map_err(|error| Error::FetchCategories { error })?; + + if !response.is_json_and_ok() { + return Err(Error::UnexpectedCategoryListResponse { + status: response.status, + content_type: response.content_type, + }); + } - let res: ListResponse = serde_json::from_str(&response.body).unwrap(); + let res: ListResponse = serde_json::from_str(&response.body).map_err(|source| Error::ParseCategoryListResponse { source })?; - res.data + Ok(res.data) } /// It adds a new category. /// -/// # Panics +/// # Errors /// -/// Will panic if it doesn't get a response form the API. -pub async fn add_category(client: &Client, name: &str) -> TextResponse { - client +/// Returns an error if the API request fails or the response cannot be handled. +pub async fn add_category(client: &Client, name: &str) -> Result { + let response = client .add_category(AddCategoryForm { name: name.to_owned(), icon: None, }) .await - .expect("API should return a response") + .map_err(|error| Error::AddCategory { error })?; + + if !response.is_json_and_ok() { + return Err(Error::UnexpectedAddCategoryResponse { + status: response.status, + content_type: response.content_type, + }); + } + + Ok(response) } /// It checks if the category list contains the given category. diff --git a/src/console/commands/seeder/app.rs b/src/console/commands/seeder/app.rs index 288547b9a..87ab33dfd 100644 --- a/src/console/commands/seeder/app.rs +++ b/src/console/commands/seeder/app.rs @@ -1,29 +1,36 @@ //! Console app to upload random torrents to a live Index API. //! +//! ADR-T-010 classifies this as a side-effect command: stdout remains empty and +//! diagnostics are JSON records on stderr. +//! //! Run with: //! //! ```text -//! cargo run --bin seeder -- \ +//! cargo run --quiet --bin seeder -- \ //! --api-base-url \ //! --number-of-torrents \ //! --user \ //! --password \ -//! --interval +//! --interval \ +//! 2>seeder.ndjson +//! jq . seeder.ndjson //! ``` //! //! For example: //! //! ```text -//! cargo run --bin seeder -- \ +//! cargo run --quiet --bin seeder -- \ //! --api-base-url "http://localhost:3001" \ //! --number-of-torrents 1000 \ //! --user admin \ //! --password 12345678 \ -//! --interval 0 +//! --interval 0 \ +//! 2>seeder.ndjson +//! jq . seeder.ndjson //! ``` //! //! That command would upload 1000 random torrents to the Index using the user -//! account admin with password 123456 and waiting 1 second between uploads. +//! account `admin` with password `12345678` and no delay between uploads. //! //! The random torrents generated are single-file torrents from a TXT file. //! All generated torrents used a UUID to identify the test torrent. The torrent @@ -127,20 +134,17 @@ //! //! As you can see the `info` dictionary is exactly the same, which produces //! the same info-hash for the torrent. -use std::str::FromStr; -use std::thread::sleep; use std::time::Duration; use clap::Parser; use reqwest::Url; -use text_colorizer::Colorize; -use tracing::level_filters::LevelFilter; -use tracing::{debug, info}; +use thiserror::Error; +use torrust_index_cli_common::BaseArgs; +use tracing::{debug, error, info}; use uuid::Uuid; -use super::api::Error; +use super::api::Error as ApiError; use crate::console::commands::seeder::api::{login, upload_torrent}; -use crate::console::commands::seeder::logging; use crate::services::torrent_file::generate_random_torrent; use crate::utils::parse_torrent; use crate::web::api::client::v1::client::Client; @@ -148,9 +152,9 @@ use crate::web::api::client::v1::contexts::torrent::forms::{BinaryFile, UploadTo use crate::web::api::client::v1::contexts::torrent::responses::UploadedTorrent; use crate::web::api::client::v1::contexts::user::responses::LoggedInUserData; -#[derive(Parser, Debug)] -#[clap(author, version, about, long_about = None)] -struct Args { +#[derive(Parser)] +#[command(name = "seeder", version, about = "Upload random torrents to a live Index API", long_about = None)] +pub struct Args { #[arg(short, long)] api_base_url: String, @@ -165,42 +169,62 @@ struct Args { #[arg(short, long)] interval: u64, + + #[command(flatten)] + pub base: BaseArgs, } -/// # Errors -/// -/// Returns an error if the API base URL cannot be parsed or if an uploaded -/// torrent cannot be serialized to JSON. -pub async fn run() -> Result<(), Box> { - logging::setup(LevelFilter::INFO); +#[derive(Debug, Error)] +pub enum CommandError { + #[error("failed to parse API base URL: {source}")] + ParseApiBaseUrl { source: url::ParseError }, - let args = Args::parse(); + #[error("failed to login to the Index API: {source}")] + Login { source: ApiError }, - let api_url = Url::from_str(&args.api_base_url).map_err(|e| format!("failed to parse API base URL: {e}"))?; + #[error("failed to upload torrent to the Index API: {source}")] + UploadTorrent { source: ApiError }, - let api_user = login_index_api(&api_url, &args.user, &args.password).await; + #[error("failed to encode generated torrent: {source}")] + EncodeTorrent { source: serde_bencode::Error }, + + #[error("failed to serialize upload response into JSON: {source}")] + SerializeUploadResponse { source: serde_json::Error }, +} + +/// # Errors +/// +/// Returns an error if setup fails. Individual torrent upload failures are +/// logged and the command continues with the next generated torrent. +pub async fn run(args: Args) -> Result<(), CommandError> { + let api_url = args + .api_base_url + .parse::() + .map_err(|source| CommandError::ParseApiBaseUrl { source })?; + + let api_user = login_index_api(&api_url, &args.user, &args.password).await?; let api_client = Client::authenticated(&api_url, &api_user.token); - info!(target:"seeder", "Uploading { } random torrents to the Torrust Index with a { } seconds interval...", args.number_of_torrents.to_string().yellow(), args.interval.to_string().yellow()); + info!(target:"seeder", number_of_torrents = args.number_of_torrents, interval_seconds = args.interval, "uploading random torrents to the Index API"); for i in 1..=args.number_of_torrents { - info!(target:"seeder", "Uploading torrent #{} ...", i.to_string().yellow()); + info!(target:"seeder", torrent_number = i, "uploading torrent"); match upload_random_torrent(&api_client).await { Ok(uploaded_torrent) => { - debug!(target:"seeder", "Uploaded torrent {uploaded_torrent:?}"); + debug!(target:"seeder", ?uploaded_torrent, "uploaded torrent"); let json = serde_json::to_string(&uploaded_torrent) - .map_err(|e| format!("failed to serialize upload response into JSON: {e}"))?; + .map_err(|source| CommandError::SerializeUploadResponse { source })?; - info!(target:"seeder", "Uploaded torrent: {}", json.yellow()); + info!(target:"seeder", uploaded_torrent = %json, "uploaded torrent"); } - Err(err) => print!("Error uploading torrent {err:?}"), + Err(error) => error!(target:"seeder", torrent_number = i, %error, "failed to upload torrent"), } if i != args.number_of_torrents { - sleep(Duration::from_secs(args.interval)); + tokio::time::sleep(Duration::from_secs(args.interval)).await; } } @@ -208,28 +232,35 @@ pub async fn run() -> Result<(), Box> { } /// It logs in a user in the Index API. -pub async fn login_index_api(api_url: &Url, username: &str, password: &str) -> LoggedInUserData { +/// +/// # Errors +/// +/// Returns an error if the login request fails or the response cannot be +/// decoded. +pub async fn login_index_api(api_url: &Url, username: &str, password: &str) -> Result { let unauthenticated_client = Client::unauthenticated(api_url); - info!(target:"seeder", "Trying to login with username: {} ...", username.yellow()); + info!(target:"seeder", username, "trying to login"); - let user: LoggedInUserData = login(&unauthenticated_client, username, password).await; + let user: LoggedInUserData = login(&unauthenticated_client, username, password) + .await + .map_err(|source| CommandError::Login { source })?; if user.role == "admin" { - info!(target:"seeder", "Logged as admin with account: {} ", username.yellow()); + info!(target:"seeder", username, role = %user.role, "logged in as admin"); } else { - info!(target:"seeder", "Logged as {} ", username.yellow()); + info!(target:"seeder", username, role = %user.role, "logged in"); } - user + Ok(user) } -async fn upload_random_torrent(api_client: &Client) -> Result { +async fn upload_random_torrent(api_client: &Client) -> Result { let uuid = Uuid::new_v4(); - info!(target:"seeder", "Uploading torrent with uuid: {} ...", uuid.to_string().yellow()); + info!(target:"seeder", %uuid, "uploading torrent with uuid"); - let torrent_file = generate_random_torrent_file(uuid); + let torrent_file = generate_random_torrent_file(uuid)?; let upload_form = UploadTorrentMultipartForm { title: format!("title-{uuid}"), @@ -238,14 +269,16 @@ async fn upload_random_torrent(api_client: &Client) -> Result BinaryFile { +fn generate_random_torrent_file(uuid: Uuid) -> Result { let torrent = generate_random_torrent(uuid); - let bytes = parse_torrent::encode_torrent(&torrent).expect("msg:the torrent should be bencoded"); + let bytes = parse_torrent::encode_torrent(&torrent).map_err(|source| CommandError::EncodeTorrent { source })?; - BinaryFile::from_bytes(torrent.info.name, bytes) + Ok(BinaryFile::from_bytes(torrent.info.name, bytes)) } diff --git a/src/console/commands/seeder/logging.rs b/src/console/commands/seeder/logging.rs index 0f9a9afc8..6763abbcf 100644 --- a/src/console/commands/seeder/logging.rs +++ b/src/console/commands/seeder/logging.rs @@ -1,12 +1,11 @@ //! Logging setup for the `seeder`. -use tracing::debug; use tracing::level_filters::LevelFilter; +use crate::bootstrap::logging; + /// # Panics /// /// pub fn setup(level: LevelFilter) { - tracing_subscriber::fmt().with_max_level(level).init(); - - debug!("Logging initialized"); + logging::setup_level_filter(level); } diff --git a/src/console/commands/tracker_statistics_importer/app.rs b/src/console/commands/tracker_statistics_importer/app.rs index 4578837ce..7f7823553 100644 --- a/src/console/commands/tracker_statistics_importer/app.rs +++ b/src/console/commands/tracker_statistics_importer/app.rs @@ -3,16 +3,17 @@ //! It imports the number of seeders and leechers for all torrents from the //! associated tracker. //! -//! You can execute it with: `cargo run --bin import_tracker_statistics`. -//! -//! After running it you will see the following output: +//! You can execute it with: //! //! ```text -//! Importing statistics from linked tracker ... -//! Loading configuration from config file `./config.toml` -//! Tracker url: udp://localhost:6969 +//! cargo run --quiet --bin import_tracker_statistics -- 2>import-tracker-statistics.ndjson +//! jq . import-tracker-statistics.ndjson //! ``` //! +//! ADR-T-010 classifies this as a side-effect command: stdout remains empty and +//! diagnostics are JSON records on stderr. Scripts should branch on the process +//! exit code and parse stderr as NDJSON when they need diagnostics. +//! //! Statistics are also imported: //! //! - Periodically by the importer job. The importer job is executed every hour @@ -21,91 +22,66 @@ //! - When a new torrent is added. //! - When the API returns data about a torrent statistics are collected from //! the tracker in real time. -use std::env; use std::sync::Arc; -use text_colorizer::Colorize; use thiserror::Error; +use tracing::info; -use crate::bootstrap::config::initialize_configuration; -use crate::bootstrap::logging; +use crate::bootstrap::config::DEFAULT_PATH_CONFIG; +use crate::config::{Configuration, Error as ConfigError, Info}; use crate::databases::database; use crate::tracker::service::Service; use crate::tracker::statistics_importer::StatisticsImporter; -const NUMBER_OF_ARGUMENTS: usize = 0; - -#[derive(Debug, PartialEq, Eq, Error)] -#[allow(dead_code)] +#[derive(Debug, Error)] pub enum ImportError { - #[error("internal server error")] - WrongNumberOfArgumentsError, -} - -fn parse_args() -> Result<(), ImportError> { - let args: Vec = env::args().skip(1).collect(); - - if args.len() != NUMBER_OF_ARGUMENTS { - eprintln!( - "{} wrong number of arguments: expected {}, got {}", - "Error".red().bold(), - NUMBER_OF_ARGUMENTS, - args.len() - ); - print_usage(); - return Err(ImportError::WrongNumberOfArgumentsError); - } - - Ok(()) -} + #[error("failed to build configuration lookup: {source}")] + BuildConfigurationInfo { source: ConfigError }, -fn print_usage() { - eprintln!( - "{} - imports torrents statistics from linked tracker. + #[error("failed to load configuration: {source}")] + LoadConfiguration { source: ConfigError }, - cargo run --bin import_tracker_statistics + #[error("failed to connect to database: {source}")] + ConnectDatabase { source: database::Error }, - ", - "Tracker Statistics Importer".green() - ); + #[error("failed to import tracker statistics: {source}")] + ImportStatistics { source: database::Error }, } /// Import Tracker Statistics Command /// -/// # Panics +/// # Errors /// -/// Panics if arguments cannot be parsed. -pub async fn run() { - parse_args().expect("unable to parse command arguments"); - import().await; +/// Returns an error if configuration loading, database connection, or the +/// statistics import fails. +pub async fn run() -> Result<(), ImportError> { + import().await } /// Import Command Arguments /// -/// # Panics +/// # Errors /// -/// Panics if it can't connect to the database. -pub async fn import() { - println!("Importing statistics from linked tracker ..."); - - let configuration = initialize_configuration(); - - let threshold = configuration.settings.read().await.logging.threshold.clone(); +/// Returns an error if configuration loading, database connection, or the +/// statistics import fails. +pub async fn import() -> Result<(), ImportError> { + info!("importing statistics from linked tracker"); - logging::setup(&threshold); + let config_info = + Info::new(DEFAULT_PATH_CONFIG.to_string()).map_err(|source| ImportError::BuildConfigurationInfo { source })?; + let configuration = Configuration::load(&config_info).map_err(|source| ImportError::LoadConfiguration { source })?; let cfg = Arc::new(configuration); let settings = cfg.settings.read().await; let tracker_url = settings.tracker.url.clone(); - - eprintln!("Tracker url: {}", tracker_url.to_string().green()); + info!(tracker_url = %tracker_url, "loaded tracker configuration"); let database = Arc::new( database::connect(settings.database.connect_url.as_ref()) .await - .expect("unable to connect to db"), + .map_err(|source| ImportError::ConnectDatabase { source })?, ); drop(settings); @@ -116,5 +92,9 @@ pub async fn import() { tracker_statistics_importer .import_all_torrents_statistics() .await - .expect("should import all torrents statistics"); + .map_err(|source| ImportError::ImportStatistics { source })?; + + info!("imported statistics from linked tracker"); + + Ok(()) } diff --git a/src/console/cronjobs/tracker_statistics_importer.rs b/src/console/cronjobs/tracker_statistics_importer.rs index 3c726a37c..15b1906f0 100644 --- a/src/console/cronjobs/tracker_statistics_importer.rs +++ b/src/console/cronjobs/tracker_statistics_importer.rs @@ -18,7 +18,6 @@ use axum::routing::{get, post}; use axum::{Json, Router}; use chrono::{DateTime, Utc}; use serde_json::{Value, json}; -use text_colorizer::Colorize; use tokio::net::TcpListener; use tokio::task::JoinHandle; use tracing::{debug, error, info}; @@ -37,9 +36,10 @@ struct ImporterState { pub torrent_info_update_interval: u64, } -/// # Panics +/// Start the tracker statistics importer launcher task. /// -/// Will panic if it can't start the tracker statistics importer API +/// Startup failures inside the spawned importer API task are logged as tracing +/// diagnostics and stop that task. #[must_use] pub fn start( importer_port: u16, @@ -70,13 +70,25 @@ pub fn start( info!("Tracker statistics importer API server listening on http://{}", addr); // # DevSkim: ignore DS137138 - let socket_addr: SocketAddr = addr.parse().expect("importer API to have a valid socket address"); + let socket_addr: SocketAddr = match addr.parse() { + Ok(socket_addr) => socket_addr, + Err(error) => { + error!(%error, %addr, "invalid importer API socket address"); + return; + } + }; - let listener = TcpListener::bind(socket_addr) - .await - .expect("importer API TCP listener to bind to socket address"); + let listener = match TcpListener::bind(socket_addr).await { + Ok(listener) => listener, + Err(error) => { + error!(%error, %socket_addr, "failed to bind importer API TCP listener"); + return; + } + }; - axum::serve(listener, app).await.unwrap(); + if let Err(error) = axum::serve(listener, app).await { + error!(%error, "importer API server stopped with an error"); + } }); // Start the Importer cronjob @@ -119,17 +131,20 @@ pub fn start( match weak_tracker_statistics_importer.upgrade() { Some(statistics_importer) => { - let one_interval_ago = seconds_ago_utc( - torrent_stats_update_interval - .try_into() - .expect("update interval should be a positive integer"), - ); + let Ok(update_interval_seconds) = torrent_stats_update_interval.try_into() else { + error!( + torrent_stats_update_interval, + "update interval does not fit in signed seconds" + ); + break; + }; + let one_interval_ago = seconds_ago_utc(update_interval_seconds); let limit = 50; debug!( - "Importing torrents statistics not updated since {} limited to a maximum of {} torrents ...", - one_interval_ago.to_string().yellow(), - limit.to_string().yellow() + since = %one_interval_ago, + limit, + "importing torrents statistics not updated since threshold" ); match statistics_importer @@ -154,13 +169,26 @@ pub fn start( /// Endpoint for container health check. async fn health_check_handler(State(state): State>) -> Json { - let margin_in_seconds = 10; + let margin_in_seconds = 10_u64; let now = Utc::now(); - let last_heartbeat = state.last_heartbeat.lock().unwrap(); - - if now.signed_duration_since(*last_heartbeat).num_seconds() - <= (state.torrent_info_update_interval + margin_in_seconds).try_into().unwrap() - { + let Ok(last_heartbeat) = state.last_heartbeat.lock() else { + error!("failed to acquire importer heartbeat lock"); + return Json(json!({ "status": "Error" })); + }; + + let Ok(max_heartbeat_age_seconds) = state + .torrent_info_update_interval + .saturating_add(margin_in_seconds) + .try_into() + else { + error!( + torrent_info_update_interval = state.torrent_info_update_interval, + "importer health check interval does not fit in signed seconds" + ); + return Json(json!({ "status": "Error" })); + }; + + if now.signed_duration_since(*last_heartbeat).num_seconds() <= max_heartbeat_age_seconds { Json(json!({ "status": "Ok" })) } else { Json(json!({ "status": "Error" })) @@ -171,7 +199,10 @@ async fn health_check_handler(State(state): State>) -> Json>) -> Json { let now = Utc::now(); - let mut last_heartbeat = state.last_heartbeat.lock().unwrap(); + let Ok(mut last_heartbeat) = state.last_heartbeat.lock() else { + error!("failed to acquire importer heartbeat lock"); + return Json(json!({ "status": "Error" })); + }; *last_heartbeat = now; drop(last_heartbeat); Json(json!({ "status": "Heartbeat received" })) diff --git a/src/jwt.rs b/src/jwt.rs index 970443d8a..3d8e15b7c 100644 --- a/src/jwt.rs +++ b/src/jwt.rs @@ -35,10 +35,12 @@ //! //! **Phase 6 — `torrust-index-auth-keypair` CLI.** A standalone binary //! (`torrust-index-auth-keypair`) generates an RSA-2048 key pair -//! and writes a JSON object with both PEM blocks to stdout. The container -//! entry script uses it to auto-generate persistent keys on first boot. -//! See `packages/index-auth-keypair/src/main.rs` for the binary and -//! ADR-T-007 Phase 6 / ADR-T-009 Phase 2 for full context. +//! and writes a schema-versioned JSON object with both PEM blocks to stdout. +//! The container entry script uses it to auto-generate persistent keys on first +//! boot. +//! See `packages/index-auth-keypair/src/bin/torrust-index-auth-keypair.rs` +//! for the binary and ADR-T-007 Phase 6 / ADR-T-009 Phase 2 / ADR-T-010 +//! for full context. //! //! **Phase 7 — Consolidated session validation.** `validate_session` //! is the sole entry point for session-token validation: it verifies diff --git a/src/lib.rs b/src/lib.rs index c2287e201..4d7a09f25 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -20,6 +20,7 @@ //! - [Development](#development) //! - [Configuration](#configuration) //! - [Usage](#usage) +//! - [Command-Line Output](#command-line-output) //! - [API](#api) //! - [Tracker Statistics Importer](#tracker-statistics-importer) //! - [Upgrader](#upgrader) @@ -225,6 +226,15 @@ //! //! # Usage //! +//! ## Command-Line Output +//! +//! ADR-T-010 classifies the `torrust-index` server as a no-stdout command: +//! stdout remains empty, and server diagnostics are JSON records on stderr. +//! Command-reachable libraries use the same path for operator-facing messages; +//! shutdown notices are structured tracing records, and mail-template failures +//! are propagated to callers for JSON diagnostic reporting instead of being +//! printed or handled by exiting from the mailer library. +//! //! ## API //! //! Running the index with the default configuration will expose the REST API on port 3001: @@ -232,6 +242,9 @@ //! ## Tracker Statistics Importer //! //! This console command allows you to manually import the tracker statistics. +//! ADR-T-010 classifies it as a side-effect command: stdout remains empty and +//! diagnostics are JSON records on stderr. Scripts should branch on the process +//! exit code and parse stderr as NDJSON when they need diagnostics. //! //! For more information about this command you can visit the documentation for //! the [`Import tracker statistics`](crate::console::commands::tracker_statistics_importer) module. @@ -240,6 +253,9 @@ //! //! This console command allows you to manually upgrade the application from one //! version to another. +//! ADR-T-010 classifies it as a side-effect command: stdout remains empty and +//! diagnostics are JSON records on stderr. Scripts should branch on the process +//! exit code and parse stderr as NDJSON when they need diagnostics. //! //! For more information about this command you can visit the documentation for //! the [`Upgrade app from version 1.0.0 to 2.0.0`](crate::upgrades::from_v1_0_0_to_v2_0_0::upgrader) module. diff --git a/src/mailer.rs b/src/mailer.rs index 926290e10..34166c9e0 100644 --- a/src/mailer.rs +++ b/src/mailer.rs @@ -7,6 +7,7 @@ use lettre::transport::smtp::authentication::{Credentials, Mechanism}; use lettre::{AsyncSmtpTransport, AsyncTransport, Message, Tokio1Executor}; use serde_json::value::{Value, to_value}; use tera::{Context, Tera, try_get_value}; +use thiserror::Error; use tracing::error; use crate::config::Configuration; @@ -19,7 +20,24 @@ use crate::web::api::server::v1::routes::API_VERSION_URL_PREFIX; /// Default verify-email template, compiled into the binary. const VERIFY_EMAIL_DEFAULT: &str = include_str!("../templates/verify.html"); -pub static TEMPLATES: LazyLock = LazyLock::new(|| { +pub(crate) static TEMPLATES: LazyLock> = LazyLock::new(build_templates); + +#[derive(Debug, Error)] +pub(crate) enum MailTemplateError { + #[error("failed to read templates/verify.html: {source}")] + ReadOverride { source: std::io::Error }, + + #[error("failed to register email template: {source}")] + RegisterTemplate { source: tera::Error }, + + #[error("failed to initialize email templates: {message}")] + Initialize { message: String }, + + #[error("failed to render email template: {source}")] + Render { source: tera::Error }, +} + +fn build_templates() -> Result { let mut tera = Tera::default(); // Allow deployers to override the template by placing a file at @@ -28,24 +46,16 @@ pub static TEMPLATES: LazyLock = LazyLock::new(|| { let template = match std::fs::read_to_string("templates/verify.html") { Ok(contents) => contents, Err(err) if err.kind() == ErrorKind::NotFound => VERIFY_EMAIL_DEFAULT.to_string(), - Err(err) => { - error!(error = %err, "Failed to read templates/verify.html"); - ::std::process::exit(1); - } + Err(source) => return Err(MailTemplateError::ReadOverride { source }), }; - match tera.add_raw_template("html_verify_email", &template) { - Ok(()) => {} - Err(e) => { - println!("Parsing error(s): {e}"); - ::std::process::exit(1); - } - } + tera.add_raw_template("html_verify_email", &template) + .map_err(|source| MailTemplateError::RegisterTemplate { source })?; tera.autoescape_on(vec![".html", ".sql"]); tera.register_filter("do_nothing", do_nothing_filter); - tera -}); + Ok(tera) +} /// This function is a dummy filter for tera. /// @@ -155,8 +165,8 @@ impl Service { } pub(crate) fn build_letter(verification_url: &str, username: &str, builder: MessageBuilder) -> Result { - let (plain_body, html_body) = build_content(verification_url, username).map_err(|e| { - tracing::error!("{e}"); + let (plain_body, html_body) = build_content(verification_url, username).map_err(|error| { + error!(%error, "failed to build verification email content"); UserError::InternalServerError })?; @@ -178,7 +188,7 @@ pub(crate) fn build_letter(verification_url: &str, username: &str, builder: Mess .expect("the `multipart` builder had an error")) } -pub(crate) fn build_content(verification_url: &str, username: &str) -> Result<(String, String), tera::Error> { +pub(crate) fn build_content(verification_url: &str, username: &str) -> Result<(String, String), MailTemplateError> { let plain_body = format!( " Welcome to Torrust, {username}! @@ -192,7 +202,12 @@ pub(crate) fn build_content(verification_url: &str, username: &str) -> Result<(S let mut context = Context::new(); context.insert("verification", &verification_url); context.insert("username", &username); - let html_body = TEMPLATES.render("html_verify_email", &context)?; + let templates = TEMPLATES.as_ref().map_err(|error| MailTemplateError::Initialize { + message: error.to_string(), + })?; + let html_body = templates + .render("html_verify_email", &context) + .map_err(|source| MailTemplateError::Render { source })?; Ok((plain_body, html_body)) } diff --git a/src/main.rs b/src/main.rs index 387c8888e..6b8193d92 100644 --- a/src/main.rs +++ b/src/main.rs @@ -1,18 +1,39 @@ +use std::process::ExitCode; + use torrust_index::app; use torrust_index::bootstrap::config::initialize_configuration; use torrust_index::web::api::Version; +use torrust_index_cli_common::{CommandExit, install_json_panic_hook}; +use tracing::error; + +const COMMAND_NAME: &str = "torrust-index"; #[tokio::main] -async fn main() -> Result<(), std::io::Error> { +async fn main() -> ExitCode { + install_json_panic_hook(COMMAND_NAME); + let configuration = initialize_configuration(); let api_version = Version::V1; let app = app::run(configuration, &api_version).await; - assert!(!app.api_server_halt_task.is_closed(), "Halt channel should be open"); + if app.api_server_halt_task.is_closed() { + error!("halt channel closed before server shutdown"); + return CommandExit::Failure.exit_code(); + } match api_version { - Version::V1 => app.api_server.await.expect("the API server was dropped"), + Version::V1 => match app.api_server.await { + Ok(Ok(())) => CommandExit::Success.exit_code(), + Ok(Err(error)) => { + error!(%error, "API server failed"); + CommandExit::Failure.exit_code() + } + Err(error) => { + error!(%error, "API server task was dropped"); + CommandExit::Failure.exit_code() + } + }, } } diff --git a/src/tracker/statistics_importer.rs b/src/tracker/statistics_importer.rs index 30b368568..666c1037f 100644 --- a/src/tracker/statistics_importer.rs +++ b/src/tracker/statistics_importer.rs @@ -2,7 +2,6 @@ use std::sync::Arc; use std::time::Instant; use chrono::{DateTime, Utc}; -use text_colorizer::Colorize; use tracing::{debug, error, info}; use url::Url; @@ -42,13 +41,13 @@ impl StatisticsImporter { return Ok(()); } - info!(target: LOG_TARGET, "Importing {} torrents statistics from tracker {} ...", torrents.len().to_string().yellow(), self.tracker_url.to_string().yellow()); + info!(target: LOG_TARGET, torrent_count = torrents.len(), tracker_url = %self.tracker_url, "importing torrents statistics from tracker"); // Start the timer before the loop let start_time = Instant::now(); for torrent in torrents { - info!(target: LOG_TARGET, "Importing torrent #{} statistics ...", torrent.torrent_id.to_string().yellow()); + info!(target: LOG_TARGET, torrent_id = torrent.torrent_id, "importing torrent statistics"); let ret = self.import_torrent_statistics(torrent.torrent_id, &torrent.info_hash).await; @@ -81,7 +80,7 @@ impl StatisticsImporter { datetime: DateTime, limit: i64, ) -> Result<(), database::Error> { - debug!(target: LOG_TARGET, "Importing torrents statistics not updated since {} limited to a maximum of {} torrents ...", datetime.to_string().yellow(), limit.to_string().yellow()); + debug!(target: LOG_TARGET, since = %datetime, limit, "importing torrents statistics not updated since threshold"); let torrents = self .database @@ -92,7 +91,7 @@ impl StatisticsImporter { return Ok(()); } - info!(target: LOG_TARGET, "Importing {} torrents statistics from tracker {} ...", torrents.len().to_string().yellow(), self.tracker_url.to_string().yellow()); + info!(target: LOG_TARGET, torrent_count = torrents.len(), tracker_url = %self.tracker_url, "importing torrents statistics from tracker"); // Import stats for all torrents in one request diff --git a/src/upgrades/from_v1_0_0_to_v2_0_0/databases/mod.rs b/src/upgrades/from_v1_0_0_to_v2_0_0/databases/mod.rs index 44af94f9d..0ea601bbf 100644 --- a/src/upgrades/from_v1_0_0_to_v2_0_0/databases/mod.rs +++ b/src/upgrades/from_v1_0_0_to_v2_0_0/databases/mod.rs @@ -1,35 +1,71 @@ use std::sync::Arc; +use tracing::info; + use self::sqlite_v1_0_0::SqliteDatabaseV1_0_0; use self::sqlite_v2_0_0::SqliteDatabaseV2_0_0; +use crate::upgrades::from_v1_0_0_to_v2_0_0::error::UpgradeError; pub mod sqlite_v1_0_0; pub mod sqlite_v2_0_0; -pub async fn current_db(db_filename: &str) -> Arc { +/// Open the source v1.0.0 `SQLite` database in read-only mode. +/// +/// # Errors +/// +/// Returns an error if the database pool cannot be created. +pub async fn current_db(db_filename: &str) -> Result, UpgradeError> { let source_database_connect_url = format!("sqlite://{db_filename}?mode=ro"); - Arc::new(SqliteDatabaseV1_0_0::new(&source_database_connect_url).await) + let database = SqliteDatabaseV1_0_0::new(&source_database_connect_url) + .await + .map_err(|source| UpgradeError::Sqlx { + context: "failed to open source database", + source, + })?; + Ok(Arc::new(database)) } -pub async fn new_db(db_filename: &str) -> Arc { +/// Open the target v2.0.0 `SQLite` database. +/// +/// # Errors +/// +/// Returns an error if the database pool cannot be created. +pub async fn new_db(db_filename: &str) -> Result, UpgradeError> { let target_database_connect_url = format!("sqlite://{db_filename}?mode=rwc"); - Arc::new(SqliteDatabaseV2_0_0::new(&target_database_connect_url).await) + let database = SqliteDatabaseV2_0_0::new(&target_database_connect_url) + .await + .map_err(|source| UpgradeError::Sqlx { + context: "failed to open target database", + source, + })?; + Ok(Arc::new(database)) } -pub async fn migrate_target_database(target_database: Arc) { - println!("Running migrations in the target database..."); - target_database.migrate().await; +/// Run target database migrations. +/// +/// # Errors +/// +/// Returns an error if any migration fails. +pub async fn migrate_target_database(target_database: Arc) -> Result<(), UpgradeError> { + info!("running migrations in the target database"); + target_database.migrate().await.map_err(|source| UpgradeError::Migration { + context: "failed to run target database migrations", + source, + }) } /// It truncates all tables in the target database. /// -/// # Panics +/// # Errors /// -/// It panics if it cannot truncate the tables. -pub async fn truncate_target_database(target_database: Arc) { - println!("Truncating all tables in target database ..."); +/// Returns an error if any target database table cannot be truncated. +pub async fn truncate_target_database(target_database: Arc) -> Result<(), UpgradeError> { + info!("truncating all tables in target database"); target_database .delete_all_database_rows() .await - .expect("Can't reset the target database."); + .map_err(|source| UpgradeError::Database { + context: "failed to truncate target database", + source, + }) } diff --git a/src/upgrades/from_v1_0_0_to_v2_0_0/databases/sqlite_v1_0_0.rs b/src/upgrades/from_v1_0_0_to_v2_0_0/databases/sqlite_v1_0_0.rs index 89be4c495..5ee8ab5d2 100644 --- a/src/upgrades/from_v1_0_0_to_v2_0_0/databases/sqlite_v1_0_0.rs +++ b/src/upgrades/from_v1_0_0_to_v2_0_0/databases/sqlite_v1_0_0.rs @@ -60,15 +60,13 @@ pub struct SqliteDatabaseV1_0_0 { impl SqliteDatabaseV1_0_0 { /// It creates a new instance of the `SqliteDatabaseV1_0_0`. /// - /// # Panics + /// # Errors /// - /// This function will panic if it is unable to create the database pool. - pub async fn new(database_url: &str) -> Self { - let db = SqlitePoolOptions::new() - .connect(database_url) - .await - .expect("Unable to create database pool."); - Self { pool: db } + /// This function will return an error if it is unable to create the + /// database pool. + pub async fn new(database_url: &str) -> Result { + let db = SqlitePoolOptions::new().connect(database_url).await?; + Ok(Self { pool: db }) } pub async fn get_categories_order_by_id(&self) -> Result, database::Error> { diff --git a/src/upgrades/from_v1_0_0_to_v2_0_0/databases/sqlite_v2_0_0.rs b/src/upgrades/from_v1_0_0_to_v2_0_0/databases/sqlite_v2_0_0.rs index 8cdb14dba..585abdd26 100644 --- a/src/upgrades/from_v1_0_0_to_v2_0_0/databases/sqlite_v2_0_0.rs +++ b/src/upgrades/from_v1_0_0_to_v2_0_0/databases/sqlite_v2_0_0.rs @@ -8,6 +8,7 @@ use sqlx::{SqlitePool, query, query_as}; use super::sqlite_v1_0_0::{TorrentRecordV1, UserRecordV1}; use crate::databases::database::{self, TABLES_TO_TRUNCATE}; use crate::models::torrent_file::{TorrentFile, TorrentInfoDictionary}; +use crate::upgrades::from_v1_0_0_to_v2_0_0::error::UpgradeError; #[derive(Debug, Serialize, Deserialize, sqlx::FromRow)] pub struct CategoryRecordV2 { @@ -32,9 +33,12 @@ pub struct TorrentRecordV2 { } impl TorrentRecordV2 { - #[must_use] - pub fn from_v1_data(torrent: &TorrentRecordV1, torrent_info: &TorrentInfoDictionary, uploader: &UserRecordV1) -> Self { - Self { + pub fn from_v1_data( + torrent: &TorrentRecordV1, + torrent_info: &TorrentInfoDictionary, + uploader: &UserRecordV1, + ) -> Result { + Ok(Self { torrent_id: torrent.torrent_id, uploader_id: uploader.user_id, category_id: torrent.category_id, @@ -46,25 +50,26 @@ impl TorrentRecordV2 { piece_length: torrent_info.piece_length, private: torrent_info.private, is_bep_30: i64::from(torrent_info.is_bep_30()), - date_uploaded: convert_timestamp_to_datetime(torrent.upload_date), - } + date_uploaded: convert_timestamp_to_datetime(torrent.upload_date)?, + }) } } /// It converts a timestamp in seconds to a datetime string. /// -/// # Panics +/// # Errors /// -/// It panics if the timestamp is too big and it overflows i64. Very future! -#[must_use] -pub fn convert_timestamp_to_datetime(timestamp: i64) -> String { +/// Returns an error if the timestamp is outside the supported range. +pub fn convert_timestamp_to_datetime(timestamp: i64) -> Result { // The expected format in database is: 2022-11-04 09:53:57 // MySQL uses a DATETIME column and SQLite uses a TEXT column. - let datetime = DateTime::from_timestamp(timestamp, 0).expect("Overflow of i64 seconds, very future!"); + let Some(datetime) = DateTime::from_timestamp(timestamp, 0) else { + return Err(UpgradeError::InvalidTimestamp { timestamp }); + }; // Format without timezone - datetime.format("%Y-%m-%d %H:%M:%S").to_string() + Ok(datetime.format("%Y-%m-%d %H:%M:%S").to_string()) } pub struct SqliteDatabaseV2_0_0 { @@ -74,27 +79,21 @@ pub struct SqliteDatabaseV2_0_0 { impl SqliteDatabaseV2_0_0 { /// Creates a new instance of the database. /// - /// # Panics + /// # Errors /// - /// It panics if it cannot create the database pool. - pub async fn new(database_url: &str) -> Self { - let db = SqlitePoolOptions::new() - .connect(database_url) - .await - .expect("Unable to create database pool."); - Self { pool: db } + /// Returns an error if it cannot create the database pool. + pub async fn new(database_url: &str) -> Result { + let db = SqlitePoolOptions::new().connect(database_url).await?; + Ok(Self { pool: db }) } /// It migrates the database to the latest version. /// - /// # Panics + /// # Errors /// - /// It panics if it cannot run the migrations. - pub async fn migrate(&self) { - sqlx::migrate!("migrations/sqlite3") - .run(&self.pool) - .await - .expect("Could not run database migrations."); + /// Returns an error if it cannot run the migrations. + pub async fn migrate(&self) -> Result<(), sqlx::migrate::MigrateError> { + sqlx::migrate!("migrations/sqlite3").run(&self.pool).await } pub async fn reset_categories_sequence(&self) -> Result { @@ -267,13 +266,12 @@ impl SqliteDatabaseV2_0_0 { .map(|v| v.last_insert_rowid()) } - #[allow(clippy::missing_panics_doc)] pub async fn delete_all_database_rows(&self) -> Result<(), database::Error> { for table in TABLES_TO_TRUNCATE { query(&format!("DELETE FROM {table};")) .execute(&self.pool) .await - .unwrap_or_else(|_| panic!("table {table} should be deleted")); + .map_err(|_| database::Error::Error)?; } Ok(()) diff --git a/src/upgrades/from_v1_0_0_to_v2_0_0/error.rs b/src/upgrades/from_v1_0_0_to_v2_0_0/error.rs new file mode 100644 index 000000000..9a22e0600 --- /dev/null +++ b/src/upgrades/from_v1_0_0_to_v2_0_0/error.rs @@ -0,0 +1,44 @@ +use thiserror::Error; + +use crate::databases::database; + +#[derive(Debug, Error)] +pub enum UpgradeError { + #[error("{context}: {source}")] + Database { context: &'static str, source: database::Error }, + + #[error("{context}: {source}")] + Sqlx { context: &'static str, source: sqlx::Error }, + + #[error("{context}: {source}")] + Migration { + context: &'static str, + source: sqlx::migrate::MigrateError, + }, + + #[error("{context}: {source}")] + Io { context: &'static str, source: std::io::Error }, + + #[error("failed to decode torrent file {path}: {message}")] + DecodeTorrent { path: String, message: String }, + + #[error("{entity} id mismatch while copying data: expected {expected}, got {actual}")] + IdMismatch { + entity: &'static str, + expected: i64, + actual: i64, + }, + + #[error("uploader mismatch for torrent {torrent_id}: expected {expected}, got {actual}")] + UploaderMismatch { + torrent_id: i64, + expected: String, + actual: String, + }, + + #[error("missing {field} for torrent {torrent_id}")] + MissingTorrentField { torrent_id: i64, field: &'static str }, + + #[error("invalid timestamp {timestamp}")] + InvalidTimestamp { timestamp: i64 }, +} diff --git a/src/upgrades/from_v1_0_0_to_v2_0_0/mod.rs b/src/upgrades/from_v1_0_0_to_v2_0_0/mod.rs index afb35f90c..894793fe6 100644 --- a/src/upgrades/from_v1_0_0_to_v2_0_0/mod.rs +++ b/src/upgrades/from_v1_0_0_to_v2_0_0/mod.rs @@ -1,3 +1,4 @@ pub mod databases; +pub mod error; pub mod transferrers; pub mod upgrader; diff --git a/src/upgrades/from_v1_0_0_to_v2_0_0/transferrers/category_transferrer.rs b/src/upgrades/from_v1_0_0_to_v2_0_0/transferrers/category_transferrer.rs index 02472ec1e..9605daa48 100644 --- a/src/upgrades/from_v1_0_0_to_v2_0_0/transferrers/category_transferrer.rs +++ b/src/upgrades/from_v1_0_0_to_v2_0_0/transferrers/category_transferrer.rs @@ -1,37 +1,73 @@ use std::sync::Arc; +use tracing::{debug, info}; + use crate::upgrades::from_v1_0_0_to_v2_0_0::databases::sqlite_v1_0_0::SqliteDatabaseV1_0_0; use crate::upgrades::from_v1_0_0_to_v2_0_0::databases::sqlite_v2_0_0::{CategoryRecordV2, SqliteDatabaseV2_0_0}; +use crate::upgrades::from_v1_0_0_to_v2_0_0::error::UpgradeError; -#[allow(clippy::missing_panics_doc)] -pub async fn transfer_categories(source_database: Arc, target_database: Arc) { - println!("Transferring categories ..."); +/// Transfer categories from the source database to the target database. +/// +/// # Errors +/// +/// Returns an error if reading, inserting, or validating copied category data +/// fails. +pub async fn transfer_categories( + source_database: Arc, + target_database: Arc, +) -> Result<(), UpgradeError> { + info!("transferring categories"); - let source_categories = source_database.get_categories_order_by_id().await.unwrap(); - println!("[v1] categories: {source_categories:?}"); + let source_categories = source_database + .get_categories_order_by_id() + .await + .map_err(|source| UpgradeError::Database { + context: "failed to read source categories", + source, + })?; + debug!(?source_categories, "read source categories"); - let result = target_database.reset_categories_sequence().await.unwrap(); - println!("[v2] reset categories sequence result: {result:?}"); + let result = target_database + .reset_categories_sequence() + .await + .map_err(|source| UpgradeError::Database { + context: "failed to reset target category sequence", + source, + })?; + debug!(?result, "reset target category sequence"); - for cat in &source_categories { - println!("[v2] adding category {:?} with id {:?} ...", cat.name, cat.category_id); + for category in &source_categories { + info!(category_id = category.category_id, name = %category.name, "adding category"); let id = target_database .insert_category(&CategoryRecordV2 { - category_id: cat.category_id, - name: cat.name.clone(), + category_id: category.category_id, + name: category.name.clone(), }) .await - .unwrap(); + .map_err(|source| UpgradeError::Sqlx { + context: "failed to insert target category", + source, + })?; - assert!( - id == cat.category_id, - "Error copying category {:?} from source DB to the target DB", - cat.category_id - ); + if id != category.category_id { + return Err(UpgradeError::IdMismatch { + entity: "category", + expected: category.category_id, + actual: id, + }); + } - println!("[v2] category: {:?} {:?} added.", id, cat.name); + info!(category_id = id, name = %category.name, "category added"); } - let target_categories = target_database.get_categories().await.unwrap(); - println!("[v2] categories: {target_categories:?}"); + let target_categories = target_database + .get_categories() + .await + .map_err(|source| UpgradeError::Database { + context: "failed to read target categories", + source, + })?; + debug!(?target_categories, "read target categories"); + + Ok(()) } diff --git a/src/upgrades/from_v1_0_0_to_v2_0_0/transferrers/torrent_transferrer.rs b/src/upgrades/from_v1_0_0_to_v2_0_0/transferrers/torrent_transferrer.rs index 9a58755ca..fa2307445 100644 --- a/src/upgrades/from_v1_0_0_to_v2_0_0/transferrers/torrent_transferrer.rs +++ b/src/upgrades/from_v1_0_0_to_v2_0_0/transferrers/torrent_transferrer.rs @@ -1,21 +1,23 @@ #![allow(clippy::missing_errors_doc)] +use std::fs; use std::sync::Arc; -use std::{error, fs}; + +use tracing::{debug, info}; use crate::models::torrent_file::Torrent; use crate::upgrades::from_v1_0_0_to_v2_0_0::databases::sqlite_v1_0_0::SqliteDatabaseV1_0_0; use crate::upgrades::from_v1_0_0_to_v2_0_0::databases::sqlite_v2_0_0::{SqliteDatabaseV2_0_0, TorrentRecordV2}; +use crate::upgrades::from_v1_0_0_to_v2_0_0::error::UpgradeError; use crate::utils::parse_torrent::decode_torrent; -#[allow(clippy::missing_panics_doc)] #[allow(clippy::too_many_lines)] pub async fn transfer_torrents( source_database: Arc, target_database: Arc, upload_path: &str, -) { - println!("Transferring torrents ..."); +) -> Result<(), UpgradeError> { + info!("transferring torrents"); // Transfer table `torrust_torrents_files` @@ -24,149 +26,172 @@ pub async fn transfer_torrents( // Transfer table `torrust_torrents` - let torrents = source_database.get_torrents().await.unwrap(); + let torrents = source_database.get_torrents().await.map_err(|source| UpgradeError::Sqlx { + context: "failed to read source torrents", + source, + })?; for torrent in &torrents { // [v2] table torrust_torrents - println!("[v2][torrust_torrents] adding the torrent: {:?} ...", torrent.torrent_id); - - let uploader = source_database.get_user_by_username(&torrent.uploader).await.unwrap(); + info!(torrent_id = torrent.torrent_id, "adding torrent"); - assert!( - uploader.username == torrent.uploader, - "Error copying torrent with id {:?}. - Username (`uploader`) in `torrust_torrents` table does not match `username` in `torrust_users` table", - torrent.torrent_id - ); + let uploader = source_database + .get_user_by_username(&torrent.uploader) + .await + .map_err(|source| UpgradeError::Sqlx { + context: "failed to read torrent uploader", + source, + })?; + + if uploader.username != torrent.uploader { + return Err(UpgradeError::UploaderMismatch { + torrent_id: torrent.torrent_id, + expected: torrent.uploader.clone(), + actual: uploader.username, + }); + } let filepath = format!("{}/{}.torrent", upload_path, torrent.torrent_id); - let torrent_from_file = - read_torrent_from_file(&filepath).unwrap_or_else(|_| panic!("Error torrent file not found: {filepath:?}")); + let torrent_from_file = read_torrent_from_file(&filepath)?; + let target_torrent = TorrentRecordV2::from_v1_data(torrent, &torrent_from_file.info, &uploader)?; let id = target_database - .insert_torrent(&TorrentRecordV2::from_v1_data(torrent, &torrent_from_file.info, &uploader)) + .insert_torrent(&target_torrent) .await - .unwrap(); - - assert!( - id == torrent.torrent_id, - "Error copying torrent {:?} from source DB to the target DB", - torrent.torrent_id - ); + .map_err(|source| UpgradeError::Sqlx { + context: "failed to insert torrent", + source, + })?; + + if id != torrent.torrent_id { + return Err(UpgradeError::IdMismatch { + entity: "torrent", + expected: torrent.torrent_id, + actual: id, + }); + } - println!("[v2][torrust_torrents] torrent with id {:?} added.", torrent.torrent_id); + info!(torrent_id = torrent.torrent_id, "torrent added"); // [v2] table torrust_torrent_files - println!("[v2][torrust_torrent_files] adding torrent files"); + info!(torrent_id = torrent.torrent_id, "adding torrent files"); if torrent_from_file.is_a_single_file_torrent() { // The torrent contains only one file then: // - "path" is NULL // - "md5sum" can be NULL - println!( - "[v2][torrust_torrent_files][single-file-torrent] adding torrent file {:?} with length {:?} ...", - torrent_from_file.info.name, torrent_from_file.info.length, - ); + let length = torrent_from_file.info.length.ok_or(UpgradeError::MissingTorrentField { + torrent_id: torrent.torrent_id, + field: "length", + })?; + info!(torrent_id = torrent.torrent_id, name = %torrent_from_file.info.name, length, "adding single-file torrent entry"); let file_id = target_database .insert_torrent_file_for_torrent_with_one_file( torrent.torrent_id, // TODO: it seems med5sum can be None. Why? When? &torrent_from_file.info.md5sum.clone(), - torrent_from_file.info.length.unwrap(), + length, ) - .await; + .await + .map_err(|source| UpgradeError::Sqlx { + context: "failed to insert single-file torrent file", + source, + })?; - println!("[v2][torrust_torrent_files][single-file-torrent] torrent file insert result: {file_id:?}"); + debug!(file_id, torrent_id = torrent.torrent_id, "inserted single-file torrent entry"); } else { // Multiple files are being shared - let files = torrent_from_file.info.files.as_ref().unwrap(); + let files = torrent_from_file + .info + .files + .as_ref() + .ok_or(UpgradeError::MissingTorrentField { + torrent_id: torrent.torrent_id, + field: "files", + })?; for file in files { - println!("[v2][torrust_torrent_files][multiple-file-torrent] adding torrent file: {file:?} ..."); + debug!(torrent_id = torrent.torrent_id, ?file, "adding multi-file torrent entry"); let file_id = target_database .insert_torrent_file_for_torrent_with_multiple_files(torrent, file) - .await; + .await + .map_err(|source| UpgradeError::Sqlx { + context: "failed to insert multi-file torrent file", + source, + })?; - println!("[v2][torrust_torrent_files][multiple-file-torrent] torrent file insert result: {file_id:?}"); + debug!(file_id, torrent_id = torrent.torrent_id, "inserted multi-file torrent entry"); } } // [v2] table torrust_torrent_info - println!( - "[v2][torrust_torrent_info] adding the torrent info for torrent id {:?} ...", - torrent.torrent_id - ); + info!(torrent_id = torrent.torrent_id, "adding torrent info"); - let id = target_database.insert_torrent_info(torrent).await; + let id = target_database + .insert_torrent_info(torrent) + .await + .map_err(|source| UpgradeError::Sqlx { + context: "failed to insert torrent info", + source, + })?; - println!("[v2][torrust_torrents] torrent info insert result: {id:?}."); + debug!(torrent_info_id = id, torrent_id = torrent.torrent_id, "inserted torrent info"); // [v2] table torrust_torrent_announce_urls - println!( - "[v2][torrust_torrent_announce_urls] adding the torrent announce url for torrent id {:?} ...", - torrent.torrent_id - ); + info!(torrent_id = torrent.torrent_id, "adding torrent announce urls"); - if torrent_from_file.announce_list.is_some() { + if let Some(announce_list) = &torrent_from_file.announce_list { // BEP-0012. Multiple trackers. - println!( - "[v2][torrust_torrent_announce_urls][announce-list] adding the torrent announce url for torrent id {:?} ...", - torrent.torrent_id - ); - - // flatten the nested vec (this will however remove the) - let announce_urls = torrent_from_file - .announce_list - .clone() - .unwrap() - .into_iter() - .flatten() - .collect::>(); - - for tracker_url in &announce_urls { - println!( - "[v2][torrust_torrent_announce_urls][announce-list] adding the torrent announce url for torrent id {:?} ...", - torrent.torrent_id - ); + for tracker_url in announce_list.iter().flatten() { + debug!(torrent_id = torrent.torrent_id, %tracker_url, "adding announce-list URL"); let announce_url_id = target_database .insert_torrent_announce_url(torrent.torrent_id, tracker_url) - .await; + .await + .map_err(|source| UpgradeError::Sqlx { + context: "failed to insert announce-list URL", + source, + })?; - println!( - "[v2][torrust_torrent_announce_urls][announce-list] torrent announce url insert result {announce_url_id:?} ..." - ); + debug!(announce_url_id, torrent_id = torrent.torrent_id, "inserted announce-list URL"); } - } else if torrent_from_file.announce.is_some() { - println!( - "[v2][torrust_torrent_announce_urls][announce] adding the torrent announce url for torrent id {:?} ...", - torrent.torrent_id - ); + } else if let Some(announce) = &torrent_from_file.announce { + debug!(torrent_id = torrent.torrent_id, tracker_url = %announce, "adding announce URL"); let announce_url_id = target_database - .insert_torrent_announce_url(torrent.torrent_id, &torrent_from_file.announce.unwrap()) - .await; - - println!("[v2][torrust_torrent_announce_urls][announce] torrent announce url insert result {announce_url_id:?} ..."); + .insert_torrent_announce_url(torrent.torrent_id, announce) + .await + .map_err(|source| UpgradeError::Sqlx { + context: "failed to insert announce URL", + source, + })?; + + debug!(announce_url_id, torrent_id = torrent.torrent_id, "inserted announce URL"); } } - println!("Torrents transferred"); + + info!("torrents transferred"); + + Ok(()) } -pub fn read_torrent_from_file(path: &str) -> Result> { - let contents = fs::read(path)?; +pub fn read_torrent_from_file(path: &str) -> Result { + let contents = fs::read(path).map_err(|source| UpgradeError::Io { + context: "failed to read torrent file", + source, + })?; - match decode_torrent(&contents) { - Ok(torrent) => Ok(torrent), - Err(e) => Err(e), - } + decode_torrent(&contents).map_err(|error| UpgradeError::DecodeTorrent { + path: path.to_string(), + message: error.to_string(), + }) } diff --git a/src/upgrades/from_v1_0_0_to_v2_0_0/transferrers/tracker_key_transferrer.rs b/src/upgrades/from_v1_0_0_to_v2_0_0/transferrers/tracker_key_transferrer.rs index 9021a32fb..efef4a480 100644 --- a/src/upgrades/from_v1_0_0_to_v2_0_0/transferrers/tracker_key_transferrer.rs +++ b/src/upgrades/from_v1_0_0_to_v2_0_0/transferrers/tracker_key_transferrer.rs @@ -1,22 +1,40 @@ use std::sync::Arc; +use tracing::info; + use crate::upgrades::from_v1_0_0_to_v2_0_0::databases::sqlite_v1_0_0::SqliteDatabaseV1_0_0; use crate::upgrades::from_v1_0_0_to_v2_0_0::databases::sqlite_v2_0_0::SqliteDatabaseV2_0_0; +use crate::upgrades::from_v1_0_0_to_v2_0_0::error::UpgradeError; -#[allow(clippy::missing_panics_doc)] -pub async fn transfer_tracker_keys(source_database: Arc, target_database: Arc) { - println!("Transferring tracker keys ..."); +/// Transfer tracker keys from the source database to the target database. +/// +/// # Errors +/// +/// Returns an error if reading, inserting, or validating copied tracker-key data +/// fails. +pub async fn transfer_tracker_keys( + source_database: Arc, + target_database: Arc, +) -> Result<(), UpgradeError> { + info!("transferring tracker keys"); // Transfer table `torrust_tracker_keys` - let tracker_keys = source_database.get_tracker_keys().await.unwrap(); + let tracker_keys = source_database + .get_tracker_keys() + .await + .map_err(|source| UpgradeError::Sqlx { + context: "failed to read source tracker keys", + source, + })?; for tracker_key in &tracker_keys { // [v2] table torrust_tracker_keys - println!( - "[v2][torrust_users] adding the tracker key with id {:?} ...", - tracker_key.key_id + info!( + tracker_key_id = tracker_key.key_id, + user_id = tracker_key.user_id, + "adding tracker key" ); let id = target_database @@ -27,17 +45,25 @@ pub async fn transfer_tracker_keys(source_database: Arc, t tracker_key.valid_until, ) .await - .unwrap(); + .map_err(|source| UpgradeError::Sqlx { + context: "failed to insert tracker key", + source, + })?; - assert!( - id == tracker_key.key_id, - "Error copying tracker key {:?} from source DB to the target DB", - tracker_key.key_id - ); + if id != tracker_key.key_id { + return Err(UpgradeError::IdMismatch { + entity: "tracker key", + expected: tracker_key.key_id, + actual: id, + }); + } - println!( - "[v2][torrust_tracker_keys] tracker key with id {:?} added.", - tracker_key.key_id + info!( + tracker_key_id = tracker_key.key_id, + user_id = tracker_key.user_id, + "tracker key added" ); } + + Ok(()) } diff --git a/src/upgrades/from_v1_0_0_to_v2_0_0/transferrers/user_transferrer.rs b/src/upgrades/from_v1_0_0_to_v2_0_0/transferrers/user_transferrer.rs index 33ce0d789..3f577d18c 100644 --- a/src/upgrades/from_v1_0_0_to_v2_0_0/transferrers/user_transferrer.rs +++ b/src/upgrades/from_v1_0_0_to_v2_0_0/transferrers/user_transferrer.rs @@ -1,73 +1,81 @@ use std::sync::Arc; +use tracing::info; + use crate::upgrades::from_v1_0_0_to_v2_0_0::databases::sqlite_v1_0_0::SqliteDatabaseV1_0_0; use crate::upgrades::from_v1_0_0_to_v2_0_0::databases::sqlite_v2_0_0::SqliteDatabaseV2_0_0; +use crate::upgrades::from_v1_0_0_to_v2_0_0::error::UpgradeError; -#[allow(clippy::missing_panics_doc)] +/// Transfer users and their related profile/authentication records. +/// +/// # Errors +/// +/// Returns an error if reading, inserting, or validating copied user data fails. pub async fn transfer_users( source_database: Arc, target_database: Arc, date_imported: &str, -) { - println!("Transferring users ..."); +) -> Result<(), UpgradeError> { + info!("transferring users"); // Transfer table `torrust_users` - let users = source_database.get_users().await.unwrap(); + let users = source_database.get_users().await.map_err(|source| UpgradeError::Sqlx { + context: "failed to read source users", + source, + })?; for user in &users { // [v2] table torrust_users - println!( - "[v2][torrust_users] adding user with username {:?} and id {:?} ...", - user.username, user.user_id - ); + info!(user_id = user.user_id, username = %user.username, "adding user"); let id = target_database .insert_imported_user(user.user_id, date_imported, user.administrator) .await - .unwrap(); + .map_err(|source| UpgradeError::Sqlx { + context: "failed to insert imported user", + source, + })?; - assert!( - id == user.user_id, - "Error copying user {:?} from source DB to the target DB", - user.user_id - ); + if id != user.user_id { + return Err(UpgradeError::IdMismatch { + entity: "user", + expected: user.user_id, + actual: id, + }); + } - println!("[v2][torrust_users] user: {:?} {:?} added.", user.user_id, user.username); + info!(user_id = user.user_id, username = %user.username, "user added"); // [v2] table torrust_user_profiles - println!( - "[v2][torrust_user_profiles] adding user profile for user with username {:?} and id {:?} ...", - user.username, user.user_id - ); + info!(user_id = user.user_id, username = %user.username, "adding user profile"); target_database .insert_user_profile(user.user_id, &user.username, &user.email, user.email_verified) .await - .unwrap(); + .map_err(|source| UpgradeError::Sqlx { + context: "failed to insert user profile", + source, + })?; - println!( - "[v2][torrust_user_profiles] user profile added for user with username {:?} and id {:?}.", - user.username, user.user_id - ); + info!(user_id = user.user_id, username = %user.username, "user profile added"); // [v2] table torrust_user_authentication - println!( - "[v2][torrust_user_authentication] adding password hash ({:?}) for user id ({:?}) ...", - user.password, user.user_id - ); + info!(user_id = user.user_id, "adding user authentication"); target_database .insert_user_password_hash(user.user_id, &user.password) .await - .unwrap(); + .map_err(|source| UpgradeError::Sqlx { + context: "failed to insert user authentication", + source, + })?; - println!( - "[v2][torrust_user_authentication] password hash ({:?}) added for user id ({:?}).", - user.password, user.user_id - ); + info!(user_id = user.user_id, "user authentication added"); } + + Ok(()) } diff --git a/src/upgrades/from_v1_0_0_to_v2_0_0/upgrader.rs b/src/upgrades/from_v1_0_0_to_v2_0_0/upgrader.rs index 51b58637e..7fd1acf4f 100644 --- a/src/upgrades/from_v1_0_0_to_v2_0_0/upgrader.rs +++ b/src/upgrades/from_v1_0_0_to_v2_0_0/upgrader.rs @@ -3,7 +3,8 @@ //! # Usage //! //! ```bash -//! cargo run --bin upgrade SOURCE_DB_FILE TARGET_DB_FILE TORRENT_UPLOAD_DIR +//! cargo run --quiet --bin upgrade -- SOURCE_DB_FILE TARGET_DB_FILE TORRENT_UPLOAD_DIR 2>upgrade.ndjson +//! jq . upgrade.ndjson //! ``` //! //! Where: @@ -15,9 +16,14 @@ //! For example: //! //! ```bash -//! cargo run --bin upgrade ./data.db ./data_v2.db ./uploads +//! cargo run --quiet --bin upgrade -- ./data.db ./data_v2.db ./uploads 2>upgrade.ndjson +//! jq . upgrade.ndjson //! ``` //! +//! ADR-T-010 classifies this as a side-effect command: stdout remains empty and +//! diagnostics are JSON records on stderr. Scripts should branch on the process +//! exit code and parse stderr as NDJSON when they need diagnostics. +//! //! This command was created to help users to migrate from version `v1.0.0` to //! `v2.0.0`. The main changes in version `v2.0.0` were: //! @@ -46,20 +52,18 @@ //! //! //! If you want more information about this command you can read the [issue 56](https://github.com/torrust/torrust-index/issues/56). -use std::env; use std::time::SystemTime; use chrono::prelude::{DateTime, Utc}; -use text_colorizer::Colorize; +use tracing::info; use crate::upgrades::from_v1_0_0_to_v2_0_0::databases::{current_db, migrate_target_database, new_db, truncate_target_database}; +use crate::upgrades::from_v1_0_0_to_v2_0_0::error::UpgradeError; use crate::upgrades::from_v1_0_0_to_v2_0_0::transferrers::category_transferrer::transfer_categories; use crate::upgrades::from_v1_0_0_to_v2_0_0::transferrers::torrent_transferrer::transfer_torrents; use crate::upgrades::from_v1_0_0_to_v2_0_0::transferrers::tracker_key_transferrer::transfer_tracker_keys; use crate::upgrades::from_v1_0_0_to_v2_0_0::transferrers::user_transferrer::transfer_users; -const NUMBER_OF_ARGUMENTS: usize = 3; - #[derive(Debug)] pub struct Arguments { /// The source database in version v1.0.0 we want to migrate @@ -70,72 +74,48 @@ pub struct Arguments { pub upload_path: String, } -fn print_usage() { - eprintln!( - "{} - migrates date from version v1.0.0 to v2.0.0. - - cargo run --bin upgrade SOURCE_DB_FILE TARGET_DB_FILE TORRENT_UPLOAD_DIR - - For example: - - cargo run --bin upgrade ./data.db ./data_v2.db ./uploads - - ", - "Upgrader".green() - ); -} - -fn parse_args() -> Arguments { - let args: Vec = env::args().skip(1).collect(); - - if args.len() != NUMBER_OF_ARGUMENTS { - eprintln!( - "{} wrong number of arguments: expected {}, got {}", - "Error".red().bold(), - NUMBER_OF_ARGUMENTS, - args.len() - ); - print_usage(); - } - - Arguments { - source_database_file: args[0].clone(), - target_database_file: args[1].clone(), - upload_path: args[2].clone(), - } -} - -pub async fn run() { +/// Run the v1.0.0 to v2.0.0 upgrade with the current timestamp. +/// +/// # Errors +/// +/// Returns an error if any database setup, migration, truncation, or transfer +/// step fails. +pub async fn run(args: Arguments) -> Result<(), UpgradeError> { let now = datetime_iso_8601(); - upgrade(&parse_args(), &now).await; + upgrade(&args, &now).await } -pub async fn upgrade(args: &Arguments, date_imported: &str) { +/// Upgrade data from v1.0.0 to v2.0.0 using a caller-supplied import timestamp. +/// +/// # Errors +/// +/// Returns an error if any database setup, migration, truncation, or transfer +/// step fails. +pub async fn upgrade(args: &Arguments, date_imported: &str) -> Result<(), UpgradeError> { // Get connection to the source database (current DB in settings) - let source_database = current_db(&args.source_database_file).await; + let source_database = current_db(&args.source_database_file).await?; // Get connection to the target database (new DB we want to migrate the data) - let target_database = new_db(&args.target_database_file).await; + let target_database = new_db(&args.target_database_file).await?; - println!("Upgrading data from version v1.0.0 to v2.0.0 ..."); + info!("upgrading data from version v1.0.0 to v2.0.0"); - migrate_target_database(target_database.clone()).await; - truncate_target_database(target_database.clone()).await; + migrate_target_database(target_database.clone()).await?; + truncate_target_database(target_database.clone()).await?; - transfer_categories(source_database.clone(), target_database.clone()).await; - transfer_users(source_database.clone(), target_database.clone(), date_imported).await; - transfer_tracker_keys(source_database.clone(), target_database.clone()).await; - transfer_torrents(source_database.clone(), target_database.clone(), &args.upload_path).await; + transfer_categories(source_database.clone(), target_database.clone()).await?; + transfer_users(source_database.clone(), target_database.clone(), date_imported).await?; + transfer_tracker_keys(source_database.clone(), target_database.clone()).await?; + transfer_torrents(source_database.clone(), target_database.clone(), &args.upload_path).await?; - println!("Upgrade data from version v1.0.0 to v2.0.0 finished!\n"); + info!("upgrade data from version v1.0.0 to v2.0.0 finished"); - eprintln!( - "{}\nWe recommend you to run the command to import torrent statistics for all torrents manually. \ - If you do not do it the statistics will be imported anyway during the normal execution of the program. \ - You can import statistics manually with:\n {}", - "SUGGESTION: \n".yellow(), - "cargo run --bin import_tracker_statistics".yellow() + info!( + command = "cargo run --quiet --bin import_tracker_statistics -- 2>import-tracker-statistics.ndjson", + "run the tracker statistics importer manually if you want all torrent statistics imported before normal execution" ); + + Ok(()) } /// Current datetime in ISO8601 without time zone. diff --git a/src/utils/parse_torrent.rs b/src/utils/parse_torrent.rs index 1a0a3fc95..62ffba389 100644 --- a/src/utils/parse_torrent.rs +++ b/src/utils/parse_torrent.rs @@ -58,13 +58,7 @@ pub fn decode_and_validate_torrent_file(bytes: &[u8]) -> Result<(Torrent, InfoHa /// /// This function will return an error if unable to parse bytes into torrent. pub fn decode_torrent(bytes: &[u8]) -> Result> { - match de::from_bytes::(bytes) { - Ok(torrent) => Ok(torrent), - Err(e) => { - println!("{e:?}"); - Err(e.into()) - } - } + de::from_bytes::(bytes).map_err(Into::into) } /// Encode a Torrent into Bencoded Bytes. @@ -73,13 +67,7 @@ pub fn decode_torrent(bytes: &[u8]) -> Result> { /// /// This function will return an error if unable to bencode torrent. pub fn encode_torrent(torrent: &Torrent) -> Result, SerdeError> { - match serde_bencode::to_bytes(torrent) { - Ok(bencode_bytes) => Ok(bencode_bytes), - Err(e) => { - eprintln!("{e:?}"); - Err(e) - } - } + serde_bencode::to_bytes(torrent) } #[derive(Serialize, Deserialize, Debug, PartialEq)] diff --git a/src/web/api/server/signals.rs b/src/web/api/server/signals.rs index e19a36893..4c1f457be 100644 --- a/src/web/api/server/signals.rs +++ b/src/web/api/server/signals.rs @@ -26,10 +26,15 @@ pub async fn graceful_shutdown( ) { shutdown_signal_with_message(rx_halt, message).await; + let graceful_shutdown_timeout = Duration::from_secs(90); + info!("Sending graceful shutdown signal"); - handle.graceful_shutdown(Some(Duration::from_secs(90))); + handle.graceful_shutdown(Some(graceful_shutdown_timeout)); - println!("!! shuting down in 90 seconds !!"); + info!( + grace_period_seconds = graceful_shutdown_timeout.as_secs(), + "shutting down gracefully" + ); loop { sleep(Duration::from_secs(1)).await; @@ -38,7 +43,7 @@ pub async fn graceful_shutdown( } } -/// Same as `shutdown_signal()`, but shows a message when it resolves. +/// Same as `shutdown_signal()`, but logs a message when it resolves. pub async fn shutdown_signal_with_message(rx_halt: tokio::sync::oneshot::Receiver, message: String) { shutdown_signal(rx_halt).await; diff --git a/tests/cli_contract.rs b/tests/cli_contract.rs new file mode 100644 index 000000000..65feb28c6 --- /dev/null +++ b/tests/cli_contract.rs @@ -0,0 +1,80 @@ +//! # ADR-T-010 CLI contract regression tests +//! +//! | Test | What it covers | +//! |--------------------------------------------------|-----------------------------------------------------| +//! | `in_scope_binaries_return_exit_code_from_main` | In-scope binaries keep explicit `ExitCode` mains. | + +const IN_SCOPE_BINARIES: &[(&str, &str)] = &[ + ("src/main.rs", include_str!("../src/main.rs")), + ( + "src/bin/create_test_torrent.rs", + include_str!("../src/bin/create_test_torrent.rs"), + ), + ( + "src/bin/import_tracker_statistics.rs", + include_str!("../src/bin/import_tracker_statistics.rs"), + ), + ("src/bin/parse_torrent.rs", include_str!("../src/bin/parse_torrent.rs")), + ("src/bin/seeder.rs", include_str!("../src/bin/seeder.rs")), + ("src/bin/upgrade.rs", include_str!("../src/bin/upgrade.rs")), + ( + "packages/index-auth-keypair/src/bin/torrust-index-auth-keypair.rs", + include_str!("../packages/index-auth-keypair/src/bin/torrust-index-auth-keypair.rs"), + ), + ( + "packages/index-config-probe/src/bin/torrust-index-config-probe.rs", + include_str!("../packages/index-config-probe/src/bin/torrust-index-config-probe.rs"), + ), + ( + "packages/index-health-check/src/bin/torrust-index-health-check.rs", + include_str!("../packages/index-health-check/src/bin/torrust-index-health-check.rs"), + ), +]; + +#[test] +fn in_scope_binaries_return_exit_code_from_main() { + let mut failures = Vec::new(); + + for &(relative_path, source) in IN_SCOPE_BINARIES { + let Some(signature) = main_signature(source) else { + failures.push(format!("{relative_path}: missing main function")); + continue; + }; + + if !signature.contains("-> ExitCode") { + failures.push(format!( + "{relative_path}: main must return ExitCode instead of using Rust's default termination: `{signature}`" + )); + } + + if signature.contains("-> Result") { + failures.push(format!( + "{relative_path}: main must not return Result because default termination writes raw stderr: `{signature}`" + )); + } + } + + let message = failures.join("\n"); + assert!(failures.is_empty(), "{message}"); +} + +fn main_signature(source: &str) -> Option { + let start = source.find("fn main(")?; + let rest = &source[start..]; + let end = rest.find('{').unwrap_or(rest.len()); + + Some(collapse_whitespace(&rest[..end])) +} + +fn collapse_whitespace(value: &str) -> String { + let mut collapsed = String::new(); + + for part in value.split_whitespace() { + if !collapsed.is_empty() { + collapsed.push(' '); + } + collapsed.push_str(part); + } + + collapsed +} diff --git a/tests/common/contexts/torrent/fixtures.rs b/tests/common/contexts/torrent/fixtures.rs index aac5ccf70..3924d7368 100644 --- a/tests/common/contexts/torrent/fixtures.rs +++ b/tests/common/contexts/torrent/fixtures.rs @@ -1,3 +1,5 @@ +#![allow(clippy::print_stderr, clippy::print_stdout)] + use std::fs::File; use std::io::Write; use std::path::{Path, PathBuf}; diff --git a/tests/e2e/web/api/v1/contexts/torrent/contract.rs b/tests/e2e/web/api/v1/contexts/torrent/contract.rs index 1bf7ace90..70d84831e 100644 --- a/tests/e2e/web/api/v1/contexts/torrent/contract.rs +++ b/tests/e2e/web/api/v1/contexts/torrent/contract.rs @@ -1,3 +1,5 @@ +#![allow(clippy::print_stdout)] + //! API contract for `torrent` context. //! //! # Test modules diff --git a/tests/e2e/web/api/v1/contexts/torrent/steps.rs b/tests/e2e/web/api/v1/contexts/torrent/steps.rs index 3f3564817..9301ab67b 100644 --- a/tests/e2e/web/api/v1/contexts/torrent/steps.rs +++ b/tests/e2e/web/api/v1/contexts/torrent/steps.rs @@ -1,3 +1,5 @@ +#![allow(clippy::print_stdout)] + use std::str::FromStr; use bittorrent_primitives::info_hash::InfoHash; diff --git a/tests/upgrades/from_v1_0_0_to_v2_0_0/sqlite_v1_0_0.rs b/tests/upgrades/from_v1_0_0_to_v2_0_0/sqlite_v1_0_0.rs index 9d13635c4..36f91b000 100644 --- a/tests/upgrades/from_v1_0_0_to_v2_0_0/sqlite_v1_0_0.rs +++ b/tests/upgrades/from_v1_0_0_to_v2_0_0/sqlite_v1_0_0.rs @@ -1,4 +1,4 @@ -#![allow(clippy::missing_errors_doc)] +#![allow(clippy::missing_errors_doc, clippy::print_stdout)] use std::fs; diff --git a/tests/upgrades/from_v1_0_0_to_v2_0_0/transferrer_testers/torrent_transferrer_tester.rs b/tests/upgrades/from_v1_0_0_to_v2_0_0/transferrer_testers/torrent_transferrer_tester.rs index c468ae7c7..f3eb45a1e 100644 --- a/tests/upgrades/from_v1_0_0_to_v2_0_0/transferrer_testers/torrent_transferrer_tester.rs +++ b/tests/upgrades/from_v1_0_0_to_v2_0_0/transferrer_testers/torrent_transferrer_tester.rs @@ -123,7 +123,7 @@ impl TorrentTester { assert_eq!(imported_torrent.is_bep_30, i64::from(torrent_file.info.is_bep_30())); assert_eq!( imported_torrent.date_uploaded, - convert_timestamp_to_datetime(torrent.upload_date) + convert_timestamp_to_datetime(torrent.upload_date).expect("fixture timestamp must be valid") ); } diff --git a/tests/upgrades/from_v1_0_0_to_v2_0_0/upgrader.rs b/tests/upgrades/from_v1_0_0_to_v2_0_0/upgrader.rs index 7344f7f2c..3a846ea70 100644 --- a/tests/upgrades/from_v1_0_0_to_v2_0_0/upgrader.rs +++ b/tests/upgrades/from_v1_0_0_to_v2_0_0/upgrader.rs @@ -83,7 +83,8 @@ async fn upgrades_data_from_version_v1_0_0_to_v2_0_0() { }, &execution_time, ) - .await; + .await + .expect("upgrade must succeed for fixture data"); // Assertions for data transferred to the new database in version v2.0.0 category_tester.assert_data_in_target_db().await; diff --git a/upgrades/from_v1_0_0_to_v2_0_0/README.md b/upgrades/from_v1_0_0_to_v2_0_0/README.md index 376091498..bc4d4ce9c 100644 --- a/upgrades/from_v1_0_0_to_v2_0_0/README.md +++ b/upgrades/from_v1_0_0_to_v2_0_0/README.md @@ -7,7 +7,13 @@ To upgrade from version `v1.0.0` to `v2.0.0` you have to follow these steps: - Back up your current database and the `uploads` folder. You can find which database and upload folder are you using in the `Config.toml` file in the root folder of your installation. - Set up a local environment exactly as you have it in production with your production data (DB and torrents folder). - Run the application locally with: `cargo run`. -- Execute the upgrader command: `cargo run --bin upgrade ./data.db ./data_v2.db ./uploads` +- Execute the upgrader command and capture its JSON stderr diagnostics: + +```sh +cargo run --quiet --bin upgrade -- ./data.db ./data_v2.db ./uploads 2>upgrade.ndjson +jq . upgrade.ndjson +``` + - A new SQLite file should have been created in the root folder: `data_v2.db` - Stop the running application and change the DB configuration to use the newly generated configuration: @@ -20,6 +26,22 @@ connect_url = "sqlite://data_v2.db?mode=rwc" - Perform some tests. - If all tests pass, stop the production service, replace the DB, and start it again. +## Command Output + +`upgrade` is an ADR-T-010 side-effect command: stdout is empty on success and on +failure. Status, help, version, usage errors, migration failures, and panic +diagnostics are emitted as JSON records on stderr. Stderr is NDJSON when the +command emits more than one record. + +Automation should branch on the process exit code and parse stderr as JSON when +it needs diagnostics. Usage failures, including invalid argv, exit with code 2; +runtime upgrade failures exit with code 1. + +```sh +cargo run --quiet --bin upgrade -- ./data.db ./data_v2.db ./uploads 2>upgrade.ndjson +jq . upgrade.ndjson +``` + ## Tests Before replacing the DB in production you can make some tests like: