From ae99924ba8e5b3e96bc8db1159f0faed52a37705 Mon Sep 17 00:00:00 2001 From: Yushan Lin Date: Sun, 12 Jul 2026 16:05:01 -0700 Subject: [PATCH 1/7] docs(config): add config/README.md documenting YAML configuration Documents the service, repository, and storage config sections with field-level tables (required/type/default/description) and example YAML snippets. --- config/README.md | 62 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 config/README.md diff --git a/config/README.md b/config/README.md new file mode 100644 index 00000000..7a3484f0 --- /dev/null +++ b/config/README.md @@ -0,0 +1,62 @@ +# Tango Configuration + +Tango is configured via a single YAML file. Pass the file path to `config.Parse` at startup. + +--- + +## `service` + +Controls how Tango operates at the service level — how many concurrent requests it can handle per repository, where it stores clones and worker checkouts on disk, and how it chunks responses over gRPC streams. + +```yaml +service: + max_worker_pool_size: 5 + workspaces_root: "/var/tango" + # streaming: + # max_num_targets: 250 + # max_num_changed_targets: 125 + # max_num_metadata_entries: 50000 +``` + +| Field | Required | Type | Default | Description | +|---|---|---|---|---| +| `max_worker_pool_size` | **Yes** | `int` | | Max number of concurrent requests per repository. Each worker is a lightweight local clone (hardlinked to the origin, not a full copy) that handles one request at a time — setting this to 5 means up to 5 concurrent graph computations per repo, and additional requests queue until a worker is free. A good starting point is the expected peak concurrent requests per repo. Must be greater than 0. | +| `workspaces_root` | **Yes** | `string` | | Root directory where Tango stores repository clones and worker checkouts. Required because Tango needs a known location to clone repositories and create worker checkouts — without it, there is no disk space allocated for graph computation. Choose a location with enough disk space to hold one origin clone plus N worker checkouts (where N is `max_worker_pool_size`) per configured repository. Use a persistent location in production — if this directory is lost, Tango re-clones from the remote on the next request, which can be slow for large repositories. Layout: `//` for origin clones and `/.workers//worker-{1..N}/` for worker checkouts. | +| `streaming.max_num_targets` | No | `int` | `250` | Max number of target entries per gRPC stream message (see `OptimizedTarget` in `proto/tango.proto`). Keep messages under gRPC's default 4MB receive limit — raise this only if the client is configured to accept larger messages and you want fewer round-trips. | +| `streaming.max_num_changed_targets` | No | `int` | `125` | Max number of changed target entries per stream message (see `ChangedTarget` in `proto/tango.proto`). Same tradeoff as `max_num_targets`. Default is lower because each entry carries both old and new targets (~2x the size). | +| `streaming.max_num_metadata_entries` | No | `int` | `50000` | Max number of ID-to-name mapping entries per stream message (see `Metadata` in `proto/tango.proto`). Large monorepos can have hundreds of thousands of metadata entries — lowering this value splits them across more messages, reducing peak memory per message. | + +## `repository` + +A list of repository entries. Each entry tells Tango how to clone, and query for a specific repository — including which Bazel settings to use. + +```yaml +repository: + - remote: "https://github.com/uber/tango.git" + bzlmod_enabled: true + # query_timeout_seconds: 600 +``` + +| Field | Required | Type | Default | Description | +|---|---|---|---|---| +| `remote` | **Yes** | `string` | | The URL used to `git clone` the repository. Tango clones from this URL and uses it as the lookup key for per-repo settings. Must be unique across all entries and match exactly what clients send in `BuildDescription.remote`. | +| `bzlmod_enabled` | No | `bool` | `true` | Whether this repository uses Bzlmod for external dependency management. Bzlmod is enforced in Bazel 9+. Set to false only for repositories still using WORKSPACE — when disabled, `//external:all-targets` is added to Bazel queries. Setting this incorrectly causes query failures — it must match the repository's actual dependency management setup. | +| `bazel_command` | No | `string` | `""` | Override the Bazel binary path. When empty, Tango automatically downloads and caches [Bazelisk](https://github.com/bazelbuild/bazelisk) from GitHub. This lets Tango run without Bazel pre-installed and allows each repository to control its own Bazel version via `.bazelversion`. Set this when the repository requires a custom Bazel wrapper script. | +| `bazel_extra_args` | No | `[]string` | `[]` | Extra arguments passed to `bazel query` invocations. These are inserted between the `query` subcommand and the query expression. Use for query flags like `--profile=/tmp/bazel-profile.json` to capture Bazel trace info for performance debugging.| +| `query_timeout_seconds` | No | `int64` | `600` | Bazel query timeout in seconds. Large monorepos with deep dependency trees may need a higher value. If queries are timing out, increase this. Setting it too low causes premature failures on valid but slow queries. | + +## `storage` (optional) + +Configures the built-in `memory` or `disk` storage backend. Omit this section if you provide a custom storage implementation. + +```yaml +# storage: +# type: "disk" +# disk: +# root_path: "/var/tango/blobs" +``` + +| Field | Required | Type | Default | Description | +|---|---|---|---|---| +| `type` | No | `string` | `"memory"` | Storage backend. `"memory"` stores blobs in-process — fast but lost on restart, suitable for development and testing. `"disk"` persists blobs to the filesystem so cached graphs survive restarts. | +| `disk.root_path` | Yes (if `type` is `"disk"`) | `string` | | Directory for on-disk blob storage. Must have enough space to hold cached target graphs for all configured repositories. | From 1f05bd84fdf4d62bfb6c97b608d156915f38ee7d Mon Sep 17 00:00:00 2001 From: Yushan Lin Date: Mon, 13 Jul 2026 15:25:57 -0700 Subject: [PATCH 2/7] Update --- config/README.md | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/config/README.md b/config/README.md index 7a3484f0..39e1ed7e 100644 --- a/config/README.md +++ b/config/README.md @@ -2,6 +2,18 @@ Tango is configured via a single YAML file. Pass the file path to `config.Parse` at startup. +## Configuration Lifecycle + +In the internal example, `config.Parse` only runs once, at startup (see `example/main.go`). The result gets fed into the `RepoManager`, `Storage`, and other constructors as a one-shot value when the service wires itself together. + +### `service` / `storage` — requires a deployment + +Editing the YAML file while the service is running does nothing until it's redeployed. To pick up a change to `service` or `storage`, redeploy the service. + +### `repository` + +`repository` config is read per request via `RepositoryConfigProvider`, not baked into a constructor at startup, so it doesn't have to come from the static file at all. It could be fetched from a third-party config service that Tango fetches per request, so a new repo or a changed setting shows up without a deployment. Or, since Tango clones and checks out the target repo anyway, it could just as easily be a plain file checked into that repo itself and read out of the checkout at request time. Either way, `repository` config can change while the service keeps running. + --- ## `service` @@ -22,9 +34,9 @@ service: |---|---|---|---|---| | `max_worker_pool_size` | **Yes** | `int` | | Max number of concurrent requests per repository. Each worker is a lightweight local clone (hardlinked to the origin, not a full copy) that handles one request at a time — setting this to 5 means up to 5 concurrent graph computations per repo, and additional requests queue until a worker is free. A good starting point is the expected peak concurrent requests per repo. Must be greater than 0. | | `workspaces_root` | **Yes** | `string` | | Root directory where Tango stores repository clones and worker checkouts. Required because Tango needs a known location to clone repositories and create worker checkouts — without it, there is no disk space allocated for graph computation. Choose a location with enough disk space to hold one origin clone plus N worker checkouts (where N is `max_worker_pool_size`) per configured repository. Use a persistent location in production — if this directory is lost, Tango re-clones from the remote on the next request, which can be slow for large repositories. Layout: `//` for origin clones and `/.workers//worker-{1..N}/` for worker checkouts. | -| `streaming.max_num_targets` | No | `int` | `250` | Max number of target entries per gRPC stream message (see `OptimizedTarget` in `proto/tango.proto`). Keep messages under gRPC's default 4MB receive limit — raise this only if the client is configured to accept larger messages and you want fewer round-trips. | -| `streaming.max_num_changed_targets` | No | `int` | `125` | Max number of changed target entries per stream message (see `ChangedTarget` in `proto/tango.proto`). Same tradeoff as `max_num_targets`. Default is lower because each entry carries both old and new targets (~2x the size). | -| `streaming.max_num_metadata_entries` | No | `int` | `50000` | Max number of ID-to-name mapping entries per stream message (see `Metadata` in `proto/tango.proto`). Large monorepos can have hundreds of thousands of metadata entries — lowering this value splits them across more messages, reducing peak memory per message. | +| `streaming.max_num_targets` | No | `int` | `250` | Max number of target entries per gRPC stream message (see `OptimizedTarget` in `proto/tango.proto`). The default is sized to stay well under gRPC's default 4MB receive limit for a typical target. If a repository has unusually large targets, tune this down to fit its needs. | +| `streaming.max_num_changed_targets` | No | `int` | `125` | Max number of changed target entries per stream message (see `ChangedTarget` in `proto/tango.proto`). Default is lower than `max_num_targets` because each entry carries both old and new targets (~2x the size). If a repository has unusually large targets, tune this down to fit its needs. | +| `streaming.max_num_metadata_entries` | No | `int` | `50000` | Max number of ID-to-name mapping entries per stream message (see `Metadata` in `proto/tango.proto`). If a repository has unusually large metadata entries, tune this down to fit its needs. | ## `repository` From cd9f9eac441d724075cd06fdba4b51ef24e2c2d0 Mon Sep 17 00:00:00 2001 From: Yushan Lin Date: Mon, 13 Jul 2026 15:27:50 -0700 Subject: [PATCH 3/7] Update --- config/README.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/config/README.md b/config/README.md index 39e1ed7e..913c08e9 100644 --- a/config/README.md +++ b/config/README.md @@ -4,11 +4,9 @@ Tango is configured via a single YAML file. Pass the file path to `config.Parse` ## Configuration Lifecycle -In the internal example, `config.Parse` only runs once, at startup (see `example/main.go`). The result gets fed into the `RepoManager`, `Storage`, and other constructors as a one-shot value when the service wires itself together. +### `service` / `storage` — requires a deployment/restart -### `service` / `storage` — requires a deployment - -Editing the YAML file while the service is running does nothing until it's redeployed. To pick up a change to `service` or `storage`, redeploy the service. +Editing the YAML file while the service is running does nothing until it's deployed or restarted. To pick up a change to `service` or `storage`, redeploy the service. ### `repository` From 91f36af517134d56cf0d953197a9066298166661 Mon Sep 17 00:00:00 2001 From: Yushan Lin Date: Tue, 14 Jul 2026 09:49:41 -0700 Subject: [PATCH 4/7] Update --- config/README.md | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/config/README.md b/config/README.md index 913c08e9..8c09ee3a 100644 --- a/config/README.md +++ b/config/README.md @@ -22,19 +22,15 @@ Controls how Tango operates at the service level — how many concurrent request service: max_worker_pool_size: 5 workspaces_root: "/var/tango" - # streaming: - # max_num_targets: 250 - # max_num_changed_targets: 125 - # max_num_metadata_entries: 50000 + # max_message_bytes: 4250000 ``` | Field | Required | Type | Default | Description | |---|---|---|---|---| | `max_worker_pool_size` | **Yes** | `int` | | Max number of concurrent requests per repository. Each worker is a lightweight local clone (hardlinked to the origin, not a full copy) that handles one request at a time — setting this to 5 means up to 5 concurrent graph computations per repo, and additional requests queue until a worker is free. A good starting point is the expected peak concurrent requests per repo. Must be greater than 0. | | `workspaces_root` | **Yes** | `string` | | Root directory where Tango stores repository clones and worker checkouts. Required because Tango needs a known location to clone repositories and create worker checkouts — without it, there is no disk space allocated for graph computation. Choose a location with enough disk space to hold one origin clone plus N worker checkouts (where N is `max_worker_pool_size`) per configured repository. Use a persistent location in production — if this directory is lost, Tango re-clones from the remote on the next request, which can be slow for large repositories. Layout: `//` for origin clones and `/.workers//worker-{1..N}/` for worker checkouts. | -| `streaming.max_num_targets` | No | `int` | `250` | Max number of target entries per gRPC stream message (see `OptimizedTarget` in `proto/tango.proto`). The default is sized to stay well under gRPC's default 4MB receive limit for a typical target. If a repository has unusually large targets, tune this down to fit its needs. | -| `streaming.max_num_changed_targets` | No | `int` | `125` | Max number of changed target entries per stream message (see `ChangedTarget` in `proto/tango.proto`). Default is lower than `max_num_targets` because each entry carries both old and new targets (~2x the size). If a repository has unusually large targets, tune this down to fit its needs. | -| `streaming.max_num_metadata_entries` | No | `int` | `50000` | Max number of ID-to-name mapping entries per stream message (see `Metadata` in `proto/tango.proto`). If a repository has unusually large metadata entries, tune this down to fit its needs. | +| `max_message_bytes` | No | `int` | `4250000` (~4.25MB) | Caps the size of each gRPC stream message Tango sends (targets, changed targets, and metadata mappings — see `proto/tango.proto`). This is a single byte budget rather than separate per-entry-type counts: Tango derives how many targets, changed targets, or metadata entries fit in one message from fixed size assumptions (a target ≈ 40KB worst case, a changed target ≈ 2x that since it carries both an old and new target, a metadata entry ≈ 85 bytes), computed once rather than measured per request. The default stays well under gRPC's default 64MB per-message limit. If a repository has unusually large targets or metadata entries, tune this down to fit its needs. | + ## `repository` From 5d5876a54c66c17c7725322206d7108359195371 Mon Sep 17 00:00:00 2001 From: Yushan Lin Date: Tue, 14 Jul 2026 09:51:47 -0700 Subject: [PATCH 5/7] Update --- config/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/config/README.md b/config/README.md index 8c09ee3a..e6b47d54 100644 --- a/config/README.md +++ b/config/README.md @@ -29,7 +29,7 @@ service: |---|---|---|---|---| | `max_worker_pool_size` | **Yes** | `int` | | Max number of concurrent requests per repository. Each worker is a lightweight local clone (hardlinked to the origin, not a full copy) that handles one request at a time — setting this to 5 means up to 5 concurrent graph computations per repo, and additional requests queue until a worker is free. A good starting point is the expected peak concurrent requests per repo. Must be greater than 0. | | `workspaces_root` | **Yes** | `string` | | Root directory where Tango stores repository clones and worker checkouts. Required because Tango needs a known location to clone repositories and create worker checkouts — without it, there is no disk space allocated for graph computation. Choose a location with enough disk space to hold one origin clone plus N worker checkouts (where N is `max_worker_pool_size`) per configured repository. Use a persistent location in production — if this directory is lost, Tango re-clones from the remote on the next request, which can be slow for large repositories. Layout: `//` for origin clones and `/.workers//worker-{1..N}/` for worker checkouts. | -| `max_message_bytes` | No | `int` | `4250000` (~4.25MB) | Caps the size of each gRPC stream message Tango sends (targets, changed targets, and metadata mappings — see `proto/tango.proto`). This is a single byte budget rather than separate per-entry-type counts: Tango derives how many targets, changed targets, or metadata entries fit in one message from fixed size assumptions (a target ≈ 40KB worst case, a changed target ≈ 2x that since it carries both an old and new target, a metadata entry ≈ 85 bytes), computed once rather than measured per request. The default stays well under gRPC's default 64MB per-message limit. If a repository has unusually large targets or metadata entries, tune this down to fit its needs. | +| `max_message_bytes` | No | `int` | `4250000` (~4.25MB) | Caps the size of each gRPC stream message Tango sends (targets, changed targets, and metadata mappings — see `proto/tango.proto`). This is a single byte budget rather than separate per-entry-type counts: Tango derives how many targets, changed targets, or metadata entries fit in one message from fixed size assumptions (a target ≈ 40KB worst case, a changed target ≈ 2x that since it carries both an old and new target, a metadata entry ≈ 85 bytes), computed once rather than measured per request. If a repository has unusually large targets or metadata entries, tune this down to fit its needs. | ## `repository` From 01ae7e8e7e1bd411ef388a6eee266121f1e657a4 Mon Sep 17 00:00:00 2001 From: Yushan Lin Date: Tue, 14 Jul 2026 13:07:22 -0700 Subject: [PATCH 6/7] Update --- config/README.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/config/README.md b/config/README.md index e6b47d54..27b04148 100644 --- a/config/README.md +++ b/config/README.md @@ -21,14 +21,14 @@ Controls how Tango operates at the service level — how many concurrent request ```yaml service: max_worker_pool_size: 5 - workspaces_root: "/var/tango" - # max_message_bytes: 4250000 + workspaces_root_path: "/var/tango" + max_message_bytes: 4250000 ``` | Field | Required | Type | Default | Description | |---|---|---|---|---| | `max_worker_pool_size` | **Yes** | `int` | | Max number of concurrent requests per repository. Each worker is a lightweight local clone (hardlinked to the origin, not a full copy) that handles one request at a time — setting this to 5 means up to 5 concurrent graph computations per repo, and additional requests queue until a worker is free. A good starting point is the expected peak concurrent requests per repo. Must be greater than 0. | -| `workspaces_root` | **Yes** | `string` | | Root directory where Tango stores repository clones and worker checkouts. Required because Tango needs a known location to clone repositories and create worker checkouts — without it, there is no disk space allocated for graph computation. Choose a location with enough disk space to hold one origin clone plus N worker checkouts (where N is `max_worker_pool_size`) per configured repository. Use a persistent location in production — if this directory is lost, Tango re-clones from the remote on the next request, which can be slow for large repositories. Layout: `//` for origin clones and `/.workers//worker-{1..N}/` for worker checkouts. | +| `workspaces_root_path` | **Yes** | `string` | | Root directory where Tango stores repository clones and worker checkouts. Required because Tango needs a known location to clone repositories and create worker checkouts — without it, there is no disk space allocated for graph computation. Choose a location with enough disk space to hold one origin clone plus N worker checkouts (where N is `max_worker_pool_size`) per configured repository. Use a persistent location in production — if this directory is lost, Tango re-clones from the remote on the next request, which can be slow for large repositories. Layout: `//` for origin clones and `/.workers//worker-{1..N}/` for worker checkouts. | | `max_message_bytes` | No | `int` | `4250000` (~4.25MB) | Caps the size of each gRPC stream message Tango sends (targets, changed targets, and metadata mappings — see `proto/tango.proto`). This is a single byte budget rather than separate per-entry-type counts: Tango derives how many targets, changed targets, or metadata entries fit in one message from fixed size assumptions (a target ≈ 40KB worst case, a changed target ≈ 2x that since it carries both an old and new target, a metadata entry ≈ 85 bytes), computed once rather than measured per request. If a repository has unusually large targets or metadata entries, tune this down to fit its needs. | @@ -40,14 +40,14 @@ A list of repository entries. Each entry tells Tango how to clone, and query for repository: - remote: "https://github.com/uber/tango.git" bzlmod_enabled: true - # query_timeout_seconds: 600 + query_timeout_seconds: 600 ``` | Field | Required | Type | Default | Description | |---|---|---|---|---| | `remote` | **Yes** | `string` | | The URL used to `git clone` the repository. Tango clones from this URL and uses it as the lookup key for per-repo settings. Must be unique across all entries and match exactly what clients send in `BuildDescription.remote`. | | `bzlmod_enabled` | No | `bool` | `true` | Whether this repository uses Bzlmod for external dependency management. Bzlmod is enforced in Bazel 9+. Set to false only for repositories still using WORKSPACE — when disabled, `//external:all-targets` is added to Bazel queries. Setting this incorrectly causes query failures — it must match the repository's actual dependency management setup. | -| `bazel_command` | No | `string` | `""` | Override the Bazel binary path. When empty, Tango automatically downloads and caches [Bazelisk](https://github.com/bazelbuild/bazelisk) from GitHub. This lets Tango run without Bazel pre-installed and allows each repository to control its own Bazel version via `.bazelversion`. Set this when the repository requires a custom Bazel wrapper script. | +| `bazel_command_path` | No | `string` | `""` | Override the Bazel binary path. When empty, Tango automatically downloads and caches [Bazelisk](https://github.com/bazelbuild/bazelisk) from GitHub. This lets Tango run without Bazel pre-installed and allows each repository to control its own Bazel version via `.bazelversion`. Set this when the repository requires a custom Bazel wrapper script. | | `bazel_extra_args` | No | `[]string` | `[]` | Extra arguments passed to `bazel query` invocations. These are inserted between the `query` subcommand and the query expression. Use for query flags like `--profile=/tmp/bazel-profile.json` to capture Bazel trace info for performance debugging.| | `query_timeout_seconds` | No | `int64` | `600` | Bazel query timeout in seconds. Large monorepos with deep dependency trees may need a higher value. If queries are timing out, increase this. Setting it too low causes premature failures on valid but slow queries. | @@ -56,10 +56,10 @@ repository: Configures the built-in `memory` or `disk` storage backend. Omit this section if you provide a custom storage implementation. ```yaml -# storage: -# type: "disk" -# disk: -# root_path: "/var/tango/blobs" +storage: + type: "disk" + disk: + root_path: "/var/tango/blobs" ``` | Field | Required | Type | Default | Description | From c8a2e8e7fbb610163f203b7422efc504e6b1f2cb Mon Sep 17 00:00:00 2001 From: Yushan Lin Date: Fri, 17 Jul 2026 10:21:27 -0700 Subject: [PATCH 7/7] Update --- config/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/config/README.md b/config/README.md index 27b04148..d29020c5 100644 --- a/config/README.md +++ b/config/README.md @@ -29,7 +29,7 @@ service: |---|---|---|---|---| | `max_worker_pool_size` | **Yes** | `int` | | Max number of concurrent requests per repository. Each worker is a lightweight local clone (hardlinked to the origin, not a full copy) that handles one request at a time — setting this to 5 means up to 5 concurrent graph computations per repo, and additional requests queue until a worker is free. A good starting point is the expected peak concurrent requests per repo. Must be greater than 0. | | `workspaces_root_path` | **Yes** | `string` | | Root directory where Tango stores repository clones and worker checkouts. Required because Tango needs a known location to clone repositories and create worker checkouts — without it, there is no disk space allocated for graph computation. Choose a location with enough disk space to hold one origin clone plus N worker checkouts (where N is `max_worker_pool_size`) per configured repository. Use a persistent location in production — if this directory is lost, Tango re-clones from the remote on the next request, which can be slow for large repositories. Layout: `//` for origin clones and `/.workers//worker-{1..N}/` for worker checkouts. | -| `max_message_bytes` | No | `int` | `4250000` (~4.25MB) | Caps the size of each gRPC stream message Tango sends (targets, changed targets, and metadata mappings — see `proto/tango.proto`). This is a single byte budget rather than separate per-entry-type counts: Tango derives how many targets, changed targets, or metadata entries fit in one message from fixed size assumptions (a target ≈ 40KB worst case, a changed target ≈ 2x that since it carries both an old and new target, a metadata entry ≈ 85 bytes), computed once rather than measured per request. If a repository has unusually large targets or metadata entries, tune this down to fit its needs. | +| `max_message_bytes` | No | `int` | `4250000` (~4.25MB) | Caps the size of each gRPC stream message Tango sends (targets, changed targets, and metadata mappings — see `proto/tango.proto`). Tango measures the actual serialized size of each message as it's built and stops adding entries once this budget is reached. | ## `repository`