Add Bazel PyPI manifest extraction

CHANGELOG.md

@@ -9,6 +9,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 ### Added
 - **`socket manifest bazel [beta]`** — Generate Bazel JVM SBOM manifests by running `bazel query` against discovered Maven repos in a Bazel workspace. Closes the inline-Maven-declaration gap that lockfile-only parsing misses for repos like envoy, ray, tensorflow, tink-java, and or-tools. Auto-detects Bzlmod and legacy `WORKSPACE`.
 - **`socket scan create --auto-manifest`** now covers Bazel workspaces in addition to Gradle/Scala/Kotlin/Conda. Repos with `MODULE.bazel`, `WORKSPACE`, or `WORKSPACE.bazel` are detected automatically and their Maven dependencies extracted as part of the standard scan-create flow.
+- **Bazel PyPI extraction** — `socket manifest bazel --ecosystem pypi` now generates `requirements.txt` for Python Bazel workspaces. Discovers custom `rules_python` pip hub names with Bazel command output first, queries `py_library` / `py_binary` / `py_test` dependencies, resolves canonical pinned versions from `requirements_lock.txt`, and emits PEP 503-normalized `name==version` lines. Supports both Bzlmod (`pip.parse`) and legacy `WORKSPACE` (`pip_parse` / `pip_install`) configurations. PyPI remains explicit opt-in for `socket scan create --auto-manifest` until real-world no-lockfile recovery is validated.
+
+### Changed
+- **Bazel diagnostics** — `socket manifest bazel --verbose` now emits bounded subprocess traces with argv, cwd, duration, exit status, output sizes, and failure stderr tails to make customer log-only triage safer and faster.
 
 ## [1.1.101](https://github.com/SocketDev/socket-cli/releases/tag/v1.1.101) - 2026-05-22
 

src/commands/manifest/README.md

@@ -16,13 +16,14 @@ manifest generator. Useful when you do not want to spell out the language.
 
 ## socket manifest bazel [beta]
 
-Generates Bazel JVM SBOM manifests (`maven_install.json`-shaped) by running
-`bazel query` against discovered Maven repos in a Bazel workspace. Output is
-consumed by `socket scan create` and closes the
-inline-Maven-declaration gap that lockfile-only parsing misses.
+Generates Bazel SBOM manifests (Maven `maven_install.json` and/or PyPI
+`requirements.txt`) by running `bazel query` against discovered ecosystem
+hubs in a Bazel workspace. Output is consumed by `socket scan create` and
+closes the inline-declaration gap that lockfile-only parsing misses for
+Bazel monorepos.
 
-> **Note**: This command generates Maven dependency manifests for Bazel JVM
-> workspaces. It does not run reachability analysis.
+> **Note**: This command generates dependency manifests for Bazel
+> workspaces (Maven and PyPI). It does not run reachability analysis.
 
 ### Usage
 
@@ -36,33 +37,98 @@ socket manifest bazel [options] [DIR=.]
 - `--bazel-rc <path>` — path to additional `.bazelrc` fragments forwarded to bazel.
 - `--bazel-flags <str>` — flags forwarded to every bazel invocation (single quoted string).
 - `--bazel-output-base <dir>` — Bazel `--output_base` for read-only-cache CI environments.
+- `--ecosystem <name>` — ecosystem(s) to extract; repeatable. Supported values: `maven`, `pypi`. When omitted, Maven is generated by default; PyPI is explicit opt-in.
 - `--out <dir>` — output directory; default `./.socket/bazel-manifests/`.
 - `--dry-run`, `--verbose` — standard diagnostic flags.
 
 > **Upload**: This subcommand only generates manifests. To generate and
 > upload in one step, use `socket scan create --auto-manifest .` — it
-> detects the workspace, runs the same extraction this subcommand performs,
-> and uploads the result.
+> detects the workspace, generates Bazel Maven manifests, and uploads the
+> result. Generate Bazel PyPI manifests explicitly with `socket manifest bazel
+> --ecosystem pypi`, then scan the generated output with `socket scan create`.
 
 ### Examples
 
 ```bash
-# Generate maven manifests from the current Bazel workspace.
+# Generate the default Bazel Maven manifest from the current workspace.
 socket manifest bazel .
 
+# Generate only the PyPI manifest.
+socket manifest bazel . --ecosystem pypi
+
+# Generate both Maven and PyPI manifests explicitly.
+socket manifest bazel . --ecosystem maven --ecosystem pypi
+
 # Use bazelisk explicitly.
 socket manifest bazel --bazel=/usr/local/bin/bazelisk .
 ```
 
+### Python/PyPI Extraction
+
+When `--ecosystem pypi` is selected, the command:
+
+1. Discovers `rules_python` pip hubs from Bazel's `mod show_extension` output when available, with bounded static parsing of `MODULE.bazel` (`pip.parse(hub_name = "...")`) and legacy `WORKSPACE` (`pip_parse(name = "...")` / `pip_install(name = "...")`) retained as fallback. Hub names are never hardcoded; custom names like `my_pypi` are detected automatically.
+2. Validates each candidate hub by probing it with `bazel query` for `:pkg` targets / `alias(` rules. Invalid candidates are dropped.
+3. Runs `bazel query 'deps(kind("py_library|py_binary|py_test", //...))'` to determine which PyPI packages are actually reached by Python rules in the repo (test dependencies included for whole-repo scope).
+4. Reads `requirements_lock.txt` (the path discovered from `pip.parse(requirements_lock = "...")`) for canonical pinned versions. When the lockfile is unavailable, falls back to parsing `pypi_name=` and `pypi_version=` tags from the spoke `py_library` rules in the hub-and-spoke architecture.
+5. Emits a sorted canonical `requirements.txt` containing `name==version` lines for every reached package.
+
+### PyPI Name and Version Semantics
+
+- **PEP 503 normalization.** Package matching uses PEP 503 normalization
+  (lowercase, then any run of `-`, `_`, or `.` is collapsed to a single
+  `-`). Bazel target names use underscores (`charset_normalizer`); PyPI
+  canonical names use hyphens (`charset-normalizer`). The emitted
+  `requirements.txt` always uses the canonical hyphenated form.
+- **Lockfile pins win.** When the lockfile and spoke-repo tags disagree on
+  a version, the lockfile wins because that is the version Bazel actually
+  resolves at analysis time. A `--verbose` warning is logged for the
+  divergence.
+- **Conflict detection.** When two reached packages normalize to the same
+  PyPI name with different versions, the command fails clearly: a single
+  `requirements.txt` cannot represent both versions, and silently
+  picking one would produce a misleading SBOM.
+
+### Unsupported PyPI Forms
+
+The PyPI extractor is intentionally narrow in this phase:
+
+- **Direct URL, editable (`-e`), and unpinned requirements** are not
+  emitted. Only canonical `name==version` lines from the resolved
+  lockfile are produced. Repositories that rely on unpinned or
+  URL-pinned requirements will see those packages omitted from
+  `requirements.txt`.
+- **Private corpus validation** requires authenticated GitHub access.
+  When credentials are unavailable, the bazel-bench harness's private
+  PyPI case skips cleanly with a distinct reason rather than failing.
+- **Whole-repo extraction.** The initial PyPI implementation emits one
+  whole-workspace manifest. Per-target PyPI slicing is not currently
+  supported.
+
+### Cross-Language Edges
+
+Bazel repos with cross-language dependencies (e.g. `rust_library` →
+`py_library` via PyO3 / cffi / etc.) are **not** traversed by the PyPI
+extractor in this phase. The PyPI extractor only covers Python rule
+dependencies reachable from `py_library`, `py_binary`, and `py_test`
+targets. Cross-language edges are assigned to Phase 4. The bazel-bench
+fixture `constructed/python-pypi` includes Go/Rust sidecars as
+validation context only; they are intentionally not asserted by the
+PyPI correctness cases.
+
 ### Requirements
 
 - `bazel` or `bazelisk` on `PATH` (or pass `--bazel <path>`).
-- Network access on cold cache. Bazel and `rules_jvm_external` own their own
-  retry policy for transient Maven resolution failures — `socket manifest bazel`
-  does not retry on top of them.
+- Network access on cold cache. Bazel and `rules_jvm_external` /
+  `rules_python` own their own retry policy for transient resolution
+  failures — `socket manifest bazel` does not retry on top of them.
 - Writable Bazel output base; pass `--bazel-output-base` for read-only-cache CI.
+- For PyPI extraction: a Python 3 interpreter on `PATH` so the
+  rules_python toolchain can analyze the workspace.
 
-This is the user-visible entry point for Bazel JVM SBOM support; the [beta] label and "Bazel JVM SBOM support" wording must stay consistent across release notes and docs.
+This is the user-visible entry point for Bazel SBOM support (Maven and
+PyPI); the [beta] label and "Bazel SBOM support" wording must stay
+consistent across release notes and docs.
 
 ## socket manifest cdxgen