Add codemap-full release and blast radius bundle#70
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds a new “full” release artifact that bundles codemap with sibling ast-grep/sg binaries, updates dependency scanning to prefer those bundled binaries, and introduces JSON output + a “blast radius” wrapper script to generate compact PR-review context.
Changes:
- Add a
codemap-fullGoReleaser archive and a pre-build download step to fetch pinnedast-greprelease assets. - Prefer bundled sibling
ast-grep/sgbinaries before falling back toPATH, and standardize the “not found” error handling. - Add JSON output for
codemap --importers, plus a boundedscripts/codemap-blast-radius.shwrapper and README documentation.
Reviewed changes
Copilot reviewed 11 out of 13 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
scripts/download-bundled-astgrep.sh |
New downloader/extractor for pinned ast-grep release zips into bundled-tools/. |
scripts/codemap-blast-radius.sh |
New wrapper to emit bounded diff/deps/importers context in Markdown/Text/JSON. |
scanner/walker.go |
Switch missing-ast-grep error to shared sentinel error. |
scanner/types.go |
Add ImportersReport struct for --importers JSON output. |
scanner/astgrep.go |
Add bundled-sibling lookup + verification of sg/ast-grep candidates; define ErrAstGrepNotFound. |
scanner/astgrep_test.go |
Add tests for rejecting non-ast-grep sg and bundled candidate behavior. |
README.md |
Document tarball vs codemap-full install paths and the blast-radius workflow. |
main.go |
Add JSON output path for importers mode; improve deps-mode guidance when ast-grep is missing. |
main_more_test.go |
Update importers test call signature and add coverage for --json --importers. |
go.mod |
Promote gopkg.in/yaml.v3 to a direct dependency. |
go.sum |
Update sums for module graph changes. |
.goreleaser.yml |
Add “full” archive and download hook; ensure package-manager outputs stay on default archive. |
.gitignore |
Ignore generated bundled-tools/ directory. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "linux arm64 app-aarch64-unknown-linux-gnu.zip" | ||
| "windows amd64 app-x86_64-pc-windows-msvc.zip" | ||
| ) | ||
|
|
There was a problem hiding this comment.
rm -rf "$output_dir" can be dangerously destructive if --output-dir is set to /, ., or an empty string (e.g., via CI env/arg mistakes). Add a safety guard that rejects empty///. paths (and ideally requires the output directory to be within the repo) before deleting, or delete only the expected per-target subdirectories.
| case "$output_dir" in | |
| ""|"/"|".") | |
| echo "Refusing to delete unsafe output directory: $output_dir" >&2 | |
| exit 1 | |
| ;; | |
| esac | |
| script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | |
| repo_root="$(cd "$script_dir/.." && pwd)" | |
| repo_root_abs="$(python3 -c 'import os, sys; print(os.path.realpath(sys.argv[1]))' "$repo_root")" | |
| output_dir_abs="$(python3 -c 'import os, sys; print(os.path.realpath(sys.argv[1]))' "$output_dir")" | |
| if [[ "$output_dir_abs" == "$repo_root_abs" ]]; then | |
| echo "Refusing to delete repository root: $output_dir" >&2 | |
| exit 1 | |
| fi | |
| case "$output_dir_abs" in | |
| "$repo_root_abs"/*) | |
| ;; | |
| *) | |
| echo "Refusing to delete output directory outside repository: $output_dir" >&2 | |
| exit 1 | |
| ;; | |
| esac |
There was a problem hiding this comment.
Addressed in 9de3a07. The downloader no longer deletes the whole output root; it now rejects unsafe base paths ("", /, .) and only removes the expected per-target subdirectories before recreating them.
scripts/download-bundled-astgrep.sh
Outdated
| import urllib.request | ||
| import zipfile | ||
|
|
||
| version, asset, dest = sys.argv[1:] | ||
| url = f"https://github.com/ast-grep/ast-grep/releases/download/{version}/{asset}" | ||
| print(f"Downloading {url}", file=sys.stderr) | ||
|
|
||
| with urllib.request.urlopen(url) as response: | ||
| data = response.read() |
There was a problem hiding this comment.
The Python downloader uses urllib.request.urlopen(url) without a timeout, which can hang a release build indefinitely on network issues. Pass an explicit timeout (and optionally add a simple retry/backoff) so CI/release runs fail fast and predictably.
| import urllib.request | |
| import zipfile | |
| version, asset, dest = sys.argv[1:] | |
| url = f"https://github.com/ast-grep/ast-grep/releases/download/{version}/{asset}" | |
| print(f"Downloading {url}", file=sys.stderr) | |
| with urllib.request.urlopen(url) as response: | |
| data = response.read() | |
| import time | |
| import urllib.error | |
| import urllib.request | |
| import zipfile | |
| DOWNLOAD_TIMEOUT_SECONDS = 30 | |
| DOWNLOAD_RETRIES = 3 | |
| RETRY_BACKOFF_SECONDS = 2 | |
| version, asset, dest = sys.argv[1:] | |
| url = f"https://github.com/ast-grep/ast-grep/releases/download/{version}/{asset}" | |
| print(f"Downloading {url}", file=sys.stderr) | |
| last_error = None | |
| for attempt in range(1, DOWNLOAD_RETRIES + 1): | |
| try: | |
| with urllib.request.urlopen(url, timeout=DOWNLOAD_TIMEOUT_SECONDS) as response: | |
| data = response.read() | |
| break | |
| except (TimeoutError, urllib.error.URLError, OSError) as exc: | |
| last_error = exc | |
| if attempt == DOWNLOAD_RETRIES: | |
| raise | |
| print( | |
| f"Download attempt {attempt} failed for {url}: {exc}. " | |
| f"Retrying in {RETRY_BACKOFF_SECONDS}s...", | |
| file=sys.stderr, | |
| ) | |
| time.sleep(RETRY_BACKOFF_SECONDS) | |
| else: | |
| raise last_error |
There was a problem hiding this comment.
Addressed in 9de3a07. The embedded Python downloader now uses an explicit timeout, retries with backoff, and fails with a clear error after the retry budget is exhausted.
.goreleaser.yml
Outdated
| before: | ||
| hooks: | ||
| - go mod tidy | ||
| - ./scripts/download-bundled-astgrep.sh |
There was a problem hiding this comment.
Adding ./scripts/download-bundled-astgrep.sh to before.hooks makes goreleaser runs depend on Python + outbound network access. Consider making this hook conditional (e.g., env-gated) or moving the download step into a dedicated CI job that prepares artifacts, so local/offline goreleaser runs remain reproducible and don't unexpectedly fail.
| - ./scripts/download-bundled-astgrep.sh | |
| - sh -c 'if [ "${DOWNLOAD_BUNDLED_ASTGREP:-0}" = "1" ]; then ./scripts/download-bundled-astgrep.sh; else echo "Skipping bundled ast-grep download (set DOWNLOAD_BUNDLED_ASTGREP=1 to enable)"; fi' |
There was a problem hiding this comment.
Addressed in 9de3a07, but with a slightly different shape. The networked codemap-full path now lives in a dedicated release-only config (.goreleaser.full.yml) and the GitHub Actions release workflow explicitly uses that config. The default .goreleaser.yml is back to the standard offline/local release path, so local goreleaser runs no longer require Python or outbound network access.
.goreleaser.yml
Outdated
| - id: full | ||
| ids: | ||
| - codemap | ||
| name_template: '{{ .ProjectName }}-full_{{ .Version }}_{{ .Os }}_{{ .Arch }}{{ with .Arm }}v{{ . }}{{ end }}{{ with .Mips }}_{{ . }}{{ end }}{{ if not (eq .Amd64 "v1") }}{{ .Amd64 }}{{ end }}' | ||
| formats: | ||
| - tar.gz | ||
| format_overrides: | ||
| - goos: windows | ||
| formats: | ||
| - zip | ||
| files: | ||
| - src: bundled-tools/{{ .Os }}_{{ .Arch }}/ast-grep{{ if eq .Os "windows" }}.exe{{ end }} | ||
| strip_parent: true | ||
| - src: bundled-tools/{{ .Os }}_{{ .Arch }}/sg{{ if eq .Os "windows" }}.exe{{ end }} | ||
| strip_parent: true |
There was a problem hiding this comment.
The full archive currently only adds ast-grep/sg to the bundle; it does not include README.md or LICENSE* like the default archive does. If these files are intentionally included in release artifacts (and for license distribution consistency), add them to the full archive files: list as well.
There was a problem hiding this comment.
Addressed in 9de3a07. The codemap-full archive now includes README.md, LICENSE*, and the bundled sg-rules alongside codemap, ast-grep, and sg for consistency with the standard archive.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 61882742eb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
scripts/download-bundled-astgrep.sh
Outdated
| with urllib.request.urlopen(url) as response: | ||
| data = response.read() |
There was a problem hiding this comment.
The release hook downloads executable archives and immediately repackages them into codemap-full without any checksum or signature verification. If GitHub traffic is intercepted, a mirror/proxy is compromised, or the upstream asset is replaced, this pipeline will ship a tampered ast-grep/sg binary to users. Please verify each downloaded asset against pinned expected hashes (or signed checksums) before extraction.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in 9de3a07. The downloader now verifies each ast-grep asset against a pinned SHA-256 before extraction, so the codemap-full pipeline no longer republishes unverified downloaded executables.
Summary
codemap-fullrelease archive that bundlescodemap,ast-grep, andsgPATHcodemap --importersscripts/codemap-blast-radius.shwrapper for diff/deps/importers review contextWhy
This makes the direct tarball/CI install path work for
--depswithout relying on Homebrew or a separate runtime install, and it gives PR-review workflows a compact blast-radius bundle that adds impact context without sending the whole repo to an LLM.Verification
go test ./...bash -n scripts/codemap-blast-radius.shbash -n scripts/download-bundled-astgrep.shgoreleaser checkgoreleaser release --snapshot --clean --skip=publishcodemap, place bundledast-grepsiblings next to it, clearPATHof ast-grep, confirmcodemap --depsstill works on a temp repoNotes
codemap-fullis an additional release assetgo mod tidypromotedgopkg.in/yaml.v3to a direct dependency because it is already imported byskills/loader.go