Add retract-phantom.sh for proxy cache retraction of phantom module paths

[alex-dna-tech]

Summary

Adds retract-phantom.sh — a bash script that retracts phantom module paths cached by the Go proxy that don't exist on master.

Problem

The Go proxy has cached module paths like go-micro.dev/v5/api/handler, go-micro.dev/v4/a, etc. as separate modules. When users run go install go-micro.dev/v5/api/handler@latest, they get:

go: go-micro.dev/v5/api/handler@latest: version constraints conflict:
    go-micro.dev/v5/api/handler@v1.18.0: parsing go.mod:
    module declares its path as: github.com/micro/go-micro
            but was required as: go-micro.dev/v5/api/handler

Solution

Tag-only retraction using orphan commits:

1. Creates empty orphan commits with retract [v0.0.0, v1.18.2] directives
2. Tags each phantom path (e.g., api/handler/v1.18.2)
3. Pushes tags to origin — master is never touched

After retraction, go install ...@latest falls back to the root module.

Phantom Paths Covered

Base	Paths
v6	api
v5	api/api, api/gateway, api/genproto, api/handler, api/handler/http
v4	a, a/codec/proto, a/transport/memory

Usage

./retract-phantom.sh [--dry-run] [--pilot] [--push]

Flag	What it does
(none)	Create all 10,211 phantom tags locally (no push).
--dry-run	Print what would happen; touches nothing.
--pilot	Process only the first path per base (3 tags total). Validate the mechanism cheaply.
--push	Create tags and push them to origin.

Typical flow:
./retract-phantom.sh --dry-run # sanity-check the enumeration
./retract-phantom.sh --pilot --push # tiny live test (3 tags)
./retract-phantom.sh --push # full run

What you need to know

• Idempotent. Re-running skips already-existing tags, so a partial/failed run can just be
  re-run.
• Invalid names are skipped. A path whose tag name fails git check-ref-format (e.g. the old
  protoc-gen-micro~) is logged and skipped — it can never be retracted by this mechanism.
• Push target is origin. Right now origin is the alex-dna-tech fork, so the tags have no effect
  on the go-micro.dev proxy (that reads micro/go-micro/upstream). To actually retract on the
  live proxy, point the push at upstream — either git remote set-url origin first, or
  add a --remote flag.
• Cross-base collisions. Paths sharing a subpath across v4/v5/v6 collapse to one tag (e.g.
  v4/api, v5/api, v6/api → api/v1.18.2), so for ~316 such paths the go.mod declares the
  last-processed base's module name. The tag exists and blocks use, but it's not a clean retract
  for those two bases.
• Mechanism. Each tag /v1.18.2 points to an orphan commit holding that path's go.mod,
  which retract [v0.0.0, v1.18.2] (self-retracting). Verify with go list -m -u / go list
  -m -versions once the tags are on the proxy-backed repo.

Verification

After push:

go list -m -versions go-micro.dev/v5/api/handler  # → no versions
go install go-micro.dev/v5/api/handler@latest     # → falls back to root module

References

• Go module retraction docs
• pkg.go.dev/go-micro.dev/v6
• pkg.go.dev/go-micro.dev/v5
• pkg.go.dev/go-micro.dev/v4

[alex-dna-tech]

1 path skipped: go-micro.dev/v4/cmd/protoc-gen-micro~ — git forbids ~ in tag names, so it can never be retracted via this mechanism. Require manual fix but with different tag name.

go-micro$ go list -m -versions go-micro.dev/v4/cmd/protoc-gen-micro~
go-micro.dev/v4/cmd/protoc-gen-micro~ v0.1.0 v0.1.1 v0.1.2 v0.1.3 v0.1.4 v0.2.0 v0.3.0 v0.4.0 v0.5.0 v0.6.0 v0.7.0 v0.8.0 v0.9.0 v0.10.0 v0.11.0 v0.12.0 v0.13.0 v0.14.0 v0.14.1 v0.15.0 v0.15.1 v0.16.0 v0.17.0 v0.20.0 v0.21.0 v0.22.0 v0.22.1 v0.23.0 v0.24.0 v0.24.1 v0.25.0 v0.26.0 v0.26.1 v0.27.0 v0.27.1 v1.0.0 v1.1.0 v1.2.0 v1.3.0 v1.3.1 v1.4.0 v1.5.0 v1.6.0 v1.7.0 v1.8.0 v1.8.1 v1.8.2 v1.8.3 v1.9.0 v1.9.1 v1.10.0 v1.11.0 v1.11.1 v1.11.2 v1.11.3 v1.12.0 v1.13.0 v1.13.1 v1.13.2 v1.14.0 v1.15.0 v1.15.1 v1.16.0 v1.17.0 v1.17.1 v1.18.0

[alex-dna-tech]

I have tested the "--push" command on my fork. Tags · alex-dna-tech/go-micro.

[alex-dna-tech]

By the way, I tried the seven-year-old project EwanValentine/shippy-service-vessel which uses the github.com/micro/go-micro@v1.18.0 one of the micro package importedby - github.com/micro/go-micro - Go Packages

~$ git clone https://github.com/EwanValentine/shippy-service-vessel.git
~$ cd shippy-service-vessel/
shippy-service-vessel$ cat go.mod
module github.com/EwanValentine/shippy-service-vessel

go 1.13

require (
	github.com/DataDog/zstd v1.4.4 // indirect
	github.com/golang/protobuf v1.3.2
	github.com/micro/go-micro v1.18.0
	github.com/xdg/scram v0.0.0-20180814205039-7eeb5667e42c // indirect
	github.com/xdg/stringprep v1.0.0 // indirect
	go.mongodb.org/mongo-driver v1.2.1
	golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553
)
shippy-service-vessel$
shippy-service-vessel$ go list
go: github.com/micro/go-micro@v1.18.0 requires
	github.com/micro/protoc-gen-micro@v1.0.0: reading github.com/micro/protoc-gen-micro/go.mod at revision v1.0.0: git ls-remote -q --end-of-options origin in ~/go/pkg/mod/cache/vcs/28cfbca4f5e41691076780b2017d5a1000b007b1d3d37b7a9d3b562c7adc9377: exit status 128:
	remote: Repository not found.
	fatal: repository 'https://github.com/micro/protoc-gen-micro/' not found

After replacing the package with the next version, everything worked fine.

replace github.com/micro/protoc-gen-micro => github.com/micro/go-micro/cmd/protoc-gen-micro/v2 v2.0.0-20210105173217-bf4ab679e18b

[asim]

@alex-dna-tech just waiting for you to give the signoff for when this PR is complete and any follow up required after merge

[alex-dna-tech]

@asim Signoff: complete. No need to merge — it's a one-time op tool. How to run it straight from the PR without touching master, plus the one manual edge case:

Run it without merging

You don't need a branch. The script never commits to one — it builds orphan commits via git commit-tree and tags them, so master and your working tree stay untouched.

# from a checkout of micro/go-micro (origin = upstream)
git fetch origin pull/4856/head
git checkout FETCH_HEAD -- retract-phantom.sh
chmod +x retract-phantom.sh

./retract-phantom.sh --dry-run        # sanity-check the ~10K enumeration
./retract-phantom.sh --pilot --push   # 3 tags, live test on the proxy
# verify from a machine that hits the proxy:
go list -m -versions go-micro.dev/v6/api   # → no versions
go install go-micro.dev/v6/api@latest        # → falls back to root module
./retract-phantom.sh --push            # full run

git checkout -- retract-phantom.sh      # drop the script when done

(Alt: gh pr checkout 4856, run, then git checkout master && git branch -D 4856.)

The script does git push origin — for you origin is upstream, so tags land on the live proxy.

The one path the script cannot handle ~ path — manual fix

One phantom path contains a ~. Git rejects ~ in ref names, so the script skips it — and you can't hand-create the tag …~/v1.18.2 either, for the same reason.

- go-micro.dev/v4/cmd/protoc-gen-micro~

It is real and cached — go list -m -versions returns:

go-micro.dev/v4/cmd/protoc-gen-micro~ v0.1.0 … v1.18.0   (62 versions)

It is the only one phantom path out of ~10K whose name contains a ~, and that
single character is why retract-phantom.sh skips it.

Because the path is already keyed to root tags, we publish the retraction as a new root tag (a valid ref name, no ~, no submodule prefix). A root tag v1.18.1 becomes the new @latest for the submodule, and its retract directive removes every v0.1.0–v1.18.0.

# run from a checkout of micro/go-micro (origin = upstream)
git checkout --orphan retract-tilde
git rm -rf . >/dev/null 2>&1

cat > go.mod <<'EOF'
module go-micro.dev/v4/cmd/protoc-gen-micro~

go 1.24

// Phantom path (tilde). The natural retraction tag cmd/protoc-gen-micro~/v1.18.2
// is an invalid git ref name, so we retract via a root tag instead — which is
// how this path is already keyed (its versions are the repo's v1.x root tags).
retract [v0.0.0, v1.18.0]
EOF

git add go.mod
git commit -m "retract go-micro.dev/v4/cmd/protoc-gen-micro~ (phantom, tilde)"
git tag v1.18.1 HEAD
git push origin v1.18.1

git checkout master
git branch -D retract-tilde

Verify (from a machine that hits the proxy):

go list -m -versions go-micro.dev/v4/cmd/protoc-gen-micro~   # → only v1.18.1, retracted
go install go-micro.dev/v4/cmd/protoc-gen-micro~@latest      # → no usable version / falls back

Caveat — this pollutes the root module's version list

v1.18.1 is a root tag, so it is also seen as a version of the root module go-micro.dev/v4. Its go.mod declares module go-micro.dev/v4/cmd/protoc-gen-micro~ (the wrong path for the root module).

• go-micro.dev/v4@latest is unaffected — the real root @latest is v4.x.y,
  which sorts higher than v1.18.1.
• But go list -m -versions go-micro.dev/v4 will now show a spurious v1.18.1,
  and anyone who explicitly pins go-micro.dev/v4@v1.18.1 gets a broken module.

[chatgpt-codex-connector]

To use Codex here, create a Codex account and connect to github.