You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(transport): drive a real headless Chrome via chromedp (XActions-style)
User pulled the v0.3+utls binary, ran it, and the Cloudflare HTML
challenge body was still in the error. utls only fakes the TLS
handshake — Cloudflare also inspects HTTP/2 frame ordering, JS
challenges, navigator fingerprinting, etc. Faking a piece of Chrome
was never going to be enough against modern bot management. Stop
fighting it. Be Chrome.
This commit ports XActions' Puppeteer approach to Go via chromedp.
Real headless Chrome instance, real TLS, real HTTP/2, real cookies,
real everything. Cloudflare can't tell us apart from the user's daily
driver because we ARE Chrome.
internal/chromebrowser/browser.go (new):
Browser type wraps a chromedp ExecAllocator + browser context.
Lazy-started on the first Fetch; Close() releases the Chrome
process. Stealth flags applied at launch:
--headless=new (Chrome 109+ headless v2)
--disable-blink-features=AutomationControlled
--disable-features=AutomationControlled,IsolateOrigins,site-per-process
--no-default-browser-check
--no-first-run
--user-agent=<Chrome 120 macOS>
--window-size=1280,800
Fetch(ctx, method, url, headers, cookies) sets cookies via
Network.SetCookies (CDP), navigates to https://x.com/robots.txt
to establish an x.com origin, then runs `fetch()` inside the page
context with credentials:'include'. Returns the HTTP status and
raw body bytes. Cloudflare clearance cookies are issued by the
navigation as a side effect, so the subsequent fetch inherits
them automatically.
internal/chromebrowser/transport.go (new):
Transport implements http.RoundTripper around Browser. Drop into
http.Client.Transport and the rest of x-cli's HTTP code (retries,
throttle, Set-Cookie merging, error parsing) keeps working
unchanged. Cookie header is split out and pushed to the browser
via CDP; everything else flows through fetch options.
api/client.go:
New Options.UseBrowser flag. When true and no HTTPClient was
provided, the default client is wired with chromebrowser.NewTransport()
and a 60s timeout (browser startup is ~1-2s on first call). When
false, falls back to the existing tlsprint http+utls path —
preserved so users without Chrome can still use x-cli with
reduced reliability.
cmd/root.go:
--http persistent flag forces the http+utls path. Default is the
browser path. newClient() propagates UseBrowser: !useHTTP into
api.Options.
cmd/auth.go:
runAuthImport now defaults to the browser path. The "verifying
session" log line tells the user which transport is in use so
they can debug. 60s context (vs the old 20s) accounts for Chrome
startup. The error message points at "Chrome could not start" as
one possible failure mode.
.github/workflows/ci.yml:
Go bumped 1.24 → 1.26 (chromedp's go.mod requires it).
go.mod:
+ github.com/chromedp/chromedp v0.15.1
+ github.com/chromedp/cdproto (latest)
Plus pure-Go indirect deps (chromedp/sysutil, gobwas/ws,
go-json-experiment).
How it works end to end:
1. `x auth import` reads cookies from the user's Chrome via kooky
(our existing browsercookies path, unchanged).
2. The Client constructs a chromebrowser.Transport. First call
spawns headless Chrome (~1-2s).
3. VerifyCredentials calls UserByRestId via the transport. The
RoundTrip pushes cookies into the headless Chrome via
Network.SetCookies, navigates to x.com/robots.txt to get
Cloudflare clearance, then runs fetch() in the x.com origin
context. The response comes back as JSON.
4. x-cli parses the JSON same as before — the api/ projection
code is transport-agnostic.
Trade-offs:
+ Cloudflare doesn't see us as a bot because we're not. Same
reason XActions' Puppeteer path works.
+ No more chasing fingerprints: TLS, HTTP/2, JS challenges, all
handled by Chrome itself.
+ The existing http+utls path stays available via --http for
users without Chrome (Linux servers, CI runners).
- Requires Chrome installed on the user's machine. macOS and
Windows: typical. Linux desktop: typical. Headless Linux:
user has to install google-chrome-stable or chromium.
- First call pays ~1-2s Chrome startup. Subsequent calls in the
same process are ~200-500ms.
- Adds ~10MB of pure-Go deps (chromedp + cdproto). Binary goes
from ~11MB to ~14MB.
If `--http` is needed (no Chrome installed), x-cli falls back to
the utls path with all the caveats it has. The user can decide
which trade-off matters more for their environment.
Copy file name to clipboardExpand all lines: cmd/root.go
+4-1Lines changed: 4 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -20,6 +20,7 @@ var (
20
20
endpointsFilestring
21
21
jsonOutbool
22
22
verbosebool
23
+
useHTTPbool
23
24
)
24
25
25
26
varrootCmd=&cobra.Command{
@@ -53,6 +54,7 @@ func init() {
53
54
rootCmd.PersistentFlags().StringVar(&endpointsFile, "endpoints", "", "endpoints.yaml (default $HOME/.config/x-cli/endpoints.yaml, then ./endpoints.yaml)")
54
55
rootCmd.PersistentFlags().BoolVar(&jsonOut, "json", false, "emit machine-readable JSON on stdout")
55
56
rootCmd.PersistentFlags().BoolVarP(&verbose, "verbose", "v", false, "verbose logging to stderr")
57
+
rootCmd.PersistentFlags().BoolVar(&useHTTP, "http", false, "force the http+utls transport (skip the headless Chrome path; needed when Chrome is not installed)")
0 commit comments