Skip to content

fix(tunnel): isolate frp identities and report relay connection health - #768

Open
Baylem wants to merge 21 commits into
GameplanePanel:masterfrom
Baylem:fix/audit-relay-health
Open

Baylem wants to merge 21 commits into
GameplanePanel:masterfrom
Baylem:fix/audit-relay-health

Conversation

@Baylem

@Baylem Baylem commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Two GameServers using the same template currently collide on frp proxy names, and a live tunnel supervisor can report Ready while the provider is disconnected. This change gives each frp client its GameServer UID identity and makes readiness depend on confirmed relay connection and forward registration.

frp and Playit expose separate liveness, readiness and sanitized status endpoints. Observations expire, child exit clears readiness, and retry escalation resets after one minute of confirmed health. The deterministic capped retry sequence is preserved as requested. Provider Deployments use Recreate so one identity has one active holder, and retain the configurable tunnel resource defaults from #762.

Playit v1.0.10 IPC exposes cached assignments without a connection heartbeat. It therefore reports Unknown after assignment and AwaitingAddress beforehand; it continues running and forwarding without claiming verified connectivity. docs/relay-health.md documents this limitation and the coordinated operator/image upgrade needed for the new frp identity.

Includes #762 (resources), #765 (tsnet forwarding), and #759 (existing web build prerequisite). Review the relay-specific changes after those prerequisites. The embedded Tailscale SDK is v1.104.2 and requires Go 1.27.3; CI, workspace and tunnel builders use that minimum. The management-placement audit bucket remains explicitly covered by upstream change-based test selection.

Expected controller retries use explicit polling delays while genuine errors retain backoff. Mod deletion and its HTTP DELETE manifest update use the same opened directory handle, preventing symlink aliases or directory replacement from redirecting removal. Regression sources cover retry pacing, retained lifecycle fences, link-target preservation and atomic metadata cleanup.

Validation: All 105 CI checks passed on 16ac0e8ae57a706b09125c603d8cd81ba1ed9a6d (main CI run), including lint, unit/integration and E2E suites, coverage gates, builds, code generation, and security checks. Linux amd64 operator/tunnel builds and compilation of unit, controller/envtest and E2E test sources passed. Required actionlint and offline zizmor passed. Tests run in CI under repository policy. Added a real-frps scenario with two servers, TCP/UDP backend checks, rejected registration, credential failure, replacement and outage/recovery. No live Playit or tailnet validation is claimed.

Closes #721
Closes #723

Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 56 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
Commits

Reviewing files that changed from the base of the PR and between a8a935c and cc1b965.

Files selected for processing: 23
  • .github/workflows/ci.yaml
  • docs/relay-health.md
  • docs/tunnels.md
  • i18n/locales/en/app.json
  • operator/hack/condition-ids/main.go
  • operator/internal/controller/gameserver_relay_health_test.go
  • operator/internal/controller/gameserver_status.go
  • operator/internal/controller/gameserver_tunnel.go
  • operator/specs.md
  • test/e2e/optional_components_e2e_test.go
  • test/e2e/relay_health_e2e_test.go
  • test/e2e/specs.md
  • tunnel/Dockerfile.frp
  • tunnel/Dockerfile.playit
  • tunnel/main.go
  • tunnel/main_test.go
  • tunnel/playit_reporter.go
  • tunnel/relay_health.go
  • tunnel/relay_health_test.go
  • tunnel/specs.md
  • tunnel/tailscale.go
  • tunnel/tailscale_retry_test.go
  • tunnel/tailscale_test.go
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Signed-off-by: Baylem <zelathis@gmail.com>
@github-actions github-actions Bot added area: chart Helm chart and deployment area: e2e kind-based end-to-end suite area: operator controller-runtime operator and CRDs area: optional-components sentinel, capture-sidecar, mcp-server, tunnel, audit-syslog-bridge, telemetry-receiver area: shared netguard, gameaction, gameproto, svcutil and other shared packages area: specs spec-kit feature specs and design docs area: web React dashboard type: fix Bug fix labels Oct 21, 2026
Baylem added 13 commits October 21, 2026 17:16
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
@Baylem
Baylem requested a review from ValgulNecron as a code owner October 24, 2026 14:53
@ValgulNecron

Copy link
Copy Markdown
Collaborator

Hey, thanks for this one. The frp identity and relay-health work looks good. Three things before merge:

  1. playit readiness can never pass. relay_health.go:31-32 only reports ready when Connection == "connected", but playitHealthObservation (:256-262) always returns "unknown". With the new /readyz probe on the playit Deployment, the playit pod sits at 0/1 Ready forever. Please make the probe pass for playit (either give it a real connected signal or don't require "connected" for playit).

  2. GAMESERVER_UID is now required (tunnel/main.go:144-145), so a tunnel pod without it exits at startup. Existing frp tunnels created by the old operator don't have it. Please make this part of the upgrade migration: the operator should add GAMESERVER_UID to existing tunnel Deployments when it reconciles them after upgrade, so nothing has to be fixed by hand. A test covering an old-style Deployment getting migrated would be great.

  3. This PR deletes tunnel/tailscale_retry_test.go from fix(tunnel): forward advertised TCP and UDP ports through tsnet #765. The positive "reset after a healthy interval" case is covered again, but nothing checks the negative case anymore: failed tailnet attempts (auth error, denied tag, listener failure) must not reset the retry escalation. Please either add a test for that against the new relayHealth, or keep the old test file.

Thanks!

Signed-off-by: Baylem <zelathis@gmail.com>

Baylem commented Oct 25, 2026

Copy link
Copy Markdown
Member Author

Addressed in dc5d584.

Playit's Kubernetes readiness probe now uses /healthz, so the running wrapper can become Pod Ready. Relay /readyz and TunnelReady still retain the honest unknown connection state because the pinned IPC has no live relay heartbeat; assigned addresses continue to be reported.

Existing frp Deployments already receive GAMESERVER_UID through the normal CreateOrUpdate reconciliation. I added an explicit old-deployment migration regression, including repeated reconciliation, and documented the operator-first/coordinated upgrade path so no manual environment edit is needed.

Restored negative Tailscale retry coverage against relayHealth for authentication errors, denied tags and listener failures, while retaining the positive healthy-interval case. The compatible #762/#765 follow-ups are included here too. Operator and tunnel production/regression sources compile locally; runtime tests remain CI-only.

Signed-off-by: Baylem <zelathis@gmail.com>
@ValgulNecron

Copy link
Copy Markdown
Collaborator

Hey, thanks for restoring the negative retry test. One small thing so it fully covers the real path:

TestTailnetFailedAttemptsCannotResetRetryEscalation runs serveTailnetObserved and relayHealth for real, but the glue between them comes from the test's own copy (observeTailnetRetryHealth) instead of the closure in tunnel/tailscale.go:98-103. If that production closure ever reported a failed attempt as healthy, the test would still pass, because it uses its own correct copy.

Could you move that closure into a small named function in tailscale.go (e.g. tailnetRetryObserver(history, ...)), have runTailscale use it, and have the test call the same function instead of its local copy? Then the test exercises exactly what production runs.

Thanks!

Signed-off-by: Baylem <zelathis@gmail.com>

Baylem commented Oct 27, 2026

Copy link
Copy Markdown
Member Author

Moved the observer into tailnetRetryObserver in tailscale.go. runTailscale and both retry test groups now use that same callback; the tests supply a controlled clock. The failed authentication, denied-tag and listener cases also assert that the recorded relay snapshot is not ready, so an incorrect production health mapping fails the regression directly. The duplicated test-only observer is removed. Operator and tunnel builds and tunnel regression-source compilation pass; runtime tests remain CI-only.

Commit: dbd0710; the shared observer correction is also carried into #771.

Signed-off-by: Baylem <zelathis@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: chart Helm chart and deployment area: e2e kind-based end-to-end suite area: operator controller-runtime operator and CRDs area: optional-components sentinel, capture-sidecar, mcp-server, tunnel, audit-syslog-bridge, telemetry-receiver area: shared netguard, gameaction, gameproto, svcutil and other shared packages area: specs spec-kit feature specs and design docs area: web React dashboard type: fix Bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(tunnel): report relay connection health and reserve tunnel resources fix(tunnel): give frp proxies unique identities across GameServers

2 participants