Telemetry runtime context keeps a stale installationId after the gateway token changes

[MicroMilo]

Summary

resolveTelemetryRuntimeContext caches the runtime context by a key that does not include the gateway token path, token mtime, or token content. However, installationId is derived from the gateway token file contents. In the same process, if the token file changes after the first call, subsequent calls with the same pilotHome / env return the cached installationId for the old token.

This is a confirmed runtime behavior. Whether it is a user-facing bug depends on the intended product semantics: if the gateway token can be rotated or regenerated during a process lifetime, the telemetry identity can become stale. If the token is intentionally immutable for the process lifetime, that invariant should probably be documented and covered by a test.

Code path

Cache / identity anchors:

• src/telemetry/context.ts:17 defines runtimeContextCache.
• src/telemetry/context.ts:23-26 builds the cache key and returns cached context before recomputing identity.
• src/telemetry/context.ts:30 stores installationId from resolveInstallationId(...) in the cached context.
• src/telemetry/context.ts:45-49 derives installationId from the gateway token file contents when the token exists.
• src/telemetry/context.ts:162-172 builds the cache key from pilotHome, commit/version env values, process.execPath, and process.argv[1], but not from the token path or token contents.

Steps to reproduce

Validation level: dynamically reproduced in one Node process using the real resolveTelemetryRuntimeContext, resolveGatewayTokenPath, and hashTelemetryId functions with a temporary pilotHome.

1. Create a temporary pilotHome.
2. Write server-token with token-A.
3. Call resolveTelemetryRuntimeContext({ pilotHome, env }).
4. Replace server-token with token-B.
5. Call resolveTelemetryRuntimeContext({ pilotHome, env }) again in the same process.

Observed result:

{
  "firstMatchesTokenA": true,
  "secondStillMatchesTokenA": true,
  "secondMatchesTokenB": false,
  "staleCacheObserved": true
}

Expected behavior

If runtime token changes are supported, the second call should either recompute installationId from token-B or invalidate the cached runtime context when the token changes.

If runtime token changes are not supported, the code should make that invariant explicit with a test or documentation so callers do not expect token rotation to update telemetry identity in-process.

Actual behavior

The second call returns the cached context created from token-A, even after the token file contains token-B.

Existing coverage

I checked current open and closed issues/PRs for runtimeContextCache token installationId, resolveTelemetryRuntimeContext token, buildRuntimeContextCacheKey installationId, and adjacent gateway/token/telemetry titles. Existing token/gateway issues appear to cover different root causes, such as WebSocket reconnect behavior or token exposure, not this telemetry cache key behavior.

Suggested fix

If token rotation should be reflected in telemetry identity, include token identity in the cache invalidation boundary, such as token path plus mtime or a token hash, or clear runtimeContextCache from the token rotation path.

If token rotation is intentionally out of scope for a running process, add a test that documents the invariant and consider a short comment near buildRuntimeContextCacheKey.

Suggested tests

• Add a same-process test with token A -> B and assert the intended installationId behavior.
• If recomputation is intended, assert that the second call matches hashTelemetryId("token-B").
• If cache stability is intended, assert that runtime token changes are ignored by design and document why.

---

Submitted with Codex.

[Gucc111]

Thanks for the detailed write-up. We reviewed the code paths and concluded this is by-design, not a bug.

The gateway token is process-lifetime-immutable by design

The server-token is read once at startup across the entire system — not just in telemetry:

• GatewayServer reads the token once via ensureGatewayAuthToken() at startup and holds it in a closure variable (auth.token) for all subsequent HTTP/WebSocket auth. It never re-reads the file.
• TelemetryCollector resolves runtimeContext once via resolveTelemetryRuntimeContext() at creation time and captures it in a closure for all subsequent event emission. It also never re-reads.

So even if we added token identity to the cache key in buildRuntimeContextCacheKey, the collector's closure would still hold the original context object — the outbound telemetry would not change.

No in-process token rotation path exists

ensureGatewayAuthToken is write-once (create-if-absent). There is no code path anywhere in the codebase that deletes, overwrites, or rotates server-token during a running process. The rm -f in docker-entrypoint.sh runs before the process starts.

If a user manually edits the file while the process is running, the correct action is to restart the process — both gateway auth and telemetry identity will pick up the new value on next startup.

Conclusion

The cache key not including token identity is consistent with the rest of the system's design: the token is treated as immutable for the process lifetime. No code change is needed. Closing as not-a-bug.