maintenance: version cluster message envelopes

[zqr10159]

Summary

This standalone update defines a versioned cluster-message envelope, a safe
fresh-install path, and an explicit mixed-version rollout path.

• bases the complete change directly on current master as one neutral commit;
  it no longer depends on [fix](security): stop sending AES secret over unauthenticated Netty #4263
• binds signed messages to a per-connection server challenge
• ships required enforcement as the Java and YAML default
• keeps optional available only as an explicit mixed-version rollout setting
• supports active and previous key IDs for non-disruptive rotation
• derives a dedicated MAC key with HKDF domain separation, or accepts an
  independently configured authentication secret
• validates configuration before the Netty endpoint starts
• makes clock skew and handshake timeout configurable
• records bounded rejection, legacy-acceptance, and handshake-timeout metrics

Compatibility matrix

Manager	Collector	Result
old	old	compatible
old	new explicit optional	compatible during rollout
new explicit optional	old	compatible during rollout
new optional	new optional	compatible during rollout
new required	new required	compatible steady state
old	new required	rejected
new required	old	rejected

Installation and upgrade

Manager and every standalone Collector now receive two independent values:

• COMMON_SECRET: the identical AES key on both sides, exactly 16, 24, or 32
  bytes
• CLUSTER_AUTH_ACTIVE_SECRET: a separate cluster-message authentication
  secret of at least 32 bytes

Fresh Docker and Compose instructions use:

umask 077
printf 'COMMON_SECRET=%s\n' "$(openssl rand -hex 16)" > .env
printf 'CLUSTER_AUTH_ACTIVE_SECRET=%s\n' "$(openssl rand -hex 32)" >> .env

rand -hex 16 produces 32 ASCII bytes and is therefore a valid AES-256
COMMON_SECRET. rand -hex 32 produces 64 ASCII characters for cluster
authentication and must not be reused directly as the AES key.

All five shipped Compose definitions fail interpolation with an actionable
message when either value is absent. Their packaged application configuration
maps COMMON_SECRET to common.secret, and the same private .env is supplied
to Manager and standalone Collector.

Existing installations must preserve their current AES secret. Replacing it
during an ordinary upgrade can make encrypted data unreadable. The upgrade
guide covers recovering the existing Manager value through restricted
administrator access, provisioning both values on every node, using explicit
optional mode only during the mixed-version window, and returning to required
mode after convergence.

Regression proof

• The default-mode contract failed on an earlier implementation with
  expected REQUIRED but was OPTIONAL; it now passes.
• The first packaging contract failed on the reviewed head because Compose did
  not require an install-specific authentication secret; it now passes for all
  five distributions.
• The reopened packaging contract failed because Compose, package
  configuration, and installation documentation omitted COMMON_SECRET; it
  now requires and documents both independent values.
• CollectorDocumentedEnvironmentIntegrationTest starts a separate real
  Collector JVM with the documented environment, waits for Spring startup, and
  verifies that it stays alive without missing- or invalid-AES-secret errors.
• A separate extracted release-package proof starts
  apache-hertzbeat-collector-1.9.0.jar with those same two variables and
  observes stable startup.

Validation on the standalone master-based commit

• Focused authentication, compatibility, Manager/Collector, packaging, and
  documented-process reactor: all selected tests passed.
• Real Collector process integration: passed and leaves generated logs under
  ignored target output.
• Docker Compose v5.0.2: all five source definitions render successfully with
  independent valid fixtures.
• Standalone Collector release package: 13-module cluster-profile assembly
  passed; the extracted packaged JAR reached stable startup with the documented
  environment.
• Startup source package proof: all 24 reactor modules passed.
• Release-profile assembly: all 24 reactor modules passed and produced server,
  Docker, and Docker Compose archives.
• The generated Compose archive retains both required interpolation
  expressions, the common.secret mapping, and the corrected AES generator in
  all five packaged distributions.
• Maven Checkstyle, added-code CJK scan, git diff --check, and
  git diff --cached --check passed.
• Branch shape: current master plus one commit (e533b4e106), with no
  dependency branch.
• GitHub CI passed on the amended head: Backend Build (9m51s), Backend Maven
  E2E (12m57s), Backend Image E2E (2m46s), DOC CI (9m25s), License Checker,
  and labeler.

Operational notes

Blank, known-default, weak authentication secrets and invalid-length AES
secrets fail before the standalone Collector endpoint starts. Required mode
rejects unsigned peers, so existing clusters must follow the documented
explicit-optional rollout sequence rather than upgrading Manager first. Clock
synchronization, secret backup, and controlled key distribution remain
deployment responsibilities.

AI assistance: used for draft implementation and test iteration.
Human validation: reproduced the documented startup failure, ran focused
authentication/compatibility and bootstrap tests, rendered all Compose
variants, started a real packaged Collector, and completed source and release
package proofs.
Risk notes: optional mode deliberately accepts unsigned legacy traffic and must
be enabled only for a bounded rollout window; required remains the shipped and
intended steady-state default.

[zqr10159]

Author remediation update and remaining blocker:

The branch now has versioned, challenge-bound envelopes; configurable optional-to-required rollout modes; HKDF-separated authentication keys; key IDs and dual-key acceptance for rotation; startup validation; configurable clock skew; rejection metrics; replay handling; and an old and new manager and collector test matrix. Focused and reactor validation passed locally.

This PR is intentionally not author-approved for direct merge yet. It still targets the unmerged #4263 branch, so it must be rebased onto the final integrated master state after #4263 lands and then receive the full repository CI. GitHub currently shows only the label check. Until that dependency and CI condition are satisfied, the merge result remains blocked.

[zqr10159]

The remaining review blockers are addressed in the latest head (7768405ced). The PR now targets current master directly and contains exactly one commit, so it no longer depends on #4263. Java plus all shipped YAML defaults are required; optional remains available only through an explicit rollout override. The new default regression failed on the prior head (expected REQUIRED but was OPTIONAL) and now passes. Latest local validation: 21 remoting authentication/compatibility tests, 16 targeted manager/collector tests, and the 24-module startup source package proof all passed. Full GitHub CI has now been triggered by the master retarget/push.

[zqr10159]

Standalone verification is now complete on head 183cf6bb21.

• The PR targets master and is exactly one commit above it; the former [fix](security): stop sending AES secret over unauthenticated Netty #4263 dependency is included in that standalone commit.
• required is the shipped Java and YAML default. optional remains available only as an explicit mixed-version rollout override, with removal after convergence documented in the PR.
• GitHub checks are green: Backend Build, Backend Maven E2E, Backend Image E2E, DOC CI, License Checker, and labeler.

For transparency, the first Maven E2E attempt stopped after 2m42s in the unrelated CommonHttpClientVirtualThreadTest.dispatchConnectionPoolCleanupClosesExpiredAndIdleConnections timing test, where a cleanup call was observed twice. This PR does not modify that client or test, and the complete class passed locally. GitHub attempt 2 passed the full Maven E2E job without any code change in 12m45s: https://github.com/apache/hertzbeat/actions/runs/30592359568

The dependency, CI coverage, and secure steady-state default blockers from the review are therefore resolved.

[zqr10159]

Addressed the remaining default-install and upgrade blocker on head
0d850535b1.

What changed:

• all five shipped Docker Compose definitions now require
  CLUSTER_AUTH_ACTIVE_SECRET during interpolation and return an actionable
  error before container startup when it is absent;
• every corresponding conf/application.yml exposes the complete
  authentication block;
• each English and Chinese distribution guide creates a private .env with
  openssl rand -hex 32, requires the value to be preserved across upgrades,
  and requires standalone Collectors to use the same value;
• root/localized quick starts plus Docker, Compose, package, native-Collector,
  Collector-reference, and upgrade documentation now describe generation,
  sharing, backup, and optional-to-required rollout;
• a Manager regression proves the all-in-one default replaces and persists its
  known-default AES root before Netty authentication validation, then reuses
  the same root after restart.

Regression proof:

• on the reviewed head, the packaging contract failed because the Compose
  distributions did not require/document an install secret and the package
  upgrade path was incomplete;
• after the change, all five definitions fail as expected when the variable is
  absent and all five render with a valid fixture under Docker Compose v5.0.2.

Validation:

• remoting authentication/compatibility: 18 tests passed, including the
  seven-combination old/new loopback matrix;
• Manager bootstrap and packaging contracts: 3 tests passed;
• startup source package: 24-module reactor passed;
• release profile: 24-module reactor passed and generated server, Docker, and
  Docker Compose archives;
• the generated Compose archive preserves the required-secret expression in
  all five definitions;
• Checkstyle, added-test CJK scan, and Git whitespace checks passed.

The branch remains exactly one commit above master. GitHub CI is running on
the amended head.

[zqr10159]

Final current-head CI update for 0d850535b1: all checks passed.

• Backend Build: 9m35s
• Backend Maven E2E: 13m01s
• Backend Image E2E: 1m52s
• DOC CI: 7m26s
• License Checker: passed
• labeler: passed

The PR remains based on master, mergeable, and exactly one commit above the
base. No rerun or code change was needed.

[zqr10159]

Addressed the remaining standalone Collector bootstrap blocker on head
e533b4e106.

The installation contract now provisions two independent values to Manager and
every standalone Collector:

• COMMON_SECRET, generated with openssl rand -hex 16, yielding a valid
  32-byte AES key;
• CLUSTER_AUTH_ACTIVE_SECRET, independently generated with
  openssl rand -hex 32.

The 64-character authentication output is explicitly not reused as
COMMON_SECRET. All five Compose definitions require both variables, all five
packaged application configurations map COMMON_SECRET to common.secret,
and the root/localized quick starts plus Docker, Compose, package, native
Collector, Collector-reference, and upgrade guides describe the same contract.
The upgrade path preserves an existing AES key rather than replacing it and
risking unreadable encrypted data.

Regression and runtime proof:

• the new packaging contract failed on the reviewed head because Compose and
  documentation omitted COMMON_SECRET;
• the updated 22-module packaging/authentication reactor passed;
• CollectorDocumentedEnvironmentIntegrationTest launches a separate real
  Collector JVM with both documented variables and verifies stable startup;
• a rebuilt and extracted apache-hertzbeat-collector-1.9.0-bin.tar.gz was
  started with the same environment, reached Started Collector, and remained
  alive without a common-secret validation error;
• all five Compose definitions rendered successfully;
• the 24-module startup package and 24-module release-profile assembly passed;
• the generated Compose archive retains both requirements in all five
  distributions.

The temporary package extraction and source-tree test logs were removed after
verification. GitHub CI is now running on the amended head.

[zqr10159]

Final current-head CI update for e533b4e106: all checks passed.

• Backend Build: 9m51s
• Backend Maven E2E: 12m57s
• Backend Image E2E: 2m46s
• DOC CI: 9m25s
• License Checker: passed
• labeler: passed

The PR remains based directly on master, mergeable, and exactly one commit
above the base. Together with the real packaged-Collector startup proof, the
standalone configuration and documentation blocker is resolved.