Skip to content

fix(operations): read switchover dispatch messages#10681

Open
weicao wants to merge 2 commits into
release-1.1from
bugfix/release-1.1-switchover-dispatch-reader
Open

fix(operations): read switchover dispatch messages#10681
weicao wants to merge 2 commits into
release-1.1from
bugfix/release-1.1-switchover-dispatch-reader

Conversation

@weicao

@weicao weicao commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Problem

The current-line switchover writer will record a durable dispatch claim and outcome in the existing per-instance progress message before or after the non-idempotent lifecycle call. A release-1.1 rollback reader must understand those messages without replaying the action or inventing success.

Two compatibility paths also need to remain distinct:

  • new writes use the logical component or sharding key;
  • older in-flight items may still be stored under a Component object-name key and may have no protocol message.

The broken edge was that a rollback reader could either miss the writer's logical key or treat old and new message formats as the same state. That can leave an item stuck or re-enter the legacy dispatch path incorrectly.

Fixes #10671.

What changes

This is a reader-only compatibility backport. It does not create dispatch claims or outcomes and does not change the API or CRD.

  • The logical component/sharding key is always checked first.
  • Only when that key is absent and ComponentObjectName is present does the reader fall back to the older object-name status key.
  • Exact claim and outcome messages validate the OpsRequest UID, logical component, instance and candidate identity.
  • Claimed or outcome-unknown work fails closed without invoking the lifecycle action.
  • A protocol message found under the legacy object-name key fails closed before dispatch, including when the item is Pending.
  • Legacy Pending and Processing items without a protocol message keep the old release-1.1 behavior.
  • If both logical and legacy keys exist, the logical key wins and the legacy detail is not modified.

Rollout boundary

Release-1.0 and release-1.1 readers must be published before the current-line writer is activated. Before writer activation, operators must also inspect and clear any legacy object-key Pending items because those intentionally retain the old action-before-status behavior.

This PR is not an activation or release-ready claim.

Validation

Exact head: f5312a323116d01217b53fa469878c224fa181dc
Tree: 1ce561189fa8c1924ec7f339299be53b1ac0d709

  • focused reader matrix: 16/16 PASS;
  • pkg/operations full PASS;
  • full race and test-fast PASS;
  • scoped vet/lint, generation, manifests, module and header checks PASS;
  • exact patch/tree/reverse checks PASS;
  • peer review reproduced the exact tree and reported zero source blockers.

The interrupted full make check-diff run is not reported as PASS. The exact changed-package and static gates above are green. Runtime validation is N=0.

@weicao
weicao requested a review from a team as a code owner July 15, 2026 21:16
@weicao weicao added the nopick Not auto cherry-pick when PR merged label Jul 15, 2026
@github-actions github-actions Bot added the size/L Denotes a PR that changes 100-499 lines. label Jul 15, 2026
@apecloud-bot

Copy link
Copy Markdown
Collaborator

Auto Cherry-pick Instructions

Usage:
  - /nopick: Not auto cherry-pick when PR merged.
  - /pick: release-x.x [release-x.x]: Auto cherry-pick to the specified branch when PR merged.

Example:
  - /nopick
  - /pick release-1.1

CLA Recheck Instructions

Usage:
  - /recheck-cla: Trigger a re-check of CLA status for this pull request.
Example:
  - /recheck-cla

@weicao

weicao commented Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

The only current PR Pre-Check failure is the issue-link check. The body contains Fixes #10671, and issue #10671 is open, but GitHub does not populate closingIssuesReferences for a pull request whose base is the non-default release-1.1 branch. The check therefore reports an empty list before any source test runs.

This is a release-branch workflow limitation, not a source failure in this patch. The issue and exact head remain linked in the PR body; I am leaving the PR title and base accurate rather than bypassing the check with a chore or docs prefix. Other CI jobs are still running and will be handled separately if they fail.

@codecov

codecov Bot commented Jul 15, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 76.36364% with 26 lines in your changes missing coverage. Please review.
✅ Project coverage is 53.13%. Comparing base (591e221) to head (6d5fa2f).
⚠️ Report is 1 commits behind head on release-1.1.

Files with missing lines Patch % Lines
pkg/operations/switchover.go 76.36% 19 Missing and 7 partials ⚠️
Additional details and impacted files
@@               Coverage Diff               @@
##           release-1.1   #10681      +/-   ##
===============================================
+ Coverage        53.02%   53.13%   +0.10%     
===============================================
  Files              545      545              
  Lines            60988    61080      +92     
===============================================
+ Hits             32340    32453     +113     
+ Misses           25521    25499      -22     
- Partials          3127     3128       +1     
Flag Coverage Δ
unittests 53.13% <76.36%> (+0.10%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread pkg/operations/switchover.go Outdated
const (
KBSwitchoverKey = "Switchover"
KBSwitchoverKey = "Switchover"
switchoverDispatchClaimMessagePrefix = "switchover dispatch claimed before lifecycle call: SwitchoverDispatch/"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Do not establish the cross-version dispatch protocol through ProgressStatusDetail.Message. The API defines Message as a human-readable explanation, but this reader treats exact text prefixes as state, version, request identity, and token. A later wording change becomes an unknown legacy message; for a switchover without a candidate, the default legacy branch can even interpret that unknown outcome as success. Dispatch state must use an explicit, versioned status API shared by the main writer and release readers, while Message remains display-only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

nopick Not auto cherry-pick when PR merged size/L Denotes a PR that changes 100-499 lines.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants