HDDS-14876. Make snapshot diff progress more granular - #10901
Open
rhalm wants to merge 1 commit into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR improves snapshot diff progress observability across OM, client, and CLI by adding stage-level substatus reporting and (when applicable) progress percentage, plus additional per-stage logging to help operators understand runtime behavior.
Changes:
- Extends the SnapshotDiffResponse wire format to include
subStatusand optionalprogressPercent, and wires these through OM server handling, client-side translation, and CLI JSON/text rendering. - Adds a new FSO-specific stage (
PATH_RESOLUTION_FSO) and refines snapshot diff stage transitions/logging insideSnapshotDiffManager. - Adds/updates unit tests to validate proto serialization behavior and SnapshotDiffResponse rendering.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/protocolPB/TestOzoneManagerRequestHandler.java | Adds tests asserting subStatus/progress are serialized into the OM protocol response (or omitted when subStatus is null). |
| hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/snapshot/TestSnapshotDiffManager.java | Updates snapshot diff manager tests and adds coverage for in-progress responses including subStatus/progress details and new PATH_RESOLUTION_FSO state. |
| hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/protocolPB/OzoneManagerRequestHandler.java | Serializes SnapshotDiffResponse subStatus/progress into the protobuf response. |
| hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/SnapshotDiffManager.java | Adds PATH_RESOLUTION_FSO stage, improves stage logging, and resets progress at the start of object-ID-map generation. |
| hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/diff/delta/CompositeDeltaDiffComputer.java | Adds a warning when DAG-based delta computation yields no result and the code falls back to full diff. |
| hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto | Adds PATH_RESOLUTION_FSO enum value and new optional progressPercent field to SnapshotDiffResponse. |
| hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/snapshot/TestSnapshotDiffResponse.java | Expands response-string rendering tests for map-gen vs non-map-gen substatuses and null subStatus behavior. |
| hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/snapshot/SnapshotDiffResponse.java | Adds getters, refines toString formatting, introduces SubStatus.hasProgress(), and adds PATH_RESOLUTION_FSO. |
| hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocolPB/OzoneManagerProtocolClientSideTranslatorPB.java | Deserializes subStatus/progress from protobuf into the client SnapshotDiffResponse. |
| hadoop-ozone/cli-shell/src/main/java/org/apache/hadoop/ozone/shell/snapshot/SnapshotDiffHandler.java | Includes subStatus and (only when applicable) progressPercent in CLI JSON output. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+1485
to
+1488
| if (response.getSubStatus() != null) { | ||
| builder.setSubStatus(response.getSubStatus().toProtoBuf()); | ||
| builder.setProgressPercent(response.getProgressPercent()); | ||
| } |
Comment on lines
+1477
to
+1480
| if (diffResponse.hasSubStatus()) { | ||
| result.setSubStatus(SnapshotDiffResponse.SubStatus.fromProtoBuf(diffResponse.getSubStatus())); | ||
| result.setProgressPercent(diffResponse.getProgressPercent()); | ||
| } |
Comment on lines
1262
to
+1267
| if (deltaFiles.isEmpty()) { | ||
| return; | ||
| } | ||
| long objectIdMapStart = Time.monotonicNow(); | ||
| updateProgress(jobKey, 0.0); | ||
| AtomicLong keysProcessed = new AtomicLong(0); |
SaketaChalamchala
added
the
snapshot
SaketaChalamchala
requested review from
SaketaChalamchala,
jojochuang,
sadanand48 and
smengcl
July 29, 2026 18:59
sadanand48
reviewed
Jul 30, 2026
| if (LOG.isDebugEnabled()) { | ||
| LOG.debug("Computed Delta SST File Set, Total count = {} ", deltaFiles.size()); | ||
| } | ||
| LOG.info("Computed Delta SST File Set for table '{}', file count: {}, elapsed: {}ms, jobId: {}", |
Contributor
There was a problem hiding this comment.
It looks like for each successful snapshot diff call, it will generate around 6-7 info log statements. I think the per stage ones could be at the debug level itself
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Improves snapshot diff progress granularity by letting clients track which stage a diff job is in and adding more informative logs. Main changes:
subStatusandprogressPercentthrough the proto layer and CLI output (with a minor formatting fix)PATH_RESOLUTION_FSO)SST_FILE_DELTA_*instead of switching toOBJECT_ID_MAP_GEN_*during object ID map generation (which now usesOBJECT_ID_MAP_GEN_OBSfor OBS and FSO file table,OBJECT_ID_MAP_GEN_FSOfor FSO directory table)Before / After
Example CLI output when polling an in-progress snapshot diff before this change:
$ ozone sh snapshot diff --get-report /vol/bucket snap1 snap2 Snapshot diff job is IN_PROGRESS. Please retry after 60000 ms. $ ozone sh snapshot diff --get-report --json /vol/bucket snap1 snap2 { "snapshotDiffReport" : { ... }, "jobStatus" : "IN_PROGRESS", "waitTimeInMs" : 60000 }And after:
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-14876
How was this patch tested?