Skip to content

HDDS-16045. Make PendingContainerTracker StorageType aware - #10928

Draft
F64116045 wants to merge 1 commit into
apache:HDDS-11233from
F64116045:HDDS-16045-pending-container-storage-type
Draft

HDDS-16045. Make PendingContainerTracker StorageType aware#10928
F64116045 wants to merge 1 commit into
apache:HDDS-11233from
F64116045:HDDS-16045-pending-container-storage-type

Conversation

@F64116045

@F64116045 F64116045 commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

PendingContainerTracker currently accounts pending container allocations at the datanode level. With storage policy support, pending allocations should be counted only against the StorageType they target. For example, a pending DISK allocation should not reduce the available space calculated for SSD or ARCHIVE allocations.

This PR makes PendingContainerTracker store the StorageType for each pending container allocation. It adds StorageType-aware APIs in PendingContainerTracker and NodeManager, while keeping the existing untyped APIs and behavior for callers that do not have storage tier information.

PipelineManager derives the StorageType from the pipeline StorageTier and passes it to NodeManager when checking space and recording pending allocations.

The tests cover StorageType-specific pending counts, StorageType-specific space checks, the existing untyped behavior, and propagation from StorageTier.DISK to StorageType.DISK.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-16045

How was this patch tested?

  • git diff --check
  • mvn -B --no-transfer-progress -pl hadoop-hdds/server-scm -am -Dtest=TestPendingContainerTracker test -DskipShade -DskipRecon -DskipDocs

@F64116045

F64116045 commented Aug 2, 2026

Copy link
Copy Markdown
Contributor Author

Note: While reviewing the allocation path, I noticed one related point and would like to check whether this is already tracked somewhere.

recordPendingAllocation(...) is defined on PipelineManager / PipelineManagerImpl, but I could not find a caller on the HDDS-11233 branch. From what I found, container allocation calls hasEnoughSpace(...) from ContainerManagerImpl, but I do not see a matching recordPendingAllocation(...) call after successful allocation.

I may be missing some context here. If this is not already covered by another task, would it make sense to track it as a follow-up? I’m happy to help with it as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant