From f38c89823554c19b454d3b5502d64af1e1bc8836 Mon Sep 17 00:00:00 2001 From: slfan1989 Date: Sun, 31 May 2026 22:52:27 +0800 Subject: [PATCH 1/4] HDDS-15441. [DiskBalancer] Fix DiskBalancer documentation inconsistencies. --- hadoop-hdds/docs/content/feature/DiskBalancer.md | 7 +++---- hadoop-hdds/docs/content/feature/DiskBalancer.zh.md | 11 +++++------ 2 files changed, 8 insertions(+), 10 deletions(-) diff --git a/hadoop-hdds/docs/content/feature/DiskBalancer.md b/hadoop-hdds/docs/content/feature/DiskBalancer.md index a022c6968433..e20383fdc75d 100644 --- a/hadoop-hdds/docs/content/feature/DiskBalancer.md +++ b/hadoop-hdds/docs/content/feature/DiskBalancer.md @@ -46,7 +46,7 @@ A disk is considered a candidate for balancing if its The Disk Balancer feature is introduced with a feature flag. By default, this feature is disabled. The feature can be **enabled** by setting the following property to `true` in the `ozone-site.xml` configuration file: -`hdds.datanode.disk.balancer.enabled = false` +`hdds.datanode.disk.balancer.enabled = true` ### Authentication and Authorization @@ -158,9 +158,9 @@ ozone admin datanode diskbalancer report [ ...] [--in-service- | `--json` | Format output as JSON. | `--json` | | `-t/--threshold-percentage` | Volume density threshold percentage (default: 10.0). Used with `start` and `update` commands. | `-t 5`
`--threshold-percentage 5.0` | | `-b/--bandwidth-in-mb` | Maximum disk bandwidth in MB/s (default: 10). Used with `start` and `update` commands. | `-b 20`
`--bandwidth-in-mb 50` | -| `-p/--parallel-thread` | Number of parallel threads (default: 1). Used with `start` and `update` commands. | `-p 5`
`--parallel-thread 10` | +| `-p/--parallel-thread` | Number of parallel threads (default: 5). Used with `start` and `update` commands. | `-p 5`
`--parallel-thread 10` | | `-s/--stop-after-disk-even` | Stop automatically after disks are balanced (default: true). Used with `start` and `update` commands. | `-s false`
`--stop-after-disk-even true` | -| `-c/--container-states` | Comma-separated container lifecycle state names that may be moved between disks . Used with `start` and `update` commands. | `-c CLOSED,QUASI_CLOSED`
`--container-states OPEN,CLOSED` | +| `-c/--container-states` | Comma-separated container lifecycle state names that may be moved between disks. Used with `start` and `update` commands. | `-c CLOSED,QUASI_CLOSED`
`--container-states CLOSED` | ### Examples @@ -252,4 +252,3 @@ The DiskBalancer's behavior can be controlled using the following configuration | `hdds.datanode.disk.balancer.container.choosing.policy` | `org.apache.hadoop.ozone.container.diskbalancer.policy.DefaultContainerChoosingPolicy` | The policy for selecting source/destination volumes and which containers to move. | | `hdds.datanode.disk.balancer.service.timeout` | `300s` | Timeout for the Datanode DiskBalancer service operations. | | `hdds.datanode.disk.balancer.should.run.default` | `false` | If the balancer fails to read its persisted configuration, this value determines if the service should run by default. | - diff --git a/hadoop-hdds/docs/content/feature/DiskBalancer.zh.md b/hadoop-hdds/docs/content/feature/DiskBalancer.zh.md index fb8c704e2035..222c73899978 100644 --- a/hadoop-hdds/docs/content/feature/DiskBalancer.zh.md +++ b/hadoop-hdds/docs/content/feature/DiskBalancer.zh.md @@ -42,7 +42,7 @@ summary: 数据节点的磁盘平衡器. 磁盘平衡器功能已通过功能标志引入。默认情况下，此功能处于禁用状态。 可以通过在“ozone-site.xml”配置文件中将以下属性设置为“true”来**启用**该功能： -`hdds.datanode.disk.balancer.enabled = false` +`hdds.datanode.disk.balancer.enabled = true` ### 身份验证和授权 @@ -153,9 +153,9 @@ ozone admin datanode diskbalancer report [ ...] [--in-service- | `--json` | 输出格式设置为JSON。 | `--json` | | `-t/--threshold-percentage` | 磁盘使用率阈值百分比（默认值：10.0）。与 `start` 和 `update` 命令一起使用。 | `-t 5`
`--threshold-percentage 5.0` | | `-b/--bandwidth-in-mb` | 最大磁盘带宽，单位为 MB/s（默认值：10）。与 `start` 和 `update` 命令一起使用。 | `-b 20`
`--bandwidth-in-mb 50` | -| `-p/--parallel-thread` | 并行线程数（默认值：1）。与 `start` 和 `update` 命令一起使用。 | `-p 5`
`--parallel-thread 10` | -| `-s/--stop-after-disk-even` | 磁盘平衡完成后自动停止（默认值：false）。与 `start` 和 `update` 命令一起使用。 | `-s false`
`--stop-after-disk-even true` | -| `-c/--container-states` | 以逗号分隔的容器生命周期状态名称，表示可在磁盘之间移动的状态。配合 `start` 和 `update` 命令使用。 | `-c CLOSED,QUASI_CLOSED`
`--container-states OPEN,CLOSED` | +| `-p/--parallel-thread` | 并行线程数（默认值：5）。与 `start` 和 `update` 命令一起使用。 | `-p 5`
`--parallel-thread 10` | +| `-s/--stop-after-disk-even` | 磁盘平衡完成后自动停止（默认值：true）。与 `start` 和 `update` 命令一起使用。 | `-s false`
`--stop-after-disk-even true` | +| `-c/--container-states` | 以逗号分隔的容器生命周期状态名称，表示可在磁盘之间移动的状态。配合 `start` 和 `update` 命令使用。 | `-c CLOSED,QUASI_CLOSED`
`--container-states CLOSED` | ### 示例 **启动 DiskBalancer：** @@ -233,7 +233,7 @@ The DiskBalancer's behavior can be controlled using the following configuration | Property | Default Value | Description | |-------------------------------------------------------------|----------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `hdds.datanode.disk.balancer.enabled` | `false` | 如果为 false，则 Datanode 上的 DiskBalancer 服务将被禁用。将其配置为 true 可启用 DiskBalancer。 | | | | +| `hdds.datanode.disk.balancer.enabled` | `false` | 如果为 false，则 Datanode 上的 DiskBalancer 服务将被禁用。将其配置为 true 可启用 DiskBalancer。 | | `hdds.datanode.disk.balancer.volume.density.threshold.percent` | `10.0` | 百分比（0-100）。如果对于每个卷，其利用率与平均数据节点利用率之差不超过此阈值，则认为数据节点处于平衡状态。 | | `hdds.datanode.disk.balancer.max.disk.throughputInMBPerSec` | `10` | 平衡器可用于移动数据的最大带宽（以 MB/s 为单位），以避免影响客户端 I/O。 | | `hdds.datanode.disk.balancer.parallel.thread` | `5` | 用于并行移动容器的工作线程数。 | @@ -244,4 +244,3 @@ The DiskBalancer's behavior can be controlled using the following configuration | `hdds.datanode.disk.balancer.container.choosing.policy` | `org.apache.hadoop.ozone.container.diskbalancer.policy.DefaultContainerChoosingPolicy` | 用于选择源/目标卷以及要移动的容器的策略。 | | `hdds.datanode.disk.balancer.service.timeout` | `300s` | Datanode DiskBalancer 服务操作超时。 | | `hdds.datanode.disk.balancer.should.run.default` | `false` | 如果平衡器无法读取其持久配置，则该值决定服务是否应默认运行。 | - From 86129208029c94471b400906490c2b6ecf656354 Mon Sep 17 00:00:00 2001 From: slfan1989 <55643692+slfan1989@users.noreply.github.com> Date: Tue, 2 Jun 2026 13:18:49 +0800 Subject: [PATCH 2/4] Update hadoop-hdds/docs/content/feature/DiskBalancer.md Co-authored-by: Gargi Jaiswal <134698352+Gargi-jais11@users.noreply.github.com> --- hadoop-hdds/docs/content/feature/DiskBalancer.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hadoop-hdds/docs/content/feature/DiskBalancer.md b/hadoop-hdds/docs/content/feature/DiskBalancer.md index e20383fdc75d..55f7786056d8 100644 --- a/hadoop-hdds/docs/content/feature/DiskBalancer.md +++ b/hadoop-hdds/docs/content/feature/DiskBalancer.md @@ -248,7 +248,7 @@ The DiskBalancer's behavior can be controlled using the following configuration | `hdds.datanode.disk.balancer.service.interval` | `60s` | The time interval at which the Datanode DiskBalancer service checks for imbalance and updates its configuration. | | `hdds.datanode.disk.balancer.stop.after.disk.even` | `true` | If true, the DiskBalancer will automatically stop its balancing activity once disks are considered balanced (i.e., all volume densities are within the threshold). | | `hdds.datanode.disk.balancer.replica.deletion.delay` | `5m` | The delay after a container is successfully moved from source volume to destination volume before the source container replica is deleted. This lazy deletion provides a grace period before failing the read thread holding the old container replica. Unit: ns, ms, s, m, h, d. | -| `hdds.datanode.disk.balancer.container.states` | `CLOSED,QUASI_CLOSED` | Comma-separated container lifecycle state names that may be moved between disks (must match enum names exactly, uppercase). Default includes **CLOSED** and **QUASI_CLOSED**; extend the list when additional states are needed to be balanced. All defined container states are OPEN, CLOSING, QUASI_CLOSED, CLOSED, UNHEALTHY, INVALID, DELETED, RECOVERING. | +| `hdds.datanode.disk.balancer.container.states` | `CLOSED,QUASI_CLOSED` | Comma-separated container lifecycle state names that may be moved between disks (must match enum names exactly, uppercase). Default includes **CLOSED** and **QUASI_CLOSED**; extend the list when additional states are needed to be balanced. All defined container states which are eligibile to move QUASI_CLOSED, CLOSED, UNHEALTHY, INVALID. | | `hdds.datanode.disk.balancer.container.choosing.policy` | `org.apache.hadoop.ozone.container.diskbalancer.policy.DefaultContainerChoosingPolicy` | The policy for selecting source/destination volumes and which containers to move. | | `hdds.datanode.disk.balancer.service.timeout` | `300s` | Timeout for the Datanode DiskBalancer service operations. | | `hdds.datanode.disk.balancer.should.run.default` | `false` | If the balancer fails to read its persisted configuration, this value determines if the service should run by default. | From 44f9abd1c5b8c93908a08b771928d0ca2d813810 Mon Sep 17 00:00:00 2001 From: slfan1989 Date: Tue, 2 Jun 2026 13:48:21 +0800 Subject: [PATCH 3/4] HDDS-15441. [DiskBalancer] Fix DiskBalancer documentation inconsistencies. --- hadoop-hdds/docs/content/feature/DiskBalancer.md | 12 ++++++------ .../docs/content/feature/DiskBalancer.zh.md | 14 +++++++------- 2 files changed, 13 insertions(+), 13 deletions(-) diff --git a/hadoop-hdds/docs/content/feature/DiskBalancer.md b/hadoop-hdds/docs/content/feature/DiskBalancer.md index 55f7786056d8..5c25ec8d4397 100644 --- a/hadoop-hdds/docs/content/feature/DiskBalancer.md +++ b/hadoop-hdds/docs/content/feature/DiskBalancer.md @@ -154,7 +154,7 @@ ozone admin datanode diskbalancer report [ ...] [--in-service- | Option | Description | Example | |-------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------| | `` | One or more datanode addresses as positional arguments. Addresses can be:
- Hostname (e.g., `DN-1`) - uses default CLIENT_RPC port (19864)
- Hostname with port (e.g., `DN-1:19864`)
- IP address (e.g., `192.168.1.10`)
- IP address with port (e.g., `192.168.1.10:19864`)
- Stdin (`-`) - reads datanode addresses from standard input, one per line | `DN-1`
`DN-1:19864`
`192.168.1.10`
`-` | -| `--in-service-datanodes` | It queries SCM for all IN_SERVICE datanodes and executes the command on all of them. | `--in-service-datanodes` | +| `--in-service-datanodes` | It queries SCM for all IN_SERVICE and HEALTHY datanodes and executes the command on all of them. | `--in-service-datanodes` | | `--json` | Format output as JSON. | `--json` | | `-t/--threshold-percentage` | Volume density threshold percentage (default: 10.0). Used with `start` and `update` commands. | `-t 5`
`--threshold-percentage 5.0` | | `-b/--bandwidth-in-mb` | Maximum disk bandwidth in MB/s (default: 10). Used with `start` and `update` commands. | `-b 20`
`--bandwidth-in-mb 50` | @@ -169,7 +169,7 @@ ozone admin datanode diskbalancer report [ ...] [--in-service- # Start DiskBalancer on multiple datanodes ozone admin datanode diskbalancer start DN-1 DN-2 DN-3 -# Start DiskBalancer on all IN_SERVICE datanodes +# Start DiskBalancer on all IN_SERVICE and HEALTHY datanodes ozone admin datanode diskbalancer start --in-service-datanodes # Start DiskBalancer with configuration parameters @@ -189,7 +189,7 @@ ozone admin datanode diskbalancer start DN-1 --json # Stop DiskBalancer on multiple datanodes ozone admin datanode diskbalancer stop DN-1 DN-2 DN-3 -# Stop DiskBalancer on all IN_SERVICE datanodes +# Stop DiskBalancer on all IN_SERVICE and HEALTHY datanodes ozone admin datanode diskbalancer stop --in-service-datanodes # Stop DiskBalancer with json output @@ -202,7 +202,7 @@ ozone admin datanode diskbalancer stop DN-1 --json # Update multiple parameters ozone admin datanode diskbalancer update DN-1 -t 5 -b 50 -p 10 -# Update on all IN_SERVICE datanodes +# Update on all IN_SERVICE and HEALTHY datanodes ozone admin datanode diskbalancer update --in-service-datanodes -t 5 # Or using the long form: ozone admin datanode diskbalancer update --in-service-datanodes --threshold-percentage 5 @@ -216,7 +216,7 @@ ozone admin datanode diskbalancer update DN-1 -b 50 --json # Get status from multiple datanodes ozone admin datanode diskbalancer status DN-1 DN-2 DN-3 -# Get status from all IN_SERVICE datanodes +# Get status from all IN_SERVICE and HEALTHY datanodes ozone admin datanode diskbalancer status --in-service-datanodes # Get status as JSON @@ -228,7 +228,7 @@ ozone admin datanode diskbalancer status --in-service-datanodes --json # Get report from multiple datanodes ozone admin datanode diskbalancer report DN-1 DN-2 DN-3 -# Get report from all IN_SERVICE datanodes +# Get report from all IN_SERVICE and HEALTHY datanodes ozone admin datanode diskbalancer report --in-service-datanodes # Get report as JSON diff --git a/hadoop-hdds/docs/content/feature/DiskBalancer.zh.md b/hadoop-hdds/docs/content/feature/DiskBalancer.zh.md index 222c73899978..76d62a5b9880 100644 --- a/hadoop-hdds/docs/content/feature/DiskBalancer.zh.md +++ b/hadoop-hdds/docs/content/feature/DiskBalancer.zh.md @@ -149,7 +149,7 @@ ozone admin datanode diskbalancer report [ ...] [--in-service- | Option | Description | Example | |-------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------| | `` | 一个或多个数据节点地址作为位置参数。地址可以是：
- 主机名（例如，`DN-1`）- 使用默认的 CLIENT_RPC 端口 (19864)
- 带端口的主机名（例如，`DN-1:19864`）
- IP 地址（例如，`192.168.1.10`）
- 带端口的 IP 地址（例如，`192.168.1.10:19864`）
- 标准输入 (`-`) - 从标准输入读取数据节点地址，每行一个 | `DN-1`
`DN-1:19864`
`192.168.1.10`
`-` | -| `--in-service-datanodes` | 它向 SCM 查询所有 IN_SERVICE 数据节点，并在所有这些数据节点上执行该命令。 | `--in-service-datanodes` | +| `--in-service-datanodes` | 它向 SCM 查询所有 IN_SERVICE 且 HEALTHY 的数据节点，并在所有这些数据节点上执行该命令。 | `--in-service-datanodes` | | `--json` | 输出格式设置为JSON。 | `--json` | | `-t/--threshold-percentage` | 磁盘使用率阈值百分比（默认值：10.0）。与 `start` 和 `update` 命令一起使用。 | `-t 5`
`--threshold-percentage 5.0` | | `-b/--bandwidth-in-mb` | 最大磁盘带宽，单位为 MB/s（默认值：10）。与 `start` 和 `update` 命令一起使用。 | `-b 20`
`--bandwidth-in-mb 50` | @@ -164,7 +164,7 @@ ozone admin datanode diskbalancer report [ ...] [--in-service- # 在多个数据节点上启动 DiskBalancer ozone admin datanode diskbalancer start DN-1 DN-2 DN-3 -# 在所有运行中的数据节点上启动 DiskBalancer +# 在所有 IN_SERVICE 且 HEALTHY 的数据节点上启动 DiskBalancer ozone admin datanode diskbalancer start --in-service-datanodes # 使用配置参数启动 DiskBalancer @@ -183,7 +183,7 @@ ozone admin datanode diskbalancer start DN-1 --json # 在多个数据节点上停止 DiskBalancer ozone admin datanode diskbalancer stop DN-1 DN-2 DN-3 -# 在所有运行中的数据节点上停止 DiskBalancer +# 在所有 IN_SERVICE 且 HEALTHY 的数据节点上停止 DiskBalancer ozone admin datanode diskbalancer stop --in-service-datanodes # 停止 DiskBalancer 并输出 JSON 信息 @@ -195,7 +195,7 @@ ozone admin datanode diskbalancer stop DN-1 --json # 更新多个参数 ozone admin datanode diskbalancer update DN-1 -t 5 -b 50 -p 10 -# 更新所有 IN_SERVICE 数据节点 +# 更新所有 IN_SERVICE 且 HEALTHY 的数据节点 ozone admin datanode diskbalancer update --in-service-datanodes -t 5 # 更新并输出 JSON 格式 @@ -208,7 +208,7 @@ ozone admin datanode diskbalancer update DN-1 -b 50 --json # 从多个数据节点获取状态 ozone admin datanode diskbalancer status DN-1 DN-2 DN-3 -# 从所有处于服务状态的数据节点获取状态 +# 从所有 IN_SERVICE 且 HEALTHY 的数据节点获取状态 ozone admin datanode diskbalancer status --in-service-datanodes # 以 JSON 格式获取状态 @@ -220,7 +220,7 @@ ozone admin datanode diskbalancer status --in-service-datanodes --json # 从多个数据节点获取报告 ozone admin datanode diskbalancer report DN-1 DN-2 DN-3 -# 从所有处于服务状态的数据节点获取报告 +# 从所有 IN_SERVICE 且 HEALTHY 的数据节点获取报告 ozone admin datanode diskbalancer report --in-service-datanodes # 以 JSON 格式获取报告 @@ -240,7 +240,7 @@ The DiskBalancer's behavior can be controlled using the following configuration | `hdds.datanode.disk.balancer.service.interval` | `60s` | Datanode DiskBalancer 服务检查不平衡并更新其配置的时间间隔。 | | `hdds.datanode.disk.balancer.stop.after.disk.even` | `true` | 如果为真，则一旦磁盘被视为平衡（即所有卷密度都在阈值内），DiskBalancer 将自动停止其平衡活动。 | | `hdds.datanode.disk.balancer.replica.deletion.delay` | `5m` | 容器成功从源卷移动到目标卷后，源容器副本被删除前的延迟时间。这种延迟删除机制旨在避免旧副本的即时删除导致持有旧容器副本的线程数据读取失败。单位：ns、ms、s、m、h、d。| -| `hdds.datanode.disk.balancer.container.states` | `CLOSED,QUASI_CLOSED` | 以逗号分隔的容器生命周期状态名称列表，指定了允许在不同磁盘之间移动的容器状态（须与枚举名完全一致，使用大写）。默认包含 **CLOSED** 和 **QUASI_CLOSED**；若需对更多状态的容器进行负载均衡，请扩展此列表。所有已定义的容器状态包括：OPEN、CLOSING、QUASI_CLOSED、CLOSED、UNHEALTHY、INVALID、DELETED 和 RECOVERING。 | +| `hdds.datanode.disk.balancer.container.states` | `CLOSED,QUASI_CLOSED` | 以逗号分隔的容器生命周期状态名称列表，指定可在不同磁盘之间移动的容器状态（须与枚举名完全一致，使用大写）。默认包含 **CLOSED** 和 **QUASI_CLOSED**；若需对更多状态的容器进行负载均衡，请扩展此列表。可移动的已定义容器状态包括：QUASI_CLOSED、CLOSED、UNHEALTHY、INVALID。 | | `hdds.datanode.disk.balancer.container.choosing.policy` | `org.apache.hadoop.ozone.container.diskbalancer.policy.DefaultContainerChoosingPolicy` | 用于选择源/目标卷以及要移动的容器的策略。 | | `hdds.datanode.disk.balancer.service.timeout` | `300s` | Datanode DiskBalancer 服务操作超时。 | | `hdds.datanode.disk.balancer.should.run.default` | `false` | 如果平衡器无法读取其持久配置，则该值决定服务是否应默认运行。 | From cf7e4cf2e48ea5083e14a3aab6b8450c60ae45e5 Mon Sep 17 00:00:00 2001 From: slfan1989 Date: Tue, 2 Jun 2026 13:52:30 +0800 Subject: [PATCH 4/4] HDDS-15441. [DiskBalancer] Fix DiskBalancer documentation inconsistencies. --- hadoop-hdds/docs/content/feature/DiskBalancer.zh.md | 1 + 1 file changed, 1 insertion(+) diff --git a/hadoop-hdds/docs/content/feature/DiskBalancer.zh.md b/hadoop-hdds/docs/content/feature/DiskBalancer.zh.md index 76d62a5b9880..a6e573c278ee 100644 --- a/hadoop-hdds/docs/content/feature/DiskBalancer.zh.md +++ b/hadoop-hdds/docs/content/feature/DiskBalancer.zh.md @@ -244,3 +244,4 @@ The DiskBalancer's behavior can be controlled using the following configuration | `hdds.datanode.disk.balancer.container.choosing.policy` | `org.apache.hadoop.ozone.container.diskbalancer.policy.DefaultContainerChoosingPolicy` | 用于选择源/目标卷以及要移动的容器的策略。 | | `hdds.datanode.disk.balancer.service.timeout` | `300s` | Datanode DiskBalancer 服务操作超时。 | | `hdds.datanode.disk.balancer.should.run.default` | `false` | 如果平衡器无法读取其持久配置，则该值决定服务是否应默认运行。 | +