HPCC-35694 Add support to WsDFU.DFUQuery to sort by physical "File Size" by asselitx · Pull Request #21119 · hpcc-systems/HPCC-Platform

asselitx · 2026-03-19T15:46:18Z

Type of change:

This change is a bug fix (non-breaking change which fixes an issue).
This change is a new feature (non-breaking change which adds functionality).
This change improves the code (refactor or other change that does not change the functionality)
This change fixes warnings (the fix does not alter the functionality or the generated code)
This change is a breaking change (fix or feature that will cause existing behavior to change).
This change alters the query API (existing queries will have to be recompiled)

Checklist:

Smoketest:

Send notifications about my Pull Request position in Smoketest queue.
Test my draft Pull Request.

Testing:

github-actions · 2026-03-19T15:56:13Z

Jira Issue: https://hpccsystems.atlassian.net//browse/HPCC-35694

Jirabot Action Result:
Changing assignee from: gordon.smith@lexisnexisrisk.com to: terrence.asselin@lexisnexisrisk.com
Workflow Transition To: Merge Pending
Additional PR: #21119

Copilot

Pull request overview

Adds backend and UI support for sorting DFUQuery results by compressed size / on-disk size semantics, and updates UI to display “File Size” and “Compression” instead of separate size columns.

Changes:

Map DFUQuery sort field to @compressedSize and adjust size field population for compressed files.
Extend DFULogicalFile to include a FileSize field (v1.68+) and populate it in helpers.
Update React/ECLWatch UI formatting and tests to reflect “File Size” / “Compression” display.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
testing/unittests/dalitests.cpp	Adds coverage asserting DFUQuery returned size uses compressed size when appropriate.
esp/src/tests/v9-files.spec.ts	Updates UI test expectations for renamed columns (“File Size”, “Compression”).
esp/src/src/Utility.ts	Extends `formatDecimal` to support prefix/postfix formatting.
esp/src/src-react/hooks/file.ts	Adds shared compression formatting logic for UI.
esp/src/src-react/components/MetricsPropertiesTables.tsx	Uses new `formatDecimal` prefix/postfix for stddev display.
esp/src/src-react/components/LogicalFileSummary.tsx	Replaces PercentCompressed with computed Compression display.
esp/src/src-react/components/IndexFileSummary.tsx	Updates percent formatting to use new `formatDecimal` signature.
esp/src/src-react/components/Files.tsx	Updates columns to “File Size” and “Compression” and adjusts size/compression formatting.
esp/src/eclwatch/DFUQueryWidget.js	Updates percent formatting to use new `formatDecimal` signature.
esp/services/ws_dfu/ws_dfuService.cpp	Adds legacy sort mapping for compressed size attribute.
esp/services/ws_dfu/ws_dfuHelpers.cpp	Populates new `FileSize` (v1.68+) based on available fields.
esp/scm/ws_dfu_common.ecm	Adds `FileSize` to `DFULogicalFile` (min_ver 1.68).
dali/base/dadfs.cpp	Sets DFUQ “size” field to compressed size for compressed files and ensures origsize exists.

+export function formatDecimal(num: number, prefix: string = "", postfix: string = ""): string {
    if (!num) return "";
    if (isNaN(num)) return num.toString();
-    return d3FormatDecimal(num);
+    return prefix + d3FormatDecimal(num) + postfix;


asselitx · 2026-03-19T18:28:56Z

+                    if (row.FileSize) {
+                        return Utility.convertedSize(row.FileSize);
+                    }
+                    return Utility.convertedSize(row.IsCompressed ? row.CompressedFileSize : row.IntSize);
+                },
+                csvFormatter: (value, row) => {
+                    if (row.FileSize) {


Ok adding suggestion to batch.

+function formatRatio(isCompressed: boolean, compressedSize: number, totalSize: number): string {
+    if (!isCompressed || totalSize <= 0) return "";
+    const ratio = compressedSize / totalSize;
+    if (!isFinite(ratio)) return "";
+    return Utility.formatDecimal(100 - ratio * 100, "", "%");
+}


github-actions · 2026-03-19T16:58:14Z

🔄 Upmerge Test Results

Status: ✅ All branches merged successfully
PR: #21119 - HPCC-35694 Add support to WsDFU.DFUQuery to sort by "CompressedSize"
Base Branch: candidate-10.2.x
Test Time: 2026-03-19 16:28:40 UTC

✅ Successful Branches (1)

master
This comment was automatically generated by the upmerge test workflow.

GordonSmith · 2026-03-27T13:44:57Z

    [min_ver("1.63"), nil_remove] int64 MaxSkew;
    [min_ver("1.63"), nil_remove] int64 MinSkewPart;
    [min_ver("1.63"), nil_remove] int64 MaxSkewPart;
+    [min_ver("1.68")] int64 FileSize;


Does this need a version bump somewhere else (for the entire service)?

GordonSmith · 2026-03-27T13:47:43Z

+                label: nlsHPCC.FileSize,
                formatter: (value, row) => {
-                    return Utility.convertedSize(row.IntSize);
+                    if (row.FileSize) {


row.FileSize !== undefined would be preferable.

GordonSmith · 2026-03-27T13:48:40Z

+                    return Utility.convertedSize(row.IsCompressed ? row.CompressedFileSize : row.IntSize);
+                },
+                csvFormatter: (value, row) => {
+                    if (row.FileSize) {


row.FileSize !== undefined is preferable

github-actions · 2026-04-01T16:38:58Z

🔄 Upmerge Test Results

Status: ✅ All branches merged successfully
PR: #21119 - HPCC-35694 Add support to WsDFU.DFUQuery to sort by physical "File Size"
Base Branch: candidate-10.2.x
Test Time: 2026-04-01 16:38:44 UTC

✅ Successful Branches (1)

master
This comment was automatically generated by the upmerge test workflow.

asselitx · 2026-04-01T16:43:03Z

@GordonSmith fixed requested changes and confirmed unittests pass, but didn't touch your code that only copilot commented on. If you do agree with copilot I can make those adjustments also.

ghalliday · 2026-04-08T11:52:29Z

@asselitx please squash

ghalliday · 2026-04-08T11:57:14Z

Also the copilot comment about formatting 0 looks relevant.

asselitx · 2026-04-08T19:17:41Z

It looks like the formatDecimal (and others) intentionally return empty string for zero input to avoid cluttering the UI with 0.00% or similar looking output to help draw attention to the more significant nonzero values.

- Expose single field 'FileSize' for physical File Size. Regardless of compression status this field has the actual physical size on disk. - Add FileSize field to DFUQuery response to hold DFUSFSize. - Update ECL Watch to use FileSize when present. - Update DFULogicalFile structure and all services using it to version 1.69. - Update unittests to return all DFU fields needed to pass new tests. Signed-off-by: Gordon Smith<GordonJSmith@gmail.com> Signed-off-by: Terrence Asselin <terrence.asselin+copilot@lexisnexisrisk.com>

github-actions · 2026-04-08T19:24:44Z

🔄 Upmerge Test Results

Status: ✅ All branches merged successfully
PR: #21119 - HPCC-35694 Add support to WsDFU.DFUQuery to sort by physical "File Size"
Base Branch: candidate-10.2.x
Test Time: 2026-04-08 19:23:56 UTC

✅ Successful Branches (1)

master
This comment was automatically generated by the upmerge test workflow.

asselitx · 2026-04-09T14:54:58Z

@ghalliday and @GordonSmith squashed and approved

ghalliday · 2026-04-10T11:02:56Z

            lFile->setMinSkewPart(file.getPropInt64(getDFUQResultFieldName(DFUQResultField::minSkewPart)));
        }

+        if (version >= 1.68)


I think this should be compare >= 1.69

ghalliday

version question

asselitx force-pushed the hpcc-35694-dfustortby-compressedsize branch from a963176 to 4814af3 Compare March 19, 2026 16:24

asselitx marked this pull request as ready for review March 19, 2026 16:30

Copilot AI review requested due to automatic review settings March 19, 2026 16:30

Copilot AI reviewed Mar 19, 2026

View reviewed changes

Copilot started reviewing on behalf of asselitx March 19, 2026 17:15 View session

asselitx requested a review from GordonSmith March 19, 2026 18:30

asselitx changed the title ~~HPCC-35694 Add support to WsDFU.DFUQuery to sort by "CompressedSize"~~ HPCC-35694 Add support to WsDFU.DFUQuery to sort by physical "File Size" Mar 19, 2026

asselitx added Bug Notable Change A notable change that is likely to be worth highlighting in any version that is released. labels Mar 19, 2026

GordonSmith requested changes Mar 27, 2026

View reviewed changes

asselitx requested a review from GordonSmith April 1, 2026 16:43

GordonSmith mentioned this pull request Apr 2, 2026

HPCC-35694 DFUQuery sort by compressed size #20885

Closed

39 tasks

GordonSmith approved these changes Apr 7, 2026

View reviewed changes

asselitx force-pushed the hpcc-35694-dfustortby-compressedsize branch from ad10c43 to 9ba041a Compare April 8, 2026 19:23

ghalliday reviewed Apr 10, 2026

View reviewed changes

ghalliday requested changes Apr 10, 2026

View reviewed changes

Conversation

asselitx commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Type of change:

Checklist:

Smoketest:

Testing:

Uh oh!

github-actions bot commented Mar 19, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

asselitx Mar 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Mar 19, 2026

🔄 Upmerge Test Results

✅ Successful Branches (1)

Uh oh!

GordonSmith Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

GordonSmith Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

GordonSmith Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 1, 2026

🔄 Upmerge Test Results

✅ Successful Branches (1)

Uh oh!

asselitx commented Apr 1, 2026

Uh oh!

ghalliday commented Apr 8, 2026

Uh oh!

ghalliday commented Apr 8, 2026

Uh oh!

asselitx commented Apr 8, 2026

Uh oh!

github-actions bot commented Apr 8, 2026

🔄 Upmerge Test Results

✅ Successful Branches (1)

Uh oh!

asselitx commented Apr 9, 2026

Uh oh!

ghalliday Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

ghalliday left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

asselitx commented Mar 19, 2026 •

edited

Loading