Skip to content

Semi aggregated query for hardware details summary endpoint#1832

Open
alanpeixinho wants to merge 1 commit intokernelci:mainfrom
profusion:fix/improve-hardware-details-summary-performance
Open

Semi aggregated query for hardware details summary endpoint#1832
alanpeixinho wants to merge 1 commit intokernelci:mainfrom
profusion:fix/improve-hardware-details-summary-performance

Conversation

@alanpeixinho
Copy link
Copy Markdown
Contributor

@alanpeixinho alanpeixinho commented Mar 30, 2026

Description

This implements performance improvement on the Hardware details summary endpoint.

  • Semi aggregating data on query to reduce number of returned rows, while allowing for filtering and detailed aggregation.
  • Ignoring fields not used on the frontend (failed_reasons).

How to test

  • Open the dashboard.
  • Go to Hardware page.
  • Select any available hardware.
  • The presented information should match previous versions (and staging/production).
  • The page should load fast even for cases with many instances of builds/boots and tests.

@MarceloRobert MarceloRobert added Backend Most or all of the changes for this issue will be in the backend code. Queries Issue that involves modifying some DB query labels Mar 31, 2026
@alanpeixinho alanpeixinho force-pushed the fix/improve-hardware-details-summary-performance branch 10 times, most recently from 9c304aa to 11b5c99 Compare April 2, 2026 18:52
@alanpeixinho alanpeixinho marked this pull request as ready for review April 2, 2026 18:53
@alanpeixinho alanpeixinho force-pushed the fix/improve-hardware-details-summary-performance branch 2 times, most recently from 162a78e to 93f201f Compare April 6, 2026 17:18
Copy link
Copy Markdown
Collaborator

@MarceloRobert MarceloRobert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like there's some discrepancy between the listing and the details, specially with larger hardware such as kubernetes

Comment on lines -71 to -74
self.origin: str = None
self.start_datetime: datetime = None
self.end_datetime: datetime = None
self.selected_commits: Dict[str, str] = None
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO you should keep at least these values since they are modified by unstable_parse_post_body and also won't have a typehint when used in the post method otherwise.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great point.

Comment on lines +426 to +439
summary = Summary(
builds=builds_summary, boots=boots_summary, tests=tests_summary
)
commons = HardwareCommon(trees=all_trees, compatibles=all_compatibles)
filters = HardwareDetailsFilters(
all=all_filters,
builds=builds_filters,
boots=boots_filters,
tests=tests_filters,
)

valid_response = HardwareDetailsSummaryResponse(
summary=Summary(
builds=self.builds_summary,
boots=self.boots_summary,
tests=self.tests_summary,
),
filters=HardwareDetailsFilters(
all=GlobalFilters(
configs=self.global_configs,
architectures=self.global_architectures,
compilers=self.global_compilers,
),
builds=LocalFilters(
issues=list(self.unfiltered_build_issues),
has_unknown_issue=self.unfiltered_uncategorized_issue_flags[
"build"
],
origins=sorted(self.unfiltered_origins["build"]),
labs=sorted(self.unfiltered_labs["build"]),
),
boots=HardwareTestLocalFilters(
issues=list(self.unfiltered_boot_issues),
platforms=list(self.unfiltered_boot_platforms),
has_unknown_issue=self.unfiltered_uncategorized_issue_flags[
"boot"
],
origins=sorted(self.unfiltered_origins["boot"]),
labs=sorted(self.unfiltered_labs["boot"]),
),
tests=HardwareTestLocalFilters(
issues=list(self.unfiltered_test_issues),
platforms=list(self.unfiltered_test_platforms),
has_unknown_issue=self.unfiltered_uncategorized_issue_flags[
"test"
],
origins=sorted(self.unfiltered_origins["test"]),
labs=sorted(self.unfiltered_labs["test"]),
),
),
common=HardwareCommon(
trees=trees,
compatibles=self.compatibles,
),
)
except ValidationError as e:
return Response(data=e.json(), status=HTTPStatus.INTERNAL_SERVER_ERROR)
valid_response = HardwareDetailsSummaryResponse(
summary=summary, filters=filters, common=commons
)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd wrap all this in a try except just in case we get a ValidationError

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct, as it is, I am changing behavior unnecessarily

@MarceloRobert
Copy link
Copy Markdown
Collaborator

Good code though, very organized 👍

@alanpeixinho alanpeixinho force-pushed the fix/improve-hardware-details-summary-performance branch 2 times, most recently from 409f8ed to 0002e6c Compare April 6, 2026 22:49
@alanpeixinho alanpeixinho force-pushed the fix/improve-hardware-details-summary-performance branch from 0002e6c to a19b158 Compare April 7, 2026 20:26
@alanpeixinho
Copy link
Copy Markdown
Contributor Author

Seems like there's some discrepancy between the listing and the details, specially with larger hardware such as kubernetes

I happened due to a misunderstanding in the filtering of dummy builds. Should be correct now.

"base_hardware, filters",
[
(ASUS_HARDWARE, {"config_name": "defconfig+kcidebug+x86-board"}),
(ASUS_HARDWARE, {"config_name": "defconfig"}),
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check if this is change is correct.

"base_hardware, filters",
[
(ASUS_HARDWARE, {"architecture": "i386"}),
(ASUS_HARDWARE, {"architecture": "asus-CM1400CXA-dalboz"}),
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check if this is change is correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Backend Most or all of the changes for this issue will be in the backend code. Queries Issue that involves modifying some DB query

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants