ROCM-26020: read render-node-opening GPU config once at create time; …#80
Merged
sarat-k merged 1 commit intoJul 21, 2026
Merged
Conversation
bhatturu
force-pushed
the
fix/rocm-26020-skip-suspended-render-open
branch
4 times, most recently
from
July 21, 2026 15:11
cea5062 to
d290b08
Compare
rsrikanth86
reviewed
Jul 21, 2026
| g_gpu_metrics[gpu_handle] = metrics_info; | ||
| } | ||
| } | ||
| // overdrive/perf/power-cap read once at create (smi_gpu_fill_config_ via |
Contributor
There was a problem hiding this comment.
say that the cached value is updated whenever value is updated
rsrikanth86
reviewed
Jul 21, 2026
|
|
||
| // config read once at create; keeps the per-GpuGet path off the render node | ||
| static void | ||
| smi_gpu_fill_config_ (aga_gpu_handle_t gpu_handle, aga_gpu_spec_t *spec, |
Contributor
There was a problem hiding this comment.
init_config_. Also fix the comment format for the function. Mention clearly the values are cached and cached values are updated during updates
bhatturu
force-pushed
the
fix/rocm-26020-skip-suspended-render-open
branch
3 times, most recently
from
July 21, 2026 20:49
01d867c to
e93a6fa
Compare
…drop UBB node power On discrete RDNA/Navi GPUs with functional PCIe runtime PM, an idle GPU runtime-suspends to D3. Several amdsmi info APIs called per GpuGet open /dev/dri/renderD* (amdsmi_get_gpu_overdrive_level via the RSMI DRM path in smi_gpu_fill_spec, the voltage-curve / enumeration-id reads in smi_gpu_fill_status, and amdsmi_get_node_handle -> amdsmi_get_gpu_asic_info in smi_gpu_fill_stats). The amdgpu DRM .open fop runs pm_runtime_get_sync(), forcing a D3->D0 resume just to read static config; the SMU then reports a stale-high gfx_activity/clock for ~9-15s (the cold-read spike). Move the render-node-opening static reads (overdrive/perf/power-cap, voltage curve, enumeration ids) out of the per-GpuGet path into smi_gpu_init_immutable_attrs (read once at create). The GPUUpdate handler writes a successful set back into the cached spec so config changes still surface without an agent restart. The per-query path now only issues amdsmi_get_gpu_metrics_info, which reads gpu_metrics sysfs and returns BUSY (no wake) on a suspended GPU. Also drop the UBB node power / node-power-cap reads (amdsmi_get_power_info ubb_power, amdsmi_get_node_handle, amdsmi_get_npm_info) from both the baremetal (amdsmi) and SR-IOV (gimamdsmi) paths: amdsmi_get_node_handle opens the render node (another idle waker), and the UBB metrics were unreliable. The gimamdsmi power_info call is retained for package power and voltages. No-op on Instinct MI2xx/MI3xx, which have no independent GPU runtime suspend. Verified on Navi48 (8x R9700S, runpm=1): suspended GPUs now show 0 render-node opens and 0 wakes on a GpuGet (was 24 opens / 8 wakes), no cold-read spike (gfx 0-6, clock ~15-50MHz vs stale 60-77% / 1800MHz); active and under-load values unchanged; no ubb metrics emitted; no crash.
bhatturu
force-pushed
the
fix/rocm-26020-skip-suspended-render-open
branch
from
July 21, 2026 22:00
e93a6fa to
653c975
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…drop UBB node power
On discrete RDNA/Navi GPUs with functional PCIe runtime PM, an idle GPU runtime-suspends to D3. Several amdsmi info APIs called per GpuGet open /dev/dri/renderD* (amdsmi_get_gpu_overdrive_level via the RSMI DRM path in smi_gpu_fill_spec, the voltage-curve / enumeration-id reads in smi_gpu_fill_status, and amdsmi_get_node_handle -> amdsmi_get_gpu_asic_info in smi_gpu_fill_stats). The amdgpu DRM .open fop runs pm_runtime_get_sync(), forcing a D3->D0 resume just to read static config; the SMU then reports a stale-high gfx_activity/clock for ~9-15s (the cold-read spike).
Move the render-node-opening static reads (overdrive/perf/power-cap, voltage curve, enumeration ids) out of the per-GpuGet path into smi_gpu_init_immutable_attrs (read once at create). The GPUUpdate handler writes a successful set back into the cached spec so config changes still surface without an agent restart. The per-query path now only issues amdsmi_get_gpu_metrics_info, which reads gpu_metrics sysfs and returns BUSY (no wake) on a suspended GPU.
Also drop the UBB node power / node-power-cap reads (amdsmi_get_power_info ubb_power, amdsmi_get_node_handle, amdsmi_get_npm_info) from both the baremetal (amdsmi) and SR-IOV (gimamdsmi) paths: amdsmi_get_node_handle opens the render node (another idle waker), and the UBB metrics were unreliable. The gimamdsmi power_info call is retained for package power and voltages.
No-op on Instinct MI2xx/MI3xx, which have no independent GPU runtime suspend.
Verified on Navi48 (8x R9700S, runpm=1): suspended GPUs now show 0 render-node opens and 0 wakes on a GpuGet (was 24 opens / 8 wakes), no cold-read spike (gfx 0-6, clock ~15-50MHz vs stale 60-77% / 1800MHz); active and under-load values unchanged; no ubb metrics emitted; no crash.
Motivation
Technical Details
Test Plan
Test Result
Submission Checklist