Skip to content

Build with TensorRT 11 and abseil 20250814 (NVCC)#28586

Draft
mc-nv wants to merge 4 commits into
microsoft:mainfrom
mc-nv:mchornyi/TRI-704/onnx
Draft

Build with TensorRT 11 and abseil 20250814 (NVCC)#28586
mc-nv wants to merge 4 commits into
microsoft:mainfrom
mc-nv:mchornyi/TRI-704/onnx

Conversation

@mc-nv
Copy link
Copy Markdown
Contributor

@mc-nv mc-nv commented May 20, 2026

Description

Two build fixes needed to compile ORT against the current NVIDIA toolchain:

  1. TensorRT EP — TensorRT 11 compatibility

    • Gate IBuilder::platformHasFastFp16() / platformHasFastInt8() and IBuilderConfig::setInt8Calibrator(nullptr) calls on NV_TENSORRT_MAJOR < 11 (these APIs were removed in TRT 11).
    • Replace dead make_unique<IExecutionContext>() branches in PerThreadContext::UpdateTensorRTContext / GetTensorRTContext (TRT 11 makes IExecutionContext abstract) with a null-guard return and ORT_ENFORCE. No in-tree callers ever hit the removed paths.
  2. Abseil 20250814 — NVCC EDG frontend

    • NVCC's cudafe++ fails to parse IfRRef<int KQual>::AddPtr<K> constructs used by the lifetime-bound insert_or_assign overload sets in raw_hash_map.h and btree_container.h when they appear inside heavily macro-expanded template parameter lists. Plain g++ accepts the same code.
    • Add a top-level IfRRefAddPtr<T, Other> alias in common.h and route the six affected use-sites through it so the member-template lookup happens outside the surrounding dependent template-id.
    • Applied as a cmake patch under cmake/patches/abseil/absl_cuda_warnings.patch.

Motivation and Context

Needed to build ONNX Runtime against TensorRT 11 and the abseil version pulled in by recent CUDA toolchain updates. Plain g++ compiles fine; only NVCC needs the abseil workaround. The TRT 11 gates preserve existing behavior on TRT 10 and earlier.

mc-nv added 3 commits May 16, 2026 06:21
NVCC's EDG-based frontend (cudafe++) fails to parse the new
`IfRRef<int KQual>::AddPtr<K>` constructs used by the lifetime-bound
`insert_or_assign` overload sets in `raw_hash_map.h` and
`btree_container.h` when they appear inside heavily macro-expanded
template parameter lists. Plain g++ accepts the same code.

Add a top-level `IfRRefAddPtr<T, Other>` alias in `common.h` and route
the six affected use-sites through it so the member-template lookup
happens outside the surrounding dependent template-id.
TensorRT 11 removes several APIs the EP still calls:

- `IBuilder::platformHasFastFp16()` / `platformHasFastInt8()`
- `IBuilderConfig::setInt8Calibrator()`
- `IExecutionContext` is now abstract, so `make_unique<IExecutionContext>()`
  no longer compiles.

Gate the removed builder/config calls with `NV_TENSORRT_MAJOR < 11`; on
TRT 11+ we skip the fast-fp16/int8 platform probe (TRT no longer exposes
it) and the redundant `setInt8Calibrator(nullptr)`.

`PerThreadContext::UpdateTensorRTContext` and `GetTensorRTContext` had
dead-code branches that constructed a default `IExecutionContext` (no
in-tree callers ever hit them). Replace with a null-guard return and an
`ORT_ENFORCE` so the file compiles under TRT 11 without changing any
reachable behavior.
@mc-nv
Copy link
Copy Markdown
Contributor Author

mc-nv commented May 20, 2026

cc: @chilo-ms , @tianleiwu

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates ONNX Runtime’s build compatibility with newer NVIDIA toolchains by (1) gating/adjusting TensorRT EP code paths that were removed or changed in TensorRT 11, and (2) extending the existing Abseil CUDA/NVCC patch to work around a cudafe++ parsing issue introduced with Abseil 20250814.

Changes:

  • TensorRT EP: Guard removed TensorRT APIs behind NV_TENSORRT_MAJOR < 11 and remove dead execution-context construction paths (TRT 11 makes IExecutionContext abstract).
  • Abseil patch: Add IfRRefAddPtr alias and route a small set of IfRRef<...>::AddPtr<...> uses through it to avoid NVCC EDG frontend parse failures.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc Gates TensorRT APIs removed in TRT 11; tightens execution-context handling to avoid constructing abstract TRT types.
cmake/patches/abseil/absl_cuda_warnings.patch Adds an Abseil internal alias and updates a handful of template-argument sites to compile under NVCC.
Comments suppressed due to low confidence (1)

onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc:4002

  • The MSVC #pragma warning(push) scope here is only popped in the NV_TENSORRT_MAJOR < 11 branch, so TensorRT 11+ builds will keep warning 4996 disabled until some later (unrelated) pop. This is especially problematic because another warning scope is pushed later in the same function. Please ensure the warning scope opened here is popped unconditionally after the INT8 dynamic-range block (regardless of TensorRT major) before any subsequent #pragma warning(push).
#pragma warning(push)
#pragma warning(disable : 4996)
#endif
      // Set INT8 Per Tensor Dynamic range
#if NV_TENSORRT_MAJOR < 11

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 3580 to +3584
#pragma warning(push)
#pragma warning(disable : 4996)
#endif
// Set INT8 per tensor dynamic range
#if NV_TENSORRT_MAJOR < 11
@tianleiwu
Copy link
Copy Markdown
Contributor

@mc-nv, please run lintrunner -a to format the files.

@mc-nv
Copy link
Copy Markdown
Contributor Author

mc-nv commented May 21, 2026

This PR interfere with #28611

@mc-nv mc-nv marked this pull request as draft May 21, 2026 17:21
@chilo-ms
Copy link
Copy Markdown
Contributor

This PR interfere with #28611

Once that PR is merged, you can still add the abseil workaround here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants