[Bug][Relax] Import failures on PaddleOCR and FasterRCNN-style models: Squeeze axes, shape Gather, and dynamic TopK

[Nanmur]

Summary

During compatibility testing, two industrial-style ONNX graphs consistently fail at the TVM Relax ONNX importer stage, before tuning can start:

1. PP-OCRv6_tiny.onnx, exported from a PaddleOCR-style model, fails around Squeeze -> Transpose.
2. FasterRCNN-12.onnx, a detection graph with dynamic post-processing, fails around shape Gather and then dynamic TopK.

I would like to ask whether these are expected importer limitations in the current Relax ONNX frontend, and whether the preprocessing workarounds described below are recommended, or if there is a better official way to handle these graphs.

Environment

TVM version: 0.24.0
Python: 3.11.15
ONNX: 1.21.0
OS: Linux
Frontend used: tvm.relax.frontend.onnx.from_onnx

Case 1: PP-OCRv6_tiny Squeeze axes are lost before Transpose

The model is PP-OCRv6_tiny.onnx, opset 11. The failing pattern in the ONNX graph is:

Squeeze.0
  inputs  = ['p2o.pd_op.pool2d.0.0']
  outputs = ['p2o.pd_op.squeeze.0.0']
  attrs   = {'axes': [2]}

Transpose.0
  inputs  = ['p2o.pd_op.squeeze.0.0']
  outputs = ['p2o.pd_op.transpose.0.0']
  attrs   = {'perm': [0, 2, 1]}

Using either OCR runtime shape below gives the same failure:

from tvm.relax.frontend.onnx import from_onnx
import onnx

model = onnx.load("PP-OCRv6_tiny.onnx")
from_onnx(
    model,
    shape_dict={"x": [1, 3, 48, 128]},
    dtype_dict={"x": "float32"},
    opset=11,
    keep_params_in_input=False,
)

Observed error:

Error converting operator Transpose, with inputs: [R.squeeze(lv151, axis=None)]
ValueError: Transpose: number of axes in perm attribute (3) must equal the number of input tensor dimensions (2)

The ONNX node has Squeeze axes=[2], so the expected output rank should be 3 and Transpose perm=[0,2,1] should be valid. The TVM error shows R.squeeze(..., axis=None), which suggests that the importer may be ignoring the ONNX axes attribute for this opset/node form.

Related PaddleOCR Conv pattern

The same model also contains Paddle-style grouped/depthwise 1xK conv patterns:

node    = Conv.35
inputs  = ['p2o.pd_op.unsqueeze.1.0', 'p2o.pd_op.unsqueeze.0.0']
outputs = ['p2o.pd_op.depthwise_conv2d.9.0']
attrs   = {
  'dilations': [1, 1],
  'kernel_shape': [1, 5],
  'strides': [1, 1],
  'group': 160,
  'pads': [0, 2, 0, 2]
}

I experimented with two graph rewrites:

1. Normalize symmetric ONNX Conv pads from [top,left,bottom,right] to [h,w] for TVM.
2. Rewrite Unsqueeze(axis=2) -> Conv(1xK) -> Squeeze(axis=2) into an equivalent Conv1D form.

However, the first rewrite is not ONNX-spec-safe as a persisted ONNX model, because ONNX Runtime validation reports:

Node (Conv.0) Op (Conv) [ShapeInferenceError] Attribute pads has incorrect size

So I currently treat this as a TVM-only workaround and avoid saving it as a general runtime ONNX graph.

Case 2: FasterRCNN-12 dynamic detection post-processing

The model is FasterRCNN-12.onnx, opset 12. It contains a full detection post-processing graph:

TopK: 7
NonMaxSuppression: 85
RoiAlign: 4
Resize: 3
Shape: 117
Gather: 798

Raw import fails first at shape Gather:

Error converting operator Gather, with inputs: [R.shape([1, 3, 1, 1]), 2034]
AssertionError: Only constant indices supported for shape gather.

After freezing/stabilizing static shape subgraphs and importing the prepared runtime ONNX with:

from_onnx(
    model,
    shape_dict={"image": [3, 224, 224]},
    dtype_dict={"image": "float32"},
    opset=12,
    keep_params_in_input=False,
)

the next importer failure is dynamic TopK:

Error converting operator TopK, with inputs: [R.sigmoid(lv242), v_5635]
ValueError: TopK k must be a constant

Current workarounds in my pipeline

The pipeline currently does the following:

• Fold static shape subgraphs into initializers when this preserves ONNX Runtime validation.
• Fix missing or empty Resize ROI inputs.
• Rewrite static Split tensor inputs to Slice where TVM treats the second input as dynamic.
• Remove or bypass inference-time Dropout.
• Skip full detection post-processing graphs containing NonMaxSuppression + dynamic TopK.
• Skip Paddle-style degenerate grouped 1xK conv graphs when the rewrite would make the persisted ONNX invalid.

Questions

1. For opset 11 Squeeze with an axes attribute, should the Relax ONNX importer preserve the axes and emit R.squeeze(..., axis=[2]) instead of axis=None?
2. Is the shape Gather failure expected when the input is R.shape(...) and the index is a scalar constant-like value?
3. Is dynamic TopK k unsupported by design in Relax ONNX import, or is there a recommended way to keep it symbolic?
4. For full detection graphs with NonMaxSuppression + TopK, does the TVM team recommend splitting the graph before import, or should users expect full-graph import to work eventually?
5. Are graph-level rewrites such as static shape folding, static Split-to-Slice, and Unsqueeze-Conv-Squeeze -> Conv1D considered reasonable preprocessing for TVM, or is there a more official path?

Expected behavior

Ideally, TVM should import the valid ONNX graph or report a precise unsupported-pattern diagnostic. In the PP-OCRv6 case, the Squeeze axes=[2] attribute appears to be valid and should not reduce the tensor with axis=None.

Attachments

log:

[11:04:07] /home/perception/Nanmur/tvm/src/relax/ir/block_builder.cc:66: Warning: BlockBuilder destroyed with remaining blocks!
[11:04:07] /home/perception/Nanmur/tvm/src/relax/ir/block_builder.cc:66: Warning: BlockBuilder destroyed with remaining blocks!
[11:04:07] /home/perception/Nanmur/tvm/src/relax/ir/block_builder.cc:66: Warning: BlockBuilder destroyed with remaining blocks!
[11:04:08] /home/perception/Nanmur/tvm/src/relax/ir/block_builder.cc:66: Warning: BlockBuilder destroyed with remaining blocks!

==========================================================================================
PP-OCRv6_tiny raw ONNX -> TVM Relax from_onnx
==========================================================================================
shape_dict = {'x': [1, 3, 48, 1]}
dtype_dict = {'x': 'float32'}
Error converting operator Transpose, with inputs: [R.squeeze(lv151, axis=None)]
IMPORT_FAILED
ValueError: Transpose: number of axes in perm attribute (3) must equal the number of input tensor dimensions (2)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5173, in from_onnx
    self._construct_nodes(graph)
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5382, in _construct_nodes
    raise err
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5376, in _construct_nodes
    op = self._convert_operator(op_name, inputs, attr, self.opset)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5476, in _convert_operator
    sym = op_function(self.bb, inputs, attrs, [self._nodes, self._params])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 931, in _impl_v13
    raise ValueError(
ValueError: Transpose: number of axes in perm attribute (3) must equal the number of input tensor dimensions (2)
[Preprocess] Normalized 37 symmetric Conv pads from 4D to 2D for TVM.

==========================================================================================
PP-OCRv6_tiny after symmetric Conv pads normalization
==========================================================================================
shape_dict = {'x': [1, 3, 48, 1]}
dtype_dict = {'x': 'float32'}
Error converting operator Transpose, with inputs: [R.squeeze(lv151, axis=None)]
IMPORT_FAILED
ValueError: Transpose: number of axes in perm attribute (3) must equal the number of input tensor dimensions (2)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5173, in from_onnx
    self._construct_nodes(graph)
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5382, in _construct_nodes
    raise err
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5376, in _construct_nodes
    op = self._convert_operator(op_name, inputs, attr, self.opset)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5476, in _convert_operator
    sym = op_function(self.bb, inputs, attrs, [self._nodes, self._params])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 931, in _impl_v13
    raise ValueError(
ValueError: Transpose: number of axes in perm attribute (3) must equal the number of input tensor dimensions (2)
[Preprocess] Normalized 37 symmetric Conv pads from 4D to 2D for TVM.

==========================================================================================
PP-OCRv6_tiny after pads normalization + Unsqueeze-Conv-Squeeze to Conv1D rewrite
==========================================================================================
shape_dict = {'x': [1, 3, 48, 1]}
dtype_dict = {'x': 'float32'}
Error converting operator Transpose, with inputs: [R.squeeze(lv151, axis=None)]
IMPORT_FAILED
ValueError: Transpose: number of axes in perm attribute (3) must equal the number of input tensor dimensions (2)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5173, in from_onnx
    self._construct_nodes(graph)
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5382, in _construct_nodes
    raise err
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5376, in _construct_nodes
    op = self._convert_operator(op_name, inputs, attr, self.opset)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5476, in _convert_operator
    sym = op_function(self.bb, inputs, attrs, [self._nodes, self._params])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 931, in _impl_v13
    raise ValueError(
ValueError: Transpose: number of axes in perm attribute (3) must equal the number of input tensor dimensions (2)

==========================================================================================
PP-OCRv6_tiny representative grouped 1xK Conv nodes
==========================================================================================
node= Conv.35 inputs= ['p2o.pd_op.unsqueeze.1.0', 'p2o.pd_op.unsqueeze.0.0'] outputs= ['p2o.pd_op.depthwise_conv2d.9.0'] attrs= {'dilations': [1, 1], 'kernel_shape': [1, 5], 'strides': [1, 1], 'group': 160, 'pads': [0, 2, 0, 2]}

==========================================================================================
FasterRCNN-12 raw ONNX -> TVM Relax from_onnx
==========================================================================================
shape_dict = {'image': [3, 1, 1]}
dtype_dict = {'image': 'float32'}
Error converting operator Gather, with inputs: [R.shape([1, 3, 1, 1]), 2034]
IMPORT_FAILED
AssertionError: Only constant indices supported for shape gather.
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5173, in from_onnx
    self._construct_nodes(graph)
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5382, in _construct_nodes
    raise err
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5376, in _construct_nodes
    op = self._convert_operator(op_name, inputs, attr, self.opset)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5476, in _convert_operator
    sym = op_function(self.bb, inputs, attrs, [self._nodes, self._params])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 1086, in _impl_v13
    assert isinstance(indices, relax.Constant), (
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Only constant indices supported for shape gather.

==========================================================================================
FasterRCNN-12 prepared runtime_fp32.onnx -> TVM Relax from_onnx
==========================================================================================
shape_dict = {'image': [3, 224, 224]}
dtype_dict = {'image': 'float32'}
Error converting operator TopK, with inputs: [R.sigmoid(lv242), v_5635]
IMPORT_FAILED
ValueError: TopK k must be a constant
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5173, in from_onnx
    self._construct_nodes(graph)
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5382, in _construct_nodes
    raise err
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5376, in _construct_nodes
    op = self._convert_operator(op_name, inputs, attr, self.opset)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5476, in _convert_operator
    sym = op_function(self.bb, inputs, attrs, [self._nodes, self._params])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 4154, in _impl_v11[11:04:08] /home/perception/Nanmur/tvm/src/relax/ir/block_builder.cc:66: Warning: BlockBuilder destroyed with remaining blocks!

    raise ValueError("TopK k must be a constant")
ValueError: TopK k must be a constant

==========================================================================================

[OmarAzizi]

I opened #19966 to address the Squeeze axes-attribute issue (case 1, PP-OCRv6_tiny). The other two cases from this issue are still open.

Before I start on those, I'd like to check with the maintainers on the intended direction for dynamic TopK specifically, since there seem to be two different ways to solve it:

1. Support it natively so the importer/op can handle a symbolic k at compile time.
2. Require it to be resolved before import and expect users to preprocess the graph so k is a constant before calling from_onnx.

[Nanmur]

Thanks a lot for looking into this and for opening #19966 for the Squeeze axes-attribute case.

For dynamic TopK, native symbolic k support would be ideal from a user perspective, especially for importing full detection post-processing graphs. That said, I do not want to insist that TVM must support dynamic k natively if this does not fit the intended Relax/ONNX frontend design.

If the preferred direction is to require k to be resolved before import, that would also be acceptable, as long as the importer gives a clear diagnostic and ideally documents the recommended preprocessing path.

I also wanted to ask about the general preprocessing direction. In testing, some ONNX models that could not be imported by TVM directly could be imported after equivalent graph rewrites, for example:

• folding static Shape/Gather/Concat-style shape subgraphs into initializers;
• fixing missing or empty Resize roi inputs;
• rewriting static Split tensor inputs into Slice nodes when the split sizes are known;
• removing or bypassing inference-time Dropout;
• in some Paddle-style OCR graphs, considering rewrites around Unsqueeze -> Conv(1xK) -> Squeeze patterns.

Are these kinds of graph-level preprocessing steps considered a reasonable/recommended way to prepare ONNX graphs for TVM import, provided that the rewritten graph is validated against ONNX Runtime and numerical outputs are checked?

Or would the maintainers recommend a different approach, such as keeping the ONNX graph unchanged and improving the importer/op support case by case?

In particular, I am concerned about whether such preprocessing could accidentally change model semantics, layout assumptions, or numerical behavior. Any guidance on which rewrites are safe, which ones should be avoided, and which ones should instead be fixed inside TVM would be very helpful.