Skip to content

[Relax] Legalize shape_to_tensor to device kernel#19957

Merged
tlopex merged 1 commit into
apache:mainfrom
guan404ming:fix-19925-shape-to-tensor-device
Jul 12, 2026
Merged

[Relax] Legalize shape_to_tensor to device kernel#19957
tlopex merged 1 commit into
apache:mainfrom
guan404ming:fix-19925-shape-to-tensor-device

Conversation

@guan404ming

@guan404ming guan404ming commented Jul 6, 2026

Copy link
Copy Markdown
Member

Why

Fixes #19925. relax.shape_to_tensor had no legalization, so it always lowered to the host packed func relax.run.shape_to_tensor, producing a CPU tensor regardless of the target device.

How

  • Register a legalization that emits the shape values as a call_tir TE kernel, passing symbolic dims via tir_vars.
  • Fall back to the packed func when the shape values are unknown (ShapeStructInfo without values).

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.

@guan404ming guan404ming marked this pull request as ready for review July 6, 2026 12:17
@guan404ming

Copy link
Copy Markdown
Member Author

/gemini review

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.

@guan404ming guan404ming force-pushed the fix-19925-shape-to-tensor-device branch from ea19c4d to f34ed21 Compare July 7, 2026 03:02
@guan404ming guan404ming requested a review from tlopex July 10, 2026 01:47

@tlopex tlopex left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tested the symbolic case and it fails in the full build pipeline.

This legalization produces a call_tir with no tensor inputs and non-empty tir_vars, e.g. [m, n]. FoldConstant treats the empty tensor argument tuple as constant and attempts to execute the generated PrimFunc without
passing m and n, resulting in:

TypeError: Expected 3 arguments when calling:
  tir_function(output, m, n)

A minimal reproducer is:

def main(x: R.Tensor(("m", "n"), "float32")):
    return R.shape_to_tensor(R.shape_of(x))

relax.build(mod, target="cuda")

The current symbolic and mixed tests only run LegalizeOps, so they do not exercise this failure. Could we make FoldConstant skip call_tir calls with non-empty symbolic tir_vars (or otherwise handle those scalar arguments),
and add an end-to-end build test?

@guan404ming guan404ming force-pushed the fix-19925-shape-to-tensor-device branch from f34ed21 to 1a5a45d Compare July 10, 2026 07:24
@guan404ming guan404ming force-pushed the fix-19925-shape-to-tensor-device branch from 1a5a45d to 39a1f1d Compare July 10, 2026 07:37
@guan404ming

Copy link
Copy Markdown
Member Author

Thanks, reproduced exactly as described. The llvm pipeline doesn't run FoldConstant, which is why the existing tests missed it.

Fixed by bailing out when call_tir carries tir_vars (args.size() > 2). Added test_call_tir_with_tir_vars_not_folded. I kept it as a FoldConstant unit test rather than an end-to-end relax.build(..., target="cuda") test so it doesn't need a CUDA CI node, happy to add the build test if you'd prefer.

@guan404ming guan404ming requested a review from tlopex July 10, 2026 07:51
@guan404ming guan404ming reopened this Jul 11, 2026

@tlopex tlopex left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tlopex tlopex merged commit 6cd73cd into apache:main Jul 12, 2026
11 checks passed
@guan404ming guan404ming deleted the fix-19925-shape-to-tensor-device branch July 12, 2026 04:01
@guan404ming

Copy link
Copy Markdown
Member Author

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] [Relax ONNX frontend] Relax ONNX import can produce CPU shape tensor consumed by CUDA call_tir

2 participants