Skip to content

user provided bound for torchtrt compile when export dimension is unb…#4213

Open
apbose wants to merge 7 commits into
mainfrom
abose/torchTRT_dynamic_user_shape_bounds
Open

user provided bound for torchtrt compile when export dimension is unb…#4213
apbose wants to merge 7 commits into
mainfrom
abose/torchTRT_dynamic_user_shape_bounds

Conversation

@apbose
Copy link
Copy Markdown
Collaborator

@apbose apbose commented Apr 27, 2026

This PR validates user bounds against any finite exporter bounds, with three branches:

  • Upper overflow (user_max > exp_max) → ValueError. TRT engine profile follows the exporter; shapes above exp_max would be rejected at runtime anyway.
  • Lower underflow (user_min < exp_min) → ValueError, except the (user_min=1, exp_min=2) case which is just PyTorch's 0/1-specialization artifact (Dim(min=1) silently becomes 2 in ShapeEnv), this particular case emits a warning.
  • Subset (user range strictly inside exporter range) → engine profile is narrowed to the user's bounds (logged at INFO). The user's Input(min_shape, max_shape) is the contract for the TRT profile; a subset of the exporter's valid range is still correct, and a tighter profile lets TRT specialize more aggressively.
  • Dim.DYNAMIC (unbounded exporter upper) → no check, user fills the gap (the intended use).

@meta-cla meta-cla Bot added the cla signed label Apr 27, 2026
@github-actions github-actions Bot added component: tests Issues re: Tests component: core Issues re: The core compiler component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels Apr 27, 2026
@github-actions github-actions Bot requested a review from lanluo-nvidia April 27, 2026 19:27
@apbose apbose force-pushed the abose/torchTRT_dynamic_user_shape_bounds branch from 4d1ff02 to eec2c04 Compare April 27, 2026 23:35
@apbose
Copy link
Copy Markdown
Collaborator Author

apbose commented Apr 30, 2026

CI index put cases failing. Looking at it.

@apbose apbose force-pushed the abose/torchTRT_dynamic_user_shape_bounds branch from 0fe114e to 2905ff6 Compare April 30, 2026 01:05
@apbose apbose requested a review from narendasan May 6, 2026 21:09
@narendasan
Copy link
Copy Markdown
Collaborator

This PR is only for externally exported programs?

@apbose
Copy link
Copy Markdown
Collaborator Author

apbose commented May 6, 2026

@narendasan yeah it addresses (torch_tensorrt.dynamo.compile(exported_program, ...)). The flow for torch_tensorrt.compile(model, inputs=..) should be unaffected, since in that case export bounds and input bounds should be same

@apbose apbose force-pushed the abose/torchTRT_dynamic_user_shape_bounds branch from dee44b1 to 22728b3 Compare May 6, 2026 22:15
Copy link
Copy Markdown
Collaborator

@narendasan narendasan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should use the user bounds in the narrow case

@apbose apbose force-pushed the abose/torchTRT_dynamic_user_shape_bounds branch from 02d94e5 to 01798b0 Compare May 21, 2026 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla signed component: api [Python] Issues re: Python API component: core Issues re: The core compiler component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths component: tests Issues re: Tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants