[executorch][nvidia][tensorrt][19/n] Add SDPA converter

[shoumikhin]

Stack from ghstack (oldest at bottom):

• [executorch][nvidia][tensorrt][25/n] Share CUDA stream across TRT delegates #17936
• [executorch][nvidia][tensorrt][24/n] Add power and unary converters with model improvements #17935
• [executorch][nvidia][tensorrt][23/n] Add benchmark infrastructure #17934
• [executorch][nvidia][tensorrt][22/n] Add correctness tests #17933
• [executorch][nvidia][tensorrt][21/n] Add comparison and slice converters #17932
• [executorch][nvidia][tensorrt][20/n] Add bmm converter #17931
• -> [executorch][nvidia][tensorrt][19/n] Add SDPA converter #17930
• [executorch][nvidia][tensorrt][18/n] Add layer_norm and pixel_shuffle converters #17929
• [executorch][nvidia][tensorrt][17/n] Add embedding, expand, upsample converters #17928
• [executorch][nvidia][tensorrt][16/n] Enable MobileNetV3 model support #17927
• [executorch][nvidia][tensorrt][15/n] Add linear model support with addmm and permute_copy converters #17926
• [executorch][nvidia][tensorrt][14/n] Add additional converters (mul, sub, div, mm, relu) #17925
• [executorch][nvidia][tensorrt][13/n] Add examples, C++ runner and CI workflow #17924
• [executorch][nvidia][tensorrt][12/n] Add C++ runtime infrastructure #17923
• [executorch][nvidia][tensorrt][11/n] Complete preprocess integration with serialization #17922
• [executorch][nvidia][tensorrt][10/n] Add blob serialization format with I/O metadata #17921
• [executorch][nvidia][tensorrt][9/n] Implement preprocess basic implementation #17920
• [executorch][nvidia][tensorrt][8/n] Add TensorRTCompileSpec dataclass #17919
• [executorch][nvidia][tensorrt][7/n] Add converter utils and add converter #17918
• [executorch][nvidia][tensorrt][6/n] Add converter registry infrastructure #17917
• [executorch][nvidia][tensorrt][5/n] Integrate partitioner #17916
• [executorch][nvidia][tensorrt][4/n] Add operator support base class #17915
• [executorch][nvidia][tensorrt][3/n] Add TensorRT partitioner stub #17914
• [executorch][nvidia][tensorrt][2/n] Add TensorRT backend stub #17913
• [executorch][nvidia][tensorrt][1/n] Add directory structure and initialization #17912

Add Scaled Dot-Product Attention (SDPA) converter to enable transformer-based attention layers.

Differential Revision: D93275047

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/17930

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 31 New Failures, 2 Unrelated Failures

As of commit e612445 with merge base 01d21fa ():

NEW FAILURES - The following jobs have failed:

• Lint / lintrunner (gh)
  >>> Lint for examples/nvidia/tensorrt/tests/test_export.py:
• Lint / lintrunner-mypy (gh)
  >>> Lint for install_utils.py:
• pull / test-coreml-bc-macos (macos-m2-stable) / macos-job (gh)
  Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '/Users/ec2-user/runner/_work/executorch/executorch/test-infra/.github/actions/check-disk-space'. Did you forget to run actions/checkout before running your local action?
• Test CUDA Builds / check-all-cuda-builds (gh)
  Process completed with exit code 1.
• Test CUDA Builds / export-model-cuda-artifact (mistralai, Voxtral-Mini-4B-Realtime-2602, quantized-int4-tile-packed) / linux-job (gh)
  RuntimeError: Command docker exec -t 7b17cf8d7a23497c4cc425b071c7ca5e52e7e55fa2c6dd8913a16316337b9fd1 /exec failed with exit code 1
• Test CUDA Builds / export-model-cuda-artifact (nvidia, parakeet-tdt, non-quantized) / linux-job (gh)
  RuntimeError: Command docker exec -t ed0529287b91e584cc4b0bf8bff6f30bee5cbc0a31abb39639d3b2dcb93320f5 /exec failed with exit code 1
• Test CUDA Builds / export-model-cuda-artifact (nvidia, parakeet-tdt, quantized-int4-weight-only) / linux-job (gh)
  RuntimeError: Command docker exec -t c8a111d9572c62390ac5f2afed5e44643d2c570b2f0506e99a947ad2b079eee2 /exec failed with exit code 1
• Test CUDA Builds / export-model-cuda-artifact (openai, whisper-small, quantized-int4-tile-packed) / linux-job (gh)
  RuntimeError: Command docker exec -t 17cdbbd7e0ba8ce7d88486206d7d33b6055b2f22e01d3100ec89b0cfd158b000 /exec failed with exit code 1
• Test CUDA Builds / export-model-cuda-artifact (Qwen, Qwen3-0.6B, non-quantized) / linux-job (gh)
  RuntimeError: Command docker exec -t dc20983c23584df363a6d2d9b3949a2b4366c97451b521684bb5da93dd7166ac /exec failed with exit code 1
• Test CUDA Builds / export-model-cuda-artifact (Qwen, Qwen3-0.6B, quantized-int4-tile-packed) / linux-job (gh)
  RuntimeError: Command docker exec -t bc92ad89d38c3512803a198d681c33f1c6da60bf71cac95d44f0b44ff559c0db /exec failed with exit code 1
• Test CUDA Builds / test-executorch-cuda-build-12.9 / linux-job (gh)
  RuntimeError: Command docker exec -t e5397a1ad9c34f0d09f7b47aa428f8012cf9320e7bfc6b57368e2537a4f64964 /exec failed with exit code 1
• Test Metal Backend / export-model-metal-artifact (mistralai, Voxtral-Mini-3B-2507, non-quantized) / macos-job (gh)
  Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '/Users/ec2-user/runner/_work/executorch/executorch/test-infra/.github/actions/check-disk-space'. Did you forget to run actions/checkout before running your local action?
• Test Metal Backend / export-model-metal-artifact (mistralai, Voxtral-Mini-3B-2507, quantized-int4-metal) / macos-job (gh)
  Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '/Users/ec2-user/runner/_work/executorch/executorch/test-infra/.github/actions/check-disk-space'. Did you forget to run actions/checkout before running your local action?
• Test Metal Backend / export-model-metal-artifact (mistralai, Voxtral-Mini-4B-Realtime-2602, quantized-int4-metal) / macos-job (gh)
  Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '/Users/ec2-user/runner/_work/executorch/executorch/test-infra/.github/actions/check-disk-space'. Did you forget to run actions/checkout before running your local action?
• Test Metal Backend / export-model-metal-artifact (nvidia, parakeet-tdt, non-quantized) / macos-job (gh)
  Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '/Users/ec2-user/runner/_work/executorch/executorch/test-infra/.github/actions/check-disk-space'. Did you forget to run actions/checkout before running your local action?
• Test Metal Backend / export-model-metal-artifact (nvidia, parakeet-tdt, quantized-int4-metal) / macos-job (gh)
  Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '/Users/ec2-user/runner/_work/executorch/executorch/test-infra/.github/actions/check-disk-space'. Did you forget to run actions/checkout before running your local action?
• Test Metal Backend / export-model-metal-artifact (openai, whisper-large-v3-turbo, non-quantized) / macos-job (gh)
  Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '/Users/ec2-user/runner/_work/executorch/executorch/test-infra/.github/actions/check-disk-space'. Did you forget to run actions/checkout before running your local action?
• Test Metal Backend / export-model-metal-artifact (openai, whisper-large-v3-turbo, quantized-int4-metal) / macos-job (gh)
  Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '/Users/ec2-user/runner/_work/executorch/executorch/test-infra/.github/actions/check-disk-space'. Did you forget to run actions/checkout before running your local action?
• Test Metal Backend / export-model-metal-artifact (openai, whisper-small, non-quantized) / macos-job (gh)
  Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '/Users/ec2-user/runner/_work/executorch/executorch/test-infra/.github/actions/check-disk-space'. Did you forget to run actions/checkout before running your local action?
• Test Metal Backend / export-model-metal-artifact (openai, whisper-small, quantized-int4-metal) / macos-job (gh)
  Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '/Users/ec2-user/runner/_work/executorch/executorch/test-infra/.github/actions/check-disk-space'. Did you forget to run actions/checkout before running your local action?
• Test Metal Backend / test-executorch-metal-build / macos-job (gh)
  Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '/Users/ec2-user/runner/_work/executorch/executorch/test-infra/.github/actions/check-disk-space'. Did you forget to run actions/checkout before running your local action?
• Test Metal Backend / test-metal-backend-modules / macos-job (gh)
  Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '/Users/ec2-user/runner/_work/executorch/executorch/test-infra/.github/actions/check-disk-space'. Did you forget to run actions/checkout before running your local action?
• Test TensorRT Backend / check-all-tensorrt-tests (gh)
  ERROR: TensorRT build test failed!
• Test TensorRT Backend / test-models-tensorrt-cpp (add) / linux-job (gh)
  RuntimeError: Command docker exec -t d53021dff2b8de3dbb8efed972c7f093b1654bdb13835914564fb60406748f51 /exec failed with exit code 1
• Test TensorRT Backend / test-models-tensorrt-cpp (linear) / linux-job (gh)
  RuntimeError: Command docker exec -t 007887c8e0e7b07435841fb83e18d86ad98339a78c1056810ce94a3eff68e43d /exec failed with exit code 1
• Test TensorRT Backend / test-models-tensorrt-cpp (mul) / linux-job (gh)
  RuntimeError: Command docker exec -t 8aa891c0f0d285ea6cf5e6f29366ffe5af39ee28a1047fff0a83727b094613c9 /exec failed with exit code 1
• Test TensorRT Backend / test-models-tensorrt-python (add) / linux-job (gh)
  RuntimeError: Command docker exec -t dee154e41d812a852ad40f8e30e771f3815e06621f766100e0b174bbf91581bb /exec failed with exit code 1
• Test TensorRT Backend / test-models-tensorrt-python (linear) / linux-job (gh)
  RuntimeError: Command docker exec -t 6ac006b556c8bbf7657bcd14f25db341a7cbf311188d11c49824a6f3f1cd08ea /exec failed with exit code 1
• Test TensorRT Backend / test-models-tensorrt-python (mul) / linux-job (gh)
  RuntimeError: Command docker exec -t 0319f44e6622d4040811371b3e2c79b3fde3b8eed5d0f55563c488e3955713e3 /exec failed with exit code 1
• Test TensorRT Backend / test-tensorrt-build / linux-job (gh)
  RuntimeError: Command docker exec -t 4061dfecf0201412b7501048f5c9a9687f8715d1230df1b52d6b0f9e0bd91409 /exec failed with exit code 1
• Test TensorRT Backend / unittest-tensorrt / linux-job (gh)
  RuntimeError: Command docker exec -t 8a102c2044d80fb620750cedbab01430bab045d716c4fe8c6efef15c73f64ba4 /exec failed with exit code 1

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / macos / macos-job (gh) (trunk failure)
  AttributeError: '_OpNamespace' 'mkldnn' object has no attribute '_is_mkldnn_acl_supported'
• pull / unittest-editable / macos / macos-job (gh) (trunk failure)
  AttributeError: '_OpNamespace' 'mkldnn' object has no attribute '_is_mkldnn_acl_supported'

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.