[ExecuTorch][WebGPU] Reuse pool for single-op scratch (-86.7 MiB)

[JCNTH]

Stack from ghstack (oldest at bottom):

• [ExecuTorch][WebGPU] Test coverage for the f16-accumulate steel GEMM #20801
• [ExecuTorch][WebGPU] f16-accumulate (pwdqf16acc) steel q4gsw prefill GEMM #20800
• [ExecuTorch][WebGPU] Test coverage for the packed-word-dequant f16 steel GEMM #20799
• [ExecuTorch][WebGPU] Packed-word-dequant f16 steel q4gsw prefill GEMM #20798
• -> [ExecuTorch][WebGPU] Reuse pool for single-op scratch (-86.7 MiB) #20797
• [ExecuTorch][WebGPU] Test coverage for the f16 KV cache #20773
• [ExecuTorch][WebGPU] Add opt-in native f16 KV cache (−280 MiB, default-OFF) #20772
• [ExecuTorch][WebGPU] Test coverage for the f16-multiply steel q4gsw GEMM #20753
• [ExecuTorch][WebGPU] Add f16-multiply variant of the steel q4gsw prefill GEMM #20752
• [ExecuTorch][WebGPU] Steel q4gsw prefill GEMM — test coverage #20731
• [ExecuTorch][WebGPU] Add 256-thread "steel" q4gsw prefill GEMM #20730
• [ExecuTorch][WebGPU] Request shader-f16 device feature (enablement, fail-open) #20729
• [ExecuTorch][WebGPU] Unit tests for the WGSL shader-variant codegen #20728
• [ExecuTorch][WebGPU] WGSL shader-variant codegen (vec1/vec4, fp16-ready) + rms_norm dedup #20727

Problem: WebGPUGraph::create_scratch_buffer keeps every fused-op scratch WGPUBuffer alive in scratch_buffers_ until graph teardown. The SDPA prefill workspace (attn_weights + attn_weights_softmax, each Hq*S*Cmax*4 bytes) and the FlashDecoding partials (part_o/part_ml) are allocated fresh in every decoder layer's lowering, so a 16-layer Llama holds ~16x copies of scratch that is only live during a single op — a large slice of the device compute/scratch footprint.

Solution: add an acquire/release reuse pool for SINGLE-OP-LIFETIME scratch. acquire_scratch() returns a free slot (best-fit, size in [n, 2n]) or creates one; the caller releases it at op-lowering scope exit via the RAII ScopedScratch guard. Across the layers the SDPA/FD scratch now recycles a small constant of buffers instead of Nx. Whole-graph-lifetime scratch stays on create_scratch_buffer unchanged.

Correctness: reuse is bit-identical because WebGPU/Dawn auto-inserts RAW hazard barriers between dispatches that share a storage buffer regardless of pass structure — the same guarantee the shipped mem_obj_id tensor aliasing already relies on. A still-in_use slot is never handed to a co-live requester (co-live safety). WEBGPU_NO_SCRATCH_POOL falls back to the per-call path for A/B.
@exported-using-ghexport

Differential Revision: D110948389

Differential Revision: D110948389

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20797

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 10 Pending, 1 Unrelated Failure

As of commit f0ccc41 with merge base f4b01a8 ():

NEW FAILURES - The following jobs have failed:

• pull / test-binary-size-linux-gcc / linux-job (gh)
  RuntimeError: Command docker exec -t d428c3c153adf46ff41a38953284a5ed543d5b42327dc5f8fe711a89542e0387 /exec failed with exit code 1
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t c708019a9f0a8d0e559e88c3f8e7a588a6c5e0d72da0e95579bb2f0c770c3575 /exec failed with exit code 1
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 510a57e9e2a72e93d1e3825fcbe39303f9335a6b948342d7c5335183a40e66d2 /exec failed with exit code 1

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / android / build-android (gh) (trunk failure)
  ImportError: /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/executorch/extension/training/pybindings/_training_lib.cpython-311-x86_64-linux-gnu.so: undefined symbol: _ZN3c104impl3cow23materialize_cow_storageERNS_11StorageImplE

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[SS-JIA]

Review automatically exported from Phabricator review in Meta.