Skip to content

fix(storage-s3): support S3 client uploads over 5 GiB via multipart#17005

Open
codyparnell wants to merge 3 commits into
payloadcms:mainfrom
codyparnell:main
Open

fix(storage-s3): support S3 client uploads over 5 GiB via multipart#17005
codyparnell wants to merge 3 commits into
payloadcms:mainfrom
codyparnell:main

Conversation

@codyparnell

Copy link
Copy Markdown

Which branch?

Targets main (Payload v4).

What?

Fixes S3 client uploads failing for files larger than 5 GiB.
AWS S3 limits single PutObject uploads to 5 GiB. This PR ensures the S3 client upload handler automatically switches to multipart upload for larger files, and documents the related Payload config requirements.

Why?

With clientUploads: true, files upload directly from the browser to S3. That bypasses server/Vercel body-size limits, but two constraints still apply:

  1. S3 single-request limit (5 GiB) — files above this cannot use a single presigned PutObject URL.
  2. Payload upload.limits.fileSize — the signed-URL signer still enforces this limit for both single-part and multipart flows.
    Without multipart routing and clear configuration guidance, large client uploads fail (e.g. Exceeded file size limit on initiateMultipart or generateSignedURL).

How?

Client (S3ClientUploadHandler)

  • Files ≤ 5 GiB → existing single-part flow (generateSignedURL → S3 PUT)
  • Files > 5 GiB → multipart flow:
    • initiateMultipart
    • concurrent part uploads with retry
    • completeMultipart (with abortMultipart on failure)

Server (generateSignedURL handler)

  • Adds multipart signer actions: initiateMultipart, signMultipartPart, completeMultipart, abortMultipart
  • Rejects single-part signed URLs above 5 GiB with a clear error directing callers to multipart
  • Treats upload.limits.fileSize: Infinity as no signer-side cap
  • Prefixes size-limit errors with the action name (e.g. initiateMultipart: Exceeded file size limit) for easier debugging

Docs & tests

  • Documents client upload limits, the 5 GiB threshold, and upload.limits.fileSize in packages/storage-s3/README.md
  • Integration tests cover multipart initiate/complete/abort and related edge cases in test/storage-s3/client-uploads/

User-facing config requirement

Users uploading large files must raise the global limit to match their largest expected file:

export default buildConfig({
  upload: {
    limits: {
      fileSize: 7 * 1024 * 1024 * 1024, // e.g. 7 GiB
      // or Infinity to disable the signer-side cap
    },
  },
})

@codyparnell codyparnell requested a review from denolfe as a code owner June 15, 2026 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant