Upload SAF files in bounded chunks with progress before creating the import process - #6247
Open
bram-atmire wants to merge 1 commit into
Open
Upload SAF files in bounded chunks with progress before creating the import process#6247bram-atmire wants to merge 1 commit into
bram-atmire wants to merge 1 commit into
Conversation
…import process The Import Batch page now stages the SAF ZIP through the REST API's generic staged-upload resource instead of one multipart request, and starts the import with the JSON script invocation that references the staged upload. Each chunk carries the current JWT and CSRF token, a lost response is retried against the server's committed offset, and the upload can resume after signing in again with its original settings. While a file is retained or uploading the page confirms the file and target collection, shows an accessible progress bar with approximate upload speed and a distinct "starting import" phase, offers only cancellation, and navigates to the existing process page once the process exists. The URL-based import is unchanged. Relates to DSpace/DSpace#13103 and DSpace/DSpace#13106. Claude-Session: https://claude.ai/code/session_01HtPYmqfFzg7fmZgovGWP5A
bram-atmire
force-pushed
the
feature/saf-chunked-uploads
branch
from
September 12, 2026 10:00
e3dba83 to
4da84d0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
References
Description
The Import Batch page now uploads the SAF ZIP in bounded chunks through the REST API's new staging endpoints, shows an accessible progress bar with an approximate upload speed, and navigates to the existing process page once the import process has been created. The URL-based import is unchanged.
Instructions for Reviewers
List of changes in this PR:
StagedUploadService(new,core/data/staged-upload.service.ts), written against the generic/api/core/uploadsresource so the submission form can reuse it (Generic resumable uploads: stage large files in bounded chunks for scripts, submissions and other upload targets DSpace#13106, phase 2). A metadata preflight runs first, so an oversized file is rejected with413before any file data is sent. Chunks are built withFile.slice()and sent sequentially asPUTrequests through the normal JWT and CSRF interceptors. After a transient failure or an in-app navigation (for example a re-login), the upload resumes from the server's committed offset. Retries are bounded and limited to network errors,408,429,502,503,504, and a401once a renewed token is available. The caller supplies the handoff, which runs once after the last chunk is acknowledged; a repeated handoff after a lost response is answered by the REST API with the same result.ScriptDataService.invokeWithUploads: the JSON form ofPOST /api/system/scripts/{name}/processesthat attaches staged uploads as input files.BatchImportPageComponent. While a file is retained or uploading, the selection form is replaced by a confirmation of the file name and the target collection (or a note that each item's SAFcollectionsfile decides). A<progress>element with arole="status"label andaria-busy, a five-second rolling upload speed, a distinct "Upload complete. Starting import" phase after the last chunk is acknowledged, and navigation to/processes/{id}only once the process ID is known. While the upload runs the only action is a red Cancel button in the button row (disabled during the brief handoff); Back and Proceed return once the upload has stopped. A resume notice appears when an unfinished upload is retained. Notifications cover success, expired login, oversized file and generic failure.admin.batch-import.upload.*inen.json5.Points for reviewers:
uploadslink from the API root. Against a REST API without that link the page reports an error instead of falling back to the multipart request. If a silent fallback is preferred for mixed deployments, that is a small addition.How to test:
spring.servlet.multipart.max-file-size: the error shows immediately and the network tab contains only the metadataPOST.jwt.login.token.expirationon the backend and upload a large file with a small chunk size: later chunk requests carry the renewed token.Validation:
npm run lint(0 errors),npm run check-circ-deps, and the focused specssaf-upload.service.spec.tsandbatch-import-page.component.spec.tspass.Checklist
mainbranch of code (unless it is a backport or is fixing an issue specific to an older branch).npm run lintnpm run check-circ-deps)package.json), I've made sure their licenses align with the DSpace BSD License based on the Licensing of Contributions documentation. (No new dependencies.)https://claude.ai/code/session_01HtPYmqfFzg7fmZgovGWP5A