Skip to content

NIFI-16159 Add support for FileResourceService in PutFile - #11495

Open
shubhluck wants to merge 1 commit into
apache:mainfrom
shubhluck:NIFI-16159
Open

NIFI-16159 Add support for FileResourceService in PutFile#11495
shubhluck wants to merge 1 commit into
apache:mainfrom
shubhluck:NIFI-16159

Conversation

@shubhluck

Copy link
Copy Markdown
Contributor

Summary

NIFI-16159

Add support for FileResourceService in PutFile, matching the Resource Transfer Source / File Resource Service pattern already available in PutS3Object (NIFI-12642), PutGCSObject (NIFI-12643), and PutHDFS (NIFI-12801).

PutFile currently writes only the incoming FlowFile's content to the destination directory. Users who want to stage a file already on the local filesystem (or reachable via a custom FileResourceService implementation) have to first pull it into a FlowFile with FetchFile/GetFile, which is wasteful for large local files.

Changes

  • Add Resource Transfer Source and File Resource Service property descriptors to PutFile, reusing ResourceTransferProperties from nifi-resource-transfer.
  • When Resource Transfer Source = File Resource Service, obtain a FileResource via ResourceTransferUtils.getFileResource(...) and stream its InputStream to the .<filename> temp file with StandardCopyOption.REPLACE_EXISTING (preserving the overwrite semantics of the original session.exportTo(...) call), then proceed with the existing rename / permissions / owner / group / lastModified logic.
  • When Resource Transfer Source = FlowFile Content (default) the behavior is byte-identical to today.
  • Add nifi-resource-transfer and nifi-file-resource-service-api (compile) and nifi-file-resource-service (test) to nifi-standard-processors/pom.xml.
  • Add TestPutFile#testPutFileFromLocalFile covering the FileResourceService path using StandardFileResourceService.

Backward compatibility

Resource Transfer Source defaults to FlowFile Content, so existing flows are unaffected. File Resource Service only appears in the UI when the source is set to File Resource Service (declared via dependsOn).

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Pull Request Tracking

  • Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
  • Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000
  • Pull request contains commits signed with a registered key indicating Verified status

Pull Request Formatting

  • Pull Request based on current revision of the main branch
  • Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

  • Build completed using ./mvnw clean install -P contrib-check
    • JDK 21
    • JDK 25

Licensing

  • New dependencies are compatible with the Apache License 2.0 according to the License Policy
  • New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

  • Documentation formatting appears as expected in rendered files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant