Skip to content

Introduce implementation for FileType#3608

Open
brkyvz wants to merge 1 commit into
apache:masterfrom
brkyvz:fileType
Open

Introduce implementation for FileType#3608
brkyvz wants to merge 1 commit into
apache:masterfrom
brkyvz:fileType

Conversation

@brkyvz

@brkyvz brkyvz commented Jun 9, 2026

Copy link
Copy Markdown

Rationale for this change

Introduces the reference implementation for the new LogicalTypeAnnotation File as discussed in this design document

What changes are included in this PR?

Introduces a new LogicalTypeAnnotation called FILE. It enforces that a group with this type annotation have the following fields by name:

path 		 		STRING	NOT NULL
size	 	 		INT64			
offset				INT64      	
etag				STRING

Are these changes tested?

Yes, unit tested

Are there any user-facing changes?

Introduces a new LogicalTypeAnnotation called FILE

} else if (!LogicalTypeAnnotation.FileLogicalTypeAnnotation.OPTIONAL_FIELD_NAMES.contains(fieldName)) {
throw new IllegalArgumentException(
"FILE type group '" + name + "' contains unrecognized field '" + fieldName
+ "'. Valid fields are: path, size, offset, etag");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: stringify FileLogicalTypeAnnotation.OPTIONAL_FIELD_NAMES and use it in the error to avoid drift if we add new fields?

}
}

private static void validateFileTypeFields(String name, List<Type> fields) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where do we validate:

  1. If offset is present, size must be present
  2. offset and size must be non-negative (note spec doesn't technically say offset must be >= 0, but I left a suggestion on that change to add it

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add tests for validity of size/offset fields, both their values (>= 0) and valid/invalid combinations

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants