Skip to content

Pass opt-in numeric CSV Boolean mode through scans and plan protobuf - #121

Merged
osipovartem merged 2 commits into
df55-list-scalar-field-metadatafrom
df55-csv-numeric-boolean-copy
Oct 8, 2026
Merged

osipovartem merged 2 commits into
df55-list-scalar-field-metadatafrom
df55-csv-numeric-boolean-copy

Conversation

@osipovartem

@osipovartem osipovartem commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Expose opt-in numeric Boolean CSV parsing on CsvFormat and CsvSource, forwarding to the Arrow reader only for callers that request it.
  • Preserve the mode across physical-plan protobuf serialization with an additive field (old plans decode to false).
  • Add source scan and protobuf round-trip tests.

Dependency

Validation

  • cargo +1.95.0 test --locked --offline -p datafusion-datasource-csv --lib: 12 passed.
  • cargo +1.95.0 test --locked --offline -p datafusion-proto --test proto_integration roundtrip_csv_scan_preserves_format_options: 1 passed.
  • cargo +1.95.0 clippy --locked --offline -p datafusion-datasource-csv -p datafusion-proto --all-targets: passed.

Independent read-only review: APPROVED after fixing the missing protobuf field (correctness, regressions, performance, API, tests).

@osipovartem
osipovartem merged commit b8eb540 into df55-list-scalar-field-metadata Oct 8, 2026
33 of 36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant