Skip to content

feat: add parquet reader - #145

Draft
georgeRobertson wants to merge 2 commits into
release_v010from
feature/gr-ndit-1761-add_parquet_reader
Draft

feat: add parquet reader#145
georgeRobertson wants to merge 2 commits into
release_v010from
feature/gr-ndit-1761-add_parquet_reader

Conversation

@georgeRobertson

@georgeRobertson georgeRobertson commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

TLDR of changes

Add parquet reader for duckdb and spark

Things left todo:

  • have the ability to define a schema in the dischema that is flattened model of struct/array of structs rather than the user having to define a field within an entity as complex (which would then require a business rule to unnest)

What kind of changes does this PR introduce?

Tick all that apply

  • fix: A bug fix. Correlates with PATCH in SemVer
  • feat: A new feature. Correlates with MINOR in SemVer
  • docs: Documentation only changes
  • style: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)
  • refactor: A code change that neither fixes a bug nor adds a feature
  • perf: A code change that improves performance
  • test: Adding missing or correcting existing tests
  • build: Changes that affect the build system or external dependencies (example scopes: pip, docker, npm)
  • ci: Changes to CI configuration files and scripts (example scopes: GitLabCI)

Please check if the PR fulfills these requirements

  • I have read and followed the Contributing guidance
  • Docs have been added / updated
  • Tests and Linting in the CI are passing
  • Changes have been reviewed and approved by a Project Maintainer

duckdb and spark parquet readers
@georgeRobertson
georgeRobertson changed the base branch from main to release_v010 August 26, 2026 15:28
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
4.2% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant