Skip to content

feat: add parquet reader - #145

Draft
georgeRobertson wants to merge 2 commits into
release_v010from
feature/gr-ndit-1761-add_parquet_reader
Draft

feat: add parquet reader#145
georgeRobertson wants to merge 2 commits into
release_v010from
feature/gr-ndit-1761-add_parquet_reader

Merge branch 'release_v010' of https://github.com/NHSDigital/data-val…

4bc4295
Select commit
Loading
Failed to load commit list.
SonarQubeCloud / SonarCloud Code Analysis failed Aug 26, 2026 in 28s

Quality Gate failed

Annotations

Check warning on line 52 in src/dve/core_engine/backends/implementations/spark/readers/parquet.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Specify both "master" and "appName" parameters to initialize this SparkSession.

See more on https://sonarcloud.io/project/issues?id=NHSDigital_data-validation-engine&issues=AaA-vPHEyrduliiMdUG7&open=AaA-vPHEyrduliiMdUG7&pullRequest=145

Check warning on line 62 in src/dve/core_engine/backends/implementations/duckdb/readers/parquet.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Complete the task associated to this "TODO" comment.

See more on https://sonarcloud.io/project/issues?id=NHSDigital_data-validation-engine&issues=AaA-vPCMyrduliiMdUG5&open=AaA-vPCMyrduliiMdUG5&pullRequest=145

Check warning on line 168 in src/dve/core_engine/backends/implementations/duckdb/contract.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Replace "list(...)[0]" with "next(iter(...))" to avoid materializing the entire iterable.

See more on https://sonarcloud.io/project/issues?id=NHSDigital_data-validation-engine&issues=AaA-vPG1yrduliiMdUG6&open=AaA-vPG1yrduliiMdUG6&pullRequest=145

Check warning on line 152 in src/dve/core_engine/backends/implementations/spark/contract.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Replace "list(...)[0]" with "next(iter(...))" to avoid materializing the entire iterable.

See more on https://sonarcloud.io/project/issues?id=NHSDigital_data-validation-engine&issues=AaA-vPH1yrduliiMdUG8&open=AaA-vPH1yrduliiMdUG8&pullRequest=145

Check warning on line 92 in src/dve/core_engine/backends/readers/utilities.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Complete the task associated to this "TODO" comment.

See more on https://sonarcloud.io/project/issues?id=NHSDigital_data-validation-engine&issues=AaA-vPIUyrduliiMdUG9&open=AaA-vPIUyrduliiMdUG9&pullRequest=145

Check warning on line 61 in src/dve/core_engine/backends/implementations/duckdb/readers/parquet.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Complete the task associated to this "TODO" comment.

See more on https://sonarcloud.io/project/issues?id=NHSDigital_data-validation-engine&issues=AaA-vPCMyrduliiMdUG4&open=AaA-vPCMyrduliiMdUG4&pullRequest=145