Skip to content

feat/codec classes in array config#3892

Open
d-v-b wants to merge 14 commits intozarr-developers:mainfrom
d-v-b:feat/codec-classes-in-array-config
Open

feat/codec classes in array config#3892
d-v-b wants to merge 14 commits intozarr-developers:mainfrom
d-v-b:feat/codec-classes-in-array-config

Conversation

@d-v-b
Copy link
Copy Markdown
Contributor

@d-v-b d-v-b commented Apr 9, 2026

This PR makes the changes necessary for adding the following features:

  • Explicitly declaring the codec classes available when reading an array without the global config.
  • Explicitly declaring the codec pipeline used when reading an array without the global config
  • Switching out the codec classes / codec pipeline for an existing array

We get these features by putting the codec classes and codec pipeline class in the array config, and adding these as explicit parameters to all the inner routines that were previously relying on the global config.

for context, our inner routines should never be relying on the global config. It's a mutable global variable that should only be used to creating default values at the boundary of zarr-python. Internally, each function should declare what it needs in its signature, without implicitly relying on the global config. These changes move us in that direction.

I am sensitive to this because I have been working on codecs / codec pipelines recently and the developer experience (changing things through the global config) is terrible. we need to fix this.

here's are some motivating example from the tests:

class FakeCodecPipeline(BatchedCodecPipeline): ...
store={}
arr = zarr.create_array(
    store, shape=(1,), dtype="uint8", config={"codec_pipeline_class": FakeCodecPipeline}
)
assert isinstance(arr.async_array.codec_pipeline, FakeCodecPipeline)

arr_2 = arr.with_config({"codec_pipeline_class": BatchedCodecPipeline})
assert isinstance(arr_2.async_array.codec_pipeline, BatchedCodecPipeline)
class FakeGzipCodec(GzipCodec): ...

store = {}  # type: ignore[var-annotated]
arr = zarr.create_array(store, shape=(1,), dtype="uint8", compressors=GzipCodec())
new_config: ArrayConfigRequest = {
    "codec_class_map": {**arr.config.codec_class_map, "gzip": FakeGzipCodec}
}

arr_2 = arr.with_config(new_config)
assert isinstance(arr_2.compressors[0], FakeGzipCodec)

arr_3 = zarr.open_array(store=store, config=new_config)
assert isinstance(arr_3.compressors[0], FakeGzipCodec)

@github-actions github-actions bot added the needs release notes Automatically applied to PRs which haven't added release notes label Apr 9, 2026
@github-actions github-actions bot removed the needs release notes Automatically applied to PRs which haven't added release notes label Apr 9, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 9, 2026

Codecov Report

❌ Patch coverage is 83.33333% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.86%. Comparing base (bd23984) to head (99967f3).

Files with missing lines Patch % Lines
src/zarr/core/array_spec.py 89.02% 9 Missing ⚠️
src/zarr/core/array.py 72.22% 5 Missing ⚠️
src/zarr/codecs/sharding.py 62.50% 3 Missing ⚠️
src/zarr/api/asynchronous.py 0.00% 1 Missing ⚠️
src/zarr/core/metadata/v2.py 50.00% 1 Missing ⚠️
src/zarr/core/metadata/v3.py 88.88% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3892      +/-   ##
==========================================
- Coverage   92.98%   92.86%   -0.13%     
==========================================
  Files          87       87              
  Lines       11246    11335      +89     
==========================================
+ Hits        10457    10526      +69     
- Misses        789      809      +20     
Files with missing lines Coverage Δ
src/zarr/api/synchronous.py 92.95% <ø> (ø)
src/zarr/api/asynchronous.py 92.88% <0.00%> (-0.34%) ⬇️
src/zarr/core/metadata/v2.py 88.50% <50.00%> (-0.52%) ⬇️
src/zarr/core/metadata/v3.py 92.53% <88.88%> (-0.24%) ⬇️
src/zarr/codecs/sharding.py 88.58% <62.50%> (-0.83%) ⬇️
src/zarr/core/array.py 97.20% <72.22%> (-0.44%) ⬇️
src/zarr/core/array_spec.py 93.33% <89.02%> (-6.67%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant