Skip to content

fix(group): create group helper arrays in the group's zarr format - #4412

Merged
d-v-b merged 5 commits into
zarr-developers:mainfrom
d-v-b:fix/group-helpers-zarr-format
Sep 30, 2026
Merged

d-v-b merged 5 commits into
zarr-developers:mainfrom
d-v-b:fix/group-helpers-zarr-format

Conversation

@d-v-b

@d-v-b d-v-b commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

This PR fixes a silly bug where v2 groups would support the creation of v3 arrays. there's a deeper problem here (our group class isn't v2 or v3 flavored) but this PR fixes the superficial problem (the incorrect behavior).

🤖 AI text below 🤖

Group.empty, zeros, ones, full and their *_like versions now create arrays in the zarr format of the group. Previously a zarr v2 group got zarr v3 arrays, which were written into the group without becoming members of it. Creating an array like an array of the other zarr format (for example zarr.zeros_like(v2_array)) no longer fails. The new array uses the default codecs of its own format, and top-level *_like functions inherit the zarr format of the source array unless zarr_format is passed.

Cause

Group.empty/zeros/ones/full and their *_like versions forwarded **kwargs to the top-level creation functions without the group's zarr_format. A zarr v2 group therefore got zarr v3 arrays. The group lists only members of its own format, so each array was written into the group's path without becoming a member of it:

g = zarr.group(zarr.storage.MemoryStore(), zarr_format=2)
z = g.zeros(name="z", shape=(4,))
z.metadata.zarr_format   # 3 before this PR, 2 after
list(g.array_keys())     # [] before this PR, ['z'] after

The agent found this during the same **kwargs audit that produced #4408 and #4410. Group.create_array already passes zarr_format=self.metadata.zarr_format; these helpers did not.

Changes

  • AsyncGroup._member_zarr_format supplies the group's format to all eight helpers. The sync Group methods delegate to these. An explicit zarr_format that differs from the group's raises ValueError.
  • Passing the format exposed a second bug. Creating an array in one format "like" an array of the other always failed, because _like_args copied format-specific codec settings (compressor/filters/order for v2, codecs for v3). For example, zarr.zeros_like(v2_array) raised ValueError: compressor cannot be used for arrays with zarr_format 3 on main. _like_args now takes the target format and copies codec settings only when the formats match.
    • The existing test_group_array_like_creation[zarr2-...] cases passed on main only because the v2 group was silently getting v3 arrays.
  • Top-level empty_like/zeros_like/ones_like/full_like now inherit the zarr format of a zarr source array unless zarr_format is passed. That matches what "like" suggests, and zarr 2.x behaviour.
  • zarr_format is now a named, keyword-only parameter of the top-level empty_like/zeros_like/ones_like/full_like/open_like, sync and async, where these functions used to read it out of **kwargs. With kwargs.get("zarr_format"), an explicit zarr_format=None made _like_args inherit the source's format and copy its codecs, and then | kwargs merged the None back over that inherited format. create then chose the default format, so zarr.zeros_like(v2_array, zarr_format=None) raised. _like_args now puts the format it settles on into the arguments it returns, and open_like takes it back out so that open_array still looks for an existing array in either format.
  • open_like does not inherit the source's format. If it did, open_array would look for an existing array in only that format. A v2 source pointed at an existing v3 array would then write a second (v2) array at the same path, because the v2 existence check doesn't see zarr.json. open_like targets the requested or default format, as before. A missing target is now created with that format's default codecs instead of raising.

Tests

  • test_group_array_helpers_use_group_format: group format 2/3 × eight helpers × source format 2/3 × with or without an explicit matching zarr_format. Checks that the array has the group's format and is listed by the group.
  • test_group_array_helpers_other_format: the one error case, a mismatched zarr_format.
  • test_like_zarr_format: top-level *_like × source format × requested format (including an explicit zarr_format=None). Checks the resulting format, and that codecs are kept only when the formats match.
  • test_open_like_zarr_format: an existing target of either format is opened as-is, and a missing one is created in the default format.
  • test_like_args gains a zarr_format argument and cases for inheriting and for crossing formats. The returned arguments include the zarr format the function settled on.

On main, 41 of the new group and *_like cases and one open_like case fail. The full test suite passes locally, and so does mypy.

Not covered

ensure_no_existing_node(zarr_format=2) doesn't detect an existing v3 node, and vice versa, so an array of one format can still be created over a node of the other through other paths (see test_v2_and_v3_exist_at_same_path). This PR avoids reaching that through open_like but doesn't change the check itself.

🤖 Generated with Claude Code

Group.empty/zeros/ones/full and their *_like versions forwarded **kwargs
to the top-level creation functions without the group's zarr_format, so
a zarr v2 group got zarr v3 arrays. The group lists only members of its
own format, so each array was written into the group without becoming a
member of it. Pass the group's format, and raise when the caller asks
for another.

Creating an array in one format like an array of the other then failed,
because _like_args copied format-specific codec settings (compressor and
filters, or codecs). _like_args now takes the target format and copies
those settings only when the formats match. Top-level *_like functions
inherit the source's format unless another is requested; open_like does
not, because open_array would then look for an existing array in only
that format.

Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-opus-5-5
@codecov

codecov Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.48%. Comparing base (083adfc) to head (15dbc79).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4412      +/-   ##
==========================================
+ Coverage   94.46%   94.48%   +0.01%     
==========================================
  Files          93       93              
  Lines       13233    13245      +12     
==========================================
+ Hits        12501    12514      +13     
+ Misses        732      731       -1     
Files with missing lines Coverage Δ
src/zarr/api/asynchronous.py 96.41% <100.00%> (+0.10%) ⬆️
src/zarr/api/synchronous.py 94.36% <100.00%> (+1.40%) ⬆️
src/zarr/core/group.py 95.62% <100.00%> (+0.01%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@d-v-b d-v-b added this to the 3.4.1 milestone Sep 29, 2026
`empty_like`, `zeros_like`, `ones_like`, `full_like` and `open_like` read the
target format with `kwargs.get("zarr_format")` and then merged `kwargs` over
the arguments derived from the source array. An explicit `zarr_format=None`
therefore made `_like_args` inherit the source's format and copy its codecs,
after which the merge put `None` back and `create` chose the default format:
`zarr.zeros_like(v2_array, zarr_format=None)` raised because v2 compressor
settings reached a v3 array. `zarr_format` is now a keyword-only parameter,
and `_like_args` records the resolved format in the arguments it returns.

Assisted-by: ClaudeCode:claude-opus-5-5
@d-v-b
d-v-b merged commit 090e1b4 into zarr-developers:main Sep 30, 2026
39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant