Skip to content

fix(array): reject setitem values that do not broadcast to the selection - #4456

Draft
d-v-b wants to merge 2 commits into
zarr-developers:mainfrom
d-v-b:fix/setitem-broadcast-check
Draft

d-v-b wants to merge 2 commits into
zarr-developers:mainfrom
d-v-b:fix/setitem-broadcast-check

Conversation

@d-v-b

@d-v-b d-v-b commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

🤖 AI text below 🤖

Writing a value whose shape does not broadcast to the selection now raises ValueError, as numpy does, instead of silently writing a truncated value; this also covers zarr.create(shape, data=...). Values that numpy broadcasts, such as a length-1 axis, a lower-rank value, or extra leading length-1 axes, are now written correctly for every selection type instead of raising or writing partial data.

Summary

import numpy as np, zarr
a = zarr.zeros((3,))
a[...] = np.arange(4)   # numpy raises; zarr wrote [0. 1. 2.] with no error

_set_selection passed the value to the codec pipeline unchecked, and the pipeline slices it with each chunk's out_selection. So a value longer than the selection was truncated, and values numpy accepts failed or wrote partial data depending on chunk layout, for example a (1,) value spanning several chunks, (3,) into (4, 3), a leading length-1 axis, or a (1,) value via vindex.

The value is now broadcast to the selection shape under numpy's assignment rules (_broadcast_to_selection) before any chunk is touched, raising numpy's could not broadcast input array from shape X into shape Y otherwise. 0-d values pass through unchanged so the pipeline's scalar fast paths still apply. Coordinate and mask selections broadcast to sel_shape and are then flattened, which replaces the ad-hoc flatten and element-count check in Array.set_coordinate_selection.

For reviewers

  • A broadcast value reaches the codecs as a read-only, stride-0 view on the complete-chunk path. The test suite plus a manual check with v2 numcodecs compressors/filters (Blosc+Delta, Zlib, F order) and v3 Blosc found no codec that rejects it, but this is the part most worth a second look.
  • The coordinate/mask error message changes from "Attempting to set a selection of N elements..." to numpy's wording. test_set_selection_rejects_value_with_wrong_rank was updated, and its docstring no longer describes an oindex write that chunked arrays accepted and sharded arrays rejected, since both now reject it.
  • No existing issue was found for this.

Tests: one test covering the valid broadcasts against numpy (basic, integer axis, oindex, vindex, mask; chunked and sharded), one per error case (axis length mismatch; extra non-length-1 leading axis), plus the zarr.create(data=...) path. 37 of the 41 new cases fail on main; the 4 that pass are exact-shape controls.

Author attestation

  • I am a human, these are my changes, and I have reviewed and understood every change and can explain why each is correct.

TODO

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions
  • New/modified features documented in docs/user-guide/*.md
  • Changes documented as a new file in changes/
  • GitHub Actions have all passed
  • Test coverage is 100% (Codecov passes)

🤖 Generated with Claude Code

`_set_selection` passed the value to the codec pipeline unchecked, and the
pipeline slices it with each chunk's `out_selection`. A value longer than
the selection was silently truncated (`zeros((3,))[...] = arange(4)`), and
values numpy would broadcast (a length-1 axis, a lower-rank value, leading
length-1 axes) raised or wrote partial data depending on chunk layout.

The value is now broadcast to the selection shape under numpy's assignment
rules before any chunk is touched, raising numpy's "could not broadcast"
`ValueError` otherwise. Coordinate and mask selections broadcast to
`sel_shape` and are then flattened, which replaces the ad-hoc flatten and
element-count check in `Array.set_coordinate_selection`.

Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-opus-5-5
@codecov

codecov Bot commented Sep 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.44%. Comparing base (9677081) to head (48a513c).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4456      +/-   ##
==========================================
- Coverage   94.46%   94.44%   -0.02%     
==========================================
  Files          93       93              
  Lines       13234    13238       +4     
==========================================
+ Hits        12501    12503       +2     
- Misses        733      735       +2     
Files with missing lines Coverage Δ
src/zarr/core/array.py 98.18% <100.00%> (+0.09%) ⬆️

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant