Optimal way of converting zarr array to numpy array

[aliaksei-chareshneu]

Dear all,

Could you tell me what is the optimal way of converting zarr array to numpy array?

Currently I am using:

np_arr = zarr_arr[...]

But this, as far as I understand, loads entire array into memory.

Are there better alternatives?

Best regards,
Aliaksei

• Value of zarr.__version__: 2.10.3
• Value of numcodecs.__version__: 0.9.1
• Version of Python interpreter: 3.9.0
• Operating system (Linux/Windows/Mac): Windows 11
• How Zarr was installed (e.g., "using pip into virtual environment", or "using conda"): using pip into virtual environment

[rabernat]

Thanks for your question @aliaksei-chareshneu!

as far as I understand, loads entire array into memory.

A numpy array is in memory, by definition. So if you want to convert your zarr array to numpy array, you have to load it into memory.

Perhaps you could say more about what you hope to achieve. What would "optimal" mean to you here?

[aliaksei-chareshneu]

@rabernat, thank you for the clarification. Well, I just think of reducing memory consumption when possible.
So if I need to use numpy array later on (e.g. to calculate max on it), there is no other way except [...]?

[rabernat]

Yes, there is another option! If you want to compute on your array but it's too big to fit in memory, you should use Dask Arrays. Dask can load directly from Zarr.