List index out of range accessing a remote Zarr

[stefanches7]

Describe the Bug

Expected Behavior

Load the data

Reproduction

from bioio import BioImage

# Load Zarr image from URL/link
zarr_url = "https://uk1s3.embassy.ebi.ac.uk/bia-integrator-data/S-BIAD845/409227a9-2f3f-421c-8086-b14253283e42/409227a9-2f3f-421c-8086-b14253283e42.zarr"

# Method 1: Direct URL loading
img = BioImage(zarr_url)

# Access image data
data = img.data  # Full image data as numpy array

Environment

bioio-3.0.0 (Conda)

[BrianWhitneyAI]

Hey @stefanches7! It looks like this file is not a .ome.zarr which has some special metadata requirements about the .zattrs file. This metadata is used for establishing axis and dask setup, so without them we cannot support reading the file with BioImage.

[stefanches7]

@BrianWhitneyAI that would be really strange because it is from IDR, and OME-NGFF validator runs it: https://ome.github.io/ome-ngff-validator/?source=https://uk1s3.embassy.ebi.ac.uk/bia-integrator-data/S-BIAD845/409227a9-2f3f-421c-8086-b14253283e42/409227a9-2f3f-421c-8086-b14253283e42.zarr . Could you please re-check?

cc @joshmoore @lubianat

[joshmoore]

@BrianWhitneyAI is this an issue of the HCS spec?

[toloudis]

Yeah it looks like this is bioio-ome-zarr not knowing anything about plate.

If you drill down to the actual images, you can probably load one of these from the right subdirectory (e.g. G/7/0):
https://uk1s3.embassy.ebi.ac.uk/bia-integrator-data/S-BIAD845/409227a9-2f3f-421c-8086-b14253283e42/409227a9-2f3f-421c-8086-b14253283e42.zarr/G/7/0

bioio in general doesn't know about these higher level structures like wells and plates. (A BioImage object would have no api to deal with it) We could possibly add something specifically for bioio-ome-zarr. Also this: ome/ngff#343

[joshmoore]

A valid simplification would be to map them to a series ("series" list) but it might still surprise consumers.

[BrianWhitneyAI]

Ah the plot thickens, thanks for the responses. Theres definitely some more going on here that I missed. My guess as to what is going on is that this file is for the 0.4.1 spec which includes the bioformats2raw.layout as a transitional organization HERE. Our Reader only supports 0.4.0 given the BioImage restrictions. This would let the file pass in the OME-NGFF validator. Im not super familiar with the bioformats2raw.layout.

[pgarrison]

From the spec:

Conforming readers:

• SHOULD make users aware of the presence of more than one image (i.e. SHOULD NOT default to only opening the first image);
• MAY use the "series" attribute in the "OME" group to determine a list of valid groups to display;
• MAY choose to show all images within the collection or offer the user a choice of images, as with HCS plates;
• MAY ignore other groups or arrays under the root of the hierarchy.

And:

Transitional metadata is added to the specification with the intention of removing it in the future. Implementations may be expected (MUST) or encouraged (SHOULD) to support the reading of the data, but writing will usually be optional (MAY).

If full support for bioformats2raw.layout is too complicated for the bioio API, maybe a barebones implementation would be to raise an exception that includes a list of paths like https://uk1s3.embassy.ebi.ac.uk/bia-integrator-data/S-BIAD845/409227a9-2f3f-421c-8086-b14253283e42/409227a9-2f3f-421c-8086-b14253283e42.zarr/G/7/0 and tells the user to adjust their code to use one of those. This could satisfy the SHOULDs.

[stefanches7]

Thanks @toloudis, I try to open a single image locally:

>>> img = BioImage("/dev/sdb/S-BIAD845/009bd3ab-eb79-4cf4-8a11-ad028b827c03/009bd3ab-eb79-4cf4-8a11-ad028b827c03.zarr/G/7/0/")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/miniforge3/envs/omexcavator/lib/python3.10/site-packages/bioio/bio_image.py", line 320, in __init__
    self._reader, self._plugin = self._get_reader(
  File "/home/ubuntu/miniforge3/envs/omexcavator/lib/python3.10/site-packages/bioio/bio_image.py", line 304, in _get_reader
    plugin = BioImage.determine_plugin(
  File "/home/ubuntu/miniforge3/envs/omexcavator/lib/python3.10/site-packages/bioio/bio_image.py", line 245, in determine_plugin
    raise biob.exceptions.UnsupportedFileFormatError(
bioio_base.exceptions.UnsupportedFileFormatError: BioImage does not support the image: '/dev/sdb/S-BIAD845/009bd3ab-eb79-4cf4-8a11-ad028b827c03/009bd3ab-eb79-4cf4-8a11-ad028b827c03.zarr/G/7/0/'. You may need to install an extra format dependency. See our list of known plugins in the bioio README here: https://github.com/bioio-devs/bioio for a list of known plugins. You can also call the 'bioio.plugins.dump_plugins()' method to report information about currently installed plugins or the 'bioio.plugin_feasibility_report(image)' method to check if a specific image can be handled by the available plugins.

Currently I have ome-tiff and ome-zarr plugins installed. Is something lacking?

UPD: could open renaming the tile directory .../G/7/0 to .../G/7/0.zarr. I suggest determining the "Zarr-ness" from the directory structure / .zgroup file presence instead; can open a PR for that.

[stefanches7]

Closed accidentally; could someone please reopen? The issue persists.