Skip to content

⬆️ Zarr V3 compatibility#904

Open
measty wants to merge 68 commits intodevelopfrom
fix-zarr-check
Open

⬆️ Zarr V3 compatibility#904
measty wants to merge 68 commits intodevelopfrom
fix-zarr-check

Conversation

@measty
Copy link
Copy Markdown
Collaborator

@measty measty commented Jan 16, 2025

Fixes the error related to zarr.errors.FSPathExistNotDir

It doesnt exist anymore since zarr v3.0.0, which instead just raises a standard FileNotFoundError

rather than checking for different error types in different versions of zarr, i've just removed the check for specific error type in favour of any errors

There are some other changes which we need to deal with from 3.0 also, as it has removed or changed zarr.LRUStoreCache, zarr.DirectoryStore, and zarr.SQLiteStore all of which tiatoolbox uses

We would also need to wait for tifffile to update to be compatible with zarr 3.0

See: https://zarr.readthedocs.io/en/latest/user-guide/v3_migration.html for more details

  • 🆙 Upgrade Zarr >=3.08, tifffile>=2025.5.21
  • Remove Python 3.10 as Zarr v3 has no support
  • Remove object_codec in dask.to_zarr
  • Refactor zarr.core.Array to zarr.Array
  • Refactor canvas_zarr.store.path to canvas_zarr.store.root
  • Refactor zarr.DirectoryStore to zarr.storage.LocalStore
  • tuple input for zarr.resize
  • Replace output.items() with output.members()
  • Remove ngff.sqlitestore as zarr.SQLiteStore is no longer supported.
  • Zarr path explicitly uses str key / i is replaced by key / str(i)

@measty measty changed the title fix zarr checking Zarr 3.0 compatibility Jan 16, 2025
@measty measty marked this pull request as draft January 16, 2025 14:11
@codecov
Copy link
Copy Markdown

codecov bot commented Jan 16, 2025

Codecov Report

❌ Patch coverage is 94.32624% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 99.69%. Comparing base (36ecfa6) to head (58762df).

Files with missing lines Patch % Lines
tiatoolbox/utils/misc.py 87.50% 1 Missing and 2 partials ⚠️
tiatoolbox/wsicore/wsireader.py 94.73% 2 Missing and 1 partial ⚠️
tiatoolbox/models/engine/multi_task_segmentor.py 94.28% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #904      +/-   ##
===========================================
- Coverage    99.88%   99.69%   -0.19%     
===========================================
  Files           85       85              
  Lines        11576    11627      +51     
  Branches      1513     1523      +10     
===========================================
+ Hits         11563    11592      +29     
- Misses           6       20      +14     
- Partials         7       15       +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@measty measty mentioned this pull request Jan 16, 2025
@shaneahmed shaneahmed added bug Something isn't working dependencies Pull requests that update a dependency file labels Jan 24, 2025
@shaneahmed
Copy link
Copy Markdown
Member

shaneahmed commented Mar 19, 2025

Zarr v3 is not compatible with tifffile cgohlke/tifffile#282 czbiohub-sf/iohub#292

@shaneahmed shaneahmed added the stale Old PRs/Issues which are inactive label Mar 21, 2025
@shaneahmed shaneahmed added this to the Release v1.7.0 milestone Mar 21, 2025
@shaneahmed
Copy link
Copy Markdown
Member

Zarr 3 is supported by tifffile. However, zarr 3 only supports Python 3.11+.

@shaneahmed shaneahmed changed the title Zarr 3.0 compatibility :Uparrow: Zarr V3 compatibility Mar 25, 2026
@shaneahmed shaneahmed changed the title :Uparrow: Zarr V3 compatibility ⬆️ Zarr V3 compatibility Mar 25, 2026
@shaneahmed
Copy link
Copy Markdown
Member

All the errors have been fixed except FsspecJsonWSIReader.

@shaneahmed
Copy link
Copy Markdown
Member

All the errors have been fixed except FsspecJsonWSIReader.

The updates to engines output breaks, test_hovernet_on_box which also needs to be fixed.

@aacic
Copy link
Copy Markdown
Collaborator

aacic commented Mar 26, 2026

All the errors have been fixed except FsspecJsonWSIReader.

@shaneahmed I was able to make it work locally. I need some more time to verify the solution, and I'll try to submit a PR sometime next week. I'll keep you posted.

@shaneahmed
Copy link
Copy Markdown
Member

All the errors have been fixed except FsspecJsonWSIReader.

@shaneahmed I was able to make it work locally. I need some more time to verify the solution, and I'll try to submit a PR sometime next week. I'll keep you posted.

Thank you @aacic That would be wonderful.

# Conflicts:
#	requirements/requirements.txt
@shaneahmed
Copy link
Copy Markdown
Member

shaneahmed commented Mar 31, 2026

All the errors have been fixed except FsspecJsonWSIReader.

The updates to engines output breaks, test_hovernet_on_box which also needs to be fixed.

@measty Please would you be able to check the error with test_hovernet_on_box? This is probably due to updates in the way new zarr output is saved.

shaneahmed and others added 2 commits March 31, 2026 12:36
* Zarr 3 fix.

* Fix multilayer image `FsspecJsonWSIReader` support.
JiwaniZakir

This comment was marked as spam.

@shaneahmed shaneahmed marked this pull request as ready for review April 17, 2026 09:20
Comment thread tests/engines/test_multi_task_segmentor.py Outdated
Comment thread tests/engines/test_multi_task_segmentor.py Outdated
Comment thread tests/engines/test_multi_task_segmentor.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working dependencies Pull requests that update a dependency file help wanted Extra attention is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants