Thinking: Should OME-Zarrs enable native thumbnails?

[lubianat]

https://github.com/BioNGFF/ome-zarr.js is valueable for the customizability, but similar to how OMERO sets default thumbnails, OME-Zarrs could come with default thumbnails with a pre-defined size

In the directions of a super light addition such as

OME-Zarr files MAY have a "thumbnail" folder. If a "thumbnail" folder is present, it MUST contain a single 2D RGB image named "thumbnail_48.jpg" of 48 x 48 pixels. Thumbnails MUST be squared. Other sizes MAY be present, in "thumbnail_X.jpg", where X indicates the size of the axes in pixels.

(based on DEFAULT_X_WIDTH at https://github.com/ome/omero-server/blob/master/src/main/java/ome/services/ThumbnailBean.java)

And probably no changes to the OME-metadata, so there is no confusion. A bit batteries included/opinionated way, as this would be a secondary feature. If extra customizability needed, one can resort to building the thumbnails with ome-zarr.js.

The thumbnail should be in widely-supported format like PNG or JPG. OME-TIF could be fun, but a lot of duplication.

1

- zarr.json
- 0
- 1
- 2
- thumbnail 
     - thumbnail_48.jpg

2

- zarr.json
- 0
- 1
- 2
- thumbnail_48.jpg

pros:

• tools that depend on thumbnails (e.g. BFF, the 2024 challenge, OMERO etc) would have a native way to getting that info (less calls, better caching, less dependences on libs)

• OME-Zarrs often are not optimized for thumbnail generation. Having a separate "thumbnail" field would reduce the need for considering that when picking the best strategy for chunking/sharding/multiscaling.

• if thumbnail is, say, PNG/JPG/(OME-)TIF, Zarr-ignorant tools could have a way to display something visual

cons:

• extra specs
• less use cases for ome-zarr.js
• mix of formats, possible confusion
• even if optional, one more thing for tooling to support
  ...

Some relevant links:

– https://forum.image.sc/t/how-are-the-thumbnails-generated-on-omero/91865/2

[joshmoore]

I'm pretty sure this has been discussed before (but am not finding the reference at the moment). It almost certainly would need to go along with one more more RFCs.

[lubianat]

I ctrl-f'd the ngff repo and found only this discussion around thumbnails:

– ome/ngff#55

but mostly about metadata for a "middle slice" thumbnail

[lubianat]

Inspired by this example from @nrepina on how previews are valuable, I am curious to think what would be the steps for OME-Zarr to show nice previews in Mac/Windows/Linux in the same way that, say .tif does:

It could be a lot of work, but if OME-Zarr is to become very very widely used...

I am not sure if it is in the "just very hard" or in the "practically impossible" box though

[joshmoore]

I am curious to think what would be the steps for OME-Zarr to show nice previews in Mac/Windows/Linux in the same way that, say .tif does...

Someone at the Broad hackathon had looked into this. Perhaps @bethac07 could make the connection?

[lubianat]

I wonder if there are other uses in the Zarr community for things like this, so maybe it would be a Zarr extension or similar

[lubianat]

for future reference, on the OS use case:

On Linux, it would basically only require a "thumbnailer" .deb that could even eventually be shipped with distros

On macOs, we would need a Quick Look (https://en.wikipedia.org/wiki/Quick_Look) plugin, what does not seem terrible, but some more investiment

e.g.

https://github.com/toland/qlmarkdown
https://github.com/whomwah/qlstephen

Windows Explorer seems to be in the "pratically impossible" category

[bethac07]

No idea, sorry

[lubianat]

@mkitti has commented on the thumbnail topic at ome/ngff#406 on the comment by @petebankhead on RFC9, authored by @jwindhager:

@jwindhager Similarly, some operating systems expect an image to be stored in a single file, as apparent by e.g. file permission systems, file type concepts (e.g. file name extensions) and file type-dependent functionality (e.g., double/right-click, drag-and-drop, preview).

@petebankhead The 'preview' aspect makes it tempting to want to embed a thumbnail, which could be supported by some applications or operating system plugins.
Should this be explicitly forbidden / discouraged / encouraged in a standard way?

@mkitti It is not clear to me that a thumbnail would be specific to the single zip file case. If this is a desired feature, I would suggest proposing a standard for all OME-Zarr images.

I pinged you all so we can perhaps centralize the thumbnail/preview conversation out of RFC-9 comments.

This may be part of a new RFC, perhaps including non-zips, or even a wider Zarr convention

Also @l-spiecker and @mkitti again via ome/ngff#418 added extra information

@l-spiecker Thumbnails: Applications might benefit from pre-rendered thumbnails. As there is no standardized way to store thumbnails for Zarr and OME-Zarr it might be a question if this should be a topic to be addressed by zipped OME-Zarr separately or if this is out of scope for this RFC. As an example many ZIP based formats (e.g. docx, 3mf) follow the Open Packaging Conventions to store thumbnails in a standardized way.

@mkitti Thumbnails should be addressed in OME-Zarr more generally.
Compatability with the Open Packaging Conventions is an interesting idea, although I am somewhat reluctant to introduce a XML standard into the base specification.

[lubianat]

BioNGFF/ome-zarr.js#30 is a way to render OME-Zarrs neatly and with more configuration and flexibility, as well as support for labels.

but I still think some lightweight thumbnail (extension?) in OME-Zarr could have merit, particularly in the context of browsing collections and displays of multiple gallery-style (like https://github.com/ome/omero-gallery).

I'll cc @krokicki @allison-truhlar as you may have thoughts of whether something like this is useful at all (e.g. for Fileglancer) or OME-Zarr.js and similar JS-level parsing is just better

[mkitti]

A basic question is "To what should we attach a thumbnail to?". Do we associate thumbnails to arrays? to groups? to Zarr roots? to multiscale objects? wells? All of the above?

[krokicki]

Thanks for thinking of us, @lubianat. Also @tischi might have a MoBIE use case for this.

I think that adding thumbnails into the OME-Zarr spec would be very useful, especially for gallery views. I'd be interested in contributing to an RFC, if that would be helpful.

The ome-zarr.js approach works well for just-in-time visualization, but it has some limitations particularly because of limited customization of the thumbnails (only via OMERO metadata) and slower browsing of large galleries. BioFileFinder overcomes these issues by adding an optional thumbnail URL in the metadata CSV. This works but it's a) not standardized and b) external to the Zarrs, which makes keeping things in sync more difficult.

I've explored embedding thumbnails directly in OME-Zarrs and had a working implementation in Zarrcade, but I didn't pursue it further due to the design questions involved. The "super light" specification proposed here is appealing because it sidesteps a lot of the complexity. At the risk of complicating things, here are some additional questions that could be considered:

• Is the thumbnail only allowed at the top level of the OME-Zarr, or for any group with multiscales metadata, or for any group at all?
  • How will this work for multiple image collections like HCS and bioformats2raw transitional layout?
  • How will it interact with RFC-8 (image collections)?
• A thumbnail must collapse a multidimensional dataset into a 2d image. Should there be metadata describing which axes were projected or selected?
  • When there are multiple time points, which one was used for the thumbnail?
  • Is the image a middle slice or a MIP?
  • If a projection, along which plane (e.g. is it an XY MIP or a XZ MIP?)
  • Should we be able to specify multiple summary images (e.g. XY, XZ, YZ)?
• Should the spec support original aspect ratios in addition to squared images? Some data sets might be difficult to square effectively (e.g. Drosophila VNC image collections)
• Should the spec support multiple "summary images"?
  • Different resolutions (large format vs thumbnail)
  • Different formats (PNG vs JPEG)
• Is there a way to specify provenance (e.g. what tool was used to generate the thumbnail?)

Most of this is probably unnecessary complexity, so I'd be happy with the simple thumbnails/thumbnail_X.jpg convention, if it can speed acceptance and adoption.

I would mainly seek to clarify where in the Zarr hierarchy the thumbnails folder can exist. Maybe only root level? If they can occur anywhere then I would suggest maintaining lightweight pointers to the thumbnails in the metadata, mainly for the purposes of reducing exploratory HTTP requests.

[lubianat]

Oh, wow, thank you very much for the details @krokicki! Happy to know a working implementation was done in Zarrcade, I had no idea.

I would mainly seek to clarify where in the Zarr hierarchy the thumbnails folder can exist. Maybe only root level? If they can occur anywhere then I would suggest maintaining lightweight pointers to the thumbnails in the metadata, mainly for the purposes of reducing exploratory HTTP requests.

Agreed. Your questions are really good, particularly the first block and how it relates to RFC 8 and to different kinds of groups/collections. I don't have a good answer for those — not yet, at least!

Most of this is probably unnecessary complexity, so I'd be happy with the simple thumbnails/thumbnail_X.jpg convention, if it can speed acceptance and adoption.

I think they are valid, and I can see reasons for allowing expressivity to cover the different scenario. It starts intersecting a bit with >5-year long discussions on rendering metadata (e.g. ome/ngff#23 and https://forum.image.sc/t/ome-zarr-how-transitional-is-the-omero-metadata/118906/9). So I guess this would need extra coordination, indeed.

I'd be interested in contributing to an RFC, if that would be helpful.

Thank you for the offer! I think @joshmoore would have a mini heart attack with a new RFC before we have 0.6 and a clear path for RFCs 3, 4, 7, 8 and 9. That said, I think if we collect use cases and sharpen ideas while the other pieces evolve, the whole thumbnail RFC process could be straightforward.

Of course, if native thumbnails would unblock some core thing, I'd be happy to dedicate more time sooner rather than later.

[lubianat]

(I also realized I never cc'd @will-moore here)