Skip to content

Repository files navigation

zarrista

PyPI

A low-level Zarr API for Python, inspired by zarrita.js, powered from Rust by Zarrs. Serving up Zarr chunks like your favorite barista!

This project is minimally vibe-coded. A person wrote most of the code by hand, and Claude prototyped some areas.

This project is for evaluation. It examines whether a native binding to Zarrs gives better performance. It is not ready for production.

Documentation

Documentation website.

Features

  • Low-level, explicit Zarr access. Open arrays and groups, read chunks, and examine metadata. The API hides no machinery.
  • Both sync and async APIs (Array / AsyncArray, Group / AsyncGroup).
  • Rust core through Zarrs, for the performance of compiled code.
  • NumPy integration. Read data into NumPy arrays through the buffer protocol. Rust and Python share the memory without a copy.
  • Variety of data access
    • AWS S3, Google Cloud Storage, Azure Storage through Obstore
    • Icechunk integration
  • Full type hinting for all operations.

Example

Open a store, then open an Array from it:

from zarrista import Array
from zarrista.store import FilesystemStore

store = FilesystemStore("data/example.zarr")
array = Array.open(store, path="/temperature")

Inspect the array's metadata:

array.shape
# [720, 1440]

array.dtype
# DataType(float32 / <f4)

array.dimension_names
# ["lat", "lon"]

Read a subset of the array. Indexing returns a DecodedArray, which converts to a NumPy array:

data = array[0:128, 0:128]
arr = data.to_numpy()
arr.shape
# (128, 128)

You can also read individual chunks by their grid index:

data = array.retrieve_chunk([0, 0])

About

A low-level Zarr API for Python, binding to Rust's Zarrs

Topics

Resources

Code of conduct

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages