Skip to content

Keep lazy_elementwise blocks as arrays for scalar-returning ops (#6965)#7126

Open
gaoflow wants to merge 2 commits into
SciTools:mainfrom
gaoflow:fix/lazy-elementwise-0d-scalar
Open

Keep lazy_elementwise blocks as arrays for scalar-returning ops (#6965)#7126
gaoflow wants to merge 2 commits into
SciTools:mainfrom
gaoflow:fix/lazy-elementwise-0d-scalar

Conversation

@gaoflow

@gaoflow gaoflow commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

🚀 Pull Request

Description

Closes #6965.

Computing (e.g. saving) a scalar lazy cube whose units had been converted failed:

import iris.analysis, dask, numpy as np, dask.array as da
from iris.coords import DimCoord
from iris.cube import Cube

cube = Cube(da.arange(10), units="m",
            dim_coords_and_dims=[(DimCoord(np.arange(10), var_name="x"), 0)])
cube = cube.collapsed("x", iris.analysis.MIN)   # -> 0-dimensional, lazy
cube.convert_units("km")

delayed = iris.save(cube, "test.nc", compute=False)
dask.compute(delayed)
AttributeError: 'float' object has no attribute 'size'

The error only appears with convert_units and only when the result is stored lazily (calling cube.data directly is fine).

Cause

convert_units applies the conversion lazily via iris._lazy_data.lazy_elementwise, which maps the operation over the blocks of the lazy array. The operation is cf_units.Unit.convert, and for a 0-dimensional block that call returns a Python float rather than a 0-d array. Dask's store step then does block.size, which a float does not have.

Fix

Wrap each block result in np.asanyarray inside lazy_elementwise, so a block always remains an array regardless of what the operation returns. For a normal (n-d) block this is a no-op; for the scalar case it restores the expected 0-d array. Masks are preserved (asanyarray).

Verification

  • New test Test_lazy_elementwise::test_scalar_returning_op_on_0d — an op that returns a Python scalar for a 0-d block now yields a 0-d ndarray. Fails on main, passes here.
  • Reproducer above now saves successfully and round-trips the correct value (0.0 km); a 1-d convert_units remains lazy with correct values.
  • Full tests/unit/lazy_data/ suite (51) and the convert_units cube tests pass.
  • ruff check / ruff format clean.

lazy_elementwise mapped the operation over the blocks of a lazy array,
but some operations return a Python scalar for a 0-dimensional block --
for example cf_units.Unit.convert during convert_units. Dask cannot
store such a block (it has no .size), so computing/saving a scalar lazy
cube after convert_units raised AttributeError. Wrap each block result
in np.asanyarray so it always remains an array.

Fixes SciTools#6965.
@gaoflow gaoflow force-pushed the fix/lazy-elementwise-0d-scalar branch from cc9ffb7 to 70e5afc Compare June 18, 2026 21:43
The wrapped_op closure in lazy_elementwise cannot be pickled by the
standard pickle module, causing test_cube_with_deferred_unit_conversion
to fail. Replace the closure with a module-level _as_array_wrapper
function combined with functools.partial, which is picklable.

See SciTools#6965
@codecov

codecov Bot commented Jun 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.15%. Comparing base (3f0dd8d) to head (0d87ab8).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #7126   +/-   ##
=======================================
  Coverage   90.15%   90.15%           
=======================================
  Files          91       91           
  Lines       24983    24985    +2     
  Branches     4685     4685           
=======================================
+ Hits        22524    22526    +2     
  Misses       1682     1682           
  Partials      777      777           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Computing a Delayed object from a collaped cube with converted units fails

1 participant