|
| 1 | +# MetPy Roadmap & Long-Term Efforts |
| 2 | + |
| 3 | +**Last updated: 2026-03-13** |
| 4 | + |
| 5 | +## NSF Grant 2103682 (deadline: April 30, 2026) |
| 6 | + |
| 7 | +MetPy is delivering on an awarded US National Science Foundation grant (award 2103682) with the goal of **speeding up MetPy's calculations** and **enabling modern big-data science** with MetPy. This grant drives the compiled C++ extensions work and related infrastructure changes. Key deliverables include native-compiled thermodynamic calculations, build/packaging infrastructure for compiled extensions, and ensuring MetPy works well in large-scale and cloud-native data workflows. |
| 8 | + |
| 9 | +## Dependency Management Modernization (under exploration) |
| 10 | + |
| 11 | +Exploring replacing pinned `ci/requirements.txt` files with modern tooling. Two approaches under evaluation on separate branches: |
| 12 | + |
| 13 | +### `uv` branch |
| 14 | + |
| 15 | +- Adds `ci-stable`/`ci-unstable` dependency groups to `pyproject.toml` with uv conflicts/sources |
| 16 | +- `uv sync --resolution lowest-direct --python 3.11` could replace the CI script that flips `>=` to `==` for minimum dep testing |
| 17 | +- Git-based source overrides for upstream/unstable testing of pint and xarray |
| 18 | + |
| 19 | +### `pixi` branch (PR #3919) |
| 20 | + |
| 21 | +- Maximal pixi config encompassing all current CI environments |
| 22 | +- Manages conda-side resolution and installation; uses uv internally for pip deps |
| 23 | +- Per-feature/per-environment deps, tasks, and platform support |
| 24 | +- See PR description for current limitations (disk size, no dependabot support, IDE plumbing) |
| 25 | + |
| 26 | +### Open questions |
| 27 | + |
| 28 | +- Whether to adopt uv, pixi, or both (they serve complementary roles — uv for PyPI, pixi for conda) |
| 29 | +- How to reduce duplication between `pyproject.toml` deps and tool-specific specs |
| 30 | +- Keep both conda and PyPI CI test paths (not redundant — conda CI covers macOS/Windows, PyPI CI covers Linux; becomes essential with compiled extensions) |
| 31 | + |
| 32 | +## Compiled C++ Extensions |
| 33 | + |
| 34 | +- MetPy will add **compiled C++ code** using **nanobind**, built with **scikit-build-core** and **cibuildwheel** |
| 35 | +- Targeting the **Python limited ABI (stable API)** — this produces abi3 wheels that work across Python versions, dramatically reducing the wheel build matrix (one wheel per platform instead of per-platform-per-Python) |
| 36 | +- Shipped via **PyPI** (cibuildwheel wheels) and **conda-forge** (conda compilers) — these produce genuinely different binaries, so both paths must be tested |
| 37 | +- pixi becomes more valuable for faithfully testing the full conda build path |
| 38 | + |
| 39 | +### `build-wheels` branch (build infrastructure — in progress) |
| 40 | + |
| 41 | +The build/packaging foundation is being established on this branch. Current state: |
| 42 | + |
| 43 | +- **Build backend switched** from setuptools to **scikit-build-core** (`build-backend = "scikit_build_core.build"`) |
| 44 | +- **Build requires**: `setuptools>=80`, `setuptools-scm[simple]>=8`, `scikit-build-core`, `pybind11` |
| 45 | +- **CMake config**: `src/CMakeLists.txt` builds a `_calc_mod` pybind11 module, installed to `metpy/` |
| 46 | +- **Stub C++ module**: `src/calcmod.cpp` contains a minimal `add()` function as a proof-of-concept |
| 47 | +- **cibuildwheel CI**: `.github/workflows/packages.yml` builds wheels across 6 platform targets: |
| 48 | + - Linux x64, Linux arm64, Windows x64, Windows arm64, macOS x64 (Intel), macOS arm64 (Apple Silicon) |
| 49 | + - Triggered via `workflow_dispatch` (`.github/workflows/run-package-builds.yml`) |
| 50 | + - Also builds sdist and gathers all artifacts |
| 51 | +- **Version scheme**: uses `scikit_build_core.metadata.setuptools_scm` for version from git tags |
| 52 | +- Branch also includes various CI updates (actions version bumps, minor fixes) |
| 53 | + |
| 54 | +**Next steps for this branch:** |
| 55 | + |
| 56 | +- Migrate from pybind11 to **nanobind** (build requires, CMakeLists.txt, module code) |
| 57 | +- Enable **limited API / abi3** in nanobind build to produce version-agnostic wheels |
| 58 | +- Integrate package builds into PR/push CI (currently manual dispatch only) |
| 59 | + |
| 60 | +### Thermo C++ PRs (calculations — draft) |
| 61 | + |
| 62 | +- **#3856** (`lcl_manual_vectorize_pybind11`) — First C++ thermo PR. Ports ~20 thermodynamic functions to C++ via pybind11: saturation vapor pressure, dew point, mixing ratio, LCL, dry/moist lapse rates, virtual temperature, and supporting helpers. Adds `src/` C++ sources (`thermo.cpp`, `constants.cpp`, `math.cpp`) with CMakeLists.txt. Modifies `calc/thermo.py` to call compiled implementations. |
| 63 | +- **#3874** (`test_cpp_cape`) — Second C++ thermo PR, building on #3856. Extends to include `ParcelProfile`, `_ParcelProfileHelper`, `LCLVectorized`, `_CheckPressure`, and `lambert_wm1`. `cape_cin`, `lfc`, `el`, and `parcel_profile` remain in Python but call underlying C++ functions. |
| 64 | + |
| 65 | +### Notes |
| 66 | + |
| 67 | +- Both thermo PRs currently use pybind11; these will be ported to **nanobind** as part of integration |
| 68 | +- C++ sources live under `src/` (alongside `src/metpy/`): `calcmod.cpp`, `thermo.cpp/.hpp`, `constants.cpp/.hpp`, `math.cpp/.hpp` |
| 69 | +- A `constants/nounit.py` module provides plain-float constants for use by the C++ layer (bypassing Pint) |
| 70 | + |
| 71 | +## v1.8.0 Milestone (26 open issues) |
| 72 | + |
| 73 | +### Plots |
| 74 | + |
| 75 | +- **#2488** (PR) — Declarative SkewT plotting |
| 76 | +- **#262** — Height scale for SkewT (good first issue) |
| 77 | +- **#2016** — Helpful declarative `__setattr__` and trait validation |
| 78 | +- **#2340** — Jupyter Lab completer not showing filtered `dir` for declarative |
| 79 | +- **#1915** — ImagePlot fails on some [0, 360] data |
| 80 | +- **#1005** — Plot colormapped axes labeling |
| 81 | + |
| 82 | +### Calculations |
| 83 | + |
| 84 | +- **#3386** — Examine wet bulb temperature implementation |
| 85 | +- **#3148** (PR) — Option to return full profile from `thickness_hydrostatic` |
| 86 | +- **#1312** — Same as above (the issue #3148 addresses) |
| 87 | +- **#2315** — ValueError calculating CAPE for a layer |
| 88 | +- **#1387** — Precipitable water returning NaN |
| 89 | +- **#3105** — `laplacian` should skip non-length dimensions |
| 90 | + |
| 91 | +### xarray Integration |
| 92 | + |
| 93 | +- **#3860** — Coordinate identification for 2D lat/lon fails with similarly named auxiliary variables |
| 94 | +- **#2370** — Unneeded warning from xarray dimension handling |
| 95 | +- **#2369** — `isentropic_interpolation_as_dataset` can create "None" DataArray |
| 96 | +- **#2368** — Add `isel` to xarray accessor |
| 97 | +- **#1599** — Align coordinate data for multiple variables |
| 98 | + |
| 99 | +### IO / Remote |
| 100 | + |
| 101 | +- **#3922** (PR) — Make WPC parser more fault tolerant |
| 102 | +- **#3862** (PR) — Recursive search for GOES products across hour boundaries |
| 103 | +- **#3842** — Make remote clients more robust |
| 104 | + |
| 105 | +### Infrastructure / Docs |
| 106 | + |
| 107 | +- **#3871** — Infrastructure guide is stale + minor code analysis review |
| 108 | +- **#1993** — Formalize and document authorship process |
| 109 | +- **#2185** — Improve "Working With Units" tutorial on array types |
| 110 | +- **#1450** — Consider sphinx-autosummary-accessors for accessor docs |
| 111 | +- **#1655** — Update roadmap |
| 112 | + |
| 113 | +### Other |
| 114 | + |
| 115 | +- **#1844** — pyproj output not accepted by MetPy |
| 116 | + |
| 117 | +## CI Simplification Goals |
| 118 | + |
| 119 | +The current CI structure is documented in `.claude/rules/ci-architecture.md`. The matrix should be simplified where possible, but the priority is **not shipping broken code** — testing and documentation must use realistic build and installation environments. Key constraints: |
| 120 | + |
| 121 | +- **PyPI path**: must build compiled wheels (cibuildwheel) and test with pip-installed dependencies. Limited API/abi3 wheels reduce the Python version dimension of the build matrix but testing still needs to cover supported versions. |
| 122 | +- **conda-forge path**: must test with conda-installed dependencies and conda-compiled extensions, since these produce different binaries than cibuildwheel. |
| 123 | +- **Cross-platform**: compiled extensions require testing on all target platforms (Linux, macOS, Windows; x64 and arm64 where applicable). |
| 124 | +- Documentation builds should use the same dependency environment as users will encounter. |
0 commit comments