Feature/mars2grib era6 reencoding support#207
Open
MircoValentiniECMWF wants to merge 10 commits into
Open
Conversation
- Add new iteration concept under backend/concepts/iteration/ following the standard 4-file layout (Enum, Matcher, Encoding, ConceptDescriptor); single Default variant, applicable at (StagePreset, SecLocalUseSection), with LocalDefinitionNumber allow-list {20, 38}; encoder sets iterationNumber and (optionally) totalNumberOfIterations.
- Add the supporting deductions backend/deductions/iterationNumber.h (resolve_IterationNumber_or_throw) and backend/deductions/totalNumberOfIterations.h (resolve_TotalNumberOfIterations_opt) used by IterationOp.
- Register IterationConcept in AllConcepts.h: include added alphabetically between generating-process and level; type appended to the AllConcepts typelist after LongrangeConcept.
- Implement iterationMatcher: returns IterationType::Default when mars has the "iteration" key, MISSING otherwise.
- Extend analysisEncoding.h LocalDefinitionNumber allow-list from {36} to {36, 38} so AnalysisOp accepts the new combined template 38 (4i analysis products).
- Add Section 2 recipes in section2Recipes.h: S2_R20 (Mars + Iteration) and S2_R38 (Mars + Iteration + Analysis); both registered in the Section2Recipes aggregator in numerical order.
- Add new modelError concept under backend/concepts/model-error/ following the standard 4-file layout (Enum, Matcher, Encoding, ConceptDescriptor); single Default variant, applicable at (StagePreset, SecLocalUseSection), with LocalDefinitionNumber allow-list {25, 39}; encoder body left as TODO (deductions / GRIB key writes to be added separately).
- Register ModelErrorConcept in AllConcepts.h: include added alphabetically between mars and nil; type appended at the end of the AllConcepts typelist to preserve existing conceptIds and global variant indices.
- Implement modelErrorMatcher: returns ModelErrorType::Default when mars["type"] == "eme"; throws Mars2GribMatcherException if the mandatory "number" key is missing in that case; returns MISSING otherwise.
- Add Section 2 recipes in section2Recipes.h: S2_R25 (Mars + ModelError) and S2_R39 (Mars + Analysis + ModelError); both registered in the Section2Recipes aggregator in numerical order.
- Extend analysisEncoding.h LocalDefinitionNumber allow-list from {36, 38} to {36, 38, 39} so AnalysisOp accepts the new template 39.
- Update ensembleMatcher.h to return MISSING when mars["type"] == "eme", since in that case the "number" key identifies the model-error realization, not an ensemble member.
…MultipleLevel - Replace the single LevelType::Hybrid variant with two variants: ModelSingleLevel for 2D fields published on the model-level system (no vertical column, no PV array) and ModelMultipleLevel for full vertical columns of model-level data, which require allocation and population of the PV array describing the hybrid coordinate transformation. Both variants map to GRIB typeOfLevel "hybrid", so encoded output is bit-identical for cases that previously used Hybrid; only encoder behaviour (PV allocation) differs between the two new variants. - Update the LevelList typelist to reflect the new variants and keep it in sync with the LevelType enumeration. - Update needPv to fire only on ModelMultipleLevel; update needLevel to cover both ModelSingleLevel and ModelMultipleLevel. - Add a new AbstractLevel variant carrying a numeric level value, sitting alongside the existing AbstractSingleLevel and AbstractMultipleLevel opaque variants. AbstractLevel is included in needLevel. - Rewrite matchML to dispatch single-level model paramIds (22, 127, 128, 129, 152) to ModelSingleLevel and the remaining multi-level set to ModelMultipleLevel. ERA6 paramIds 127 and 128 on ML, which were previously rejected, are now accepted as ModelSingleLevel. - Refresh Doxygen for the level concept to document the three orthogonal predicates needPv, needLevel, needTopBottomLevel and to describe the rationale for splitting Hybrid into single-level and multi-level model variants. - Fix typo in the level encoder header comment: "Se of typeOfLevel" becomes "Setting of typeOfLevel".
…fc detection, model-error types)
- params.yaml: register ECMWF covariance paramIds 254001..254017 on levtype=sfc.
- levelMatcher: map paramIds 254001..254017 to LevelType::AbstractLevel
(typeOfFirstFixedSurface=254); extend matchO2D with ocean paramIds
262146/262147 (DepthBelowSeaLayer) and 262148 (OceanSurfaceToBottom).
- pointInTimeMatcher: add 254001..254017 and 262146..262148 to the default
point-in-time set so Section 4 recipe selection succeeds for these params.
- significanceOfReferenceTime: recognize MARS type "est" as a forecast type.
- typeOfGeneratingProcess:
* For type=fc, detect ensemble evidence (numberOfForecastsInEnsemble>1,
typeOfEnsembleForecast present, or mars.number>0) and resolve to
EnsembleForecast instead of Forecast; default behavior is preserved
when no ensemble evidence is present. Adds detail to RESOLVE log.
* Map type=eme/me (4D-Var model errors) to Analysis, matching the
existing {4i,4v,me,eme} grouping in significanceOfReferenceTime.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #207 +/- ##
===========================================
- Coverage 62.03% 61.57% -0.46%
===========================================
Files 303 315 +12
Lines 11671 11776 +105
Branches 1049 1054 +5
===========================================
+ Hits 7240 7251 +11
- Misses 4431 4525 +94 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add support for new parameters and new concepts needed by ERA6
Contributor Declaration
By opening this pull request, I affirm the following: