[AINode]: Integrate MOMENT as a builtin forecasting model #17386
Open
Eliaaazzz wants to merge 1 commit intoapache:masterfrom
Open
[AINode]: Integrate MOMENT as a builtin forecasting model #17386Eliaaazzz wants to merge 1 commit intoapache:masterfrom
Eliaaazzz wants to merge 1 commit intoapache:masterfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Integrate MOMENT Time Series Foundation Model
Integrate MOMENT (A Family of Open Time-series Foundation Models, ICML 2024, Auton Lab/CMU) into IoTDB AINode as a built-in forecasting model.
MOMENT uses a T5 encoder-only backbone with patch-based input embedding and RevIN (Reversible Instance Normalization). It supports zero-shot forecasting with a fixed input length of 512 timesteps
and channel-independent multivariate processing.
Architecture:
Input [batch, n_channels, 512]
→ RevIN normalization
→ Patching (patch_len=8, stride=8 → 64 patches)
→ Patch embedding (linear → d_model)
→ T5 Encoder (self-attention)
→ ForecastingHead (linear → forecast_horizon)
→ RevIN denormalization
→ Output [batch, n_channels, forecast_horizon]
The default variant is MOMENT-1-small (~152 MB, d_model=512, 6 layers). Weights are downloaded from HuggingFace Hub on first use, consistent with existing built-in models (Sundial, Timer-XL,
Chronos2, etc.).
Design Decisions
momentfmpip package, following the same pattern as Chronos2 ([AINode] Integrate Chronos2 as builtin forecasting model #16903) and Toto ([AINode]: Integrate toto as a builtin forecasting model #17322). This avoids an additional runtimedependency and gives full control over the loading process.
PreTrainedModelwith customfrom_pretrained()to handle upstream MOMENT's flat state-dict key mapping (prefixing withmoment.*) and T5 config extraction fromconfig.json.output_length > model_forecast_horizon, predictions are appended to the context window and fed back as input, similar toautoregressive decoding.
Bug Fix
Fixed unquoted
$JAVAvariable inconfignode-env.shanddatanode-env.shto handle paths with spaces (e.g.C:\Program Files\...).This PR has:
Key changed/added classes (or packages if there are too many classes) in this PR
iotdb-core/ainode/iotdb/ainode/core/model/moment/— New MOMENT model packageconfiguration_moment.py—MomentConfig(PretrainedConfig)modeling_moment.py—MomentForPrediction(PreTrainedModel)with RevIN, Patching, T5 Encoder, ForecastingHeadpipeline_moment.py—MomentPipeline(ForecastPipeline)with padding/truncation and iterative forecastingiotdb-core/ainode/iotdb/ainode/core/model/model_info.py— AddedmomenttoBUILTIN_HF_TRANSFORMERS_MODEL_MAPiotdb-core/ainode/iotdb/ainode/core/constant.py— Added memory estimate for MOMENTintegration-test/.../AINodeTestUtils.java— AddedmomenttoBUILTIN_LTSM_MAPLICENSE— Added MIT attribution for MOMENT source codescripts/conf/confignode-env.sh,scripts/conf/datanode-env.sh— Fixed unquoted$JAVAvariable