Skip to content

[AINode]: Integrate MOMENT as a builtin forecasting model #17386

Open
Eliaaazzz wants to merge 1 commit intoapache:masterfrom
Eliaaazzz:f_ainode_moment
Open

[AINode]: Integrate MOMENT as a builtin forecasting model #17386
Eliaaazzz wants to merge 1 commit intoapache:masterfrom
Eliaaazzz:f_ainode_moment

Conversation

@Eliaaazzz
Copy link
Copy Markdown

Description

Integrate MOMENT Time Series Foundation Model

Integrate MOMENT (A Family of Open Time-series Foundation Models, ICML 2024, Auton Lab/CMU) into IoTDB AINode as a built-in forecasting model.

MOMENT uses a T5 encoder-only backbone with patch-based input embedding and RevIN (Reversible Instance Normalization). It supports zero-shot forecasting with a fixed input length of 512 timesteps
and channel-independent multivariate processing.

Architecture:
Input [batch, n_channels, 512]
→ RevIN normalization
→ Patching (patch_len=8, stride=8 → 64 patches)
→ Patch embedding (linear → d_model)
→ T5 Encoder (self-attention)
→ ForecastingHead (linear → forecast_horizon)
→ RevIN denormalization
→ Output [batch, n_channels, forecast_horizon]

The default variant is MOMENT-1-small (~152 MB, d_model=512, 6 layers). Weights are downloaded from HuggingFace Hub on first use, consistent with existing built-in models (Sundial, Timer-XL,
Chronos2, etc.).

Design Decisions

  • Vendor the model code rather than depending on the external momentfm pip package, following the same pattern as Chronos2 ([AINode] Integrate Chronos2 as builtin forecasting model #16903) and Toto ([AINode]: Integrate toto as a builtin forecasting model #17322). This avoids an additional runtime
    dependency and gives full control over the loading process.
  • Wrap as PreTrainedModel with custom from_pretrained() to handle upstream MOMENT's flat state-dict key mapping (prefixing with moment.*) and T5 config extraction from config.json.
  • Use MOMENT-1-small as the default built-in variant to minimize download size and memory usage while still providing useful forecasting capability.
  • Iterative long-horizon forecasting in the pipeline: for output_length > model_forecast_horizon, predictions are appended to the context window and fed back as input, similar to
    autoregressive decoding.
  • Fixed 512-length input handling: inputs shorter than 512 are left-padded with zeros and masked; inputs longer than 512 are truncated to the last 512 timesteps.

Bug Fix

Fixed unquoted $JAVA variable in confignode-env.sh and datanode-env.sh to handle paths with spaces (e.g. C:\Program Files\...).


This PR has:

  • been self-reviewed.
  • added or updated version, license, or notice information
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added integration tests.
  • been tested in a test IoTDB cluster.

Key changed/added classes (or packages if there are too many classes) in this PR
  • iotdb-core/ainode/iotdb/ainode/core/model/moment/ — New MOMENT model package
    • configuration_moment.pyMomentConfig(PretrainedConfig)
    • modeling_moment.pyMomentForPrediction(PreTrainedModel) with RevIN, Patching, T5 Encoder, ForecastingHead
    • pipeline_moment.pyMomentPipeline(ForecastPipeline) with padding/truncation and iterative forecasting
  • iotdb-core/ainode/iotdb/ainode/core/model/model_info.py — Added moment to BUILTIN_HF_TRANSFORMERS_MODEL_MAP
  • iotdb-core/ainode/iotdb/ainode/core/constant.py — Added memory estimate for MOMENT
  • integration-test/.../AINodeTestUtils.java — Added moment to BUILTIN_LTSM_MAP
  • LICENSE — Added MIT attribution for MOMENT source code
  • scripts/conf/confignode-env.sh, scripts/conf/datanode-env.sh — Fixed unquoted $JAVA variable

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant