Skip to content

fix(gcp_cloud_storage sink): apply timezone to key_prefix strftime#25541

Open
xfocus3 wants to merge 1 commit into
vectordotdev:masterfrom
xfocus3:fix/gcs-key-prefix-timezone-25090
Open

fix(gcp_cloud_storage sink): apply timezone to key_prefix strftime#25541
xfocus3 wants to merge 1 commit into
vectordotdev:masterfrom
xfocus3:fix/gcs-key-prefix-timezone-25090

Conversation

@xfocus3
Copy link
Copy Markdown

@xfocus3 xfocus3 commented May 31, 2026

Summary

Fixes #25090. The gcp_cloud_storage sink ignored the configured timezone option when rendering strftime specifiers in key_prefix, so date-based object partitioning (e.g. key_prefix = "%Y%m%d/") always used UTC. The aws_s3 sink already honors timezone for key_prefix via Template::with_tz_offset; this brings the GCS sink in line.

Root cause

GcsSinkConfig::key_partitioner built the KeyPartitioner straight from the raw key_prefix template:

fn key_partitioner(&self) -> crate::Result<KeyPartitioner> {
    Ok(KeyPartitioner::new(
        Template::try_from(self.key_prefix.as_deref().unwrap_or("date=%F/"))
            .context(KeyPrefixTemplateSnafu)?,
        None,
    ))
}

The timezone offset (timezone → falling back to the global timezone) was computed in RequestSettings::new and applied only to the filename timestamp, never to the key_prefix template. aws_s3 instead does Template::try_from(...)?.with_tz_offset(offset) before constructing its partitioner.

Fix

Compute the offset in build_sink the same way aws_s3 does, thread it into key_partitioner, and apply it with Template::with_tz_offset. Behavior with no timezone configured is unchanged (renders in UTC).

Testing

  • cargo test --lib --no-default-features --features sinks-gcp gcp::cloud_storage — 16 passed, 0 failed, including the new gcs_key_prefix_honors_timezone_offset regression test which asserts a 2026-04-01T00:30:00Z event renders as 20260401/ under UTC+08:00, 20260331/ under UTC-08:00, and 20260401/ with no timezone.
  • cargo clippy --lib --no-default-features --features sinks-gcp -- -D warnings — clean.
  • cargo fmt --check — clean.
  • Changelog fragment changelog.d/25090_gcs_key_prefix_timezone.fix.md validated with ./scripts/check_changelog_fragments.sh.

The `gcp_cloud_storage` sink built its key partitioner from the raw
`key_prefix` template without applying the configured `timezone`
offset, so strftime specifiers in `key_prefix` (e.g. `%Y%m%d/`) were
always rendered in UTC. The `aws_s3` sink already applies the offset
via `Template::with_tz_offset`; this change mirrors that behavior so
date-based object partitioning honors the `timezone` (or global
`timezone`) setting.

Adds a regression test covering positive/negative/none offsets around
the UTC day boundary.

Closes vectordotdev#25090
@xfocus3 xfocus3 requested a review from a team as a code owner May 31, 2026 18:34
@github-actions github-actions Bot added the domain: sinks Anything related to the Vector's sinks label May 31, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e414b8467a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

let offset = self
.timezone
.or(cx.globals.timezone)
.and_then(timezone_to_offset);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve DST-aware timezone rules

When timezone is a named zone with daylight saving time, this converts it once with timezone_to_offset, which freezes the offset based on Utc::now() at sink build time before Template formats each event timestamp. In a sink configured with America/Los_Angeles, if Vector starts during PDT (-07) but renders a winter event such as 2026-12-01T07:30:00Z (still Nov 30 in PST -08), the key prefix is formatted using -07 and lands under Dec 1 instead of Nov 30. The template needs the named timezone, or an offset computed from the event timestamp, rather than a build-time fixed offset.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: sinks Anything related to the Vector's sinks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gcp_cloud_storage sink: timezone option not applied to key_prefix strftime rendering

1 participant