Skip to content

feat: support per-runner-flavor SQS batch size and window in multi_runner_config#5108

Open
Copilot wants to merge 6 commits into
mainfrom
copilot/support-sqs-batch-size-window-config
Open

feat: support per-runner-flavor SQS batch size and window in multi_runner_config#5108
Copilot wants to merge 6 commits into
mainfrom
copilot/support-sqs-batch-size-window-config

Conversation

Copilot AI commented Apr 14, 2026

Copy link
Copy Markdown
Contributor

Description

lambda_event_source_mapping_batch_size and lambda_event_source_mapping_maximum_batching_window_in_seconds are module-level variables applied identically to every runner flavor. Deployments with mixed load profiles (e.g. 1000-runner high-volume vs 10-runner low-volume) need per-flavor tuning to avoid unnecessary latency, wasted SSM throughput, and GitHub API rate limit pressure on low-volume flavors.

Adds both as optional(number, null) fields in multi_runner_config.runner_config, with coalesce() fallback to the existing module-level variables. Follows the established pattern used by scale_up_reserved_concurrent_executions.

Changes:

  • modules/multi-runner/variables.tf — new optional fields in the runner_config object type + descriptions; restored iam_overrides fields accidentally dropped during merge
  • modules/multi-runner/runners.tfcoalesce(per_flavor, module_level) for both parameters; restored iam_overrides wiring
  • modules/multi-runner/README.md — regenerated with terraform-docs to reflect all changes; kept in sync with latest main

Usage:

multi_runner_config = {
  runner-large = {
    runner_config = {
      lambda_event_source_mapping_batch_size                         = 50   # override
      lambda_event_source_mapping_maximum_batching_window_in_seconds = 10   # override
      # ...
    }
  }
  runner-metal = {
    runner_config = {
      # no override — uses module-level defaults
      # ...
    }
  }
}

Fully backwards compatible. Existing configs require no changes.

Test Plan

  • Verified with terraform fmt (clean)
  • CodeQL security scan passed (no issues)
  • Code review passed (no issues found)
  • Confirmed iam_overrides is fully restored in runner_config object type, heredoc description, standalone variable, and runners.tf wiring
  • Merged latest main and regenerated README; no remaining conflicts

Related Issues

Copilot AI changed the title [WIP] Support per-runner-flavor SQS batch size and window configuration feat: support per-runner-flavor SQS batch size and window in multi_runner_config Apr 14, 2026
Copilot AI requested a review from Brend-Smits April 14, 2026 08:32
@github-actions

github-actions Bot commented Apr 14, 2026

Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@wadherv

wadherv commented May 18, 2026

Copy link
Copy Markdown
Contributor

@Brend-Smits can you please review this PR?

@Brend-Smits

Copy link
Copy Markdown
Contributor

@copilot resolve the merge conflicts in this pull request

@Brend-Smits Brend-Smits requested a review from Copilot June 10, 2026 14:16
@Brend-Smits Brend-Smits marked this pull request as ready for review June 10, 2026 14:16
@Brend-Smits Brend-Smits requested a review from a team as a code owner June 10, 2026 14:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds per-runner-flavor overrides for SQS event source mapping tuning in the multi-runner module, allowing mixed runner fleets to use different Lambda batching behavior without changing global defaults.

Changes:

  • Extends multi_runner_config.runner_config with per-flavor optional overrides for lambda_event_source_mapping_batch_size and lambda_event_source_mapping_maximum_batching_window_in_seconds.
  • Updates the per-flavor module "runners" wiring to coalesce(per_flavor_override, module_level_default) for both settings.
  • Regenerates/updates modules/multi-runner docs to reflect the new fields.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
modules/multi-runner/variables.tf Adds optional per-flavor batch size/window fields to the runner_config object type and documents them in the multi_runner_config heredoc.
modules/multi-runner/runners.tf Applies per-flavor overrides via coalesce() when passing event source mapping settings into the runners module.
modules/multi-runner/README.md Updates the generated inputs documentation to include the new per-flavor configuration options.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread modules/multi-runner/variables.tf Outdated
Comment thread modules/multi-runner/variables.tf Outdated
Comment thread modules/multi-runner/README.md Outdated

Copilot AI commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

@copilot resolve the merge conflicts in this pull request

Resolved the merge conflict in modules/multi-runner/README.md by merging origin/main and regenerating the terraform-docs section. Pushed in commit e530643.

@Brend-Smits Brend-Smits force-pushed the copilot/support-sqs-batch-size-window-config branch from 4cddd3c to aaa109f Compare June 10, 2026 15:01
@Brend-Smits Brend-Smits requested a review from a team as a code owner June 10, 2026 15:01
…nner_config

Add sqs_batch_size and sqs_batch_window fields to multi_runner_config,
allowing per-runner-flavor configuration of SQS event source mapping
batch settings.

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
@Brend-Smits Brend-Smits force-pushed the copilot/support-sqs-batch-size-window-config branch from aaa109f to 757a59a Compare June 10, 2026 15:05
@edersonbrilhante

Copy link
Copy Markdown
Contributor

@Brend-Smits this PR is removing the iam_overrides added in the PR #4875

@Brend-Smits

Copy link
Copy Markdown
Contributor

@Brend-Smits this PR is removing the iam_overrides added in the PR #4875

Good catch, thanks. Must have done something wrong when merging.

Comment thread modules/multi-runner/variables.tf
Comment thread modules/multi-runner/variables.tf
Comment thread modules/multi-runner/variables.tf
Copilot AI requested a review from Brend-Smits June 10, 2026 20:17
@Brend-Smits

Copy link
Copy Markdown
Contributor

@copilot resolve the merge conflicts in this pull request

…atch-size-window-config

# Conflicts:
#	modules/multi-runner/README.md

Copilot AI commented Jun 11, 2026

Copy link
Copy Markdown
Contributor Author

@copilot resolve the merge conflicts in this pull request

Merged origin/main into the branch and regenerated modules/multi-runner/README.md with terraform-docs. No conflicts remain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support per-runner-flavor SQS batch size and window configuration in multi_runner_config

5 participants