Skip to content

Export mutateSparkPodOption and its implementations#2857

Open
everpeace wants to merge 1 commit intokubeflow:masterfrom
everpeace:export-mutatesparkpodoption
Open

Export mutateSparkPodOption and its implementations#2857
everpeace wants to merge 1 commit intokubeflow:masterfrom
everpeace:export-mutatesparkpodoption

Conversation

@everpeace
Copy link
Copy Markdown
Contributor

@everpeace everpeace commented Mar 2, 2026

Purpose of this PR

Proposed changes:

  • Export mutateSparkPodOption, which was originally defined in internal/webhook, in pkg/webhook so that so that external tools(e.g. kueue) can re-use the mutation logic

Change Category

  • Bugfix (non-breaking change which fixes an issue)
  • Feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that could affect existing functionality)
  • Documentation update

Rationale

Spark operator integration in Kueue should calculate spark driver/executor pods' shape(resource requests/limits, volume, etc.) so that kueue correctly decide the workload should be admitted or not.

Thus, mutateSparkPodOption logics are needed to re-use in Kueue integration.

ref: kubernetes-sigs/kueue#7268 (comment)

Checklist

  • I have conducted a self-review of my own code.
  • I have updated documentation accordingly.
  • I have added tests that prove my changes are effective or that my feature works.
  • Existing unit tests pass locally with my changes.

Additional Notes

Copilot AI review requested due to automatic review settings March 2, 2026 09:28
@google-oss-prow google-oss-prow Bot requested review from ImpSy and nabuskey March 2, 2026 09:28
@google-oss-prow
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign yuchaoran2011 for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@everpeace everpeace force-pushed the export-mutatesparkpodoption branch 2 times, most recently from ce4933e to 811ca12 Compare March 2, 2026 09:32
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR moves the Spark pod mutation option type and its mutation implementations out of internal/webhook into pkg/webhook, so external integrations (e.g., Kueue) can reuse the same driver/executor pod shaping logic as the operator’s mutating webhook.

Changes:

  • Added a new public pkg/webhook package file that contains MutateSparkPodOption and the Spark pod mutation functions.
  • Refactored internal/webhook Spark pod defaulter to call the exported mutation functions from pkg/webhook.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
pkg/webhook/mutate_spark_option.go Introduces exported pod mutation option type and exported mutation/helper functions for reuse by external tools.
internal/webhook/sparkpod_defaulter.go Switches the internal webhook to use the exported mutation options from pkg/webhook.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pkg/webhook/mutate_spark_option.go
Comment thread pkg/webhook/mutate_spark_option.go
Comment thread pkg/webhook/mutate_spark_option.go Outdated
… in pkg/webhook so that external tools(e.g. kueue) can re-use the mutation logic.

Signed-off-by: Shingo Omura <everpeace@gmail.com>
@everpeace everpeace force-pushed the export-mutatesparkpodoption branch from 8ca9f0e to e28433d Compare March 2, 2026 10:06
Copy link
Copy Markdown

@kannon92 kannon92 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are exporting mutate_spark_operaton, we should make sure to have unit tests.

@everpeace
Copy link
Copy Markdown
Contributor Author

Since we are exporting mutate_spark_operaton, we should make sure to have unit tests.

Actually, these methods are well tested here: https://github.com/kubeflow/spark-operator/blob/master/internal/webhook/sparkpod_defaulter_test.go

I will move the tests into pkg/webhook package from internal/webhook.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants