fix(ci): make validation jobs capable of failing#294
Merged
Conversation
Four jobs across both pipelines ran their check loop on the right-hand side of a pipe. That runs the loop in a subshell, so the failure flag was discarded when the loop ended — GitHub yaml-lint and dry-run-apply, and GitLab yaml-lint and dry-run-apply all passed unconditionally regardless of input. GitLab's yaml-lint had the same bug in a different shape: its exit 1 exited only the subshell. This is why a Jinja syntax error reached the H4 on 2026-07-25 rather than being caught in CI. Fixing it exposed a bug it had been hiding: GitHub's dry-run-apply lacked the values.yaml/kustomization.yaml exclusions that kubeconform and the GitLab job both have, so it fed Helm values files to kubectl apply. Those failures were real but invisible. Fixing only the subshell would have turned CI red on the next commit, so both are fixed together. Loops now read from a file redirect, keeping them in the current shell. Written for POSIX sh rather than bash process substitution since the GitLab runner image's shell is not guaranteed; verified under sh and dash. Refs: docs/REVIEW-2026-07-24.md C4
kubectl apply --dry-run=client downloads the OpenAPI schema from the API server, so on a GitHub-hosted runner with no cluster it failed on every manifest: failed to download openapi: Get http://localhost:8080/openapi/v2: connect: connection refused It had done so since the day it was written. Nobody knew, because the loop ran in a subshell and the job could not report failure. Making the job capable of failing is what surfaced it. Deleted rather than patched with --validate=false, which would skip the OpenAPI check and reduce the job to 'is this parseable YAML' — already covered by yaml-lint and covered better by kubeconform. Offline schema and policy validation stay with kubeconform and conftest; real dry-run stays with GitLab CI against the actual cluster.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four jobs across both pipelines ran their check loop on the right-hand side
of a pipe. That runs the loop in a subshell, so the failure flag was
discarded when the loop ended — GitHub yaml-lint and dry-run-apply, and
GitLab yaml-lint and dry-run-apply all passed unconditionally regardless of
input. GitLab's yaml-lint had the same bug in a different shape: its exit 1
exited only the subshell.
This is why a Jinja syntax error reached the H4 on 2026-07-25 rather than
being caught in CI.
Fixing it exposed a bug it had been hiding: GitHub's dry-run-apply lacked the
values.yaml/kustomization.yaml exclusions that kubeconform and the GitLab job
both have, so it fed Helm values files to kubectl apply. Those failures were
real but invisible. Fixing only the subshell would have turned CI red on the
next commit, so both are fixed together.
Loops now read from a file redirect, keeping them in the current shell.
Written for POSIX sh rather than bash process substitution since the GitLab
runner image's shell is not guaranteed; verified under sh and dash.
Refs: docs/REVIEW-2026-07-24.md C4