Skip to content

gen circleci wires run-tests-with-ats but does not scaffold .ats/main.yaml -- execute-chart-tests fails on every repo without one #2075

Description

@teemow

Problem

The generated CircleCI config for app/chart repos wires architect/run-tests-with-ats (execute-chart-tests on branches, execute-chart-tests-release on tags), but ATS reads its configuration from .ats/main.yaml, which the generator does not create. On a repo without that file, ATS's SmokeTestScenario pre-run step fails before anything is tested:

Error for config option '--smoke-tests-cluster-type': Unknown cluster type 'None' requested for tests of type 'smoke'. Known cluster types are: '['external', 'kind']'.

Observed 2026-07-04 in two repos:

  • giantswarm/fulfillment: failed on every PR and on main (release job execute-chart-tests-release red on every tag build) -- fixed by adding .ats/main.yaml in fulfillment#87
  • giantswarm/area-oncall-scheduler: the align PR (Release v5.11.0 #456) introduced the job and it failed immediately -- fixed by adding .ats/main.yaml on the alignment branch

Every repo that picks up the chart-test wiring via align-files without having ATS config will hit the same failure, showing a permanently red (non-required) check that erodes trust in CI signal and costs sweep triage time.

Proposed solution

When devctl gen circleci emits the run-tests-with-ats jobs, also scaffold a minimal .ats/main.yaml if the repo has none (only generate when missing -- the file is repo-owned afterwards):

smoke-tests-cluster-type: kind
upgrade-tests-cluster-type: kind
skip-steps: [smoke,functional,upgrade]

Repos with real ATS scenarios then remove skip-steps. Alternative: skip generating the chart-test jobs entirely when .ats/main.yaml is absent.

Acceptance criteria

  • A freshly generated app repo without ATS scenarios has a green execute-chart-tests job
  • Existing repos' hand-maintained .ats/main.yaml files are never overwritten

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions