Fixed CLI Compatibility Matrix runs

[Catalin-Stratulat-Ericsson]

Fixed CLI Compatibility Matrix runs

---

Description

• What changed:
  • only 3 runs (latest + n-2)
  • using main latest (no longer 1.5)
  • using only semantic versions from releases (not api/test/dummy releases e.g. v1.6.0-pre1 | api/v1.0.0 )
  • each server vs ctl test is isolated in logs (for easier debug) (leads to a bit of duplication but should be a cleaner read)
• Why it’s needed: it was broken due to scraping porch "releases" that didnt have binaries/assets
• How it works: every week same as before a run is made

---

Related Issue(s)

• Closes/Fixes #

---

Type of Change

• Bug fix
• New feature
• Enhancement
• Refactor
• Documentation
• Tests
• Other: ________

---

Checklist

• Code follows project style guidelines
• Self-reviewed changes
• Tests added/updated
• Documentation added/updated
• All tests and gating checks pass

---

Testing Instructions (Optional)

1. just view the weekly/pr job

---

Additional Notes (Optional)

• Known issues:
• Further improvements:
• Review notes:

---

AI Disclosure

• I have NOT used AI in the creation of this PR.

[netlify]

✅ Deploy Preview for kpt-porch ready!

Name	Link
🔨 Latest commit	014e8ea
🔍 Latest deploy log	https://app.netlify.com/projects/kpt-porch/deploys/6a623d36d89590000940edc1
😎 Deploy Preview	https://deploy-preview-1128--kpt-porch.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[Copilot]

Pull request overview

Updates the scheduled “CLI Compatibility Matrix” GitHub Actions workflow to run a smaller, more reliable compatibility matrix by selecting only stable semver releases, testing against main for “latest”, and producing more isolated per-CLI logs/results for easier debugging.

Changes:

• Filter upstream releases to stable vX.Y.Z tags and reduce the server matrix to latest + 2 recent releases.
• Switch “latest” server checkout to main (instead of the 1.5 branch).
• Split the CLI E2E runs into separate steps (latest, n-1, n-2) with per-run logs and JSON result files.

Comments suppressed due to low confidence (2)

.github/workflows/porch-compat-matrix.yaml:228

• The regex used to extract failing tests uses \s with grep -E, but \s is not portable in POSIX ERE and won’t match whitespace on many systems (it typically matches a literal s). This can cause failed_tests to be empty even when tests fail. Use a POSIX character class instead.

            FAILED_TESTS=$(grep -E '^\s*--- FAIL:' "${LOG_FILE}" | sed 's/.*--- FAIL: //' | sed 's/ (.*//' | tr '\n' ',' | sed 's/,$//')

.github/workflows/porch-compat-matrix.yaml:261

• The regex used to extract failing tests uses \s with grep -E, but \s is not portable in POSIX ERE and won’t match whitespace on many systems (it typically matches a literal s). This can cause failed_tests to be empty even when tests fail. Use a POSIX character class instead.

            FAILED_TESTS=$(grep -E '^\s*--- FAIL:' "${LOG_FILE}" | sed 's/.*--- FAIL: //' | sed 's/ (.*//' | tr '\n' ',' | sed 's/,$//')

[mozesl-nokia]

I think the inlined scripts in the run blocks should be extracted the sh files. Since there is a lot of duplicate stuff in there, probably one parameterized script will do.

[Catalin-Stratulat-Ericsson]

I think the inlined scripts in the run blocks should be extracted the sh files. Since there is a lot of duplicate stuff in there, probably one parameterized script will do.

yes exactly. these were previously all in a for loop to not duplicate but i needed to break that to have them run under a separate name so they can be viewed and debugged independently. Ill see if i can do them in a separate script and call that. Good Suggestion

BEFORE

AFTER

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[Catalin-Stratulat-Ericsson]

hey @mozesl-nokia i believe this script should work in the job to remove the duplicated code but i cannot test it until it is merged.