Skip to content

ci: switch POM dependency detection to Scalpel-only#24579

Draft
gnodet wants to merge 1 commit into
mainfrom
ci/scalpel-only-detection
Draft

ci: switch POM dependency detection to Scalpel-only#24579
gnodet wants to merge 1 commit into
mainfrom
ci/scalpel-only-detection

Conversation

@gnodet

@gnodet gnodet commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Remove the grep-based POM dependency detection and shadow comparison infrastructure. Maveniverse Scalpel becomes the sole mechanism for detecting modules affected by POM changes.

  • Remove grep-based functions (extractPomDiff, detectChangedProperties, findAffectedModules, analyzePomDependencies) — ~330 lines
  • Remove shadow comparison function and all comparison data collection
  • Simplify PR comment to show Scalpel-detected changes directly (properties, managed deps, managed plugins)
  • Show Scalpel failure warning in PR comment when detection is unavailable
  • Update CI-ARCHITECTURE.md to reflect Scalpel-only architecture

Background: Why Scalpel-only is ready

The dual grep+Scalpel approach was designed as a gradual migration path:

  1. Phase 1: Add Scalpel alongside grep (April 2026)
  2. Phase 2: Shadow comparison to validate Scalpel across many PRs
  3. Phase 3: This PR — switch to Scalpel-only

Analysis of all successful builds since Scalpel was introduced

An investigation of all merged PRs since the Scalpel integration (April 2026) found:

For parent POM property changes (the grep approach's target):

  • Scalpel consistently detected all modules that grep found — a strict superset
  • Scalpel additionally caught modules that grep missed (managed deps without explicit ${property}, plugin versions, BOM imports)

Known grep limitations resolved by Scalpel:

  1. Managed dependencies without explicit <version> — modules inheriting versions via <dependencyManagement> without declaring <version>${property}</version> were missed
  2. Maven plugin version changes — plugin version properties consumed in parent/pom.xml via <pluginManagement> were invisible to child modules
  3. BOM imports — modules using artifacts from a BOM were not linked to the BOM version property
  4. Transitive dependency changes — only direct property references were detected
  5. Non-property version changes — structural <dependencyManagement> edits without property substitution were not caught

For container upgrade PRs (neither grep nor Scalpel involved):

Historical context: the src/testsrc/main fix

The investigation also uncovered a historical regression. On March 27, 2026, the script rewrite introduced a src/test directory check for -amd eligibility. Since test-infra modules only have src/main (no src/test), they were misclassified as "pom-only" and their downstream component modules were silently dropped from the CI reactor. ~18 container upgrade PRs merged without downstream integration tests between March 27 and April 16, 2026 — including the Infinispan 16.2 upgrade (#22316) that later required a revert (#22954) due to flaky tests that were never caught by CI. This was fixed in PR #22514 by changing the check to src/main.

Test plan

  • Script passes bash -n syntax check
  • CI runs successfully on this PR (the PR itself changes CI files, so the paths-ignore filter may skip the build — a /component-test run or manual workflow dispatch can validate)
  • Verified all grep-related functions are fully removed
  • Verified no stale variable references remain
  • Verified CI-ARCHITECTURE.md accurately describes the new architecture

Claude Code on behalf of gnodet

🤖 Generated with Claude Code

…-only

Remove the grep-based POM dependency detection and the shadow comparison
infrastructure, making Maveniverse Scalpel the sole mechanism for
detecting modules affected by POM changes.

The grep approach had known limitations that Scalpel resolves:
- Managed deps without explicit <version> were missed
- Plugin version changes were invisible to child modules
- BOM imports were not linked to version properties
- Transitive dependency changes were not detected
- Non-property version changes in <dependencyManagement> were not caught

Scalpel has been running in shadow mode alongside grep since April 2026,
validating its detection across hundreds of PRs. The dual approach was
designed as a gradual migration path (grep → dual → Scalpel-only), and
this completes the final step.

This removes ~330 lines of grep detection code and shadow comparison
infrastructure that is no longer needed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run
  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
  • You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
  • Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

@github-actions github-actions Bot added the docs label Jul 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Scalpel POM analysis unavailable: Scalpel report not found (merge-base may be unreachable in shallow clone)
ℹ️ CI did not run targeted module tests.


⚙️ View full build and test results

@davsclaus davsclaus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice cleanup completing the Scalpel migration. The code changes are clean — no stale variable references, correct function signatures, shell syntax passes bash -n. One documentation inaccuracy to fix (inline comment below).

This review does not replace specialized tools such as CodeRabbit or SonarCloud.

This review was generated by an AI agent (Claude Code on behalf of davsclaus) and may contain inaccuracies. Please verify all suggestions before applying.

The script overrides `fullBuildTriggers` to empty (`-Dscalpel.fullBuildTriggers=`) because Scalpel's default (`.mvn/**`) would trigger a full build whenever `.mvn/extensions.xml` itself changes (e.g., Dependabot bumping Scalpel).

The grep-based script fetches the PR diff via the GitHub REST API (unchanged). Scalpel uses local git history to compare effective POM models and detect source file changes — the CI workflow progressively deepens the shallow clone (`50 → 200 → 1000 → full`) until the merge-base is reachable. Scalpel runs for **all PRs** (not just POM changes) so the shadow comparison covers source-only changes too, building confidence for a future switch to Scalpel-driven builds. Scalpel disables its built-in JGit fetch (`-Dscalpel.fetchBaseBranch=false`) because the CI workflow already pre-fetches the base branch with native git (Scalpel uses JGit 5.x which has limited shallow clone support).
Root `pom.xml` changes are excluded via `-Dscalpel.excludePaths=.github/**` because it contains build-infrastructure config (license plugin, checkstyle, etc.) that does not affect module compilation or test behavior.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This statement is inaccurate. -Dscalpel.excludePaths=.github/** excludes .github directory files from Scalpel's change detection — it does not exclude the root pom.xml.

After this PR, root pom.xml changes are no longer explicitly excluded. Scalpel will compare effective POM models and only flag modules with actual property/managed-dep differences, which is likely the correct behavior. But the documentation attributes this to a mechanism that doesn't do what it claims.

Suggested change
Root `pom.xml` changes are excluded via `-Dscalpel.excludePaths=.github/**` because it contains build-infrastructure config (license plugin, checkstyle, etc.) that does not affect module compilation or test behavior.
Root `pom.xml` changes are handled naturally by Scalpel's effective POM comparison — only modules with actual property, managed dependency, or managed plugin differences are flagged. The `excludePaths=.github/**` setting excludes CI infrastructure files from Scalpel's source change detection.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants