Skip to content

build: migrate from Allure 2 to Allure 3#2501

Open
maxandersen wants to merge 5 commits into
mainfrom
build/allure3-migration
Open

build: migrate from Allure 2 to Allure 3#2501
maxandersen wants to merge 5 commits into
mainfrom
build/allure3-migration

Conversation

@maxandersen

@maxandersen maxandersen commented May 31, 2026

Copy link
Copy Markdown
Collaborator

TL;DR: pr to see how allure 3 looks like. I really don't want to introduce yet another layer that relies on nodejs but we already use it for docs generation so maybe not too much of a loss?

Summary

Migrate test reporting from Allure 2 to Allure 3.

Changes

  • Plugin: Replace allure-report + allure-adapter-base v2.12.0 with single io.qameta.allure v4.0.2
  • Report generator: 2.33.0 → 3.9.0 (Allure 3)
  • allure-bom: 2.29.1 → 2.35.2
  • AspectJ weaver: Remove manual -javaagent config — plugin 4.x auto-manages it
  • Gradle API: Modernize reporting.baseDirreporting.baseDirectory
  • Workaround: Add buildscript { classpath 'commons-io:commons-io:2.22.0' } to fix a classpath conflict (Gradle ships commons-io 2.15.1 which lacks a method required by the plugin's commons-compress 1.28.0)

What stays the same

  • No test source code changes — allure-java stays at 2.x, all @Description, @Issue, and Allure.step() usage is unchanged
  • CI workflow unchanged — report output path (build/reports/allure-report/allureReport/index.html), artifact upload patterns, and multi-OS result merging all work the same
  • Single-file HTML report still generated

Verified locally

  • ./gradlew clean test allureReport — 629 tests pass (601 passed, 28 skipped)
  • ./gradlew --no-daemon allureReport --clean — works (matches CI usage)
  • Single-file report generated at unchanged path
  • allureRawResultElements merge mechanism intact

Summary by CodeRabbit

  • Chores
    • Updated Allure reporting configuration to version 3 with custom grouping by test suite and tag
    • Updated build dependencies including Allure Gradle plugin to version 4.0.2
    • Modified test execution setup by removing AspectJ instrumentation and refactoring JVM argument handling
    • Updated reporting directory configuration to use standardized build directory structure

Adds the Allure 3 report configuration file at the project root.
Configures custom grouping by parentSuite (platform) then tag
(unit-test/integration-test).

Note: The Gradle plugin 4.x generates its own config and passes it via
--config, so this file is not auto-discovered by Gradle. It is used when
running the Allure 3 CLI directly. The plugin's default grouping
(parentSuite/suite/subSuite) already satisfies the grouping requirement.
- Replace allure-report + allure-adapter-base v2.12.0 with single
  io.qameta.allure plugin v4.0.2
- Update report generator from 2.33.0 to 3.9.0
- Bump allure-bom from 2.29.1 to 2.35.2
- Remove manual AspectJ weaver config (plugin 4.x auto-manages it)
- Modernize reporting.baseDir to reporting.baseDirectory
- Add buildscript commons-io 2.22.0 to work around Gradle's bundled
  commons-io 2.15.1 conflicting with the plugin's commons-compress

No test source code changes required — allure-java API stays at 2.x.
CI workflow unchanged — report output path is the same.
@coderabbitai

coderabbitai Bot commented May 31, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 803fe6ad-7205-4865-9fef-cbcb1b7d8646

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@maxandersen maxandersen added the ai-review used to explicitly request ai-review label May 31, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@build.gradle`:
- Around line 3-10: The buildscript block currently declares a
buildscript.dependencies entry for classpath 'commons-io:commons-io:2.22.0' but
has no buildscript.repositories, so that dependency can fail to resolve; add a
buildscript.repositories block (e.g., include mavenCentral() and any other
needed repos like google() or gradlePluginPortal()) inside the buildscript scope
before the dependencies block so the commons-io artifact can be fetched during
configuration.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d0765316-929d-436c-9da3-284ac6a40621

📥 Commits

Reviewing files that changed from the base of the PR and between 021cee0 and d899060.

📒 Files selected for processing (2)
  • allurerc.mjs
  • build.gradle

Comment thread build.gradle
Comment on lines +3 to +10
buildscript {
dependencies {
// Force commons-io 2.22.0+ on the buildscript classpath.
// Gradle ships commons-io 2.15.1 which lacks the Supplier overload
// of IOUtils.skip() required by commons-compress 1.28.0 (used by
// the Allure Gradle plugin's Node.js download task).
classpath 'commons-io:commons-io:2.22.0'
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Locate the Gradle file(s) referenced
ls -la
find . -maxdepth 3 -name "build.gradle" -o -name "build.gradle.kts" | sed 's|^\./||'

echo "---- build.gradle (first 140 lines) ----"
if [ -f build.gradle ]; then
  nl -ba build.gradle | sed -n '1,140p'
else
  echo "build.gradle not found at repo root"
fi

echo "---- build.gradle (search for buildscript.repositories / buildscript { repositories ) ----"
if [ -f build.gradle ]; then
  rg -n --hidden --no-ignore-vcs "buildscript\\s*\\{|buildscript\\.repositories|repositories\\s*\\{" build.gradle
  echo "---- grep specifically for buildscript repositories block ----"
  rg -n --hidden --no-ignore-vcs "buildscript\\s*\\{[\\s\\S]{0,400}?repositories\\s*\\{" build.gradle || true
fi

echo "---- Search entire repo for buildscript classpath / repositories in buildscript ----"
rg -n --hidden --no-ignore-vcs "classpath ['\"]commons-io:commons-io:2\\.22\\.0['\"]|buildscript\\.repositories|buildscript\\s*\\{[\\s\\S]{0,300}?repositories\\s*\\{" . || true

Repository: jbangdev/jbang

Length of output: 2470


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Show build.gradle with line numbers (up to 160)
echo "---- build.gradle (1-160) ----"
if [ -f build.gradle ]; then
  awk '{printf "%6d\t%s\n", NR, $0}' build.gradle | sed -n '1,160p'
else
  echo "build.gradle not found"
fi

echo "---- rg: buildscript.repositories or buildscript { ... repositories { ----"
rg -n --hidden --no-ignore-vcs "buildscript\\.repositories|buildscript\\s*\\{[[:space:][:print:]]{0,600}?repositories\\s*\\{" build.gradle || true

echo "---- rg: commons-io classpath ----"
rg -n --hidden --no-ignore-vcs "commons-io:commons-io:2\\.22\\.0" build.gradle || true

Repository: jbangdev/jbang

Length of output: 5634


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "---- buildscript.repositories across repo ----"
rg -n --hidden --no-ignore-vcs "buildscript\\.repositories" . || true

echo "---- any buildscript { blocks across repo ----"
rg -n --hidden --no-ignore-vcs "buildscript\\s*\\{" . || true

echo "---- commons-io classpath across repo ----"
rg -n --hidden --no-ignore-vcs "commons-io:commons-io:2\\.22\\.0" . || true

Repository: jbangdev/jbang

Length of output: 286


🏁 Script executed:

#!/bin/bash
set -euo pipefail
echo "---- settings.gradle (1-220) ----"
if [ -f settings.gradle ]; then
  awk '{printf "%6d\t%s\n", NR, $0}' settings.gradle | sed -n '1,220p'
else
  echo "settings.gradle not found"
fi

Repository: jbangdev/jbang

Length of output: 895


🌐 Web query:

Gradle buildscript { dependencies { classpath ... } } without buildscript.repositories use project repositories or default repositories?

💡 Result:

In Gradle, the buildscript block does not automatically inherit or use the repositories defined in the project's standard repositories {} block [1][2]. The buildscript block exists in a separate scope specifically for configuring the build script's own classpath (e.g., loading plugins or custom build logic), while the standard repositories {} block is used to resolve the project's application dependencies [3][1][2]. If you define dependencies within a buildscript block but omit the repositories {} block inside it, Gradle will not automatically look at your project-level repositories [1]. Instead, Gradle typically fails with an error indicating that it cannot resolve the dependencies because no repositories are defined [4][5]. While some users may observe behavior where specific dependencies resolve even without an explicit repository block—sometimes leading to the assumption that implicit repositories are included—this is generally due to how specific Gradle versions or environments are configured (e.g., pre-configured default repositories in certain plugins or environments) [4]. For predictable and portable builds, you must explicitly declare the necessary repositories inside the buildscript block if your dependencies are not available in standard public locations or if you want to ensure the build does not rely on implicit, environment-specific defaults [4][6]. In modern Gradle development, the recommended approach is to move away from the legacy buildscript block entirely in favor of the plugins {} block [7]. The plugins {} block integrates with pluginManagement in settings.gradle, which provides a centralized and more robust way to manage plugin repositories and versions across multi-project builds [7].

Citations:


Add buildscript.repositories for the commons-io buildscript classpath dependency.

buildscript.dependencies won’t use the later project-level repositories { ... }, so commons-io:commons-io:2.22.0 may fail to resolve on a clean environment.

Suggested fix
 buildscript {
+	repositories {
+		mavenCentral()
+	}
 	dependencies {
 		// Force commons-io 2.22.0+ on the buildscript classpath.
 		// Gradle ships commons-io 2.15.1 which lacks the Supplier overload
 		// of IOUtils.skip() required by commons-compress 1.28.0 (used by
 		// the Allure Gradle plugin's Node.js download task).
 		classpath 'commons-io:commons-io:2.22.0'
 	}
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@build.gradle` around lines 3 - 10, The buildscript block currently declares a
buildscript.dependencies entry for classpath 'commons-io:commons-io:2.22.0' but
has no buildscript.repositories, so that dependency can fail to resolve; add a
buildscript.repositories block (e.g., include mavenCentral() and any other
needed repos like google() or gradlePluginPortal()) inside the buildscript scope
before the dependencies block so the commons-io artifact can be fetched during
configuration.

Add allurerc.mjs with environment matchers that group test results by
OS (Linux, macOS, Windows) using the existing parentSuite labels. This
enables Allure 3's side-by-side Environments view in the merged report.

The Gradle plugin's allureReport task generates a minimal JSON config
that can't express JS-based matchers, so a finalizedBy task re-runs the
CLI with allurerc.mjs to overwrite the report. This runs automatically
whenever allureReport is invoked, including from CI.
The old URL returned 404. Updated to the correct Allure 3 docs path.
@maxandersen maxandersen force-pushed the build/allure3-migration branch from f6c03b0 to c7f550f Compare May 31, 2026 07:51
@maxandersen

Copy link
Copy Markdown
Collaborator Author

opened these issues thus far:

allure-framework/allure-gradle#201 - cant tell gradle about allure3 config
allure-framework/allure-gradle#200 - jar conflict

The allureReportCustomConfig task hardcoded the Unix executable path
(allure), which doesn't exist on Windows. Use allure.bat (or .cmd
fallback) matching how the Allure Gradle plugin resolves the executable.
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-review used to explicitly request ai-review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant