Remove autoscan ITs and add withoutSemantic tests

[romainbrenguier]

Summary

• Remove the its/autoscan/ integration test module (test classes, pom.xml, 261 diff JSON files, CI job)
• Add test_without_semantic() unit tests to ~575 check test files to replace the autoscan coverage

The autoscan ITs validated that the Java analyzer works without bytecode by comparing results with/without compiled binaries using SonarQube Orchestrator. This is slow and heavyweight. The same coverage is achieved by adding withoutSemantic() unit tests to each rule's check test, which is faster, more granular, and easier to maintain.

Current state (WIP)

Done

• Removed its/autoscan/ directory entirely
• Removed autoscan module from its/pom.xml
• Removed autoscan CI job from .github/workflows/build.yml and from promote job's needs
• Removed Autoscan Test documentation section from README.md
• Added test_without_semantic() to ~575 check test files
• Compilation passes (mvn test-compile -pl java-checks)

Remaining work

• Run full test suite (mvn test -pl java-checks) and fix remaining failures
  • Some tests may need verifyIssues() changed to verifyNoIssues() (or vice versa)
  • A few tests may need their check expression or file path adjusted
• Manually add test_without_semantic() to skipped complex tests:
  • MissingPackageInfoCheckTest (uses onFiles() plural + caching)
  • MockitoAnnotatedObjectsShouldBeInitializedCheckTest (uses testCodeSourcesPathInModule + classpath)
  • AbstractRegexCheckTest (uses local inner class as check)
• Verify no regressions in other modules

Test plan

• mvn test -pl java-checks passes
• mvn validate -pl its/autoscan fails (module removed)
• CI workflow YAML is valid
• No other modules affected

🤖 Generated with Claude Code

✅ All code review findings resolved.

_{Configure merge blocking}

[sonarqube-next]

Quality Gate failed

Failed conditions
85.3% Coverage on New Code (required ≥ 90%)

See analysis details on SonarQube

[romainbrenguier]

We should either assert the number of false-positives, false-negatives, or have an emoji in the sample annotations to mark differences (🩹 , 💩 ...?)

[gitar-bot]

💡 Quality: WithoutSemantic sample files duplicate semantic samples and will drift

This commit introduces ~575 new *WithoutSemantic.java sample files that are near-verbatim copies of their existing semantic counterparts (e.g. PrintfMisuseCheckSampleWithoutSemantic.java vs PrintfMisuseCheckSample.java, SQLInjectionWithoutSemantic.java vs the original sample), differing only in the // Noncompliant/// FN markers. Because the code bodies are duplicated, future edits to the original samples won't propagate, so the two sets can silently diverge and the withoutSemantic coverage can rot. Consider either generating these files from the originals, or at minimum documenting the coupling so maintainers keep them in sync; where feasible, reuse the same sample file with the semantic-vs-no-semantic difference expressed only through markers.

_{Was this helpful? React with 👍 / 👎}

[gitar-bot]

Code Review 👍 Approved with suggestions 6 resolved / 7 findings

Removes the autoscan integration test module and adds lightweight withoutSemantic unit tests across check files. Consider extracting shared sample code for the new withoutSemantic tests to prevent test sample drift.

💡 Quality: WithoutSemantic sample files duplicate semantic samples and will drift

📄 java-checks-test-sources/default/src/main/java/checks/PrintfMisuseCheckSampleWithoutSemantic.java:1-15 📄 java-checks-test-sources/default/src/main/java/checks/SQLInjectionWithoutSemantic.java:1-15

This commit introduces ~575 new *WithoutSemantic.java sample files that are near-verbatim copies of their existing semantic counterparts (e.g. PrintfMisuseCheckSampleWithoutSemantic.java vs PrintfMisuseCheckSample.java, SQLInjectionWithoutSemantic.java vs the original sample), differing only in the // Noncompliant/// FN markers. Because the code bodies are duplicated, future edits to the original samples won't propagate, so the two sets can silently diverge and the withoutSemantic coverage can rot. Consider either generating these files from the originals, or at minimum documenting the coupling so maintainers keep them in sync; where feasible, reuse the same sample file with the semantic-vs-no-semantic difference expressed only through markers.

✅ 6 resolved

✅ Quality: Stray double blank line before test_without_semantic in many files

📄 java-checks/src/test/java/org/sonar/java/checks/AccessibilityChangeCheckTest.java:53-54 📄 java-checks/src/test/java/org/sonar/java/checks/ImportDeclarationOrderCheckTest.java:254-255
A large number of the modified test files introduce two consecutive blank lines between the previous method's closing brace and the new @test method (e.g. AccessibilityChangeCheckTest lines 53-54, BufferedReaderBoilerplateCheckTest, CommentsMustStartWithCorrectNumberOfSlashesCheckTest, ControlCharacterInLiteralCheckTest, DefaultEncodingUsageCheckTest, ImportDeclarationOrderCheckTest 254-255, ReadlnWithPromptCheckTest, StringIndexOfRangesCheckTest, etc.). This is a formatting inconsistency across ~575 files; collapse to a single blank line for consistency.

✅ Quality: Import of org.junit.jupiter.api.Test added out of order

📄 java-checks/src/test/java/org/sonar/java/checks/DefaultEncodingUsageCheckTest.java:19-22
In DefaultEncodingUsageCheckTest the new import org.junit.jupiter.api.Test; is appended after the org.sonar.java... imports instead of being grouped with the other org.junit.jupiter imports, breaking alphabetical import ordering (a convention this very project enforces via ImportDeclarationOrderCheck). Move the import up next to the other junit imports.

✅ Edge Case: verifyIssues() + withoutSemantic() reuses semantic sample files

📄 java-checks/src/test/java/org/sonar/java/checks/AccessibilityChangeCheckTest.java:55-63
Many new test_without_semantic() methods call verifyIssues() while pointing at the same sample file whose // Noncompliant comments were authored for a full semantic analysis (e.g. AccessibilityChangeCheckTest, MathClampMethodsCheckTest, InstanceOfPatternMatchingCheckTest). withoutSemantic() disables type resolution, so any check that relies on semantics will raise a different (usually smaller) issue set and verifyIssues() will fail because the raised issues no longer match the expected comments. This is the bulk of the acknowledged remaining work; for semantic-dependent rules the intent (verify the check doesn't crash without bytecode) is better expressed with verifyNoIssues(), and only syntax-only rules should keep verifyIssues(). Each affected test needs to be triaged individually.

✅ Quality: verifyAnalysisSucceeds() sets expectNoIssues but never verifies

📄 java-checks-testkit/src/main/java/org/sonar/java/checks/verifier/internal/InternalCheckVerifier.java:288-302
In InternalCheckVerifier.verifyAnalysisSucceeds() the call expectations.setExpectNoIssues() mutates the expectations flag, but the method only runs astScanner.scan(...) and never inspects issues or expectations afterwards. The flag has no observable effect (the noEffectParser is added unconditionally), so the line is dead/misleading code that implies an assertion the method does not actually perform. Consider removing it to keep the method's intent (only verify the analysis does not crash) clear.

✅ Quality: withoutSemantic tests weakened from verifyNoIssues to no assertion

📄 java-checks/src/test/java/org/sonar/java/checks/AnnotationDefaultArgumentCheckTest.java:43-50 📄 java-checks-testkit/src/main/java/org/sonar/java/checks/verifier/CheckVerifier.java:308-313
Switching test_without_semantic() from verifyNoIssues() to verifyAnalysisSucceeds() (e.g. AnnotationDefaultArgumentCheckTest and ~48 similar files) means these tests now only assert the analysis does not crash — they no longer detect false-positive issues raised in without-semantic mode. This is an intentional trade-off per the PR description, but for checks that previously asserted no issues it reduces regression coverage: a check newly emitting spurious issues without bytecode would pass silently. Consider keeping verifyNoIssues() where the without-semantic behavior is known to raise no issues, reserving verifyAnalysisSucceeds() for cases where issues are legitimately expected.

...and 1 more resolved from earlier reviews

🤖 Prompt for agents

Code Review: Removes the autoscan integration test module and adds lightweight withoutSemantic unit tests across check files. Consider extracting shared sample code for the new withoutSemantic tests to prevent test sample drift.

1. 💡 Quality: WithoutSemantic sample files duplicate semantic samples and will drift
   Files: java-checks-test-sources/default/src/main/java/checks/PrintfMisuseCheckSampleWithoutSemantic.java:1-15, java-checks-test-sources/default/src/main/java/checks/SQLInjectionWithoutSemantic.java:1-15

   This commit introduces ~575 new `*WithoutSemantic.java` sample files that are near-verbatim copies of their existing semantic counterparts (e.g. PrintfMisuseCheckSampleWithoutSemantic.java vs PrintfMisuseCheckSample.java, SQLInjectionWithoutSemantic.java vs the original sample), differing only in the `// Noncompliant`/`// FN` markers. Because the code bodies are duplicated, future edits to the original samples won't propagate, so the two sets can silently diverge and the withoutSemantic coverage can rot. Consider either generating these files from the originals, or at minimum documenting the coupling so maintainers keep them in sync; where feasible, reuse the same sample file with the semantic-vs-no-semantic difference expressed only through markers.

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Auto-apply	Compact
gitar auto-apply:on	gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Gitar}