Skip to content

PoC: "smart" module-path resolution#2520

Closed
koppor wants to merge 5 commits into
jbangdev:mainfrom
koppor:feature/hybrid-module-resolve
Closed

PoC: "smart" module-path resolution#2520
koppor wants to merge 5 commits into
jbangdev:mainfrom
koppor:feature/hybrid-module-resolve

Conversation

@koppor

@koppor koppor commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Related comments and issues


Claude-generated information:

Proof of concept — hybrid module-path resolution

Follow-up to #2511 / #559. PoC for the idea @quintesse raised (at #2511 (comment)): instead of special-casing JavaFX, treat the script as a class-path application but move any dependency that is a module onto the --module-path, and see if that resolves the JavaFX/jdk.jsobject case (and works in general).

Gated behind a system property so default behaviour is unchanged:

jbang -Djbang.hybrid.module.resolve=true myscript.java

What it does

When the flag is set, ModularClassPath.getAutoDectectedModuleArguments skips the JavaFX path entirely and instead:

  1. Promotes every resolved dependency that is a real (non-automatic) module onto --module-path.
  2. Walks their requires edges transitively and promotes any required module present in the resolved artifacts — including an automatic module, which is safe because a valid requires clause guarantees a usable module name.
  3. Adds the promoted modules as --add-modules roots so the boot layer resolves them.
  4. Leaves plain jars (and automatic modules nobody requires) on the class-path.

The key rule: never force an automatic module onto the module-path unless it is required by a real module. Their name is derived from the file name and may be an invalid Java identifier (e.g. fastparse_2.13-2.3.3.jarfastparse.2.13), which aborts boot-layer init — this is exactly what killed the naive "put everything on the module-path" attempt.

MWE / verification

Reproducer for the original bug (fails on stock jbang 0.138):

//JAVA 26+
//DEPS org.openjfx:javafx-web:26.0.1:${os.detected.jfxname}
//RUNTIME_OPTIONS --enable-native-access=ALL-UNNAMED

public class fxweb {
    public static void main(String[] args) {
        System.out.println("Boot layer OK: javafx.web + jdk.jsobject resolved");
    }
}

0.138FindException: Module jdk.jsobject not found, required by javafx.web.

Tested the hybrid path on a mixed dependency set that contains both real modules and the offending non-modular jar:

//DEPS org.openjfx:javafx-web:26.0.1:${os.detected.jfxname}
//DEPS com.lihaoyi:fastparse_2.13:2.3.3

With -Djbang.hybrid.module.resolve=true:

  • javafx.* + jdk.jsobject → module-path
  • fastparse_2.13 (+ transitives) → class-path
  • boots fine, no JavaFX special-casing involved

Status

This is a proof of concept for discussion, not a finished feature — happy to iterate on how/whether to expose it to users (and whether it should eventually replace the JavaFX special-casing merged in #2511).

🤖 Generated with Claude Code

koppor and others added 2 commits June 8, 2026 11:49
Proof of concept for jbangdev#2511: instead of special-casing JavaFX,
treat the script as a class-path application but promote every resolved
dependency that is a real (non-automatic) module onto the `--module-path`,
adding those modules as `--add-modules` roots so the boot layer resolves
them and their `requires` transitively. Plain jars and automatic modules
stay on the class-path.

Leaving automatic modules on the class-path is the key: their name is
derived from the file name and may be an invalid Java identifier (e.g.
`fastparse_2.13-2.3.3.jar` -> `fastparse.2.13`), which aborts boot-layer
initialization if such a jar is forced onto the module-path.

Gated behind the system property `jbang.hybrid.module.resolve` so the
default behaviour is unchanged. Verified on a mixed dependency set
(org.openjfx:javafx-web:26.0.1 + com.lihaoyi:fastparse_2.13:2.3.3): with
the flag set, javafx.web/jdk.jsobject land on the module-path, fastparse
stays on the class-path, and the app boots -- without any JavaFX
special-casing.

Refs jbangdev#559, jbangdev#2511

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…dule

Extends the hybrid PoC to follow `requires` edges. The seed is still the set
of real (non-automatic) modules, but resolution now walks their `requires`
transitively and promotes any required module that is present in the resolved
artifacts -- including an *automatic* module.

This is safe: a module can only appear in a `requires` clause if its name is a
valid Java identifier, so an automatic module that is actually required is
guaranteed to have a usable name and can live on the module-path. Automatic and
plain jars that nobody requires still stay on the class-path, where their
file-name-derived (possibly invalid) module name does no harm.

Refs jbangdev#559, jbangdev#2511

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)
  • ai-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d0bd4c44-72c9-4dc3-bc19-9d9240f77828

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@koppor koppor changed the title PoC: hybrid module-path resolution (-Djbang.hybrid.module.resolve) PoC: hybrid module-path resolution (-Djbang.hybrid.module.resolve) Jun 8, 2026
@quintesse

Copy link
Copy Markdown
Contributor

Nice. Can we try and see what happens if we remove all the special JavaFx detection code? Do things still keep working?

@quintesse

Copy link
Copy Markdown
Contributor

Or better yet, do what @maxandersen suggested (assuming things keep working) and automatically enable that feature when we detect JavaFx.

…casing

Module-path resolution used to kick in only for JavaFX (string-matched on
`org/openjfx/javafx-`), while the generic hybrid path sat behind the
`-Djbang.hybrid.module.resolve` PoC flag.

Run hybrid resolution unconditionally (JDK 9+): promote every resolved real
(non-automatic) module plus its transitive `requires` onto the `--module-path`
and add them as roots, leaving plain/automatic jars on the class-path. This
covers JavaFX, `//MODULE` scripts, and any other modular dependency uniformly,
so the JavaFX-specific detection (`hasJavaFX`, the `javafx`/`jdk.jsobject` name
matching) and the PoC flag are no longer needed.

`jdk.jsobject` (required by `javafx.web`, shipped separately since JDK 26) is
still handled, now via the requires-graph rather than a hard-coded name.

Verified: TestRun, DependencyResolverTest and TestArtifactInfo pass, including
all testJavaFX* cases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@koppor koppor changed the title PoC: hybrid module-path resolution (-Djbang.hybrid.module.resolve) PoC: "smart" module-path resolution Jun 8, 2026
@koppor

koppor commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

Tried it — removed all the JavaFX-specific detection (hasJavaFX, the org/openjfx/javafx./jdk.jsobject name-matching) and made the generic hybrid resolution the unconditional path (JDK 9+). Things keep working: TestRun, DependencyResolverTest and TestArtifactInfo are all green, including every testJavaFX* case. jdk.jsobject (needed by javafx.web, shipped separately since JDK 26) is still picked up — now via the requires graph instead of a hard-coded name.

On @maxandersen's suggestion to only flip the feature on when we detect JavaFX: I started down that road but //MODULE scripts kill it. Those launch via -m <module>/<main> and need a proper --module-path for their deps even when there's no JavaFX in sight, so we'd have to trigger on "JavaFX or //MODULE or …" — and the next modular case after that. Running hybrid resolution unconditionally covers JavaFX, //MODULE, and any other modular dependency with one code path, so I kept it always-on.

As a bonus the redundancy worry goes away: the only jars now living on both module-path and class-path are the JavaFX ones, which is exactly what the old code already did.


Edit: Investigating failing tests

Making the hybrid module-path promotion unconditional (dc3ef46) broke the
smoke-quarkus-test. A plain class-path app such as a Quarkus CLI fails at
startup with:

  Exception in thread "main" java.lang.ExceptionInInitializerError
  Caused by: java.lang.IllegalArgumentException: This library does not have
    private access to interface io.smallrye.config._private.ConfigLogging

because hybrid promotes `jboss-logging` to a named module on the
`--module-path` while `smallrye-config` stays on the class-path (unnamed
module). A named module cannot read the unnamed module, so jboss-logging's
reflective logger-proxy generation into smallrye-config's `_private` package
is denied. Frameworks like Quarkus rely on the relaxed access of the
class-path and cannot run as auto-generated named modules.

Gate the hybrid resolution on `hasJavaFX()` again (keeping the generic
requires-graph logic, dropping the name-matching). `//MODULE` scripts don't
need this path: the command generators already put their whole class-path on
the module-path via `-p` and launch with `-m`, so JavaFX detection is the
correct and sufficient trigger.

Verified: TestRun, DependencyResolverTest, TestArtifactInfo and TestModule
pass; the qcli smoke test runs again (prints "Hello World!!", class-path only).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@koppor

koppor commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

Seems like we can't be "brute force", but with some "intelligence" -- 1f0e725 - revealed by the tests 😅


Claude's text

Tried @maxandersen's "remove all the special JavaFX detection" idea (commit dc3ef46e, hybrid resolution unconditional for every script). Unit suites (TestRun, DependencyResolverTest, TestArtifactInfo, TestModule) all stayed green and JavaFX kept working — but the smoke-quarkus-test broke:

Exception in thread "main" java.lang.ExceptionInInitializerError
Caused by: java.lang.IllegalArgumentException: This library does not have private access to interface io.smallrye.config._private.ConfigLogging`

Root cause: hybrid promotes jboss-logging (a real module) to a named module on the --module-path, while smallrye-config (automatic, nobody requires it) stays on the class-path = unnamed module. A named module can't read the unnamed module, so jboss-logging's reflective logger-proxy generation into smallrye-config's _private package is denied. Frameworks like Quarkus depend on the relaxed access of the class-path and can't run as auto-generated named modules. So promoting everything isn't viable.

Where it landed (1f0e7257): keep JavaFX detection as the trigger — exactly your "enable when we detect JavaFX" suggestion — but replace the brittle name-matching (javafx.* / jdk.jsobject) with the generic requires-graph hybrid logic, and drop the -Djbang.hybrid.module.resolve PoC flag. So:

  • JavaFX → hybrid promotes the real modules + their transitive requires (this is how jdk.jsobject, required by javafx.web and shipped separately since JDK 26, lands on the module-path — no hard-coded name anymore).
  • Plain class-path apps (Quarkus, …) → class-path only, exactly as before.
  • //MODULE scripts → unaffected; the worry that pushed me toward always-on turned out moot. They never used this code path — the command generators already put the whole class-path on the module-path via -p and launch with -m. Triggering hybrid for them would just add a conflicting second --module-path.

Verified the four test suites pass and the qcli smoke test runs again (Hello World!!, class-path only).

@maxandersen

Copy link
Copy Markdown
Collaborator

...so what happens now when using quarkus-fx that includes javafx? :)

@quintesse

quintesse commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

I don't think the idea was to always turn on the hybrid mode, there's bound to be situations where that will not work, or where it is simply unwanted.

And what I meant by Max's idea was : don't have special code to change the class/module path (which is what this PR does), but it should still use the special detection that decides when to turn this feature on!

Also, this should be skipped when //MODULE is used because then everything should just go on the module path, no need to check anything. (you already explained that this code is never used in that case)

...so what happens now when using quarkus-fx that includes javafx? :)

Well ... what does the current JBang do in that case?

@koppor

koppor commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

I don't think the idea was to always turn on the hybrid mode, there's bound to be situations where that will not work, or where it is simply unwanted.

Currently, it seems, it works in most cases 😅. I was so happy that it worked w/ JavaFX without any issues - and then I saw this Quarkus test 🙈


Will keep you updated.

@quintesse

Copy link
Copy Markdown
Contributor

Oh, you responded quick, so you might not have read the edit:

And what I meant by Max's idea was : don't have special code to change the class/module path (which is what this PR does), but it should still use the special detection that decides when to turn this feature on!

So it's either the PoC property that turns it on or the old JavaFx detection. But using the new smart path code.

The hybrid resolution promoted *every* real (non-automatic) module onto
the module-path once JavaFX was detected. For a quarkus-fx style app this
still broke startup identically to the original Quarkus regression: a
named jboss-logging on the module-path cannot build a logger proxy for
smallrye-config's ConfigLogging interface that stays in the unnamed
module (class-path).

Seed the requires-graph from the JavaFX modules only and follow their
requires edges, matching PR jbangdev#2511's actual intent ("put modules required
by JavaFX on the module-path"). javafx.* and what they require (e.g.
jdk.jsobject) land on the module-path; everything else - including
unrelated real modules like jboss-logging - stays on the class-path.

Add a regression test for the non-//MODULE case (JavaFX next to an
unrelated real module).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@koppor

koppor commented Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Generated with Claude Code

Good catch on quarkus-fx — I checked, and it did still break, even with the JavaFX gate. 🤖

A quarkus-fx app (Quarkus + JavaFX) failed identically to the original Quarkus regression:

Caused by: java.lang.IllegalArgumentException: This library does not have private access to interface io.smallrye.config._private.ConfigLogging
	at org.jboss.logging@3.6.0.Final/org.jboss.logging.Logger.getMessageLogger(Logger.java:2569)

Root cause: once JavaFX was detected, the hybrid resolution promoted every real (non-automatic) module onto the module-path — including jboss-logging (the @3.6.0.Final in the trace → it's a named module). Its collaborator smallrye-config stayed on the class-path (unnamed module), so the named jboss-logging couldn't reflect into smallrye-config's ConfigLogging to build its logger proxy.

Fix: seed the requires-graph from the JavaFX modules only and follow their requires edges — exactly what PR #2511 set out to do ("put modules required by JavaFX on the module-path"):

  • javafx.* + what they require (e.g. jdk.jsobject) → module-path ✅
  • unrelated real modules like jboss-logging / smallrye-config → stay on the class-path ✅

This narrow scoping is also why no -Djbang.hybrid.module.resolve opt-in flag is needed (the PoC had one only because it was unconditional and risky):

  • Non-JavaFX apps (incl. plain Quarkus): getAutoDectectedModuleArguments returns empty → byte-for-byte the same class-path launch as before. Nothing to guard or opt out of.
  • JavaFX apps: the module-path isn't a preference, it's required to run at all. A flag would just mean "JavaFX is broken until you find the magic property."

So it differs only where it must and is provably inert everywhere else — safe as the default, with nothing for a toggle to add.

Verified qfx.java now runs with a JavaFX-only --module-path and jboss-logging/smallrye-config back on -classpath. Added a regression test for the non-//MODULE JavaFX-plus-unrelated-real-module case.

Notes / scope
  • Governs the non-//MODULE path only — getAutoDectectedModuleArguments is the sole module-path source for plain scripts. //MODULE scripts use the generator's -p <whole class-path> + -m, unaffected here.
  • Side finding (out of scope): //MODULE + JavaFX fails earlier at compile time (package javafx.collections is not visible) — pre-existing, independent of this PR.

@koppor

koppor commented Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

Uff, I think, I am running in a circle now -- maybe, only support the test case as PR - and remove the other thing? -- I think, it's rubbish. Only JavaFX handling. No one asked for it to make the code worse as before. -- Hybrid was an idea, but creating dozens (?) of MWEs (by Claude) showed: Not working...

@quintesse

Copy link
Copy Markdown
Contributor

Well, then we just leave the JavaFx detection but using the more generic path modification. That we we don't have to hard-code extra dependencies that might or might not exists and it also means we could more easily add other modules to the list if we ever find other well-known often-used modules that we'd like to put on the module path by default.

@koppor

koppor commented Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

Current state & verification 🤖

After the quarkus-fx fix I wanted to pin down what this branch actually changes vs main (#2511), so I built both main (31fa8d53) and the current tip (3e8ca2da) and ran the same 7 minimal scripts against each (scripts at the bottom of this comment).

Results

✅ = runs, ❌ = fails. ᵛ = verified by actually running it (or reproduced earlier in this PR); unmarked cells are reasoned from the code.

Commit / mode C1 plain
(non-fx, mod+non-mod, Quarkus)
C2 fx only C3 fx + non-mod C4 fx + mod C5 fx + mod + non-mod
(quarkus-fx)
31fa8d53 main (#2511) — javafx names + hardcoded jdk.jsobject ✅ᵛ ✅ᵛ ✅ᵛ ✅ᵛ ✅ᵛ
2aa50767 PoC hybrid, toggle off
2aa50767 / dc3ef46e hybrid on / unconditional default ❌ᵛ
1f0e7257 fx-gated, promote all real modules ❌ᵛ
3e8ca2da current — fx-gated, seed from fx only ✅ᵛ ✅ᵛ ✅ᵛ ✅ᵛ ✅ᵛ
//MODULE path (orthogonal; identical on main & current) ✅ᵛ (non-fx) ❌ᵛ ❌ᵛ ❌ᵛ ❌ᵛ

Note on C1/C5: jbang's Quarkus integration replaces main with GeneratedMain, which boots the app and blocks. A clean boot therefore hangs (success); the regression instead fails fast during Config static-init. So these were run under a timeout — "booted, blocked → timeout" = pass, the ConfigLogging exception = fail.

Honest takeaway

For all five runtime scenarios, main and the current tip behave identically. The hybrid detour (unconditional → fx-gated → promote-all → seed-from-fx) lands back where main already was. The durable delta over main is:

  1. Removed the hardcoded jdk.jsobject / JAVAFX_PREFIX special-cases → a generic requires-graph walk seeded from JavaFX modules (equivalent for JavaFX today, but auto-discovers any module a future JDK might extract that JavaFX requires).
  2. Dropped the experimental -Djbang.hybrid.module.resolve toggle.
  3. Added the quarkus-fx regression test.

i.e. cleanup/hardening, not new capability. The "promote arbitrary modular deps to the module-path" idea was proven to break reflective frameworks (C1, then C5) and was reverted.

The one genuinely unsolved limitation

//MODULE + JavaFX fails on both main and this branch, at compile:

error: package javafx.collections is not visible

Root cause: openjfx ships a stub javafx-base.jar (empty automatic module javafx.baseEmpty) plus a classifier javafx-base-<os>.jar (the real module javafx.base carrying the classes). ModuleUtil.generateModuleInfo matches the root dep org.openjfx:javafx-base only against the non-classifier artifact's management key, so it emits requires javafx.baseEmpty; (empty) instead of requires javafx.base; — and the module can't see javafx.collections. Fixing this would be real new value over main.


Minimal jbang test scripts used above

C1 — plain Quarkus (non-fx, modular + non-modular, reflective)

//DEPS io.quarkus:quarkus-core:3.15.1
public class C1PlainQuarkus {
    public static void main(String... args) { System.out.println("C1 plain-quarkus ok"); }
}

C2 — JavaFX only

//DEPS org.openjfx:javafx-base:21.0.2
import javafx.collections.FXCollections;
import javafx.collections.ObservableList;
public class C2Fx {
    public static void main(String... args) {
        ObservableList<String> l = FXCollections.observableArrayList("hi");
        System.out.println("C2 fx ok: " + l);
    }
}

C3 — JavaFX + non-modular jar (docopt)

//DEPS org.openjfx:javafx-base:21.0.2
//DEPS com.offbytwo:docopt:0.6.0.20150202
import javafx.collections.FXCollections;
import javafx.collections.ObservableList;
public class C3FxPlain {
    public static void main(String... args) {
        ObservableList<String> l = FXCollections.observableArrayList("hi");
        System.out.println("C3 fx+plain ok: " + l + " / " + org.docopt.Docopt.class.getName());
    }
}

C4 — JavaFX + modular jar (jboss-logging), no cross-module reflection

//DEPS org.openjfx:javafx-base:21.0.2
//DEPS org.jboss.logging:jboss-logging:3.6.0.Final
import javafx.collections.FXCollections;
import javafx.collections.ObservableList;
import org.jboss.logging.Logger;
public class C4FxMod {
    public static void main(String... args) {
        ObservableList<String> l = FXCollections.observableArrayList("hi");
        Logger.getLogger(C4FxMod.class).info("hello from jboss-logging");
        System.out.println("C4 fx+mod ok: " + l);
    }
}

C5 — JavaFX + Quarkus (modular + non-modular + reflection = quarkus-fx)

//DEPS io.quarkus:quarkus-core:3.15.1
//DEPS org.openjfx:javafx-base:21.0.2
public class C5FxQuarkus {
    public static void main(String... args) { System.out.println("C5 fx+quarkus ok"); }
}

//MODULE + non-JavaFX (picocli) — works

//MODULE
//DEPS info.picocli:picocli:4.6.3
package mplain;
public class MPlain {
    public static void main(String... args) {
        System.out.println("M-plain module ok: " + picocli.CommandLine.class.getName());
    }
}

//MODULE + JavaFX — fails to compile (the limitation)

//MODULE
//DEPS org.openjfx:javafx-base:21.0.2
package mfx;
import javafx.collections.FXCollections;
public class MFx {
    public static void main(String... args) {
        System.out.println("M-fx module ok: " + FXCollections.observableArrayList("hi"));
    }
}

🤖 Generated with Claude Code

@koppor koppor closed this Jun 9, 2026
@koppor koppor deleted the feature/hybrid-module-resolve branch June 9, 2026 10:58
@quintesse

Copy link
Copy Markdown
Contributor

Why did you close the PR? Isn't point 1 interesting enough to merge?

@koppor

koppor commented Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

Why did you close the PR? Isn't point 1 interesting enough to merge?

I thought some clean working //MODULE thing would be nicer - #2525

That we we don't have to hard-code extra dependencies that might or might not exists and it also means we could more easily add other modules to the list if we ever find other well-known often-used modules that we'd like to put on the module path by default.

My bet is: JavaFX will never add other non-javafx modules as dependency. If they will, we can change the algorithm to a more generic lookup. - OK, I might lean towards the YAGNI principle too hard?

@quintesse

quintesse commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

OK, I might lean towards the YAGNI principle too hard?

I simply dislike hard-coded stuff :-)
I could also imagine that at some point there would be another situation where we'd need something like this and we'd go "cool, we already have code for that!". Now, if you still had to go and write it, I'd definitely agree, but having written it to only throw it away ... that seems like a waste :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants