Skip to content

compat: Support the TypeEq type-system refactor (JuliaLang/julia#61915)#718

Open
Keno wants to merge 1 commit into
JuliaIO:masterfrom
KenoAIStaging:kf/typeeq-compat
Open

compat: Support the TypeEq type-system refactor (JuliaLang/julia#61915)#718
Keno wants to merge 1 commit into
JuliaIO:masterfrom
KenoAIStaging:kf/typeeq-compat

Conversation

@Keno

@Keno Keno commented Jun 4, 2026

Copy link
Copy Markdown
Member

The TypeEq refactor changes typeof(Type{T}) from DataType to the new Core.TypeEq. As a consequence, a Type{T} value is no longer an instance of DataType (isa(Type{T}, DataType) is now false), even though Type{T} <: DataType still holds and Base.isType(Type{T}) is true.

JLD2 dispatches its DataType serialization machinery on typeof(x), so on such a Julia a value whose type is a Type{T} (for example the field of a struct declared x::Type{Float64}) was routed through the generic compound path instead of the dedicated DataType path. This surfaced as a MethodError in commit and downstream fieldcount/convert errors when writing such values.

Introduce a typeof_serializable helper that maps Core.TypeEq-typed values back to DataType (guarded by isdefined(Core, :TypeEq) so the code still loads on released Julia), and relax the remaining value-level DataType annotations and isa(x, DataType) checks to also accept Type{T} values. These changes keep the on-disk format identical to older Julia versions.

This change was made with the assistance of generative AI (Claude Code).

The TypeEq refactor changes `typeof(Type{T})` from `DataType` to the new
`Core.TypeEq`. As a consequence, a `Type{T}` value is no longer an instance
of `DataType` (`isa(Type{T}, DataType)` is now `false`), even though
`Type{T} <: DataType` still holds and `Base.isType(Type{T})` is `true`.

JLD2 dispatches its `DataType` serialization machinery on `typeof(x)`, so on
such a Julia a value whose type is a `Type{T}` (for example the field of a
struct declared `x::Type{Float64}`) was routed through the generic compound
path instead of the dedicated `DataType` path. This surfaced as a
`MethodError` in `commit` and downstream `fieldcount`/`convert` errors when
writing such values.

Introduce a `typeof_serializable` helper that maps `Core.TypeEq`-typed values
back to `DataType` (guarded by `isdefined(Core, :TypeEq)` so the code still
loads on released Julia), and relax the remaining value-level `DataType`
annotations and `isa(x, DataType)` checks to also accept `Type{T}` values.
These changes keep the on-disk format identical to older Julia versions.

This change was made with the assistance of generative AI (Claude Code).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@codecov

codecov Bot commented Jun 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.39%. Comparing base (5d7a409) to head (e282cb5).
⚠️ Report is 7 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #718      +/-   ##
==========================================
+ Coverage   87.38%   87.39%   +0.01%     
==========================================
  Files          37       37              
  Lines        4588     4593       +5     
==========================================
+ Hits         4009     4014       +5     
  Misses        579      579              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@JonasIsensee

Copy link
Copy Markdown
Collaborator

Thanks for providing the update!

Codecov complains that the isdefined(Core, :TypeEq) == true branch is not hit as part of the tests.
Is this change not yet merged into julia nightly ?

+ @static if isdefined(Core, :TypeEq)
+     @nospecializeinfer typeof_serializable(@nospecialize(x)) =
+         isa(x, Core.TypeEq) ? DataType : typeof(x)
+ else
+     @nospecializeinfer typeof_serializable(@nospecialize(x)) = typeof(x)
+ end

@Keno

Keno commented Jun 4, 2026

Copy link
Copy Markdown
Member Author

Codecov complains that the isdefined(Core, :TypeEq) == true branch is not hit as part of the tests.
Is this change not yet merged into julia nightly ?

Merged after the PR, but before your comment ;)

@Keno

Keno commented Jun 4, 2026

Copy link
Copy Markdown
Member Author

That said, the nightly build with it probably isn't done yet.

@JonasIsensee

Copy link
Copy Markdown
Collaborator

Let's wait until then, retrigger CI, and merge when tests pass with the relevant code change covered.

# A ghost type, so no need to store at all
return nothing
elseif isa(T, DataType) && sizeof(T) ≤ MAX_INLINE_SIZE
elseif (isa(T, DataType) || Base.isType(T)) && sizeof(T) ≤ MAX_INLINE_SIZE

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think a change is needed here because this is inside an if isconcretetype(T) block.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
elseif (isa(T, DataType) || Base.isType(T)) && sizeof(T) MAX_INLINE_SIZE
elseif isa(T, DataType) && sizeof(T) MAX_INLINE_SIZE

@nhz2

nhz2 commented Jul 6, 2026

Copy link
Copy Markdown
Member

The tests are still missing coverage for these changes.

Examples of things still broken:

Unions are not saved correctly.

julia> jldsave("union_new.jld2"; x = Union{Nothing, Type{Int}})

julia> load("union_new.jld2")
Dict{String, Any} with 1 entry:
  "x" => Nothing

Type{Int} values cannot be read in older julia versions.

In Julia nightly

julia> jldsave("datatypes_new.jld2"; x = Type{Int})

julia> load("datatypes_new.jld2")
Dict{String, Any} with 1 entry:
  "x" => Type{Int64}

Loading this file in Julia 1.12.6

julia> load("datatypes_new.jld2")
┌ Warning: custom serialization of Core.TypeEq encountered, but the type does not exist in the workspace; the data will be read unconverted
└ @ JLD2 ~/.julia/packages/JLD2/qcxKY/src/data/reconstructing_datatypes.jl:112
Dict{String, Any} with 1 entry:
  "x" => UnknownType{Symbol("Core.TypeEq"), Tuple{Int64}}

There is a 2x file size regression storing arrays of Type{Int}, not sure this is practical problem, but it does show there was some change to the representation.

1.12.6:

julia> jldsave("typearray.jld2"; x = fill(Type{Int}, 100000))

julia> filesize("typearray.jld2")
8801445

nightly:

julia> jldsave("typearray.jld2"; x = fill(Type{Int}, 100000))

julia> filesize("typearray.jld2")
16379568

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants