compat: Support the TypeEq type-system refactor (JuliaLang/julia#61915)#718
compat: Support the TypeEq type-system refactor (JuliaLang/julia#61915)#718Keno wants to merge 1 commit into
Conversation
The TypeEq refactor changes `typeof(Type{T})` from `DataType` to the new
`Core.TypeEq`. As a consequence, a `Type{T}` value is no longer an instance
of `DataType` (`isa(Type{T}, DataType)` is now `false`), even though
`Type{T} <: DataType` still holds and `Base.isType(Type{T})` is `true`.
JLD2 dispatches its `DataType` serialization machinery on `typeof(x)`, so on
such a Julia a value whose type is a `Type{T}` (for example the field of a
struct declared `x::Type{Float64}`) was routed through the generic compound
path instead of the dedicated `DataType` path. This surfaced as a
`MethodError` in `commit` and downstream `fieldcount`/`convert` errors when
writing such values.
Introduce a `typeof_serializable` helper that maps `Core.TypeEq`-typed values
back to `DataType` (guarded by `isdefined(Core, :TypeEq)` so the code still
loads on released Julia), and relax the remaining value-level `DataType`
annotations and `isa(x, DataType)` checks to also accept `Type{T}` values.
These changes keep the on-disk format identical to older Julia versions.
This change was made with the assistance of generative AI (Claude Code).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #718 +/- ##
==========================================
+ Coverage 87.38% 87.39% +0.01%
==========================================
Files 37 37
Lines 4588 4593 +5
==========================================
+ Hits 4009 4014 +5
Misses 579 579 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Thanks for providing the update! Codecov complains that the |
Merged after the PR, but before your comment ;) |
|
That said, the nightly build with it probably isn't done yet. |
|
Let's wait until then, retrigger CI, and merge when tests pass with the relevant code change covered. |
| # A ghost type, so no need to store at all | ||
| return nothing | ||
| elseif isa(T, DataType) && sizeof(T) ≤ MAX_INLINE_SIZE | ||
| elseif (isa(T, DataType) || Base.isType(T)) && sizeof(T) ≤ MAX_INLINE_SIZE |
There was a problem hiding this comment.
I don't think a change is needed here because this is inside an if isconcretetype(T) block.
There was a problem hiding this comment.
| elseif (isa(T, DataType) || Base.isType(T)) && sizeof(T) ≤ MAX_INLINE_SIZE | |
| elseif isa(T, DataType) && sizeof(T) ≤ MAX_INLINE_SIZE |
|
The tests are still missing coverage for these changes. Examples of things still broken: Unions are not saved correctly. julia> jldsave("union_new.jld2"; x = Union{Nothing, Type{Int}})
julia> load("union_new.jld2")
Dict{String, Any} with 1 entry:
"x" => Nothing
In Julia nightly julia> jldsave("datatypes_new.jld2"; x = Type{Int})
julia> load("datatypes_new.jld2")
Dict{String, Any} with 1 entry:
"x" => Type{Int64}
Loading this file in Julia 1.12.6 julia> load("datatypes_new.jld2")
┌ Warning: custom serialization of Core.TypeEq encountered, but the type does not exist in the workspace; the data will be read unconverted
└ @ JLD2 ~/.julia/packages/JLD2/qcxKY/src/data/reconstructing_datatypes.jl:112
Dict{String, Any} with 1 entry:
"x" => UnknownType{Symbol("Core.TypeEq"), Tuple{Int64}}
There is a 2x file size regression storing arrays of 1.12.6: julia> jldsave("typearray.jld2"; x = fill(Type{Int}, 100000))
julia> filesize("typearray.jld2")
8801445
nightly: julia> jldsave("typearray.jld2"; x = fill(Type{Int}, 100000))
julia> filesize("typearray.jld2")
16379568
|
The TypeEq refactor changes
typeof(Type{T})fromDataTypeto the newCore.TypeEq. As a consequence, aType{T}value is no longer an instance ofDataType(isa(Type{T}, DataType)is nowfalse), even thoughType{T} <: DataTypestill holds andBase.isType(Type{T})istrue.JLD2 dispatches its
DataTypeserialization machinery ontypeof(x), so on such a Julia a value whose type is aType{T}(for example the field of a struct declaredx::Type{Float64}) was routed through the generic compound path instead of the dedicatedDataTypepath. This surfaced as aMethodErrorincommitand downstreamfieldcount/converterrors when writing such values.Introduce a
typeof_serializablehelper that mapsCore.TypeEq-typed values back toDataType(guarded byisdefined(Core, :TypeEq)so the code still loads on released Julia), and relax the remaining value-levelDataTypeannotations andisa(x, DataType)checks to also acceptType{T}values. These changes keep the on-disk format identical to older Julia versions.This change was made with the assistance of generative AI (Claude Code).