Is your feature request related to a problem or challenge?
Follow-up to #23704 (which fixed the SIMILAR TO "failed to downcast array" panic from #22886 by coercing operands in the analyzer).
The analyzer now coerces SIMILAR TO operands to a common string type, but the SQL planner still rejects LargeUtf8 and Utf8View patterns before the analyzer ever runs. In datafusion/sql/src/expr/mod.rs:
let pattern_type = pattern.get_type(schema)?;
if pattern_type != DataType::Utf8 && pattern_type != DataType::Null {
return plan_err!("Invalid pattern in SIMILAR TO expression");
}
So a non-literal pattern of type LargeUtf8 or Utf8View fails at plan time:
-- errors: "Invalid pattern in SIMILAR TO expression"
SELECT s FROM test, patterns WHERE s SIMILAR TO arrow_cast(pat, 'Utf8View');
#23704's examples all keep the Utf8View/LargeUtf8 on the input side with a Utf8 pattern, so this path isn't exercised there. Now that the analyzer coerces both operands, this planner restriction is unnecessary.
Describe the solution you'd like
Relax the pattern-type check to also accept LargeUtf8 and Utf8View:
if !matches!(
pattern_type,
DataType::Utf8 | DataType::LargeUtf8 | DataType::Utf8View | DataType::Null
) {
return plan_err!("Invalid pattern in SIMILAR TO expression");
}
plus a sqllogictest case with a non-literal Utf8View (and LargeUtf8) pattern.
Describe alternatives you've considered
An earlier alternative implementation of the same fix (#22887) already included this planner relaxation and the corresponding test, so the change is small and self-contained.
Additional context
cc @u70b3 (author of #23704) — flagging in case you want to fold this into the same area. Happy to send the PR otherwise.
Is your feature request related to a problem or challenge?
Follow-up to #23704 (which fixed the
SIMILAR TO"failed to downcast array" panic from #22886 by coercing operands in the analyzer).The analyzer now coerces
SIMILAR TOoperands to a common string type, but the SQL planner still rejectsLargeUtf8andUtf8Viewpatterns before the analyzer ever runs. Indatafusion/sql/src/expr/mod.rs:So a non-literal pattern of type
LargeUtf8orUtf8Viewfails at plan time:#23704's examples all keep the
Utf8View/LargeUtf8on the input side with aUtf8pattern, so this path isn't exercised there. Now that the analyzer coerces both operands, this planner restriction is unnecessary.Describe the solution you'd like
Relax the pattern-type check to also accept
LargeUtf8andUtf8View:plus a sqllogictest case with a non-literal
Utf8View(andLargeUtf8) pattern.Describe alternatives you've considered
An earlier alternative implementation of the same fix (#22887) already included this planner relaxation and the corresponding test, so the change is small and self-contained.
Additional context
cc @u70b3 (author of #23704) — flagging in case you want to fold this into the same area. Happy to send the PR otherwise.