Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions vortex-array/src/scalar_fn/fns/list_length.rs
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,8 @@ impl ScalarFnVTable for ListLength {
}

fn is_strict(&self, _options: &Self::Options) -> bool {
// A null list has a null length, and the length of a valid list is a non-null value
// determined by that list alone, with `return_dtype` carrying over the input nullability.
true
}

Expand Down
3 changes: 3 additions & 0 deletions vortex-array/src/scalar_fn/fns/list_sum.rs
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,9 @@ impl ScalarFnVTable for ListSum {
}

fn is_strict(&self, _options: &Self::Options) -> bool {
// A null list sums to null, which is all that strictness requires. Element nulls are part
// of the list value rather than row-level nulls, and a valid list may still sum to null
// (empty, all-null, or overflow) since strictness is only one-directional.
true
}

Expand Down
21 changes: 16 additions & 5 deletions vortex-array/src/scalar_fn/vtable.rs
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,8 @@ pub trait ScalarFnVTable: 'static + Sized + Clone + Send + Sync {

/// Returns whether this expression itself is strict.
///
/// ## `STRICT` semantics
///
/// Strict has the same value-level meaning as PostgreSQL `STRICT`: if any input value at row
/// `i` is null, the output value at row `i` is null. Formally, `f` is strict iff at each row,
/// for some total function `g` over non-null values,
Expand All @@ -203,8 +205,8 @@ pub trait ScalarFnVTable: 'static + Sized + Clone + Send + Sync {
/// f(v1, .., vk) = g(v1, .., vk) otherwise
Comment on lines 201 to 205

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this not contradicts the below

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why does it contradict the stuff below?

/// ```
///
/// That is, any null input yields a null output, and non-null outputs depend only on the (all
/// non-null) inputs at that row. Lifted columnwise, this is equivalent to the mask-hoisting
/// That is, any null input yields a null output, and the output at a fully non-null row depends
/// only on the inputs at that row. Lifted columnwise, this is equivalent to the mask-hoisting
/// law: for any argument `aj = mask(aj', m)`,
///
/// ```text
Expand All @@ -217,10 +219,17 @@ pub trait ScalarFnVTable: 'static + Sized + Clone + Send + Sync {
/// strict because, for example, `false AND null = false`.
///
/// Two consequences that optimizations rely on:
/// 1. Output validity is precomputable as the `AND` of the input validities, so
/// `valid(f(a1, .., ak)) = valid(a1) ∧ .. ∧ valid(ak)`.
/// 1. Any null input slot nulls the output slot, so the output validity is bounded above by the
/// `AND` of the input validities: `valid(f(a1, .., ak)) valid(a1) ∧ .. ∧ valid(ak)`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it a bound or equal?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bounded, thats why there is a subset there.

formally, g is allowed to return a null even if its inputs are not null. This is subtly different from how f MUST return null if any inputs ARE null.

/// 2. Values behind null slots are irrelevant, so kernels may compute `g` densely over all
/// lanes (including garbage) and apply validity afterwards.
/// lanes (including garbage) and intersect the input validities afterwards.
///
/// Strictness is one-directional: `g` may itself produce null, as `list_sum` does for a valid
/// but empty list. This is weaker than the validity-intersection property (Arrow's
/// `NullHandling::INTERSECTION`), where the output validity always *equals* the `AND` of the
/// input validities. Most strict functions satisfy that stronger property and can advertise it
/// via [`ScalarFnVTable::validity`], but optimizations must not assume it from strictness
/// alone.
///
/// Returning `true` also requires [`ScalarFnVTable::return_dtype`] to propagate nullability:
/// if any input dtype is nullable, the output dtype must be nullable. For example, `cast` is
Expand All @@ -229,6 +238,8 @@ pub trait ScalarFnVTable: 'static + Sized + Clone + Send + Sync {
///
/// Nullary functions are vacuously strict because they have no input values.
///
/// ## Returns
///
/// Conservatively defaults to `false` (non-strict).
///
/// This method only checks the expression itself, not its children.
Expand Down
Loading