Skip to content

[C++][Python] pairwise_diff ignores array offsets for sliced inputs #50524

Description

@hsusul

Describe the bug

pairwise_diff and pairwise_diff_checked produce incorrect results when the input array has a nonzero offset, such as an array created with slice().

The kernel reads values from the parent array before the logical slice instead of consistently indexing relative to the sliced input.

Reproduction

import pyarrow as pa
import pyarrow.compute as pc

base = pa.array([99, 1, 4, 9, 16, 88], type=pa.int64())
sliced = base.slice(1, 4)
rebuilt = pa.array(sliced.to_pylist(), type=sliced.type)

print("offset:", sliced.offset)
print("input:", sliced.to_pylist())
print("actual:", pc.pairwise_diff(sliced, period=1).to_pylist())
print("expected:", pc.pairwise_diff(rebuilt, period=1).to_pylist())

Output:

offset: 1
input: [1, 4, 9, 16]
actual: [None, -98, 3, 5]
expected: [None, 3, 5, 7]

The -98 result comes from subtracting the parent array's prefix value 99, which is outside the logical slice.

The same behavior is reproducible through the public C++ API.

Expected behavior

A sliced array should produce the same result as an equivalent zero-offset array containing the same logical values:

[None, 3, 5, 7]

Actual behavior

The first computed difference reads from the parent array before the slice:

[None, -98, 3, 5]

Root cause

In cpp/src/arrow/compute/kernels/vector_pairwise.cc, PairwiseExecImpl copies the input ArraySpan and then calls:

left.SetSlice(left_start, computed_length);
right.SetSlice(right_start, computed_length);

ArraySpan::SetSlice() assigns the provided value as an absolute offset rather than adding it to the existing input offset.

For a sliced input, the shifted spans therefore index from the parent buffers rather than from input.offset.

The validity calculation still uses the original input offset, so the value and validity spans may also refer to different logical elements.

Affected inputs

The issue affects sliced inputs where:

offset > 0
0 < abs(period) < length

It is reproducible for both positive and negative periods and affects numeric, decimal, and temporal kernel signatures that share this implementation.

Proposed direction

Preserve the original input offset when creating the shifted operand spans, and add focused C++ regression coverage for sliced inputs.

I searched existing open and closed issues, pull requests, commits, release notes, tests, and source history and did not find an existing report for this sliced-array offset bug.

Component(s)

C++, Python

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions