Skip to content

Incorrect question/answer for 12864 (prompt asks for dates + column C, golden has neither) #35

Description

@swenyang

Task: 12864 (Verified-400)

The prompt is inconsistent with the golden answer in two ways:

  1. "dates" that don't exist. The prompt says "retrieve corresponding
    dates from Column B on sheet 2 by matching Column D on sheet 1."
    But the
    value actually returned (Sheet1 Column B, header "Deal Name") is
    Monthly / Daily / Quarterly — these are reporting frequencies, and
    there are no dates anywhere in the workbook. The word "dates" mislabels
    the lookup target and makes the task ambiguous.

  2. Column C is requested but absent from the golden. The prompt says "I
    require the associated data from both Column B and C on sheet 2."
    The
    golden's Sheet2 has no Column C at all (not even a header) — it fills
    only Sheet2!B2:B12 with
    =INDEX(Sheet1!B$2:B$17, MATCH($A2, Sheet1!$D$2:$D$17, 0)).

answer_position is 'Sheet2'!B2:B12, so scoring only checks Column B — that
part is self-consistent and scoreable. But the prompt's references to "dates"
and "Column C" match neither the data nor the golden, which makes the intended
task ambiguous.

Suggested fix:

  • Replace "dates" with "deal name" (the actual header of the looked-up
    column, Sheet1!B1 / Sheet2!B1), so the prompt matches the workbook's own
    labels and the golden. (If a value-accurate term is preferred, the values are
    reporting frequencies, e.g. "reporting frequency".)
  • Either drop the "Column C" clause, or extend the golden to include the
    Column C the prompt asks for.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions