Skip to content

Flaky wav2vec2 transcription test #2024

Description

@robinp

As noted in an other PR by @QuentinFuxa, it seems all recent PRs fail on the same flaky or faulty test:

FAILED test_transformers.py::TestWav2Vec2::test_transformers_wav2vec2[facebook/wav2vec2-large-robust-ft-swbd-300h-expected_transcription0-cpu]

Expected: MISTER QUILTER IS THE APOSSEL OF THE MIDDLE CLASSES AND WE ARE GLAD TO WELCOME HIS GOSPEL
Got:      MISTER QUILTER IS THE APOSSTEL OF THE MIDDLE CLASSES AND WE ARE GLAD TO WELCOME HIS GOSPEL

The model produces a slightly different transcription (APOSSTEL vs APOSSEL) likely due to environment/library version differences. All other 166 tests passed. Could a maintainer re-run the failed job?

Originally posted by @QuentinFuxa in #2010 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions