Skip to content

run_benchmark: exit non-zero on import failure#2693

Open
fallintoplace wants to merge 1 commit into
pytorch:mainfrom
fallintoplace:fix-run-benchmark-import-exit
Open

run_benchmark: exit non-zero on import failure#2693
fallintoplace wants to merge 1 commit into
pytorch:mainfrom
fallintoplace:fix-run-benchmark-import-exit

Conversation

@fallintoplace

@fallintoplace fallintoplace commented Jun 21, 2026

Copy link
Copy Markdown

Summary

  • Make user benchmark import failures in run_benchmark.py exit with a non-zero status (SystemExit(1)) so automation doesn’t treat them as success.
  • Keep benchmark execution failures bubbling up naturally so real runtime errors remain visible.
  • Add regression coverage in test_run_benchmark.py for both behaviors.

Why this matters

When benchmark imports fail, the script currently only logs a traceback and returns success, which can let broken benchmark runs pass CI checks.

@meta-cla meta-cla Bot added the cla signed label Jun 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant