Skip to content

Validator fail-open re-run failures print a full traceback #357

Description

@hertznsk

Description

When a semantic validator rejects an agent output and the validator-triggered primary re-run fails, Conductor correctly fails open and keeps the original output. However, the expected fail-open path is logged with exc_info=True, so users see a full Python traceback even though the workflow continues successfully.

This makes a handled validator recovery failure look like an unhandled workflow crash and adds noisy implementation details to normal CLI output.

Reproduction

On current main (f39bd63c3f8fe540f164102ebedb598526b2f686):

  1. Configure an agent with validator.max_retries: 1.
  2. Return passed: false from the validator.
  3. Make the validator-triggered primary re-run raise a provider/runtime exception.
  4. Run the workflow with warning logs visible.

The existing regression scenario demonstrates the behavior directly:

uv run pytest tests/test_engine/test_validator_integration.py::TestValidatorCostAndFailurePaths::test_rerun_failure_keeps_original_no_double_count_and_emits_event -q -s --log-cli-level=WARNING

Observed output includes:

WARNING conductor.engine.workflow: Validator re-run failed for 'reviewer'; using original output
Traceback (most recent call last):
  ...
RuntimeError: rerun boom
PASSED

Expected behavior

The workflow should still fail open and retain the original output, but the handled re-run failure should be reported as a concise warning without a full traceback in normal user-facing logs. Diagnostic exception details can remain available at debug level or in structured diagnostic metadata/logs.

Actual behavior

WorkflowEngine._apply_validator() catches the exception and returns the original output, but logs the warning with exc_info=True in src/conductor/engine/workflow.py. This prints the entire traceback at WARNING level despite the failure being handled.

Additional context

The existing test verifies output preservation, usage accounting, and the rerun_errored event, but does not assert that the user-facing fail-open path avoids traceback noise.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions