Fix dag.test() callbacks missing context["exception"]#66294
Conversation
The in-process supervisor used by dag.test() did not set context["exception"] before invoking finalize(), so on_failure_callback and on_retry_callback could not read the raised exception in dag.test(), unlike normal subprocess execution. Mirror the subprocess path in task_runner.main() by assigning context["exception"] = error between run() and finalize().
|
@Vamsi-klu can you close your PR, I have made the fix and waiting for the PR #64568 approval for merge and close this issue. |
|
@Vamsi-klu A few things need addressing before review — see our Pull Request quality criteria.
No rush. Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you. Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting |
|
@potiuk I apologize for making it ready for review without checking the status of test runs. Will raise a revision in couple of days max. Thanks!
|
task_runner.main()(the subprocess path) setscontext["exception"] = errorimmediately afterrun()and beforefinalize()so failure/retry callbacks can read the raised exception from context. The in-process supervisor used bydag.test()did not mirror that contract, soon_failure_callbackandon_retry_callbacksawcontext["exception"]as missing/None when invoked viadag.test(). This patchesInProcessTestSupervisor.start()to mirror the subprocess path.Tests
test_supervisor.py::TestInProcessTestSupervisor::test_start_sets_exception_in_context_before_finalize(parametrized overFAILEDandUP_FOR_RETRY) — verified to fail without the fix and pass with it.dag.test()regression intest_dag.py::TestDag::test_dag_test_failure_callback_receives_exception_in_context.closes: #64568
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.7) following the guidelines