Skip to content

Fix dag.test() callbacks missing context["exception"]#66294

Open
Vamsi-klu wants to merge 1 commit into
apache:mainfrom
Vamsi-klu:fix-dag-test-context-exception
Open

Fix dag.test() callbacks missing context["exception"]#66294
Vamsi-klu wants to merge 1 commit into
apache:mainfrom
Vamsi-klu:fix-dag-test-context-exception

Conversation

@Vamsi-klu
Copy link
Copy Markdown
Contributor

task_runner.main() (the subprocess path) sets context["exception"] = error immediately after run() and before finalize() so failure/retry callbacks can read the raised exception from context. The in-process supervisor used by dag.test() did not mirror that contract, so on_failure_callback and on_retry_callback saw context["exception"] as missing/None when invoked via dag.test(). This patches InProcessTestSupervisor.start() to mirror the subprocess path.

Tests

  • Unit regression in test_supervisor.py::TestInProcessTestSupervisor::test_start_sets_exception_in_context_before_finalize (parametrized over FAILED and UP_FOR_RETRY) — verified to fail without the fix and pass with it.
  • End-to-end dag.test() regression in test_dag.py::TestDag::test_dag_test_failure_callback_receives_exception_in_context.

closes: #64568


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

The in-process supervisor used by dag.test() did not set
context["exception"] before invoking finalize(), so on_failure_callback
and on_retry_callback could not read the raised exception in
dag.test(), unlike normal subprocess execution. Mirror the subprocess
path in task_runner.main() by assigning context["exception"] = error
between run() and finalize().
@ASk1
Copy link
Copy Markdown

ASk1 commented May 6, 2026

@Vamsi-klu can you close your PR, I have made the fix and waiting for the PR #64568 approval for merge and close this issue.

@potiuk
Copy link
Copy Markdown
Member

potiuk commented May 11, 2026

@Vamsi-klu A few things need addressing before review — see our Pull Request quality criteria.

  • Pre-commit / static checks — Failing: CI image checks / Static checks. See docs.

No rush.


Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.


Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting

@Vamsi-klu
Copy link
Copy Markdown
Contributor Author

@potiuk I apologize for making it ready for review without checking the status of test runs. Will raise a revision in couple of days max. Thanks!

@Vamsi-klu A few things need addressing before review — see our Pull Request quality criteria.

  • Pre-commit / static checks — Failing: CI image checks / Static checks. See docs.

No rush.


Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.


Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants