Add Jinja template rendering for async deadline callbacks#66496
Closed
seanghaeli wants to merge 2 commits into
Closed
Add Jinja template rendering for async deadline callbacks#66496seanghaeli wants to merge 2 commits into
seanghaeli wants to merge 2 commits into
Conversation
Renders Jinja2 templates in callback kwargs using the simple context
(dag_run, deadline info) that the scheduler passes to the triggerer.
This allows users to write callbacks like:
AsyncCallback(my_func, kwargs={"msg": "DAG {{ dag_id }} missed deadline"})
Key design decisions:
- Uses BaseTrigger's inherited render_template() (via Templater) for
consistent rendering behavior with the rest of Airflow
- Skips pre-rendering for Notifier classes since they handle their own
template rendering in __await__ via render_template_fields()
- Only renders when context is present (no-op for callbacks without context)
- Acknowledges this is interim: full context should be fetched via the
Execution API at execution time (tracked in PR apache#64984 TODO comments)
Addresses feedback from Ramit (PR apache#64984 comment) by documenting the
path forward while still providing template rendering value on top of
the existing simple context from PR apache#55241.
Contributor
Author
|
Closing this PR — the approach of building template rendering on top of the stored simple context doesn't address the architectural concern that context shouldn't be stored in the DB at all. The proper implementation should fetch full context via the Execution API at callback execution time (now possible since #55068 merged), which means reverting the simple context from #55241 and replacing it with runtime context fetching. Will reopen with the correct approach. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add Jinja template rendering support for async deadline callbacks running in the triggerer. Uses Airflow's standard
render_template()(viaBaseTrigger→Templater) rather than raw Jinja, and correctly skips rendering for Notifiers which handle their own template fields.Builds on the simple context introduced in #55241, with a clear path toward full Execution API context fetching documented as a follow-up.
Changes
CallbackTrigger.run()using inheritedrender_template()_is_notifier_class()duck-typing helper to skip double-rendering for Notifiersselectinload(Deadline.deadline_alert)from scheduler queryRelated