fix serialize_template_field handling callable value in dict

[wjddn279]

closed: #63334, #65705, #65674

Cause

The serialization result of the DAG is as follows.

{
  "__var": {
    "op_args": [],
    "task_id": "consume",
    "ui_color": "#ffefeb",
    "op_kwargs": "{'values': [3, 1, 2], 'sort_key': <function unstable_dag.<locals>.<lambda> at 0xf2dc11567e20>}",
    "task_type": "_PythonDecoratedOperator",
    "retry_delay": 300.0,
    "_task_module": "airflow.providers.standard.decorators.python",
    "_operator_name": "@task",
    "template_fields": [
      "templates_dict",
      "op_args",
      "op_kwargs"
    ],
    "_needs_expansion": false,
    "python_callable_name": "consume",
    "template_fields_renderers": {
      "op_args": "py",
      "op_kwargs": "py",
      "templates_dict": "json"
    }
  },

When serializing a dict containing lambda function definitions, the object's id value is reflected as-is, causing version changes on every serialization. The root cause is that when a dict contains callable objects, is_jsonable returns False, and since the dict itself is not callable, it falls through to that code path.

Solution

I added logic to filter out this case and convert callable objects consistently within the existing dict handling (sorting) logic. I have confirmed locally that this resolves the issue.

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[Copilot]

Pull request overview

Fixes non-deterministic DAG serialization when templated dict fields contain callable values (e.g., lambdas), preventing serialized output from changing on every run due to function object addresses.

Changes:

• Sanitizes callable values during recursive dict sorting in serialize_template_field.
• Refactors callable stringification into a helper for consistent formatting.
• Adds unit tests intended to validate stable serialization when dicts contain callables.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
airflow-core/src/airflow/serialization/helpers.py	Makes callable values in dicts serialize to a stable string during recursive sorting.
airflow-core/tests/unit/serialization/test_helpers.py	Adds regression tests around dicts containing callables to prevent unstable serialization.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

[ephraimbuddy]

Thanks — root-cause analysis is right and the <callable …> token is the correct deterministic fix. One blocker, plus a few nits.

Blocker: sorted(obj.items()) regresses mixed-key dicts.

The new dict branch in not is_jsonable(...) calls sort_and_make_static_dict_recursively, which does sorted(obj.items()) at helpers.py:73. A non-jsonable dict with mixed key types like {1: "a", "b": lambda: None} previously fell through to str(template_field) and serialized fine; now it raises TypeError: '<' not supported between instances of 'str' and 'int', which propagates (the try only catches AttributeError) and breaks DAG serialization.

>>> serialize_template_field({1: "a", "b": lambda: None}, "op_kwargs")
TypeError: '<' not supported between instances of 'str' and 'int'

The pre-existing sort_dict_recursively (now line 107) has the same bug today for jsonable mixed-key dicts — {1: "a", "b": "c"} already raises on main. One fix can cover both call sites:

sorted(obj.items(), key=lambda kv: (type(kv[0]).__name__, repr(kv[0])))

This keeps the determinism the sort_* helpers exist for, without crashing on heterogeneous keys. A try/except TypeError fallback to unsorted iteration would also work but loses the consistency guarantee. Please add a regression test covering the mixed-key + callable case.

[amoghrajesh]

Sorry for the delayed review, some comments.

[wjddn279]

@ephraimbuddy

I've written logic that handles those issues all at once. (with #65705). Could you take another look?

[wjddn279]

ping @ephraimbuddy !

[ephraimbuddy]

@ephraimbuddy

I've written logic that handles those issues all at once. (with #65705). Could you take another look?

This now regressed well from earlier.

[wjddn279]

Update tests to reflect logic change: set and frozenset are now converted to list instead of being string-cast.

related https://github.com/apache/airflow/pull/63871/changes#r3212530026

[wjddn279]

same https://github.com/apache/airflow/pull/63871/changes#r3212524999

[wjddn279]

same https://github.com/apache/airflow/pull/63871/changes#r3212524999

[wjddn279]

Previously set/frozenset values were converted via str(), which leaked memory addresses for custom objects and destabilized the DAG hash on every parse. The new logic recursively serializes each element and returns a sorted list, producing deterministic, JSON-encodable output.

[ephraimbuddy]

We also need to align this with the same function in task runner

[wjddn279]

Since keys are now fixed as strings by normalize_dict_key, the logic that sorted by key type alongside the key value has been removed. Sorting is now performed solely by the key value.