Skip to content

Add deferrable support to LLM operators in common.ai provider #70093

Description

@guan404ming

Why

LLM calls in LLMOperator / AgentOperator block a worker slot for the whole API call (30s to minutes). Airflow's deferral mechanism fits this perfectly.

Proposal

Step 1 — LLMTrigger + deferrable LLMOperator

  • Run the LLM call in the triggerer via async agent.run(), freeing the worker slot
  • deferrable=True on LLMOperator, rehydrate output_type on resume
  • Chain with require_approval (HITL) so the whole flow never holds a worker

Step 2 — Deferrable subclasses

  • LLMBranchOperator (branch on resume)
  • LLMSQLOperator, LLMSchemaCompareOperator

Step 3 — AgentTrigger for AgentOperator

  • Multi-turn agent in the triggerer, message history serialized
  • Toolsets rebuilt from serializable spec (conn_id + hook path)
  • Durable-storage replay on triggerer restart, no re-billed LLM calls

Use case

  • Hundreds of concurrent LLM tasks without holding hundreds of worker slots
  • Long-running reasoning/agent tasks (10+ min) become cheap to run

Metadata

Metadata

Assignees

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions