Http Template - Allow HTTP/API Tasks to Execute Without Spawning Separate Pods (Execute from Controller/Server Threads)

[rajarshipal-lab]

Summary

Currently, in Argo Workflows, Http Template execution results in the creation of a separate Pod. This introduces additional overhead in terms of Pod scheduling latency, resource consumption, and Kubernetes object churn.

This feature request proposes adding an option to execute lightweight HTTP/API tasks directly from the Argo Workflow Controller or Argo Server process (using internal worker threads/goroutines) instead of spawning a separate Pod for each step.

Motivation

In high-scale environments with large numbers of short-lived HTTP/API calls:

• Pod creation latency significantly impacts overall workflow execution time.
• Kubernetes API server load increases due to large volumes of Pod objects.
• Resource overhead (CPU/memory) increases due to container runtime initialization.
• Workflows that orchestrate external APIs (e.g., control-plane automation, cloud API calls) are not compute-bound and do not require container isolation.

Proposal

Http Template execution happens inside the Workflow Controller or Argo Server process.
May be Uses a bounded worker pool (goroutines) for concurrency.
Also, Avoids creating Kubernetes Pods.
Is explicitly opt-in via a new field, for example:

templates:
  - name: call-api
    http:
      url: https://example.com
      method: GET
    executionMode: Inline   # or Controller