Problem
The Orchestrator currently runs and executes steps within itself; in Node, this is technically not threaded. Still executing at most one thing at a time, even though there is a significant amount of dependency latency that gets filled with more work via async/await.
Proposed solution
We should migrate to a true split Orchestrator + N agent runners that can be either local or remote instances. This puts the majority of the workload on the remote instance, and keeps the orchestrator itself the core loop of get work + dispatch, and allows horizontal scaling nearly infinitely.
Bidirectional asymmetric keys for security should be fine for auth between orchestrator + agent runner since we don't need to encrypt; only sign payloads.
Alternatives considered
We don't need full OAuth, JWTs, refresh tokens, etc, since these aren't users. If the agent or orchestrator is compromised, all those solutions would fail too so we should go with the simple plan.
Area
Orchestrator
Additional context
No response
Problem
The Orchestrator currently runs and executes steps within itself; in Node, this is technically not threaded. Still executing at most one thing at a time, even though there is a significant amount of dependency latency that gets filled with more work via async/await.
Proposed solution
We should migrate to a true split Orchestrator + N agent runners that can be either local or remote instances. This puts the majority of the workload on the remote instance, and keeps the orchestrator itself the core loop of get work + dispatch, and allows horizontal scaling nearly infinitely.
Bidirectional asymmetric keys for security should be fine for auth between orchestrator + agent runner since we don't need to encrypt; only sign payloads.
Alternatives considered
We don't need full OAuth, JWTs, refresh tokens, etc, since these aren't users. If the agent or orchestrator is compromised, all those solutions would fail too so we should go with the simple plan.
Area
Orchestrator
Additional context
No response