LLM API invocations are slow and many applications have significant parallelism between invocations that we should be able to exploit.
If we choose to use Python's async support, we should avoid leaking async behavior into the rest of effectful. This means that the core functions in effectful.ops should not become async functions and we should not add async duplicates of these functions.
LLM API invocations are slow and many applications have significant parallelism between invocations that we should be able to exploit.
If we choose to use Python's async support, we should avoid leaking async behavior into the rest of
effectful. This means that the core functions ineffectful.opsshould not become async functions and we should not add async duplicates of these functions.