Skip to content

Static batched decode (LLaMA + Qwen3): B independent sequences per step, up to 41x aggregate throughput#129

Open
mikepapadim wants to merge 10 commits into
beehive-lab:feat/mma_cudafrom
mikepapadim:feat/static-batched-decode
Open

Static batched decode (LLaMA + Qwen3): B independent sequences per step, up to 41x aggregate throughput#129
mikepapadim wants to merge 10 commits into
beehive-lab:feat/mma_cudafrom
mikepapadim:feat/static-batched-decode

Doc: on-device sampling section + numbers

0cc44dc
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs