Issues

• Large top-k values overflow int32 in sampler preparation

  FlashML-org/FreeToken#789 · Filter by author wenlive opened on Oct 22, 2026

• Context clipping mutates sampling parameters shared by offline requests

  FlashML-org/FreeToken#787 · Filter by author wenlive opened on Oct 22, 2026

• New architecture request: Recursive Flint (RecursiveSI/flint — 39.8B MoE coding model, ~2.5B active)

  FlashML-org/FreeToken#786 · Filter by author ukind opened on Oct 22, 2026

• [Desktop] weights failed to load: Qwen3.6-35B-A3B NVFP4

  duplicate
  FlashML-org/FreeToken#784 · Filter by author mehedih985 opened on Oct 21, 2026
  1

• perf(qsa): prefill indexer scoring pads 4 heads to 16 per row; 16-row tiles are ~5x faster and bitwise identical

  FlashML-org/FreeToken#783 · Filter by author gdevenyi opened on Oct 21, 2026

• [Desktop] weights failed to load: Qwen3.6-35B-A3B NVFP4

  bug
  FlashML-org/FreeToken#782 · Filter by author srvhbkking-collab opened on Oct 21, 2026
  2

• [Perf] Fuse the BF16 attention output gate in short gfx1201 decode graphs

  amd
  FlashML-org/FreeToken#780 · Filter by author zihaomu opened on Oct 21, 2026

• Capture-time cuGraphUpload (#697) widens inter-kernel gaps in replayed decode graphs: ~1-2% slower decode

  FlashML-org/FreeToken#775 · Filter by author gdevenyi opened on Oct 21, 2026

• Proposal: isolate logprob reads (max_tokens 1) from batch composition and the prefix cache so scoring is reproducible

  FlashML-org/FreeToken#772 · Filter by author gdevenyi opened on Oct 20, 2026

• Engine bug report: recurring allocator OOM warnings during chunked GGUF prefill

  buglinux
  FlashML-org/FreeToken#770 · Filter by author void-mckenzie opened on Oct 20, 2026
  3

• Proposal: load an FTW with fp16 NVFP4 global scales (upcast with a warning) instead of refusing it after #739

  FlashML-org/FreeToken#769 · Filter by author gdevenyi opened on Oct 20, 2026

• Proposal: vLLM scoring fields on /v1/completions (allowed_token_ids, return_tokens_as_token_ids, token-id prompts) for classifier-style clients

  FlashML-org/FreeToken#768 · Filter by author gdevenyi opened on Oct 20, 2026

• /v1/completions: prompt: [] returns 200 with no choices, and both completions paths drop the error code (context_length_exceeded)

  FlashML-org/FreeToken#767 · Filter by author gdevenyi opened on Oct 20, 2026

• An FTW checkpoint can never serve the W4A4 NVFP4 MoE prefill, and the refusal tells you to reconvert with the option that produced it

  FlashML-org/FreeToken#766 · Filter by author gdevenyi opened on Oct 20, 2026

• Two tests fail on a CUDA host since #736 / #697: test_nvfp4_experts_restricted_to_triton_for_swigluoai (sm_120) and test_graph_capture_reuses_warm_offload_cache_before_capture (any Linux host with libcuda)

  FlashML-org/FreeToken#765 · Filter by author gdevenyi opened on Oct 20, 2026

• [Feature] Optional host-resident token-embedding table (pinned, mapped, UVA lookups): ~1 GB of VRAM back for offloaded expert slots

  FlashML-org/FreeToken#754 · Filter by author SUKUNA-AI opened on Oct 19, 2026

• [Perf] Offloaded decode: overlap the expert-miss gather with the same layer's shared expert, event-ordered inside the CUDA graph (bit-exact implementation)

  FlashML-org/FreeToken#753 · Filter by author SUKUNA-AI opened on Oct 19, 2026

• [Perf] Offloaded prefill: keep buffer invalidation off the prefill copy stream and give that stream top priority (short-prompt H2D 21 -> 42 GB/s under Nsight)

  FlashML-org/FreeToken#752 · Filter by author SUKUNA-AI opened on Oct 19, 2026

• [Perf] Offloaded decode: gather missed experts with a few TMA bulk-copy CTAs (cp.async.bulk, .shared::cta) instead of 48 x 1024 LDG threads

  FlashML-org/FreeToken#751 · Filter by author SUKUNA-AI opened on Oct 19, 2026

• [Perf] Layer-major prefill for offloaded MoE: stream each expert layer once per prompt, not once per chunk (qwen3_5_moe)

  FlashML-org/FreeToken#750 · Filter by author SUKUNA-AI opened on Oct 19, 2026

• [Bug] Context overflow is checked only after tokenization in the worker: a streamed request gets 200 plus an in-stream error, and an oversized max_tokens is silently clamped

  FlashML-org/FreeToken#749 · Filter by author SUKUNA-AI opened on Oct 19, 2026

• [Feature] Hybrid radix cache: greedy outputs depend on request history even when cached_tokens is 0 (request: cache flush endpoint / deterministic mode)

  FlashML-org/FreeToken#748 · Filter by author SUKUNA-AI opened on Oct 19, 2026

• [Desktop] weights failed to load: DeepSeek-V4.1-Flash-1017 FP8

  new-model
  FlashML-org/FreeToken#745 · Filter by author pacnet opened on Oct 19, 2026
  1

• hybrid decode halves (23 → 11 tok/s) whenever a foreign thread lands on a pinned spinning CPU-MoE pool CPU; please detect/warn, consider SCHED_FIFO or a yielding spin

  FlashML-org/FreeToken#744 · Filter by author philporter1 opened on Oct 19, 2026

• perf(moe): fused triton router for sigmoid + selection-bias routing

  perf
  FlashML-org/FreeToken#730 · Filter by author jason-fxz opened on Oct 17, 2026