chore: update whisper.cpp to v1.9.1 with RTX 50 (sm_120) CUDA support#3

Closed

xAlcahest wants to merge 832 commits into

OpenWhispr:masterfrom

xAlcahest:sync-upstream-v1.9.1

This pull request is big! We're only showing the most recent 250 commits

Commits on May 14, 2026

ggml : bump version to 0.11.1 (ggml/1484)
ggerganov
committed
sync : ggml
ggerganov
committed
talk-llama : sync llama.cpp
ggerganov
committed
try to fix window cublas CI failure

danbev
authored and
ggerganov
committed
Revert "try to fix window cublas CI failure"

danbev
authored and
ggerganov
committed
try using CCCL 12.4.127 with cuda 11.8.0 to fix CI failure

danbev
authored and
ggerganov
committed
Revert "try using CCCL 12.4.127 with cuda 11.8.0 to fix CI failure"

danbev
authored and
ggerganov
committed
devops : add spirv-headers to vulkan dockerfile

danbev
authored and
ggerganov
committed
ggml-cuda : add explicit casts to -INFINITY for float and half2 types

danbev
authored and
ggerganov
committed
ggml-cuda : add ar_add() to avoid ambiguous operator+ for half/bfloat16 in CUDA 11.8

danbev
authored and
ggerganov
committed
ci : update ONEAPI version to 2025.3.3-0-devel-ubuntu24.04

danbev
authored and
ggerganov
committed
squash! ci : update ONEAPI version to 2025.3.3-0-devel-ubuntu24.04

danbev
authored and
ggerganov
committed
Revert "ggml-cuda : add ar_add() to avoid ambiguous operator+ for half/bfloat16 in CUDA 11.8"

danbev
authored and
ggerganov
committed
Revert "ggml-cuda : add explicit casts to -INFINITY for float and half2 types"

danbev
authored and
ggerganov
committed
ggml: install ggml.pc in <libdir>/pkgconfig (ggml/1480)

robUx4
authored and
ggerganov
committed
metal : tighten input-position loop in kernel_conv_transpose_1d (ggml/1477)

CrispStrobe
authored and
ggerganov
committed
ggml-virtgpu : include missing mutex header (llama/22810)

olliewalsh
authored and
ggerganov
committed
Add OP im2col_3d (llama/22903)

arthw
authored and
ggerganov
committed
CUDA: directly include cuda/iterator (llama/22936)

ORippler
authored and
ggerganov
committed
vulkan: Support asymmetric FA in scalar/mmq/coopmat1 paths (llama/22589)

jeffbolznv
authored and
ggerganov
committed
Ggml/cuda snake fusion hardening (llama/22912)

ServeurpersoCom
authored and
ggerganov
committed
CUDA: handle OW > 65535 in im2col (2D and 3D) (llama/22944)

CrispStrobe
authored and
ggerganov
committed
opencl: add q4_1 MoE for Adreno (llama/22856)

authored and
ggerganov
committed
metal : promote mul_mv/mul_mm batch divisors to function constants (llama/22711)

guyfischman
authored and
ggerganov
committed
vulkan: Check shared memory size for mmq shaders (llama/22693)

jeffbolznv
authored and
ggerganov
committed
vulkan: Fix Windows performance regression on Intel GPU BF16 workloads for Xe2 and newer (llama/22461)

rillomas
authored and
ggerganov
committed
ggml-webgpu: address precision issues for multimodal (llama/22808)

Constannnnnt
authored and
ggerganov
committed
ggml-webgpu: Enables running gpt-oss-20b (llama/22906)

yomaytk
authored and
ggerganov
committed
opencl: add opt-in Adreno xmem F16xF32 GEMM for prefill (llama/22755)

authored and
ggerganov
committed
hexagon: eliminate scalar VTCM loads via HVX splat helpers (llama/22993)

authored and
ggerganov
committed
ggml-zendnn : adaptive fallback to CPU backend for small batch sizes (llama/22681)

z-sachin
authored and
ggerganov
committed
hexagon: add unary tanh op (llama/22999)

max-krasnyansky
authored and
ggerganov
committed
flush the gpu profile timestamp before the queryset is overflowed (llama/22995)

yomaytk
authored and
ggerganov
committed
opencl: fix crash when warming up MoE on Adreno (llama/22876)

lhez
authored and
ggerganov
committed
opencl: add q5_0 and q5_1 MoE for Adreno (llama/22985)

authored and
ggerganov
committed
Fix for issue #22974. Cast intermediate results to float before adding and casting the result to the destination type. Avoids half+half operator ambiguity. (llama/22994)

scutler-nv
authored and
ggerganov
committed
ggml-webgpu: only use subgroup-matrix path when head dims are divisible by sg_mat_k / sg_mat_n (llama/23020)

ArberSephirotheca
authored and
ggerganov
committed
sync : ggml
ggerganov
committed
talk-llama : sync llama.cpp
ggerganov
committed

Commits on May 15, 2026

server: add support for carry_initial_prompt (#3781)
alubbe
authored

Commits on May 18, 2026

Commits on May 19, 2026

whisper : set bench data for each iteration (#3812)
danbev
authored

Commits on May 21, 2026

ci : use github ubuntu-22.04-arm runner instead of qemu (#3815)
danbev
authored

Commits on May 22, 2026

common : fix server /inference fails to decode in-memory audio (regression) (#3818)
ServeurpersoCom
authored

Commits on May 25, 2026

fix: in bindings/ruby/test/jfk_reader/jfk_reader in jfk_reader.c (#3756)

orbisai0security
and
claude
authored
cmake : add CMakePresets.json [no ci] (#3808)
danbev
authored
SYCL: fix multi-GPU system RAM exhaustion by using Level Zero allocations (llama/21597)

authored and
ggerganov
committed
vulkan: fix matmul integer pipeline selection (llama/23005)

0cc4m
authored and
ggerganov
committed
ggml-cpu: Add IME2 Instruction Support for the SpacemiT Backend (llama/22863)

alex-spacemit
authored and
ggerganov
committed
logs : reduce (llama/23021)
ggerganov
committed
ggml-webgpu: makes the flash attn vec path subgroup-aware (llama/23040)

ArberSephirotheca
authored and
ggerganov
committed
HIP: RDNA3 mma FA, faster AMD transpose, tune AMD (llama/22880)

JohannesGaessler
authored and
ggerganov
committed
ggml-hexagon: cpy: add contiguous fast-path in reshape copy (llama/23076)

pdhinaka
authored and
ggerganov
committed
llama + spec: MTP Support (llama/22673)

am17an
and
ggerganov
committed
ggml : bump version to 0.12.0 (ggml/1494)
ggerganov
committed
ggml-alloc: fix out-of-bounds read in ggml_dyn_tallocr_remove_block (ggml/1492)

Dev-X25874
authored and
ggerganov
committed
ggml.h: correct ggml_silu_back arg docstring (a=dy, b=x) (ggml/1500)

OriPekelman
authored and
ggerganov
committed
vulkan: removed duplicate #include <memory> in headers (llama/23144)

winstonma
authored and
ggerganov
committed
vulkan: fuse SSM_CONV + BIAS + SILU (llama/22653)

jeffbolznv
authored and
ggerganov
committed
vulkan: Support unaligned tensors for ROPE (llama/22637)

jeffbolznv
authored and
ggerganov
committed
vulkan: add cpy bf16 -> f32 pipelines (llama/22677)

ServeurpersoCom
authored and
ggerganov
committed
ggml-vulkan/CMakeLists: add a check for SPIRV-Headers (llama/22009)

jeeb
authored and
ggerganov
committed
CUDA: Continue directly including cuda/iterator (llama/23102)

ORippler
authored and
ggerganov
committed
feat: Support d_conv=15 for ssm-conv.cu (llama/23017)

gabe-l-hart
authored and
ggerganov
committed
sycl: route small f32 matmuls to oneMKL, bypass oneDNN (llama/22150)

authored and
ggerganov
committed
sycl: scalar SWAR byte-subtract in Q6_K MMVQ dot product (llama/22156)

authored and
ggerganov
committed
ggml-hexagon: add PAD op HVX kernel (llama/23078)

authored and
ggerganov
committed
hexagon: add support for TRI op (llama/22822)

authored and
ggerganov
committed
rpc : keep last_graph_uid in the device context (llama/23273)

rgerganov
authored and
ggerganov
committed
sycl: add GGML_SYCL_USE_ASYNC_MEM_OP env toggle (llama/22153)

authored and
ggerganov
committed
ggml-webgpu : extend GDN for K>1 (llama/23299)

reeselevine
authored and
ggerganov
committed
hexagon: enable support for NORM op (llama/23319)

aparmp-quic
authored and
ggerganov
committed
hexagon: add MROPE and IMROPE support in HTP rope op (llama/23317)

aparmp-quic
authored and
ggerganov
committed
opencl: add MoE support for q4_k, q5_k, q6_k on Adreno (llama/23303)

authored and
ggerganov
committed
ggml-cuda: tune RDNA3 Q6_K MMVQ nwarps (llama/23349)

ravel7524
authored and
ggerganov
committed
metal : optimize pad + cpy (llama/23354)
ggerganov
committed
Programmatic Dependent Launch (PDL) for more performance on newer NVIDIA GPUs (Hopper+) (llama/22522)

aendk
authored and
ggerganov
committed
hexagon: HMX quantized matmul rework (llama/23368)

authored and
ggerganov
committed
vulkan: optimize operations in the IM2COL shader (llama/22685)

daniandtheweb
authored and
ggerganov
committed
opencl: refactor backend initilization (llama/23318)

lhez
authored and
ggerganov
committed
hexagon: ssm-conv fix for large prompts (llama/23307)

authored and
ggerganov
committed
ggml : Check the right iface method before using the fallback 2d get (llama/23306)

TheBlueMatt
authored and
ggerganov
committed
metal : optimize concat kernel and fix set kernel threads (llama/23411)
ggerganov
committed
fix(flash-attn): replace f32 with kv_type and q_type (llama/23372)

Constannnnnt
authored and
ggerganov
committed
vulkan: fuse snake activation (mul, sin, sqr, mul, add) (llama/22855)

ServeurpersoCom
authored and
ggerganov
committed
CUDA: fix PDL CC check for JIT compilation (llama/23471)

JohannesGaessler
authored and
ggerganov
committed
ggml-zendnn : add Q8_0 quantization support (llama/23414)

z-sachin
authored and
ggerganov
committed
SYCL: add BF16 to DMMV kernel path (~4x tg speedup on Intel Arc) (llama/21580)

authored and
ggerganov
committed
SYCL : gated_delta_net K>1 (llama/23174)

karavayev
authored and
ggerganov
committed
sycl : Level Zero detection in ggml_sycl_init (llama/23097)

sanmai
authored and
ggerganov
committed
SYCL: improve MoE prefill throughput (llama/23142)

sanmai
authored and
ggerganov
committed
opencl: generalize Adreno MoE kernels on M (llama/23449)

shawngu-quic
authored and
ggerganov
committed
vulkan: fix windows find_package of SPIRV-Headers (llama/23215)

jeffbolznv
authored and
ggerganov
committed
ggml : Check the right iface method before using the fallback 2d get (llama/23514)

dskwe
authored and
ggerganov
committed
hexagon: apply repl optimization in flash attn softmax as #22993 (llama/23455)

njsyw1997
authored and
ggerganov
committed
opencl: batch profiling to improve speed and prevent memory leaks (llama/23495)

shaofeiqi
authored and
ggerganov
committed
TP: fix entirely zero-sized slices per device (llama/23525)

JohannesGaessler
authored and
ggerganov
committed
ggml : Parallelize quant LUT init (llama/23595)

jeffbolznv
authored and
ggerganov
committed
ggml : bump version to 0.12.1 (ggml/1508)
ggerganov
committed
sync : ggml
ggerganov
committed
talk-llama : sync llama.cpp
ggerganov
committed
readme : add AMD ROCm/HIP GPU build instructions (#3823)
Kaihui-AMD
authored
ggml: `gguf_init_from_callback` and `gguf_init_from_buffer` (llama/22341)

authored and
ggerganov
committed
TP: fix ggml context size calculation (llama/22616)

JohannesGaessler
authored and
ggerganov
committed
ggml : bump version to 0.13.0 (ggml/1510)
ggerganov
committed
sync : ggml
ggerganov
committed
benches : update
ggerganov
committed
release : v1.8.5
ggerganov
committed

Commits on May 26, 2026

cli : merge tokens split across UTF-8 boundaries in JSON output (#3751)

texasich
and
texasich
authored

Commits on May 27, 2026

Commits on May 29, 2026

CUDA: add fast walsh-hadamard transform (llama/23615)

authored and
ggerganov
committed
metal : add apple device id (llama/23566)

authored and
ggerganov
committed
CUDA: missing PDL sync for FWHT, better fallback (llama/23690)

JohannesGaessler
authored and
ggerganov
committed
Check batch_compute_passes before sending passes when not doing GPU profiling (llama/23457)

nikhilJain17
authored and
ggerganov
committed
ggml-webgpu: Add MMVQ path for Q4/Q8/Q2_K/Q4_K and clean up legacy MUL_MAT pipeline (llama/23594)

yomaytk
authored and
ggerganov
committed
SYCL: implement ggml_sycl_pool_vmm (llama/22862)

authored and
ggerganov
committed
hexagon: add support for CONCAT op (llama/23648)

max-krasnyansky
authored and
ggerganov
committed
vulkan: optimize conv2d and implement coopmat1 support (llama/22620)

jeffbolznv
authored and
ggerganov
committed
ggml-zendnn : fixed naming of matmul function (llama/20964)

authored and
ggerganov
committed
vulkan: avoid preferring transfer queue on AMD UMA devices (llama/22455)

winstonma
authored and
ggerganov
committed
CUDA: restrict PDL to CTK >= 12.3 due to MSVC issues (llama/23742)

ORippler
authored and
ggerganov
committed
vulkan: add REPEAT op support for f16 to f16. (llama/23298)

l8bloom
authored and
ggerganov
committed
vulkan: use GL_NV_cooperative_matrix_decode_vector for faster matmul (llama/23541)

jeffbolznv
authored and
ggerganov
committed
vulkan: Switch MUL_MAT_VEC to 4 K per iteration for F16/32 (llama/22887)

TheBlueMatt
authored and
ggerganov
committed
ggml-webgpu: Fix how to dispatch WG to some ops (llama/23750)

yomaytk
authored and
ggerganov
committed
hexagon: add support for Q4_1 in MUL_MAT and MUL_MAT_ID (llama/23647)

authored and
ggerganov
committed
ggml-webgpu: remove legacy constants (llama/23672)

reeselevine
authored and
ggerganov
committed
opencl: OP_GATED_DELTA_NET (llama/23312)

ymcki
authored and
ggerganov
committed
Hexagon: OP_GATED_DELTA_NET K>1 support (llama/23531)

ymcki
authored and
ggerganov
committed
ggml: fixed Arm SVE usage bug in vec.h, vec.cpp (llama/22841)

authored and
ggerganov
committed
cuda : fix KQ mask offset integer overflow in fattn MMA kernel (llama/23610)

authored and
ggerganov
committed
vulkan: Fix memory logger unsafe iterator access (llama/23667)

winstonma
authored and
ggerganov
committed
vulkan: fix wrong index variable in inner loop (llama/23665)

winstonma
authored and
ggerganov
committed
vulkan: fast path for walsh-hadamard transform (llama/23687)

jeffbolznv
authored and
ggerganov
committed
hexagon: minor refresh for HMX FA and MM (llama/23796)

authored and
ggerganov
committed
CUDA: route batch>=4 quantized matmul to MMQ on AMD MFMA hardware (llama/23227)

authored and
ggerganov
committed
mmvq Optim: add MMVQ_PARAMETERS_TURING(mmvq_parameter_table_id) for … (#23729)

authored and
ggerganov
committed
ggml: auto apply iGPU flag CUDA/HIP if integrated device (llama/23007)

fl0rianr
authored and
ggerganov
committed
opencl: move backend info printing into its own function (llama/23702)

lhez
authored and
ggerganov
committed
hexagon: basic/generic op fusion support and RMS_NORM+MUL fusion (llama/23835)

max-krasnyansky
authored and
ggerganov
committed
meta : Add missing `buffer` set in allreduce fallback !COMPUTE clear (llama/23480)

TheBlueMatt
authored and
ggerganov
committed
cuda : disables launch_fattn PDL enrollment due to compiler bug (llama/23825)

aendk
authored and
ggerganov
committed
sync : ggml
ggerganov
committed
talk-llama : sync llama.cpp
ggerganov
committed
ggml : bump version to 0.13.1 (ggml/1523)
ggerganov
committed
sync : ggml
ggerganov
committed

Commits on Jun 1, 2026

Commits on Jun 2, 2026

Commits on Jun 6, 2026

Commits on Jun 8, 2026

ci : add ccache to build-sycl [no ci] (#3859)
danbev
authored
model : support for DeepseekV32ForCausalLM with generic DeepSeek Sparse Attention (DSA) implementation (llama/23346)

committed
CUDA: Check PTX version on host side to guard PDL dispatch (llama/23530)

authored and
ggerganov
committed
ggml-webgpu: add q4_0/q8_0 SET_ROWS (llama/23760)

reeselevine
authored and
ggerganov
committed
ggml-webgpu: Check earlier for WebGPU required features (llama/23879)

reeselevine
authored and
ggerganov
committed
vulkan: add Flash Attention support for BFloat16 KV cache (llama/23420)

0cc4m
authored and
ggerganov
committed
ggml : add some lsx support (llama/23798)

MQ-mengqing
authored and
ggerganov
committed
metal : restore im2col implementation for large kernels (llama/23901)
ggerganov
committed
opencl: support bf16 by converting to f16 (llama/23839)

lhez
authored and
ggerganov
committed
sycl : Optimize Q3_K mul_mat by reorder (llama/23725)

arthw
authored and
ggerganov
committed
Add more types in GET_ROWS OP (llama/23710)

arthw
authored and
ggerganov
committed
Support Q4_1, Q5_0, Q5_1 in Flash-attention (llama/23812)

arthw
authored and
ggerganov
committed
vulkan: Removed unused functions (llama/23175)

winstonma
authored and
ggerganov
committed
vulkan: Block-load Q3_K/Q6_K block data and subtract on 32b ints (llama/23056)

TheBlueMatt
authored and
ggerganov
committed
TP: quantized KV cache support (llama/23792)

JohannesGaessler
authored and
ggerganov
committed
vulkan: reduce host memory lock contention (llama/23376)

winstonma
authored and
ggerganov
committed
vulkan: don't hold the device mutex while compiling pipelines (llama/23641)

jeffbolznv
authored and
ggerganov
committed
metal: template GLU kernels to support f16/f32 (llama/23882)

shrivasshankar
authored and
ggerganov
committed
opencl: add basic support for q5_0 and q5_1 (llama/23548)

authored and
ggerganov
committed
revert to using global_invocation_id for cpy shader (llama/23955)

yomaytk
authored and
ggerganov
committed
opencl: fix compiler warnings for non-adreno path (llama/23922)

lhez
authored and
ggerganov
committed
clean up unused variables warnings (llama/23975)

anavp-nvidia
authored and
ggerganov
committed
hexagon: add gelu_quick (llama/24007)

tboinovski1
authored and
ggerganov
committed
hexagon: MUL_MAT, MUL_MAT_ID, FLASH_ATTN and GDN cleanup and optimizations for latest models (llama/23989)

max-krasnyansky
authored and
ggerganov
committed
hexagon: profiler output fix and script updates (llama/24042)

max-krasnyansky
authored and
ggerganov
committed
opencl: use flat variants of q4_K and q6_K gemv for very large M (llama/24006)

lhez
authored and
ggerganov
committed
cuda: reserve space for quantize kv-cache at startup (llama/23907)

authored and
ggerganov
committed
ggml-cpu: use runtime SVE width in FWHT (llama/24059)

chaxu01
authored and
ggerganov
committed
Avoid PDL race conditions by disabling __restrict__ when PDL is used (llama/24030)

authored and
ggerganov
committed
ggml-cpu: extend RVV quantization vec dot to higher VLENs (llama/22754)

authored and
ggerganov
committed
ggml-webgpu: FlashAttention refactor + standardize quantization support (llama/23834)

reeselevine
authored and
ggerganov
committed
metal : reduce rset heartbeat from 500ms -> 5ms (llama/24074)
ggerganov
committed
ggml: vectorize ggml_vec_dot_q4_1_q8_1 with WASM SIMD128 (llama/22209)

sirohikartik
authored and
ggerganov
committed
sycl : port multi-column MMVQ from CUDA backend (llama/21845)

masonmilby
authored and
ggerganov
committed
CUDA: enroll mul_mat_vec_q_moe into pdl (llama/24087)

ORippler
authored and
ggerganov
committed
kleidiai : dynamic chunck-based scheduling for hybrid execution (llama/23819)

chaxu01
authored and
ggerganov
committed
vulkan: add fwht support for Intel with shmem reduction (llama/23964)

0cc4m
authored and
ggerganov
committed
opencl: improve get_rows, cpy, concat and q6_k flat gemv (llama/24160)

lhez
authored and
ggerganov
committed
vulkan: check coopmat2 features before reporting support (llama/24186)

0cc4m
authored and
ggerganov
committed
metal : fix im2col 1D case (audio models) (llama/24220)

ngxson
authored and
ggerganov
committed
HIP: add gfx1152 and gfx1153 to RDNA3.5 (llama/24129)

harkgill-amd
authored and
ggerganov
committed
sync : ggml
ggerganov
committed
ggml : bump version to 0.14.0 (ggml/1533)
ggerganov
committed
sync : ggml
ggerganov
committed
talk-llama : sync llama.cpp
ggerganov
committed

Commits on Jun 9, 2026

Commits on Jun 15, 2026

cuda: reset cuda context after reading memory size (llama/23935)

0cc4m
authored and
ggerganov
committed
vulkan: Use cm2 decode_vector for mul_mat_id B matrix loads (llama/23991)

jeffbolznv
authored and
ggerganov
committed
Implement 2D workgroups for scale, binary, and unary ops (llama/24044)

nikhilJain17
authored and
ggerganov
committed
Handle buffer overlap / buffer aliasing for concat operator (llama/24000)

authored and
ggerganov
committed
ggml-webgpu: Improve prefill speeds for k-quants + refactor matmul for Q4/Q5/Q8 and k-quants (llama/24225)

yomaytk
authored and
ggerganov
committed
ggml-webgpu: Add clang-format job (llama/24308)

reeselevine
authored and
ggerganov
committed
Remove case for GGML_TYPE_Q4_K in mvvq.cu (llama/23528)

ravel7524
authored and
ggerganov
committed
ggml-cpu : fix rms_norm_back wrong output under in-place aliasing (llama/24305)

devYRPauli
and
ggerganov
committed
ggml : add GGML_OP_COL2IM_1D (llama/24206)

ServeurpersoCom
authored and
ggerganov
committed
vulkan: add `v_dot2_f32_f16` support in matrix-matrix multiplication and Flash Attention (llama/24123)

0cc4m
authored and
ggerganov
committed
vulkan: reduce iq1 shared memory usage for mul_mm (llama/24287)

jeffbolznv
authored and
ggerganov
committed
CUDA: Fix ssm_scan_f32 data-races (llama/24360)

ORippler
authored and
ggerganov
committed
Remove padding and multiple D2D copies for MTP (llama/24086)

gaugarg-nv
authored and
ggerganov
committed
vulkan: use medium matmul tile on Asahi Linux (llama/24306)

xingjianll
authored and
ggerganov
committed
vulkan: add fast path for contiguous buffer transfers (llama/23973)

winstonma
authored and
ggerganov
committed
ggml : bump version to 0.15.0 (ggml/1539)
ggerganov
committed
vulkan: ifdef eMesaHoneykrisp (build fix) (llama/24479)

jeffbolznv
authored and
ggerganov
committed
opencl: add q5_0/q5_1 gemm and gemv kernels for Adreno (llama/24319)

authored and
ggerganov
committed
ggml: support concat for scalar types at cuda backend (llama/24011)

zihaomu
authored and
ggerganov
committed
ggml : bump version to 0.15.1 (ggml/1541)
ggerganov
committed
sync : ggml
ggerganov
committed
talk-llama : sync llama.cpp
ggerganov
committed

Commits on Jun 17, 2026

Commits on Jun 18, 2026

ci : add GGML_NATIVE=OFF and GGML_BMI2=OFF to windows-blas (#3891)
danbev
authored

Commits on Jun 19, 2026

release : v1.9.1 (#3892)
danbev
authored

Commits on Jun 23, 2026

Merge tag 'v1.9.1' into sync-upstream-v1.9.1
xAlcahest
committed

Commits on Jun 24, 2026

ci: build CUDA for Blackwell sm_120 (RTX 50) on CUDA 12.9
xAlcahest
committed