AI Systems / LLM Inference · vLLM / vLLM-Omni · C++ / CUDA / Python
_{From CUDA kernels and a hand-built runtime up to vLLM upstream — source first, minimal scope, verified changes.}

• LLM Inference Systems — built and optimized inference runtimes across scheduling, KV cache, memory management, and model execution · C++ / CUDA / Python
• Prefix Cache & Cross-Stage Inference — designed cache lifecycle and state-management abstractions for cross-stage intermediate-state reuse in multimodal inference · vLLM / vLLM-Omni
• Multimodal Serving & Scheduling — worked on cross-stage execution, request batching, and scheduling for autoregressive and diffusion pipelines · Distributed Inference / PyTorch

• vllm-project/vllm-omni — cross-stage inference, prefix caching, batching, and image-generation infrastructure · #7902 #7920 #8134 #8174
• ThinkFlowLab/vllm-rlt — integrated Nanbeige4.2 with PD serving and identified model/serving integration issues · #77