Dr. Haojun Xia

Dr. Haojun Xia

RSE at Google Research, PhD at USYD, USTCer, ML system, LLM Inference

Talks and presentations

Conference Oral: Kitty: Accurate and Efficient 2-bit KV Cache Quantization with Dynamic Channel-wise Precision Boost

May 20, 2026

Oral Presentation, International Conference on Machine Learning and Systems (MLSys 2026), Bellevue, WA, USA

Delivered an oral presentation on our research paper “Kitty” at the International Conference on Machine Learning and Systems (MLSys 2026). The talk focused on the critical bottlenecks of KV cache in Large Language Model (LLM) inference, introducing an innovative approach that combines low-bit quantization with specialized hardware execution to dramatically reduce memory footprint and unlock massive serving throughput improvements.

Conference Talk: Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity

August 29, 2024

Talk, International Conference on Very Large Data Bases (VLDB 2024), Guangzhou, China

Presented our full-paper research on Flash-LLM at the 50th International Conference on Very Large Data Bases (VLDB 2024). The presentation detailed a highly-efficient algorithm-system co-design that effectively leverages unstructured sparsity on modern Tensor Cores, offering a powerful and cost-effective remedy to overcome the memory wall for large generative model inference.

Conference Talk: Quant-LLM: Accelerating the Serving of Large Language Models via FP6-Centric Algorithm-System Co-Design on Modern GPUs

July 11, 2024

Talk, USENIX Annual Technical Conference (USENIX ATC 2024), Santa Clara, CA, USA

Presented our full-paper research on FP6-LLM at the USENIX Annual Technical Conference (USENIX ATC 2024). The presentation covered the core compiler and runtime co-design to enable unified Tensor Core support for 6-bit quantized LLM inference, demonstrating an end-to-end throughput speedup of up to 2.65x on LLaMA-70B models.