Published in Annual Conference on Machine Learning and Systems (MLSys), 2026
This paper introduces an algorithm-system co-design for accurate 2-bit KV cache quantization, significantly reducing GPU memory consumption and increasing inference throughput during LLM inference.
Published in International Conference on Very Large Data Bases (VLDB), 2024
This work develops a highly efficient LLM acceleration framework that provides runtime support for LLM inference with unstructured sparsity, reducing inference costs by up to 50%.
Published in USENIX Annual Technical Conference (ATC), 2024
We designed and implemented a GPU kernel with unified Tensor Core support for various quantization bit-widths, achieving up to 2.65x throughput improvement on LLaMA-70B.
Published in IEEE/ACM International Symposium on Microarchitecture (MICRO), 2021
Co-designed a hardware accelerator and memory optimization strategy for highly-efficient Bayesian Neural Network (BNN) training on server and edge devices.
Published in International Symposium on Computer Architecture (ISCA), 2021
Prototyped an efficient hardware architecture for large-scale LSTM network training, utilizing variable compression and cell skipping.
Published in Design, Automation and Test in Europe Conference (DATE), 2021
Designed and instantiated a lightweight pattern-matching hardware processor on real FPGA boards, integrating it with ARM CPUs.
Published in The University of Sydney (USYD), 2026
My doctoral dissertation focuses on alleviating the memory wall in Large Language Model (LLM) inference through algorithm-system co-design, including low-bit weight/KV quantization and unstructured weight sparsity acceleration.
Published in University of Science and Technology of China (USTC), 2021
This thesis focuses on the architecture design, Verilog HDL prototyping, and hardware-software co-design of a high-performance, memory-efficient lightweight automata processing engine for large-scale pattern matching.