Published:
Designed and implemented a multi-stage pipelined processor core (MIPS / RISC-V ISA) from scratch using Verilog HDL, featuring hazardous forwarding logic and comprehensive simulation verification.
Published:
Designed and implemented a domain-specific core optimized for ADFA algorithms and integrated it into an ARM-based heterogeneous SoC via high-throughput AXI channels, evaluated on a physical Zedboard FPGA platform.
Published in Design, Automation and Test in Europe Conference (DATE), 2021
Designed and instantiated a lightweight pattern-matching hardware processor on real FPGA boards, integrating it with ARM CPUs.
Published in International Symposium on Computer Architecture (ISCA), 2021
Prototyped an efficient hardware architecture for large-scale LSTM network training, utilizing variable compression and cell skipping.
Published in University of Science and Technology of China (USTC), 2021
This thesis focuses on the architecture design, Verilog HDL prototyping, and hardware-software co-design of a high-performance, memory-efficient lightweight automata processing engine for large-scale pattern matching.
Published in IEEE/ACM International Symposium on Microarchitecture (MICRO), 2021
Co-designed a hardware accelerator and memory optimization strategy for highly-efficient Bayesian Neural Network (BNN) training on server and edge devices.
Published in USENIX Annual Technical Conference (ATC), 2024
We designed and implemented a GPU kernel with unified Tensor Core support for various quantization bit-widths, achieving up to 2.65x throughput improvement on LLaMA-70B.
Published in International Conference on Very Large Data Bases (VLDB), 2024
This work develops a highly efficient LLM acceleration framework that provides runtime support for LLM inference with unstructured sparsity, reducing inference costs by up to 50%.
Published in Annual Conference on Machine Learning and Systems (MLSys), 2026
This paper introduces an algorithm-system co-design for accurate 2-bit KV cache quantization, significantly reducing GPU memory consumption and increasing inference throughput during LLM inference.
Published in The University of Sydney (USYD), 2026
My doctoral dissertation focuses on alleviating the memory wall in Large Language Model (LLM) inference through algorithm-system co-design, including low-bit weight/KV quantization and unstructured weight sparsity acceleration.
Published:
Presented our full-paper research on FP6-LLM at the USENIX Annual Technical Conference (USENIX ATC 2024). The presentation covered the core compiler and runtime co-design to enable unified Tensor Core support for 6-bit quantized LLM inference, demonstrating an end-to-end throughput speedup of up to 2.65x on LLaMA-70B models.
Published:
Presented our full-paper research on Flash-LLM at the 50th International Conference on Very Large Data Bases (VLDB 2024). The presentation detailed a highly-efficient algorithm-system co-design that effectively leverages unstructured sparsity on modern Tensor Cores, offering a powerful and cost-effective remedy to overcome the memory wall for large generative model inference.
Published:
Delivered an oral presentation on our research paper “Kitty” at the International Conference on Machine Learning and Systems (MLSys 2026). The talk focused on the critical bottlenecks of KV cache in Large Language Model (LLM) inference, introducing an innovative approach that combines low-bit quantization with specialized hardware execution to dramatically reduce memory footprint and unlock massive serving throughput improvements.
Undergraduate Lab Course, University of Science and Technology of China (USTC), 2016
Served as a Lab Teaching Assistant for Digital Circuit Experiments in Fall 2016. Guided undergraduate students through hands-on hardware laboratory sessions, troubleshooting hardware prototype testing, and grading experimental reports.
Undergraduate Course, University of Science and Technology of China (USTC), 2017
Served as a Teaching Assistant for the Digital Circuit Theory course in Fall 2017. Assisted students in mastering fundamental logic design, sequential circuits, and theoretical building blocks of digital hardware.
Undergraduate Course, University of Science and Technology of China (USTC), 2018
Served as a Teaching Assistant for the Computer Architecture course in Spring 2018. Responsible for guiding students through advanced computer architecture topics, managing course logistics, and evaluating academic performance.
Undergraduate Course, University of Science and Technology of China (USTC), 2019
Served as a Teaching Assistant for the Computer Architecture course in Spring 2019. Duties included holding office hours, grading assignments, and helping undergraduate students understand core computer architecture concepts.