SpAtten
[HPCA'21] SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning
About
We propose sparse attention (SpAtten) with KV token pruning, local V pruning, head pruning, and KV progressive quantization to improve LLM efficiency.
Open Source Health
- Stars
- 140
- Forks
- 11
- License
- MIT
- Last commit
- 2 years ago
Related Categories
Vendor
MIT HAN Lab
Efficient AI Computing. PI: Song Han
Quick Links
Open Source
More by MIT HAN Lab
Related Products
Once For All
[ICLR 2020] Once for All: Train One Network and Specialize it for Efficient Deployment
More from this vendor
CHaiDNN
HLS based Deep Neural Network Accelerator Library for Xilinx Ultrascale+ MPSoCs
Top category match
SoulX-FlashTalk
SoulX-FlashTalk is the first 14B model to achieve sub-second start-up latency (0.87s) while maintaining a real-time throughput of 32 FPS on an 8xH800 node.
Top category match
Ts Pattern
The exhaustive Pattern Matching library for TypeScript, with smart type inference.
Top category match
Nobodywho
NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.
Top category match