Star-Attention
Efficient LLM Inference over Long Sequences
About
Star Attention improves the inference time by up to 11x while preserving 97-100% of accuracy. The method is compatible with most Transformer-based LLMs trained with global attention, operating seamlessly out-of-the-box without additional training/finetuning. Furthermore, Star Attention is orthogonal to other optimization methods, including Flash Attention and KV cache compression techniques, allowing for potential combined enhancements.
Open Source Health
- Stars
- 392
- Forks
- 24
- License
- Apache-2.0
- Last commit
- 1 years ago
Related Categories
Vendor
NVIDIA
Publisher of DALI (NVIDIA Data Loading Library) and Tensorrt
Quick Links
Open Source
More by NVIDIA
Related Products
SkillSpector
Security scanner for AI agent skills. Detect vulnerabilities, malicious patterns, security risks, prompt injection, data exfiltration, and supply-chain risks in Claude Code, Codex, and MCP skills before you install them.
More from this vendor
Nvbench
CUDA Kernel Benchmarking Library
More from this vendor
OSMO
The developer-first platform for scaling complex Physical AI workloads across heterogeneous compute—unifying training GPUs, simulation clusters, and edge devices in a simple YAML
More from this vendor
The Algorithm
Source code for the X Recommendation Algorithm
Shared categories
Recommenders
TensorFlow Recommenders is a library for building recommender system models using TensorFlow.
Shared categories