LongAlign
byTHUKEG
[EMNLP 2024] LongAlign: A Recipe for Long Context Alignment of LLMs
LLM OrchestrationOpen Source AI & Machine LearningOpen Source
About
LongAlign is the first full recipe for LLM alignment on long context. We propose the LongAlign-10k dataset, containing 10,000 long instruction data of 8k-64k in length. We investigate on training strategies, namely packing (with loss weighting) and sorted batching, which are all implemented in our code. For real-world long context evaluation, we introduce LongBench-Chat that evaluates the instruction-following capability on queries of 10k-100k length.
Open Source Health
- Stars
- 262
- Forks
- 21
- License
- Apache-2.0
- Last commit
- 2 years ago
Python
Alternatives to LongAlign
The Algorithmby TwitterSource code for the X Recommendation Algorithm
Recommendersby TensorFlowTensorFlow Recommenders is a library for building recommender system models using TensorFlow.- RPG_KDD2025by FacebookresearchThis repository provides the code for implementing RPG described in our KDD'25 paper "Generating Long Semantic IDs in Parallel for Recommendation".
- Pairecby AlibabaA Go web framework for quickly building recommendation online services based on JSON configuration.
Textrankby Summa NLPTextRank implementation for Python 3.
Resources & Links
Related Categories
Using LongAlign?
Track its cost next to the rest of your stack and get a reminder before it renews.
Add to my stackVendor
THUKEG
Open-source projects on GitHub: Slime, AgentBench and P-tuning-v2
