K2 Horizon Collection K2 Horizon models, datasets, and supporting resources • 22 items • Updated about 14 hours ago • 77
Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning Paper • 2607.07508 • Published Jul 8 • 32
view article Article Keep the Tokens Flowing: Lessons from 16 Open-Source RL Libraries +7 aminediroHF, qgallouedec, kashif, lewtun, edbeeching, albertvillanova, nouamanetazi, lvwerra, sergiopaniego • Mar 10 • 188
view article Article Mixture of Experts Explained +4 osanseviero, lewtun, philschmid, smangrul, ybelkada, pcuenq • Dec 11, 2023 • 1.18k
view article Article SmolLM3: smol, multilingual, long-context reasoner +21 eliebak, cmpatino, anton-l, edbeeching, m-ric, nouamanetazi, akseljoonas, guipenedo, hynky, clefourrier, SaylorTwift, kashif, qgallouedec, hlarcher, glutamatt, Xenova, reach-vb, ngxson, craffel, lewtun, loubnabnl, lvwerra, thomwolf • Jul 8, 2025 • 791
Running on CPU Upgrade Featured 3.29k The Smol Training Playbook 📚 3.29k The secrets to building world-class LLMs
The Ultimate Collection of Code Classifiers Collection 🔥 15 classifiers, 124M parameters, one per programming language— for assessing the educational value of GitHub code • 15 items • Updated May 5, 2025 • 16
Essential-Web v1.0: 24T tokens of organized web data Paper • 2506.14111 • Published Jun 17, 2025 • 48
view article Article nanoJAXGPT: A pedagogical introduction to JAX/Equinox sachithgunasekara • Oct 23, 2024 • 8
CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training Paper • 2504.13161 • Published Apr 17, 2025 • 98
Running 136 TxT360: Trillion Extracted Text 📖 136 Explore and download the TxT360 LLM pretraining dataset
Running 4.01k The Ultra-Scale Playbook 🌌 4.01k The ultimate guide to training LLM on large GPU Clusters