Thunder-Tok: Revolutionizing Tokenization with Smarter Strategies
Thunder-Tok, a new subword tokenizer, optimizes language model efficiency by reducing token fertility by 25% in English and 9% in Korean. It challenges the status quo without sacrificing performance.
Read the full articleYou might also wanna read
An Algorithm for Computing Optimal Tokenizers in Practice
In this post, I will present an algorithm that was able to compute an optimal tokenizer in some settings. This result is cool because optima
Das Ende des Hypes? – Tokenomics statt Tokenmaxxing: KI kostet immer mehr
Der Einsatz von Künstlicher Intelligenz wird immer kostspieliger. Wieso das so ist und was dagegen unternommen wird.
Systematic Evaluation of Deep Learning Optimizers Reveals Limited Speedup Over AdamW in Language Model Pretraining
AdamW has long been the dominant optimizer in language model pretraining, despite numerous claims that alternative optimizers offer 1.4 to 2
arxiv.org·10mo ago
PHOTON: Hierarchical Autoregressive Model for Efficient Language Generation
Transformers operate as horizontal token-by-token scanners; at each generation step, the model attends to an ever-growing sequence of token-
arxiv.org·6mo ago
Beyond Autoregression: LLaDA2.1 and the Case for Self-Editing Language Models
Introduction Every mainstream large language model today generates text the same way: one token at a time, left to right, no looking back. I
DSpark: How DeepSeek Made Its Own Serving Stack 60-85% Faster Without Touching Model Quality
A large language model writes one token at a time.
Comments
Sign in to join the conversation.
No comments yet. Be the first.