Rethinking Chinese Text Representation: From Tokens to Pixels
A new approach to Chinese language models replaces text tokens with images of characters, outperforming traditional methods by 21%. Could this challenge existing assumptions about language processing?
Read the full articleYou might also wanna read
Large Language Models Do Not Always Need Readable Language
Authors: Jiayi Zhu, Haoxuan Peng, Junxi Wang, Liang Ke, Chen Zhang, Linfeng Zhang
Chonky_mmbert_small_multilingual_v1: Transformer Model for Semantic Text Segmentation in RAG Systems
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Research Proves Transformer Language Models Are Injective and Invertible
Transformer components such as non-linear activations and normalization are inherently non-injective, suggesting that different inputs could
arxiv.org·8mo ago
Beyond Autoregression: LLaDA2.1 and the Case for Self-Editing Language Models
Introduction Every mainstream large language model today generates text the same way: one token at a time, left to right, no looking back. I
MegaTrain: Full Precision Training of 100B+ Parameter Large Language Models on a Single GPU
Authors: Zhengqing Yuan, Hanchi Sun, Lichao Sun, Yanfang Ye
An Algorithm for Computing Optimal Tokenizers in Practice
In this post, I will present an algorithm that was able to compute an optimal tokenizer in some settings. This result is cool because optima
Comments
Sign in to join the conversation.
No comments yet. Be the first.