X's For You Feed Algorithm: Open-Source Recommendation System Using Grok-Based Transformer
By
grainier
Fresh out the oven, still warm. Top of the tray.
Summary
This article describes the open-source algorithm powering X's "For You" feed recommendation system. The algorithm combines in-network content from followed accounts with out-of-network content discovered through machine learning retrieval, then ranks everything using a Grok-based transformer model adapted from xAI's Grok-1 release. The system retrieves posts from two sources: "Thunder" for in-network content and "Phoenix" for out-of-network content, with the goal of providing personalized content recommendations.
Key quotes
· 4 pulledThis repository contains the core recommendation system powering the 'For You' feed on X.
It combines in-network content (from accounts you follow) with out-of-network content (discovered through ML-based retrieval) and ranks everything using a Grok-based transformer model.
The transformer implementation is ported from the Grok-1 open source release by xAI, adapted for recommendation system use cases.
The For You feed algorithm retrieves, ranks, and filters posts from two sources: In-Network (Thunder): Posts from accounts you follow, Out-of-Network (Phoenix): Posts discovered through ML-based retrieval.
You might also wanna read
DeepSeek-V4: Hybrid Sparse-Attention Architecture Enables Efficient Million-Token Context Inference
DeepSeek-V4 introduces a hybrid sparse-attention architecture combined with on-policy distillation across domain specialists, enabling 1M-to
artgor.medium.com·6h ago
Rotary GPU: Enabling Large Mixture-of-Experts Models on Consumer Laptop GPUs with Limited Memory
This paper presents Rotary GPU, an exploratory approach to running large Mixture-of-Experts (MoE) language models on consumer-grade hardware
arxiv.org·1d ago
LinkedIn cuts GPU training hours by 65% with Generative Recommender system optimizations
LinkedIn has developed a Generative Recommender (GR) system that models user activity as token sequences, offering richer long-context perso
startuphub.ai·3d ago
Rank-Aware Decomposition Technique Reduces Computation in Recommender Systems by 87.5%
This paper presents a rank-aware decomposition technique for deep ranking models in industrial recommender systems. The key insight is that
arxiv.org·3d ago
Hands-on evaluation of MiniMax M2.7 via API on ML and coding workflows
The author evaluates MiniMax M2.7 by using it through Claude Code on three real-world ML and coding workflows: scaffolding a Kaggle competit
andlukyane.com·11d ago
Orthrus: A Dual-Architecture Framework for Fast, Lossless LLM Inference via Diffusion Decoding
Orthrus is a dual-architecture framework that combines autoregressive LLMs with diffusion models to enable fast, lossless parallel token gen
github.com·16d ago