Inside FSDP with PyTorch and Ray: Scaling Model Training with Fully Sharded Data Parallel
Read the full article28d ago
You might also wanna read
MegaTrain: Full Precision Training of 100B+ Parameter Large Language Models on a Single GPU
Authors: Zhengqing Yuan, Hanchi Sun, Lichao Sun, Yanfang Ye
arxiviq.substack.com·12d ago
Scikit-Learn Pipelines: Build & Optimize ML Workflows
Chain preprocessing and model training steps into one Scikit-Learn pipeline, tune hyperparameters with GridSearchCV, prevent data leakage
daily.dev·1y ago
Modular: Modular Platform 25.5: Introducing Large Scale Batch Inference
Modular Platform 25.5: Introducing Large Scale Batch Inference
modular.com·11mo ago
MegaTrain: System for Training 100B+ Parameter LLMs on Single GPU Using CPU Memory
We present MegaTrain, a memory-centric system that efficiently trains 100B+ parameter large language models at full precision on a single GP
arxiv.org·3mo ago
Modular: PyTorch and LLVM in 2025 — Keeping up With AI Innovation
PyTorch and LLVM in 2025 — Keeping up With AI Innovation
modular.com·8mo ago
GPEmu: A GPU Emulator for Rapid, Low-Cost Deep Learning Prototyping [pdf]
Article URL: Comments URL: Points: 3 # Comments: 0
vldb.org·1y ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.