Answer.AI Develops System to Train 70 Billion Language Model on Regular Desktop
By
amrrs
A five-star bake. Worth schmearing, sharing, saving.
Summary
Answer.AI has developed an open-source system that can efficiently train a 70 billion large language model on a regular desktop computer with two or more standard gaming GPUs. The system is a collaboration between Answer.AI, Tim Dettmers from U Washington, and Hugging Face's Titus von Koeller and Sourab Mangrulkar, combining FSDP and QLoRA technologies. This advancement will benefit the open-source community by enabling the release of better models.
Key quotes
· 3 pulledToday, we’re releasing Answer.AI’s first project: a fully open source system that, for the first time, can efficiently train a 70b large language model on a regular desktop computer with two or more standard gaming GPUs (RTX 3090 or 4090).
This system will help the open source community release better models.
We’re releasing an open source system, based on FSDP and QLoRA, that can train a 70b model on two 24GB GPUs.
You might also wanna read
Researchers use IBM quantum computer to boost AI language model accuracy by reducing perplexity
Researchers have demonstrated the first use of quantum computers to enhance a production-scale large language model (LLM). By running an AI
livescience.com·4d ago
Monostate: All-in-One AI Training Platform for Fine-Tuning LLMs
Monostate is an all-in-one AI training platform that enables users to fine-tune large language models (LLMs) with their own data using vario
Product Hunt·2mo ago