ByteDance trains massive AI model in bid to rival Anthropic

TikTok owner training a model with 10 trillion parameters.

The short version

• ByteDance is training an AI model that could approach the size of Anthropic’s most cutting-edge Mythos system, as Chinese companies continue to narrow the gap with the top US labs.
• The Chinese tech giant is at an early stage of training a model with as many as 10 trillion parameters—three times larger than Moonshot’s Kimi K3, the biggest Chinese model released to date, according to three people with knowledge of the matter.
• The ByteDance model is being pre-trained—a stage that typically takes three to six months—before it is fine-tuned and released if all goes well, one of the people said.

What happened

The exact model size would only be determined at a later stage. Anthropic doesn’t disclose the size of its models, but industry estimates say its most advanced Mythos 5 has about 8 trillion parameters and Fable 5 about 5 trillion.

Why it matters

While parameter count sets the fundamental capacity or memory limits for the models to store information, actual capability also depends on other factors, such as data quality and training methods.

Summary by Nerd News Network. Read the full article at Ars Technica — AI via the links above and below.