AMD releases open-source AI model trained on its own GPUs

AMD Advanced Micro Devices, Inc. ↗ 2.49% released Instella-MoE on July 24, a fully open mixture-of-experts language model with 16 billion total parameters trained from scratch on AMD Instinct MI300X and MI325X GPUs. The release includes model weights from every training stage, training configurations, data mixtures, intermediate checkpoints, and inference code — making it one of the most transparent large-scale AI model releases to date.

A Competitive Open Model at Modest Compute Cost

Instella-MoE activates only 2.8 billion of its 16 billion parameters per token, keeping inference costs closer to those of a much smaller dense model while delivering results that AMD says lead among fully open-source models of comparable size. On standard benchmarks, the base model achieved an average score of 76.7, ahead of SmolLM3-3B-Base and OLMo-3-7B, and competitive with open-weight models such as Moonlight-16B-A3B. The model supports 64K-token context windows and went through a six-stage training pipeline spanning pre-training, mid-training, long-context extension, supervised fine-tuning, direct preference optimization, and reinforcement learning.weex+1

The architecture incorporates two novel techniques: Gated Multi-head Latent Attention, which adds a learned gate to selectively filter low-utility attention responses, and FarSkip-Collective, a connectivity method that overlaps communication with computation during expert-parallel training, yielding a 12.7% speedup in pre-training and up to 39.2% reduction in time-to-first-token during inference.rocm.blogs.amd

AMD's Play for AI Training Credibility

The release builds on AMD's earlier Instella series of 3-billion-parameter models introduced in 2025 and follows Zyphra's ZAYA1, another large-scale MoE model trained on MI300X hardware last year. Together, these efforts represent AMD's campaign to prove its GPUs and ROCm software stack can support frontier-scale AI development — a market long dominated by Nvidia ↘ 0.03% and its CUDA ecosystem.reddit+2

AMD licensed Instella-MoE under a Research RAIL license for academic and research use. The company said it plans to continue extending this work with larger models and further efficiency improvements.rocm.blogs.amd