Holistic Scaling Laws for Optimal Mixture-of-Experts Architecture Optimization
Weilin Wan, Jingtao Han, Weizhong Zhang, Cheng Jin
Accepted at NeurIPS, 2026
This paper establishes the first holistic framework for global MoE architectural optimization, reducing the intractable 16-dimensional search space to a two-phase sequential procedure via mathematical constraints and rank-preserving properties.
