Abstract:Mixture-of-experts (MoE) models utilize expert parallelism (EP) inference to reduce memory pressure on individual GPUs and enable parallel expert computation. However, on hardware platforms with limited interconnect bandwidth, All2All communication introduces a severe communication bottleneck. The prevalent DeepSeek two-batch parallel scheduling scheme achieves computation-communication overlap, while it faces two critical challenges in real-world deployments: the degradation of computational intensity in attention operators due to batch splitting, and insufficient overlap caused by the temporal asymmetry between computation and communication operators. To address these issues, this study proposes a novel computation-communication overlap scheduling optimization for MoE model inference. First, multi-batch computation-communication overlap is selectively applied to certain operators, while the attention module processes full batches to preserve arithmetic intensity. Second, a multi-stream overlap scheduling mechanism based on streaming multiprocessor allocation is designed to achieve greater overlap between computation and communication. This approach alleviates the overhead caused by the reduced computational intensity of some operators and better exploits computation-communication overlap. Experimental results on a single-server, 8-GPU H20 platform with the Qwen3-30B model show that the proposed scheme improves decoding throughput by 6.0%–23.6%, achieving 1.74×–6.29× the performance gain of the two-batch parallel scheduling scheme.