Abstract:For large-model online inference on the Ascend 910B platform, the Prefill and Decode stages exhibit significant heterogeneous characteristics in computational density, memory access bandwidth, and communication sensitivity. As a result, the mainstream “Prefill-TP+Decode-DP” deployment strategy suffers from issues such as memory redundancy due to weight replication, operator memory access bottlenecks, and sequence-length load imbalance in long-context, high-concurrency scenarios. Based on the PD-separation architecture, this study proposes an operator-level hybrid parallel optimization framework. In the Decode stage, a hybrid partitioning and communication optimization strategy is designed to target memory-bound token embedding (Embedding), the attention output projection matrix (o_proj), and the language model head (LMHead), effectively alleviating KV Cache memory pressure while significantly reducing per-token generation latency. In the Prefill stage, a staged hybrid parallel scheme combining data parallelism (DP), tensor parallelism (TP), and sequence parallelism (SP) is proposed. Sequence-dimension balanced partitioning mitigates the “bucket effect” caused by variable-length requests, and a Shard Linear weight de-redundancy mechanism oriented to Ascend operator/format constraints is further proposed to achieve computation-communication overlap. Based on the deployment and evaluation of DeepSeek-R1 671B (W8A8 quantization) on an Ascend 910B cluster using vLLM-Ascend, the results show that, without accuracy loss, peak memory per device in the Decode stage is reduced by 8.68 GiB and per-token latency by about 2 ms. In the Prefill stage, the average TTFT is reduced by about 20% across multiple input lengths, and Decode QPS improves by 40.2% under high memory pressure.