Abstract:In recent years, large-scale models based on the Transformer architecture have been widely applied across various domains and have demonstrated exceptional performance. However, the efficient deployment of their core operators on heterogeneous platforms faces significant challenges, including complex adaptation requirements and high costs associated with manual optimization. Furthermore, existing compilers exhibit limitations in block-level logical expression and cross-platform performance portability. To address these issues, this study proposes DeepGen, an MLIR-based high-performance operator generation framework. By constructing a block-level hierarchical intermediate representation (IR), the proposed framework enables direct characterization of computation patterns, memory access behaviors, and parallel structures of operators. Concurrently, a suite of universal cross-platform optimization passes is designed to achieve progressive optimization of computation scheduling and data access. Moreover, by decoupling computational logic from data movement processes, DeepGen provides a unified optimization mechanism for high-performance operator generation on heterogeneous platforms. Experimental results indicate that on NVIDIA A100/V100 and Hygon Z100/K100 platforms, DeepGen improves the computational efficiency of matrix multiplication operators by an average factor of 1.10× and 1.18× compared to cuBLAS and rocBLAS, respectively. In comparison with Triton, the efficiency of the attention operator is enhanced by up to 3.71× with DeepGen. Furthermore, the end-to-end inference performance of LLaMA2 and BERT models is improved by average factors of 1.76× and 2.01×, respectively. These experimental results validate that DeepGen possesses excellent cross-platform performance portability and versatility for operator generation and optimization in Transformer models.