With the development of petroleum industry and the improvement of oil recovery technology, the fault diagnosis of pumping unit is particularly important in the production process. The deficiencies in diagnosis of traditional indicator diagram towards pumping unit are analyzed in the paper. The main insufficiencies are focused on the diagnosis and analytic methods which are the qualitative analysis, and the diagnosis dimension is too limited. The faults of pumping unit are summarized in this paper. Meanwhile, the fuzzy neural network is introducedand the reasoning process is achieved by introducing a step-changing BP algorithm based on gold-segmentation is given. From the different dimensions to solve the problems of pumping unit fault it creates a diagnosis scheme referring to the characteristics of the fuzzy neural network, and the indicator diagram to comprehensively evaluate the faults. Finally, the theoretical feasibility is verified through the simulation experiments.