This paper solves the regulation optimal control problem for nonlinear systems based on the adaptive dynamic programinf(ADP) and neural networks(NNs). Firstly, we propose the regulation nonlinear system; afterwards, the cost function and its Hamilton-Jacobi-Bellman(HJB) function were given; furthermore, an actor-critic frame structure was put forward to get the optimal control, and the neural network algorithms were utilized to approximate the performance index and the optimal control ofactor-critic structures respectively. The NN weights are estimated by gradient algorithms, so that the optimal control is obtained. Simultaneously, the stability of the whole system and the convergences of actor-critic NN weights are proved based on the Lyapunov theory. Finally, the simulation results are provided to verify the effectiveness of the proposed methods.