For the traditional Lukas-Kanade (LK) method, it is hard to capture the ball center in the Ball-Plate system and the accuracy of tracking is not exact. A way to improve the LK method based on the Hough transform is proposed. Traditional LK method chooses corner where the change of gray scale is obvious in the image. Usually the cornet is distributed at the edge of circle and cannot go deep into the center of the circle. By using the improved LK method to select the corner, the two-dimensional XY coordinate system is transformed into 3D ABR coordinate system. The center is determined by accumulate number and accumulate weight. The ball is tracked by this center point's gray scale. The results show that the improved optical flow method is better than the traditional LK optical flow and histogram method.