This paper proposes an image registration method to address the problem of high registration delay, object matching inaccuracy and even object loss caused by fast camera motion in mobile augmented reality (MAR). Firstly, TLD (Tracking Learning Detection algorithm) is improved by creating the background model for the feature points and their neighborhood to reduce the number of scanning windows and to accelerate object tracking. Then, an improved ORB (Oriented FAST and Rotated BRIEF algorithm) is presented, where Hessian matrix and gray centroid method are adopted to give the scale and direction information to the feature points, Hamming distance is used to match the feature points, and RANSAC (RANdom SAmple Consensus algorithm) further optimizes the matching results. Furthermore, the image registration algorithm based on improved TLD and improved ORB is proposed, which reduces the detection range of feature points, improves the speed of object tracking, and can quickly calculate the transformation matrix between virtual and real scenes. The experimental results show that the proposed image registration algorithm has high accuracy and good robustness in the mobile augmented reality, so it can better meet the application needs of augmented reality in the mobile terminal.