This letter introduces a new reference frame to improve the performance of motion estimation and compensation in video coding, based on a video stabilization technique. The proposed method synthesizes the new reference frame from the previous frame in a way that the new reference and current frames have the same camera orientations. The overhead data for each frame to transmit from an encoder to a decoder is only three rotational angles along the x, y, and z axes. Since the new reference and current frames have the same camera orientations, the proposed method significantly improves the performance of motion estimation and compensation for video sequences having dynamic camera motion by up to 0.98 dB with negligible overhead data.