In this paper, a novel algorithm for the automatic online improvement of the extrinsic camera parameters of a stereo image pair is introduced. To this end, the well-known dense stereo matching method PatchMatch stereo (PM) is extended for the pixelwise estimation of a discrepancy between the expected epipolar line and the actual correspondence. The availability of an initial guess of the camera parameters is assumed. Next, the estimated disparity map is filtered for highly stable and accurate correspondences that cover preferably the complete image. For this reason, we extend a quality estimation method adapted to Semi-Global Matching (SGM) derived disparity maps for general disparity maps. Finally, the set of stable and accurate correspondences from the disparity map is used for the estimation of the extrinsic camera parameters by means of the five-point algorithm in a RANSAC (random sample consensus) framework. Our algorithm can estimate optimized disparity maps and is able to adjust for errors in the relative camera pose. It can even correct epipolar errors of tens of pixels in highresolution images. We demonstrate that the proposed algorithm allows for robust and accurate estimation of the extrinsic camera parameters on datasets that provide weaklycalibrated image pairs.