This paper presents a novel formulation for object tracking. We model the second-order statistics of image regions and perform covariance matching under the variational level set framework. Specifically, covariance matrix is adopted as a visual object representation for partial differential equation (PDE) based contour tracking. Log-Euclidean calculus is used as a covariance distance metric instead of Euclidean distance which is unsuitable for measuring the similarities between covariance matrices, because the matrices typically lie on a non-Euclidean manifold. A novel image energy functional is formulated by minimizing the distance metrics between the candidate object region and a given template, and maximizing the ones between the background region and the template. The corresponding gradient flow is then derived according to a variational approach, enabling PDE-based visual tracking. Experiments on synthetic and real video sequences prove the validity of the proposed method.