This paper introduces a new variational Gaussian filtering approach for estimating the state of a nonlinear dynamic system. We first assume that the predictive distribution of the state is Gaussian and derive an iterative method for updating the state posterior in the natural parameter space through KullbackLeibler divergence minimization. The obtained update rule is the same as that of the conjugate-computation variational inference technique in Bayesian learning. The derivation here is simpler and more insightful. We then impose a Wishart prior on the inverse of the state prediction covariance to take into account the impact of approximating the state predictive distribution using a Gaussian density on the state posterior estimation. The prediction covariance is identified jointly with the state using variational inference and the established state posterior update rule to achieve the desired Gaussian filtering. Simulation study examines the performance of the proposed filtering framework in target tracking based on bearing and range measurements.