We give necessary conditions for geometric and polynomial convergence rates of randomwalk- type Markov chains to stationarity in terms of existence of exponential and polynomial moments of the invariant distribution and the Markov transition kernel. These results complement the use of Foster-Lyapunov drift conditions for establishing geometric and polynomial ergodicity. For polynomially ergodic Markov chains, the results allow us to derive exact rates of convergence and exact relations between the moments of the invariant distribution and the Markov transition kernel. In an application to Markov chain Monte Carlo we derive tight rates of convergence for symmetric random walk Metropolis.