The object of this work is to design an adequate regularization for the problem of recovering missing Fourier coefficients, particularly in some nonstandard situations where low-frequency coefficients are lost. In the framework of nonlocal regularization, we propose a technique for building an original patchwise similarity measure that is adapted to the missing spectrum. Then, a simple nonlocal quadratic energy is minimized. By construction, the similarity criterion is invariant under the corruption process so that the distance between two patches of the corrupted image is almost exactly equal to that computed on the clean image. We illustrate our method with experiments which show its efficiency, in terms of both speed and quality of the results, with respect to other common approaches. We show that the method is practical on synthetic examples which are built upon models of inverse scattering problems, synthetic aperture mirrors for spatial imaging, or medical imaging problems.