With the recent attention towards audio processing in the time-frequency domain we increasingly encounter the problem of missing data within that representation. In this paper we present an approach that allows us to recover missing values in the time-frequency domain of audio signals. The presented approach is able to deal with real-world polyphonic signals by operating seamlessly even in the presence of complex acoustic mixtures. We demonstrate that this approach outperforms generic missing data approaches, and we present a variety of situations that highlight its utility.