Mosaic super-resolution via sequential feature pyramid networks
M Shoeiby, A Armin, S Aliakbarian… - Proceedings of the …, 2020 - openaccess.thecvf.com
Proceedings of the IEEE/CVF Conference on Computer Vision and …, 2020•openaccess.thecvf.com
Advances in the design of multi-spectral cameras have led to great interests in a wide range
of applications, from astronomy to autonomous driving. However, such cameras inherently
suffer from a trade-off between the spatial and spectral resolution. In this paper, we propose
to address this limitation by introducing a novel method to carry out super-resolution on raw
mosaic images, multi-spectral or RGB Bayer, captured by modern real-time single-shot
mosaic sensors. To this end, we design a deep super-resolution architecture that benefits …
of applications, from astronomy to autonomous driving. However, such cameras inherently
suffer from a trade-off between the spatial and spectral resolution. In this paper, we propose
to address this limitation by introducing a novel method to carry out super-resolution on raw
mosaic images, multi-spectral or RGB Bayer, captured by modern real-time single-shot
mosaic sensors. To this end, we design a deep super-resolution architecture that benefits …
Abstract
Advances in the design of multi-spectral cameras have led to great interests in a wide range of applications, from astronomy to autonomous driving. However, such cameras inherently suffer from a trade-off between the spatial and spectral resolution. In this paper, we propose to address this limitation by introducing a novel method to carry out super-resolution on raw mosaic images, multi-spectral or RGB Bayer, captured by modern real-time single-shot mosaic sensors. To this end, we design a deep super-resolution architecture that benefits from a sequential feature pyramid along the depth of the network. This, in fact, is achieved by utilizing a convolutional LSTM (ConvLSTM) to learn the inter-dependencies between features at different receptive fields. Additionally, by investigating the effect of different attention mechanisms in our framework, we show that a ConvLSTM inspired module is able to provide superior attention in our context. Our extensive experiments and analyses evidence that our approach yields significant super-resolution quality, outperforming current state-of-the-art mosaic super-resolution methods on both Bayer and multi-spectral images. Additionally, to the best of our knowledge, our method is the first specialized method to super-resolve mosaic images, whether it be multi-spectral or Bayer.
openaccess.thecvf.com
Showing the best result for this search. See all results