Multimodal remote sensing images registration ensures the consistency of the spatial positions for different images. It can provide accurate geographic information and supports the fusion of multisource data for geospatial analyses and applications. The rigid registration method shows high performance in dealing with large-scale deformation, but it is difficult to achieve high-precision image registration. In contrast, nonrigid registration method is suitable for processing local differences, but cannot effectively deal with large-scale deformation differences. Therefore, the combination of rigid and nonrigid registration methods becomes a necessary strategy to address such issues. In this article, we propose a novel ADRNet method for multimodal remote sensing image registration. The proposed ADRNet method contains three main modules: affine registration module, deformable registration module, and spatial transformer module that integrates the affine and deformable transformation parameters (TPs) to obtain the final aligned images.Meanwhile, we design a new feature enhancement module (FE) and an attention module with dilated convolutions that have different dilation rates, which are used to alleviate the limitations imposed by receptive fields in the convolution operation. Moreover, we propose a specific symmetric loss (SL) function to optimize the whole network from the perspective of inverse consistency. To assess the efficiency and performance of the network, we extend the experimental data, ranging from cross-modal images in a conventional viewpoint to cross-modal images in a remote sensing viewpoint. The experimental results show that our method exhibits excellent performance for the images with different viewpoints and deformation scales. The relevant code will be released at: https://github.com/Ahuer-Lei/ADRNet .