A Transformer-Unet Generative Adversarial Network for the Super-Resolution Reconstruction of DEMs

Abstract

A new model called the Transformer-Unet Generative Adversarial Network (TUGAN) is proposed for super-resolution reconstruction of digital elevation models (DEMs). Digital elevation models are used in many fields, including environmental science, geology and agriculture. The proposed model uses a self-similarity Transformer (SSTrans) as the generator and U-Net as the discriminator. SSTrans, a model that we previously proposed, can yield good reconstruction results in structurally complex areas but has little advantage when the surface is simple and smooth because too many additional details have been added to the data. To resolve this issue, we propose the novel TUGAN model, where U-Net is capable of multilayer jump connections, which enables the discriminator to consider both global and local information when making judgments. The experiments show that TUGAN achieves state-of-the-art results for all types of terrain details.

1. Introduction

The digital elevation model (DEM) is a kind of ground model that denotes the height of the ground in the form of a series of ordered digital arrays and numerically simulates the terrain through limited topographic elevation data from the Earth’s surface. DEMs have been widely used in the fields of natural disaster prevention, surveying, mapping, hydrology, meteorology, geological geomorphology, etc. [1,2,3]. DEMs with high resolution contain more information and can provide accurate descriptions. For example, urban river flood models constructed using high-resolution DEMs can improve the accuracy of flood predictions [4]; thus, the quality of DEMs plays an important role in their investigation and application.

The resolution of a DEM depends on the method used to generate the data, and many of the existing methods [5,6] for acquiring DEM data (e.g., LiDAR, InSAR interferometry, photogrammetry and existing topographic maps) are either too costly, not widely enough used in the region, vulnerable to weather or unable to provide the most up-to-date data. In addition, many super-resolution implementations in digital image field have proved to be effective. Therefore, super-resolution reconstruction of DEMs to obtain high-precision and high-resolution DEM data has become one of the most effective and feasible methods for improving the resolution of DEMs.

Recently, with the rapid advancement of deep learning, methods based on deep learning have swiftly developed in the DEM super-resolution domain. These methods [7,8,9,10] use artificial neural networks to learn the complex connections between low-resolution and high-resolution DEMs, and then use the connections to generate high-quality super-resolution DEMs. This approach overcomes the shortcomings of traditional interpolation methods [11,12,13,14,15], learns from a large amount of training data and generates high-resolution DEMs even in regions with complex terrains or incomplete data.

Image super-resolution reconstruction algorithms with deep learning [16] are categorized into single-image super-resolution (SISR) reconstruction methods [17] and reference-based image super-resolution (RefSR) reconstruction methods [18,19]. RefSR generates output images with high resolution by utilizing high-resolution reference images combined with input images with low resolution. In the super-resolution reconstruction process, a high-resolution reference image serves as a guide, supplementing the low-resolution input image with missing extra information while providing a more accurate representation of the details [20,21]. In the realm of DEM super-resolution reconstruction, algorithms based on deep learning primarily employ the SISR method for reconstruction [7,8,9,10]. Previously, we proposed SSTrans, which makes innovative use of terrain self-similarity to reconstruct images via super-resolution [22], largely solving the problem of difficult-to-obtain reference images. SSTrans can achieve good results in nonsmooth areas; however, the effect is not significant in smooth areas.

The remainder of this article is organized as follows. Section 2 introduces the background related to our research, which includes the basic model SSTrans and the discriminator U-Net. Section 3 introduces the model structure, including the overall framework of TUGAN, the architecture of the U-Net discriminator network, the introduction of spectral normalization and the introduction of the loss function. Section 4 analyses the outcomes of the experiments, including an evaluation of the reconstructed outcomes, a comparison with the other super-resolution algorithms and a comparison of the quality of reconstruction of the terrain attributes of the DEM. Section 5 concludes the paper.

2. Background

The Transformer network conceptualized by Vaswani et al. [23] is a neural network utilizing the self-attention mechanism, which was originally employed in natural language processing applications for tasks such as machine translation. In traditional neural network models (such as CNNs), the information passes through layers of neurons, and each neuron is connected only to the neurons above and below it. The position of each word or clause is an important parameter in this calculation. The self-attention mechanism strengthens the connections between units, which in turn improves the exact value.

Previously, to find a solution to the lack of reference image issue, we proposed SSTrans, which utilizes the terrain self-similarity feature to construct a DEM reference dataset; this approach is capable of obtaining a deep relationship between the DEM data with low resolution and the reference data and solves the problem of not having a publicly available DEM reference dataset for RefSR work. Using topographic self-similarity, constructed reference data can provide effective detailed information for low-resolution DEMs during the reconstruction process.

However, the reconstruction results for SSTrans are better in nonsmooth areas (for example, areas with large elevation differences) but not in smooth areas. This is because self-similarity makes the model focus more on the special and complex parts of the terrain, while the global features of the smooth terrain are more important than the local content is; therefore, the model cannot take advantage of the data. For the purpose of solving this problem, we propose a new GAN structure in which we use SSTrans as a generator and employ U-Net as a discriminator, with the aim of concurrently addressing both the global information and local conditions.

In addition, Schonfeld et al. [24] introduced a GAN architecture based on a U-Net [25,26,27] discriminator. U-Net GAN can supply comprehensive feedback, encompassing both global and local aspects to the generator through an encoding–decoding process, and the discriminator used in this method can assist the generator in producing more realistic images both globally and locally.

In this paper, we integrate ideas from the U-Net GAN to construct a newer, more efficient super-resolution network model for DEMS called TUGAN, mainly by using U-Net as the discriminator. On this basis, we can provide both global and local feedback to the generator through the encoding–decoding process, helping the generator to produce realistic images.

3. Model Structure

We create a new DEM super-resolution reconstruction method with Transformer and a generative adversarial network on the basis of SSTrans. In this new method, we use SSTrans as the generator and U-Net as the discriminator to construct the TUGAN to better feed the generator with the reconstruction information of each elevation point.

As shown in Figure 1, the DEM data with low-resolution $I_{LR}$ and the corresponding reference DEM data $I_{Ref}$ are input into the generator, and the output of the generator, $G(I_{LR},I_{Ref})$, is obtained after the backbone network operation, residual feature extraction and feature fusion in the generator. $I_{HR}$ is the real high-resolution DEM data, then $G(I_{LR},I_{Ref})$ and $I_{HR}$ are inputted into the discriminator $D_{enc}$, which is the encoder part of the U-Net discriminator. The encoder obtains the global information by downsampling step by step and categorizes each data point. The encoder obtains the global information by stepwise downsampling and classifies each data point, and then, $D_{dec}$, the decoder of U-Net, performs the stepwise upsampling operation to obtain the output data with the same resolution as the input resolution. As a result, we can attain accurate positioning between the elevation points. The ability of U-Net to accurately segment detailed information can be further enhanced by adding jump connections between the encoder module and the decoder module with matching resolution data. By combining the encoder module and the decoder module, the discriminator can acquire the ability to distinguish between authentic data and synthetic data and transfer the loss information to the generator so that the generator can improve the local and global constructions to better deceive the discriminator.

3.1. Generator Network Architecture

We use SSTrans [22] as a generator for TUGAN, which is a Transformer structural model with significant generative power whose structure is shown in Figure 2. SSTrans utilizes the self-similarity of the terrain to construct the corresponding reference data for each DEM data. $I_{Ref}\uparrow \downarrow$ refers to the reference after double triple downsampling and double triple upsampling operations and $I_{LR}\uparrow$ refers to the LR after double triple upsampling, which is included in data preprocessing step. Details in $I_{LR}$ and $I_{Ref}$ are extracted in redisual feature extraction. It employs an attention mechanism to identify the most pertinent details for super-resolution reconstruction. Finally, features $V^{\prime }$ and F, along with maximum confidence weight matrix W for $I_{LR}$ and $I_{Ref}$, generate the final $I_{SR}$ after feature synthesis.

3.2. Discriminator Network Architecture

We use U-Net structure as our discriminator. Here are two reasons for it: (1) The reconstruction quality of each reconstruction method is seriously degraded in regions with complex terrain surfaces. Therefore, it is important to enhance the performance of both the generator and the discriminator, where U-Net comes in; (2) The U-Net can also offer both global and detailed understanding on the terrain, providing adequate error feedback to the generator on each elevation point.

As shown in Figure 3, the network uses a convolutional kernel of size $4\times 4$ for downsampling operations to perform the encoding process on the DEM data. A $2\times 2$ upsampling and a $3\times 3$ convolution operation are then used to obtain the data, and there is a consistent resolution with the input data to achieve accurate localization.

Since the U-Net structure contributes to an increase in training instability, to enhance the stability, Lipschitz constraints must be added to limit dramatic changes in the function, and the TUGAN method adds a regular spectral normalization expression after the convolutional layers of the discriminator to better maintain the continuity of Lipschitz. In a GAN, for discriminator D, if any x and y in the image space satisfy Equation (1),

$$\parallel D\left(x\right)-D\left(y\right)\parallel \le K\parallel x-y\parallel ,$$

(1)

then the discriminator is said to be k-Lipschitz continuous, where $\parallel ·\parallel$ is the $L_2$ regularization and k is the Lipschitz constant. The 1-Lipschitz continuity is achieved by dividing the parameter matrix W of the convolutional layer of the discriminator D by the spectral parameter. Spectral normalization is achieved by first decomposing the parameter matrix W by a singular value to achieve the maximum singular value and then dividing the matrix W through the utilization of the maximum singular value of W to achieve 1-Lipschitz continuity. The formula for spectral normalization is given by

$$W_{SN}={\displaystyle \frac{W}{\sigma \left(W\right)}},$$

(2)

where $\sigma \left(W\right)$ is the spectral parameter of matrix W.

3.3. Loss Function

Many GAN model super-resolution algorithms use the objective function proposed by the original GAN, which may result in unstable training, which leads to a disappearance of gradients or gradient explosion. In terms of the discriminator, TUGAN draws on the proposed spectral normalization method by adding spectrally normalized GANs (SNGANs) [28] to the discriminator network to strengthen the stability of the training; and with regard to the loss function, TUGAN borrows from the Wasserstein GAN with a gradient penalty (WGAN-GP) [29] and uses the method of adding a gradient penalty to enhance the stability of the training process.

Before introducing WGAN-GP, its predecessor Wasserstein GAN(WGAN) [30] should first be introduced. WGAN is also known as a GAN and uses the Wasserstein distance; most studies have proven that the use of the Wasserstein distance instead of $KL$ dispersion and $JS$ dispersion as similarity measures can resolve the gradient disappearance and pattern collapse problems during training. Calculating the Wasserstein distance requires implementing Lipschitz continuity so that the neural network $f\left(x\right)$ fixed elements $x_1$ and $x_2$ are satisfied:

$$|f\left(x_1\right)-f\left(x_2\right)| \le k|x_1-x_2|.$$

(3)

After organizing, the distribution distance formula can be formulated as follows:

$$L=D\left(real\right)-D\left(G\right(x\left)\right).$$

(4)

The discriminator’s aim D is to distinguish the real data from the simulated data and increase the distance between these two data points; thus, the discriminator’s loss can be defined as follows:

$$D\left(loss\right)=D\left(G\right(x\left)\right)-D\left(real\right).$$

(5)

The generator’s aim is to make the simulated data close to the real data; thus, the distance L needs to be minimized, and the generator’s loss can be defined as follows:

$$G\left(loss\right)=-D\left(G\right(x\left)\right).$$

(6)

WGAN utilizes weight trimming to maintain training stability but is relatively rigid; when the discriminator is a multilayer network, it is difficult to set an appropriate trimming threshold, so WGAN is prone to gradient disappearance or gradient explosion problems. Therefore, WGAN-GP introduces the method of gradient penalization to replace weight cropping, and the formula for gradient penalization is expressed as follows:

$$Norm=grad\left(D(X_{inter},\left[X_{inter}\right])\right),$$

(7)

$$gradient\_penaltys=MSE(Norm-k),$$

(8)

where $X_{inter}$ is the sampling of x in the joint distribution space, $Norm$ denotes the gradient of x corresponding to D in the joint distribution space, and $gradent\_penaltys$ is the mean-square error between $Norm$ and k. Usually, k is set to one. The loss of the discriminator after the addition of the gradient penalty can be defined as follows:

$$D\left(loss\right)=D\left(G\right(x\left)\right)-D\left(real\right)+\lambda gradient\_penaltys,$$

(9)

where $\lambda$ is the parameter utilized to adjust the intensity of the gradient penalty.

The loss function of the TUGAN generator consists of two parts, adversarial loss and reconstruction loss, which can be expressed as follows:

$${\mathcal{L}}_G={\lambda }_{rec}{\mathcal{L}}_{rec}+{\lambda }_{adv}{\mathcal{L}}_{adv},$$

(10)

where $L_{rec}$ uses the $L_1$ loss denoted as

$${\mathcal{L}}_{rec}={\displaystyle \frac{1}{WH}}\sum _{x=1}^W\sum _{y=1}^H{\parallel G\left(I_{x,y}^{LR}\right)-I_{x,y}^{HR}\parallel }_1.$$

(11)

The adversarial loss $L_{adv}$ of the generator can be denoted as

$${\mathcal{L}}_{adv}=-D\left(G\left(I^{LR}\right)\right).$$

(12)

The loss function of the TUGAN discriminator includes three parts: the encoding loss, the decoding loss and the gradient penalty, which can be represented as follows:

$${\mathcal{L}}_D={\mathcal{L}}_{D_{enc}}+{\mathcal{L}}_{D_{dec}}+\lambda gradient\_penaltys.$$

(13)

where $L_{D_{enc}}$ is the loss of the encoder module of the discriminator denoted as

$${\mathcal{L}}_{D_{enc}}=D(G\left(I^{LR}\right)-D\left(I^{HR}\right).$$

(14)

$L_{D_{dec}}$ is the loss of the discriminator decoder module denoted as

$${\mathcal{L}}_{D_{dec}}={\displaystyle \frac{1}{WH}}\sum _{x=1}^W\sum _{y=1}^{H}D\left(G\left(I_{x,y}^{LR}\right)\right)-D\left(I_{x,y}^{HR}\right)\parallel$$

(15)

$gradient\_penaltys$ is shown in Equation (8), which is $\lambda$ and usually takes the value of 10.

3.4. Data Normalization

Consistent with the SSTrans method, we choose four regions in China, namely the Inner Mongolia Plateau, Qinling Mountains, Tarim Basin, and North China Plain, and automatically match a reference DEM for each high-resolution DEM. Due to the large range of elevation difference values in the DEM data, normalizing the elevation values to the range of $[-1,1]$ is essential, as shown below:

$$x_{normalization}={\displaystyle \frac{2(x-x_{min})}{x_{max}-x_{min}}}-1,$$

(16)

where x denotes the elevation value in the DEM data, $x_{min}$ expresses the minimum elevation value in the DEM data, $x_{max}$ expresses the maximum elevation value in the DEM data and $x_{normalization}$ expresses the result after normalization.

The hyperparameters used in training were the Adam optimizer [31] with ${\beta }_1=0.9$, ${\beta }_2=0.99$, and $\epsilon =1\times {10}^{-8}$; the learning rate was set to $1\times {10}^{-4}$; the reconstruction loss ${\lambda }_{rec}=1$; and the antagonistic loss ${\lambda }_{adv}=1\times {10}^{-3}$. The network with only the anti-loss function is trained for 2 epochs, and then the network with all losses is trained for 50 epochs.

4. Experimental Results and Analysis

4.1. Data Descriptions

The ASTER GDEM V3 dataset provides the experimental data needed for this study with a data resolution of 30 m. As shown in Figure 4, we selected four representative regions in mainland China, namely the Inner Mongolia Plateau, the Qinling Mountains, the Tarim Basin and North China Plain, as the geographical dataset for assessing the performance of TUGAN. These regions have diverse topographic features with significant differences in terrain and elevation. Based on the characteristics of these regions, we established a DEM dataset on the basis of self-similarity. A total of $40,000$ DEM data pairs were included, of which $30,000$ pairs were used as our training set and the other pairs were used as our validation set. Each region stores $10,000$ data pairs, of which 7500 pairs were used for training and 2500 pairs for confirmation. After super-resolution, the dataset was improved from 30 m to 10 m.

The reconstructed DEM super-resolution results were assessed using the root mean square error $RMSE$ and mean absolute error $MAE$.

$$RMSE=\sqrt{{\displaystyle \frac{1}{N}}\sum _{i=1}^N{(I_{HR}-I_{SR})}^2},$$

(17)

$$MAE={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}|I_{HR}-I_{SR}|.$$

(18)

The peak signal-to-noise ratio $PSNR$ and structural similarity index measure $SSIM$ were also evaluated in this article. A larger PSNR indicates that the gap between the original image and the reconstructed image is smaller.

$$SSIM(x,y)={\displaystyle \frac{(2{\mu }_x{\mu }_y+c_1)(2{\sigma }_{xy}+c_2)}{({\mu }_x^2+{\mu }_y^2+c_1)({\sigma }_x^2+{\sigma }_y^2+c_2)}},$$

(19)

$$PSNR=10{log}_{10}\left({\displaystyle \frac{\Delta S}{MSE}}\right).$$

(20)

The SSIM is an index used to evaluate the similarity between reconstructed data and original data, and the calculation results are very close to the human visual judgment standard; the higher the SSIM, the more similar the reconstructed data to the original data and the better the reconstruction quality.

SSIM is definited as in Equation (19), where x denotes the primary DEM data, y denotes the reconstructed DEM data, ${\mu }_x$ expresses the average grayscale of x, ${\mu }_y$ expresses the average grayscale of y, ${\sigma }_{xy}$ expresses the covariance of x and y, and ${\sigma }_x$ and ${\sigma }_y$ express the standard deviations of x and y, respectively. Moreover, parameters $c_1$ and $c_2$ are two constants set according to the empirical values.

The evaluation index of the terrain attributes of DEM data employs the mean absolute error (MAE), which is represented by $E_{tp}$ as follows:

$$E_{tp}={\displaystyle \frac{1}{N}}\sum _{i=1}^N\left|t_i-t_i^{\prime }\right|$$

(21)

where $t_i$ and $t_i^{\prime }$ represents the terrain attribute of the original high-resolution DEM data and the reconstructed DEM data, respectively.

4.2. Results for the Four Test Regions

For the purpose of verifying the validity of the TUGAN method proposed in our article, $900\times 900$-sized DEM data from the test sets of the above four regions are selected. The maximum height differences among the four regions are shown in

Table 1. The maximum elevation difference of the four regions.

Regions	Maximum Elevation (m)	Minimal Elevation (m)	Maximum Elevation Difference (m)
Region 1	2206	1260	946
Region 2	2528	190	2338
Region 3	1109	906	203
Region 4	129	5	124

, and the evaluation indices of the TUGAN reconstruction results in the four regions are shown in

Table 2. The TUGAN reconstruction results in the four regions.

Regions	MAE (m)	RMSE (m)	PSNR (dB)	SSIM (%)
Region 1	3.08	4.24	35.57	99.21%
Region 2	10.32	13.52	25.51	99.32%
Region 3	1.29	1.69	43.57	97.21%
Region 4	1.25	1.64	43.84	95.99%

. The visualization results of the TUGAN reconstruction are shown in Figure 5, where Figure 5b1–b4 shows the results of the TUGAN reconstruction of the four regions, and the experimental results are analyzed as follows.

Region 1, situated in the Inner Mongolia Plateau, has a high relief and simple and smooth surfaces, and the maximum elevation difference is 946 m. As shown in Table 2, the mean absolute error $MAE$ is $3.08$, the $RMSE$ is $4.24$, and the average $MAE$ per 1000 m elevation difference is $3.25$. The PSNR reaches $35.57$ and the structural similarity $SSIM$ reaches $99.21\%$, which is also shown in Figure 5b1, where the reconstructed DEM of the region is visually consistent with the original one. The error between the original DEM and the reconstructed DEM in this region is small, and the reconstruction result is stable.

Region 2 is located in the Qinling Mountains and has large topographic changes and many mountain ranges; however, the surface characteristics are similar to those of Region 1, both of which are relatively simple and smooth. The maximum elevation difference in Region 2 is 2338 m, which is the largest among the four regions. As shown in Table 2, the $MAE$ value of this region is $10.32$, the $RMSE$ value is $13.52$, the average $MAE$ per 1000 m elevation difference is $4.41$ and the PSNR value reaches $25.51$. Compared with Region 1, the SSIM value of this region is further improved to $99.32\%$, which is close to the same as human visual judgement standard, as shown in Figure 5b2. Due to the large elevation difference in this region, the errors in all the indicators except for the SSIM greatly increase, which proves that the errors in the DEM super-resolution reconstruction strongly correlate with the elevation difference.

Region 3 is situated in the Tarim Basin and features little change in terrain, with a smaller maximum elevation difference of 203 m and a more complex topographic surface. As shown in Table 2, the $MAE$ value of this region is $1.29$, the RMSE value is $1.69$, the average MAE per 1000 m of elevation difference is $6.35$, the PSNR value reaches $43.57$ and the SSIM value reaches $97.21\%$. As shown in Figure 5b3, due to the complexity of the topographic surface of the region, the visual effect results in more small pixels, and some subtle differences can be observed after local zooming. However, as the average MAE per 1000 m of elevation difference increases, the SSIM, an index for assessing similarity, decreases.

Region 4 is a plain located in North China, where the maximum elevation difference is 124 m and most locations are below 50 m. In terms of the complexity of the terrain surface, the reconstruction task is more difficult. In addition, the MAE of this region is $1.25$, the RMSE is $1.64$, the average reconstruction time per 1000 m is $1.5$ m and the average reconstruction time per 1000 m is $2.5$ m, which is greater than that of Region 3. As shown in Table 2, the MAE is $1.25$, the RMSE is $1.64$, the mean absolute error per 1000 m is $10.08$, the PSNR is $43.84$ and the SSIM is $95.99\%$. As shown in Figure 5b4, the errors in this region can be observed via local magnification in comparison with the original DEM. Due to the smaller height difference and the increased number of complex surface in Region 4 compared to Region 3, the quality of the reconstruction is reduced, and the mean absolute error per 1000 m is significantly greater.

4.3. Comparison with Other SR Methods

In this article, the TUGAN method is compared with the bicubic method [32], super-resolution convolutional neural network (SRCNN) [8], super-resolution generative adversarial network (SRGAN) [9], enhanced deep super-resolution network (EDSR) [7] and SSTrans [22] methods.

A comparison of each method for the four regions is shown in

Table 3. The comparison of each method in the four regions.

Regions	Methods	MAE (m)	RMSE (m)	PSNR (dB)	SSIM (%)
Region 1	Bicubic	6.12	7.30	30.47	97.31%
	SRGAN	6.17	8.15	29.90	96.09%
	SRCNN	5.02	6.26	31.91	98.38%
	EDSR	3.80	5.06	35.31	98.60%
	SSTRans	4.44	5.65	33.06	98.93%
	TUGAN	3.08	4.24	35.57	99.21%
Region 2	Bicubic	15.24	19.28	21.39	98.21%
	SRGAN	17.79	23.10	20.86	97.54%
	SRCNN	14.86	18.20	22.02	98.85%
	EDSR	11.40	15.34	27.38	99.32%
	SSTRans	12.96	16.52	23.77	99.04%
	TUGAN	10.32	13.52	25.51	99.32%
Region 3	Bicubic	2.46	3.18	38.08	86.32%
	SRGAN	2.10	2.78	39.22	87.71%
	SRCNN	2.22	2.87	38.99	89.37%
	EDSR	1.98	3.24	39.91	89.80%
	SSTRans	1.55	2.03	41.99	96.13%
	TUGAN	1.29	1.69	43.57	97.21%
Region 4	Bicubic	2.53	3.32	37.07	74.76%
	SRGAN	2.48	3.29	37.79	77.08%
	SRCNN	2.40	3.17	38.10	78.35%
	EDSR	2.31	3.08	38.36	79.22%
	SSTRans	1.63	2.14	41.51	94.11%
	TUGAN	1.25	1.64	43.84	95.99%

and Figure 6, where the first column of Figure 6 is the visualization image of the primary DEM and the visualization image of the local magnified region of the primary DEMs is in the second column. The subsequent visualization reconstruction outcomes of each method in four regions correspond to each method.

Region 1 and Region 2 have large elevation differences and simple and smooth surfaces, and the EDSR method has slight advantages over the SSTrans method in both regions. In this article, the evaluation indicators of TUGAN improved by the SSTrans method exceeded those of the EDSR method, indicating significant improvement. In Region 1, the MAE of TUGAN was lower by $30.6\%$ than that of SSTrans, the RMSE was lower by $24.96\%$ than that of SSTrans, the PSNR was greater by $7.59\%$ than that of SSTrans and the SSIM was $0.28\%$ greater than that of SSTrans.

In Region 2, the TUGAN model still achieved good reconstruction results. The method reduces the MAE by $20.37\%$ compared to that of SSTrans; the RMSE by $18.16\%$ compared to that of SSTrans; the PSNR by $7.32\%$ compared to that of SSTrans; and the SSIM is $0.28\%$ higher than that of SSTrans, which is the same as the SSIM value of the EDSR. As shown in Figure 6, visual analysis revealed that Regions 1 and 2 have high SSIM values and high similarity, making it challenging to subjectively assess the advantages and disadvantages of the methods through visual perception.

The elevation difference between Region 3 and Region 4 is relatively small, and the terrain surface is more complex, requiring more detailed information for reconstruction. Therefore, the reconstruction quality of each method decreases to some extent compared to that of Region 1 and Region 2. The TUGAN method proposed in this article, an improved method based on SSTrans, has achieved further improvements in reconstruction evaluation indicators.

In Region 3, the MAE of TUGAN was lower by $16.7\%$ than that of SSTrans, the RMSE was lower by $16.75\%$ than that of SSTrans, the PSNR was greater by $3.76\%$ than that of SSTrans and the SSIM was greater by $1.08\%$ than that of SSTrans. In Region 3, the reconstruction quality of TUGAN was much better than that of the other methods, with both methods achieving an SSIM above $90\%$, while the other methods did not exceed $90\%$.

The terrain surface of Region 4 is more complex than that of Region 3, and the reconstruction quality of each method decreased. However, SSTrans and TUGAN maintained the advantages obtained on the basis of reference data, with TUGAN having a slightly higher quality than SSTrans. The MAE of TUGAN was $23.31\%$ lower than that of SSTrans, the RMSE was $23.36\%$ lower than that of SSTrans, the PSNR was $5.61\%$ higher than that of SSTrans and the SSIM was $1.88\%$ higher than that of EDSR. In regions with more complex topographic surfaces, such as Region 4, TUGAN achieved greater improvement than Region 3.

In general, in Regions 1 and 2, the reconstruction effect of the SSTrans was slightly worse than that of the EDSR proposed in this article. After improving the SSTrans, the reconstruction quality of the TUGAN model exceeded that of the EDSR model in these two regions. In Regions 3 and 4, the TUGAN maintained the advantages of SSTrans in these two regions and further improved the quality of reconstruction.

To provide further elucidation of the research findings, Figure 7 presents a visual depiction of the reconstruction of North China. The figure illustrates the absolute value of the difference between each pixel point in the HR image and the DEM image after super-resolution processing. When the difference is minimal, the corresponding pixel color appears as a grey shade. Conversely, when the difference is significant, the pixel color takes on a blue tint. This chromatic variation not only aids in the identification of the precision of each region in the reconstruction process but also reflects the effectiveness of the model in responding to diverse terrain characteristics. In terms of visual assessment, other models exhibit a marked deficiency compared to our model with respect to the aforementioned difference.

4.4. Terrain Attribute Analysis for DEM

In addition to an analysis of the reconstruction results of the DEM elevation value, this article also selects slope and aspect to analyze the reconstruction effect of the model.

4.4.1. Slope Reconstruction Analysis

The slope attribute refers to the terrain slope at each location. It can be used to describe the steepness and inclination direction of the terrain. The slope is an important indicator in many geographic analyses and applications, usually expressed in percentage or angle. The slope calculation formula is

$$Slope=57.29578\times arctan\sqrt{{({\displaystyle \frac{dz}{dx}})}^2+({\displaystyle \frac{dz}{dy}})}$$

(22)

where $57.29578$ is the result of ${\displaystyle \frac{180}{\pi }}$ and the scanning window is shown in Figure 8. ${\displaystyle \frac{dz}{dx}}$ and ${\displaystyle \frac{dz}{dy}}$ can be represented as follows:

$${\displaystyle \frac{dz}{dx}}={\displaystyle \frac{\frac{4\times (c+2f+i)}{wght1}-\frac{4\times (a+2d+g)}{wght2}}{8\times xCellsize}}$$

(23)

$${\displaystyle \frac{dz}{dy}}={\displaystyle \frac{\frac{4\times (g+2h+i)}{wght3}-\frac{4\times (a+2b+c)}{wght4}}{8\times yCellsize}}$$

(24)

where $wght1$, $wght2$, $wght3$, $wght4$ represent the number of valid cells. For example, if the values cf and i are all valid, then $wght1=(1+2+1)=4$. This same calculation applies to $wght2$, $wght3$ and $wght4$. Variables $xCellsize$ and $yCellsize$ represent the resolution of the DEM in the x-axis direction and the y-axis direction, respectively.

The evaluation of the slope reconstruction results of TUGAN and other methods in four regions is shown in

Table 4. Quantitative evaluation of slope reconstruction results of TUGAN and other methods.

Regions	Bicubic	SRGAN	SRCNN	EDSR	SSTrans	TUGAN
Region 1	3.30	4.07	3.05	3.02	1.96	1.85
Region 2	5.28	7.66	5.17	4.66	4.04	3.61
Region 3	2.50	2.13	2.11	2.19	1.12	1.01
Region 4	2.93	2.42	2.52	2.48	1.64	0.92

. The slope value range is [0, 90], and the calculation method of the evaluation index is shown in Equation (21). Figure 9 shows the visual comparison of the slope reconstruction results of TUGAN and other methods. This article selects the slope reconstruction results of TUGAN in Region 4 for visual comparison with SRGAN, EDSR and SSTrans.

In comparison with alternative methods, TUGAN demonstrated superior performance across all four regions. In Region 1, the reconstruction index of TUGAN was found to be $5.61\%$ lower than that of SSTrans. Similarly, in Region 2, the reconstruction index of TUGAN was observed to be $10.64\%$ lower than that of SSTrans. In Region 3, the reconstruction index of TUGAN was found to be $9.82\%$ lower than that of SSTrans. Similarly, in Region 4, the reconstruction index of TUGAN was observed to be $43.9\%$ lower than that of SSTrans. The visualization results for Region 4 are presented in Figure 9. It is evident that the reconstruction outcomes of TUGAN restored more intricate details in comparison to those of SSTrans, SRGAN and EDSR. The reconstruction results of TUGAN were further enhanced in terms of both indicators and visual effects.

4.4.2. Aspect Reconstruction Analysis

The aspect attribute refers to the direction of terrain slope. The aspect attribute can be used to describe the direction and orientation of terrain. It is an important indicator in many geographic analyses and applications. It is usually expressed in degrees or azimuth. The calculation formula for the aspect is

$$Aspect=57.29578\times \mathrm{atan}2({\displaystyle \frac{dz}{dx}},-{\displaystyle \frac{dz}{dy}})$$

(25)

where the scanning window is shown in Figure 8. Then, ${\displaystyle \frac{dz}{dx}}$ and ${\displaystyle \frac{dz}{dy}}$ can be calculated as follows:

$${\displaystyle \frac{dz}{dx}}={\displaystyle \frac{\frac{4\times (c+2f+i)}{wght1}-\frac{4\times (a+2d+g)}{wght2}}{8}}$$

(26)

$${\displaystyle \frac{dz}{dy}}={\displaystyle \frac{\frac{4\times (g+2h+i)}{wght3}-\frac{4\times (a+2b+c)}{wght4}}{8}}$$

(27)

where $wght1$, $wght2$, $wght3$, $wght4$ are consistent with those indicated in Equations (23) and (24).

Table 5. Quantitative evaluation of aspect reconstruction results of TUGAN and other methods.

Regions	Bicubic	SRGAN	SRCNN	EDSR	SSTrans	TUGAN
Region 1	68.11	75.39	63.97	62.58	39.78	37.51
Region 2	29.60	42.39	28.71	25.37	22.17	20.36
Region 3	84.41	86.05	79.25	78.13	33.29	33.15
Region 4	86.99	87.07	83.74	81.83	33.75	30.87

presents a quantitative evaluation of the aspect reconstruction results of TUGAN and various methods in four regions. The range of aspect values is [0, 360], and the calculation method of the evaluation index is illustrated in Equation (21). Figure 10 depicts a visual comparison of the aspect reconstruction results of TUGAN and various methods. This article selects the aspect reconstruction results of TUGAN in Region 4 for visual comparison with SRGAN, EDSR and SSTrans methods.

In Region 1, TUGAN’s reconstruction index is $5.7\%$ lower than SSTrans; in Region 2, TUGAN’s reconstruction index is $8.16\%$ lower than SSTrans; in Region 3, TUGAN’s reconstruction index is $0.42\%$ lower than SSTrans; in Region 4, TUGAN’s reconstruction index is $8.53\%$ lower than SSTrans. In comparison to alternative methods, the reconstruction results of TUGAN are considerably superior. The visualization results for Region 4 are shown in Figure 10. In this region, TUGAN exhibits high-quality, detailed information that is more accurate than that generated by SSTrans, SRGAN and EDSR methods.

5. Future Work

The TUGAN proposed in this article generated high-quality reconstruction results, but there is still some room for further research, of which the outlook is as follows.

This article intended to conduct a study in different topographic regions in China, where only publicly available datasets with 30 m resolution can be downloaded. The method proposed in this article was shown to perform well on the 30 m resolution dataset, but was not validated on the higher-resolution, for example, 10 m dataset. Therefore, future research will try to deepen the cooperation with researchers in the field of geography to obtain higher resolution data in China and even in the world to validate the effectiveness of TUGAN in high-resolution DEM data.

In addition, in super-resolution reconstruction, the larger the scale factor, the worse the reconstruction quality. In this article, the experiments were carried out with a magnification scale of four times, and in future work, larger scale reconstruction experiments will be attempted in order to obtain higher-resolution DEM data.

6. Conclusions

In conclusion, this article proposes a novel DEM super-resolution reconstruction method called TUGAN, which inherits the advantages of SSTrans and U-Net GAN to enhance the resolution of DEMs. TUGAN uses self-similarity and transformer-based features to construct high-resolution DEMs on the basis of low-resolution inputs, which overcomes the challenges posed by smooth terrain surfaces.

The results of our experiments prove the effectiveness of TUGAN in the super-resolution reconstruction of DEMs, especially in regions with varying terrain complexities and elevation differences. TUGAN outperforms several existing super-resolution methods, including Bicubic, SRGAN, SRCNN, EDSR and its base method, SSTrans, with regard to diverse evaluation metrics such as the MAE, RMSE, PSNR and SSIM. The improvements in reconstruction quality are particularly noticeable in regions with complex terrains.

The TUGAN model is capable of obtaining more accurate and detailed terrain information at a reduced cost due to its superior super-resolution reconstruction effect. The TUGAN model can be employed to conduct a more comprehensive analysis of terrain features, such as assisting geological departments in the development of more precise water flow models, which in turn can facilitate more accurate predictions of flood occurrences.

Future research directions may involve further optimization of the TUGAN model and its application to other geographical regions and datasets. Additionally, exploring other effective models for super-resolution reconstruction might prove to be valuable for future endeavors in this field.