Local and Nonlocal Steering Kernel Weighted Total Variation Model for Image Denoising

Abstract

To eliminate heavy noise and retain more scene details, we propose a structure-oriented total variation (TV) model based on data dependent kernel function and TV criterion for image denoising application. The innovative model introduces the weights produced from the local and nonlocal symmetry features involved in the image itself to pick more precise solutions in the TV denoising process. As a result, the proposed local and nonlocal steering kernel weighted TV model yields excellent noise suppression and structure-preserving performance. The experimental results verify the validity of the proposed model in objective quantitative indices and subjective visual appearance.

1. Introduction

Image denoising is a vital preprocessing step for image based object detection, recognition, and tracking [1,2,3,4,5,6]. Since high frequency image details are mixed with noise in most cases, most of the existing image denoising methods have difficult preserving the edge and texture information while thoroughly eliminating the noise [7,8,9,10,11,12].

Many traditional image processing methods are exploited on the basis of the local structural regularity assumption present in natural images. The rationale of denoising algorithms is to make use of the structural patterns to regularize the ill-posed restoration problem and make the texture region less blurry and the flat region smoother [13,14,15]. The gradient based total variation (TV) is a state-of-the-art method that has been proven to restore real scenes from noisy images effectively [16]. However, the TV model tends to introduce staircase effect and texture loss. To surmount the inherent defects of TV regularization, some improved TV models with structure preserving performance are presented. By combining intensity into the definition of the distance between pixels, bilateral filtering [17] clearly relieves the blurring effect of the Gaussian filter and provides detail preserving performance. In view of this, the bilateral total variation (BTV) model [18] and non-local total variation (NLTV) model [19] are successively exploited to more precisely restore the details by fusing the idea of bilateral filtering and non-local means filtering [20] with TV criterion. However, the BTV model only considers the spatial distance but ignores the neighborhood similarity in obtaining the gradient of a pixel, which leads to the derogation of structure information in the recovered image. Moreover, the nonsymmetrical structure preserving ability of the NLTV model tends to be weakened with growing noise strength, which may result from neglecting the robust local structure constraints.

Existing investigations have shown that feature descriptors using local steering kernel (LSK) are robust to noise interruption [21]. The reason lies in that LSK is exploited to solve image noise and uncertainty by estimating the local structure. Moreover, a patch containing a flat region, textural clutter, and structural part have significant differences for LSK-based descriptors. In view of this, we encode the noise corrupting image patches using the LSK method [22] to robustly recover the original image structure and remove the noise disturbance.

Inspired by the fact that the disconnected nonlocal components with spatial support provide more useful information in image restoration [23], we further combine the nonlocal self-similarity [24,25] with the local constraints of LSK to weight the respective measurements of TV, which enhances the structure preserving capability of the TV model in denoising application. In this way, our proposed local and nonlocal steering kernel weighted TV model for image denoising can robustly estimate the local structure of the image, as well as effectively remove the annoying noise. Obviously, it utilizes the redundancy of symmetrically similar patches in the corrupted image and the sensitivity of local feature description to implement the denoising task.

The rest of this paper is structured as follows. Section 2 briefly reviews the related works of this study, Section 3 introduces the presented algorithm and discusses its mechanism, Section 4 states experimental results and analysis, and Section 5 gives the conclusion of the paper.

2. Related Works

Consider a discrete noisy image

$$Y=X+V$$

(1)

where $Y$ denotes the observation, $V$ indicates the zero-mean additive white noise perturbation that is uncorrelated to the true image $X$.

2.1. Regularization Based Denoising Framework

For the image denoising problem, we need to solve the minimization problem as [26]

$$\widehat{X}=\underset{\widehat{X}}{Arg\min }\left\{{\Vert Y-\widehat{X}\Vert }_p^p+\mu \mathsf{\Upsilon }\left(\widehat{X}\right)\right\}$$

(2)

where $\widehat{X}$ is the denoised image, ${\Vert Y-\widehat{X}\Vert }_p^p$ is the data fidelity item used to retain the original image characteristics and reduce the image distortion, the scalar $\mu$ is used to properly balance the fidelity term ${\Vert Y-\widehat{X}\Vert }_p^p$ with the regularizer $\mathsf{\Upsilon }\left(\widehat{X}\right)$, a smaller $\mu$ increases the impact of fidelity item and provides stable convergence, while a larger $\mu$ tends to enhance the influence of regularization term and induces fast tracking; ${\Vert \cdot \Vert }_p^p$ denotes the $L_p\begin{array}{c}\left(1\le p\le 2\right)\end{array}$ norm of residual.

The well-known Tikhonov regularizer [27,28] is defined as

$${\mathsf{\Upsilon }}_T\left(\widehat{X}\right)={\Vert \Gamma \widehat{X}\Vert }_2^2$$

(3)

where $\Gamma$ represents the high-pass operator. It is clear that the Tikhonov regularization method tends to constrain the total energy of the image or implements spatial smoothing.

Since the noise and image textures both contain abundant high frequency components, the regularization procedure easily remove both of them indiscriminately, and the corresponding result is that the denoised images lose most of their sharp edges and details.

The popular edge preserving regularization strategy for image restoration applications is the TV method [16]. The TV criterion penalizes the $L_1$ norm of gradient amplitude that measures the total change of image and is expressed as

$${\mathsf{\Upsilon }}_{\mathrm{TV}}\left(\widehat{X}\right)={\Vert \nabla \widehat{X}\Vert }_1$$

(4)

where $\nabla$ is the gradient operator. The advantage of TV criterion lies in that it can retain edges during refactoring without seriously penalizing the steep gradients [29,30].

On the basis of TV criterion and bilateral filter, a spatially adaptive regularizer called bilateral total variation (BTV) that attempts to eliminate the staircase effect and detail the loss problem of the TV prior model is presented in [18]. The BTV regularizer is mathematically formulated as

$${\mathsf{\Upsilon }}_{\mathrm{BTV}}\left(\widehat{X}\right)=\underset{l+m\ge 0}{\underbrace{{\displaystyle \sum }_{l=-P}^P{\displaystyle \sum }_{m=0}^P}}{\alpha }^{\left|\mathrm{m}\right|+\left|l\right|}{\Vert \widehat{X}-S_x^l{S}_y^m\widehat{X}\Vert }_1$$

(5)

where operators $S_x^l$ and $S_y^m$, respectively, implement $l$ pixel horizontal translation and $m$ pixel vertical translation, so as to present multi-scale derivatives. $P$ indicates the radius of the search window. The scalar weight $\alpha \left(0<\alpha <1\right)$ is utilized to produce spatial decay for the regularization terms. Larger $\alpha$ provides a larger impact of neighbor pixels, but an exceeded $\alpha$ generally blots out the scene details, as is the case with the TV prior, and results in over-smooth effects. On the contrary, a tiny $\alpha$ sharply attenuates the spread of weight with the increase of spatial distance, which decreases the noise suppression capability and leads to slower convergence. A proper selection of $\alpha$ plays an important role in balancing noise suppression and detail preservation.

Local image structures are often repeated themselves within the image and across the image sequences. The redundant information contained in similar patches has a vital significance for solving most ill-posed image restoration problems. This is because similar patches are generally considered to be various observations of the same real scene.

To overcome the performance decline of local derivative-based prior models in noise suppression, a self-similarity-based nonlocal prior model is employed in the regularized framework, which gives the nonlocal total variation (NLTV) regularizer as [19]

$${\mathsf{\Upsilon }}_{\mathrm{NLTV}}\left(\widehat{X}\right)={\displaystyle \sum _{i\in \Omega }{\displaystyle \sum _{j\in {\mathrm{N}}_i}{W}_{NL}\left(i,j\right)}}{\Vert \widehat{X}\left(i\right)-\widehat{X}\left(j\right)\Vert }_1$$

(6)

where $i$ indicates any of the pixel in the image $\widehat{X}:\Omega \to ℝ$, $j$ represents the pixels located in the $R$ sized search window around $i$ and denoted as ${\mathrm{N}}_i$, the weight formula is represented as

$$W_{NL}\left(i,j\right)=\exp \left(-\frac{{\Vert N_i\left(u\right)-N_j\left(u\right)\Vert }_p^p}{{\sigma }^2}\right)$$

(7)

where $N_i(\cdot )$ and $N_j(\cdot )$ denote the $r$ sized square similarity patch surrounding pixel $i$ and $j$, respectively; $\sigma$ represents the filtering parameter that controls the smoothness.

2.2. Local Steering Kernel

Steering kernel regression (SKR) [21] depends on not only the position and intensity, but also the intrinsic local structure of the samples. Therefore, the size and shape of the regression kernel will significantly affect its spread and feature extraction characteristics [31,32]. The core of the SKR method is the local steering kernel (LSK) function, which estimates the local structures accurately, even in strong noise.

Let coordinate vector $x_i={\left[x_i^1,x_i^2\right]}^T$ represents the position of a certain pixel, and $\widehat{X}\left(x_i\right)$ denotes the intensity of the $x_i$ pixel. The structural representation capability of LSK mainly relies on the so-called gradient covariance matrix or steering matrix [21]. Assume there exists a $p$ pixels involved patch $N(x_i)=\{x_1,...,x_i,...x_p\}$ centered at $x_i$, the structure adaptive steering kernel representation can be modeled as

$$K\left(x_i,x_j\right)=\frac{\sqrt{\det ({\mathrm{C}}_j)}}{h^2}\cdot \exp \left(\frac{{\left(x_i-x_j\right)}^T{\mathrm{C}}_j\left(x_i-x_j\right)}{-2h^2}\right)$$

(8)

where $x_j\in N(x_i)$, the symmetric covariance matrix $C_j$ is evaluated by the spatial gradient vectors surrounding $x_j$ [33]. A good choice of $C_j$ is vital for estimating the LSK and will expand the kernel weight along the local edges; $h\left(2\le h\le 3\right)$ is a global smoothing parameter.

Let $\Omega \left(x_j\right)=\left\{x_1,\cdots ,x_j,\cdots ,x_M\right\}$ represent the positions of the $M$ adjacent pixels surrounding $x_j$. Natively, $C_j$ can be directly estimated by $G_j^T{G}_j$, in which $G_j$ is expressed as

$$G_j=\left[\begin{array}{cc}{\widehat{X}}_h(x_1)& {\widehat{X}}_v(x_1)\\ ⋮& ⋮\\ {\widehat{X}}_h(x_j)& {\widehat{X}}_v(x_j)\\ ⋮& ⋮\\ {\widehat{X}}_h(x_M)& {\widehat{X}}_v(x_M)\end{array}\right]$$

(9)

where ${\widehat{X}}_h(\cdot )$ and ${\widehat{X}}_v(\cdot )$, respectively, indicate the first-order gradients in the horizontal and vertical direction.

In order to enhance robustness and promote stability, the proposed method estimates the covariance matrix using a regularized parametric method. Based on the singular value decomposition (SVD) formula of $G_j$,

$$G_j=U_j{S}_j{V}_j=U_{j}diag{\left[s_1,s_2\right]}_j{\left[u_1,u_2\right]}_j^T$$

(10)

we can obtain the singular values $(s_1,s_2)$ and the singular vectors $(u_1,u_2)$, which are further used to calculate the stable covariance matrix

$$C_j={\left(\frac{s_1{s}_2+eps}{M}\right)}^{\varphi }\left(f{u}_1{u}_1{}^T+g{u}_2{u}_2{}^T\right)$$

(11)

where the amplification factor $\varphi$ and the constant $eps$ are set to 0.5 and ${10}^{-7}$, respectively. According to the intrinsic structure, we can regulate $f$ and $g$ to make the induced kernel be isotropic at the smooth area and uniform along the image contour ($g>f$) as

$$f\left(\beta ,\phi \right)=\frac{s_1+\gamma }{s_2+\gamma }\left(\gamma \ge 0\right)$$

(12)

$$g\left(\beta ,\phi \right)=\frac{s_2+\gamma }{s_1+\gamma }\left(\gamma \ge 0\right)$$

(13)

where $\gamma$ is a tuning parameter for controlling the kernel spread and is set to 1, which suppresses the noise, as well as decreases the detail loss; $\beta$ and $\phi$ are the eigen values of the structure tensor [34] that reflects the gradient strength along the direction of each eigenvector.

3. Local and Nonlocal Steering Kernel Weighted Total Variation Model

The local derivative-based models (such as TV and BTV) are sensitive to noise in the homogenous region, while the patch similarity-based models (such as NLTV) are not fit to deal with the noise in the cluttered texture region. In view of this, we incorporate the steering kernel [35]-based structure descriptor into the TV regularization framework and present an innovative regularizer. This local structure based regularizer can smooth out noise while preserving the details, even in very noisy circumstances. The weight function based on the local steering kernel (LSK) can be defined as

$$W_{LSK}^{l,m}(i)=\frac{K\left(x_i,{\overrightarrow{x}}_i^{l,m}\right)}{\underset{l+m\ge 0}{\underbrace{{\displaystyle {\displaystyle \sum }_{l=-P}^P}{\displaystyle {\displaystyle \sum }_{m=0}^P}}}K\left(x_i,{\overrightarrow{x}}_i^{l,m}\right)}$$

(14)

Furthermore, we consider both the nonlocal similarity and local structure properties and propose a joint local and nonlocal structural weight, which is then normalized in its neighborhood to prevent the nonuniform weight in various patches, and is characterized by

$$W_{NLSK}^{l,m}(i)=\frac{W_{NL}\left(x_i,{\overrightarrow{x}}_i^{l,m}\right)\cdot K\left(x_i,{\overrightarrow{x}}_i^{l,m}\right)}{\underset{l+m\ge 0}{\underbrace{{\displaystyle {\displaystyle \sum }_{l=-P}^P}{\displaystyle {\displaystyle \sum }_{m=0}^P}}}{W}_{NL}\left(x_i,{\overrightarrow{x}}_i^{l,m}\right)\cdot K\left(x_i,{\overrightarrow{x}}_i^{l,m}\right)}$$

(15)

where ${\overrightarrow{x}}_i^{l,m}$ is a pixel located in the patch that shifts $l$ and $m$ pixels from $x_i$ in the horizontal and vertical direction. For $x_i\in \Omega$, either of the $W_{LSK}^{l,m}(i)$ calculated by (14) or $W_{NLSK}^{l,m}(i)$ calculated by (15) can be used to form the weight matrix uniformly represented by $W^{l,m}$.

In order to compare the spread characterization of different kernels, Figure 1 shows the subjective visual representation of the steering kernel for different local structures (texture, strong edge) in the “House” image on the noisy and noiseless cases. Seeing the weight map of LSK, the shape and orientation of its footprints elongate along the edge to realize edge preservation and noise smoothing. The weight map of non-local kernel (NLK) is strewn according to the nonlocal similarity but neglects the influence of local structure. In contrast, the weight map of non-local steering kernel (NLSK) contains both neighborhood similarity and local spatial support, and assigns large weights to the locally and structurally similar pixels along with the central pixels of nonlocal similar patches. Specifically speaking, the weight map of NLSK spread closer than other kernels in the direction perpendicular to edges, which adaptively reduces the blurriness with respect to the local feature of the image. Moreover, we can easily find that all of the kernels show a rapid decline for spread characterization in the noisy condition and NLSK is still the most precise structure descriptor.

Rather than defining the weight by local intensity or nonlocal similarity, we, respectively, introduce the LSK and NLSK prior to weight for the neighboring gradient according to the nonlocal and intrinsic structure of the image itself. On the basis of TV criterion and LSK or NLSK prior, we uniformly define the noise tolerant LSKTV and NLSKTV regularizer as

$$\mathsf{\Upsilon }(\widehat{X})=\underset{l+m\ge 0}{\underbrace{{\displaystyle \sum }_{l=-P}^P{\displaystyle \sum }_{m=0}^P}}{\Vert W^{l,m}\cdot (\widehat{X}-S_x^l{S}_y^m\widehat{X})\Vert }_1$$

(16)

Combining the above presented idea, we propose an innovative structure feature-guided cost function for the denoising problem as

$$\widehat{X}=\underset{\widehat{X}}{\mathrm{ArgMin}}\left\{\underset{l+m\ge 0}{\underbrace{{\displaystyle \sum }_{l=-P}^P{\displaystyle \sum }_{m=0}^P}}{\Vert W^{l,m}\cdot {\nabla }_{l,m}\widehat{X}\Vert }_1+\frac{\mu }{2}{\Vert \widehat{X}-Y\Vert }_2^2\right\}$$

(17)

where ${\nabla }_{l,m}\widehat{X}=\widehat{X}-S_x^l{S}_y^m\widehat{X}$.

Considering the equivalent constrained problem to (17), such that $W^{l,m}\cdot {\nabla }_{l,m}\widehat{X}=d_{l,m}$, we can get the unconstrained version and solve it by using the split Bregman algorithm [36,37] given by

$$\{\begin{array}{l}({\widehat{X}}^{k+1},d_{l,m}^{k+1})=\underset{u,d}{\mathrm{ArgMin}}\{\frac{\mu }{2}{\Vert {\widehat{X}}^k-Y\Vert }_2^2+\underset{l+m\ge 0}{\underbrace{{\displaystyle {\displaystyle \sum }_{l=-P}^P}{\displaystyle {\displaystyle \sum }_{m=0}^P}}}\left[{\Vert d_{l,m}^k\Vert }_1+\frac{\lambda }{2}{\Vert d_{l,m}^k-W^{l,m}\cdot {\nabla }_{l,m}{\widehat{X}}^k-b_{l,m}^k\Vert }_2^2\right]\}\\ b_{l,m}^{k+1}{=\mathrm{b}}_{l,m}^k+(W^{l,m}\cdot {\nabla }_{l,m}{\widehat{X}}^{k+1}-d_{l,m}^{k+1})\end{array}$$

(18)

where $\lambda >0$ is a constant.

Following (18), we solve the equations using the Gauss-Seidel iteration written as

$${\widehat{X}}_{i,j}^{k+1}=\frac{\lambda }{\mu +4\lambda }\left[{\widehat{X}}_{i+1,j}^k+{\widehat{X}}_{i-1,j}^k+{\widehat{X}}_{i,j-1}^k+{\widehat{X}}_{i,j+1}^k+\underset{l+m\ge 0}{\underbrace{{\displaystyle \sum }_{l=-P}^P{\displaystyle \sum }_{m=0}^P}}\left(d_{l,m,i-1,j}^k-d_{l,m,i,j}^k-b_{l,m,i,j-1}^k+b_{l,m,i,j}^k\right)\right]+\frac{\mu }{\mu +4\lambda }{Y}_{i,j}$$

(19)

At the boundaries of the domain, one-sided finite differences are used instead of the centered finite differences. We get

$$d_{l,m}^{k+1}=shrink(W^{l,m}\cdot {\nabla }_{l,m}{\widehat{X}}^{k+1}+d_{l,m}^k,\frac{1}{\lambda })$$

(20)

The whole algorithm procedure is presented in Algorithm 1.

Algorithm 1: Proposed image denoising algorithm

Input: noisy observation $Y$.

1. Initialization:    ${\widehat{X}}^0=Y,d_{l,m}^0=b_{l,m}^0=0$

2. Iteration:  While${\Vert {\widehat{X}}^{k+1}-{\widehat{X}}^k\Vert }_2^2/{\Vert {\widehat{X}}^k\Vert }_2^2<\epsilon$do  Calculate the normalized weight $W_k^{l,m}$ by Equation (15)  Update ${\widehat{X}}^{k+1}$ according to Equation (19)  $d_{l,m}^{k+1}=shrink(W^{l,m}\cdot {\nabla }_{l,m}{\widehat{X}}^{k+1}+d_{l,m}^k,1/\lambda )$  $b_{l,m}^{k+1}=shrink(W^{l,m}\cdot {\nabla }_{l,m}{\widehat{X}}^{k+1}+b_{l,m}^k,1/\lambda )$  $k=k+1$  end While Output: desired image $X$

4. Experimental Results and Analysis

In this section, we will contrast the performance of the presented method with the previous variational denoising methods on artificially degraded samples that are generated by adding Gaussian noise with zero mean and standard deviation of 10, 25, and 40 to the $512\times 512$ sized standard bitmap (BMP) format test images. In the following experiments, the parameter sensitivity will be firstly discussed for finding a tune strategy to obtain balanced and higher performance. It is worthy to note that the parameters selected according to the advice of original documents are employed to pursue the best performance. In the following experiments, the stopping criteria $\epsilon$ of the proposed local steering kernel total variation (LSKTV) and non-local steering kernel total variation (NLSKTV) algorithm is set to $1\times {10}^{-3}$ to guarantee stable convergence.

In order to facilitate quantitative comparison, Peak Signal-to-Noise Ratio (PSNR) and Structure Similarity index (SSIM) [38] are employed to objectively assess the performance of various denoising methods. The PSNR is defined as

$$\mathrm{PSNR}\left(x,y\right)=10\cdot \log \frac{{255}^2}{1/(H\cdot W){\displaystyle {\sum }_{i=0}^H{\displaystyle {\sum }_{j=0}^W{\left(x\left(i,j\right)-y\left(i,j\right)\right)}^2}}}$$

(21)

where $H$ and $W$ indicate the height and width of the image, respectively.

In addition, the SSIM index can be calculated by

$$SSIM\left(x,y\right)=\frac{\left(2{\mu }_x{\mu }_y+c_1\right)\left(2{\sigma }_{xy}+c_2\right)}{\left({\mu }_x^2{\mu }_y^2+c_1\right)\left({\sigma }_x^2+{\sigma }_y^2+c_2\right)}$$

(22)

where ${\mu }_x$ and ${\mu }_y$ stand for the mean value of $x$ and $y$, ${\sigma }_x$ and ${\sigma }_y$ indicate the variance of $x$ and $y$, ${\sigma }_{xy}$ denotes the covariance of $x$ and $y$. The constants $c_1$ and $c_2$ are, respectively, set to 6.5025 and 58.6225 to stabilize the division with the weak denominator.

4.1. Parameter Sensitivity Analysis

As for the proposed NLSK weight, appropriate window size is crucial for precisely measuring the local and nonlocal structure, which will directly affect the final restoration effect and subsequent parameter adjustment. In view of this, we select different sizes of similar windows and search windows, and then implement the proposed denoising method upon the same simulated image on three noise strengths. For ease of comparison, we uniformly symbolize the search window size of NLK, LSK, and NLSK weight with $R=2l+1=2m+1$. The PSNR curves of the iterative denoising process for different parameter configurations are shown in Figure 2. Through comparison, we can find that the size of similar window $r$ and search window $R$ should be aptly raised with the increase of noise level to effectively utilize redundant information in the neighborhood for high precision and stable convergence. However, extremely large similar windows and search windows may remove the image details and cause image over-smoothness effect, which instead results in performance degradation.

The learning rate $\mu$ is the key impact factor to control the convergence property of the proposed method [39,40]. As can be seen from the Figure 3, the increase of $\mu$ promotes the convergence rate, but if $\mu$ is too large it will interrupt the convergence process. The reason lies in that a larger $\mu$ is beneficial for accelerating the process to reach maximum PSNR, but excessive $\mu$ will produce non-convergence issues, which instead depresses the PSNR. As for higher noise level, the learning rate $\mu$ should be set larger to ensure the algorithm can effectively remove noise and improve convergence speed.

Considering the smoothing parameter $h$ in the steering kernel, we will further find the optimal selection of $h$ is beneficial to achieve higher restoration precision. As can be seen from Figure 4, $h$ should be raised with the increase of noise strength on the premise of stable convergence. A larger $h$ is helpful to smooth out noise and promote PSNR, but if $h$ is too large it will lead to over-smoothness effect and destroy the intrinsic structure of images, which instead depresses the PSNR.

4.2. Performance Comparisons

Figure 5 shows the PSNR for the denoised “Zebra” image of TV, BTV, TGV [41], NLTV, and the proposed algorithms on variant noise strength. Obviously, the proposed NLSKTV and LSKTV method achieve an expanded leading advantage in PSNR with the increase of noise strength $\sigma$ when compared to TV, BTV, TGV, and NLTV method. As for the highest noise strength $\sigma =40$, NLSKTV can still reach 27.56 dB (the input PSNR is only 16.08 dB at that time), which is 2.56 dB, 1.64 dB, 1.72 dB, 1.29 dB, and 0.09 dB higher than TV, BTV, TGV, NLTV, and LSKTV, respectively. This phenomenon indicates that the proposed NLSKTV and LSKTV deal with the strong noise more effectively.

In order to intuitively observe the visual effect, we implement the denoising with different algorithms. We can see clearly from Figure 6 that TV and TGV methods generate over-smoothed details and produce staircase effect in varying degrees. Furthermore, there are partial noise residuals and texture distortions in the results of BTV and NLTV method. Obviously, the noise suppression ability of the proposed LSKTV and NLSKTV are superior to others. It is worth noting that the presented NLSKTV method preserves more details for its special local structure and nonlocal similarity balanced weight.

To quantitatively evaluate the precision of the abovementioned denoising methods, we carry out comparisons of PSNR and SSIM on different test images respectively corrupted by additive Gaussian noise with $\sigma =10,25,40$. The results summarized in

Table 1. PSNR and SSIM of denoised images for different denoising methods.

Method	Zebra	Boat	Barbara	Dollar	Lighthouse	House
	PSNR/SSIM	PSNR/SSIM	PSNR/SSIM	PSNR/SSIM	PSNR/SSIM	PSNR/SSIM
σ = 10
TV	31.84/0.8053	31.45/0.8204	29.90/0.7998	28.69/0.8235	30.58/0.8287	35.23/0.8567
BTV	33.13/0.8825	32.37/0.8598	30.96/0.8690	29.73/0.8816	31.26/0.8633	36.54/0.9303
TGV	33.36/0.8771	32.81/0.8688	31.42/0.8797	29.54/0.8750	31.54/0.8661	36.77/0.9466
NLTV	34.06/0.8952	32.85/0.8692	32.26/0.9091	30.60/0.9296	31.84/0.8743	37.24/0.9392
LSKTV	33.73/0.8891	32.87/0.8701	31.91/0.8983	29.88/0.9021	32.04/0.8729	37.39/0.9347
NLSKTV	34.18/0.8955	32.97/0.8745	32.60/0.9139	30.80/0.9329	32.20/0.8852	37.76/0.9394
σ = 25
TV	27.30/0.6553	27.30/0.6810	24.98/0.6568	22.72/0.6566	25.89/0.6721	31.15/0.7583
BTV	28.35/0.8143	28.10/0.7392	25.64/0.7079	23.52/0.7236	26.30/0.6982	32.46/0.8842
TGV	28.45/0.7724	28.24/0.7399	25.71/0.6876	23.17/0.6974	26.32/0.6788	32.63/0.8512
NLTV	28.70/0.8288	28.19/0.7477	26.02/0.7368	23.57/0.7474	26.25/0.7133	32.64/0.8705
LSKTV	29.72/0.8345	28.73/0.7543	26.72/0.7642	24.15/0.7767	27.71/0.7297	33.47/0.8783
NLSKTV	30.21/0.8495	29.31/0.7737	27.37/0.8043	24.73/0.8455	28.03/0.7505	34.05/0.8938
σ =40
TV	25.00/0.5862	25.45/0.6064	23.55/0.5869	20.54/0.5633	23.90/0.5824	29.06/0.7064
BTV	25.92/0.7640	26.20/0.6682	23.77/0.6238	21.05/0.6072	24.04/0.6024	30.44/0.8540
TGV	25.84/0.7340	26.19/0.6670	24.00/0.5995	20.80/0.5753	24.01/0.5683	30.34/0.8232
NLTV	26.27/0.7405	26.26/0.6653	24.52/0.6756	21.75/0.7101	24.50/0.6220	29.99/0.7834
LSKTV	27.47/0.7735	26.75/0.6801	24.53/0.6626	21.90/0.6805	25.67/0.6476	31.18/0.8255
NLSKTV	27.56/0.8092	26.83/0.6937	24.74/0.6980	22.60/0.7420	25.74/0.6604	31.31/0.8498

indicate that the PSNR and SSIM of the proposed LSKTV is better than that of TV, BTV, total generalized variation (TGV), and NLTV in most cases; furthermore, its superiority becomes increasingly significant with the increasing noise strength. Note that NLSKTV constantly achieves the best performance by fusing additional nonlocal self-similarity to describe the image structure on the basis of LSKTV. In summary, the proposed method effectively preserves the image details and removes the noise simultaneously.

4.3. Image Deblurring Application

In this section, we will validate the deblurring performance of the proposed method with the existing TV, BTV, TGV, and NLTV methods on standard test image “cameraman”. Following the blurring degradation model in [42], the test image is blurred by a 9 × 9 Gaussian kernel with standard deviation 5 and 10 and corrupted with Gaussian noise with standard deviation 10. The PSNR and SSIM results of different methods are shown in

Table 2. PSNR and SSIM of the deblurring results for different methods.

Method	TV	BTV	TGV	NLTV	LSKTV	NLSKTV
	PSNR/SSIM	PSNR/SSIM	PSNR/SSIM	PSNR/SSIM	PSNR/SSIM	PSNR/SSIM
${\sigma }_h=5$	24.68/0.7574	24.67/0.7698	24.74/0.7666	24.88/0.7719	24.98/0.7887	25.14/0.7923
${\sigma }_h=10$	23.67/0.6385	23.75/0.6827	23.91/0.6693	24.01/0.6986	24.37/0.7506	24.53/0.7550

. It is apparent that the LSKTV and NLSKTV achieve higher PSNR and SSIM than TV, BTV, TGV, and NLTV.

Figure 7 shows the visual effects of different deblurring methods. It is obvious that other competitive methods, such as TV, BTV, TGV, and NLTV, tend to generate staircase effects in flat regions and zigzag effect on edges, which reduces the structural similarity significantly. In contrast, the proposed LSKTV and NLSKTV preserve sharp edges and recover high frequency details, as well as yield visually pleasant results. In summary, the quantitative and qualitative results indicate that the proposed local and nonlocal steering kernel-weighted total variation regularizer is a good candidate for image deblurring application.

4.4. Runtime Comparisons

In order to compare the computation complexity of different methods, we perform the simulation on various sized test images and list the corresponding runtime in

Table 3. Runtime(s) of different methods for various sized images.

Image Size	TV	BTV	TGV	NLTV	LSKTV	NLSKTV
128 × 128	0.001	0.033	0.032	0.118	0.485	0.753
256 × 256	0.051	0.112	0.089	0.423	1.356	1.975
512 × 512	0.175	0.451	0.338	2.024	4.122	6.771

. These results are implemented in MATLAB (R2017a) on a computer with Intel (R) Core(TM) [email protected] CPU. As can be seen from Table 3, the proposed LSKTV and NLSKTV demand more computational load than others to calculate the steering kernel weight, which is the price for their outstanding denoising performance. As the computing of the steering kernel has larger parallelism, the LSKTV and NLSKTV will be further accelerated by the specially designed parallel hardware, such as Graphics Processing Unit (GPU) and Field Programmable Gate Array (FPGA).

5. Conclusions

Noise suppression and detail preservation are generally difficult to balance for most of the existing TV model-based image denoising methods. In view of this, we proposed an innovative structure-oriented TV model that incorporates the local structural regularity of local steering kernel (LSK) as well as nonlocal self-similarity of nonlocal kernel (NLK) to achieve a more precise denoising effect. Experimental results demonstrated that the presented LSKTV and NLSKTV model is conducive to simultaneously restoring the image structures and eliminating the noise disturbance, which is validated by the remarkable objective performance assessment and the favorable subjective visual effect.