$\delta ARD$ loss for low-contrast medical image segmentation

Loss functions used in medical image segmentation tasks can be divided into four categories as follows:

(i) Distribution-based loss

Distribution-based loss functions aim to minimize the difference between the real probability distribution and the predicted probability distribution. The most typical distribution-based loss function is cross-entropy (CE), which is prone to slow progress in the iterative process in medical image datasets with class-imbalanced problems and may not be optimized to the best. Hence, some loss functions were proposed based on CE loss, such as TopK loss [7] and Focal loss [8],

$$\begin{equation} Loss_{\textrm{CE}} = \sum\limits_\Omega^{i = 1,j = 1} \left(u_{i,j}\textrm{log}\left(v_{i,j}\right)+\left(1-u_{i,j}\right)\textrm{log}\left(\left(1-v_{i,j}\right)\right)\right) \end{equation} \tag{ 1 }$$

where $I, J$ denotes the set of all the points on the horizontal and vertical dimensions, $i, j$ denote the points in the set $I, J$, respectively. The GT pixel is denoted by $u_{i,j}$, and the predicted pixels as $v_{i,j}$. Ω denotes the set of all points contained in the whole image.

(ii) Region-based loss

Typical region-based loss functions are dice loss [9], ioU loss and tversky loss [2], etc. Due to the lack of attention on the contour, region-based loss functions could not perform well on those medical images that contain ROIs with rough contours (e.g. figure 1(b)),

$$\begin{equation} Loss_{\textrm{Dice}} = 1-\frac {\sum\limits_\Omega^{i = 1,j = 1} 2\mid u _{i,j} \cap v _{i,j}\mid} {\sum\limits_\Omega^{i = 1,j = 1}\mid u _{i,j}\mid +\sum\limits_\Omega^{i = 1,j = 1} \mid v _{i,j}\mid} \end{equation} \tag{ 2 }$$

(iii) BD-based loss

BD-based loss functions compute the difference between the predictions and the GT in the shape-aware and the length-aware BD space. These include BD loss [4] and HD loss. Karimi and Septimiu [6] proposed HD loss and proved that hausdorff distance could be approximated by GT and distance transformation of prediction segmentation, avoiding unstable training. However, only considering BD is not conducive to accurate segmentation [5]. Thus, AC loss combines length and region area together and helps CNN achieve better segmentation performance on medical image datasets,

$$\begin{equation} Loss_{\textrm{AC}} = Length+\lambda \cdot Region \end{equation} \tag{ 3 }$$

$$\begin{equation} Loss_{\textrm{BD}} = \theta \cdot Loss_{\textrm{Dice}}+\left(1-\theta\right) \cdot Loss_{\textrm{Boundary}} \end{equation} \tag{ 4 }$$

where λ and θ are two hyperparameters,

$$\begin{equation} Loss_{\textrm{HD}} = \frac{1}{N}\sum\limits_C^{c = 1}\sum\limits_N^{i = 1} \left(s_{i}^c-g_{i}^c\right)\cdot\left(d_{G_{i}^c}^2+d_{S_{i}^c}^2\right) \end{equation} \tag{ 5 }$$

where $d_{G_{i}^c}^2$ and $d_{S_{i}^c}^2$ are the distance transformation of GT and segmentation.

(iv) Compound-based loss

By combining different loss functions in some way, the compound-based loss functions could obtain more accurate segmentation results [2, 10]. However, the parameters introduced may have decreased efficiency.

CNNs, such as AlexNet [11], UNet [12], UNet ++ [13], Dense-UNet [14] and SE-net [15], extract hierarchical structure and features with excellent performance, and have been widely used in segmentation tasks. In the field of medical image analysis, Ronnenberg et al [12] proposed a convolutional network for biomedical image segmentation, UNet, which can be trained end-to-end from very few images, has been widely used and improved. Rundo et al [16] proposed USE-Net for prostate region segmentation in multi-institutional MRI datasets, and the SE module's adaptive feature recalibration provides excellent cross-dataset generalization. Protonotarios et al [17] proposed a few-shot U-Net deep learning model for segmentation of lung cancer lesions in PET/CT imaging and continuously adjusted the weight of the model online according to user feedback to improve detection and classification accuracy. Zhuang et al [18] proposed a residual-extended-attention gate-unet (RDAUNet) that combines expansion residuals and attention gates into UNet architectures. UNet and its variants have proven to be the most efficient and commonly used medical image segmentation method.

In addition, to improve CNNs' lack of the ability to model long-range interactions [19, 20], Transformer [21] was introduced into the field of computer vision. Subsequently in medical image segmentation tasks, and achieved successful results. Valanarasu et al [22] describe the Medical Transformer (MedT), which uses gated axial attention and performs best on small medical image datasets without pre-training. Shen et al [23] proposed a dilated transformer (DT) utilizing residual axial attention to further improve the performance of transformer-based approaches. Chen et al [24] proposed that TransUNet, Transformers can be used as a powerful encoder for medical image segmentation tasks, combined with U-Net to enhance finer details by recovering local spatial information.

The proposed δARD loss function uses the BD and region information to accommodate the low-contrast data. As shown in equation (6), $Loss_{\delta ARDL}$ consists of three calculation components. One contour-based loss function and one region-based loss function.

$$\begin{equation} Loss_{\delta ARD} = \alpha \cdot \delta ARD +\beta \cdot \delta L+R, \end{equation} \tag{ 6 }$$

where α and β are two hyperparameters that can be tuned for the specific dataset and the based segmentation model to achieve the best performance; δ denotes the difference between the prediction and the GT; L and R denote the length and region area of the ROI. More descriptions of the three components of δARD will be given in the following sections 3.1.1 and 3.1.2.

3.1.1. BD correlation (δARD and δ L):

BD information is vital for subsequent image-based disease analysis and diagnosis. Gomez et al [25]. have obtained outstanding segmentation performance by taking the candidate contour with the largest ARD value as the ideal result, which shows that ARD has a certain effect on deciding the true contour in medical images. At the same time, length also plays an important role in the cardiac data in AC loss. Hence, we introduce ARD and length to the loss function to enhance the network's sensitivity to true lesion contours.

$$\begin{equation} ARD = \int_{J} \Delta G \cdot \hat{r} \mathrm dx \end{equation} \tag{ 7 }$$

where J is the set of the lesion edges, $\Delta G$ is the edge derivation of the image, and $\hat{r}$ is a radial vector based on the center of the lesion area,

$$\begin{equation} L_u = \int_{J} \mid \delta u\mid \mathrm ds , \end{equation} \tag{ 8 }$$

where u represents GT value.

However, we need to find the BD set J of the lesion first, and then find ARD for each point within the BD, which is very time-consuming for high-resolution medical images. Therefore, we simplify this process by solving the ARD and length of the entire image. The following takes the GT image as an example to find its ARD and length.

In order to simplify the calculation, we approximate the result of GT image dislocation subtraction as the BD of the lesion area and approximate the result of BD dislocation subtraction as the edge derivation,

$$\begin{equation} B_u\left(I-1,J-1\right) = u\left(1:I,1:J\right)-u\left(0:I-1,0:J-1\right), \end{equation} \tag{ 9 }$$

where I and J represent the set of points in the horizontal and vertical dimensions, respectively. $B_u(I-1, J-1)$ denotes the BD value of the simplified solution,

$$\begin{equation} \Delta G_u\left(I-2,J-2\right) = B_u\left(1:I-1,1:J-1\right)-B_u\left(0:I-2,0:J-2\right), \end{equation} \tag{ 10 }$$

The radial vector is simplified to calculate the difference between each point and the center.

$$\begin{equation} r_u\left(i,j\right) = d\left(i,j\right)\cdot u - \overline{d\cdot u}, \end{equation} \tag{ 11 }$$

where $d(i,j)$ denotes the position of (i, j), i and j represent points in the set I and J, respectively. $\overline{d\cdot u}$ denotes the average of $d\cdot u$, that is, a central point of all lesion areas on the GT,

$$\begin{equation} ARD_u = \frac{\Delta G_u\left(0,0\right)\cdot r_u\left(0,0\right)+\Delta G_u\left(0,1\right)\cdot r_u\left(0,1\right)+{\cdots}+\Delta G_u\left(I,J\right)\cdot r_u\left(I,J\right)}{I\cdot J} , \end{equation} \tag{ 12 }$$

$$\begin{equation} L_u = \sqrt{\mid B_u\left(0,0\right)^2 \mid+\epsilon}+\sqrt{\mid B_u\left(0,1\right)^2 \mid+\epsilon}+{\cdots}+\sqrt{\mid B_u\left(I,J\right)^2 \mid+\epsilon}, \end{equation} \tag{ 13 }$$

where ε is the introduced error term and ε > 0, the GT pixel is denoted by u, and the predicted pixels as v,

$$\begin{equation} \delta ARD = \sqrt{\mid ARD_v-\gamma ARD_u \mid^2} , \end{equation} \tag{ 14 }$$

$$\begin{equation} \delta L = \sqrt{\mid L_v-\lambda L_u \mid^2} , \end{equation} \tag{ 15 }$$

where $0\lt\lambda, \gamma,\alpha, \beta \lt1$.

3.1.2. Region correlation loss(R):

In the introduction, we proved the importance of regional information in loss from the paper [4] and experimental results. When there is regional information, the localization effect of the region is significantly enhanced. Therefore, it is necessary to introduce regional information,

$$\begin{equation} R = \int_{I,J}\left(\left(c_{\mathrm{in}}-v\right)^2-\left(c_{\mathrm{out}}-v\right)^2\right)u\mathrm dx , \end{equation} \tag{ 16 }$$

According to equation (16), R is divided into two parts $R_{\mathrm{in}}$ and $R_{\mathrm{out}}$ for calculation,

$$\begin{equation} R = R_{\mathrm{in}}+R_{\mathrm{out}}, \end{equation} \tag{ 17 }$$

$$\begin{equation} R_{\mathrm{in}} = \frac{\mid u\left(0,0\right) \left(c_{\mathrm{in}}-v\left(0,0\right)\right)^2+{\cdots}+u\left(I,J\right) \left(c_{\mathrm{in}}-v\left(I,J\right)\right)^2\mid}{I\cdot J}, \end{equation} \tag{ 18 }$$

$$\begin{equation} R_{\mathrm{out}} = \frac{\mid (1-u(0,0))(c_{\mathrm{out}}-v(0,0))^2+{\cdots}+(1-u(I,J))(c_{\mathrm{out}}-v(I,J)^2 \mid}{I\cdot J} , \end{equation} \tag{ 19 }$$

where $c_{\mathrm{in}}$ and $c_{\mathrm{out}}$ represent the inside area and outside area of the GT region, respectively.

For evaluating the proposed loss function, we choose UNet [12] as the fundamental segmentation framework. Figure 3 illustrates that UNet has an encoder-decoder-based architecture. Each encoder module is responsible for extracting image features. It contains two $3 \times 3$ convolutions layers, followed by a rectified linear unit and a $2 \times 2$ max pooling layer; while each decoder module is responsible for upsampling. It consists of a $2 \times 2$ up-convolution layer. Furthermore, UNet uses skip connection to combine multi-level feature maps and achieve good performance on medical image segmentation tasks.

We evaluate the proposed δARD loss on one private and four public medical image datasets by five-fold cross-validation. Specifically, the private breast ultrasound image dataset (PBUSI) was collected from two ultrasound equipment, i.e. GE LOGIQ E9 and PHILIPS EPIQ 5. It consists of 878 breast ultrasound images each of which contains a tumor with varying size and shape. Two experienced radiologists participated in labeling ROI; BUSIS dataset [1] and Busi dataset [26] are two public breast ultrasound image datasets. BUSIS consists of 562 breast ultrasound images, each of which contains one tumor; while Busi consists of 647 images that contain multiple tumor types; MoNuSeg dataset [27] and GLAnd Segmentation (GLAS) dataset [28] are two public microscopic image datasets, among which MoNuSeg dataset contains images across multiple organs and patients, including 30 training data with 22 000 nuclear BD annotations and 14 testing data with 7000 nuclear. This dataset was created by downloading H&E stained tissue images captured at 40x magnification from TCGA archive; GLAS Segmentation consists of 165 images derived from 16 H&E stained histological sections of stage T3 or T42 colorectal adenocarcinoma. Each section belongs to a different patient, and sections were processed in the laboratory on different occasions. The dataset exhibits high inter-subject variability in both stain distribution and tissue architecture. The pathologist delineated the BD of each individual glandular object on that visual field.

In this paper, we design three experiments to verify the effect of the proposed loss on the segmentation of lesions in five medical image datasets, i.e. PBUSI, BUSIS, Busi, MoNuSeg and GLAS datasets.

First, to verify the robustness of the proposed loss function, we compare it with the most related loss functions, i.e. AC loss, BD loss, HD loss and the most commonly used loss functions, i.e. CE loss and Dice loss. All the above loss functions were evaluated using UNet as the base segmentation network on the five medical image datasets.

Second, considering the performance of the proposed loss could be affected by the hyperparameters, including α and β, we conducted related experiments to find the best choice for α and β which can help UNet obtain the best segmentation performance on PBUSI dataset. Based on experience, we set 10 pairs of values for α and β. Specifically, when β is 1, we set α to be 0.5, 1, 1.1, 1.2, 1.5 and 2; and when α is 1, we set β to be 1.1, 1.2, 1.5 and 2.

Third, to further verify the effectiveness of the proposed loss function, we conducted a comparative analysis of state-of-the-art (SOTA) deep learning segmentation methods using the CE loss function or our proposed loss function in conjunction. By doing so, we aimed to explore the efficacy of our proposed loss function compared to the currently used CE loss function. SOTA methods i.e. UNet++ [13], RDAUNet [18], Attention UNet [29], Ghost UNet [30], MedT [22] and Trans UNet [24]. We tested this experiment on three typical medical image datasets, i.e. PBUSI, MoNuSeg and GLAS datasets.

Furthermore, we used PyTorch to implement the whole framework and conducted the related experiments on one NVIDIA GeForce GTX 1080ti GPU. During the training period, the learning rate, batch size and epochs were set to 0.0001, 10 and 200, respectively; and all input images were resized to 256 × 256. Moreover, we adopted three typical evaluation metrics that include true positive ratio (TPR), false positive ratio (FPR), Jaccard index (JI), Dice similarity coefficient (DSC), F1 score (F1), Precision (PC) and Specificity (SP).

In experiment 1, to verify the robustness of the proposed loss function, we compare it with the most related loss functions, i.e. AC loss, BD loss, HD loss and the most commonly used loss functions, i.e. CE loss and Dice loss. All the above loss functions were evaluated using UNet as the base segmentation network on the five medical image datasets.

Table 1 displays the experimental outcomes for typical loss functions such as CE loss, Dice loss, AC loss, HD loss, BD+Dice loss, and the newly proposed δARD loss presented in this paper. It is segmented into five experiments based on the datasets. The results of the segmentation of PBUSI dataset are demonstrated in lines 2–7 of table 1. This experiment signifies that δARD loss yields the highest values of TPR, FPR, JI, DSC, F1, and SP values. δARD loss outperformed the widely used CE loss with an uplift in TPR, FPR, JI, DSC, F1, and SP of 3.0%, 11.4%, 4.4%, 3.6%, 4.0%, and 0.2%, correspondingly. Comparing it to Dice loss, δARD loss shows enhancement in TPR, FPR, JI, DSC, F1, SP, and PC by 1.5%, 1.0%, 1.6%, 1.6%, 1.7%, 0.1%, and 1.1%, respectively. Additionally, when compared with similar BD+Dice loss, δARD loss achieved an improvement in TPR, FPR, JI, DSC, and F1by 1.7%, 0.1%, 0.2%, 1.3%, and 0.6%, respectively. In comparison with similar AC loss, δARD loss showed improvement in TPR, FPR, JI, DSC, F1, SP, and PC by 0.7%, 6.7%, 3.2%, 2.5%, 2.6%, 0.3%, and 4.4%, correspondingly. δARD loss shows the highest accuracy in the segmentation of the small breast ultrasound PBUSI dataset. Furthermore, the proposed δARD loss function improves the accuracy by concentrating on shape information. The same conclusion is observed in the quantization results of BUSIS, Busi, MoNuSeg, and GLAS datasets. Compared to other loss functions, δARD loss achieves optimal values in most indicators, significantly enhancing the overall segmentation results for low-contrast medical images.

From the above experimental results, we come to the following conclusion. First, we can observe that δARD has the highest TPR, JI, DSC, F1, SP, and PC values and the lowest FPR values on five low-contrast medical image datasets (slightly inferior to BD+Dice loss on GLAS dataset), which indicates that δARD has the best robustness and could help UNet achieve the best segmentation performance on medical image datasets. Second, compared with the most related AC loss, our study highlights that incorporating ARD into the UNet architecture significantly enhances its capacity to capture more intricate BD information for low-contrast medical images. Additionally, we demonstrate that our proposed loss function outperforms well-established typical loss functions.

As shown in the first line of figure 4, BD loss, HD loss and Dice loss identified multiple areas due to low-contrast, and changes in lesion size, shape, and location. In contrast, the proposed δARD loss is more effective at enforcing BD and regional characteristics for low-contrast medical images by including ARD, length and area to enhance shape consistency. The result using the proposed loss in the second line of figure 4 is superior to other loss functions in accurately identifying and refining shape contours. Line 3 of figure 4 shows that δARD loss performs accurate segmentation of multiple small breast tumors. UNet identifies two small ROI without any false positives. AC loss, BD loss, and Dice loss models show more error areas, while CE loss and HD loss are worse than δARD loss. UNet was unable to obtain accurate and complete tumors when using the five loss functions shown in line 4 of figure 4. In contrast, interference can be successfully resisted using δARD loss. Therefore, this shows that δARD provides significant information, which helps UNet identify the true ROI region and BD for low-contrast medical images. It is worth noting that during the segmentation process, the false positive area increases, as shown in line 5 of figure 4. However, compared to other methods, the δARD model is closest to GT.

Zoom In Zoom Out Reset image size

Table 2 shows the results obtained by UNet using the proposed loss δARD under different hyperparameters that include α and β on PBUSI dataset. It can be found that the optimal performance is obtained when α = 1.1 and β = 1. In addition, when α is increased from 0.5 to 1.2, TPR, JI and DSC values show a continuous increase. It indicates that within a certain upper limit, the larger portion of ARD value considered into the loss function leads to better segmentation performance, but it becomes worse when the portion of α and β is larger than 1.1 (i.e. the loss function considered too much on the BD, and too less on length).

We selected three datasets, PBUSI, GLAS, and MoNuSeg, for the experiment due to their complexity and features such as tumor segmentation, multiple nuclei, and two types of region segmentation on the GLAS dataset. Figure 5 provides a visual representation of seven methods for segmenting ultrasound breast tumors using CE loss or δARD loss on the three datasets. To compare the effectiveness of each method, five representative images were chosen for demonstration. These include the segmentation of benign (line 1) and malignant (line 2) breast ultrasound tumors, small single regions with smooth shapes (line 3), large regions (line 4), and the multicores image (line 5). The results indicate that δARD achieves accurate ROI region consistency and resolves the multipoint prediction challenges encountered by other models. The use of δARD loss improves the model's performance by detecting complete ROI while reducing false positives, particularly in medical images with varying sizes, shapes, and ROI volumes. Our findings demonstrate that implementing the δARD loss function can enhance the contour shape and position accuracy of different models.

Zoom In Zoom Out Reset image size

Table 3 presents the validation results of δARD on various datasets and models including UNet, UNet++, RDAUNet, Attention UNet, Ghost UNet, MedT, and Trans UNet. The study compared the effects of CE and δARD loss function on the segmentation of PBUSI, GLAS, and MoNuSeg datasets with seven models. Our findings indicate that δARD outperforms CE on both PBUSI and GLAS datasets. Notably, on 38 out of 49 measures in the MoNuSeg dataset, we achieved better results than CE loss. The significant improvement in the segmentation performance of different models through the use of our proposed loss function suggests its validity and effectiveness. Therefore, the results in table 3 provide evidence for the practicality of δARD loss function.