In this section, we discuss the use of three sets of synthetic data to evaluate the performance of ABCEE and present the results. The algorithms considered for comparison include ABCEE-R, ABCEE-V, AVMAX, MVC-NMF, MVSA, RMVES and VCA. These algorithms are all based on the LSMM and are the most typical ones among geometric-based approaches. AVMAX and VCA use model Equation (5). The difference between them is that VCA adopts the pure pixel assumption, which means that the endmember set is a subset of the pixels, whereas AVMAX does not require this assumption. MVSA and RMVES use model Equation (6), which is the closest model to that used in ABCEE. Although MVC-NMF uses NMF as the mathematical tool for endmember extraction and abundance inversion, its underlying principle can be regarded as an improvement to model Equation (6) through the use of the minimum-volume constraint.
3.1. Synthetic Dataset 1
The first synthetic dataset was generated using the following method. First, the spectra of five different materials (826 bands) were selected from the USGS Spectral Library to serve as the endmembers, denoted by
. Then, the abundances for each endmember were generated following [
29]. For example, if the abundance of
in the upper left point of the image is 1, then the abundance of this material progressively decreases to a minimum value of 0 for points increasingly far from that point; similarly, if the abundance of
at the center is 1, then the abundance of this material also progressively decreases to a minimum value of 0 for points increasingly far from that point. Third, based on these abundances, the endmembers were mixed in accordance with the LSMM approach to obtain a synthetic image of 100 × 100 pixels. Finally, random noise with an SNR of 100:1 was added.
In practice, the exact number of endmembers in an image is typically unknown, and the number of endmembers estimated based on the image information may be greater or smaller than the true number of endmembers. Through experiments with different numbers of endmembers, this section presents an evaluation of the accuracy of endmember extraction when the number of endmembers is more than (
), equal to (
) or less than (
) the true number of endmembers; the results are shown in
Table 1. As shown in the table, in cases of different numbers of endmembers, ABCEE-R achieves the best RMSE results.
For the case of , the ABCEE-V RMSE is worse than that of ABCEE-R, but its volume is the smallest. For the case of , although the RMVES volume, which is the best in this case, is slightly better than that of ABCEE-V, the ABCEE-V RMSE is vastly superior to that of RMVES. In other words, when the number of endmembers extracted is more or less than the true number of endmembers, ABCEE can produce good results, proving that an algorithm’s robustness can be enhanced by improving its objective function.
Table 1.
Experimental results for synthetic hyperspectral dataset 1.
Table 1.
Experimental results for synthetic hyperspectral dataset 1.
M | Index | AVMAX | MVCNMF | MVSA * | RMVES | VCA | ABCEE-R | ABCEE-V |
---|
4 | RMSE | 0.18621 | 0.084863 | 0.012774 | 0.087644 | 0.283535 | 0.011625 | 0.082551 |
Volume | 501.1301 | 359.0325 | 1799.304 | 710.6319 | 440.9159 | 777.213 | 320.0297 |
5 | RMSE | 0.004896 | 0.195454 | 0.009964 | 0.001838 | 0.007107 | 0.000681 | 0.003071 |
Volume | 1001.615 | 1055841 | 1253.954 | 1053.175 | 1001.296 | 2048.226 | 1537.273 |
SAD (rad) | 0.002236 | 0.081847 | 0.016715 | 0.004058 | 0.002254 | 0.051416 | 0.0868 |
6 | RMSE | 0.019094 | 0.472892 | 0.009954 | 0.221258 | 0.008665 | 0.004744 | 0.150269 |
Volume | 80.83104 | 60541510 | 827.5818 | 19.67032 | 61.82337 | 58.11748 | 21.25579 |
For the case of , the RMSEs of both ABCEE algorithms are better than those of AVMAX, MVCNMF, MVSA and VCA, but their volumes are slightly worse. This is because the number of endmembers extracted equals to the true number of endmembers, and the pixels in the feature space truly exhibit the characteristics of a simplex after dimensionality reduction; moreover, the volumes calculated based on internal maximum-volume models (AVMAX and VCA) will obviously be smaller than the volumes calculated based on external minimum-volume models (the other four algorithms), and a larger volume means that more points are located inside the simplex, resulting in a smaller RMSE.
In addition, for the case of
, the SAD was considered to evaluate the differences between the extracted endmembers and the true endmembers; the results are shown in
Table 1. As seen from the table, the SAD results of AVMAX and VCA (based on internal maximum-volume models) indicate the best performance, whereas the other five algorithms performed somewhat poorly. This is because the synthetic data contained pure pixels and a high SNR, conditions like that favor the internal maximum-volume approach. Among the five algorithms based on external minimum-volume models (excluding MVSA), the RMVES result is of the highest quality, followed by the ABCEE-R result, whereas the endmembers extracted via ABCEE-V deviate the most from the true endmembers; these findings are consistent with the design of ABCEE-V, whose objective function attempts to achieve a tradeoff between volume and error.
The objective function values of the different iterations of ABCEE algorithms are shown in
Figure 3, where the horizontal axis represents the number of iterations, the vertical axis represents the logarithm of the objective function, and the colors represent the colony size, which is the sum of the numbers of employed and onlooker bees (
i.e.,
). For the experiments presented in this paper, these numbers were set to be identical (
i.e.,
). It is evident from the figure that significant changes in the objective function values occur over the first 200 iterations. This is because the quality of the randomly generated initial solutions is poor and has a large potential for optimization, giving rise to a high optimization speed. As the number of iterations increases, the optimization speed is reduced, and in most cases, after 600 iterations, the objective function may be difficult to optimize further. The computation times for different colony sizes are shown in
Table 2. As indicated in
Figure 3 and
Table 2, a larger colony size yields a better final objective function value but also leads to longer computation time. This is because with a greater colony size, the number of search neighborhoods increases, and a better solution is more likely to be found. The computation time for each neighborhood search is nearly identical; therefore, the colony size (
i.e., the number of searches) directly determines the total computation time.
Table 2.
ABCEE computation time comparison for different populations (seconds).
Table 2.
ABCEE computation time comparison for different populations (seconds).
Algorithm | | Population | 20 | 50 | 80 | 100 |
---|
Number of Endmembers | |
---|
ABCEE-R | 4 | 630.94 | 1540.09 | 2475.27 | 3070.64 |
5 | 680.75 | 1718.50 | 4117.73 | 3433.15 |
6 | 733.81 | 1728.14 | 2751.05 | 3424.61 |
ABCEE-V | 4 | 24.84 | 62.73 | 98.05 | 125.10 |
5 | 32.78 | 80.20 | 144.30 | 189.03 |
6 | 33.51 | 83.11 | 132.93 | 169.73 |
Figure 3.
Changes in the ABCEE objective function over the course of multiple iterations. The horizontal axis represents the iteration number, and the vertical axis represents the logarithm of the objective function value. (a) ABCEE-R, M=4; (b) ABCEE-V, M=4; (c) ABCEE-R, M=5; (d) ABCEE-V, M=5; (e) ABCEE-R, M=6; (f) ABCEE-V, M=6.
Figure 3.
Changes in the ABCEE objective function over the course of multiple iterations. The horizontal axis represents the iteration number, and the vertical axis represents the logarithm of the objective function value. (a) ABCEE-R, M=4; (b) ABCEE-V, M=4; (c) ABCEE-R, M=5; (d) ABCEE-V, M=5; (e) ABCEE-R, M=6; (f) ABCEE-V, M=6.
In this experiment, nine values of and were considered to find the best value of penalty factor, respectively. We set here, where is the order-of-magnitude difference between the two parts of the objective function, and its value can be determined by an initial endmember extraction result obtained by any kind of algorithms of short computing time, such as AVMAX and VCA, without considering the pure pixel assumption. While , was set to 5 for and 3 for .
Figure 4 reflects the influence of the penalty factors
and
on the endmember extraction results. The horizontal axis represents the corresponding value of
for each penalty factor. The average SAD, the volume of the simplex and the RMSE were used as indicators for evaluating
and
. It is apparent from
Figure 4a,b that for both
and
, the results are optimal when
. Besides, from
Figure 4c,d, it can be found that the simplex’s volume of extracted endmembers is most similar to that of real endmembers while
, which is consistent of the results of SAD. If
, the proportion of volume in all the factors contained in objective function is too large, which results to the very small value of simplex volume, the very large values of RMSE and SAD. Obviously, this kind of result is not right. On the other hand, if
, the proportion of volume in all the factors contained in objective function is too small, which means that with the increase of
, the algorithm is more likely to find the solutions of smaller RMSE, even though the simplex volume may become bigger or few points may not contained in the simplex. Actually, if a simplex volume is big enough, all pixels are contained in the simplex, so it is extremely hard to optimize RMSE, which can be proved by the experiments shown in
Figure 4c,d that RMSE varied little while enlarging its volume.
Figure 4.
Influence of the penalty factors and (a) the SAD of ABCEE-R; (b) the SAD of ABCEE-V; (c) the Volume and RMSE of ABCEE-R; (d) the Volume and RMSE of ABCEE-V.
Figure 4.
Influence of the penalty factors and (a) the SAD of ABCEE-R; (b) the SAD of ABCEE-V; (c) the Volume and RMSE of ABCEE-R; (d) the Volume and RMSE of ABCEE-V.
Based on the analysis above, it is known that the value setting of
and
plays a crucial role in the final result of endmember extraction. In addition, we can conclude that while
, a result of the best tradeoff between Volume and RMSE can be obtained. Therefore, in this paper, the value of
and
in other experiments can be defined by the following steps, (1) extract a set of initial endmembers using AVMAX, (2) calculate its corresponding volume, penalty term (RMSE in Equation (19) or the number of points outside simplex in Equation (16)) and the ratio
, and (3) calculate
and
according the following formula.
3.2. Synthetic Dataset 2
The second synthetic dataset included 16 data calculated with four different numbers of endmembers (
m = 4, 8, 16 and 20) and four different SNRs (90:1, 70:1, 50:1 and 30:1). All endmembers were drawn from the USGS Spectral Library, and the abundance of each endmember in each pixel, following beta distributions, was generated in accordance with [
6]. Once the endmembers and abundances were prepared, they were multiplied together and summed to constitute the first term of Equation (1), and then, Gaussian white noise spectra with different SNRs were added in.
The RMSEs of the 16 data in synthetic Dataset 1 are shown in
Figure 5. The RMSEs for MVSA are not contained in this figure because the endmembers extracted via MVSA contain negative values, which is clearly unreasonable. The reason for this phenomenon is that MVSA is based on an external minimum-volume model, and its calculation using Sequential Quadratic Programming (SQP), but its feasible solution space is not constrained to be non-negative. The core algorithm of MVC-NMF is non-negative matrix factorization and thus contains a non-negative constraint, and ABCEE’s feasible solution space is also subject to a non-negative constraint; therefore, the three corresponding algorithms, though also based on the minimum-volume approach, do not produce results that contain negative values.
Figure 5.
RMSEs for synthetic Dataset 2. (a) m=4; (b) m=8; (c) m=16; (d) m=20.
Figure 5.
RMSEs for synthetic Dataset 2. (a) m=4; (b) m=8; (c) m=16; (d) m=20.
Figure 5 demonstrates that ABCEE-R is clearly superior to the other algorithms when the number of endmembers is relatively small (
m = 4 or 8), whereas in other circumstances (
m = 16 or 20), the performances of ABCEE-R, AVMAX and VCA are nearly identical. Among these 16 experiments, ABCEE-R achieved the smallest RMSE 12 times, whereas ABCEE-V and VCA achieved the smallest RMSE two times each; in the latter cases, the RMSEs of ABCEE-R were ranked second, very close to the best results. Therefore, we can infer from these results that ABCEE-V is more suitable for experimental data with fewer endmembers and higher SNRs, whereas ABCEE-R is more robust. In addition, with a reduction in SNR, the RMSE values for each algorithm increase. However, ABCEE-R always retains its advantage over the other algorithms, illustrating its insensitivity to the number of endmembers and the SNR.
The SADs of the 16 data in synthetic Dataset 1 are shown in
Figure 6. For the same reason discussed above, the results for MVSA were not added to this figure. No explicit trend is evident in the change of the SADs of the two ABCEE algorithms for the data with different SNRs. AVMAX and VCA can always obtain better SADs. This is because certain pixels can be very similar to certain endmembers when the abundances obey a beta distribution, which results in high proportions of certain endmembers in certain pixels. For example, for
m = 4, these four endmembers’ maximum abundances were 0.9884, 0.9898, 0.9876 and 0.9887, indicating that in these simulated images, although noise was present, the pure pixel assumption was essentially satisfied. Therefore, any errors between the extracted endmembers and the real ones would predominantly originate from noise when an algorithm based on the internal maximum-volume method was used. The external minimum-volume method demonstrated no obvious advantages compared with the other methods, and because of the presence of noise, its endmember extraction results could be quite different from the real results when the endmembers were relatively few (
m = 4 or 8). However, with an increasing number of endmembers (
m = 16 or 20), the largest proportion of each endmember in a pixel gradually decreases; for example, for
m = 20, the smallest value among the maximum abundances of these 20 endmembers was merely 0.8784. Therefore, under these circumstances, the SAD of ABCEE-R gradually overtakes and eventually exceeds those of AVMAX and VCA. The results of ABCEE-V are also clearly superior those of MVC-NMF and RMVES, although they are inferior to those of ABCEE-R, especially when the SNR is low.
Figure 6.
SADs for synthetic Dataset 2. (a) m=4; (b) m=8; (c) m=16; (d) m=20.
Figure 6.
SADs for synthetic Dataset 2. (a) m=4; (b) m=8; (c) m=16; (d) m=20.
3.3. Synthetic Dataset 3
The third synthetic dataset is mixed by four endmembers, which are similar to the endmembers in the first synthetic dataset, while m = 4. The production process of this dataset is the same as the first synthetic dataset. It is noteworthy that the endmember abundance of the third synthetic dataset follows normal distribution, and each endmember’s maximum abundance in any pixel is no more than 80%. Besides, in the final step of adding Gaussian white noise, SNRs are 100:1 and 50:1, respectively. In general, this dataset simulates the circumstance that data contains no pure pixels.
The results are shown in
Table 3. It can be seen that for the circumstances of two different SNRs, ABCEE-R and ABCEE-V can always obtain the minimum RMSE and SAD, respectively, which means that ABCEE can gain better endmember extraction results no matter how high or how low the SNR is while processing images without pure pixels. The experiment results are coordinated with the purpose of ABCEE’s objective function design. Equation (19) assures the volume of simplex can be well maintained while optimizing RMSE, which makes the spectral angle of the extracted and real endmembers be maintained in a suitable ranges (better than MCV-NMF and RMVES) while getting the minimum RMSE using ABCEE-R. Besides, Equation (16) allows few data points located in the outside of simplex while searching the external minimum-volume, which brings the endmembers extracted by ABCEE-V closer to real endmembers, and its RMSE only inferior to ABCEE-R and MVSA.
Table 3.
Experimental results for synthetic hyperspectral dataset 3.
Table 3.
Experimental results for synthetic hyperspectral dataset 3.
SNR | Metric | AVMAX | MVC-NMF | MVSA | RMVES | VCA | ABCEE-R | ABCEE-V |
---|
100:1 | SAD | 0.12444 | 0.174573 | 0.057896 | 0.208851 | 0.124838 | 0.149328 | 0.040319 |
RMSE | 0.028656 | 0.076074 | 1.18 × 10−16 | 0.261966 | 0.034615 | 1.02 × 10−16 | 0.003675 |
50:1 | SAD | 0.120362 | 0.177811 | 0.08611 | 0.153927 | 0.118427 | 0.145519 | 0.049782 |
RMSE | 0.023661 | 0.078122 | 1.35 × 10−16 | 0.006434 | 0.0319 | 1.02 × 10−16 | 0.006023 |
Besides, the synthetic dataset 3 is also used to evaluate the robustness of ABCEE.
Table 4 represents the SAD and RMSE’s average value and standard deviation of ABCEE-R and ABCEE-V’s results by computing 15 times.
Figure 7 shows the histogram of these results. From
Table 4, it can be seen that the standard deviations of ABCEE are always smaller than 0.05, which shows the stabilization of the algorithm, even though the endmember extraction results are different. In addition, from the point view of relative value of standard deviation and average value, the robustness of ABCEE-R is better than ABCEE-V.
On another hand, we can see from
Figure 7 that most results are concentrated in a small interval under the same circumstances and only very few results are abnormal, such as the result at the far right in
Figure 7a, and the two results at the far right in
Figure 7b. It can be found that in 120 results, only 20 results are abnormal, which means that 83.33% results can be located in a good interval by ABCEE.
Table 4.
The mean and standard deviation of 15 times run of ABCEE.
Table 4.
The mean and standard deviation of 15 times run of ABCEE.
| | ABCEE-R | ABCEE-V |
---|
| | RMSE | SAD | RMSE | SAD |
---|
SNR = 100:1 | Mean | 1.02 × 10−16 | 0.143641 | 0.004334 | 0.064347 |
Stdev | 5.79 × 10−18 | 0.040741 | 0.003203 | 0.026924 |
SNR = 50:1 | Mean | 1.04 × 10−16 | 0.1681471 | 0.007672 | 0.082615 |
Stdev | 4.57 × 10−18 | 0.0380169 | 0.004312 | 0.035142 |
Figure 7.
Hist of the synthetic Dataset 3. (a) ABCEE-R, SNR=100:1, RMSE; (b) ABCEE-V, SNR=100:1, RMSE; (c) ABCEE-R, SNR=100:1, SAD; (d) ABCEE-V, SNR=100:1, SAD; (e) ABCEE-R, SNR=50:1, RMSE; (f) ABCEE-V, SNR=50:1, RMSE; (g) ABCEE-R, SNR=50:1, SAD; (h) ABCEE-V, SNR=50:1, SAD.
Figure 7.
Hist of the synthetic Dataset 3. (a) ABCEE-R, SNR=100:1, RMSE; (b) ABCEE-V, SNR=100:1, RMSE; (c) ABCEE-R, SNR=100:1, SAD; (d) ABCEE-V, SNR=100:1, SAD; (e) ABCEE-R, SNR=50:1, RMSE; (f) ABCEE-V, SNR=50:1, RMSE; (g) ABCEE-R, SNR=50:1, SAD; (h) ABCEE-V, SNR=50:1, SAD.