Extracting the Green Fractional Vegetation Cover from Digital Images Using a Shadow-Resistant Algorithm (SHAR-LABFVC)

Abstract

Taking photographs with a commercially available digital camera is an efficient and objective method for determining the green fractional vegetation cover (FVC) for field validation of satellite products. However, classifying leaves under shadows in processing digital images remains challenging and results in classification errors. To address this problem, an automatic shadow-resistant algorithm in the Commission Internationale d’Eclairage L*a*b* color space (SHAR-LABFVC) based on a documented FVC estimation algorithm (LABFVC) is proposed in this paper. The hue saturation intensity (HSI) is introduced in SHAR-LABFVC to enhance the brightness of shaded parts of the image. The lognormal distribution is used to fit the frequency of vegetation greenness and to classify vegetation and the background. Real and synthesized images are used for evaluation, and the results are in good agreement with the visual interpretation, particularly when the FVC is high and the shadows are deep, indicating that SHAR-LABFVC is shadow resistant. Without specific improvements to reduce the shadow effect, the underestimation of FVC can be up to 0.2 in the flourishing period of vegetation at a scale of 10 m. Therefore, the proposed algorithm is expected to improve the validation accuracy of remote sensing products.

1. Introduction

Fractional vegetation cover (FVC) is widely used to describe vegetation quality and ecosystem changes and is a controlling factor in transpiration, photosynthesis and other terrestrial processes [1,2,3]. Estimating FVC in field measurements is critical because it provides a baseline for improving remote sensing algorithms and validating products.

Visual estimation, sampling [4], photography [5] and other techniques are commonly used in field measurements. Among these, photography with digital cameras is one of the most important methods. Analyzing digital images to calculate FVC is efficient and accurate in most circumstances [3,5,6,7,8,9,10,11]. The parts of an image that contain vegetation can be determined based on their physical, shape and color characteristics and other features [6,7,8,12]. In general, these methods can be grouped into two classes: (1) cluster analysis based on training samples, e.g., supervised and unsupervised classifications [13,14] and object-based image analysis methodology (OBIA) [15,16]; and (2) threshold-based methods according to the vegetation index, such as the color index of vegetation extraction (CIVE) [17], the excess green index (ExG) [18,19], excess green minus excess red (ExG−ExR) [20], and many other indices [11,21]. When these two types of methods are used for image classification, different color spaces are usually introduced and analyzed, for example red green blue (RGB) and hue saturation intensity (HSI) color spaces [12], for the mean-shift-based color segmentation method and the Commission Internationale d’Eclairage L*a*b* (LAB) color space for the documented FVC estimation algorithm named LABFVC [22] and LAB2 [23].

However, shadows should be addressed when FVC is extracted from a digital image. Shadows projected by vegetation increase the contrast in an image, alter the color in shaded areas and affect the image analysis. Shadows occur not only on the soil, but also inside the vegetation canopy [15].

Although a series of methods have been proposed, the shadow problem has not been perfectly solved. Visual interpretation using the supervised classification tool via commercial image processing software can perfectly distinguish vegetation in shadows through human to computer interaction. However, this method requires many manual steps and is less automatic and efficient [13,14,16]. Manual steps can cause bias and inconsistency between observers [24]. Methods based on physical characteristics [15,16] and feature space analysis [12] have been proposed to solve the shadow problem, but are time consuming and unsuitable for real-time applications. Using artificial shelters to change the illumination conditions and to reduce the contrast between sunlit and shaded areas can avoid the shadow effect in small areas [21,25]. However, shading a large area is difficult. Extremely dark shadows compared with sunlit leaves complicate the classification of shaded leaves using threshold-based methods and certain methods based on physical characteristics [15,23,26]. Extremely dark shadows are most likely to occur when photos of dense vegetation are taken on a sunny day.

In this study, we propose a modified LABFVC algorithm that is shadow resistant and can classify green vegetation with reasonable accuracy. The whole method is realized in Matrix Laboratory (MATLAB; the MathWorks, USA) and can extract FVC automatically and efficiently. The accuracy of our method was evaluated using real and synthesized images.

2. Study Area and Data Processing

2.1. Study Area and Field Measurements

All data used in this article were obtained from the Heihe Watershed Allied Telemetry Experimental Research (HiWATER) field campaign conducted in the Heihe River basin, China [27]. Field measurements of FVC were taken in corn fields, orchards and fields planted with vegetables, such as peppers and red beans, from 24 May 2012 to 14 September 2012, every five days on average [28,29].

The land surface is relatively homogeneous in this area, and a patch of field was usually larger than 15 m × 15 m. This size matches the spatial scales of moderate-resolution satellites, e.g., the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) [30]. A plot of 10 m × 10 m representing the scale of ASTER pixels was designed at every sampling position. Digital photographs were taken along two diagonals of the plot (taken once for the overlapping section at the cross point of the two diagonals). Twenty-two sampling plots were distributed throughout the area, of which 15 were located in corn fields. Nine photographs of each plot were taken. Approximately 4100 images were obtained during HiWATER 2012. Figure 1 shows the pattern in which photographs were taken in each plot.

Figure 1. Schematic of a plot showing how the photographs were taken. The nine small squares in the middle of the figure represent the photography positions in a plot. The large square around these squares represents the area of the sample plot.

Figure 1. Schematic of a plot showing how the photographs were taken. The nine small squares in the middle of the figure represent the photography positions in a plot. The large square around these squares represents the area of the sample plot.

2.2. Data Description

2.2.1. Synthesized Images

The synthesized image was composed of different mosaics of digital images obtained in the HiWATER experiment, 2012 [29]. The reference FVC of the synthesized image can be exactly calculated as the ratio of the number of pixels of all vegetation mosaics to the total number of pixels in this image. Each of the square vegetation mosaics had 224 × 132 pixels. The entire synthesized image, with both vegetation and soil, had 2629 × 1765 pixels. The mosaics were derived from various images of leaves and soil, including those with deep shadows. Four different types of vegetation mosaics, namely bright-green vegetation, jade-green vegetation, bottle-green vegetation and shaded vegetation, were used to compose the vegetation part in the image (Table 1). Another four types of soil mosaics, namely, light-brown soil, brown soil, dark-grey soil and shaded soil, were selected to form the square image. The vegetation mosaics were regularly embedded in the synthesized image.

The shadow-resistant LABFVC (SHAR-LABFVC) algorithm proposed in this study, the LABFVC algorithm and visual interpretation were then implemented to process this image to evaluate the accuracy of these methods.

Table 1. Different mosaics composing the simulated picture.

Vegetation	Representation	Soil	Representation
	Bright-green vegetation		Light-brown soil
	Jade-green vegetation		Brown soil
	Bottle-green vegetation		Dark-grey soil
	Shaded vegetation		Shaded soil

Table 1. Different mosaics composing the simulated picture.

Vegetation	Representation	Soil	Representation
	Bright-green vegetation		Light-brown soil
	Jade-green vegetation		Brown soil
	Bottle-green vegetation		Dark-grey soil
	Shaded vegetation		Shaded soil

2.2.2. Real Images

Fifteen real images of different crop types, including corn, peppers, cabbage, celery and beans, in different growing periods in 2012 were selected from HiWATER for comparison. All images were classified by visual interpretation, the LABFVC algorithm and SHAR-LABFVC. Visual interpretation was performed using Environment for Visualizing Images (ENVI; Exelis, Inc., Boulder, CO, USA) to obtain a reference FVC of the real images. First, we attempted to choose as many samples as possible and performed a supervised classification method using the maximum likelihood classifier. The image was divided into sunlit vegetation, shaded vegetation, sunlit soil and shaded soil. Then, the four classes were combined into two classes, namely vegetation and background, and an interactive tool was used to examine and correct the region of each class in the image. Finally, the proportion of vegetation was computed as the reference FVC of the image.

3. Methodology

SHAR-LABFVC was developed from and improves upon the work of Liu et al. [22], which is hereafter referred to as LABFVC. In the proposed algorithm, the hue saturation intensity (HSI) color space is introduced to decrease the difference between sunlit and shaded leaves. Subsequently, the image is transformed into the LAB color space, where the green red (a*) component is used to distinguish vegetation and background; negative a* values theoretically indicate green, and positive a* values indicate red. The distributions of green vegetation and background are respectively modeled with lognormal and Gaussian distribution functions on the a* component. The threshold is then calculated to classify the vegetation and background.

3.1. LABFVC Algorithm

The LABFVC algorithm proposed by Liu et al. [22] is an automatic FVC-extracting algorithm for digital images with an accuracy similar to that of the greenness (2G-R-B) method [31]. LABFVC is based on the premise that the representations of vegetation and soil in the LAB color spaces approximately follow Gaussian distributions. Therefore, green vegetation is classified by solving a Gaussian mixture model in the LAB color space. The threshold separating the vegetation from the background is automatically selected and used to calculate the FVC in an image.

The LABFVC algorithm consists of the following three steps:

First, the red-green-blue (RGB) digital image is converted to the LAB color space. The LAB color space is close to the human perception of color, and its channels are less correlated than those of the RGB color space [32]. Methods for transforming between RGB and LAB can be found in the references. The a* channel in the LAB color space is recommended for detecting green vegetation.

Second, assuming that the distributions of the vegetation and background in the a* dimension are Gaussian distributions, the following function can be used to fit the distribution of their frequency on the a* component:

$$F(x)=\frac{w_v}{\sqrt{2\pi }{\sigma }_v}{e}^{\frac{-{(x-{\mu }_v)}^2}{2{\sigma }_v^2}}+\frac{w_b}{\sqrt{2\pi }{\sigma }_b}{e}^{\frac{-{(x-{\mu }_b)}^2}{2{\sigma }_b^2}}$$

(1)

where the dependent variable F is the frequency of the green vegetation and the background on the a* component; the independent variable x represents their a* values; μ_v and μ_b are the average values of the green vegetation and background, respectively; σ_v and σ_b are the standard deviations of the green vegetation and background, respectively; and w_v and w_b are the weights of the green vegetation and background, respectively.

Third, the threshold value T is determined based on the assumption that the probabilities of misclassifying vegetation and background are equal, which is used to classify the vegetation and background. After fitting the distributions of the vegetation and background by Equation (1), the average values of the green vegetation (μ_v) and background (μ_b), their standard deviations (σ_v and σ_b) and their weights (w_v and w_b) are known. Threshold T can be obtained by solving Equation (2):

$$w_v\cdot erfc(\frac{T-{\mu }_v}{\sqrt{2}{\sigma }_v})=w_b\cdot erfc(\frac{T-{\mu }_b}{\sqrt{2}{\sigma }_b})$$

(2)

where the complementary error function erfc is incorporated to determine the threshold:

$$erfc(x)=1-erf(x)=\frac{2}{\sqrt{\pi }}{\displaystyle {\int }_x^{\infty }{e}^{-t^2}dt}$$

(3)

where erf is the Gaussian error function.

The fraction of vegetation cover in an image is calculated after the vegetation is classified using the threshold T. For a more detailed description of the LABFVC algorithm, we refer to Liu et al. [22].

3.2. Shadow-Resistant LABFVC

Step 3 in the LABFVC method is identical to a step in SHAR-LABFVC, but Steps 1 and 2 are combined with new ideas to resist the shadow effect. In Figure 2, the following additional and improved processes are highlighted in yellow in the SHAR-LABFVC flow chart: (1) the use of the HSI color space to equalize the intensity histogram; and (2) fitting of the vegetation on the a* component of the LAB color space with a lognormal model.

Figure 2. Schematic of shadow-resistant SHAR-LABFVC (LAB, the Commission Internationale d’Eclairage L*a*b* color space; FVC, fractional vegetation cover). The portions highlighted in yellow are the main differences between the LABFVC algorithm and SHAR-LABFVC. a* is the green red component in the LAB color space. When a pixel’s a* value is smaller than the threshold T, the pixel is classified as green vegetation.

Figure 2. Schematic of shadow-resistant SHAR-LABFVC (LAB, the Commission Internationale d’Eclairage L*a*b* color space; FVC, fractional vegetation cover). The portions highlighted in yellow are the main differences between the LABFVC algorithm and SHAR-LABFVC. a* is the green red component in the LAB color space. When a pixel’s a* value is smaller than the threshold T, the pixel is classified as green vegetation.

In the first step of SHAR-LABFVC, the equalization of the intensity histogram aims to brighten the shaded leaves and to promote their separation probability. The HSI color space is chosen to enhance the intensity of the three components of an image in this color space, which are less correlated with each other than in the RGB color space [26]. For an RGB image, the intensity I of each pixel in HSI space is given by the following [26]:

$$I=\frac{1}{3}(R+G+B)$$

(4)

To brighten the shaded leaves, we transform the RGB color space into the HSI space, select its intensity component, equalize the intensity histogram, replace the original intensity with the equalized one and retransform the image to the RGB color space.

In the second step (Figure 2), the distributions of the green vegetation and background are modeled on the a* component in the LAB color space, as in the LABFVC method. The modification to this step is that SHAR-LABFVC uses a lognormal model for the vegetation histogram. Green vegetation has a wide distribution on the negative axis of the a* component. The shaded vegetation is nearly black and has an a* value of approximately 0. Therefore, the complete color distribution of the vegetation is similar to a lognormal distribution. Figure 3 shows a histogram distribution of the a* component of an image of a corn field. The distribution of the background is assumed to be Gaussian, as in the LABFVC method.

Figure 3. Histogram of the a* (green red) component of an image. The green and brown curves represent the lognormal distribution of the vegetation and the Gaussian distribution of the background, respectively. The small image in the left corner is the original image.

Figure 3. Histogram of the a* (green red) component of an image. The green and brown curves represent the lognormal distribution of the vegetation and the Gaussian distribution of the background, respectively. The small image in the left corner is the original image.

The following function can be used to fit the distributions of both the green vegetation and the background’s frequency on the a* component:

$$F(x)=\frac{w_v}{x\sqrt{2\pi }{\sigma }_v}{e}^{\frac{-{(\ln x-{\mu }_v)}^2}{2{\sigma }_v^2}}+\frac{w_b}{\sqrt{2\pi }{\sigma }_b}{e}^{\frac{-{(x-{\mu }_b)}^2}{2{\sigma }_b^2}}$$

(5)

where vegetation is fitted by a lognormal function.

In the third step, the green FVC is extracted by classifying the a* values of the images at the threshold T, determined using Equation (2). This step assumes that the misclassification probabilities of the vegetation and background are approximately equal. In Figure 4, s_v and s_b represent the areas of the misclassified vegetation and background, respectively. T is chosen such that s_v and s_b are equal.

Figure 4. Determining the threshold for image classification. The red curve is based on a lognormal distribution and represents the vegetation, and the black curve is based on a Gaussian distribution and represents the background. The classification is based on the a* (green red) component of the LAB color space.

Figure 4. Determining the threshold for image classification. The red curve is based on a lognormal distribution and represents the vegetation, and the black curve is based on a Gaussian distribution and represents the background. The classification is based on the a* (green red) component of the LAB color space.

4. Results

4.1. Image Enhancement

Image enhancement is an important module in SHAR-LABFVC. Figure 5 shows the original, intensity-stretched and RGB stretched images, which present the effect of our image enhancement module. After the intensity histogram is equalized in the HSI color space, the entire image becomes brighter, particularly the shaded leaves in the red boxes (Figure 5b), facilitating the identification of shaded components. Equalizing all three components in the RGB color space can also highlight the shaded leaves, but changes the color of the image (Figure 5c). Therefore, stretching all of the RGB components is not usable when further classification of the vegetation is required.

Figure 5a was classified by visual interpretation, and the average values of the classes at the axis of the a* component are presented in Table 2, which also shows that the average values of the vegetation in the original image and the intensity-stretched image are both on the negative axis of the a* component. As shown in Table 2, the difference between the vegetation and background in the intensity-stretched image is greater than that in the original image. Image enhancement effectively enhances the difference between the vegetation and background and can help separate these two classes.

Figure 5. Image enhancement in different color spaces and the classification result. (a) The original image. (b) The intensity-stretched image after transformation to the HSI color space and intensity histogram equalization. (c) The RGB stretched image, in which the histograms of the three components (R, G, B) have been equalized separately. (d) The SHAR-LABFVC classification result. The red boxes marked with the same number in images (a), (b), (c) and (d) represent the same area.

Figure 5. Image enhancement in different color spaces and the classification result. (a) The original image. (b) The intensity-stretched image after transformation to the HSI color space and intensity histogram equalization. (c) The RGB stretched image, in which the histograms of the three components (R, G, B) have been equalized separately. (d) The SHAR-LABFVC classification result. The red boxes marked with the same number in images (a), (b), (c) and (d) represent the same area.

Table 2. Average values of the different classes and their differences in the image’s a* component with different image enhancement methods.

Data	Vegetation	Background	Distance*
The original picture	−17.204	0.708	17.912
Intensity-stretched picture	−30.539	1.418	31.957
RGB stretched picture	−0.576	15.299	15.875

* The distance represents the difference between the averaged values of two parts.

Table 2. Average values of the different classes and their differences in the image’s a* component with different image enhancement methods.

Data	Vegetation	Background	Distance*
The original picture	−17.204	0.708	17.912
Intensity-stretched picture	−30.539	1.418	31.957
RGB stretched picture	−0.576	15.299	15.875

* The distance represents the difference between the averaged values of two parts.

4.2. Analysis and Evaluation of the Synthesized Image

The synthesized image was used to compare SHAR-LABFVC, LABFVC and visual interpretation. The classification results are presented in Figure 6. Table 3 presents the FVC estimates. The result for SHAR-LABFVC is 0.237, identical to the reference FVC (0.237). The result for the LABFVC algorithm is lower, because it cannot distinguish leaves with deep shadows from the background (Figure 6c).

Figure 6. (a) The synthesized image and classification using (b) visual interpretation, (c) LABFVC and (d) SHAR-LABFVC. The white and black regions were classified as vegetation and soil, respectively.

Figure 6. (a) The synthesized image and classification using (b) visual interpretation, (c) LABFVC and (d) SHAR-LABFVC. The white and black regions were classified as vegetation and soil, respectively.

Table 3. Reference FVC and FVC estimates from the synthesized image test.

Approach	FVC
The Reference FVC *	0.237
LABFVC	0.209
SHAR-LABFVC	0.237

* The reference FVC is equal to the proportion of the area of all vegetation mosaics in the synthesized image.

Table 3. Reference FVC and FVC estimates from the synthesized image test.

Approach	FVC
The Reference FVC *	0.237
LABFVC	0.209
SHAR-LABFVC	0.237

* The reference FVC is equal to the proportion of the area of all vegetation mosaics in the synthesized image.

4.3. Analysis and Evaluation Based on Real Images

4.3.1. Different Types and Growth Stages of Vegetation

Fifteen real images from different crop types and different growing periods were selected for the evaluation. All images were classified strictly and carefully by visual interpretation via ENVI to obtain the reference FVC of the real images. Table 4 presents the classification accuracy of each image. In this evaluation, the average overall accuracy of visual interpretation was 99.54%.

Table 5 presents the comparison of the results of the methods for 15 randomly-selected real images that were carefully interpreted visually. When the vegetation is sparse, the reference FVC is less than 0.5, and the RMSEs of the two methods are similar. When the vegetation increases, the reference FVC is greater than 0.5, and the two methods appear different. Based on the reference FVC determined by visual interpretation, the RMSE of SHAR-LABFVC is 0.025. However, the RMSE of LABFVC is as high as 0.1. As the density of the vegetation increases, shadows deepen and influence the performance of LABFVC.

Table 4. Accuracy evaluation of the visual interpretation of the 15 real images.

Image	Crop Type	Product Accuracy (%)		User Accuracy (%)		Overall Accuracy (%)	Kappa Coefficient
		Vegetation	Background	Vegetation	Background
1	corn	94.10	100.00	100.00	88.60	95.96	0.9093
2	corn	99.94	99.94	99.99	99.77	99.94	0.9982
3	corn	98.68	100.00	100.00	99.60	99.70	0.9914
4	corn	100.00	100.00	100.00	100.00	100.00	1.0000
5	corn	100.00	99.34	99.91	100.00	99.92	0.9962
6	bell pepper	100.00	100.00	100.00	100.00	100.00	1.0000
7	bell pepper	99.42	99.88	100.00	99.81	99.76	0.9936
8	hot pepper	96.83	100.00	100.00	98.59	99.02	0.9768
9	hot pepper	100.00	99.81	100.00	100.00	99.93	0.9985
10	celery	99.91	99.48	99.60	99.88	99.72	0.9943
11	celery	99.95	99.91	99.96	99.98	99.94	0.9986
12	cabbage	99.99	100.00	100.00	99.99	100.00	0.9999
13	cabbage	100.00	100.00	100.00	99.99	100.00	0.9999
14	sapling	98.79	100.00	100.00	98.84	99.40	0.9880
15	sapling	99.97	99.57	99.64	99.98	99.79	0.9957

Table 4. Accuracy evaluation of the visual interpretation of the 15 real images.

Image	Crop Type	Product Accuracy (%)		User Accuracy (%)		Overall Accuracy (%)	Kappa Coefficient
		Vegetation	Background	Vegetation	Background
1	corn	94.10	100.00	100.00	88.60	95.96	0.9093
2	corn	99.94	99.94	99.99	99.77	99.94	0.9982
3	corn	98.68	100.00	100.00	99.60	99.70	0.9914
4	corn	100.00	100.00	100.00	100.00	100.00	1.0000
5	corn	100.00	99.34	99.91	100.00	99.92	0.9962
6	bell pepper	100.00	100.00	100.00	100.00	100.00	1.0000
7	bell pepper	99.42	99.88	100.00	99.81	99.76	0.9936
8	hot pepper	96.83	100.00	100.00	98.59	99.02	0.9768
9	hot pepper	100.00	99.81	100.00	100.00	99.93	0.9985
10	celery	99.91	99.48	99.60	99.88	99.72	0.9943
11	celery	99.95	99.91	99.96	99.98	99.94	0.9986
12	cabbage	99.99	100.00	100.00	99.99	100.00	0.9999
13	cabbage	100.00	100.00	100.00	99.99	100.00	0.9999
14	sapling	98.79	100.00	100.00	98.84	99.40	0.9880
15	sapling	99.97	99.57	99.64	99.98	99.79	0.9957

Table 5. Evaluation of the LABFVC algorithm and SHAR-LABFVC.

Approach	RMSE
	FVC < 0.5	FVC > 0.5
LABFVC	0.022	0.100
SHAR-LABFVC	0.025	0.025

Table 5. Evaluation of the LABFVC algorithm and SHAR-LABFVC.

Approach	RMSE
	FVC < 0.5	FVC > 0.5
LABFVC	0.022	0.100
SHAR-LABFVC	0.025	0.025

Example results of the comparison are displayed in Figure 7 and Figure 8. The two original images shown here were captured in July when the plants were in their lush vegetation period. Figure 8a was taken on a cloudy day; thus, the differences between the results of visual interpretation, SHAR-LABFVC and the LABFVC algorithm are small (Figure 8 and Table 6). Figure 7a was taken on a sunny day; thus, the difference between sunlit and shaded leaves tends to be large, and the advantages of SHAR-LABFVC become important (Figure 7 and Table 6).

Table 6. FVC estimates of the real images.

Approach	Figure 7	Figure 8
Visual interpretation	0.909	0.791
LABFVC	0.667	0.738
SHAR	0.902	0.799

Table 6. FVC estimates of the real images.

Approach	Figure 7	Figure 8
Visual interpretation	0.909	0.791
LABFVC	0.667	0.738
SHAR	0.902	0.799

Figure 7. A digital image and its classification results. (a) The original image of corn. (b) An enlarged view of the red box in (a). The classification results of (c) visual interpretation, (d) the LABFVC algorithm and (e) SHAR-LABFVC. In (c–e), the black and white areas are classified as background and vegetation, respectively. The red boxes in pictures represent the same area.

Figure 7. A digital image and its classification results. (a) The original image of corn. (b) An enlarged view of the red box in (a). The classification results of (c) visual interpretation, (d) the LABFVC algorithm and (e) SHAR-LABFVC. In (c–e), the black and white areas are classified as background and vegetation, respectively. The red boxes in pictures represent the same area.

Figure 8. The same as Figure 7, except that the original image is of celery. (a) The original image of celery. (b) An enlarged view of the red box in (a). The classification results of (c) visual interpretation, (d) the LABFVC algorithm and (e) SHAR-LABFVC. In (c–e), the black and white areas are classified as background and vegetation, respectively. The red boxes in pictures represent the same area.

Figure 8. The same as Figure 7, except that the original image is of celery. (a) The original image of celery. (b) An enlarged view of the red box in (a). The classification results of (c) visual interpretation, (d) the LABFVC algorithm and (e) SHAR-LABFVC. In (c–e), the black and white areas are classified as background and vegetation, respectively. The red boxes in pictures represent the same area.

4.3.2. Extreme Situations

Four extreme situations, including no shadow (Figure 9A), many shallow shadows (Figure 9B), no leaf (Figure 9C) and many leaves with deep shadows (Figure 9D), are represented in the corn images taken under different illumination conditions and growth periods. These images were carefully classified using visual interpretation via ENVI to obtain the reference FVC. The classification results are shown in Figure 9 and Table 7.

Figure 9. The classification results in four extreme situations. (A1) was taken on a cloudy day, when there was no shadow. (B1) was taken on a sunny day, when there were many shallow shadows covering both the vegetation and background. (C1) was taken in the Gobi desert, where there is seldom green vegetation. (D1) was taken when the corn was fully grown and there were limited gaps between the leaves. The images in Columns (2–4) are the classification results from visual interpretation, LABFVC and SHAR-LABFVC, respectively.

Figure 9. The classification results in four extreme situations. (A1) was taken on a cloudy day, when there was no shadow. (B1) was taken on a sunny day, when there were many shallow shadows covering both the vegetation and background. (C1) was taken in the Gobi desert, where there is seldom green vegetation. (D1) was taken when the corn was fully grown and there were limited gaps between the leaves. The images in Columns (2–4) are the classification results from visual interpretation, LABFVC and SHAR-LABFVC, respectively.

Table 7. FVC estimates of each extreme situation.

Image	Situation	Approach
		Visual Interpretation	LABFVC	SHAR-LABFVC
A	No shadow	0.579	0.559	0.563
B	Full of shallow shadows	0.683	0.674	0.684
C	No leaf	0.000	0.000	0.003
D	Full of leaves	0.942	0.642	0.922

Table 7. FVC estimates of each extreme situation.

Image	Situation	Approach
		Visual Interpretation	LABFVC	SHAR-LABFVC
A	No shadow	0.579	0.559	0.563
B	Full of shallow shadows	0.683	0.674	0.684
C	No leaf	0.000	0.000	0.003
D	Full of leaves	0.942	0.642	0.922

4.4. Comparison with Time Series Data

The results of the LABFVC algorithm and SHAR-LABFVC were compared for the 15 corn plots of the 2012 HiWATER experiment. The results of the comparison are shown in Figure 10. The two results were quite similar before 26 June, when the corn was young and the leaves did not shade each other significantly. As the crop matured, the underestimation of LABFVC became obvious. For example, on 13 July, the LABFVC algorithm underestimated the result by more than 0.2. After August, when corn was harvested and the leaves had turned yellow, the FVC determined by the different methods exhibited fewer differences.

Figure 10. Comparison of the results of the LABFVC algorithm and SHAR-LABFVC over the entire growing period of corn. Error bars represent the positive and negative standard errors of the mean.

Figure 10. Comparison of the results of the LABFVC algorithm and SHAR-LABFVC over the entire growing period of corn. Error bars represent the positive and negative standard errors of the mean.

5. Discussion

Shadows occur due to obstructions from terrain topography, cloud cover or dense vegetation and can cause errors in image classification. In this study, we proposed an automatic shadow-resistant FVC extraction method (SHAR-LABFVC) that can classify green vegetation efficiently and achieve stable results with reasonable accuracy.

SHAR-LABFVC was developed based on LABFVC, which was chosen because of its automaticity and efficiency [26]. SHAR-LABFVC can automatically determine the threshold to classify green vegetation and to compute FVC from digital images. In contrast to the supervised classification [13,14] and other image analysis approaches based on physical characteristics [15,16], neither LABFVC nor SHAR-LABFVC require manual steps to process images.

The time required to extract FVC using LABFVC is less than 12 seconds per image, much faster than other automated image classification methods based on physical characteristics (PC-based) [16] and approaches for feature space analysis (FSA-based) [12] (Table 8). The time required to extract FVC using SHAR-LABFVC is approximately five seconds per image. SHAR-LABFVC is thus sufficiently efficient for real-time application.

Table 8. Average computing time of the different methods.

Approach	Time (per image)	Computer Processor	Random-AccessMemory (RAM)	Program/Software
PC-based *	15 min	-	-	eCognition 8.7 (Trimble Navigation Ltd. USA)
FSA-based *	91.9 s	1.2 GHz	512 MB	VC++
LABFVC	12.0 s	3.1 GHz	4 GB	MATLAB
SHAR-LABFVC	5.0 s	3.1 GHz	4 GB	MATLAB

* The efficiency for the PC-based and the FSA-based methods was determined based on the work of Bauer et al. [16] and Zheng et al. [12].

Table 8. Average computing time of the different methods.

Approach	Time (per image)	Computer Processor	Random-AccessMemory (RAM)	Program/Software
PC-based *	15 min	-	-	eCognition 8.7 (Trimble Navigation Ltd. USA)
FSA-based *	91.9 s	1.2 GHz	512 MB	VC++
LABFVC	12.0 s	3.1 GHz	4 GB	MATLAB
SHAR-LABFVC	5.0 s	3.1 GHz	4 GB	MATLAB

* The efficiency for the PC-based and the FSA-based methods was determined based on the work of Bauer et al. [16] and Zheng et al. [12].

Real and synthesized image evaluation results were presented in Section 4.2 and Section 4.3. The classification accuracy of the real images (Table 4) and the process of image synthesis (Section 2.2.1) indicate a high reliability of the reference values.

The LABFVC algorithm works well when the vegetation is sparse. However, LABFVC is more sensitive to the contrast between sunlit parts and shadows, which causes systematic errors in classification. With the use of image enhancement and lognormal distribution, SHAR-LABFVC is much less sensitive to the shadow effect than LABFVC. Image enhancement was introduced to decrease the shadow effect, which is more convenient to implement than using artificial shelters to change the illumination conditions [21,25]. In contrast to other image analysis approaches [15,16], SHAR-LABFVC brightens and separates the shaded vegetation and background rather than classifying shadow as another class. This process also improves the accuracy of obtained green FVC.

SHAR-LABFVC was designed to extract FVC from the images taken with a digital camera. Photography is a commonly-used method for obtaining FVC from crop, grassland, low shrub areas and understory in sparse forests. When the digital image contains heavily-shaded parts, LABFVC cannot obtain vegetation efficiently. Compared to LABFVC, SHAR-LABFVC obtains results more similar to the reference FVC, particularly when heavily-shaded components are present. Representing the extreme situations that may appear in these areas, the classification results for images with no shadow, many shadows, no leaves and many leaves with deep shadows are shown in Figure 9. When no shadow or a few shadows cover the vegetation or background (Figure 9A1, B1 and C1), the differences among the results of LABFVC, SHAR-LABFVC and visual interpretation are small. When the leaves heavily shade one another (Figure 9D1), shadows tend to be dark and LABFVC significantly underestimates the vegetation (Figure 9D3).

Time series images of corn were processed to evaluate the accuracy of SHAR-LABFVC (Figure 10). The results of SHAR-LABFVC and LABFVC are quite similar during both the budding period and the period when leaves are withered. When the crops mature, leaves significantly shade the incoming light from the Sun and the sky. Deep shadows occur during this period, and the advantages of SHAR-LABFVC are apparent.

SHAR-LABFVC is shadow-resistant and reliable based on the evaluations. However, there are still some shortfalls in SHAR-LABFVC. Figure 7 presents a case in which the corn plants are tall and project many deep shadows that cover more than half of the image. Figure 7c shows that visual interpretation produces a smooth classification image. The main differences between the classification methods can be observed in the area in which the leaves are crimped or overlap with one another and generate dark shadows. Some shaded leaves are too dark to be stretched by SHAR-LABFVC. The leaf veins and solar glints also cause classification errors because they are extremely bright and nearly white. After adjusting the intensity, these leaves are still saturated and far from green. These situations correspond to the parts of the soil and vegetation histograms that are mixed in the image’s a* component (areas s_v and s_b in Figure 3), resulting in misclassification. Generally, the FVC estimates provided by SHAR-LABFVC are fairly close to those of visual interpretation under all possible illumination conditions.

6. Conclusions

Deep shadows severely affect the results of digital image classification, particularly when vegetation growth is at its peak. In this study, we developed a shadow-resistant LABFVC algorithm (SHAR-LABFVC) to extract FVC from digital images. SHAR-LABFVC improves the documented LABFVC method and resists shadows by equalizing the histogram of the image’s intensity component in the HSI color space and brightening the shaded parts of the leaves and background. Generally, the histogram of the vegetation’s green red component exhibits a lognormal distribution because of the shaded leaves. This property is considered when dividing the images into vegetation and background regions in the LAB (the Commission Internationale d’Eclairage L*a*b*) color space.

The evaluation of SHAR-LABFVC revealed high accuracy similar to that of visual interpretation. The latter is assumed to be an accurate classification method that allows a reference FVC to be obtained from a digital image. As demonstrated by the results of this study, the RMSE of SHAR-LABFVC based on visual interpretation is 0.025, indicating similar performances of SHAR-LABFVC and visual interpretation. However, SHAR-LABFVC is more pragmatic and automatic. Compared to other automatic classification methods based on physical characteristics or feature space analysis, SHAR-LABFVC is less time consuming. Thus, SHAR-LABFVC can be used to obtain the FVCs of a large number of images.

Shadows caused errors of up to 0.2 when estimating FVC at moderate resolutions (e.g., the scale of ASTER) in the flourishing vegetation period. This underestimation of FVC caused by shadow effects will also affect the evaluation of present global FVC products at coarse resolutions (e.g., 1 km), because the systematic errors of moderate-spatial resolution FVC cannot be eliminated in the up-scaling process. Therefore, the development of shadow-resistant algorithms for field measurements is required. Given the uncertainties in our algorithm, further research is needed to investigate additional vegetation types and data sources. However, SHAR-LABFVC is expected to facilitate the validation of satellite-based products in this context due to its efficiency.