Computerized Classification of Pneumoconiosis on Digital Chest Radiography Artificial Neural Network with Three Stages

Materials

As shown in Table 1, we used 38 posteroanterior (PA) chest radiographs developed by the Ministry of Labor. These included three normal cases, five cases of silicosis (category 1: three cases, category 2: one case, category 3: one case), five cases of asbestosis (category 1: two cases, category 2: two cases, category 3: one case), six cases of other pneumoconiosis (category 1: four cases, category 2: one case, category 3: one case), four cases of category 0 (normal case), six cases of category 1, six cases of category 2, and three cases of category 3 with pneumoconiosis [3, 33]. In addition, we used 17 PA chest radiographs developed by ILO. These included two cases of category 0 (normal case), five cases of category 1, five cases of category 2, and five cases of category 3 with pneumoconiosis [2]. The small opacities were classified into four categories, ranging from a completely normal case (category 0) to severe pneumoconiosis (category 3) [2]. These images were digitized with a pixel size of 0.175 mm, matrix sizes of 2468 × 2034 and 2034 × 2034, and 12-bit gray level.

Overall Scheme of Pneumoconiosis Classification with Four Categories

Figure 1 shows the overall scheme of the classification with four texture features. An experienced radiological technologist manually selected approximately 100–1200 ROIs on the lung with a matrix size of 32 × 32 in the intercostal spaces and over rib spaces for each chest radiograph [5]. We eliminated overlap with ROIs. Table 2 shows the number of ROIs selected for each case. We obtained a background trend in all selected ROIs using a two-dimensional surface-fitting technique based on the least squares method [5]. After the number of gray levels in these ROIs was reduced by quantization level, we then obtained gray-level histograms, histograms of gray-level differences, GLRLM feature images, and GLCOM feature images. The first ANN was trained with each texture feature for the ROI-based classification of patterns among the four categories. Next, the second ANN was trained with combined patterns including output values obtained from the first ANN with each texture feature. Finally, we obtained the case-based classification performed for distinguishing among the four categories with the third ANN.

Four Texture Features

First, the gray-level histogram was applied to the trend-corrected images quantized to the 4-bit gray level. Compared with the images in category 3, the frequencies of pixel values 6 and 7 were higher for the images in the other categories (Fig. 2). The histogram of gray-level differences indicated the frequency of the difference of gray levels i and j at a specified distance d and specified direction θ. The histogram of the gray-level difference (distance level: 7 pixels and direction: 0°) was applied to the trend-corrected images quantized to the 5-bit gray level. Compared with category 3, the frequencies of the other categories were higher in the 0-pixel value (Fig. 3). The effects of the number of gray levels and the distance levels on the gray-level histograms and histograms of gray-level differences in the ROI-based classification among the four categories will be discussed later.

A gray-level run-length was defined as a set of consecutive pixels of the same gray level in a given direction [21]. An element of a GLRLM measures the number of occurrences of a run with a specific gray level in a given direction [21]. In this study, the GLRLM feature images (direction: 0°) were applied to the trend-corrected images quantized to the 6-bit gray level (Fig. 4e–h). The horizontal and vertical directions of the GLRLM feature images were composed of the length of runs and the number of gray levels, respectively (Fig. 4e–h). These four categories showed quite different values in some parts of the GLRLM feature images. For example, as the profusion of small opacities was concentrated from category 0 to category 3, the GLRLM feature images for category 0 showed many long runs at the lower pixel values (Fig. 4e), whereas the GLRLM feature images for category 3 showed many short runs at the highest pixel values (Fig. 4h). We then determined the power spectrum (PS) images in these GLRLM feature images by fast Fourier transform (Fig. 4i–l). We used only PS values on the principal and secondary axes, which represent the maximum and next maximum PS values on the radial line from the central point, respectively [31, 32]. We used the PS values on the positive main and second axes representing spatial frequency, because they were symmetric with respect to the center of the PS image (Fig. 5) [31, 32]. Compared with category 3, the PS values of the other categories were lower at all spatial frequencies on the principal and secondary axes. The effects of the number of gray levels on the GLRLM feature images on the ROI-based classification among the four categories will be discussed later.

An element at location (i, j) of the GLCOM showed that the joint probability density function indicated the possibility of gray level between the i pixel and the j pixel in a specified direction θ and a specified distance d [15]. In the same manner as for the GLRLM feature images, the GLCOM feature images (distance level: 3 pixels and direction: 45°, 225°) were applied to the trend-corrected images quantized to the 6-bit gray level (Fig. 6e–h). The horizontal and vertical directions of the GLCOM feature images were composed of the number of gray levels [32]. We calculated PS images by Fourier transform (Fig. 6i–l) and obtained only the PS values on the principal and secondary axes (Fig. 7) [32]. The PS values for each category were different at spatial frequencies of 0.5–1.5 on the principal and secondary axes. The effects of the distance levels on the GLCOM feature images on ROI-based classification among the four categories will be discussed later.

First ANN with Four Texture Features

We used a feedforward ANN with a back propagation algorithm. On the first ANN for gray-level histograms, the ANN was composed of three layers consisting of input, hidden, and output layers. The ANN was composed of 16 input units, 9 hidden units, and 4 output units (Fig. 8). We used a total of 16 pieces (normalized frequency from 0 to 15 pixel values) on gray-level histograms as the input data (Fig. 2). The ANN was trained for the classification results (truth category = 0.9, other category = 0.1) [24]. In testing of ANN, when the output value was highest on the output unit of the truth category, the input data obtained from the ROI were considered as a truth category [24]. The first ANN for other texture features was composed of the same structure as the first ANN for gray-level histograms. As in the case of the first ANN for gray-level histograms, we used a total of 32 pieces (normalized frequency among ±16 pixel values), 32 pieces (16 normalized PS values for each of the principal and secondary axes), and 64 pieces (32 normalized PS values for each of the principal and secondary axes) as input data on the first ANN in the histogram of gray-level differences (Fig. 3), the GLRLM feature images (Fig. 5), and the GLCOM feature images (Fig. 7), respectively.

Second ANN Combined with Four Texture Features

Figure 9 shows the output values obtained from the first ANN for each texture feature. On the output unit of category 0 (no. 0) on the gray-level histogram, the output value of category 0 was the highest of all categories (Fig. 9a). In addition, on the output unit of category 3 (no. 3) on the gray-level histogram, the output value of category 3 was the highest of all categories (Fig. 9a). Therefore, the second ANN was trained with output patterns obtained from the first ANN for each texture feature (Fig. 10). The second ANN was composed of 16 input units, 9 hidden units, and 4 output units. The input data consisted of a total of 16 pieces (four output values obtained from the first ANN with four texture features). In the same manner as the first ANN, the ANN was trained for the classification results (truth category = 0.9, other category = 0.1) [24].

Third ANN

Figure 11 shows histograms of output values obtained from the second ANN for each case with four categories. Compared with the other categories, the histogram of category 1 was globally shifted to the right (Fig. 11b). In addition, the histogram of category 2 was globally shifted to the right (Fig. 11c). Therefore, for input data on the third ANN, we used a histogram of the output values obtained from the second ANN for each case (Fig. 12). The third ANN was composed of 40 input units, 21 hidden units, and 4 output units. Input data consisted of a total of 40 pieces (10 input values correspond to the 10 frequency values at 10 bins, from 0–0.1 to 0.9–1.0) [11]. In the same manner as the first and second ANN, the ANN was trained for the classification results (truth category = 0.9, other category = 0.1) [24].

Evaluation of Classification Performance

To investigate the effects of the number of gray levels and the distance level on each texture feature, we randomly divided the data into training data (category 0: 814 ROIs; category 1: 529 ROIs; category 2: 449 ROIs; and category 3: 544 ROIs) and non-training data (category 0: 813 ROIs; category 1: 528 ROIs; category 2: 449 ROIs; and category 3: 544 ROIs). We obtained averages of the ROC curves derived with the ten different pairs of non-training data sets. In addition, we calculated the statistical significance of differences between ROC curves using the two-tailed paired t test with each non-training data set. As shown in the Fig. 13, in the case of the ROC curve in category 3, the output values of “category 3” on a certain texture feature were regarded as “true positives,” the output values of the “other category” on a certain texture feature were regarded as “true negatives” [28]. We obtained the area under the ROC curve (AUC) on each category in this manner [28].

For evaluation of the classification performance with the third ANN method, the original database (category 0: 9 cases; category 1: 20 cases; category 2: 15 cases; and category 3: 11 cases) was divided randomly into training data and non-training data for each category. We obtained the case-based averages of the ROC curves derived with the ten different pairs of non-training data sets.

Effects of the Number of Gray Levels and the Distance Level of Texture Features

Figure 14 shows the effects of the number of gray levels and distance level on (a) the gray-level histogram, (b) the gray-level difference, (c) the GLRLM feature image, and (d) the GLCOM feature image in the first ANN. As shown in Fig. 14a, the classification performance with the gray-level histogram of the 4-bit gray level was greater than with the previous method (P < 0.05). As shown in Fig. 14b, the classification performance with the histogram of the gray-level difference (distance level: 7 pixels) was larger than with the previous method (P < 0.05). As shown in Fig. 14c, the classification performance with the GLRLM feature image on the 5-bit gray level was larger than with the previous method (P < 0.05). Finally, as shown in Fig. 14d, the classification performance with the GLCOM feature image (distance level: 3 pixels) was larger than with the previous method (P < 0.05).

With regard to the effect of the gray level, as the number of gray levels increased, the pixel values were spread and ungraded, and the texture features were decreased among the four categories [7]. On the other hand, as the gray level decreased, the pixel values were focused on some pixel value: these texture features were increased among the four categories [7]. With regard to the effect of the distance level, for segmentation of the lung in severe interstitial lung disease and for detection of diffuse lung disease on chest CT images, the distance level of GLCOM ranged from 0.50 to 0.89 mm [21, 31]. Therefore, we selected the gray-level histogram of the 4-bit gray level, the histogram of the gray-level difference on the 5-bit gray level and 7-pixel (1.23 mm) distance level, the GLRLM feature image on the 5-bit gray level, and the GLCOM feature image on the 6-bit gray level and 3-pixel (0.53 mm) distance level.

Classification Performance By Overall Scheme

Figure 15 shows the ROC curves for (a) category 0, (b) category 1, (c) category 2, and (d) category 3 in the third ANN. Table 3 shows the AUC and specificity per case at 90% sensitivity obtained with the classification method using the third ANN for category 0, category 1, category 2, and category 3. On the non-training data, the AUC value and specificity for category 0 were 0.78 ± 0.10 and 34.2 ± 28.7%, respectively. The AUC value and specificity for category 1 were 0.73 ± 0.08 and 38.3 ± 11.7%, respectively. The AUC value and specificity for category 2 were 0.79 ± 0.06 and 52.5 ± 11.3%, respectively. The AUC value and specificity for category 3 were 0.85 ± 0.09 and 60.1 ± 24.9%, respectively. The AUC values for category 0, category 1, category 2, and category 3 were 0.72 ± 0.04, 0.59 ± 0.05, 0.51 ± 0.06, and 0.56 ± 0.05, respectively, on ROI-based performance with second ANN combined with the gray-level histograms, histograms of gray-level differences, GLRLM, and GLCOM feature images. However, use the histograms of output values obtained from the second ANN for each case suggested that the case performance with the third ANN was increased.

In a database including 300 normal and 125 pneumoconiosis cases (category 1, category 2, and category 3), Yu et al. reported that AUC value, sensitivity, and specificity were 0.978, 92.0, and 87.7%, respectively [15]. In addition, in a database including 85 normal and 40 pneumoconiosis cases (category 1 and category 2), Zhu et al. reported that AUC value, accuracy, sensitivity, and specificity were 0.974 ± 0.018 and 0.929 ± 0.018, 0.957 ± 0.021, and 0.873 ± 0.024, respectively [16]. However, the lower lung field of the right (AUC: 0.698) and left (AUC: 0.563) lungs that showed lower performance was not used [16]. Including our previous studies [31, 32], these CAD systems showed relatively high performance for detection of pneumoconiosis on chest radiographs. However, as our study involved classification among the four categories with four texture features and ANN simultaneously, it was difficult to compare the performance of classification of a number of cases, such as in our study, with previous studies that involved classification between two cases, such as normal vs. abnormal.

In the case of ANN for classification of a number of cases, the performance of ANN for differential diagnosis of interstitial lung disease on chest radiographs and diffuse lung disease on HRCT showed AUC for ANN performance of 0.911 and 0.956, respectively [27, 28]. As these previous studies used patient information (sex, age, etc.) and patient interview information (incubation period for symptoms, etc.) instead of radiological findings determined by radiologists, we suggest that the performance of the ANN for differential diagnosis was higher than in our study.

Without analyzing and extracting features of the images, it was useful for recognizing images with a convolutional neural network (CNN). The CNN was used for classification of interstitial lung disease and lung nodules on chest CT images [34, 35]. The CNN required many training times for detection of specific texture features. The input data for first ANN consisted of 16, 32, 32, and 64 pieces on gray-level histograms (as shown in Fig. 2), histograms of gray-level differences (as shown in Fig. 3), GLRLM (as shown in Fig. 5), and GLCOM feature images (as shown in Fig. 7), respectively. Our computerized method was classified among the four categories with less training time than the CNN.

Wang et al. created five GLRLM features for detection of diffuse lung disease patterns based on new GLRLMs: (1) the number of short runs with relatively high CT values, (2) the number of long runs with relatively high CT values, (3) the number of short runs with low CT values, (4) the number of long runs with low CT values, and (5) the total number of runs [21]. These CAD systems for detection of pneumoconiosis and diffuse lung disease used a combination of GLRLM and GLCOM features [15, 21]. By obtaining different texture features from the gray-level histogram, the histogram of the gray-level difference, the PS values on GLRLM, and GLCOM feature images, wherever the texture patterns were placed within the ROI, were thought to obtain adequate texture features.

In the future, as training of the ANN depends on the number of cases and texture types on pneumoconiosis, for evaluating the clinical efficacy of this CAD system, we will obtain a comprehensive database including a wide distribution of patterns for each category.

Conflict of Interest

The authors declare that they have no conflict of interest.