A large-scale image dataset of wood surface defects for automated vision-based quality control processes

Introduction

In the wood industry, each step of the manufacturing process affects material utilization and cost efficiency.¹ The heterogeneity of wood material with the complexity of these manufacturing processes may result in various defects, which not only degrade the mechanical properties of the wood such as the strength and stiffness but also reduce its aesthetic value.² These mechanical and aesthetical defects have furthermore a large impact on the commercial value of the wood and can diminish the utilization of such materials for further processing. There are many various types of defects arising from many different causes. The major wood defects include knots, fungal damage, cracks, warping, slanting, wormholes, and pitch defects. The seriousness of a defect, and therefore the grade and the cost of the material, is primarily determined by four criteria, including the size, location, type of the defect, and the purpose for which the wooden product will be used.^3,4

Even though the automation in this industrial sector is growing, many market leader companies still utilize trained domain experts to detect undesirable features and to perform quality grading.⁵ Besides the fact that the manual examination is tedious and biased, it was found that domain experts are not able to check large production volumes. Moreover, the study conducted by Urbonas et al.⁶ stated that due to factors such as eye fatigue or distraction, manual inspection rarely achieves 70 % reliability. To overcome the drawbacks of the manual examination, researchers try to develop automated systems, which are accurate and won't slow down the manufacturing process. According to the repeatability and quality of the inspection, the study performed by Lycken⁷ has already proved that automatic systems slightly outperformed human graders. Most of these systems were based on conventional image processing techniques in combination with supervised learning algorithms, however, over the last decade deep learning has achieved remarkable success in the forestry and wood products industry.⁸ Although researchers in the field were able to achieve satisfying results with the average recognition rate above 90 %,⁹ most of the authors worked with small-scale image datasets obtained in laboratory conditions by using self-developed vision system setups. Performing experiments in such conditions usually entails the disadvantage of a limited number of available products. In most of the studies,^2,6,10,11 researchers compensate for the lack of real products by using data augmentation techniques, which can expand the dataset up to 10 times its original size. From one point of view, data augmentation is considered to be an excellent tool to generalize the classification model and therefore prevent overfitting.¹² Nonetheless, it cannot ensure that the variability of the observed phenomenon will be sufficiently captured, especially in cases where the variability might be limitless.

In order to address the lack of extensive databases in the field, we performed an experiment with the goal to acquire a large-scale dataset of timber surface defects. Unlike other conducted studies, our experiment was placed in an industrial environment during real production, which allowed us to acquire a large amount of authentic data from the production line. To face the challenges arising from the manufacturing process, such as the high speed of the conveyor belt and heavy vibrations, we designed hardware as well as a software solution, which enabled acquisition of high-resolution images at the acquisition rate of 66 kHz. In this experiment, we acquire 20 276 original data samples of sawn timber surface, from which 1 992 images were without any surface defects, and 18 284 images captured one or more defects covering overall 10 types of common wood surface defects. The most frequent defects include live knots and dead knots, with an overall occurrence in the dataset of 58.8 % and 41.2 %, respectively. Furthermore, to provide more valuable information in this data descriptor, all dataset samples were complemented with two types of labels: a semantic label map for the semantic segmentation and a bounding box label.

Methods

Due to the industrial environment where the experiment was set, the most challenging part of this work was the dataset acquisition. Performing data acquisition in such an environment entailed several negative factors. One of those factors was that the sawmill production line utilized for this experiment is used for more than 300 days per year, with minimal pauses, which maximizes the manufacturer's profits. Also, we had to deal with the high speed of the sawmill conveyor belt, which reached a value of 9.6 m s⁻¹ at the place of the acquisition. This high speed of the conveyor causes constant heavy vibrations that in some peaks may result in fluctuations that are even centimetres in length. The main goal of the technical solution was therefore to create a robust and at the same time portable construction, which can be easily implemented in the sawmill environment.

Acquisition equipment

To overcome the limitation of this environment, we developed a mechanical construction for carrying the camera and the light source. The final construction assembled from ITEM aluminium profiles was at the place of the acquisition fixed to the production line construction and the floor, which helped to avoid the acquisition of blurry images. Although this solution didn’t deal directly with heavy vibrations, it ensured the harmonization of the conveyor vibrations with the mounted camera. The final mechanical solution implemented in the sawmill environment is demonstrated in Figure 1.

Figure 1. The mechanical construction, including the mounted camera and light source.

The distance between the line scan camera and the light source from the conveyor belt is 40 and 15 centimetres, respectively.

In order to obtain high-quality images at a speed of 9.6 m s⁻¹, a trilinear line scan camera SW-4000TL-PMCL manufactured by JAI was chosen. This camera was able to acquire 3 × 4096 pixels per line at the speed of 66 kHz. The required speed of the image acquisition was achieved by connecting the camera interface to a high-performance Camera Link frame grabber with the transfer speed parameter set to 10 tap mode. For this application, we selected the Silicon Software microEnable five marathon VCLx frame grabber with a PCIe interface that allows on-board high-speed data processing and high data throughput up to 1800 MB s⁻¹. The required field of view, which obtains a part of the sawn timber piece, with a width of 15 cm and the full length of 500 cm was achieved by using the Kowa LM50LF line scan camera lens. The selected camera, together with a 50 mm focal length lens placed at a distance of 40 cm from the measured object, led to a horizontal resolution of 16.66 pixels per millimetre. The vertical resolution R_v of the image was computed before the experiment by the following formula.

(1)

$$R_v=\frac{1}{\frac{\frac{v_w}{60}\ast L}{v_c}}$$

where v_w is the velocity of the conveyor, L is the number of lines per image, and v_c is the line rate of the camera. The resulting vertical resolution of 6.67 pixels per millimeter was afterward experimentally verified during the acquisition process.

Since the shutter of the camera was set to 3 μs to ensure the high-speed image acquisition, we had to use a powerful light source, which would sufficiently illuminate the desired field of view. For this purpose, we selected one of the most powerful light sources on the market, a linear LED light Corona II by Chromasens with the ability to provide a light intensity of 3.5 million lux. To achieve the best possible images, a white spectrum of the light was utilized.

Data processing

During four hours of acquisition, 60 480 images were acquired overall. Due to the limited third-party software functionality, the acquisition process had to be performed in a continuous mode, without any triggering option. This resulted in a large number of images of an empty conveyor or partly captured wood surface. To filter these meaningless data from the dataset, an offline histogram-based algorithm was created. The basic idea behind this algorithm is the sum calculation of the image green color space histogram. This sum value of the histogram can be in the next step divided by any number from the range of 5 to 10 (values in the range were deduced from the size of the images). The last step of the algorithm is based on a simple threshold, where all images with a resulting value of less than 10 were removed. Using this value of threshold ensured that only images that contained in the horizontal direction at least 40 % of the wood surface were kept. Since this filtration approach proved 100 % reliable in successfully filtering images with no wooden surface on 1500 randomly selected and manually sorted samples, we applied this filtering algorithm on the whole dataset. The filtering process reduced the dataset to a final number of 20 275 images.

Additionally, besides the filtration, we performed image cropping to remove the undesirable background from the images. This operation not only reduced the file size but also decreased the potential computation time for future use. To automatically crop each image in the dataset without any relevant data loss, we employed a simple straight-line edge detection technique in a vertical direction. Basically, the main principle of the algorithm is finding as many raising edge points in the desired direction as are needed to construct a line. The cropping operation was then performed on the image bounding box derived from the following formula.

(2)

$$\mathit{BB}\left(x_1,y_1,x_2,y_2\right)=\left(\frac{L_{x_1+}{L}_{x_2}}{2}-150,L_{y_1},\frac{L_{x_1+}{L}_{x_2}}{2}+2650,L_{y_2}\right)$$

where $\mathit{BB}\left(x_1,y_1,x_2,y_2\right)$ is the cropped bounding box, and $L_{x_1{y}_2{x}_1{y}_2}$ stands for the image coordinates of the detected straight edge. Cropping the image, changed the image resolution to 2800 × 1024, and reduced the overall dataset size by almost 80 GB. An example of the image after the image crop operation is demonstrated in Figure 2.

Figure 2. A dataset example of a sawn timber surface with dead knots.

Data records

The dataset containing the data acquired in this experiment is publicly available.¹⁵ The dataset includes 1 992 images of sawn timbers without any defects and 18 283 timber images with one or more surface defects. On average, there are 2.2 defects per image, while only 6.7 % of images contain more than three defects. The highest occurrence of defects, which was captured during the experiment, was 16 defects per image. In this dataset, we present altogether 10 types of wood surface defects, including several types of knots, cracks, blue stains, resins, or marrows. An overall overview of all available wood surface defects with a number of occurrences is summarized in Table 1.

Each colour image with a resolution of 2800×1024 is provided in a BMP format in 10 separated zip folders labelled as Images.¹⁵ Additionally, we provide two types of annotations, semantic label maps, and bounding box labels. Both labels are provided in separate zip folders. The bounding box labels are located in a folder Bounding_Boxes and named as imagenumber_anno.txt, where the image number corresponds to the name of the original image in the dataset. Each original image has therefore one assigned text file, which can have multiple label records for each defect in the image. All bounding box labels have the following structure, where the first record represents the object label, and the subsequent values correspond to left, top, bottom, and right absolute positions of the defect in the image divided by 100.

Semantic label maps, used for semantic segmentation, are located in a folder, Semantic Maps. For each image in the dataset there exists just one semantic map in a BMP format with the label name in the form of imagenumber_segm.bmp, where the image number represents the corresponding name of the original image. In comparison to bounding box labels, each pixel of the semantic map image has its label, which is determined by a specified colour (see Figure 3).

Figure 3. Example of a semantic segmentation label.

The red label represents dead knots, the green label stands for live knots, and the dark yellow represents knots with cracks.

To see the exact label specification for the provided wood surface defect dataset, refer to Semantic Map Specification text file,¹⁵ or Table 2.

Technical validation

The technical validation of the dataset was conducted by assessing the quality of the assigned labels by employing deep learning-based classification. For this purpose, we utilized a standard state-of-the-art Convolution neural network detector based on the ResNet-50 model.¹⁶ The selected neural network architecture was modified by adding Batch Normalization and ReLu layers after each convolution layer. The input layer of the network, and therefore all dataset images were downsampled to 1024×357. To train the neural network, we employed a transfer learning paradigm using pre-trained weights from the COCO dataset.¹⁷ Moreover, we performed data augmentation, including horizontal, vertical flip, translation and scaling, and divided the dataset into training and testing set in a conventional ratio of 40/60. To increase the detection of the labelled defects by the ResNet-50 model, several parameters were additionally modified on the basis of the trial-and-error process. These included sizes, strides, ratios and scales (see Table 3).

At the beginning of the training, the first four layers of the network were frozen. After freezing the layers, the neural network was tuned by unfreezing the layers in a reverse order except for the Batch Normalization layer. The whole neural network was then finally fine-tuned at a low training speed. The overall number of epochs during the training was 30, while the training speed ranged between 10^-4 at the beginning and 10^-6 at the end of the training.

The trained ResNet-50 model resulted in an accuracy of 81 %. Since the neural network output a large number of false positives, the dataset was re-evaluated by a trained person who didn’t participate in the primary dataset labelling process.

Data availability

Software availability

Zenodo: Software for labeling wood surface defects and managing images. ‘Supporting tools for managing and labeling raw wood defect images’. http://doi.org/10.5281/zenodo.4904736.¹⁴

This project contains the following underlying data:

Labeler tool:

SubVI

Support Utils:

Data are available under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0).