Enhancing brain tumor diagnosis: an optimized CNN hyperparameter model for improved accuracy and reliability

Dataset details

This study utilized the two brain tumor MRI datasets publicly available at Kaggle. The dataset 1 comprises a total of 7,023 images of the human brain having dimensions of 512 $\times$ 512 and JPG format. It consists of four classes: glioma (1,621), meningioma (1,645), no tumor (2,000), and pituitary (1,757). The “no tumor” class images were sourced from the Br35H dataset. Figure 2 illustrates the different categories present in the dataset, including no tumors, meningioma, pituitary, and glioma. The dataset 2 consists of 253 images with yes (155) and no (98) classes. To train and evaluate the hyperparameter-tuned model for brain tumor detection, the datasets were divided into training, validation, and testing sets for dataset 1 and 80:20 ratio of training and testing for dataset 2. A detailed description of the dataset can be found in Tables 2 and 3.

Pre-processing

Pre-processing a brain tumor MRI dataset involves several steps to optimize the data for analysis and modeling. This includes addressing challenges like varying resolutions and intensity ranges in MRI images. Rescaling the images to a standardized resolution ensures consistency across the dataset while normalizing intensity values enhances subsequent algorithms and models. Techniques like rescaling pixel values to a specific range (0, 1) or using z-score normalization are commonly employed. Aligning the MRI images to a standard reference frame is essential due to variations in position and orientation. Image registration techniques facilitate spatial alignment, enabling improved comparison and analysis. Additionally, addressing noise and artifacts in MRI images caused by factors like patient motion or equipment variations is crucial. Denoising techniques like Gaussian smoothing or non-local means filtering effectively reduce noise while preserving vital details. These pre-processing steps optimize the brain tumor MRI dataset, enabling accurate analysis and meaningful insights.

Hyperparameters of convolutional neural networks for training

Hyperparameters in CNNs are parameters not learned during the training process but instead set by the user prior to training. The input layer is the parameter where the CNN training process starts, and the classification layer is where it ends the process in a feed-forward fashion. Nevertheless, the opposite process begins with classification and moves through the first convo layer. Neuron J sends information in a forward fashion computed according to Eq. (1) to the value of neurons N in layer L. The non-linear ReLU function determines the output, as indicated in Eq. (2).

(1) $$I{P}_N^L=\sum _{J=1}^N{W}_N{J}^L{x}_j+b_N$$

(2) $$O{P}_N^L=max(0,I{P}_N^L)$$where IP: input, OP: output, W: weight, and $b$: neuron number. All neurons use Eqs. (1) and (2) to construct output results and form the non-linear activation function by taking input values. The pooling layer employs a k × k window to gather the results to calculate maximum average feature values. Equation (3) explains how to perform this calculation using the SoftMax function for each tumor type.

(3) $$O{P}_N^L=\frac{e^{I{P}_N^L}}{{\sum }_N{e}^{O{P}_k^L}}$$

The back-propagation cost function is calculated by minimizing the new weights, as presented in Eq. (4).

(4) $$C=-\frac{1}{S}\sum _N^{S}IP(X(a^i/b^i))$$

In the training process, the letter S represents the training set sample, while $b^i$ represents the ith sample of the training set with its corresponding label $a^i$. The probability of classification, denoted as $X(a^i/b^i)$, is used to minimize the cost C through the stochastic gradient function. To calculate the weights of each convolutional layer L, the weight of the convolutional layer L at iteration t is represented by $W_L^t$, as depicted in Eq. (5).

(5) $$W_L^{t+1}=W_L^t+V_L^{t+1}$$

Here $W_L^t$: weight of the convolutional layer L, $V_L^{t+1}$ is the updated weight value at iteration $t$. The essential component of convolutional neural networks, feature extraction, is made possible by the convolutional layer. Several filters to extract features are present in this layer. The resultant value and layer sizes are evaluated by using Eqs. (6) and (7); correspondingly, here $n_L^i$ is the resultant feature map of the images, $\sigma$ is the function of activation, $y_L$ is the input width and $x_L^i\in f,z^i\in f$ is the filter $(f)$ channels.

(6) $$n_L^i=\sigma (x_L^i-y_L+z^i)$$

(7) $$Convolutionallayeroutputsize=\left(\frac{inputsize-filtersize}{stride}\right)+1$$

A convolutional neural network often employs the pooling layer after each convolutional layer. This layer manages the parameters, which are also in charge of overfitting. The most widely used pooling layer is max pooling, which performs distinct functions from other layers like min pooling and average pooling. Equations (8) and (9) are used to determine the output and size of the pooling layer when x is the output and R is the pooling region, respectively.

(8) $$Pl{o}_{i,j}=\underset{r,s\in R}{max}$$

(9) $$Poolinglayeroutputsize=\left(\frac{convolutionallayeroutputsize-PoolingSize}{stride}\right)+1$$

Fine-tuning of CNN hyperparameters

As depicted in Algorithms 1 and 2, optimizing various aspects of the network’s architecture and the training procedure is necessary to fine-tune the hyperparameters of a CNN for brain tumor detection, classification. The quantity and size of filters control convolutional layer depth and receptive field. Consider alternative numbers, such as (16, 32, 64), and different filter sizes, like [3 × 3, 5 × 5]—altering the stride value to regulate the output feature maps’ spatial dimensions. The spatial resolution is decreased with a more excellent stride, whereas the spatial resolution is increased with a smaller stride. The input volume’s spatial size is preserved during convolutional processes by choosing the suitable padding parameters. Padding may reduce the impact of margins without retaining crucial spatial information.

The varied pooling methods used during the tests, such as min, max, or average pooling, aid in capturing invariant information and lower the spatial dimensions. ReLU, sigmoid, and tanh are three appropriate activation functions for the CNN layers that introduce non-linearity and allow the network to model complicated relationships in the input. A lower learning rate may necessitate more repetitions but can produce better convergence. In contrast, a more significant learning rate may result in faster convergence but runs the risk of overshooting the ideal solution. The experiment with various batch sizes will decide how many training examples will be processed in each iteration. Different network depths can be achieved by changing the number of convolutional and fully connected layers. Regularization techniques like dropout or L2 regularization improve generalization and reduce overfitting. Various optimization techniques, which regulate how the network’s weights are changed during training, are being tested, including stochastic gradient descent (SGD) and Adam.

Working of hyperparameter CNN

Hyperparameteric CNN refers to a CNN model that utilizes hyperparameters to configure its architecture and optimize its performance. The grid search hyperparameters create a grid of possible values for each hyperparameter that we want to tune including the number of filters (num_filters), number of units (num_units), dropout rate (dropout), and optimizer. These hyperparameters are crucial in determining the model capacity, regularization, and learning behavior. As shown in Figs. 3 and 4 for Dataset 1 and Dataset 2, the given data depicts that the different combinations of hyperparameter values are tested, and their corresponding accuracies are recorded, tested, and their corresponding accuracies are recorded. The hyperparameter values are varied for num_filters, num_units, dropout, and optimizer, while the accuracy represents the model’s performance with those specific hyperparameter settings.

Evaluation criterion

Several statistical equations are used to test the proposed model for classifying and detecting brain tumors Eqs. (10)–(13). True positive images are those correctly classified and denoted as $Tp$, while true negative images are those incorrectly classified as negative and denoted as $Tn$. $Fp$ stands for the number of incorrectly positive classified images, while $Fn$ stands for incorrectly negative classified images. The statistical metrics accuracy, precision, recall, and F1-score were used to gauge the model output. The F1-score was used to evaluate the outcomes when there was a conflict between accuracy and sensitivity, informative and technical.

(10) $$Precision(Pre)=\frac{Tp}{Tp+Fp}$$

(11) $$Recall(R)=\frac{Tp}{Tp+Fn}$$

(12) $$F1-score(F1)=\frac{2(Se\times Pre)}{Se+Pre}$$

(13) $$Accuracy(Acc)=\frac{Tp+Tn}{Tn+Tn+Fp+Fn}$$

Model results

The confusion matrix of the test data for the four-class (dataset 1) and two-class (dataset 2) classification is shown in Figs. 5 and 6. The test dataset 1 includes four categories: meningioma, pituitary, no tumor, and glioma while dataset 2 contains yes and no class. The confusion matrix is a square matrix with dimensions equal to the number of classes. In this scenario, with four classes, the confusion matrix is a $n\times n$ matrix; where $n$ is the number of classes. Each cell in the matrix represents a combination of predicted and actual class labels. The numbers within the matrix represent the total number of images utilized for classification. Each entry in the matrix corresponds to the count of images that belong to a specific actual class (represented by rows) and were predicted to belong to a specific predicted class (represented by columns).

The model’s ability to generalize new data is measured by the validation and training accuracy, as shown in Figs. 7 and 8, which is determined using a different dataset not used during training. The x-axis represents the number of epochs, and the y-axis represents the accuracy. The graph has two lines, one for training accuracy and one for validation accuracy. The training accuracy line is a dashed blue line, and the validation accuracy line is a dotted orange line. The training accuracy starts at around 0.75 and increases steadily to around 0.98. The validation accuracy starts at around 0.8 and increases steadily to around 0.95. It helps identify overfitting and evaluates the model performance on real-world data. A model may have overfitted to the training data and not generalized well if training accuracy is high, but validation accuracy is noticeably lower. Validation Loss denotes the inconsistency between the predictions of the model and the actual targets in the validation dataset, as shown in Figs. 9 and 10 for dataset 1 and dataset 2. It acts as a gauge of the model’s effectiveness using previously unobserved data. It is estimated using a loss function, like training loss, and the objective is to minimize it to improve the model’s accuracy on the novel, untried samples.

The ROC curve explains the concession among the true positive (sensitivity) and the false favorable rates (specificity minus sensitivity) for various categorization thresholds, as shown in Figs. 11 and 12 for dataset 1 and dataset 2. A point on the curve shows the model’s performance at each threshold, which represents a different threshold setting. As the threshold changes, the actual positive rate is drawn against the false positive rate to form the curve. The AUC value calculates the total effectiveness of the model. The likelihood that a casually designated positive sample will be graded higher than a negative sample is represented by this parameter. The range of AUC values is 0 and 1, with higher values representing improved performance and discrimination capacity.

The statistical information in Tables 4 and 5 summarizes brain tumor classification, and detection performance. Impressive results are displayed in the table, including an average precision, recall, and F1-score of 0.94, showing remarkable accuracy in detecting brain tumors. The accuracy of 0.96 shows the model’s overall performance in accurately classifying cases of brain tumors. At the same time, the AUC value of 0.99 indicates outstanding discrimination abilities due to the fine-tuning of hyperparameters of CNN for dataset 1.

In dataset 2, for the ‘Yes’ class, the precision, recall, and F1 score are all 0.90, indicating a consistent performance in predicting this class. The Support for ‘Yes’ is 31, implying that this class appeared 31 times in the dataset. The AUC for ‘Yes’ is also 0.90, indicating good discrimination ability for this class.

The ‘No’ class shows slightly lower precision, recall, and F1 score at 0.85 but has an AUC of 0.10, which seems unusually low. Typically, AUC values are between 0.5 and 1. This is a point of interest, possibly indicating issues with the model’s ability to distinguish the ‘No’ class correctly. The ‘Average’ row displays the mean values of precision, recall, and F1 score for both classes, which are all 87.5. The average AUC is 0.5, suggesting good overall discrimination ability across both classes.

Figure 13 show the accuracy values comparison for different hyperparameters of the CNN model. The accuracy values occurred against several filters (num_filters), number of units (num_units), dropout rate (dropout), and optimizer. Two optimizers are used: Stochastic Gradient Descent (SGD) and Adaptive Moment Estimation (Adam). Adam is an adaptive optimizer that dynamically adjusts the learning rate, while SGD uses a fixed one. The highest accuracy value of 96% shows an occurrence rate of 11.1%, and the lowest accuracy value is 75, with an occurrence rate of 2.8%. Generally, the accuracy occurrence range a maximum value of 19.4% for both 92% and 95%.

For dataset 2, the accuracy values range from 0.39 to 0.90, indicating the correctness of our proposed model as shown in Fig. 14. The associated percentages vary, showcasing the distribution of how frequently each accuracy level occurs within the dataset. For instance, an accuracy of 0.61 is associated with a high percentage of 27.6%, suggesting a relatively low precision in that specific case. Conversely, some values, like 0.81, 0.82, 0.85, and 0.86, exhibit consistent accuracy percentages of 3.4%, implying more stable performance. Intriguingly, accuracies of 0.39, 0.73, 0.76, 0.84 and 0.90 have percentages of 6.9%, potentially indicating areas of interest or significance within the context of the data. Moreover, there are instances where relatively high accuracy levels, such as 0.88, are accompanied by a 10.3% percentage, possibly suggesting that precision is balanced with occurrences.

Table 6 presents a comprehensive comparison of various brain tumor classification methods, revealing their respective accuracy scores. Each method’s accuracy value indicates the percentage of correctly classified brain tumor images in the dataset. Notably, the “Fine-tuned ResNet-50 with CNN” achieves an accuracy of 0.95, leveraging the combined strengths of the fine-tuned ResNet-50 model and a CNN architecture. Similarly, the “Hybrid approach” attains an accuracy of 0.90 by employing a combination of traditional machine learning algorithms and deep learning models, enhancing classification robustness. The “U-Net with fine-tuned ResNet50” method achieves an accuracy of 0.94, benefiting from U-Net’s segmentation capabilities in conjunction with a fine-tuned ResNet-50 model. The “Hybrid Ensemble” and “CNN Ensemble” methods both achieve an accuracy of 0.95 through the ensemble of diverse models and techniques, resulting in enhanced accuracy. The “Proposed model” stands out with the highest accuracy of 0.96, outperforming all other methods, potentially advancing brain tumor diagnosis in medical imaging applications. In one of our experiments, we conducted tests on a smaller dataset comprising 253 images from two distinct classes. Despite the reduced data size, our system demonstrated remarkable performance, achieving an accuracy of 88%. This outcome underscores the robustness and versatility of our hyperparameter tuning approach in scenarios where data availability is constrained.