Computers and Electrical Engineering 105 (2023) 108562

Contents lists available at ScienceDirect

Computers and Electrical Engineering

journal homepage: www.elsevier.com/locate/compeleceng

A hybrid end-to-end learning approach for breast cancer

diagnosis: convolutional recurrent network
Muhammet Fatih Aslan *
Electrical and Electronics Engineering, Karamanoglu Mehmetbey University, Karaman, Turkey

A R T I C L E I N F O A B S T R A C T

Keywords: In this study, mammography images are classified as normal, benign, and malignant using the
BiLSTM Mammographic Image Analysis Society (MIAS) and INbreast datasets. After the preprocessing of
Breast cancer each image, the processed images are given as input to two different end-to-end deep networks.
CNN
The first network contains only a Convolutional Neural Network (CNN), while the second
Deep learning
End-to-end learning
network is a hybrid structure that includes both the CNN and Bidirectional Long Short Term
Hybrid architecture Memories (BiLSTM). The classification accuracy obtained using the first and second hybrid ar­
Image preprocessing chitectures is 97.60% and 98.56% for the MIAS dataset, respectively. In addition, experiments
Mammography performed for the INbreast dataset at the study’s end prove the proposed method’s effectiveness.
These results are comparable to those obtained in previous popular studies. The proposed study
contributes to previous studies in terms of preprocessing steps, deep network design, and high
diagnostic accuracy.

1. Introduction

Cancer is a public health problem with a high mortality risk worldwide. Breast cancer is the second most prevalent type of cancer in
women. Additionally, the mortality rate is relatively high in comparison to other cancer kinds and most commonly affects people
between the ages of 20 and 59 [1]. Currently, it’s thought that 1 in 14 women will develop breast cancer [2]. Breast cancer risk for
women in the United States is roughly 13%, according to Cancer.org [3]. And to make matters worse, in roughly 80% of these people,
the disease has advanced. This reduces the success rate of treatments. Therefore, early detection of breast cancer is vital [4]. In early
diagnoses, the breast cancer mortality rate can be reduced by 80%. The two most popular imaging techniques for making the diagnosis
are mammography and biopsy [5]. Mammography is the most effective approach used to determine whether the breast tumor is benign
or malignant. In addition, ultrasound images, Thermographic, Histopathology, and Magnetic Resonance Imaging (MRI) are commonly
used for breast cancer diagnosis [6]. Specialists research these methods, and early detection is the key objective. The specialist
carefully examines the images, detects cancer, if there is any, and then distinguishes these cancer cells as benign or malignant. While
the benign tumor is not aggressive and can be removed to relieve the patient’s suffering, the malignant diagnosis has an aggressive
attitude that increases irregularly. However, the presence of the tumor or the difference between the two tumors cannot be easily
distinguished by the expert. Mammography images have disadvantages that will make it difficult for the radiologist to analyze the
images. The main disadvantage of mammography images is that the difference between dense tissue and cancerous tissue is difficult to

* Corresponding author at: Karamanoglu Mehmetbey University: Karamanoglu Mehmetbey Universitesi, Karamanoglu Mehmetbey ÿni,
Mühendi, 70100 Karaman, Turkey.
E-mail address: [email protected].

https://doi.org/10.1016/j.compeleceng.2022.108562
Received 27 July 2022; Received in revised form 20 December 2022; Accepted 21 December 2022
Available online 28 December 2022
0045-7906/© 2022 Elsevier Ltd. All rights reserved.
M.F. Aslan Computers and Electrical Engineering 105 (2023) 108562

understand. Breast cancer analysis depends on the experience of pathologists and causes wrong decisions due to factors such as fatigue,
decreased attention, and is also quite time-consuming. Computer-Aided Diagnosis (CAD)-based diagnostic methods are very impor­
tant, both for allowing experts to focus on other difficult-to-diagnose cases and to support expert decisions [7].
CAD techniques are required to give the right direction towards qualitative products for cancer diagnoses, provide uniformity in the
results during the observation process, and enhance objectivity. With recently developed image processing algorithms and artificial
intelligence techniques, small differences in dense breast images can be detected. The successful use of computer vision and artificial
intelligence in numerous applications is the cause of this rapid advancement in CAD techniques [8]. Therefore, numerous research
groups have attempted to study and propose new algorithms for breast mass classification in mammographic images. Commonly used
methods consist of feature extraction and classification steps [9–12]. Although the results from such studies are quite successful, their
performance is poor against the deep learning architectures that have been used recently. The complex structure of tasks such as
preprocessing, segmentation, feature extraction, etc. in traditional learning methods reduces the performance of the system [13]. In
order to simplify classification without utilizing these processes and to achieve high accuracy values by utilizing the rich information in
the raw image, this new area has drawn the attention of researchers.
Deep learning-based classification, diagnostic, and detection investigations in computer vision have gained popularity nowadays
and are progressing quickly. Convolutional Neural Networks (CNN) are now almost a standard in image recognition applications. In
different tasks performed on robotic, autonomous systems and mobile applications, more skilled software than humans is carried out
with deep learning-based methods. CNN, in particular, can easily distinguish fine details that people miss. However, since CNN is a new
topic of interest, its use in medical imaging methods is not sufficient. Currently, in the field of medicine, it has not yet met the expected
expectations in support of expert decisions, early diagnosis, determination of disease level, etc. Due to the high mortality rate and
frequent development of breast cancer in women, CAD in medicine is a very active study area [14].

1.1. Related Works

Recently, researchers have proposed several new methods for breast cancer detection [15]. Hai, Tan [9] performed cancer
detection on breast mammography images using an end-to-end learning algorithm. That study, in contrast to ours, extracted textural
features and used mammographic images to classify pathological grades. Setiawan, Wesley [16] aimed to increase the speed of the
proposed method by using the Region of Interest (ROI) values of the raw images in the MIAS dataset. Since ROI images contain the
cancerous region, it was aimed to powerfully distinguish abnormal and normal images using Laws’ texture energy measurements
(LAWS) features extracted from these images. The extracted features were then classified with a single hidden layer Artificial Neural
Network (ANN), resulting in the discrimination of Normal and abnormal images with 93.9% accuracy. Xie, Li [17] applied some
preprocessing to the images in the MIAS dataset and obtained the abnormal regions without noise using the ROI values. They applied
the Hough transform to automatically segment the mass contained within this ROI, thereby detecting the edges of the mass. They
extracted 32 physical features from this mass. To eliminate these features, they combined the Support Vector Machine (SVM) and
Extreme Learning Machine (ELM) classifiers. ELM accurately classified the 30 selected features with a 96.02% accuracy rate. Wang, Li
[18] applied mass detection to breast images using CNN and Unsupervised Extreme Learning Machine (US-ELM) algorithms. They
extracted the deep, morphological, texture and density features from the detected mass and made the benign / malign classification
with ELM and calculated the highest classification accuracy at 86.5%. In addition to excluding the feature extraction step, our study is
preferable because it also includes the normal class in addition to benign and malignant. Patil and Biradar [19], proposed a breast
cancer detection model from mammogram images using an optimized hybrid classifier. The tumor region was detected using the
Firefly Updated Chicken Based Chicken Swarm Optimization (FC-CSO) algorithm in the input images, whose noise was removed using
a median filter. Then, CNN and texture features were extracted from the tumor image and these features were combined. As a result,
the tumor was classified as normal, benign or malignant with a success rate of 98.49%. That study’s limitations are that instead of raw
mammography images, images of the tumor region were fed into the network, and texture features were used. Nahid, Mehrabi [20],
classified breast images as benign and malign using the BreakHis [21] dataset. They proposed CNN, Long Short-Term Memories
(LSTM) and a combination of both. Classification success rates were calculated at 91% and 96%. Although this study suggests a hybrid
structure, it differs from our study in terms of layer types in architecture, dataset and number of classes. Furthermore, although
BreakHis only includes a microscopic image of the breast tumor tissue, the Mammographic Image Analysis Society (MIAS) dataset used
in our study includes mammography of the entire breast. In a different study, Ting, Tan [22] used the MIAS dataset’s region of interest
(ROI) values to crop the images. Then the ROI is transformed into eight different patches according to the rotation angle and the flipped
position. Afterwards, these patches were trained with CNN, and each mammography image was classified as benign, malignant, and
normal with a 90.71% success rate. The most important disadvantage of that study is that the selection of ROI depends on external
data. Another study that uses the ROI values in the MIAS dataset and gives breast cancer mass images as an input to the network was
done by Jiao, Gao [23]. Unlike other studies, a joint model was introduced with both CNNs layers and parasitic metric layers. This
proposed new network structure was named the parametric metric learning net, and the accuracy value was obtained at 97.4% in a
two-class study performed as benign / malign. Using dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI), Rasti,
Teshnehlab [24] reported a CAD system for breast cancer diagnosis. For this, they made use of their dataset. For tumor localization,
they applied an automatic segmentation method based on local active contours to find the ROI from the breast MRI image. The
proposed CAD system was designed based on a mixture ensemble of CNN (ME-CNN) to discriminate between benign and malignant
breast tumors. The resulting classification success was 96.39%. Wang, Feng [25] proposed a Deep Neural Network based on
Multi-View (MV-DNN) data to improve overall mass classification performance. First, they cropped the tumor patches and performed
feature extraction using pre-trained GoogLeNet. Then they applied attention-based feature selection. They utilized LSTM to combine

2
M.F. Aslan Computers and Electrical Engineering 105 (2023) 108562

clinical information obtained from radiologist reports with visual features in order to produce a hybrid structure. As a result of ex­
periments performed with the Breast Cancer Digital Repository (BCDR) dataset, they classified mammography images as benign and
malignant by 85%. Duraisamy and Emperumal [26] presented a novel method using the Chan-Vese (C-V) level set segmentation
method and deep learning. Steps such as cropping and data augmentation were made before the deep learning step. ROI values in the
MIAS dataset were used for cropping. Mammography images were classified with 99% accuracy using the Fully Complex Valued
Relaxation Network (FCRN) classifier, which was performed after the deep learning step. Despite the high accuracy, the cropping step
before training is the weak point of this study. Houssein, Emam [27] performed a hybrid breast cancer detection study based on the
ResNet50 CNN model and an improved version of the Marine Predators Algorithm (MPA) optimization (IMPA). They first resized the
images and then increased the number of sample images to 1290 with data augmentation. They fed the augmented images into the
CNN model. IMPA aimed to find the best values for the hyperparameters of the CNN architecture. As a result, the proposed
IMPA-ResNet50 method provided a 98.88% accurate diagnosis as a result of the experimental analysis of the MIAS dataset.
To identify breast cancer from mammography images, Ragab, Attallah [28] first used preprocessing that comprised ROI and
histogram equalization steps. The scan images were then rotated at specific angles to accomplish data augmentation, creating 1288
images. They then used these images for four different experiments and got different results. In the first, only different CNN models
were classified, while in the second, deep features were classified with SVM. In the third experiment, they fused deep features and
classified them with SVM. And in the final experiment, for deep features being fused, they first reduced the feature size with Principal
Component Analysis (PCA) and then applied SVM. As a result, they had the third experiment’s success rate of 97.4%, which was the
greatest. Sannasi Chakravarthy, Bharanidharan [29] applied preprocessing steps such as cropping, histogram equalization, and edge
detection to raw mammography images for breast cancer diagnosis. They then increased the number of images in the MIAS dataset to
2576 with data augmentation. They hybridized the features extracted from four different CNN models and fed these features into the
SVM, which has three different kernel functions. At the end of the study, they achieved the highest classification success with the radial
basis function (RBF) kernel function, at 97.93%. Salama and Aly [30] first applied data augmentation to raw mammography images for
breast cancer detection. As a result of the rotation technique alone, the number of images in the MIAS dataset reached 1028. They used
the U-Net network to segment the breast region from these augmented images. The segmented regions were fed into five different CNN
models (InceptionV3, ResNet50, DenseNet121, MobileNetV2, and VGG16). As a result, the InceptionV3 model classified the
segmented regions with 96.87% accuracy. The authors performed a binary classification (Benign-Malignant) in that study. Mele­
koodappattu, Dhas [31] proposed an ensemble approach combining the CNN model and feature extraction steps to detect breast
cancer. Deep features were extracted with a 9-layer CNN model from mammography images with a preprocessing step. In addition,
Local Binary Pattern (LBP) features were extracted from preprocessed scan images and Uniform Manifold Approximation and Pro­
jection (UMAP) method was used for feature reduction. As a result of combining both features, the proposed ensemble method
achieved 98% success for the MIAS dataset. They achieved this accuracy for binary classification. Muduli, Dash [32] designed a simple
CNN architecture for breast cancer diagnosis. For mammography images, they first applied pre-processes such as scaling, cropping,
and ROI. They used data augmentation techniques such as scaling, rotation, and flipping for the pre-processed images, resulting in
3200 images. That study, which applied binary classification, provided 96.55% accuracy on the MIAS dataset with the proposed CNN
architecture. El Houby and Yassin [33] applied image preprocessing steps such as noise removal, contrast enhancement, and cropping
to raw mammography images for breast cancer detection. They implemented data augmentation for the pre-processed images
belonging to the malignant class and conducted experiments with a total of 530 images. Finally, they designed a CNN architecture for
the processed and augmented images and achieved 95.3% success. For raw mammography images, Karthiga, Narasimhan [34] used a
variety of preprocessing techniques, including Contrast Limited Adaptive Histogram Equalization (CLAHE), cropping, polynomial
curve fitting for segmentation, etc. These processed images were fed into the AlexNet, VGG19 and VGG16 pre-trained CNN models. The
VGG19 model with the Stochastic Gradient Descent with Momentum (SGDM) optimizer provided 94.39% accuracy for the MIAS
dataset.

1.2. Aim and Contributions of This Paper

More examples can be given of the above-mentioned studies on breast cancer detection and classification. In the results section of
this paper, a table has been added to compare the proposed method with previous studies. In fact, the results of the proposed studies
differ depending on the method, preprocessing steps, data augmentation, and dataset. This type of work aims to make the method used
uniform and to interpret the results more objectively.
The purpose of this study is to use mammography images to identify breast cancer. The original mammography images in the MIAS
dataset are first preprocessed, and then the output images are given to the deep learning network. Two types of end-to-end deep
learning networks are designed for training and classification. The first is the CNN network, and the second is the CNN + Bidirectional
Long Short-Term Memories (BiLSTM) network, which extracts features based on temporal relationships.
Considering Section 1.1., the differences/superiorities of this study over previous studies are as follows: The second architecture
(CNN+BiLSTM) is one of the contributions of this study. Furthermore, one of the most important differences between this study and
others is that it does not use the ROI value. Therefore, pre-training tumor area boundaries are not required, only the presence of a raw
mammography image is sufficient. Moreover, some studies classify only diseased tumors as benign and malignant. This study can also
detect normal breast mammography images. These show the differences of this study in terms of application/methodology compared
to previous studies. However, the main subject of this study is to design a different deep architecture from previous studies, to provide a
more successful breast cancer detection on the same dataset (MIAS) and to achieve this success in a very simple way.
The main innovation, which is the motivation of this study, is to propose a new end-to-end architecture to increase the success of

3
M.F. Aslan Computers and Electrical Engineering 105 (2023) 108562

Fig. 1. Some images of the MIAS and INbreast dataset.

existing CNN models. For this reason, AlexNet, a simpler CNN model, is preferred in experiments. At the end of the study, it is evident
that adding BiLSTM layers to an existing deep network with a simple structure can easily increase its success. In this way, a successful
diagnosis of breast cancer is achieved with a simple end-to-end architecture, without the need for different manual application steps
such as feature extraction and optimization, which were applied to increase the success in previous studies. It is quite simple in terms of
implementation, only the preprocessed images are fed into the trained hybrid network. In addition, this study shows that the success of
existing CNN models can be increased with additional BiLSTM layers. In addition to the MIAS dataset, the proposed method has also
been tested with the INbreast dataset to prove its effectiveness. In general, the contributions/superiorities of this study can be sum­
marized as follows:

1) For breast cancer diagnosis, a deep network with a CNN+BiLSTM hybrid structure is proposed.
2) Since the designed architecture has an end-to-end structure, there is no need for a manual feature extraction step, and it is easy to
implement.
3) Thanks to the pre-processing steps applied, ROI values are not required.
4) Three classes (normal, benign and malignant) are used for breast cancer diagnosis.
5) Considering the proposed methods and classification accuracy, this study is superior to many other modern studies.

This paper is organized as follows: After the introduction, Section 2 introduces the datasets used in the experimental analysis and
the proposed deep architectures for these datasets. Section 3 explains the application steps of the proposed method in detail. The
results of the proposed method and the comparison of these results with previous studies are discussed in Section 4 with controversial
language. Section 5 highlights the advantages and disadvantages of the proposed method and accordingly explains future work plans.
Finally, Section 6 concludes the study.

2. Methodology

This section first introduces the MIAS and INbreast datasets used for cancer diagnosis, then provides information on deep learning-
based architectures.

2.1. MIAS Dataset

The proposed method has been tested using the MIAS [35] database. This dataset contains mammography images (1024 × 1024)
with a total of 322 taken from 161 women. Each image is in grayscale and has a horizontal and vertical resolution of 96 dpi. The dataset
contains annotations for these 322 images. Breasts in the image are labeled as normal or abnormal. If there is an abnormality, this
abnormality has a location (ROI values) and a benign-malignant label. In the dataset, each image is also labeled according to the breast
density. Accordingly, fatty, glandular and dense information for each breast image is also available. In Fig. 1 (a), some images of the
MIAS data are shown with the label information.

4
M.F. Aslan Computers and Electrical Engineering 105 (2023) 108562

Fig. 2. Structure of CNN-based deep learning architecture.

Fig. 3. Structure of CNN-BiLSTM (Hybrid) based deep learning architecture.

2.2. INbreast Dataset

After the proposed method was applied to the MIAS dataset, the INbreast dataset [36] was also used to prove the robustness of the
method. The INbreast dataset was obtained with the permission of the Ethics Committee at the Chest Center in Porto. The resulting
mammography images had dimensions of 2560 × 3328 or 3328 × 4084 pixels. Images from a total of 115 cases include benign,
malignant and normal cases (see Fig. 1 (b)). In the dataset, there are 220 benign, 67 Normal and 49 Malignant mammography images
[37].

2.3. Architecture

This study aims to provide cancer detection from breast mammography images by using end-to-end learning-based deep learning
architectures. Thanks to end-to-end learning, there is no feature extraction step as in traditional learning methods. In traditional
machine learning methods, firstly, the features that will provide the distinction in the data should be determined, and the redundant
features should be removed. However, at this stage, it is also possible to remove useful features that distinguish between classes. It is
also very difficult to obtain a feature vector that fully represents a class. In such cases, as in the end-to-end approach, feeding the raw
image to the network provides richer information. In traditional methods, since the determination of features is the most important
task, the most important step that determines the result is feature extraction. However, since this step is not used in end-to-end
learning, architectural design that determines the features comes to the fore. Convolution layers, Rectified Linear Unit (ReLU) acti­
vation function and Pooling layers in the designed architecture contain mathematical calculations for feature extraction, nonlinear

5
M.F. Aslan Computers and Electrical Engineering 105 (2023) 108562

Fig. 4. Implementation steps of the proposed algorithm.

transformation and size reduction, respectively. In this way, different transformations are applied through different layers to the
features extracted using convolution layers. Apart from these, dropout, normalization, etc. layers can also be used. As a result, the fully
connected layer prepares these extracted features for classification.
In this paper, two different deep learning networks that diagnose breast cancer from mammography images are designed. More
precisely, the second architecture is created in a hybrid structure to enhance the results after those achieved utilizing the first ar­
chitecture. Cropped images obtained as a result of preprocessing are given to both architectures as input. The first architecture shown
in Fig. 2 includes only the CNN structure. A CNN network consists of many layers. Each layer has its own specific purpose and these
layers can be repeated with different parameters as part of the CNN network. In Fig. 2, it can be seen that the same layers containing
different parameters are used more than once. The convolution layer (Conv.) extracts features from each image by applying sliding
convolutional filters to the input images. The batch normalization layer (Batch Norm.) normalizes each input channel according to the
mini-batch value, resulting in features divided into mini-batch sizes and normalized. The ReLU layer is the activation function
frequently used in deep learning architectures due to its advantage in terms of speed. Thanks to the ReLU function, only for positive
values, neurons are activated, which speeds up the training algorithm. The Average Pooling (Average Pool.) layer represents the
sampling process that takes the average values. The fully connected layer (Fully Con.) establishes the connection using the weight and
bias values between the inputs and the outputs. Finally, Softmax classifies multi-class data as probabilistic and is widely preferred in
deep learning architectures [38]. Depending on these explanations, Fig. 2 can be examined in three steps: 1) Input image, 2) Feature
Extraction, 3) Classification. More detailed information about this architecture will be described in Section 3.
The second architecture designed for breast cancer detection is created by adding additional layers to the first architecture. As seen
in Fig. 3, the additional layers are one flatten, two BiLSTM and two dropout layers. Features in the output of Fully Connected
Layer+ReLU are converted into a one-dimensional array with the Flattening layer. Thus, the features extracted with the CNN network
are prepared for the input of the BiLSTM layer. The BiLSTM layer takes the features extracted from the first architecture as input data
and extracts features. The dropout layer after the BiLSTM layers is intended to improve training performance.
Recurrent Neural Networks (RNNs) are designed to extract features from ordered data and are therefore often preferred for
modeling dynamic systems. The longer the link to the past states in the RNN, the less the effect of the information from the past states
(vanishing gradient problem). Long Short-Term Memories (LSTM) [39] have become very popular recently as a way to solve the
vanishing gradient problem and provide a solution for long sequences. BiLSTM [40] is an enhanced version of LSTM. To access both
previous and subsequent information, BiLSTM combines the forward hidden layer and the backward hidden layer. While LSTM uses
only past information, BiLSTM is based on both past and future information. Therefore, BiLSTM has a high generalization ability and
provides a stronger prediction. Studies have shown that BiLSTM has a better ability to solve sequential modeling tasks than LSTM [41].
There is no sequential relationship or dependency among the image samples used in breast cancer detection. The output or pre­
diction is simply a function of the current inputs. That is, the previous inputs have no relation to the current output. Recently, re­
searchers have favored RNN-based network designs for this kind of data [42]. Although the LSTM and BiLSTM networks were designed
to analyze temporal data, they also showed high performance in non-temporal or non-sequential data [43–47]. Recurrent paths are
ignored if RNN-based architectures are used for non-sequential data. In this case, recurrent structures perform operations unneces­
sarily. Although these structures are redundant, they do not adversely affect the prediction performance of the network. In this sense,
RNN-based BiLSTM and LSTM structures can also be used for non-sequential data in terms of feature diversity. In this context, this
study also does not use temporal data but makes use of the powerful feature extraction capability of BiLSTM. Due to the BiLSTM layer,
an additional step is added to Fig. 2, as shown in Fig. 3. The second architecture consists of 1) Input image, 2) Feature Extraction, 3)
Temporal Feature Extraction and 4) Classification steps. The parameters of each of the layers of this architecture will be discussed in
detail in Section 3.

3. Proposed methodology

All the actions necessary to carry out the suggested method are thoroughly explained in this section. Each application step of the
proposed method is shown in Fig. 4. In Fig. 4, the algorithms used, numerical parameters, proposed architectures, etc. information for
each step are explained under four subtitles.

3.1. Preprocessing

The uniformity of the images to be given as input to the deep learning network increases the success of the network. In the pre­
processing step applied for this purpose, the images are made suitable for learning algorithm input. Because of this, any preprocessing
that was done on the original mammography images before they were uploaded to the deep learning network is noted there. The raw

6
M.F. Aslan Computers and Electrical Engineering 105 (2023) 108562

Fig. 5. Image preprocessing steps applied to original breast mammography images.

Table 1
Parameter limits of the methods applied in data augmentation.
Lower bound Upper bound

Gaussian noise Mean 0.005 0.015

Variance 0.001 0.005
Salt & Pepper noise Noise density 0.01 0.03
Sharpen filter Radius 2 5
Amount 1 3
Gaussian Filter Kernel Size 1 9

mammography images, as shown in Fig. 1, contain both noise and different directions of the breasts (right, left). To eliminate these
differences, the preprocessing steps shown in Fig. 5 are carried out. The sample images shown in Fig. 5 belong to the MIAS dataset.
Each preprocessing step performed is summarized as follows. First, the threshold value is determined, and thresholding is per­
formed according to this value. The original image’s mask is obtained in this manner. In image processing, morphological processes are
generally applied during segmentation to remove defects in the images. Morphological processes called erode and dilate are carried out
to eliminate the noise caused by the scanning process in the mammography image. The breast image given in Fig. 5 is in the left
direction. In this study, the left breasts are mirrored so that all breasts are in the same direction. Therefore, the next preprocessing step
is mirroring and this step is not found in the preprocessing steps for the right breast. The mirrored original image and the mirrored
noise-free mask image are multiplied because the next goal is to obtain the original mirrored noise-free image. As a result of this step, as
seen in Fig. 5, the left breast is now in the right direction and the existing noise in the original image is removed. The next step is to
determine the limits of the breast so that the learning algorithm can produce faster and more accurate results because the black regions
outside the breast (background) are unimportant to the learning algorithm. The final step involves cropping using these border lines.
The cropped images in the last step are now suitable for feature extraction and classification. The size of each cropped image is 512 ×
1024.

3.2. Data Augmentation

Having a lot of samples is one of the strategies to improve deep learning success. However, the number of data is not always at the
desired amount. When it is impossible to obtain a lot of data, data augmentation is a crucial alternative method. Data augmentation is

7
M.F. Aslan Computers and Electrical Engineering 105 (2023) 108562

Fig. 6. New images resulting from data augmentation.

Table 2
Number of samples in classes before and after data augmentation.
Benign Malign Normal

Before the data augmentation 64 51 207

After the data augmentation 320 306 414

to generate new data for training step by processing the original datasets. Thus, the size of the dataset to be used in train is increased
and the predictions become stronger [48]. In studies involving deep learning and computer vision, data augmentation is very common.
There are many techniques for image data augmentation, such as geometric transformations, flipping, operations on color space,
cropping, zooming, rotating, adding noise, filtering, mixing multiple images and deleting random regions. All of these techniques can
be used, but only special techniques are preferred depending on the structure of the study [49].
In this study, in order to increase the number of breast mammography images with the data augmentation techniques, white gauss
noise and salt & pepper noise are added at a random rate between the limits given in Table 1. In addition, Sharpen and Gaussian filters
are applied respectively to change the sharpness and smoothness of mammography images. While determining these types of data
augmentation techniques, those that would not affect the tumor disease or tissue density are preferred. Table 1 shows the limits of the
parameters of these data augmentation techniques. By changing the parameters in the upper and lower limit range, new images can be
created from the original images. Fig. 6 shows a preprocessed mammography image and new images produced from this image using
data augmentation techniques in described Table 1.
In practice, after the preprocessing step, data augmentation is performed on the whole mammography image. In order to ensure
that the number of Benign, Normal and Malignant mammogram images in the new augmented dataset is also close, different rates of
data augmentation have been made for each class. The sample image numbers of classes in the new dataset before and after data
augmentation are given in Table 2.

3.3. CNN-Based Classification

In this section, information is given about the hyperparameters of the CNN network architecture (see Fig. 2), which is designed in
an end-to-end approach structure. In recent years, thanks to its ability to learn various distinctive features, CNN has been used

8
M.F. Aslan Computers and Electrical Engineering 105 (2023) 108562

Table 3
Hyperparameters of CNN network layers, and training options.
CNN Layers
Layer Name Filter Size Activation Function Padding Stride Batch Normalization Output Channel

Conv-1 3 ReLU 0 1 True 8

Conv-2 3 ReLU 0 1 True 16
Conv-3 3 ReLU 0 1 True 32
Conv-4 3 ReLU 0 1 True 32
FC Layers
Layer Name Output Size State Activation Function ReLU Drop Out
FC-1 25 tanh True False
FC-2 3 tanh True False
Training Options
Optimization Algorithm Epoch Mini Batch Size (MBS) Learning Rate (α) Momentum (γ)
SGDM 15 50 0.001 0.95

Fig. 7. Confusion matrices of proposed methods (for MIAS dataset).

frequently in various computer vision applications such as image classification, object detection, semantic segmentation, etc. Unlike
traditional learning methods, CNN performs feature extraction and classification/prediction [50].
As seen in Fig. 2, noise-free cropped images with the same size obtained by pre-processing are given as input to the CNN layer. The
sizes of the input images are multiplied by 0.25 for faster training. That is, the size of each mammogram image given as an input is 128
× 256. The features extracted from these images and the success of classification directly depend on the hyperparameters of the layers
used in architecture. In the CNN architecture shown in Fig. 2, Table 3 lists the hyperparameters for the four convolutional (Conv-1,
Conv-2,...) and two fully connected (FC-1, FC-2) layers.
Thanks to the convolutional layer, images are converted into feature vectors. A fully connected layer is used to transfer these
feature vectors to a neural network. Normal, Benign, and Malign classification is performed with the Softmax layer, which evaluates
the results as probabilistic. Since the number of classes in the output is three, the output size of the FC-2 layer before Softmax is three.
Since the FC-1 layer is used in the second architecture shown in Fig. 3, it is also used here for an accurate comparison. In the FC-1 and
FC-2 layers, the tanh activation function is preferred for training. The hyperparameter values used for both FC layers and CNN layers
are determined by trial and error. According to Table 3, the value of MBS that enables the training data to be divided into small groups
is 256, and the optimization algorithm that provides the training by reducing the error is SGDM. Table 3 also shows the parameter
values of the SGDM algorithm used.
θl+1 = θl − α∇E(θl ) + γ(θl − θl− 1 ) (1)
Eq. (1) is the parameter update equation of the SGDM algorithm. The purpose of this equation is to decrease the next error value by
updating the weight values according to the Loss function (E(θl)). For this, using small steps constrained by the learning rate (α), the
global minimum error value is sought at each iteration in the direction of the negative gradient of the Loss function. The back­
propagation technique is used to update the weight values throughout each iteration of the search for the optimum value.The value
that determines the relationship between the weight values between the iterations is the Momentum (γ) coefficient. For training the
designed network, the parameter values specified in Table 3 are used in Eq. (1). As a result of the training, the results calculated in the
classification process with the test data are shown in Fig. 7 and Table 5 in the result section.

9
M.F. Aslan Computers and Electrical Engineering 105 (2023) 108562

Table 4
Hyperparameters of hybrid network layers, and training options.
BiLSTM 1 BiLSTM 2 FC 1
Number of State Activation Gate Activation Number of State Activation Gate Activation Output Size State Activation
hidden units Function Function hidden units Function Function Function

125 tanh sigmoid 100 tanh sigmoid 3 tanh

Training Options
Optimization Gradient Decay Squared Gradient Maximum Mini Batch Size Learning Epsilon (є)
Algorithm Factor (β1) Decay Factor (β2) Epoch Rate (α)
8
Adam 0.9 0.999 200 100 0.001 10−

Table 5
Performance metrics of the proposed methods (for the MIAS dataset).
Architecture Acc. (%) Recall Specificity Precision F1-score MCC

CNN 97.60 0.9804 0.9889 0.9735 0.9761 0.9648

CNN+BiLSTM 98.56 0.9882 0.9933 0.9836 0.9856 0.9786

3.4. CNN-BiLSTM (Hybrid)-based Classification

This section provides details on the CNN-BiLSTM network architecture’s hyperparameters, which were constructed using the Fig. 3
structure. The second network with a hybrid structure is created by adding the BiLSTM layer to the previous network. With BiLSTM,
the relationship between previous outputs and existing inputs is addressed and this relationship is determined in two ways, from past to
future and from future to past. BiLSTM, which differs from LSTM in this aspect, provides more successful results and generally learns
faster than LSTM.
Since the BiLSTM algorithm is designed for sequential data, feature matrices (or feature vectors) are first created from preprocessed
images. These matrices are created by the CNN network. Therefore, as seen in Fig. 3, the second network includes both CNN and
BiLSTM layers (CNN-BiLSTM). CNN has the advantage of obtaining global information. On the other hand, a BiLSTM can take
advantage of its long-term dependencies between data. Hybrid CNN and BiLSTM models are used to combine both of these benefits into
a single algorithm [20]. The training options and network hyperparameters of the designed CNN-BiLSTM hybrid network are shown in
Table 4. Since the CNN architecture structure is the same as in Section 3.3, the values in Table 3 also apply to the CNN network in the
hybrid architecture.
As seen in Table 4 and Fig. 3, the hybrid architecture includes two consecutive layers of BiLSTM (BiLSTM 1, BiLSTM 2). The fully
connected layer (FC-1) is given as input the temporal features extracted by the BiLSTM layers. Finally, the classification is completed
with the Softmax function. Parameter values of the number of hidden layer neurons, activation functions, etc. are found by trial and
error method. Using the last layer FC-1, temporal features are represented by three different classes. The Adam optimization algorithm
[51] has reduced the error during each iteration of the training phase. The Adam optimization algorithm, which is specially designed
for training deep neural networks, is an adaptive learning algorithm. Low memory requirement makes the Adam algorithm preferable
[51]. The Adam algorithm is a combination of SGDM and Root Mean Squared Propagation (RMSprop) [52] optimization methods.
Adam has a parameter update similar to the RMSProp algorithm, however, it also includes the term momentum. The parameter update
formula for the Adam algorithm is given between Eqs. (2)-(4). The Adam method modifies the learning rate for each neural network
weight by using the first and second moments of gradient estimations. In other words, the expected value of the first and second power
of the term to be updated is used.
ml = B1 ml− 1 + (1 − β1 )∇E(θl ) (2)

vl = B2 vl− 1 + (1 − β2 )[∇E(θl )]2 (3)

αml
θl+1 = θl − √̅̅̅̅ (4)
vl + є

m: Gradient moving averages

v: Squared gradient moving averages
θ: Network parameter to be updated
∇E(θ): Gradient of the loss function
β1: Gradient Decay Factor
β2: Squared Gradient Decay Factor
є: Epsilon
α: Learning Rate
l: Iteration Number
By using the parameter values in Table 4 in the Adam optimization algorithm between Equality 2-4, weights are updated in each

10
M.F. Aslan Computers and Electrical Engineering 105 (2023) 108562

Table 6
Comparative analysis with previous studies using the MIAS dataset.
Previous Work Dataset Method* Number of Parameters* Class Train- Acc.
Samples Test (%) (%)

Setiawan, Wesley [16] MIAS LAWS&ANN - Hidden nodes: 10 A,N 70-15 93.90
Xie, Li [17] MIAS ELM 115 - B,M - 96.02
Ting, Tan [22] MIAS CNN - - B,M, - 90.50
N
Houssein, Emam [27] MIAS IMPA&ResNet50 1290 - A,N 70-30 98.88
Ragab, Attallah [28] MIAS Deep Feature Fusion & 1288 LR: 10− 3 Iter. num.:104 MB size:4 B,M 80-20 97.40
SVM Num. Ep.: 20 Opt: SGDM
Sannasi Chakravarthy, MIAS Deep Feature Fusion & 2576 LR: 10− 3 Iter. num.:104 MB size:4 B,M, 70-30 97.93
Bharanidharan [29] SVM-RBF Num. Ep.: 10 Opt: SGDM N
Salama and Aly [30] MIAS Inception V3 & U-Net 1028 LR: 10− 2 Iter. num.:105 Num. Ep.: 80 B,M 80-20 96.87
Melekoodappattu, Dhas [31] MIAS Ensemble CNN - - B,M - 98.00
Muduli, Dash [32] MIAS CNN 3200 LR: 10− 4 Iter. num.:16x103 MB B,M 80-20 96.55
size:25 Num. Ep.: 100 Opt: SGDM
El Houby and Yassin [33] MIAS CNN 530 LR: 10− 3 MB size:90 Opt: Adam B,M 80-20 95.3
Karthiga, Narasimhan [34] MIAS VGG19 3816 LR: 10− 4 Iter. num.:4800 Num. Ep.: B,M 80-20 94.39
30 Opt: SGDM
Proposed Algorithm MIAS CNN-BiLSTM 1040 LR: 10− 3 MB size:50,100 Num. Ep.: B,M, 80-20 98.56
15,200 Opt: SGDM, Adam N

* LR: Learning Rate, Iter. num.: Iteration number, MB: Mini-Batch, Num. Ep.: Number of Epochs, Opt: Optimizer, ANN: Artificial Neural Network,
ELM: Extreme Learning Machine, kNN: K Nearest Neighbors, A: Abnormal, N: Normal, B: Benign, M: Malignant, LAWS: Law’s Texture Energy
Measure, GA: Genetic Algorithm, BP: Back Propagation

iteration during the training phase. After the iteration number limit has been reached and the training has been completed, the ac­
curacy values determined using the test data are shown in Fig. 7 and Table 5 in the result section.

4. Results

The training options and hyperparameters of the networks described in Section 3 are used, and the performance of both networks is
determined using test data. The confusion matrices obtained with the MIAS dataset and showing the classification success are shown in
Fig. 7. Also, various performance measures proving the success of the network are shown in Table 5. Formulas are defined for all
metrics in Eq. (5)-Eq. (10). The results calculated according to these formulas are shown in Table 5.
tp + tn
Accuracy = x100 (5)
tp + fp + tn + fn

tp
Precision = (6)
tp + fp

tp
Recall = (7)
tp + fn

tn
Specificity = (8)
tn + fp

2tp
F1 − score = (9)
2tp + fp + fn

(tp ∗ tn) − (fn ∗ fp)

MCC = √̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅ (10)
(tp + fn) ∗ (tn + fp) ∗ (tp + fp) ∗ (tn + fn)

tp: True Positive

tn: True Negative
fp: False Positive
fn: False Negative
The results in Table 5 show that both architectures successfully diagnose breast cancer. However, thanks to the BiLSTM layers
added to the CNN network, the classification success rate increases. The accuracy values for the designed first (CNN) and second
(CNN+BiLSTM) networks are 97.60% and 98.56%, respectively. Deep models classified both Benign (B) and Malignant (M) image
samples with 100% accuracy. However, CNN and CNN+BiLSTM models classify 5.9% and 3.5% of the Normal (N) samples as Benign
(B), respectively. As a result, CNN+BILSTM performs a more accurate estimation. Additionally, Table 5 shows that classification using
a hybrid architecture results in higher values for Recall, Specificity, Precision, F1-Score, and MCC. While these values for CNN are
0.9804, 0.9889, 0.9735, 0.9761, 0.9648, respectively, these values for CNN+BiLSTM are 0.9882, 0.9933, 0.9836, 0.9856, 0.9786,

11
M.F. Aslan Computers and Electrical Engineering 105 (2023) 108562

Fig. 8. Confusion matrices of proposed methods (for INbreast dataset).

respectively. As a result, the hybrid CNN+BiLSTM network outperforms the CNN network thanks to its better performance and un­
biased classification abilities.
The results in Table 5 show that breast cancer detection in mammogram images can be classified successfully using only raw
images. As explained in Section 1.1, although most of the previous studies applied deep learning, they also performed a manual feature
extraction step. In addition, some studies only detect the tumor in the image and perform training only with the tumor region (ROI).
Some studies classify benign and malign using only the diseased breast image. This study has the advantages of convenience, such as
the absence of manual feature extraction steps and the direct use of original mammogram images without the need for ROI values.
Additionally, it can be noted that the suggested approach performs better than the majority of earlier studies when compared to the
breast cancer classification studies carried out using the MIAS dataset in Table 6.
Besides the results of previous studies, Table 6 also contains different information. Most of the studies in the table are current and
therefore involve deep learning. The number of samples is therefore very important. In addition, parameter information, which has a
big impact on how well learning algorithms train, also has an impact on the study’s result. Numerous studies have used only two
(binary) classes (A, N or B, M) for breast cancer. As in this study, having more classes (B, M, N) provides a more useful and
comprehensive application. The train-test rate, which is another piece of information in Table 6, is also effective for the result of the
study. Generally, the preferred rate is 70%-30% or 80%-20%. As seen in Table 6, a comparison with exactly the same conditions is not
very possible. Generally, each study prefers different parameters, training rates, etc. for the learning algorithm. For this reason,
comparisons are usually made over accuracy values.
Comparison based on accuracy only is not enough. In addition to the diagnostic accuracy shown in Table 6, a more comprehensive
analysis is made if previous studies are compared in different aspects, especially in computational complexity. The low computational
complexity is important for the intelligibility and preferability of the proposed method. Out of two models with comparable accuracy,
the one with the lowest computing complexity is chosen. Apart from this, memory space and diagnostic speed also significantly affect
the applicability of the method. These are also directly related to computational complexity. Setiawan, Wesley [16]’s manual feature
extraction from smaller images (128 × 128) using ROI values and classification with a single hidden layer ANN reduced the
computational complexity of the proposed study. The method, therefore, requires little memory space and runs fast. However, not
every dataset has ROI values, so the method’s dependence on ROI values limits its use. In this context, the proposed method has an
important limitation. Also, extracted manual features may not be strongly representative of classes. The dependence of that study by
Xie, Li [17] on the ROI values limits the study area, similar to the previous study. In addition, the Hough transform, feature extraction,
and feature selection also increase the computational complexity and reduce the speed of the method. The authors classified data using
the ELM approach, which has a very quick processing speed, to make up for this speed loss. The manual extraction of 32 features is
another drawback of the study.The study by Ting, Tan [22] also depends on the ROI values in the dataset, similar to previous studies.
Extracted features were high-level, as the authors adopted the end-to-end structure. But the CNN that was designed had 29 layers.
Therefore, it has more computational complexity than the CNN+BiLSTM structure designed in our study. Although Houssein, Emam
[27] provided a high diagnostic accuracy, the computational complexity was much higher than in our study. Because the ResNet50
used is more complex. In addition, the use of the optimization algorithm adversely affected the working speed of the method. Ragab,
Attallah [28] developed a method based on ROI values. In addition, extracting features from GoogleNet, ResNet-18, ResNet50 and
ResNet101 models and then combining these features greatly increased the computational complexity of that study. The study by
Sannasi Chakravarthy, Bharanidharan [29] had a high computational complexity as it extracted features from AlexNet, ResNet50,
GoogleNet and DenseNet121 models, similar to the previous study. Salama and Aly [30],s U-Net-based segmentation and Inception V3
steps increased the study’s reliability and accuracy, however, they also significantly raised the computational complexity. Melekoo­
dappattu, Dhas [31]’s proposal for a 9-layer CNN model had a low computational complexity. The application of feature reduction and
the extraction of texture features, however, slowed down that experiment. Furthermore, that study also depended on ROI values.
Muduli, Dash [32] designed a very simple and low computational CNN consisting of four convolution layers and one fully connected

12
M.F. Aslan Computers and Electrical Engineering 105 (2023) 108562

Table 7
Performance metrics of the proposed methods (for INbreast dataset).
Architecture Acc. (%) Recall Specificity Precision F1-score MCC

CNN 91.67 0.8833 0.9366 0.8754 0.8789 0.8166

CNN+BiLSTM 92.26 0.8621 0.9295 0.9122 0.8853 0.8263

layer. Moreover, as a result of cropping mammography images with ROI, the CNN model input data was quite small (128 × 128). The
limitations of that study compared to ours are that it depends on the ROI values and only distinguishes between benign and malignant
types. With a straightforward CNN they created, El Houby and Yassin [33] classified ROI images only into malignant and
non-malignant categories. In this context, this study had little computational cost, but only two species classification was a disad­
vantage. Finally, Karthiga, Narasimhan [34] provided 94.39% accuracy on the MIAS dataset for the SGDM optimizer. The VGG19
architecture, on the other hand, had a very high computational complexity. By modifying the AlexNet model (8 layers), which has a
straightforward design, and then feeding the extracted features from this CNN model into the BiLSTM layers, our approach offers a
diagnosis that is significantly more accurate than any of these earlier research. Since BiLSTM layers take single-column matrices as
input, they have very fast feature processing capability. Also, in the proposed method, the raw data is input data, so no cropping was
done with the ROI values. However, the proposed method provided a high accuracy for MIAS with three classes (Benign, malignant and
normal). As a result, when accuracy, speed, and computational complexity are thoroughly addressed, the proposed method performs
better than earlier methods.
In addition, in order to prove the robustness of the method proposed in this study, an experiment is also performed on the INbreast
dataset. Each of the mammography images in the INbreast dataset (see Fig. 1(b)) is preprocessed as in the MIAS dataset. Then, the
number of pre-processed images is increased with the data augmentation techniques shown in Fig. 6. As a result of data augmentation,
the total number of processed leaf images increases to 1680. 80% of these images are used for training and the rest is reserved for
testing. The confusion matrix containing the results of the CNN and CNN+BILSTM networks as a result of the training and testing steps
is shown in Fig. 8. In addition, other performance metrics are shared in Table 7. As a result, the average classification accuracy for the
CNN and CNN+BILSTM structures for the INbreast dataset images is 91.67% and 92.26%, respectively. These accuracy values support
the results from the MIAS dataset. However, the accuracies obtained for the INbreast dataset are lower than the MIAS dataset. For the
INBreast dataset, CNN and CNN+BiLSTM are very successful for benign samples, with success rates of 95% and 97.1%, respectively.
The most incorrect classification for both models occurs for Normal samples. These error rates for CNN and CNN+BILSTM are 19.7%
and 21.2%, respectively. When the Recall, Specificity, Precision, F1-Score and MCC metric values in Table 7 are examined, it is seen
that both models are quite successful. However, CNN+BiLSTM provides a more successful classification. For CNN, these metrics have
the following values: 0.8833, 0.9366, 0.8754, 0.8789, and 0.8166. For CNN+BiLSTM, these metrics have the following values: 0.8621,
0.9295, 0.9122, 0.8853, and 0.8263.

5. Discussion and future works

The absence of a feature extraction step, proposed deep learning architectures, the classification of normal breast tissues, and high
diagnostic accuracy are the study’s contributions. Compared to many previous studies, the strength of our study is that it can classify
the raw mammography images easily and with high success. This enables detection of breast cancer entirely independent of humans.
Also, since the proposed algorithm is useful, this algorithm can be applied to breast images obtained with different imaging methods.
For this reason, in medical applications in a similar field, researchers can successfully use the proposed architecture. As mentioned in
the introduction, some studies have focused on the detection of the tumor area. Our method only carries out cancer classification; the
mammography image’s tumour area is not marked. This can be considered the study’s weakness because it is essential for fully
automated decision support. Another weakness may be the parameter values in the proposed architecture. These values should be
optimally determined based on classification success. Although the CNN-BiLSTM structure proposed in this study increases the success
compared to CNN, it causes additional parameter adjustments and time loss. A more complex architecture is obtained compared to the
existing CNN network. In addition, preprocessing applied to raw mammography images may not produce efficient results for different
datasets. Applications should be developed on various and large datasets so that the study can produce more reliable and compre­
hensive results.
The complicated structure of human organs makes it impossible to make a general conclusion from a single method or a few
datasets, despite the development of artificial intelligence-based approaches. Therefore, different types of images and large numbers of
samples are essential for artificial intelligence studies. Thankfully, the public dataset has increased recently. However, the increase in
datasets brings along the problems of reliability and uniformity. In addition, while it is pleasing that the data augmentation techniques
applied in the studies increase the accuracy of artificial intelligence, its contribution to the reliability of the diagnostic results, in
general, is controversial. Therefore, unsupervised or semi-supervised approaches that require less labeled data are noteworthy. New
algorithms to be developed to overcome these weaknesses are the focus of our future work.
In future studies, hyperparameter optimization methods will be used to provide a stable classification and higher accuracy. Another
method planned to increase classification accuracy is to use the breast density information in the MIAS database for cancer detection.
Also, for a more comprehensive and fully automated system, both cancer detection and marking of the cancer area are planned. To
reduce data dependency, unsupervised or semi-supervised methods will be developed in the future. Newly developed methods will be
used to analyze 3D mammography images as well as 2D mammography images. Finally, it is planned to increase success with new pre-

13
M.F. Aslan Computers and Electrical Engineering 105 (2023) 108562

Table 8
List of symbols used in this paper.
Symbols Description

α Learning Rate
γ Momentum
θl Network parameter to be updated
E(θl) Loss function
m Gradient moving averages
v Squared gradient moving averages
∇E(θ) Gradient of the loss function
β1 Gradient Decay Factor
β2 Squared Gradient Decay Factor
є Epsilon
l Iteration Number
tp True Positive
tn True Negative
fp False Positive
fn False Negative

processing steps. This paper feeds a large number of nodule-independent pixels into the deep network. In future studies, only an image
of the tumor region will be obtained from the raw mammography images after preprocessing.These images will be analyzed by
artificial intelligence algorithms.

6. Conclusion

In this study, cancer detection was performed using mammography images in the MIAS dataset. An end-to-end approach has been
proposed. In this context, Convolutional and hybrid (CNN + BiLSTM) network designs have been developed. The raw images were first
processed with the preprocessing step, and then the data augmentation step was applied for the preprocessed images since the number
of original data was low and the distribution in the classes was irregular. The augmented images were fed to CNN and hybrid network
and trainings were carried out. Then, in the test step, mammography images were classified with high accuracy. The hybrid archi­
tecture performed a more successful classification when the results were examined
The proposed method has important practical implications in real life. For example, when different mammography images are fed
directly into the trained network via the computer, it generates an estimate for diagnosis. Meanwhile, the specialist who interprets the
mammography images can also benefit from the artificial intelligence-based diagnostic result. Therefore, it can be used as a decision-
support mechanism. In this way, more accurate and faster disease diagnoses can be provided. In addition, these artificial intelligence-
based decisions can be produced with a smartphone. In addition to the decision support system, the preprocessing steps applied to the
images can make it easier for the specialist to detect the nodule in the image. The image preprocessing applied in this study also makes
the existing nodule more noticeable. The proposed method can also be applied in diagnostic applications involving various medical
images because it makes a methodological contribution.
Symbols used in this Paper: Table 8 summarizes the list of symbols used in this article.

Declaration of Competing Interest

The author declares that there is no conflict of interest.

Data availability

The data used is public, there is an access address in the article.

References

[1] Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for
36 cancers in 185 countries. CA: Cancer J Clin 2018;68:394–424.
[2] Subashini T, Ramalingam V, Palanivel S. Automated assessment of breast tissue density in digital mammograms. Comput Vision Image Understanding 2010;114:
33–43.
[3] Cancer.org. How Common Is Breast Cancer?, 2020.
[4] Yan R, Ren F, Wang Z, Wang L, Zhang T, Liu Y, et al. Breast cancer histopathological image classification using a hybrid deep neural network. Methods 2020;
173:52–60.
[5] Gecer B, Aksoy S, Mercan E, Shapiro LG, Weaver DL, Elmore JG. Detection and classification of cancer in whole slide breast histopathology images using deep
convolutional networks. Pattern Recognit 2018;84:345–56.
[6] Nahid A-A, Kong Y. Involvement of machine learning for breast cancer image classification: a survey, 2017. Comput Math Methods Med; 2017.
[7] Sudharshan P, Petitjean C, Spanhol F, Oliveira LE, Heutte L, Honeine P. Multiple instance learning for histopathological breast cancer image classification.
Expert Syst Appl 2019;117:103–11.

14
M.F. Aslan Computers and Electrical Engineering 105 (2023) 108562

[8] Li H, Zhuang S, Li D-a, Zhao J, Ma Y. Benign and malignant classification of mammogram images based on deep learning. Biomed Signal Process Control 2019;
51:347–54.
[9] Hai J, Tan H, Chen J, Wu M, Qiao K, Xu J, et al. Multi-level features combined end-to-end learning for automated pathological grading of breast cancer on digital
mammograms. Comput Med Imaging Graphics 2019;71:58–66.
[10] Punitha S, Amuthan A, Joseph KS. Benign and malignant breast cancer segmentation using optimized region growing technique. Future Comput Inf J 2018;3:
348–58.
[11] Rouhi R, Jafari M, Kasaei S, Keshavarzian P. Benign and malignant breast tumors classification based on region growing and CNN segmentation. Expert Syst
Appl 2015;42:990–1002.
[12] Chen Z, Strange H, Oliver A, Denton ER, Boggis C, Zwiggelaar R. Topological modeling and classification of mammographic microcalcification clusters. IEEE
Trans Biomed Eng 2014;62:1203–14.
[13] Khan S, Islam N, Jan Z, Din IU, Rodrigues JJC. A novel deep learning based framework for the detection and classification of breast cancer using transfer
learning. Pattern Recognit Lett 2019;125:1–6.
[14] Abdelrahman L, Al Ghamdi M, Collado-Mesa F, Abdel-Mottaleb M. Convolutional neural networks for breast cancer detection in mammography: A survey.
Comput Biol Med 2021;131:104248.
[15] Litjens G, Sánchez CI, Timofeeva N, Hermsen M, Nagtegaal I, Kovacs I, et al. Deep learning as a tool for increased accuracy and efficiency of histopathological
diagnosis. Sci Rep 2016;6:26286.
[16] Setiawan AS, Wesley J, Purnama Y. Mammogram classification using law’s texture energy measure and neural networks. Procedia Comput Sci 2015;59:92–7.
[17] Xie W, Li Y, Ma Y. Breast mass classification in digital mammography based on extreme learning machine. Neurocomputing 2016;173:930–41.
[18] Wang Z, Li M, Wang H, Jiang H, Yao Y, Zhang H, et al. Breast cancer detection using extreme learning machine based on feature fusion with CNN deep features.
IEEE Access 2019;7:105146–58.
[19] Patil RS, Biradar N. Automated mammogram breast cancer detection using the optimized combination of convolutional and recurrent neural network. Evol
Intell 2020:1–16.
[20] Nahid A-A, Mehrabi MA, Kong Y. Histopathological breast cancer image classification by deep neural network techniques guided by local clustering, 2018.
Biomed Res Int; 2018.
[21] Spanhol FA, Oliveira LS, Petitjean C, Heutte L. A dataset for breast cancer histopathological image classification. IEEE Trans Biomed Eng 2015;63:1455–62.
[22] Ting FF, Tan YJ, Sim KS. Convolutional neural network improvement for breast cancer classification. Expert Syst Appl 2019;120:103–15.
[23] Jiao Z, Gao X, Wang Y, Li J. A parasitic metric learning net for breast mass classification based on mammography. Pattern Recognit 2018;75:292–301.
[24] Rasti R, Teshnehlab M, Phung SL. Breast cancer diagnosis in DCE-MRI using mixture ensemble of convolutional neural networks. Pattern Recognit 2017;72:
381–90.
[25] Wang H, Feng J, Zhang Z, Su H, Cui L, He H, et al. Breast mass classification via deeply integrating the contextual information from multi-view data. Pattern
Recognit 2018;80:42–52.
[26] Duraisamy S, Emperumal S. Computer-aided mammogram diagnosis system using deep learning convolutional fully complex-valued relaxation neural network
classifier. IET Comput Vision 2017;11:656–62.
[27] Houssein EH, Emam MM, Ali AA. An optimized deep learning architecture for breast cancer diagnosis based on improved marine predators algorithm. Neural
Comput Appl 2022.
[28] Ragab DA, Attallah O, Sharkas M, Ren J, Marshall S. A framework for breast cancer classification using multi-DCNNs. Comput Biol Med 2021;131:104245.
[29] Sannasi Chakravarthy S, Bharanidharan N, Rajaguru H. Multi-deep CNN based experimentations for early diagnosis of breast cancer. IETE J Res 2022:1–16.
[30] Salama WM, Aly MH. Deep learning in mammography images segmentation and classification: Automated CNN approach. Alexandria Eng J 2021;60:4701–9.
[31] Melekoodappattu JG, Dhas AS, Kandathil BK, Adarsh KS. Breast cancer detection in mammogram: combining modified CNN and texture feature based approach.
J Ambient Intell Hum Comput 2022.
[32] Muduli D, Dash R, Majhi B. Automated diagnosis of breast cancer using multi-modal datasets: A deep convolution neural network based approach. Biomed
Signal Process Control 2022;71:102825.
[33] El Houby EMF, Yassin NIR. Malignant and nonmalignant classification of breast lesions in mammograms using convolutional neural networks. Biomed Signal
Process Control 2021;70:102954.
[34] Karthiga R, Narasimhan K, Amirtharajan R. Diagnosis of breast cancer for modern mammography using artificial intelligence. Math Comput Simul 2022;202:
316–30.
[35] SUCKLING J P. The mammographic image analysis society digital mammogram database. Digital Mammo 1994:375–86.
[36] Moreira IC, Amaral I, Domingues I, Cardoso A, Cardoso MJ, Cardoso JS. INbreast: Toward a Full-field Digital Mammographic Database. Acad Radiol 2012;19:
236–48.
[37] Maqsood S, Damaševičius R, Maskeliūnas R. TTCNN: A Breast Cancer Detection and Classification towards Computer-Aided Diagnosis Using Digital
Mammography in Early Stages. Appl Sci 2022;12:3273.
[38] Paluszek M, Thomas S. Practical MATLAB Deep Learning: A Project-Based Approach. Apress; 2020.
[39] Hochreiter S, Schmidhuber J. Long Short-Term Memory. Neural Comput 1997;9:1735–80.
[40] Graves A, Schmidhuber J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural networks 2005;18:
602–10.
[41] Liu G, Guo J. Bidirectional LSTM with attention mechanism and convolutional layer for text classification. Neurocomputing 2019;337:325–38.
[42] M. Paul, R.K. Karsh, F.A. Talukdar. Image hashing based on shape context and speeded up robust features (SURF). 2019 International Conference on
Automation, Computational and Technology Management (ICACTM). IEEE, 2019. p. 464-8.
[43] Aslan MF, Unlersen MF, Sabanci K, Durdu A. CNN-based transfer learning–BiLSTM network: A novel approach for COVID-19 infection detection. Appl Soft
Comput 2021;98:106912.
[44] Q. Cong, Z. Feng, F. Li, Y. Xiang, G. Rao, C. Tao. XA-BiLSTM: a deep learning approach for depression detection in imbalanced data. 2018 IEEE International
Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 2018. p. 1624-7.
[45] Sabanci K, Aslan MF, Ropelewska E, Unlersen MF, Durdu A. A Novel Convolutional-Recurrent Hybrid Network for Sunn Pest–Damaged Wheat Grain Detection.
Food Anal Methods 2022.
[46] Islam MZ, Islam MM, Asraf A. A combined deep CNN-LSTM network for the detection of novel coronavirus (COVID-19) using X-ray images. Inf Med Unlocked
2020;20:100412.
[47] Zainudin Z, Shamsuddin SM, Hasan S. Convolutional Neural Network Long Short-Term Memory (CNN + LSTM) for Histopathology Cancer Image Classification.
Singapore: Springer Singapore; 2020. p. 235–45.
[48] T. Ko, V. Peddinti, D. Povey, S. Khudanpur. Audio augmentation for speech recognition. Sixteenth Annual Conference of the International Speech
Communication Association, 2015.
[49] Shorten C, Khoshgoftaar TM. A survey on image data augmentation for deep learning. J Big Data 2019;6:60.
[50] Hu Z, Tang J, Wang Z, Zhang K, Zhang L, Sun Q. Deep learning for image-based cancer detection and diagnosis− a survey. Pattern Recognit 2018;83:134–49.
[51] D.P. Kingma, J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:14126980. (2014).
[52] Tieleman T, Hinton G. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine
learning 2012;4:26–31.

Muhammet Fatih Aslan is a research assistant at Karamanoglu Mehmetbey University (KMU), Karaman, Turkey. He completed his master’s degree at Selcuk University
in 2018. He completed his Ph.D. in Electrical and Electronic Engineering at Konya Technical University in 2022. His research interests include autonomous mobile
robotics, image processing, artificial intelligence, biomedical signal and image processing.

15