applied

sciences
Article
Multiclass Skin Lesion Classification Using a Novel
Lightweight Deep Learning Framework for Smart Healthcare
Long Hoang 1 , Suk-Hwan Lee 2 , Eung-Joo Lee 3 and Ki-Ryong Kwon 1, *

1 Department of Artificial Intelligence Convergence, Pukyong National University, Busan 48513, Korea;
[email protected]
2 Department of Computer Engineering, Dong-A University, Busan 49315, Korea; [email protected]
3 Division of Artificial Intelligence, Tongmyong University, Busan 48520, Korea; [email protected]
* Correspondence: [email protected]; Tel.: +82-51-629-6257

Abstract: Skin lesion classification has recently attracted significant attention. Regularly, physicians
take much time to analyze the skin lesions because of the high similarity between these skin lesions.
An automated classification system using deep learning can assist physicians in detecting the skin
lesion type and enhance the patient’s health. The skin lesion classification has become a hot research
area with the evolution of deep learning architecture. In this study, we propose a novel method
using a new segmentation approach and wide-ShuffleNet for skin lesion classification. First, we
calculate the entropy-based weighting and first-order cumulative moment (EW-FCM) of the skin
image. These values are used to separate the lesion from the background. Then, we input the
segmentation result into a new deep learning structure wide-ShuffleNet and determine the skin
lesion type. We evaluated the proposed method on two large datasets: HAM10000 and ISIC2019.
Based on our numerical results, EW-FCM and wide-ShuffleNet achieve more accuracy than state-of-
the-art approaches. Additionally, the proposed method is superior lightweight and suitable with a



small system like a mobile healthcare system.


Citation: Hoang, L.; Lee, S.-H.; Lee, Keywords: skin lesion classification; medical image processing; deep learning; artificial intelligence;
E.-J.; Kwon, K.-R. Multiclass Skin big data; wide-ShuffleNet; mobile healthcare system
Lesion Classification Using a Novel
Lightweight Deep Learning
Framework for Smart Healthcare.
Appl. Sci. 2022, 12, 2677. https:// 1. Introduction
doi.org/10.3390/app12052677
Skin lesions, which are irregular skin changes compared to the neighboring tissue,
Academic Editor: Francesco Bianconi can evolve into skin cancer, one of the most dangerous cancers. There are two main
Received: 10 January 2022
types of skin cancer: nonmelanoma and melanoma. Melanoma lesions are responsible
Accepted: 1 March 2022
for the significant increase in mortality and morbidity in recent years; they are the most
Published: 4 March 2022
destructive and dangerous among various lesion types [1]. If the physicians detect the
lesions earlier, they can increase the curing rate to 90% [2]. Moreover, visual inspection
Publisher’s Note: MDPI stays neutral
for skin cancer is complex because of high similarity among different skin lesion types
with regard to jurisdictional claims in
(e.g., nonmelanoma and melanoma), leading to misdiagnosis. A solution for healthcare
published maps and institutional affil-
systems [3] and image inspection [4] is using the automatic classification of lesion pictures
iations.
by machine learning (ML).
Presently, 132,000 melanoma skin lesion cases and approximately three million non-
melanoma skin lesion cases occur yearly in the world. Furthermore, 60,000 people died
Copyright: © 2022 by the authors.
due to prolonged sun exposure (12,000 nonmelanoma and 48,000 melanoma), according
Licensee MDPI, Basel, Switzerland. to the World Health Organization. Approximately 80% of skin cancer mortalities occur
This article is an open access article with melanoma lesions [5]. Besides long sun exposure, a record of sunburn has been
distributed under the terms and linked to the development of skin cancer, especially melanoma. In the beginning grades,
conditions of the Creative Commons patient survival rates can be improved if melanoma is identified correctly [6]. To handle
Attribution (CC BY) license (https:// the interobservation differences, technicians are guided to recognize melanoma manually.
creativecommons.org/licenses/by/ Consequently, an automatic classification system can enhance the precision and efficiency
4.0/). of the early discovery of this cancer type.

Appl. Sci. 2022, 12, 2677. https://doi.org/10.3390/app12052677 https://www.mdpi.com/journal/applsci

Appl. Sci. 2022, 12, 2677 2 of 21

Melanoma is similar to benign moles in their early stages of development; thus, it

is not easy to distinguish malignant and benign (even for qualified dermatologists [7]).
Several methods have been proposed to solve these problems, including handcrafted and
artificial intelligence (AI) approaches.
First, low-level properties, such as border, color, and visual texture features. were used
to separate melanoma and nonmelanoma lesions [8]. Celebi et al. [9] used shape, color,
and texture features, but experiences with huge observed intraclass similarity led to low
results. Segmentation is another approach to drop unwanted features and background,
as presented in [10]. Tommasi et al. [10] used a binary mask to segment the images and
a support vector machine (SVM) to classify the segmented images. The segmentation
process in [11] calculates the thresholding values using Gabor filter masks, which yields
poor outcomes.
The second approach is AI, an area with numerous possible utilizations, such as
mining, ecology, urban planning. AI is subdivided into ML and deep learning (DL) [12].
ML builds algorithms to identify data and obtain predictions [13–15]. DL can study
related features of images and extract the features with various architectures. Additionally,
DL is highly efficient for big data investigation [16–22]. One of the DL prototypes is a
convolutional neural network (CNN), which has presented an excellent performance in
video and images processing with the development of graphics processing unit (GPU)
computing systems. CNN is a powerful mechanism for bioimage examination based on a
recent study [23–25]. Hence, it allows the high potential in melanoma classification [26,27].
Moreover, CNN ensemble approaches have shown success for this classification task [28].
Skin cancer is extremely common, and early detection is crucial. Although computer-
aided diagnostic tools have been extensively studied, they still lack clinical practice.
ML (and particularly DL) models have demonstrated great promise in skin lesion clas-
sification tasks; however, some challenges limit their adoption: data availability for
some lesion categories and the requirement of trained professionals handling equipment
(e.g., dermatoscopy to collect and annotate the data). Thus, developing powerful but
efficient models that can run in decentralized devices (e.g., smartphones) is critical.
Conventional DL methods require high parameters and are not suitable for a portable
system. Therefore, creating a framework for a mobile system is a challenging task. Unlike
previous methods, we propose a novel approach for skin lesion classification that uses low
parameters while keeping high accuracy. The proposed method is appropriate for a portable
system like a mobile healthcare system. Our framework combines a novel segmentation
technique and a new wide-ShuffleNet to classify skin lesions. The proposed segmentation
method increases the segmentation accuracy compared with previous methods and helps
the network detect the skin lesion object better boost the classification process.
The contributions of this study are as follows:
• We propose a novel method to segment the skin image using the entropy-based
weighting (EW) and first-order cumulative moment (FCM) of the skin image.
• A two-dimensional wide-ShuffleNet network is applied to classify the segmented
image after applying EW-FCM. To the best of our knowledge, EW-FCM and wide-
ShuffleNet are novel approaches.
• Based on our numerical results on HAM10000 and ISIC2019 datasets, the proposed
framework is more efficient and accurate than state-of-the-art methods.
The remainder of the paper is organized as follows. We explore the related works in
Section 2. In Section 3, we present the proposed method. Section 4 presents the numerical
results and analysis. Finally, Section 5 presents the conclusion and future studies.
Appl. Sci. 2022, 12, 2677 3 of 21

2. Related Works
There are two strategies of skin classification: ML and DL methods [29].

2.1. ML Approaches
K-nearest neighbor (KNN) is a supervised ML algorithm used in predictive and
forecasting models [30]. The accuracy of the KNN algorithm is considerably good [31].
Sajid et al. [32] proposed another KNN-based automated skin cancer diagnostic system.
Their proposed system employed a median filter to remove the image noise using a
collection of statistical and textural features. Textural features were extracted from a curvelet
domain, whereas statistical features were extracted from lesion images. Furthermore, the
proposed framework classified the input images into noncancerous or cancerous. The
KNN model requires a long time to perform the output predictions, and it is unsuitable
for a big dataset. Moreover, the KNN algorithm operates worst with improper feature
information of high dimensional input data, making the algorithm unsuitable for the skin
lesion classification [33].
Alam et al. [34] applied SVM to discover eczema. The approach in [35] manipulates
several steps: image segmentation, feature determination applying texture-based data, and
finally deciding the type of eczema with SVM. Upadhyay et al. [36] extracted orientation
histogram, gradient, and location of skin lesion features. These features were fused and
classified as malignant or benign using an SVM algorithm. The SVM algorithm is unsuitable
for managing the noisy input image [37]. If the number of training samples is less than the
number of feature vector parameters, SVM gives a lower performance.
The Bayesian algorithm is another approach used in skin lesion classification with
multiple trained skin image databases [38]. Performing the Naïve Bayes algorithm for mul-
tiobjective areas is not easy [39]. The decision tree [40] model has been widely used for skin
lesion classification, forecast of under limbs lesions, and cervical disease. Arasi et al. [41]
presented intelligence techniques, decision tree, and Naïve Bayes to diagnose malignant
melanoma. The extracted features are based on principle component analysis and hybrid
discrete wavelet transform. These features become the input to different classification
methods, such as decision tree and Naïve Bayes, for classifying the lesions as benign or ma-
lignant. The decision tree algorithm demands big training data for achieving considerable
accuracy. Moreover, the decision tree model requires a large amount of memory and more
computational time [42].

2.2. DL Approaches
There are two DL classification strategies for skin classification: non-segmentation [43]
and segmentation approaches.

2.2.1. Non-Segmentation DL Approaches

Menegola et al. [44] applied six open datasets for lesion image classification. They used
Google Inception-v4 and ResNet-101 architectures to detect seborrheic keratosis, malignant
melanoma, and nevus. They also confirmed that combined datasets increase training data
and improve classification accuracy. Han et al. [45] introduced Resnet-152 for the lesions
image classification. The lesions include squamous cell carcinoma, basal cell carcinoma,
actinic keratosis, intraepithelial carcinoma, and malignant melanoma. The factors that
reduce the recognition accuracy of skin lesions are image contrast and ethnicity.
Esteva et al. [46] used Inception v3 architecture to classify the lesion into three
groups. The method first distinguishes between benign and malignant and then sepa-
rates seborrheic keratoses and keratinocyte carcinoma types. It also recognizes nevi and
malignant melanomas.
Fujisawa et al. [47] classified skin lesions into 21 classes and introduced a four-level
lesion identify method. Skin pictures are arranged in four levels using the GoogleLeNet
model. Benign and malignant samples are classified first, followed by recognition of
melanocytic and epithelial. The method also identified seborrheic keratosis, actinic kerato-
Appl. Sci. 2022, 12, 2677 4 of 21

sis, Bowen, and basal cell carcinoma lesions. Zhang et al. [18] inserted an attention layer at
the end of ResNet architecture and created new attention residual network. They classified
seborrheic keratosis, melanoma, and nevus lesion.
Mahbod et al. [48] introduced a new method to recognize skin lesions using the fine-
tuned pretrained network. They first applied AlexNet, VGG16, and ResNet to extract skin
image features from the last fully connected layers. Then, they applied SVM to fuse these
extracted features. Harangi [49] presented a method that fuses the outcome probabilities of
the four CNN models: AlexNet, GoogLeNet, VGGNet, and ResNet. The study proposed
the four fusion techniques to identify skin lesions: seborrheic keratosis, melanoma, and
nevus. The sum fusion provides better results than other rules (simple majority voting,
product fusion, and the maximal probability).
Nyiri and Kiss [50] presented the classification of the dermatological picture using
various CNN models, such as VGG19, VGG16, Inception, Xception, DenseNet, and ResNet.
They applied these models to extract features of two different inputs: the original skin and
segmented images. The proposed method combined these two extracted features to predict
skin lesions. Numerical results confirm that ensemble CNN has a better performance
than single CNN in skin classification. Consequently, ensemble architecture outperforms
individual architecture.
Matsunaga et al. [51] proposed an aggregate DL technique to match three lesion classes:
seborrheic keratosis, melanoma, and nevus. They used two binary classifiers on the basis
of ResNet-50 architecture. The first classifier distinguishes between melanoma and another
lesion. Meanwhile, the second classifier identifies the relationship between seborrheic
keratosis and another lesion. The proposed method recognizes the skin lesion images
by combining the output probabilities of these two binary classifiers. Li and Shen [52]
combined two ResNet architectures and obtained the features at the fully connected layer
of each ResNet model. They combined the extracted features to classify skin lesions into
seborrheic keratosis, melanoma, and nevus.

2.2.2. Segmentation DL Approaches

Gonzalez–Diaz [53] introduced three architectures to identify skin lesions: segmenta-
tion, structure segmentation, and diagnosis stages. First, the skin picture is segmented; then,
the output of this stage is used as the input of the structure segmentation stage. Finally, the
diagnosis stage links the outputs of the two previous steps and forecasts the skin lesion
type. The generation of a labeled training database is the main challenge to creating a
structure segmentation network in which each picture has an associated ground truth. This
annotation is usually hard to obtain, as it demands a massive effort of the dermatologists
to outline the segmentations manually.
The study in [54] employed a DL segmentation model (U-Net) to create a segmented
map of the lesion image, cluster sections of abnormal skin, and give the output to a
classification network. Meanwhile, Son et al. [54] drew contours of each cluster with the
output mask created by U-Net and applied a convex hull algorithm to crop each cluster.
Each cluster was applied as input to the EfficientNet to predict the lesion type.
Al-Masni et al. [55] presented an integrated framework that combines skin lesion
boundary segmentation and classification steps. First, the study used DL full-resolution
convolutional network (FrCN) to segment the skin lesion boundaries. Then, a CNN
classifier (ResNet-50, Inception-v3, DenseNet-201, and InceptionResNet-v2) was used to
classify the segmented skin lesions. The segmentation step is a crucial prerequisite for skin
lesion diagnosis because it extracts principal features of different skin lesions.
Table 1 presents the datasets of all DL methods mentioned in the related work section,
including the number of images and classes. Three approaches test with the extensive
databases (more than 10,000 images).
 Table 1. Datasets of all DL methods.

Methods
Appl. Sci. 2022, 12, 2677 Dataset Number of Images Number of Classes
5 of 21
Menegola et al. [44] ISIC 2017 2000 2
Han et al. [45] Asan 17,125 12
Esteva et al. [46] Table 1. Datasets of all DL methods.
DERMOFIT 1300 10
Fujisawa et al. [47] University of Tsukuba Hospital 6009 21
Methods Dataset Number of Images Number of Classes
Mahbod et al. [48] ISIC 2017 2000 2
Menegola et al. [44] ISIC 2017 2000 2
Harangi [49]
Han et al. [45] ISIC
Asan2017 17,1252000 12 2
Nyiri and Kiss [50]
Esteva et al. [46] ISIC
DERMOFIT 2017 1300 2000 10 2
Matsunaga et al. [47]
Fujisawa et al. [51] University of
ISIC 2017 Hospital
Tsukuba 6009 2000 21 2
Mahbod et al. [48] ISIC 2017 2000 2
Li and Shen [52] ISIC 2017 2000 2
Harangi [49] ISIC 2017 2000 2
Gonzalez-Diaz
Nyiri and Kiss[53][50] ISIC2017
ISIC 2017 2000 2000 2 2
Unet [54]
Matsunaga et al. [51] DERMNET
ISIC 2017 200015,851 2 18
LiFrcn
and Shen
[55] [52] ISIC 2017
HAM10000 200010,015 2 7
Gonzalez-Diaz [53] ISIC 2017 2000 2
Unet [54] DERMNET 15,851 18
Frcn [55] BesidesHAM10000
DL, other segmentation approaches
10,015achieved good results for
7 different fields,
such as surface defect detection and mineral separation. Truong et al. [56] presented an
automatic thresholding approach that improves Otsu’s method by applying an entropy
Besides DL, other segmentation approaches achieved good results for different fields,
weighting scheme and overcoming the weakness of Otsu’s technique in defect detection.
such as surface defect detection and mineral separation. Truong et al. [56] presented an
Zhan et al.
automatic [57] presented
thresholding approachan that
ore improves
image segmentation
Otsu’s method algorithm using
by applying an aentropy
histogram
accumulation
weighting schememoment applied tothe
and overcoming multiscenario ore object
weakness of Otsu’s recognition.
technique in defectOre pictures in
detection.
three separate scenarios are used to demonstrate the effectiveness
Zhan et al. [57] presented an ore image segmentation algorithm using a histogram accu-and accuracy of the
proposedmoment
mulation approach. It is reasonable
applied to inherit
to multiscenario these ideas
ore object into bioimage
recognition. fields.
Ore pictures in three
There
separate are many
scenarios methods
are used for skinthelesion
to demonstrate classification
effectiveness in theofliterature
and accuracy the proposed review,
includingItML
approach. and DL to
is reasonable frameworks.
inherit theseHowever, these methods
ideas into bioimage fields. suffer from one of the
following drawbacks:
There are (1) missing
many methods for skintest results
lesion with bigin
classification data, (2) having
the literature insufficiently
review, includinggood
ML and DL frameworks.
performance, However, these
and (3) heavy-weight methods
model. suffer
In this from
study, we one of the a
present following draw- to
novel approach
backs: (1) missing
overcome test results with big data, (2) having insufficiently good performance,
these limitations.
and (3) heavy-weight model. In this study, we present a novel approach to overcome these
limitations.
3. Methodology
We introduce the novel EW-FCM segmentation technique and a new wide-
3. Methodology
ShuffleNet for skin
We introduce thelesion
novelclassification. The segmentation
EW-FCM segmentation techniquestep
and helps
a newthe network separate
wide-ShuffleNet
between
for the classification.
skin lesion skin lesion object and background
The segmentation and the
step helps boosts the recognition
network process.
separate between the The
skin lesion object and background and boosts the recognition process. The segmentationused
segmentation results (full lesion image, including background and foreground) were
as the (full
results input of wide-ShuffleNet
lesion image, includingfor feature extraction
background and classification.
and foreground) were used as Figure 1 shows
the input
the
of structures of for
wide-ShuffleNet thefeature
proposed method.
extraction and We explain how
classification. Figurethe1 shows
EW and the the first-order
structures
cumulative
of the proposed moment
method. were
We combined
explain how tothe
form
EWtheandnew EW-FCM segmentation
the first-order cumulative momenttechnique
were combined to form the new EW-FCM segmentation technique and maintain
and maintain their good characteristics in Section 3.1. Section 3.2 introduces the their goodwide-
characteristics
ShuffleNet. in Section 3.1. Section 3.2 introduces the wide-ShuffleNet.

entropy-based
weighting (EW) Melanocytic
Nevi (NV)

skin lesion segmented wide-

image image ShuffleNet

first-order
cumulative
moment (FCM)

Figure1.1.Structures
Figure Structuresofof the
the proposed
proposed method.
method.
Appl. Sci. 2022, 12, 2677 6 of 21

3.1. EW-FCM Segmentation Technique

In this section, we first present a short analysis of the Otsu technique. Then, we
introduce EW and histogram accumulation moment, including FCM. Finally, the new
image threshold technique for image segmentation is presented.
Otsu is one of the most commonly referenced thresholding methods. Let I = g(x,y)
be an image with the gray value belonging to the interval [0, 1, . . . , L − 1]. Assign ni as
the pixel numbers with the gray value i and the total pixel numbers in g(x,y) as N. The
existence probability of the gray level i is given by
ni
Pi = . (1)
N
Assume the threshold th (0 ≤ th ≤ L − 1) divides g(x,y) into two classes: background
CO = {( x, y)|0 ≤ g( x, y) ≤ th} and object CB = {( x, y)|th + 1 ≤ g( x, y) ≤ L − 1}.
The medium gray level of individual class and the probability of category occurrence,
respectively, are calculated as follows:

th L −1
iP iPi
µo (th) = ∑ ωO (th
i
)
, µ B (th) = ∑ ω (th)
, (2)
i =0 i =th+1 B

where
th L −1
ωO (th) = ∑ Pi , ωB (th) = ∑ Pi . (3)
i =0 i =th+1

In Otsu’s technique, the resulting threshold performance is measured by analyzing

the difference between the background and foreground. The optimal threshold th∗ when
applying this criterion must maximize between-class variance as follows:

th∗ = argmax ωO (th)µO

2 ( th ) + ω ( th ) µ2 ( th ) .

B B
(4)
0 ≤ th ≤ L − 1

The basic idea to improve Otsu’s technique is the addition of a weight W to the
objective function in Equation (4) to regulate the output threshold, given as follows:

th∗ = argmax W ωO (th)µO

2 ( th ) + ω ( th ) µ2 ( th ) .


B B
(5)
0 ≤ th ≤ L − 1

Image entropy describes the properties of an image; it is a mathematical measure of

randomness. Images with low entropy values possess minimal information and hold many
pixels with similar intensity values. An image with zero entropy means that all pixels hold
the same gray value. Reference [56] suggested an EW scheme by substituting weight W
with the entropy objective function ψ(th) in Equation (5) to create a new objective function

th∗ = argmax ψ(th) ωO (th)µO

2 ( th ) + ω ( th ) µ2 ( th ) .


B B
(6)
0 ≤ th ≤ L − 1

The entropy objective function is defined as follows:

Hth Hn
ψ(th) = ln(ωO (th)ω B (th)) + + , (7)
ωO (th) ω B (th)

where
th L −1
Hth = − ∑ Pi ln( Pi ), Hn = − ∑ Pi ln( Pi ). (8)
i =0 i =0
 Appl. Sci. 2022, 12, x FOR PEER REVIEW 7 of 23

Appl. Sci. 2022, 12, 2677 7 of 21

Next, we discuss the first order of the cumulative moment. Let ( ℎ) denotes the
FCMNext,
of thewe
gray the mean gray value from 0 to ℎ
discuss the first order of the cumulative moment. Let MO (th) by
histogram, which is given denotes the
FCM of the gray histogram, which is the mean gray value from 0 to th given by
( ℎ) = . (9)
th
MO (th) = ∑ iPi . (9)
The mean gray of the entire image is , defined
i =0 by

The mean gray of the entire image is MT , defined by

= . (10)
L −1
MT = ∑ iPi . (10)
The FCM helps the optimal threshold avoid dropping into the local optimum
i =0
[57].
We FCM
The combine
MT the EWthe (th)
helps and FCM
optimal to obtain
threshold avoid the optimal threshold
dropping andoptimum
into the local create a new
[57].
objective functionthe
We combine forEW
image
ψ(th)segmentation
and FCM toas follows:
obtain the optimal threshold and create a new
objective function for image segmentation as follows:
ℎ∗ = { ( ℎ)( )}.
∗
(11)
th = argmax0 ≤ ℎ{ ψ
≤ ( th 1 MT )}.
−)(
(11)
We adopt the segmentation process 0 ≤ thin
≤ reference
L−1 [58], including texture filtering,
threshold and binarize, and plot the boundaries. Zade [58] uses the Otsu method to
We adopt the segmentation process in reference [58], including texture filtering, thresh-
calculate the threshold; meanwhile, our framework uses the new objective function in
old and binarize, and plot the boundaries. Zade [58] uses the Otsu method to calculate the
Equation (11) to determine the threshold. Our objective function provides a better
threshold; meanwhile, our framework uses the new objective function in Equation (11) to
segmentation technique due to the reservation of all properties of EW-FCM. Figure 2
determine the threshold. Our objective function provides a better segmentation technique
shows the segmentation results of the original Otsu technique, EW scheme [56], and the
due to the reservation of all properties of EW-FCM. Figure 2 shows the segmentation results
proposed EW-FCM segmentation method. As seen in Figure 2, the proposed EW-FCM
of the original Otsu technique, EW scheme [56], and the proposed EW-FCM segmentation
approach provides better segmentation accuracy than the original Otsu and EW scheme
method. As seen in Figure 2, the proposed EW-FCM approach provides better segmentation
EWS.
accuracy than the original Otsu and EW scheme EWS.

Figure2.2.Segmentation
Figure Segmentation results
results of
of various
various methods: Otsu, EWS,
EWS, and
and EW-FCM.
EW-FCM.

3.2.Wide-ShuffleNet
3.2. Wide-ShuffleNet
Weprovide
We providea brief
a brief analysis
analysis of ShuffleNet
of ShuffleNet inventing
inventing for portable
for portable devices.
devices. Several
Several terms
terms
are are reviewed,
reviewed, including
including efficient
efficient model model designs,
designs, groupgroup convolution,
convolution, channel
channel shuffle
shuffle for
for group
group convolutions,
convolutions, and ShuffleNet
and ShuffleNet unit. We unit.
alsoWe also introduce
introduce a new
a new variant variant of
of ShuffleNet,
ShuffleNet,
called called wide-ShuffleNet,
wide-ShuffleNet, that has beenthat has beenfor
developed developed for skin classification.
skin classification.
Efficient model
Efficient model designs:
designs: Recently,
Recently,efficient
efficientmodel
modeldesigns
designs played an an
played essential rolerole
essential in
achieving
in achieving DLDL networks in many
networks computer
in many visionvision
computer tasks [59–61]. The growing
tasks [59–61]. demand for
The growing de-
streaming
mand high-quality
for streaming DL architectures
high-quality on embedded
DL architectures systems boosts
on embedded systemstheboosts
research
the of
re-
effective
search model designs
of effective model [62].
designsFor[62].
instance, instead of
For instance, assembling
instead convolution
of assembling layers,
convolution
GoogLeNet
layers, [63] expands
GoogLeNet the network
[63] expands depth with
the network sufficient
depth with lower complexity.
sufficient ResNet in
lower complexity.
[64,65] in
ResNet achieves remarkable
[64,65] achieves performance
remarkable using theusing
performance effective bottleneck
the effective architecture.
bottleneck archi-
SqueezeNet
tecture. [66] preserves
SqueezeNet accuracy; accuracy;
[66] preserves however, ithowever,
decreasesitcomputation and parameters
decreases computation and
significantly.
parameters significantly.
Group convolution: AlexNet [59] is the first model using the idea of group convolution
and spreading the network across two GPUs, and ResNeXt [67] confirms the efficacy of
Appl. Sci. 2022, 12, 2677 8 of 21

group convolution. MobileNet [68] applies the depthwise separable convolution (DWConv)
and achieves the best results between lightweight networks. ShuffleNet performs DWConv
and group convolution (GConv) in a new style.
Channel shuffle operation: In the early studies on efficient network layout, the opera-
tion of channel shuffle is seldom noticed. Lately, another study [69] applied this concept for
a two-stage convolution. However, the study [69] did not examine the efficacy of channel
shuffle and its application in light network layout.
Channel shuffle for group convolutions: New DL architectures [63–65] consist of dupli-
cated structure blocks with identical designs. Among these architectures, ResNeXt [67] and
Xception [70] offer an effective GConv or DWConv toward the structure blocks to discover
an outstanding trade-off among computational cost and representation capacity. However,
both architectures do not use pointwise convolutions [68] (or one-by-one convolutions),
which require significant complexity. Increasing the number of one-by-one convolutions,
we must restrict the number of channels to meet the complexity constraint in a small
network. One possible approach is using the channel links with GConv on the one-by-one
layers to handle the limitation. GConv remarkably decreases computation cost by assuring
that individual convolution runs on the relative input channel group.
ShuffleNet unit was invented for the light model, benefiting from channel shuffle
functioning. ShuffleNet unit gets the ideal from bottleneck unit [64] (see Figure 2a in
reference [61]). Building a ShuffleNet unit, the first one-by-one layer in the bottleneck
unit is replaced by pointwise GConv accompanied by a channel shuffle functioning (see
Figure 2b in reference [61]). The second pointwise GConv retrieves the dimension of the
channel to match with the shortcut pathway.
There are two types of ShuffleNet units: nonstride and stride. Two modifications are
made to create ShuffleNet with stride (see Figure 2c in reference [61]). First, a three-by-three
average pooling is added to the shortcut path. Then, the elementwise addition is altered
by channel concatenation, and it is simple to expand the channel dimension with a small
additional computational cost. The ShuffleNet model is formed by the ShuffleNet units.
The ShuffleNet architecture is made by stacking ShuffleNet units together and classified
into three stages. The first block in every stage is implemented with a stride equal to two.
Other parameters in the same stage stay identical, and the number of output channels
doubled for the following stage.
The proposed wide-ShuffleNet develops from ShuffleNet units and the idea of skip
connections. Now, we explore the skip connections. He et al. [64] introduced skip connec-
tions that bypass one or many layers (see Figure 2 in reference [64]). Skip connections create
the fundamental unit of the residual network, which is considered a residual module [71].
It maintains the feature data over all layers, having longer networks while preserving
low parameter numbers. Alternately of getting a desired mapping (indicated as H(x)),
the network with skip connection gets a residual mapping (indicated as F(x) + x, with
F(x) = H(x) − x). Skip connections make identity mapping and add the result to the output
of the two convolutional layers (see Figure 2 in reference [64]).
Next, we apply the long variant of the skip connections to extend the width of the
DL architecture. An architecture with a long residual connection converges more quickly
and offers excellent performance (see reference [72]). A long residual connection helps the
network increase the accuracy as it enhances reused features through the entire network.
It also helps the network to get the detailed and general characteristics of objects. A one-
by-one convolution layer is inserted in the shortcut connection to create a long residual
connection between different sized layers, making the size of two inputs of the additional
layer equal.
Figure 3 shows the final architecture of the proposed wide-ShuffleNet. ShuffleNet 1, 5,
and 13 are stride units, whereas the others are the nonstride units. In three skip connections,
we use three skip convolution layers at the same kernel size of 1 × 1 to connect the input
layers of ShuffleNet units: 1, 5, 13, and an output layer of the ShuffleNet units: 4, 12, 16.
Appl. Sci. 2022, 12, 2677 9 of 21

Appl. Sci. 2022, 12, x FOR PEER REVIEW

Skip convolution layers: 1, 2, 3, 4, and 5 have 136, 272, 544, 544, and 544 filters,10respectively.
of 23

The stride for all skip convolution layers is equal to 2.

input

convolution
BN
leaky Relu

ShuffleNet Unit 1

ShuffleNet Unit 2
skip convolution 1
skip BN 1
ShuffleNet Unit 3 Skip leaky Relu 1

ShuffleNet Unit 4

skip convolution 4 ShuffleNet Unit 5

skip BN 4
Skip leaky Relu 4 ShuffleNet Unit 6

ShuffleNet Unit 7

ShuffleNet Unit 8
skip convolution 2
skip BN 2
ShuffleNet Unit 9 Skip leaky Relu 2

ShuffleNet Unit 10

skip convolution 5 ShuffleNet Unit 11

skip BN 5
Skip leaky Relu 5
ShuffleNet Unit 12

ShuffleNet Unit 13

ShuffleNet Unit 14 skip convolution 3

skip BN 3
ShuffleNet Unit 15 Skip leaky Relu 3

ShuffleNet Unit 16

leaky Relu

global average global average global average

pooling 3 pooling 2 pooling 1

fully connected

output

Figure 3. Proposed wide-ShuffleNet.

Figure 3. Proposed wide-ShuffleNet.
Appl. Sci. 2022, 12, 2677 10 of 21

Additionally, we insert the batch normalization (BN) layer after every skip convolu-
tion layer due to some reasons. DL training process becomes quick when applying BN.
Increasing the deep of the network means that the training process gets more challenging
because of many problems faced while training. The architecture provides greater test data
precision with the BN layer than the original model (without this layer). BN decreases the
internal covariate shift; thus, improving the performance of the network. The classification
accuracy significantly increases when applying BN. The position of the BN layer is after the
convolutional layer and before the leaky ReLU layer. This structure speeds up the training
and decreases testing and training time [73]. Furthermore, we replace all ReLU layers in
all ShuffleNet units with the leaky ReLU layers. Leaky ReLU gives better results than the
ReLU activation function (see reference [74]).

4. Experiment
In this section, we present the numerical results.

4.1. Datasets
The dataset plays an essential role in evaluating the performance of the proposed
framework. We test the proposed method on the two datasets: HAM10000 and ISIC 2019.
HAM10000 is the dermatoscopic database, which uses a benchmark dataset [75]. This
database comprises more than 10,000 dermatoscopic pictures obtained from many people
worldwide. The HAM10000 database also holds metadata formats like comma-separated-
values data (.CSV), containing gender, age, and cell class. This dataset consists of seven
different types of skin diseases: actinic keratoses and intraepithelial carcinoma (AKIEC),
basal cell carcinoma (BCC), benign keratosis-like lesions (BKL), dermatofibroma (DF),
melanoma (MEL), melanocytic Nevi (NV), and vascular lesions (VASC). The principal
problem of the HAM10000 database is classes imbalance and the irregular distribution
of skin disease numbers. NV class exceeds 70% of the total image numbers. This factor
influences the training and creates an extreme imbalance database. The second large class
is BKL, with approximately 13% of the pictures. The other classes contribute a minority
number of the images. Especially, less than 2% of the total images belong to the DF class,
which is the most difficult class for prediction. Figure 4 shows some sample images from
the HAM10000 dataset. HAM10000 dataset is a part of the ISIC 2018 challenge with three
Appl.
tasks:Sci. 2022, 12, x FOR
lesion PEER REVIEWsegmentation (task 1), lesion attribute detection (task 2), disease
boundary 12 of 23

classification (task 3).

Figure 4.in
Figure 4. Sample skin lesions Sample skin lesions in the
the HAM10000 HAM10000
dataset. Typedataset. Type 1: Type
1: MEL; MEL; Type 2: NV;
2: NV; Type 3:
Type 3: BCC;
BCC;
Type 4: AKIEC; Type 5: BKL; Type 6: DF; Type 7: VASC.
Type 4: AKIEC; Type 5: BKL; Type 6: DF; Type 7: VASC.
The second dataset is the ISIC-2019 [76], consisting of 25,331 dermoscopic pictures
belonging to eight categories: AKIEC, BCC, BKL, DF, MEL, NV, VASC, and squamous cell
carcinoma (SCC). ISIC-2019 data are obtained from the following sources: BCN_20000
Dataset, HAM10000 Dataset, and MSK Dataset. The ISIC-2019 challenge has only one task:
disease classification. Table 2 gives the class distribution of the two datasets, HAM10000
and ISIC-2019.
Appl. Sci. 2022, 12, 2677 11 of 21

The second dataset is the ISIC-2019 [76], consisting of 25,331 dermoscopic pictures
belonging to eight categories: AKIEC, BCC, BKL, DF, MEL, NV, VASC, and squamous
cell carcinoma (SCC). ISIC-2019 data are obtained from the following sources: BCN_20000
Dataset, HAM10000 Dataset, and MSK Dataset. The ISIC-2019 challenge has only one task:
disease classification. Table 2 gives the class distribution of the two datasets, HAM10000
and ISIC-2019.

Table 2. Class distribution in the HAM10000 and ISIC 2019 datasets.

HAM10000 ISIC2019
Class Name
Number of Images Number of Images
AKIEC 327 867
BCC 514 3323
BKL 1099 2624
DF 115 239
MEL 1113 4522
NV 6705 12,875
SCC - 628
VASC 142 253
Total 10,015 25,331

4.2. Evaluation
We apply the following metrics to evaluate the performance of the proposed method.

TP
Precision = , (12)
TP + FP
TP
Sensitivity = , (13)
TP + FN
TN
Speci f icity = ,
TN + FP
Precision × Sensitivity
F1 − score = 2 × , (14)
Precision + Sensitivity
TP + TN
Accuracy = , (15)
TP + FP + TN + FN
where TN is the true negative, TP is the true positive, FN is the false negative, and FP is
the false positive.

4.3. Implementation Details

The original HAM10000 and ISIC-2019 databases can be downloaded from the link in
the Data Availability Statement. All tests were conducted on the i7 8700 PC, with 32 GB
memory, GPU NVIDIA 1070, MATLAB (9.8 (R2020a), Natick, MA, USA). We apply the
initial learning rate at 0.001, the mini-batch at 32, and the momentum at 0.9 for SGD for
network training. After every 20 epochs, we divide the learning rate by half.

4.4. Comparison of the HAM10000 and ISIC 2019 Datasets

We evaluate the proposed method with two big datasets, HAM10000 and ISIC 2019.
Then, we compare the performance of the proposed framework and that of state-of-the-
art approaches for the skin lesion classification task, including non-segmentation and
segmentation approaches. We split two datasets into training and testing parts with the
same amount mentioned in references [23,29,76]. The HAM10000 dataset is used in the
first and second experiments, with the percentages of the testing parts being 20% and
10%, respectively. The third experiment utilizes the ISCI2019 dataset, with 10% of the total
images belonging to the testing part. We implemented three different experiments to make
a fair comparison between the proposed EW-FCM framework and other methods. The
Appl. Sci. 2022, 12, 2677 12 of 21

reason is that each method uses a different dataset and a different proportion of the testing
data from the whole dataset. For example, Thurnhofer-Hemsi et al. [23] use 20% of the
dataset as the testing data, while Srinivasu et al. [29] use only 10% for the testing data on
the same dataset HAM10000.

4.4.1. Comparison with Non-Segmentation Methods

Table 3 presents the classification results of all approaches in three experiments.

Table 3. Comparison with different methods on the HAM10000 and ISIC2019 datasets.

Experiment Method ACC

PNASNet [77] 76.00%
ResNet-50 + gcForest [78] 80.04%
VGG-16 + GoogLeNet
81.50%
Ensemble [79]
Experiment 1 Densenet-121 with SVM [80] 82.70%
HAM10000 DenseNet-169 [80] 81.35%
(80% training, 20% testing) Bayesian DenseNet-169 [81] 83.59%
Shifted MobileNetV2 [23] 81.90%
Shifted GoogLeNet [23] 80.50%
Shifted 2-Nets [23] 83.20%
The proposed method 84.80%
HARTS [82] 77.00%
FTNN [83] 79.00%
CNN [84] 80.00%
Experiment 2
VGG19 [85] 81.00%
HAM 10000
MobileNet V1 [68] 82.00%
(90% training, 10% testing)
MobileNet V2 [86] 84.00%
MobileNet V2-LSTM [29] 85.34%
The proposed method 86.33%
VGG19 [85] 80.17%
Experiment 3 ResNet-152 [64] 84.15%
ISIC 2019 Efficient-B0 [87] 81.75%
(90% training, 10% testing) Efficient-B7 [87] 84.87%
The proposed method 82.56%

In the first experiment, the HAM10000 database is divided into two parts: one part
with 8008 training images and the other part with 2007 testing images. Four methods in
Table 3 provide the confusion matrix of the HAM10000 classification, including Shifted
MobileNetV2, Shifted GoogLeNet, Shifted 2-Nets, and the proposed method. Shifted 2-Nets
introduces an aggregate of enhanced CNN joined (ensemble of the DL approach) with a
test-time regularly spaced shifting method for skin recognition. Shifted 2-Nets reached
83.20% accuracy (10.4 M parameters), whereas our framework produced 84.80% accuracy
(1.8 M parameters). The proposed framework is more efficient than Shifted GoogLeNet,
MobileNetV2, and Shifted 2-Nets. Figure 5 shows the confusion matrix of the four methods.
The first note is that the proposed method has a lower wrong classification than other
methods. Additionally, the number of right predictions has improved for most categories:
AKIEC, BCC, DF, and MEL categories. BKL, NV, and VASC are three experiments that
our framework ranks as the third, second, and third positions among four approaches,
respectively. We calculate the following metrics from the obtained confusion matrix:
specificity, sensitivity, precision, and F1 score. Table 4, which is visualized in Figure 6,
presents our proposed framework’s performance and various approaches with five metrics:
accuracy, specificity, sensitivity, precision, and F1 score (macro-average).
Appl. Sci. 2022, 12, x2677
FOR PEER REVIEW 1413of
of 23
21

Figure
Figure 5.
5. Confusion
Confusion matrix
matrix of
of the
the four
four methods.
methods.

Table 3.
Table Comparisonwith
4. Comparison withdifferent
different methods
methods onon
thethe HAM10000
HAM10000 anddataset in datasets.
ISIC2019 the first experiment
(macro-average).
Experiment Method ACC
PNASNet [77]
Methods 76.00%
Shifted MobileNetV2 Shifted GoogLeNet ResNet-50 Shifted
+ gcForest
2-Nets[78] 80.04%
The Proposed Method
Specificity 95.20% VGG-16
94.70% + GoogLeNet Ensemble
95.30% [79] 81.50%
96.03%
Sensitivity 65.90% 58.10% Densenet-121 with SVM [80]
64.40% 82.70%
70.71%
Experiment 1
Precision 71.40% 68.50% DenseNet-169 [80]
76.10% 81.35%
75.15%
F1 score 67.00%HAM10000 60.80% Bayesian DenseNet-16967.80% [81] 72.61%
83.59%
Accuracy (80% training, 20% testing)80.50%
81.90% 83.20% 84.80%
Shifted MobileNetV2 [23] 81.90%
Parameters 3.4 M 7M 10.4 M 1.8 M
Shifted GoogLeNet [23] 80.50%
Shifted 2-Nets [23] 83.20%
In the second experiment, the HAM10000 database
The proposed method is split into two parts: 90% of the
84.80%
total images are the training part, and the rest are
HARTS [82]the testing part. None of the
77.00%methods
in Table 3 in the second experiment yield the precision, F1 score, and confusion matrix
FTNN [83] 79.00%
(micro-average metric). We use three metrics to evaluate the performance of various meth-
CNN [84] 80.00%
ods: accuracy, sensitivity,
Experiment 2 and specificity. Table 5 presents the outcomes of all approaches.
Figure 7HAM
visualizes the results of Table 5. OurVGG19
10000
[85] achieves the highest81.00%
framework performance
in terms of accuracy and specificity. MobileNet
Meanwhile, theV1MobileNet
[68] V2 with 82.00%
long short-term
(90% training, 10% testing)
memory (LSTM) [29] component has theMobileNet V2 [86] MobileNet V2 improves
highest sensitivity. 84.00% effi-
MobileNet
ciency more than the first version MobileNet V1.V2-LSTM
Consequently,[29] the number of 85.34%
parameters
decreases 19% from 4.2 to 3.4 M. MobileNet The proposed method
V2 is approximately two times the86.33%
parameters
of our framework (1.8 M), even with lower parameters.
VGG19 [85] Additionally, VGG19
80.17% has the
highestExperiment
number of parameters
3 with about 143 M (79[64]
ResNet-152 times the proposed method).
84.15% This
comparison ISICconfirms
2019 the efficiency of theEfficient-B0
proposed framework.
[87] 81.75%
(90% training, 10% testing) Efficient-B7 [87] 84.87%
The proposed method 82.56%
 Specificity 95.20% 94.70% 95.30% 96.03%
Sensitivity 65.90% 58.10% 64.40% 70.71%
Precision 71.40% 68.50% 76.10% 75.15%
F1 score 67.00% 60.80% 67.80% 72.61%
Appl. Sci.Accuracy
2022, 12, 2677 81.90% 80.50% 83.20% 84.80% 14 of 21
Parameters 3.4 M 7M 10.4 M 1.8 M

Figure 6. Bar chart results of various methods in the first experiment.

Figure 6. Bar chart results of various methods in the first experiment.
Table 5. Comparison with different methods on the HAM10000 dataset in the second experiment
In the second experiment, the HAM10000 database is split into two parts: 90% of the
(micro-average).
total images are the training part, and the rest are the testing part. None of the methods in
Table 1 in Method
the second experiment Accuracy Specificity F1 score,
yield the precision, Sensitivity Parameters
and confusion matrix
(micro-average metric).
HARTS [82] We use three
77.00% metrics to evaluate
83.00% the performance
78.21% of - various
methods: FTNN
accuracy,
[83] sensitivity, and specificity.84.00%
79.00% Table 5 presents
79.54%the outcomes - of all
VGG19
approaches. [85]7 visualizes the
Figure 81.00% 87.00%
results of Table 82.46% achieves the
5. Our framework 143 highest
M
MobileNet V1 [68] 82.00% 89.00% 84.04% 4.2
performance in terms of accuracy and specificity. Meanwhile, the MobileNet V2 with long M
MobileNet V2 [86] 84.00% 90.00% 86.41% 3.4 M
short-term memory (LSTM) [29] component has the highest sensitivity. MobileNet V2
MobileNet V2-LSTM [29] 85.34% 92.00% 88.24% ≈3.4 M
The proposed method 86.33% 97.72% 86.33% 1.8 M

In the third experiment, we follow the dataset divided according to the previous
study [76]. The authors in reference [76] tested the skin classification on the ISIC-2019
dataset with different transfer learning models, such as state-of-the-art architecture Effi-
cientNet. EfficientNet has eight models that start from B0 to B7 and achieve better efficiency
and accuracy than earlier ConvNets. EfficientNet employs swish activation instead of
applying the ReLU function (see reference [76]).
Table 6 compares all approaches. All methods provide only the accuracy metric,
except the proposed methods. EfficientNetB0 uses a small number of parameters. It is
the simplest of all eight architectures in EfficientNet. However, the total parameters of
EfficientNet-B0 are approximately three times that of our method (5 M compared with
1.8 M), whereas the accuracy is lower (81.75% compared with 82.56%). EfficientNet-B7
and ResNet-152 are the first and second rank in terms of accuracy, respectively. Both
architectures have high parameters (66 and 50 M, respectively) and achieved a better result
than our method (the proposed network uses less than 4% of the parameters compared to
these two models). VGG19 is the worst method with the highest parameters (143 M) and
the lowest accuracy (80.17%).
 FTNN [83] 79.00% 84.00% 79.54% -
VGG19 [85] 81.00% 87.00% 82.46% 143 M
MobileNet V1 [68] 82.00% 89.00% 84.04% 4.2 M
MobileNet V2 [86] 84.00% 90.00% 86.41% 3.4 M
Appl. Sci. 2022, 12, 2677 MobileNet V2-LSTM [29] 85.34% 92.00% 88.24% 15 of 21
3.4 M
The proposed method 86.33% 97.72% 86.33% 1.8 M

Figure 7. Bar chart results of various methods in the second experiment.

experiment.

Table 6. Comparison with different methods on the ISIC2019 dataset in the third experiment
In the third experiment, we follow the dataset divided according to the previous
(micro-average).
study [76]. The authors in reference [76] tested the skin classification on the ISIC-2019
dataset with different transfer Accuracy
Method learning models, such as Sensitivity
Specificity state-of-the-artParameters
architecture
EfficientNet. EfficientNet has eight models that start from B0 to B7 and achieve better
VGG19 [85] 80.17% - - 143 M
efficiency and accuracy
ResNet-152 [64] than earlier ConvNets. EfficientNet
84.15% - employs
- swish 50
activation
M
instead Efficient-B0
of applying the ReLU function
[87] 81.75%(see reference
- [76]). - 5M
Table 6 compares
Efficient-B7 [87] all approaches.
84.87% All methods provide only the accuracy metric,
66 M
except the proposed
The proposed methodmethods. EfficientNetB0
82.56% uses a small
97.51% number of
82.56% parameters.
1.8 M is the
It
simplest of all eight architectures in EfficientNet. However, the total parameters of
EfficientNet-B0 are approximately three times that of our method (5 M compared with 1.8
We havethe
M), whereas already compared
accuracy is lowerdifferent
(81.75%methods
compared in with
three 82.56%).
experiments. Even with and
EfficientNet-B7 the
highest efficiency result, our method has a weakness that could not control the imbalanced
classes of two datasets: HAM10000 and ISIC2019. Data sampling methods, which are the
future research work, can balance the classes distribution.

4.4.2. Comparison with Segmentation Methods

The proposed method has higher accuracy and is more efficient than other non-
segmentation approaches. In this section, we compare the proposed EW-FCM with other
segmentation techniques, such as non-DL and DL segmentation methods. All segmentation
methods use the full lesion image (background and foreground) as the input image for the
classification network.
First, we compare the proposed EW-FCM with other non-DL methods, such as the
original Otsu, Otsu momentum, an EW scheme. Table 7 presents the results. EW-FCM
achieved the highest accuracy among non-DL segmentation approaches. Figure 8 shows
the segmentation results of various segmentation methods.
 result than our method (the proposed network uses less than 4% of the parameters
compared to these two models). VGG19 is the worst method with the highest parameters
(143 M) and the lowest accuracy (80.17%).

Appl. Sci. 2022, 12, 2677 Table 6. Comparison with different methods on the ISIC2019 dataset in the third experiment (micro-
16 of 21
average).

Method Accuracy Specificity Sensitivity Parameters

Comparison
Table 7. VGG19 [85] with segmentation
80.17% methods on- the HAM10000- and ISIC2019
143datasets
M
(micro-average).
ResNet-152 [64] 84.15% - - 50 M
Efficient-B0 [87] 81.75% - - 5M
Experiment Method ACC Sensitivity Specificity
Efficient-B7 [87] 84.87% 66 M
Original Otsu + wide-ShuffleNet 79.62% 79.62% 96.60%
The proposed method 82.56% 97.51% 82.56% 1.8 M
Otsu Momentum [57] + wide-ShuffleNet 81.51% 81.51% 96.92%
Image Entropy [56] + wide-ShuffleNet 83.91% 83.91% 97.32%
Experiment 1 We have+ wide-ShuffleNet
U-Net already compared different 85.10%
methods in three85.10%
experiments. Even with the
97.52%
HAM10000 highest efficiency
U-Net result, our[54]
+ EfficientNet-B0 method has a weakness
85.65% that could not control the97.61%
85.65% imbalanced
(80% training, 20% testing) FrCNof+ two
classes Inception-ResNet-v2
datasets: HAM10000[55] and ISIC2019.
87.74% Data sampling - methods, which - are the
FrCN + Inception-v3 [55] 88.05%
future research work, can balance the classes distribution. - -
FrCN + DenseNet-201 [55] 88.70% - -
4.4.2. EW-FCM + wide-ShuffleNet
Comparison 84.80%
with Segmentation Methods 84.80% 97.48%
Original Otsu + wide-ShuffleNet
The proposed 80.14% and is more
method has higher accuracy 80.14%
efficient than96.69%
other non-
Experiment 2
HAM 10000 segmentation approaches. In this section, we compare the proposed EW-FCM97.09%
Otsu Momentum [57] + wide-ShuffleNet 82.54% 82.54% with other
Image Entropy [56] + wide-ShuffleNet 84.83% 84.83% 97.47%
(90% training, 10% testing) segmentation techniques, such as non-DL and DL segmentation methods. All
EW-FCM methods
segmentation + wide-ShuffleNet 86.33%
use the full lesion image (background86.33%
and foreground)97.72%
as the input
image for the
Original classification
Otsu network.
+ wide-ShuffleNet 78.55% 78.55% 96.94%
Experiment 3
Otsu Momentum [57] + wide-ShuffleNet
First, we compare the proposed EW-FCM 80.34%with other non-DL
80.34% methods,97.19%
such as the
ISIC 2019
Image Entropy
original Otsu, [56]
Otsu+ wide-ShuffleNet
momentum, an EW scheme. 81.20% Table 7 presents
81.20% the results.
97.31%
EW-FCM
(90% training, 10% testing)
achieved the highest
EW-FCM accuracy among non-DL
+ wide-ShuffleNet segmentation82.56%
82.56% approaches. Figure 8 shows
97.51%
the segmentation results of various segmentation methods.

Figure 8. Segmentation results of various segmentation methods.

Figure 8. Segmentation results of various segmentation methods.

TableSecond,
7. Comparison with segmentation
we compare the proposedmethods
EW-FCM on with
the HAM10000 and ISIC2019
DL segmentation datasets
methods. (micro-
Al-Masni
average).
et al. [55] provides only the accuracy of the training of the ISIC 2018 task 3 (HAM10000
dataset).
ExperimentMoreover, Al-Masni etMethod
al. [55] uses DL FrCN to segment the skin Specificity
ACC Sensitivity lesion and
classify full
Experiment 1 lesion images with various networks,
Original Otsu + wide-ShuffleNetsuch as Inception-ResNet-v2,
79.62% 79.62% DenseNet-
96.60%
201, and Inception-v3. Meanwhile, Son et al. [54] crop the segmented image with U-Net and
HAM10000 Otsu Momentum [57] + wide-ShuffleNet 81.51% 81.51% 96.92%
input the result to Efficient-B0 for classification. We adopt the idea of U-Net and Efficient-
B0 in [54] to evaluate the HAM10000 dataset using the U-Net as a segmentation method
with full lesion image (background and foreground). EW-FCM achieved a lower accuracy
than the DL segmentation methods (see Table 7); it also achieved higher results than
non-DL segmentation methods. There are two main drawbacks of the DL segmentation
methods. First, creating a labeled training database is the primary challenge to creating
a DL segmentation network in which each picture has an associated ground truth. A
massive effort of dermatologists is used to obtain this ground truth pixel-wise segmentation.
Meanwhile, the proposed EW-FCM uses a threshold technique for image segmentation
and does not depend on the ground truth. As a result, we cannot calculate the quantitative
Appl. Sci. 2022, 12, 2677 17 of 21

analysis with the EW-FCM segmentation technique (such as using the Jaccard index).
Second, the complexity increases when using DL for segmentation and classification. For
instance, the total parameters of the DL segmentation U-Net and DL classification Efficient-
B0 are 12.7 M (7.7 M + 5 M = 12.7 M). EW-FCM uses DL only for the classification state,
thus decreasing the complexity and being suitable with a portable system.
Next, we evaluate the performance of the skin classification with various classifiers
and present the results in Table 8. The proposed method improves the skin classification
for two reasons: wide-ShuffleNet is better than ShuffleNet, and the new segmentation
technique is better than the original image. Efficient-B0 has the highest accuracy, but
its parameters increase approximately three times compared to wide-ShuffleNet. In fu-
ture studies, we will investigate the performance of EW-FCM with ShuffleNet V2 (2.4 M
parameters) and compare it with the proposed wide-ShuffleNet (1.8 M parameters).

Table 8. Comparison with different networks on the HAM10000 and ISIC2019 datasets
(micro-average).

Experiment Method ACC Sensitivity Specificity

Original Image + ShuffleNet 76.83% 76.83% 96.14%
Experiment 1 Original Image + wide-ShuffleNet 77.88% 77.88% 96.31%
HAM10000 EW-FCM + ShuffleNet 83.66% 83.66% 97.28%
(80% training, 20% testing) EW-FCM + wide-ShuffleNet 84.80% 84.80% 97.48%
EW-FCM + EfficientNet-B0 85.50% 85.50% 97.58%
Original Image + ShuffleNet 77.25% 77.25% 96.21%
Experiment 2 Original Image + wide-ShuffleNet 78.64% 78.64% 96.44%
HAM 10000 EW-FCM + ShuffleNet 84.43% 84.43% 97.41%
(90% training, 10% testing) EW-FCM + wide-ShuffleNet 86.33% 86.33% 97.72%
EW-FCM + EfficientNet-B0 87.23% 87.23% 97.87%
Original Image + ShuffleNet 74.23% 74.23% 96.32%
Experiment 3 Original Image + wide-ShuffleNet 75.73% 75.73% 96.53%
ISIC 2019 EW-FCM + ShuffleNet 81.62% 81.62% 97.38%
(90% training, 10% testing) EW-FCM + wide-ShuffleNet 82.56% 82.56% 97.51%
EW-FCM + EfficientNet-B0 84.66% 84.66% 97.81%

5. Conclusions
Skin cancer is one of the most dangerous diseases in humans. The automated classifi-
cation of skin lesions using DL will save time for physicians and increase the curing rate.
Typical DL frameworks require high parameters and cannot work on the mobile system.
Hence, developing the lightweight DL framework for skin lesion classification is essential.
In this paper, we propose a novel method for skin lesion classification. Our lightweight
method improves the limitation of evaluating a small number of skin lesion images in
the past. The numerical results show that the proposed framework is more efficient and
accurate than the 20 other approaches (see Table 3). The proposed method reduces the
number of the parameters to approximately 79 times that of another method (VGG19) while
maintaining higher accuracy. Additionally, the proposed method achieves higher accuracy
than other non-segmentation and non-DL segmentation methods and the approximate
results at the level of the DL segmentation methods while reducing the complexity of
the DL segmentation methods. Our framework does not require ground truth for image
segmentation, whereas DL segmentation methods cannot work without ground truth. Thus,
the proposed method decreases the effort of the dermatologists to manually outline the
ground truth pixel-wise segmentation. We create an accurate and efficient framework by
combining the new EW-FCM segmentation technique and wide-ShuffleNet.
We will compare the proposed framework with more networks in future work. An-
other future direction for research and development is to integrate the proposed method
into real-world problems like the mobile healthcare system.
Appl. Sci. 2022, 12, 2677 18 of 21

Author Contributions: Conceptualization, L.H.; Funding acquisition, S.-H.L. and K.-R.K.; Investiga-
tion, L.H.; Methodology, L.H.; Project administration, S.-H.L. and K.-R.K.; Software, L.H.; S.-H.L. and
K.-R.K.; Supervision, S.-H.L., E.-J.L., and K.-R.K.; Validation, S.-H.L. and K.-R.K.; Writing—original
draft, L.H.; Writing—review and editing, L.H. and S.-H.L. All authors have read and agreed to the
published version of the manuscript.
Funding: This research was funded by the Brain Korea 21 project (BK21).
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: The original HAM10000 and ISIC2019 datasets are available online at
https://www.kaggle.com/kmader/skin-cancer-mnist-HAM10000 (accessed on 19 October 2021)
and https://challenge2019.isic-archive.com/ (accessed on 19 October 2021).
Acknowledgments: This research was supported by the Basic Science Research Program through the
National Research Foundation of Korea (NRF) funded by the Ministry of Education (2020R1I1A306659411,
2020R1F1A1069124) and by the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Informa-
tion Technology Research Center) support program (IITP-2022-2020-0-01797) supervised by the IITP
(Institute for Information & Communications Technology Planning & Evaluation).
Conflicts of Interest: The authors declare no conflict of interest.

Abbreviations

AI Artificial Intelligence
AKIEC Actinic Keratoses and Intraepithelial Carcinoma
BCC Basal Cell Carcinoma
BKL Benign Keratosis-like Lesions
BN Batch Normalization
CNN Convolutional Neural Network
DF Dermatofibroma
DL Deep Learning
DWConv Depthwise Separable Convolution
EW Entropy-based Weighting
EWS Entropy-based Weighting Scheme (including Otsu)
FCM First-order Cumulative Moment
GConv Group Convolution
GPU Graphics Processing Unit
KNN K-Nearest Neighbor
MEL Melanoma
ML Machine Learning
NV Melanocytic Nevi
SCC Squamous Cell Carcinoma
SVM Support Vector Machine
VASC Vascular Lesions
WHO World Health Organization

References
1. Rey-Barroso, L.; Peña-Gutiérrez, S.; Yáñez, C.; Burgos-Fernández, F.J.; Vilaseca, M.; Royo, S. Optical technologies for the
improvement of skin cancer diagnosis: A review. Sensors 2021, 21, 252. [CrossRef]
2. Hosny, K.M.; Kassem, M.A.; Foaud, M.M. Classification of skin lesions using transfer learning and augmentation with Alex-net.
PLoS ONE 2019, 14, e0217293. [CrossRef] [PubMed]
3. Zicari, R.V.; Ahmed, S.; Amann, J.; Braun, S.A.; Brodersen, J.; Bruneault, F.; Wurth, R. Co-Design of a trustworthy AI System in
healthcare: Deep learning based skin lesion classifier. Front. Hum. Dyn. 2021, 3, 40. [CrossRef]
4. Mishra, N.; Celebi, M. An overview of melanoma detection in dermoscopy images using image processing and machine learning.
arXiv 2016, arXiv:1601.07843.
5. World Health Organization. Radiation: Ultraviolet (UV) Radiation and Skin Cancer. Available online: https://www.who.int/
news-room/questions-and-answers/item/radiation-ultraviolet-(uv)-radiation-and-skin-cancer#:~{}:text=Currently%2C%20
between%202%20and%203,skin%20cancer%20in%20their%20lifetime (accessed on 19 October 2021).
Appl. Sci. 2022, 12, 2677 19 of 21

6. Jerant, A.F.; Johnson, J.T.; Sheridan, C.D.; Caffrey, T.J. Early detection and treatment of skin cancer. Am. Fam. Physician 2000, 62,
357–368.
7. Trufant, J.; Jones, E. Skin cancer for primary care. In Common Dermatologic Conditions in Primary Care; John, J.R., Edward, F.R., Jr.,
Eds.; Springer: Cham, Switzerland, 2019; pp. 171–208.
8. Barata, C.; Celebi, M.E.; Marques, J.S. A survey of feature extraction in dermoscopy image analysis of skin cancer. IEEE J. Biomed.
Health Inform. 2019, 23, 1096–1109. [CrossRef] [PubMed]
9. Celebi, M.E.; Kingravi, H.A.; Uddin, B.; Iyatomi, H.; Aslandogan, Y.A.; Stoecker, W.V.; Moss, R.H. A methodological approach to
the classification of dermoscopy images. Comput. Med. Imaging Graph. 2007, 31, 362–373. [CrossRef]
10. Tommasi, T.; La Torre, E.; Caputo, B. Melanoma recognition using representative and discriminative kernel classifiers.
In Proceedings of the International Workshop on Computer Vision Approaches to Medical Image Analysis (CVAMIA), Graz,
Austria, 12 May 2006; pp. 1–12. [CrossRef]
11. Pathan, S.; Prabhu, K.G.; Siddalingaswamy, P.C. A methodological approach to classify typical and atypical pigment network
patterns for melanoma diagnosis. Biomed. Signal Process. Control 2018, 44, 25–37. [CrossRef]
12. Taner, A.; Öztekin, Y.B.; Duran, H. Performance analysis of deep learning CNN models for variety classification in hazelnut.
Sustainability 2021, 13, 6527. [CrossRef]
13. Wang, W.; Siau, K. Artificial intelligence, machine learning, automation, robotics, future of work and future of humanity: A review
and research agenda. J. Database Manag. 2019, 30, 61–79. [CrossRef]
14. Samuel, A.L. Some studies in machine learning using the game of checkers. II—Recent progress. In Computer Games I; Springer:
Berlin/Heidelberg, Germany, 1988; pp. 366–400.
15. Liu, W.; Wang, Z.; Liu, X.; Zeng, N.; Liu, Y.; Alsaadi, F.E. A survey of deep neural network architectures and their applications.
Neurocomputing 2017, 234, 11–26. [CrossRef]
16. Qiu, Z.; Chen, J.; Zhao, Y.; Zhu, S.; He, Y.; Zhang, C. Variety identification of single rice seed using hyperspectral imaging
combined with convolutional neural network. Appl. Sci. 2018, 8, 212. [CrossRef]
17. Acquarelli, J.; Van, L.T.; Gerretzen, J.; Tran, T.N.; Buydens, L.M.; Marchiori, E. Convolutional neural networks for vibrational
spectroscopic data analysis. Anal. Chim. Acta 2017, 954, 22–31. [CrossRef] [PubMed]
18. Zhang, X.; Lin, T.; Xu, J.; Luo, X.; Ying, Y. DeepSpectra: An end-to-end deep learning approach for quantitative spectral analysis.
Anal. Chim. Acta 2019, 1058, 48–57. [CrossRef] [PubMed]
19. Yang, X.; Ye, Y.; Li, X.; Lau, R.Y.; Zhang, X.; Huang, X. Hyperspectral image classification with deep learning models. IEEE Trans.
Geosci. Remote Sens. 2018, 56, 5408–5423. [CrossRef]
20. Yu, X.; Tang, L.; Wu, X.; Lu, H. Nondestructive freshness discriminating of shrimp using visible/near-infrared hyperspectral
imaging technique and deep learning algorithm. Food Anal. Methods 2018, 11, 768–780. [CrossRef]
21. Yue, J.; Mao, S.; Li, M. A deep learning framework for hyperspectral image classification using spatial pyramid pooling. Remote
Sens. Lett. 2016, 7, 875–884. [CrossRef]
22. Signoroni, A.; Savardi, M.; Baronio, A.; Benini, S. Deep learning meets hyperspectral image analysis: A multidisciplinary review.
J. Imaging 2019, 5, 52. [CrossRef]
23. Thurnhofer-Hemsi, K.; López-Rubio, E.; Domínguez, E.; Elizondo, D.A. Skin lesion classification by ensembles of deep convolu-
tional networks and regularly spaced shifting. IEEE Access 2021, 9, 112193–112205. [CrossRef]
24. Litjens, G.; Kooi, T.; Bejnordi, B.E.; Setio, A.A.A.; Ciompi, F.; Ghafoorian, M.; van der Laak, J.A.; van Ginneken, B.; Sánchez, C.I.
A survey on deep learning in medical image analysis. Med. Image Anal. 2017, 42, 60–88. [CrossRef] [PubMed]
25. Cui, C.; Thurnhofer-Hemsi, K.; Soroushmehr, R.; Mishra, A.; Gryak, J.; Dominguez, E.; Najarian, K.; Lopez-Rubio, E. Diabetic
wound segmentation using convolutional neural networks. In Proceedings of the 41th Annual International Conference of the
IEEE Engineering in Medicine and Biology Society (EMBC), Berlin, Germany, 23–27 July 2019; pp. 1002–1005.
26. Thurnhofer-Hemsi, K.; Domínguez, E. Analyzing digital image by deep learning for melanoma diagnosis. In Advances in
Computational Intelligence; Rojas, I., Joya, G., Catala, A., Eds.; Springer: Cham, Switzerland, 2019; pp. 270–279.
27. Thurnhofer-Hemsi, K.; Domínguez, E. A convolutional neural network framework for accurate skin cancer detection. Neural
Process. Lett. 2021, 53, 3073–3093. [CrossRef]
28. Codella, N.C.; Gutman, D.; Celebi, M.E.; Helba, B.; Marchetti, M.A.; Dusza, S.W.; Halpern, A. Skin lesion analysis toward
melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (ISBI), hosted by the international
skin imaging collaboration (ISIC). In Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI
2018), Washington, DC, USA, 4–7 April 2018; pp. 168–172. [CrossRef]
29. Srinivasu, P.N.; SivaSai, J.G.; Ijaz, M.F.; Bhoi, A.K.; Kim, W.; Kang, J.J. Classification of skin disease using deep learning neural
networks with MobileNet V2 and LSTM. Sensors 2021, 21, 2852. [CrossRef] [PubMed]
30. Dang, Y.; Jiang, N.; Hu, H.; Ji, Z.; Zhang, W. Image classification based on quantum K-nearest-neighbor algorithm. Quantum Inf.
Process. 2018, 17, 239. [CrossRef]
31. Sumithra, R.; Suhil, M.; Guru, D.S. Segmentation and classification of skin lesions for disease diagnosis. Procedia Comput. Sci.
2015, 45, 76–85. [CrossRef]
32. Sajid, P.M.; Rajesh, D.A. Performance evaluation of classifiers for automatic early detection of skin cancer. J. Adv. Res. Dyn.
Control. Syst. 2018, 10, 454–461.
Appl. Sci. 2022, 12, 2677 20 of 21

33. Zhang, S.; Wu, Y.; Chang, J. Survey of image recognition algorithms. In Proceedings of the 2020 IEEE 4th Information Technology,
Networking, Electronic and Automation Control Conference (ITNEC), Chongqing, China, 12–14 June 2020; pp. 542–548.
[CrossRef]
34. Alam, M.; Munia, T.T.K.; Tavakolian, K.; Vasefi, F.; MacKinnon, N.; Fazel-Rezai, R. Automatic detection and severity measurement
of eczema using image processing. In Proceedings of the 2016 38th Annual International Conference of the IEEE Engineering in
Medicine and Biology Society (EMBC), Orlando, FL, USA, 16–20 August 2016; pp. 1365–1368. [CrossRef]
35. Immagulate, I.; Vijaya, M.S. Categorization of non-melanoma skin lesion diseases using support vector machine and its variants.
Int. J. Med. Imaging 2015, 3, 34–40. [CrossRef]
36. Upadhyay, P.K.; Chandra, S. An improved bag of dense features for skin lesion recognition. J. King Saud Univ. Comput. Inf. Sci.
2019; in press. [CrossRef]
37. Awad, M.; Khanna, R. Support vector machines for classification. In Efficient Learning Machines; Apress: Berkeley, CA, USA, 2015;
pp. 39–66.
38. Hsu, W. Bayesian classification. In Encyclopedia of Database Systems, 2nd ed.; Liu, L., Özsu, M.T., Eds.; Springer: New York, NY,
USA, 2018; pp. 3854–3857.
39. Tahmassebi, A.; Gandomi, A.; Schulte, M.; Goudriaan, A.; Foo, S.; Meyer-Base, A. Optimized naive-bayes and decision tree
approaches for fMRI smoking cessation classification. Complexity 2018, 2018, 2740817. [CrossRef]
40. Seixas, J.L.; Mantovani, R.G. Decision trees for the detection of skin lesion patterns in lower limbs ulcers. In Proceedings of the
2016 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA, 15–17
December 2018; pp. 677–681. [CrossRef]
41. Arasi, M.A.; El-Horbaty, E.S.M.; El-Sayed, A. Classification of dermoscopy images using naive bayesian and decision tree
techniques. In Proceedings of the 2018 1st Annual International Conference on Information and Sciences (AiCIS), Fallujah, Iraq,
20–21 November 2018; pp. 7–12. [CrossRef]
42. Hamad, M.A.; Zeki, A.M. Accuracy vs. cost in decision trees: A survey. In Proceedings of the 2018 International Conference
on Innovation and Intelligence for Informatics, Computing, and Technologies (3ICT), Sakhier, Bahrain, 18–20 November 2018;
pp. 1–4. [CrossRef]
43. Serte, S.; Demirel, H. Gabor wavelet-based deep learning for skin lesion classification. Comput. Biol. Med. 2019, 113, 103423.
[CrossRef]
44. Menegola, A.; Tavares, J.; Fornaciali, M.; Li, L.T.; Avila, S.; Valle, E. RECOD Titans at ISIC Challenge 2017. 2017. Available online:
https://arxiv.org/abs/1703.04819 (accessed on 19 October 2021).
45. Han, S.S.; Kim, M.S.; Lim, W.; Park, G.H.; Park, I.; Chang, S.E. Classification of the clinical images for benign and malignant
cutaneous tumors using a deep learning algorithm. J. Investig. Dermatol. 2018, 138, 1529–1538. [CrossRef]
46. Esteva, A.; Kuprel, B.; Novoa, R.A.; Ko, J.; Swetter, S.M.; Blau, H.M.; Thrun, S. Dermatologist-level classification of skin cancer
with deep neural networks. Nature 2017, 542, 115–118. [CrossRef] [PubMed]
47. Fujisawa, Y.; Otomo, Y.; Ogata, Y.; Nakamura, Y.; Fujita, R.; Ishitsuka, Y.; Fujimoto, M. Deep-learning-based, computer-aided
classifier developed with a small dataset of clinical images surpasses boardcertified dermatologists in skin tumour diagnosis.
Br. J. Dermatol. 2019, 180, 373–381. [CrossRef]
48. Mahbod, A.; Ecker, R.; Ellinger, I. Skin lesion classification using hybrid deep neural networks. In Proceedings of the ICASSP
2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK, 12–17 May 2019;
pp. 1229–1233.
49. Harangi, B. Skin lesion classification with ensembles of deep convolutional neural networks. Biomed. Inf. 2018, 86, 25–32.
[CrossRef] [PubMed]
50. Nyíri, T.; Kiss, A. Novel ensembling methods for dermatological image classification. In Proceedings of the International
Conference on Theory and Practice of Natural Computing, Dublin, Ireland, 12–14 December 2018; pp. 438–448.
51. Matsunaga, K.; Hamada, A.; Minagawa, A.; Koga, H. Image classification of melanoma, nevus and seborrheic keratosis by deep
neural network ensemble. arXiv 2017, arXiv:1703.03108.
52. Li, Y.; Shen, L. Skin lesion analysis towards melanoma detection using deep learning network. Sensors 2018, 18, 556. [CrossRef]
53. Díaz, I.G. Incorporating the knowledge of dermatologists to convolutional neural networks for the diagnosis of skin lesions.
arXiv 2017, arXiv:1703.01976v1.
54. Son, H.M.; Jeon, W.; Kim, J.; Heo, C.Y.; Yoon, H.J.; Park, J.U.; Chung, T.M. AI-based localization and classification of skin disease
with erythema. Sci. Rep. 2021, 11, 1–14. [CrossRef]
55. Al-Masni, M.A.; Kim, D.H.; Kim, T.S. Multiple skin lesions diagnostics via integrated deep convolutional networks for segmenta-
tion and classification. Comput. Methods Programs Biomed. 2020, 190, 105351. [CrossRef]
56. Truong, M.T.N.; Kim, S. Automatic image thresholding using Otsu’s method and entropy weighting scheme for surface defect
detection. Soft Comput. 2018, 22, 4197–4203. [CrossRef]
57. Zhan, Y.; Zhang, G. An improved OTSU algorithm using histogram accumulation moment for ore segmentation. Symmetry 2019,
11, 431. [CrossRef]
58. Zade, S. Medical-Image-Segmentation. Available online: https://github.com/mathworks/Medical-Image-Segmentation/
releases/tag/v1.0 (accessed on 19 October 2021).
Appl. Sci. 2022, 12, 2677 21 of 21

59. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. In Proceedings
of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA, 3–6 December 2012;
pp. 1097–1105.
60. Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards real-time object detection with region proposal networks.
In Proceedings of the Advances in Neural Information Processing Systems 28 (NIPS 2015), Montreal, QC, Canada, 7–12 December
2015; pp. 91–99.
61. Zhang, X.; Zhou, X.; Lin, M.; Sun, J. Shufflenet: An extremely efficient convolutional neural network for mobile devices. arXiv
2017, arXiv:1707.01083.
62. He, K.; Sun, J. Convolutional neural networks at constrained time cost. In Proceedings of the IEEE Conference on Computer
Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 5353–5360. [CrossRef]
63. Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with
convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 8–10 June
2015; pp. 1–9. [CrossRef]
64. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [CrossRef]
65. He, K.; Zhang, X.; Ren, S.; Sun, J. Identity mappings in deep residual networks. In Proceedings of the European Conference on
Computer Vision, Amsterdam, The Netherlands, 11–14 October 2016. [CrossRef]
66. Iandola, F.N.; Han, S.; Moskewicz, M.W.; Ashraf, K.; Dally, W.J.; Keutzer, K. Squeezenet: Alexnet-level accuracy with 50× fewer
parameters and 0.5 mb model size. arXiv 2016, arXiv:1602.07360.
67. Xie, S.; Girshick, R.; Dollár, P.; Tu, Z.; He, K. Aggregated residual transformations for deep neural networks. arXiv 2016,
arXiv:1611.05431.
68. Howard, A.G.; Zhu, M.; Chen, B.; Kalenichenko, D.; Wang, W.; Weyand, T.; Adam, H. Mobilenets: Efficient convolutional neural
networks for mobile vision applications. arXiv 2017, arXiv:1704.04861.
69. Zhang, T.; Qi, G.; Xiao, B.; Wang, J. Interleaved group convolutions for deep neural networks. arXiv 2017, arXiv:1707.02725.
70. Chollet, F. Xception: Deep learning with depthwise separable convolutions. arXiv 2016, arXiv:1610.02357.
71. Zagoruyko, S.; Komodaki, N. Wide residual networks. arXiv 2016, arXiv:1605.07146.
72. Hoang, H.H.; Trinh, H.H. Improvement for convolutional neural networks in image classification using long skip connection.
Appl. Sci. 2021, 11, 2092. [CrossRef]
73. Yahya, A.A.; Tan, J.; Hu, M. A novel handwritten digit classification system based on convolutional neural network approach.
Sensors 2021, 21, 6273. [CrossRef]
74. Xu, B.; Wang, N.; Chen, T.; Li, M. Empirical evaluation of rectified activations in convolutional network. arXiv 2015,
arXiv:1505.00853.
75. Tschandl, P.; Rosendahl, C.; Kittler, H. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of
common pigmented skin lesions. Sci. Data 2018, 5, 180161. [CrossRef] [PubMed]
76. Zanddizari, H.; Nguyen, N.; Zeinali, B.; Chang, J.M. A new preprocessing approach to improve the performance of CNN-based
skin lesion classification. Med. Biol. Eng. Comput. 2021, 59, 1123–1131. [CrossRef] [PubMed]
77. Milton, M.A.A. Automated skin lesion classification using ensemble of deep neural networks in ISIC 2018: Skin lesion analysis
towards melanoma detection challenge. arXiv 2019, arXiv:1901.10802.
78. Ray, S. Disease classification within dermascopic images using features extracted by ResNet50 and classification through deep
forest. arXiv 2018, arXiv:1807.05711.
79. Perez, F.; Avila, S.; Valle, E. Solo or ensemble? Choosing a CNN architecture for melanoma classification. arXiv 2019,
arXiv:1904.12724.
80. Gessert, N.; Sentker, T.; Madesta, F.; Schmitz, R.; Kniep, H.; Baltruschat, I.; Werner, R.; Schlaefer, A. Skin lesion diagnosis using
ensembles, unscaled multi-crop evaluation and loss weighting. arXiv 2018, arXiv:1808.01694.
81. Mobiny, A.; Singh, A.; Van Nguyen, H. Risk-aware machine learning classifier for skin lesion diagnosis. J. Clin. Med. 2019, 8, 1241.
[CrossRef]
82. Naga, S.P.; Rao, T.; Balas, V. A systematic approach for identification of tumor regions in the human brain through HARIS
algorithm. In Deep Learning Techniques for Biomedical and Health Informatics; Academic Press: Cambridge, MA, USA, 2020;
pp. 97–118. [CrossRef]
83. Cetinic, E.; Lipic, T.; Grgic, S. Fine-tuning convolutional neural networks for fine art classification. Expert Syst. Appl. 2018, 114,
107–118. [CrossRef]
84. Rathod, J.; Waghmode, V.; Sodha, A.; Bhavathankar, P. Diagnosis of skin diseases using convolutional neural networks. In
Proceedings of the 2018 Second International Conference on Electronics, Communication and Aerospace Technology (ICECA),
Coimbatore, India, 29–31 March 2018; pp. 1048–1051. [CrossRef]
85. Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556.
86. Hartanto, C.A.; Wibowo, A. Development of mobile skin cancer detection using faster R-CNN and MobileNet V2 model.
In Proceedings of the 2020 7th International Conference on Information Technology, Computer, and Electrical Engineering
(ICITACEE), Semarang, Indonesia, 24–25 September 2020; pp. 58–63.
87. Tan, M.; Le, Q. Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv 2019, arXiv:1905.11946.