CSA-DE-LR: enhancing cardiovascular disease diagnosis with a novel hybrid machine learning approach

Machine learning techniques

In the initial research works, ML algorithms such as LR, XGBoost, SVM, NB, etc. and similar methods were used for CVD prediction (Kolukısa et al., 2019; Kolukisa et al., 2020; Kolukisa & Bakir-Gungor, 2023; Dhanka, Bhardwaj & Maini, 2023).

Kolukısa et al. (2019) expanded the range of feature selection methodologies to enhance performance. Additionally, Fisher linear discriminant analysis was applied to reduce computational time by decreasing the number of features in diagnosing coronary artery disease, resulting in well-performing models for each dataset. Utilizing the MLP classifier, they achieved an accuracy of 82.5% and an F-measure of 83.80% on the Cleveland dataset. In Kolukisa et al. (2020), a novel self-optimized and adaptive ensemble ML algorithm was introduced. This algorithm autonomously identifies the most appropriate machine learning models, ensuring high accuracy across diverse coronary artery disease datasets. On the Cleveland dataset, an accuracy of 83.43% was achieved. Furthermore, Kolukisa and Bakir-Gungor’s work (2023), which incorporated the Z-Alizadehsani, Cleveland, and Statlog datasets, proposed an exhaustive ensemble feature selection (FS) method and a probabilistic ensemble FS approach. The evaluation encompassed six distinct classification algorithms and four variants of voting algorithms. The obtained accuracy values were 85.47% for the Cleveland dataset and 85.55% for the Statlog dataset.

Dhanka, Bhardwaj & Maini (2023) present a thorough examination of LR and XGBoost in the Statlog heart disease dataset as a benchmark. Model parameters are optimized through Random SearchCV hyperparameter tuning. The study encompasses an analysis of both non-optimized and optimized models. The results derived from 10-fold cross-validation indicate that LR and XGBoost accuracies are 85.2% and 81.5%, respectively.

Although standard ML techniques have shown notable success in CVD prediction, they often struggle with handling complex and multidimensional data, which can lead to lower success rates (Ramudu et al., 2023). Furthermore, the local minima problem presents a major obstacle to typical machine learning approaches, affecting algorithms’ convergence to optimum solutions. The complexity and non-convexity of objective functions, which can have several local minima in addition to a global minimum, are at the core of this problem. Points in the parameter space where the objective function approaches a local low are known as local minima, however they are not always the lowest points overall. In order to minimize the objective function, model parameters are frequently updated iteratively using gradient-based optimization techniques such as gradient descent. Nevertheless, if initialization or parameter updates push these algorithms into areas of the parameter space that match these less-than-ideal solutions, they may become stuck in local minima (Ghassemi et al., 2020). Reaching the best possible model performance requires breaking out of such local minima. To address these problems, various metaheuristics algorithms have been employed by researchers with ML algorithms.

Hybrid approaches using metaheuristics and ML algorithms

In the literature, metaheuristics have been employed by researchers to select features, to optimize parameters, and to train the standard ML algorithms for improving their classification accuracy by avoiding local minima.

Most studies in the literature have used metaheuristic approaches for feature selection problems without focusing on training ML algorithms, to select the most appropriate features among different types of CVD datasets (Murugesan et al., 2021; Muliawan, Rizal & Hadiyoso, 2023; Torthi et al., 2024; Sampathkumar & Periyasamy, 2024). These works differ from the proposed work in that metaheuristic methods are used to train ML algorithms while preserving their good qualities. In feature selection problems, metaheuristics help to reduce dimensionality and speed up computation time. One of these studies is the work of Murugesan et al. (2021), where authors created a super learner by fusing three bioinspired algorithms with ANN. Using the methods for BFO (bacterial foraging optimization), KH (krill herd), and CSO (cat swarm optimization), three sets of features were chosen. Using the characteristics chosen by each method, a backpropagation neural network (BPNN) was trained. The Statlog dataset yielded an accuracy of 86.36%, whereas the Cleveland dataset produced an accuracy of 84%.

The study by Muliawan, Rizal & Hadiyoso (2023) uses ensemble classifiers with parameter optimization to predict heart disease using a public dataset from the UCI machine learning repository. The dataset includes 13 variables influencing heart disease. Particle swarm optimization (PSO) was used for feature selection and principal component analysis (PCA) for feature extraction. Parameter optimization was applied to machine learning methods like SVM, deep learning, and ensemble classifier. The results showed the highest accuracy in Deep Learning and SVM parameters, with bagging on SVM achieving 83.51% accuracy.

Torthi et al. (2024) proposed BAPSO-RF, a Bat algorithm and particle swarm optimization-based Random Forest, to improve heart disease prediction accuracy. Using 270 records and 14 variables from the UCI heart disease dataset, the proposed BAPSO-RF is assessed. The model outperformed other methods like GAPSO-RF, GA, and GA-RBF by using metrics like accuracy, precision, recall, and f1-score values of approximately 98.71%, 98.67%, 98.23%, and 98.45%, respectively.

Sampathkumar & Periyasamy (2024) proposed a method using binary particle swarm optimization and attention-based deep network (BPSO-ADN) to extract significant features from a cardiac dataset for improved prediction accuracy. The technique uses BPSO for feature selection and ADN for detailed pattern analysis, with BPSO directed by a fitness evaluation method to find the most suitable subset for heart disease prediction.

In some research works, metaheuristics have been employed by researchers to optimize parameters for improving their classification accuracy (Nalluri et al., 2017; Dhanka & Maini, 2024). These works differ from the proposed work in that metaheuristic methods are used to train ML algorithms while preserving their good qualities. Metaheuristics are a useful tool for effectively exploring complicated search spaces in hyperparameter optimization issues. This aids in the finding of optimum or nearly optimal configurations for machine learning models. These techniques, which use parallelization, stochastic search, and adaptive exploration to break free from local optima and enhance model performance over a wide range of tasks and datasets, provide adaptable and reliable solutions. One of these studies is the work of Nalluri et al. (2017), where authors diagnosed heart disease using the hybrid system. The classification was done using SVM and MLP classifiers. The parameters were optimized using three evolutionary algorithms: PSO, FA (frefy algorithm), and GSA (gravity search algorithm). Learning rate and momentum were optimized in MLP. Margin was optimized in SVM. Five datasets related to cardiovascular disease were used to verify the system. On the Cleveland dataset, 90.74% on the Statlog dataset, 89.5% on the SPECT dataset, 90.6% on the SPECTF dataset, and 91.4% on the Eric dataset, the system achieved an accuracy of 94.1%.

Dhanka & Maini (2024) developed two clever models—HyOPTRF (Model 1) and HyOPTXGBoost Classifier (Model 2)—that were applied to the Statlog HD dataset using both hyper-tuned and default parameters. On Trial (2) the HyOPTRF recorded the highest Accuracy of 92.59% and F1 score 93.75%, while on Trial (33) the HyOPTXGBoost Classifier recorded the highest Accuracy of 96.30% and F1 Score 96.77%. The suggested models were compared to the other models that were already in use and verified using the Stratify Kfold Cross-Validation approach.

In a few research works, metaheuristics have been employed by researchers to train the standard ML algorithms for improving their classification accuracy by avoiding local minima (Leema, Nehemiah & Kannan, 2016; Arabasadi et al., 2017; Poornima & Gladis, 2018; Shahid & Singh, 2020; Al Bataineh & Manacek, 2022). The resulting algorithms which combine metaheuristics and ML algorithms are referred as hybrid algorithms in the literature. For example, In Leema, Nehemiah & Kannan (2016), the authors aimed to enhance the performance of ANNs by applying a hybrid optimization algorithm. This approach was implemented on three benchmark datasets: Wisconsin Breast Cancer, Pima Indian Diabetes, and Cleveland. The ANN training incorporated a combination of DE and PSO for global search and the backpropagation (BP) algorithm for local search. Prior to constructing the model, the datasets underwent min-max normalization. A 10-fold cross-validation was conducted, and the proposed approach, termed Differential Evolution with Global Information and Back Propagation (DEGI-BP), was compared with DE-BP and PSO-BP. The experiments conducted on the Cleveland dataset demonstrated that the proposed approach outperformed other hybrid optimization algorithms, achieving an accuracy of 86.66%.

A neural network-based approach for diagnosing CAD was proposed by Arabasadi et al. (2017). GA optimized the neural network’s weights. The ANN was trained using the backpropagation technique. GA started with an initial population of 100 chromosomes. The root mean square error, or RMSE, of the untrained ANN was used to determine the chromosomes’ fitness value. GA employed the roulette wheel algorithm for selection. With a crossover probability of 1, two-point crossover was employed. Gaussian mutation was used to carry out the mutation. Every gene on a chromosome carried one neural network weight, and every chromosome had all of the neural network’s weights. SVM was used to choose features. The system was evaluated on the Z-Alizadeh Sani dataset. The system achieved an accuracy of 93.85%.

A hybrid classifier was presented by Poornima & Gladis (2018) to predict cardiac disease. Using the orthogonal local preserving projection (OLPP), features were chosen. The ANN was used to carry out the classification. The neural network consisted of four neurons in the input layer, one hundred neurons in the hidden layer, and five neurons in the output layer. The range of weights for the connections between neurons was between −10 and 10. Levenberg–Marquardt (LM) and group search optimization (GSO) were used to optimize the network by determining the weights. The optimal weights in the network were selected from the two sets of weights that LM and GSO had produced. The results were validated using three datasets: Switzerland, Hungarian, and Cleveland. Using the Cleveland dataset, the accuracy rate of the method was 94%.

Shahid & Singh (2020) introduced a pioneering approach, amalgamating PSO with an emotional neural network (EmNN). The performance of this novel approach was compared with a hybrid model named PSO-ANFIS, which integrates an artificial neural network (ANN) with fuzzy logic. The study concentrated on leveraging brain-based emotional learning within EmNNs, renowned for their heightened accuracy. PSO was employed to optimize the proposed neural network. The evaluation encompassed three datasets: Z-Alizadeh Sani, Cleveland, and Statlog. While data preprocessing was omitted, feature selection was conducted, resulting in the selection of 8 features for the Statlog dataset and 7 features for the Cleveland dataset. The achieved accuracy was 84% for the Cleveland dataset and 85.2% for the Statlog dataset.

In a study conducted by Al Bataineh & Manacek (2022) utilizing the Cleveland dataset with 13 features and 303 samples, a hybrid algorithm named MLP-PSO was introduced. This algorithm involved replacing missing values with feature-specific mean values and incorporated categorical data encoding and feature scaling techniques. The MLP model was trained using weights and biases optimized through the particle swarm optimization (PSO) algorithm. Performance evaluation was conducted using 5-fold cross-validation, and hyperparameter tuning was executed through the grid search method. Upon comparing the results with ten different machine learning algorithms, the proposed MLP-PSO method demonstrated the highest accuracy, reaching 84.60%. To enhance classification performance, they refined the feature extraction process and training step of a neural network (NN) following the methodology described in Cherian, Thomas & Venkitachalam (2020). Statistical and higher-order statistical features were extracted from the dataset, and principal component analysis (PCA) was performed.

The findings demonstrate that metaheuristic-based ML methods for diagnosing CVD exhibit superior performance over alternative ML techniques in terms of different performance criteria. However, these methods generally require significant time, to improve regarding low detection rates and effort to achieve high classification performance, which negatively impacts their applicability and effectiveness. To minimize these disadvantages of current hybrid methods, the ML method to be used must first be computationally efficient for large datasets and have simplicity-interpretability in disease diagnosis problems. Afterwards, it is necessary to choose the metaheuristic algorithm that will best overcome the disadvantages of the ML method to be used and will be successful for the train and suitable for the relevant problem (Naser et al., 2024).

To eliminate the shortcomings of existing studies and increase the low detection rate in CVD prediction, a pioneering hybrid classifier, denoted as CSA-DE-LR, is introduced in this research endeavor (Naser et al., 2024). In this study, the diagnosis of CVD utilizes LR as a classification model because it is computationally efficient for large data sets and simple interpretability in disease diagnosis problems. Then, CSA-DE optimization method employees for model training instead of the gradient descent algorithm to improve LR’s classification accuracy by avoiding local minima.

The reasons for choosing CSA-DE as an optimization method can be justified as follows. The CSA is a crucial AIS algorithm that improves the immune system’s response to antigens by generating antibodies with greater affinity over time. CSA is popular for optimization tasks because it can search for local and global solutions in the solution space using hyper-mutation and receptor editing processes (Rahman et al., 2023). Furthermore, studies have demonstrated that CSA-based techniques outperform other bio-inspired and optimization methods in various contexts (Haktanirlar Ulutas & Kulturel-Konak, 2011; Duru et al., 2022; Rahman et al., 2023). However, traditional CSA techniques may not offer sufficient search capabilities and could benefit from improvement (Gong, Jiao & Zhang, 2010; Zhang et al., 2008; Xu et al., 2017). Therefore, to improve the local search performance of the CSA method, a hybrid optimization method can be developed by using methods that are known to have remarkable search performance, such as the differential evolution (DE) algorithm (Mostafa et al., 2024; Azevedo, Rocha & Pereira, 2024; Song et al., 2024).

As a summary, proposed hybrid classifier CSA-DE-LR combines two optimization algorithms, leveraging their strengths and applying them to LR during training to attain heightened performance. Significantly, it offers three optimization options—based on F1 score, MCC, and MAE—enhancing its adaptability. Additionally, implementing feature selection in this study significantly enhanced the outcomes. By identifying and utilizing the most relevant features, the CSA-DE-LR method achieved remarkable accuracy and efficiency, demonstrating the value of meticulous feature selection in improving diagnostic models for CVD. Employing Bayesian optimization for fine-tuning hyperparameters and utilizing ten-fold cross-validation, CSA-DE-LR demonstrates a notable improvement in diagnostic accuracy across datasets, presenting a substantial advancement in medical diagnostics.

Logistic regression

Selecting Logistic Regression (LR) is a form of statistical modeling that’s especially suitable for situations where the outcome variable is binary, meaning it takes on two possible outcomes. The primary concept behind LR is to model the probability that a given input point belongs to a particular category. This probability estimation is achieved by fitting the data to a logistic curve, hence the name “Logistic Regression.”

The training set of features $\left\{\left({\overrightarrow{x}}_1,y_1\right),\dots ,\left({\overrightarrow{x}}_M,y_M\right)\right\}$ comprises M instances. Each instance has a feature vector ${\overrightarrow{x}}_i\in R^D$ and the corresponding y_i is the label for each feature vector, which, in this binary classification scenario, can either be 0 or 1.

In mathematical terms, the prediction for a given ${\overrightarrow{x}}_i$ is determined by inputting the weighted sum of its features into the sigmoid function, and Eq. (1). is used to describe the class of Eq. (1). (1)${\displaystyle y_i^{{}^{\prime }}=\left\{0,p_i<0.51,p_i\ge 0.5\right.}$

where p_i is determined by Eq. (2). LR uses the sigmoid function, which outputs a value between 0 and 1 for any input as outlined in Eq. (3). This value can be interpreted as the probability that the input instance belongs to the class labeled as 1. (2)${\displaystyle p_i=\sigma \left(\overrightarrow{w}{\overrightarrow{x}}_i\right)}$ (3)${\displaystyle \sigma \left(a\right)=\frac{1}{1+e^{-a}}.}$

The goal of the learning algorithm is to adjust its internal parameters (typically weights associated with each feature and a bias term) to minimize the difference between its predicted probabilities and the actual outcomes in the training set. This difference is often captured by a cross-entropy cost function given in Eq. (4). (4)${\displaystyle J\left(\overrightarrow{w}\right)=-\sum _{i=1}^m{y}_{i}log\left(p_i\right)+\left(1-y_i\right)log\left(1-p_i\right)}$

Clonal selection algorithm

The CSA (de Castro & Von Zuben, 2002) emulates the adaptive immune system’s response to antigenic stimuli. An algorithm, CLONALG, was developed based on the principles of clonal selection and affinity maturation inherent in immune responses. Different adaptations of CLONALG were employed to handle pattern recognition and optimization problems (de Castro & Von Zuben, 2002; Azevedo, Rocha & Pereira, 2024; Duru et al., 2022). For the application of CSA, a version of CLONALG optimized for such tasks was employed. The primary objective of CSA is to identify an antibody with peak affinity (de Castro & Von Zuben, 2002; Rahman et al., 2023). The key steps of CSA are outlined in Algorithm 1 .

 
_______________________ 
Algorithm 1 The Clonal Section Algorithm_____________________________________________ 
 1:  Set up the control parameters:  total antibodies (P), the highest iteration count, 
     receptor editing rate (B), and clonal multiplication coefficient (α). 
 2:  Create an initial set of P antibodies. 
 3:  Determine the affinity score for every antibody Ab. 
 4:  for each iteration do 
 5:       Clone the antibodies α times and calculate the affinity scores for these clones. 
 6:       for each clone Ci do 
 7:           Apply reverse mutation to Ci, resulting in the mutated clone σi. 
 8:           if f(σi) > f(Ci) then 
 9:                Ci := σi 
10:           else 
11:                Carry out pair-wise mutation on Ci to produce σi. 
12:                Compute the affinity value of σi. 
13:                if f(σi) > f(Ci) then 
14:                     Ci := σi 
15:                else 
16:                     Ci := Ci 
17:                end if 
18:           end if 
19:       end for 
20:       for each antibody Abi do 
21:           Choose the clone Cj from Abi clones with the topmost affinity 
22:           Abi := Cj 
23:       end for 
24:       Substitute the lowest performing B% of antibodies with the newly formed ones. 
25:  end for____________________________________________________________________________________    

The set of P antibodies Ab = {Ab₁, Ab₂, …, Ab_P} is initially formed randomly as indicated in Eq. (5). This set undergoes enhancement in every cycle through processes like selection, cloning, hyper-mutation, re-selection, and receptor-editing until the highest iteration count is reached (de Castro & Von Zuben, 2002). Each antibody Ab_i = [Ab_i,1, Ab_i,2, …, Ab_i,D] ∈ R^D within the group represents a potential solution. The ultimate goal is to identify an antibody with the utmost affinity score once all cycles have concluded. (5)${\displaystyle A{b}_{i,j}=l{b}_j+rand\left(0,1\right)\times \left(u{b}_j-l{b}_j\right)}$

where rand(0, 1) is a function producing random numbers uniformly spread between 0 and 1. The terms lb_j and ub_j denote the minimum and maximum limits for the jth parameter, respectively. After establishing a population of P antibodies, the fitness score for every antibody in the Ab set can be determined as illustrated in Eq. (6). (6)${\displaystyle f\left(A{b}_i\right)=\frac{1}{1+J\left(A{b}_i\right)}}$

f(Ab_i) represents the fitness function determining the fitness score of Ab_i. Meanwhile, J(Ab_i) serves as the cost function, as depicted in Eq. (4), providing the cost measure of Ab_i. The clone count(α_i) for every chosen antibody Ab may remain consistent (de Castro & Von Zuben, 2002). The aggregate count of clones within the clone group C is derived from Eq. (7). (7)${\displaystyle \left|C\right|=\sum _{i=1}^n{\alpha }_i=\alpha \times n}$

where α represents the clonal multiplication coefficient and is a positive whole number. In this study, every antibody in the group is chosen for cloning (n = P), and the quantity of clones for each selected antibody remains consistent (α_i = α) to aid in identifying multiple optimal solutions (de Castro & Von Zuben, 2002).

Following the formation of the clone group, the antibodies undergo enhancement via hyper-mutation processes, namely inverse mutation and pair-wise mutation. In the inverse mutation method, for each clone C_i = [C₁, C₂, …, C_D], parameters j and l are chosen at random, ensuring |j − l| > 2, and the parameters between j and l within C_i are inverted to produce the mutated clone σ_i. If σ_i’s affinity surpasses that of C_i, it replaces C_i. If not, pair-wise mutation is applied to C_i. Here, parameters j and l of C_i are randomly selected and swapped. The affinity of σ_i, resulting from pair-wise mutation, is assessed. If σ_i’s affinity is superior to that of C_i, it takes the place of C_i; if not, C_i remains unaltered.

Post hyper-mutation, a re-selection step ensures the antibody population size stays consistent. For each antibody, Ab_i, the highest affinity clone among Ab_i’s clones is chosen and allocated to Ab_i. Concludingly, receptor editing is executed, substituting the least efficient B% of antibodies with new ones. The procedures of the CSA optimization approach are detailed in Algorithm 1 .

Differential evolution

The DE algorithm (Storn & Price, 1997) operates as a collective method that encompasses processes like crossover, mutation, and selection. Its core mechanism hinges on mutation, which derives from the distinctions between randomly chosen solution pairs within the collective. This algorithm harnesses mutation as an exploration tool and the selection process to guide the exploration towards favorable areas in the solution environment. The DE algorithm also employs a distinctive crossover that may favor parameters from one parent over another. By leveraging attributes from current collective members to formulate trial solutions, the crossover operator adeptly redistributes insights about potent combinations, facilitating a more effective search for optimal solutions. Initially, DE establishes a random set of solution vectors. This set undergoes enhancements through the application of mutation, crossover, and selection processes. In the DE method, every newly generated solution is compared against a mutated one, and the superior of the two emerges victorious. The DE algorithm has captured the attention of scholars in diverse fields and has proven valuable in solving numerous real-world challenges (Storn & Price, 1997; Corne et al., 1999; Mostafa et al., 2024; Song et al., 2024; Azevedo, Rocha & Pereira, 2024). The essential steps of the DE algorithm are described as follows:

 
_________________________________________________________________________________________________ 
Algorithm 2 Differential Evolution Algorithm_________________________________________ 
 1:  Initialize Population 
 2:  Evaluation 
 3:  repeat 
 4:       Mutation 
 5:       Recombination 
 6:       Evaluation 
 7:       Selection 
 8:  until requirements are met_______________________________________________________________________    

During the mutation process, all of the M parameter vectors are subjected to mutation. This mutation step broadens the exploration area. A mutated solution vector, denoted as w_i, is formulated by Eq. (8): (8)${\displaystyle {\overrightarrow{w}}_i^{{}^{\prime }}={\overrightarrow{w}}_{i_1}+sf\times \left({\overrightarrow{w}}_{i_3}-{\overrightarrow{w}}_{i_2}\right),1\le i\le M}$ where sf represents the scaling factor with values from [0, 1], and the solution vectors i₁, i₂, and i₃ are selected randomly and are required to conform to i₁ ≠ i₂ ≠ i₃ ≠ i, where i represents the current solution’s index. During the crossover procedure, the parent vector merges with the mutated vector, generating a trial vector as given in Eq. (9). (9)${\displaystyle {\overrightarrow{w}}_{i,j}=\left\{{\overrightarrow{w}}_{i,j}^{{}^{\prime }},r_j\le cr{\overrightarrow{w}}_{i,j},r_j>cr\right.}$ where cr represents the crossover constant, r_j is a real number picked randomly from the range [0, 1], and j indicates the jth element of the related array.

Every solution within the population possesses an equal probability of being chosen as a parent, regardless of its fitness value. After undergoing mutation and crossover processes, the offspring’s performance is assessed. Subsequently, a comparison between the offspring and its parent takes place, with the superior entity prevailing. If the parent remains superior, it is preserved in the population.

Proposed method (CSA-DE-LR)

This study introduces a novel classification methodology, leveraging a hybrid approach that combines CSA and DE to optimize the LR weights for classification tasks. The proposed method, henceforth referred to as CSA-DE-LR, offers three optimization techniques based on different performance metrics: F1 score, MCC, and MAE. These metrics guide the training process to fine-tune the model weights to achieve an optimal balance between precision and generalizability.

 
________________________________________________________________________________________________________________ 
Algorithm 3 Proposed CSA-DE-LR classification method__________________________ 
1:  Determine the input parameters: Input matrix XM×N, target yM, number 
of antibodies P, population of P antibodies WP×D, percentage of receptor editing 
B, maximum evaluation number MEN, lower bound lb, upper bound ub, number of 
clones for each antibody α, scaling factor sf, crossover rate cr 
Output: 
 1:  D ← N + 1 
 2:  W ← CreateAntibodies(P,D) 
 3:  W′ 
 ← W 
 4:  fit ← CalculateFitness(W) 
 5:  evaluation_number ← 0 
 6:  while evaluation_number < MEN do 
 7:       Cloning() 
 8:       LocalSearchV iaDE() 
 9:       Selection() 
10:       ReceptorEditing() 
11:       FindBestAntibody() 
12:  end while 
13:  return   gpar                                  ⊳ return global best antibody params    

 
________________________________________________________________________________________________________________ 
Algorithm 4 Create population of P antibodies_______________________________________ 
 1:  procedure CreateAntibodies(P, D) 
 2:       for i ← 1 : P do 
 3:           for j ← 1 : D do 
 4:                W[i,j] ← lb + rand(0, 1) × (ub − lb) 
 5:           end for 
 6:       end for 
 7:       return   W 
 8:  end procedure__________________________________________________________________________    

 
_________________________________________________________________________________________________ 
Algorithm 5 Clone each antibody α times______________________________________________ 
 1:  procedure Cloning() 
 2:       for i ← 1 : P do 
 3:           C[i × α : (i + 1) × α, :] ← W[i, :] 
 4:       end for 
 5:  end procedure__________________________________________________________________________ 

 
_________________________________________________________________________________________________ 
Algorithm 6 Local search via DE__________________________________________________________ 
 1:  procedure LocalSearchV iaDE() 
 2:       for i ← 1 : P × α do 
 3:           j ← i//α                                                    ⊳ //, floor division 
 4:           inds ←{x|x ∈ Z, 0 ≤ x < P,x ⁄= j} 
 5:           pars = rand_choice(inds, 3) × α + randint(0,α, 3)      ⊳ randomly select 3 
     neighbour clones 
 6:           arr ← C[pars[0], :] + sf × (C[pars[2], :] − C[pars[1], :]) 
 7:           ar ← rand(low = 0,high = 1,size = (D)) 
 8:           ρ ← ar ≤ cr                                                 ⊳ param to change 
 9:           C[i,ρ] ← arr[ρ] 
10:           vec ← C[i,ρ] 
11:           vec[vec < lb] ← lb 
12:           vec[vec > ub] ← ub 
13:           C[i,ρ] ← vec 
14:       end for 
15:       cfit ← CalculateFitness(C) 
16:  end procedure__________________________________________________________________________    

 
_________________________________________________________________________________________________ 
Algorithm 7 Selection Phase________________________________________________________________ 
 1:  procedure Selection() 
 2:       cfit′ 
 ← reshape(cfit,size = (P,α)) 
 3:       maxidxs ← argmax(cfit′ 
 ,axis = 1) 
 4:       inds ← [0 : 1 : P] × α 
 5:       idxs ← inds + maxidxs 
 6:       bestIdxs = cfit[idxs] > fit 
 7:       W[bestIdxs, :] = C[idxs, :][bestIdxs, :] 
 8:       fit[bestIdxs] = cfit[idxs][bestIdxs] 
 9:  end procedure__________________________________________________________________________ 

 
_________________________________________________________________________________________________ 
Algorithm 8 Receptor Editing Phase_____________________________________________________ 
 1:  procedure ReceptorEditing() 
 2:       fIndex ← argsort(fit) 
 3:       n ← round(P × B) 
 4:       worstNindex ← fIndex[0 : n] 
 5:       newNantibodies ← CreateAntibodies(n,D) 
 6:       newNfitness ← CalculateFitness(newNantibodies) 
 7:       W[worstNindex, :] = newNantibodies 
 8:       fit[worstNindex] ← newNfitness 
 9:  end procedure__________________________________________________________________________    

 
_________________________________________________________________________________________________ 
Algorithm 9 Find best antibody___________________________________________________________ 
 1:  procedure FindBestAntibody() 
 2:       index ← argmax(fit) 
 3:       gmax ← fit[index]                                                     ⊳ global maximum 
 4:       gpar ← W[index, :]                                                        ⊳ global params 
 5:  end procedure__________________________________________________________________________ 

 
_________________________________________________________________________________________________ 
Algorithm 10 Calculate prediction________________________________________________________ 
  1:  procedure CalculatePrediction(ϕ) 
  2:       w ← ϕ[:,1 :] 
  3:       b ← ϕ[:,0] 
  4:       prediction ← σ(X.dot(wT) + b)                            ⊳ σ is sigmoid func 
  5:       return   prediction 
 6:  end procedure___________________________________________________________________    

 
__________________________________________________________________________________________ 
Algorithm 11 Calculate fitness function using Matthews correlation coefficient 
 1:  procedure CalculateFitnessMCC(ϕ) 
 2:       a ← CalculatePrediction(ϕ) 
 3:       p ← round(a) ⊳ round function round elements 1 if elements ≥ 0.5, otherwise 
     0 
 4:       f ← MCC(yM,p) 
 5:       evaluation_number ← evaluation_number + len(f) 
 6:       return   f 
 7:  end procedure__________________________________________________________________________ 

 
_________________________________________________________________________________________________ 
Algorithm 12 Calculate fitness function using F1 score_____________________________ 
 1:  procedure CalculateFitnessF1(ϕ) 
 2:       a ← CalculatePrediction(ϕ) 
 3:       p ← round(a) ⊳ round function round elements 1 if elements ≥ 0.5, otherwise 
     0 
 4:       f ← F1(yM,p) 
 5:       evaluation_number ← evaluation_number + len(f) 
 6:       return   f 
 7:  end procedure__________________________________________________________________________    

 
_________________________________________________________________________________________________ 
Algorithm 13 Calculate fitness function using mean absolute error (MAE)____ 
 1:  procedure CalculateFitnessMAE(ϕ) 
 2:       p ← CalculatePrediction(ϕ) 
 3:       f ← MAE(yM,p) 
 4:       f ← 1/(f + 1) 
 5:       evaluation_number ← evaluation_number + len(f) 
 6:       return   f 
 7:  end procedure__________________________________________________________________________ 

 
_________________________________________________________________________________________________ 
Algorithm 14 Calculate Matthews correlation coefficient___________________________ 
 1:  procedure MCC(actual,predicted) 
 2:       tp ← sum(predicted ∗ actual,axis = 0) 
 3:       tn ← sum((1 − predicted) ∗ (1 − actual)),axis = 0) 
 4:       fp ← sum(predicted,axis = 0) − tp 
 5:       fn ← sum(actual,axis = 0) − tp 
 6:       mcc ← (tp ∗ tn − fp ∗ fn)/(tp + fn) ∗ (tp + fp) ∗ (tn + fn) ∗ (tn + fp) 
 7:       return   mcc 
 8:  end procedure__________________________________________________________________________    

 
_________________________________________________________________________________________________ 
Algorithm 15 Calculate F1 score__________________________________________________________ 
 1:  procedure F1(actual,predicted) 
 2:       tp ← sum(predicted ∗ actual,axis = 0) 
 3:       fp ← sum(predicted,axis = 0) − tp 
 4:       fn ← sum(actual,axis = 0),tp 
 5:       precision ← tp/(tp + fp) 
 6:       recall ← tp/(tp + fn) 
 7:       F1 ← 2 ∗ precision ∗ recall/(precision + recall) 
 8:       return   F1 
 9:  end procedure__________________________________________________________________________ 

 
_________________________________________________________________________________________________ 
Algorithm 16 Calculate MAE______________________________________________________________ 
 1:  procedure MSE(actual,predicted) 
 2:       mae = mean((actual − predicted)2,axis = 0) 
 3:       return   mae 
 4:  end procedure__________________________________________________________________________    

The CSA-DE-LR method begins by initializing a population of P antibodies, each representing a potential solution to the classification problem. The antibodies undergo cloning and local search procedures via DE, followed by a selection phase that favors the most promising candidates. Receptor editing is applied to introduce diversity by replacing a portion of the least-fit antibodies with new candidates. This iterative process continues until the maximum evaluation number (MEN) is reached, at which point it returns the best-performing antibody, indicative of the optimal model weights.

The detailed pseudocode for the CSA-DE-LR classification method is presented from Algorithm 3 to Algorithm 11 . Algorithm 3 outlines the main procedure, which utilizes sub-procedures defined in Algorithms 4 to Algorithm 11 . Each sub-procedure is dedicated to a specific task within the optimization process, including the initialization of the antibody population ( Algorithm 4 ), cloning ( Algorithm 5 ), local search via DE ( Algorithm 6 ), selection ( Algorithm 7 ), receptor editing ( Algorithm 8 ), and the identification of the best antibody ( Algorithm 9 ).

A critical phase within this process is detailed in Algorithm 7 , the Selection Phase, where the efficacy of each antibody’s clone is rigorously evaluated. Every antibody’s clones are compared, and the clone with the highest fitness value is identified. If this clone surpasses the original antibody in terms of fitness, it is preferentially selected as a superior solution. This approach ensures that the proposed model continuously evolves towards higher accuracy by adopting the most advantageous traits of each generation.

Following this, Algorithm 8 , the Receptor Editing Phase, comes into play. Here, antibodies are ranked based on their fitness values, and the bottom percentile, amounting to PxB antibodies, is identified for replacement. This mechanism introduces strategic diversity to the population by substituting the least-fit antibodies with newly created ones, thus preventing premature convergence and maintaining a robust search within the solution space.

Prediction calculations are performed per Algorithm 10, where the sigmoid function (σ) is applied to the weighted sum of inputs plus the bias term. The fitness of each antibody is evaluated using one of the three fitness functions (Algorithms 11 to 13), each corresponding to one of the selected optimization metrics.

Algorithms 14 to 16 encapsulate the calculation of the MCC, F1 score, and MAE, respectively. The MCC computation follows the standard formula involving true positives, true negatives, false positives, and false negatives. Similarly, the F1 score is computed using precision and recall derived from the confusion matrix. The MAE, on the other hand, is inverted to ensure that a lower error results in higher fitness.

By integrating CSA and DE with LR, the proposed CSA-DE-LR method aims to effectively navigate the search space and converge to an optimal set of weights for the logistic regression classifier, thereby enhancing classification accuracy and model robustness.

Datasets

In this study, four well-known datasets were used for empirical analysis: the Cleveland, Statlog, Breast Cancer Wisconsin Original (WBCO), and Breast Cancer Wisconsin Diagnostic (WBCD) datasets. All four datasets are publicly available through the UCI Machine Learning Repository and are commonly used in medical classification research.

• The Cleveland dataset contains 303 instances, each described by 13 attributes. It is used to classify instances as indicative of CAD or representing a healthy state.

• The Statlog dataset consists of 270 instances, also described by 13 attributes. It is similar in structure to the Cleveland dataset and is used to classify the presence of CAD.

• The WBCO dataset comprises 699 instances, each described by nine attributes based on biopsy data. This dataset is used to classify tumors as either benign or malignant.

• The WBCD dataset contains 569 instances, each described by 30 attributes. This dataset is used to classify tumors as benign or malignant.

Preprocessing

Data preprocessing was crucial to ensure consistent scaling and accurate results. First, any missing values were identified and addressed to maintain data integrity. In the Cleveland dataset, six instances with missing values were removed, and in the WBCO dataset, 16 instances containing missing values were excluded. The Statlog and WBCD datasets did not contain any missing values.

After addressing the missing data, the scaling process was carried out. For the Cleveland, Statlog, and WBCO datasets, the training data for each fold was normalized using the MinMaxScaler, which scales data values to the [0, 1] range. For the WBCD dataset, the StandardScaler was applied to normalize the data by centering and scaling based on the mean and standard deviation. The scalers were first fit and applied to the training data, and subsequently, the transformation was applied to the test data to ensure consistent scaling.

Following the scaling procedures, a 10-fold cross-validation process was employed to evaluate the model’s performance. This involved dividing the data into 10 equally sized folds. Each fold was used as a test set while the remaining nine folds formed a training set, allowing the model to be trained and evaluated 10 separate times. The individual results from each fold were then averaged to provide a more comprehensive and reliable assessment of the model’s overall performance.

Evaluation metrics

In this study, several key metrics were utilized to evaluate and compare the performance of the classification processes. These include:

• Accuracy (ACC): This metric measures the ratio of correctly predicted observations to the total observations and the formula for ACC is provided by Eq. (10). (10)${\displaystyle ACC=\frac{TP+TN}{TP+TN+FP+FN}}$ where TP = True Positives, TN = True Negatives, FP = False Positives, FN = False Negatives.

• F1 score: The F1 score is a metric that balances precision (the quality of the positives identified) and recall (the ability to find all relevant instances). The F1 score is calculated as: (11)${\displaystyle F1=2\times \frac{Precision\times Recall}{Precision+Recall}}$ where Precision = TP/(TP + FP), Recall = TP/(TP + FN). This metric is critical for understanding the model’s accuracy in classifying positive cases.

• Matthews correlation coefficient (MCC): This coefficient is a reliable statistical rate which yields a value between -1 and +1. It is especially useful for imbalanced datasets. The formula for MCC is: (12)${\displaystyle MCC=\frac{\left(TP\times TN\right)-\left(FP\times FN\right)}{\sqrt{\left(TP+FP\right)\left(TP+FN\right)\left(TN+FP\right)\left(TN+FN\right)}}}$

• False negative rate (FNR) and false positive rate (FPR): These rates are crucial for understanding the types of errors made by the model. FNR measures the rate at which positive cases are mistakenly classified as negative, while FPR measures the rate of negative cases incorrectly classified as positive. Their formulas are: (13)${\displaystyle FNR=\frac{FN}{TP+FN}}$ (14)${\displaystyle FPR=\frac{FP}{TN+FP}}$

• ROC-AUC score: The receiver operating characteristic (ROC) curve and the area under the curve (AUC) represent the model’s ability to distinguish between classes. A score close to 1 indicates perfect classification, while a score around 0.5 is no better than random guessing.

Hyper-parameter optimization

A crucial aspect of the experimental design involved meticulously optimizing hyperparameters for each classification method, including the CSA-DE-LR, CSA-LR, DE-LR, and other popular machine learning techniques such as decision tree (DT), linear discriminant analysis (LDA), Logistic Regression (LR), Multi-Layer Perceptron (MLP), Random Forest (RF), XGBoost, and Support Vector Machine (SVM). Table 1 details the hyperparameter ranges and the best values obtained for each classifier on the Statlog, Cleveland, WBCD, and WBCO datasets, achieved after 300 iterations of tuning using the Hyperopt (Bergstra, Yamins & Cox, 2013) method.

For the CSA-LR approach, the hyperparameters tuned included the lower and upper bounds (lb and ub), population size (P), number of clones (α), and receptor editing rate (B). The DE-LR method involved optimizing the lower and upper bounds (lb and ub), population size (P), scaling factor (sf), and crossover rate (cr). These adjustments ensured that both methods could operate efficiently within their designed optimization frameworks.

In the proposed CSA-DE-LR approach, a combination of hyperparameters was tuned, including the lower and upper bounds (lb and ub), population size (P), number of clones (α), receptor editing rate (B), scaling factor (sf), and crossover rate (cr). The optimization process balanced exploration and exploitation, finding the best parameter settings to maximize performance.

For other classifiers, the DT focused on ‘Min Samples Split’ and ‘Min Samples Leaf’ to control tree depth and prevent overfitting. LDA tuned the ‘Shrinkage’ parameter for improved generalization. LR optimized the ‘C’ parameter, which controls regularization strength.

In the MLP, hyperparameters like the learning rate, number of hidden units, batch size, and the number of epochs were tuned to discover the best settings. For RF, the number of trees in the forest was the focal point of optimization.

The SVM involved tuning the ‘C’ parameter, which defines the trade-off between smooth decision boundaries and classifying training points correctly. Finally, XGBoost focused on ‘Eta’ (learning rate) and ‘Depth’ (tree depth) to maximize predictive performance.

These comprehensive optimizations via Hyperopt were crucial in maximizing each classifier’s performance. They allowed an exhaustive search of the hyperparameter space to ensure optimal settings were used across the various datasets.

Ethical implications of deploying ML models in healthcare

Healthcare professionals may greatly benefit from ML models, which have the potential to improve patient outcomes and clinical decision-making. To guarantee the proper and ethical use of predictive technology, it is necessary to address important ethical concerns in conjunction with these breakthroughs (Chotrani, 2021). Securing patient privacy and confidentiality is one of the ethical issues with machine learning in healthcare (Manoharan et al., 2023). Data protection laws were followed, and steps were taken to anonymize patient information in order to reduce privacy concerns. In this work, a cardiovascular disease prediction model was trained using publicly accessible healthcare datasets, including Cleveland and Statlog. These datasets frequently have usage and sharing agreements that the researchers agreed to abide by. These policies may include limitations on redistribution, citation requirements, and ethical considerations.

The possibility of biases in the machine learning model either from the training data or the algorithms themselves is a further ethical concern. Biases can manifest in various forms, including demographic, socioeconomic, or access-related biases (Mehrabi et al., 2021). This work included rigorous data pretreatment methods to reduce biases. Addressing biases in the model aims to ensure fairness and equity in predictive healthcare analytics. In this work, we emphasize the significance of human supervision in addition to machine learning forecasts for cardiovascular disorders. Although our models provide insightful information, they should not be used as a substitute for physicians but rather as decision support tools (Bankins, 2021). In order to guarantee the proper implementation of models and patient-centered care, collaboration between data scientists, healthcare professionals, and ethicists is essential. Additionally, this work supports accountability and transparency, which are essential values in the moral application of machine learning algorithms (Hosai̇n et al., 2023). This involves transparent reporting of performance metrics, documentation of decisions, and addressing errors, fostering trust in the proposed predictive model for clinical practice.