A Decision Support System Based On IA

Expert Systems With Applications 242 (2024) 122746
Contents lists available at ScienceDirect
Expert Systems With Applications

journal homepage: www.elsevier.com/locate/eswa
A decision support system for supplier quality evaluation based on

MCDM-aggregation and machine learning
Qiuping Ma, Hongyan Li ∗
Department of Economics and Business Economics, BSS, Aarhus University, Aarhus, 8210, Denmark
ARTICLE INFO ABSTRACT
Keywords: Evaluating suppliers’ quality performance is one of critical tasks of quality management since it is directly
Supplier quality evaluation related to quality assurance, improvement and development, especially for manufacturing companies. However,
Decision support system the supplier quality evaluation remains challenging due to the involvement of a large amount of qualitative
Machine learning
and quantitative data, multi-dimensional attributes, as well as a large number of items and suppliers, etc. In
MCDM
addition, while various multiple criteria decision making methods (MCDMs) exist and can be used for supplier
Aggregation
quality evaluation, individual choice and application of the methods often result in inconsistent or conflicting
evaluation results. To overcome the challenges, in this study, we develop a decision support system for supplier
quality evaluation to deal with the practical complexity and to take advantage of the availability of the big
industrial data. The system is presented as a conceptual framework including three modules: the decision
matrix and criteria initialization module, the MCDM method selection and implementation module, and the
aggregation module. The sequential conduction of each module formulates and solves the multiple supplier and
item quality evaluation problem. The implementation of the system can be applied to dynamically monitor
and evaluate supplier and item quality performance. Moreover, as the core intelligent part of the decision
support system, in the aggregation module, we propose an aggregated MCDM model based on the machine
learning concepts. It aims at providing more reliable evaluations across multiple suppliers and items. The
aggregation model also contributes to the MCDM literature as an independent method which is not restricted
to the purpose of supplier quality evaluation. The methods shed lights on how to construct and solve MCDM
problems in the big-data era. Finally, the application of the decision support system is illustrated by a case
of a large automotive company. The case shows that our decision support system provide more robust and
reliable evaluation results than the traditional individual multiple criteria decision making method.
1. Introduction support system for supplier quality evaluation and develop a ML based
aggregation method to improve the evaluation performance.
With the rapid development of information technology, the great In modern manufacturing, almost all manufacturers are involved
amount of data is available in manufacturing and service industry. with different suppliers. Empirical studies show that suppliers exert sig-
Data-driven decision making methods are urgently required for many
nificant influence on the quality of final products (Agrawal et al., 2017;
organizations across almost all domains (Boehmke & Greenwell, 2019),
Dong et al., 2016). Moreover, it is common to trace product quality
especially in modern manufacturing, such as maintenance manage-
ment, quality management, production planning and control, and sup- issues back to supplier quality defects (Handley & Gray, 2013; Lee &
ply chain management (Bertolini et al., 2021; Dogan & Birant, 2021; Li, 2018; Rui & Lai, 2015). Therefore, continuously improving supplier
Nguyen et al., 2018). Machine learning (ML) has become prevalent quality has become an essential and strategic activity for manufacturers
to solve the big data-driven decision making problems. The applica- to maintain competitive advantages, especially for high-technology
tion of ML has shown positive effects not only on the supply chain manufacturers facing intense pressure from customers (Agrawal et al.,
performance (Kamble & Gunasekaran, 2020) but also on managerial 2017). However, evaluating supplier quality performance is still a
side (Dogan & Birant, 2021). However, how to take advantage of the
challenging task due to the lack of a unified evaluation framework and
essence of ML has not been fully explored in the context of supply chain
efficient evaluation methods.
management (Li & Liu, 2019). In this paper, we propose a decision
∗ Corresponding author.
E-mail addresses: [email protected] (Q. Ma), [email protected] (H. Li).
https://doi.org/10.1016/j.eswa.2023.122746
Received 12 July 2023; Received in revised form 22 October 2023; Accepted 25 November 2023
Available online 1 December 2023
0957-4174/© 2023 Elsevier Ltd. All rights reserved.
Q. Ma and H. Li Expert Systems With Applications 242 (2024) 122746
The research about the supplier quality evaluation is rather limited aggregated MCDM model based on the ML techniques can be applied
and most of the existing literature focus on supplier overall perfor- independently beyond the purpose of the supplier quality evaluation
mance in which quality is treated as an element (e.g. Restrepo & and thereby contributes to the literature on how to solve MCDM
Villegas, 2019; Rezaei & Fallah Lajimi, 2019 and Hamdi et al., 2018). problems in the era of big data. The DSS can evaluate and monitor
Moreover, many studies have a narrow focus on either items or suppli- supplier quality performance in real time and enable decision-makers
ers (e.g. Bai et al., 2017; Santos et al., 2017 and Rezaei et al., 2015). to respond promptly for further quality improvement.
They only take the external (critical) features of the market and the The rest of the paper is structured as follows. Section 2 provides an
internal (strategic) features of the manufacturer or the relationship with overview of the literature and clarifies the research gaps. In Section 3,
suppliers into consideration. In reality, the supplier evaluation problem we develop a DSS for the supplier quality evaluation process in which
involves with diverse data sources, multi-dimensional attributes, as three steps are included. In Section 4, we present methodologies used
well as a large number of items and suppliers involved. Therefore, it is in our proposed DSS. An application case is used to illustrate the
necessary to establish a generalized supplier quality evaluation system system and methodologies in Section 5. Finally, the paper ends with
to handle the large quantity of data and the high complexity of the a conclusion, research limitations, and future research directions in
evaluation. Section 6.
Within a supplier quality evaluation system, the supplier quality
2. Literature review
needs to be evaluated efficiently and regularly. MCDM methods are
often used to evaluate the performance of a single supplier or compo-
Our work mainly intersects with the following research areas: sup-
nent. In our study, we look into the evaluation of multiple suppliers and
plier evaluation and aggregation of MCDM methods. We hereby review
product items from different dimensions. Thus, it is difficult to find a
the most recent studies in the two streams.
single MCDM method which is robust for all suppliers and components
with multiple evaluation dimensions. Recently, several studies strive to 2.1. Supplier evaluation
aggregate the evaluations from different MCDM methods, for example,
based on subjective judgment (Jha et al., 2014), taking the aver- According to the phase of occurrence and purpose of the applica-
age (Wang et al., 2016) of multiple evaluations, or adding a singular tion, Galo et al. (2018) classify supplier evaluation studies into two
index (Petrović et al., 2018). However, given the characteristics of big types: the first type occurs in the supplier selection phase and aims
data and multiple dimensions in supplier quality evaluation problems, at selecting appropriate suppliers (Chai et al., 2013). This has been
the existing aggregation methods are not sufficiently efficient. Thus, studied intensively in literature. The second type is regularly applied to
innovative evaluation methods are needed for bid-data-driven supplier current suppliers to evaluate and improve supplier performance (Osiro
evaluation problems so that the consistent evaluation can be obtained. et al., 2014). In supply chains, manufacturers need to assess suppliers’
Therefore, in this paper, we focus on developing a unified supplier product quality so that the final product quality is maintained and
quality evaluation framework and data-driven MCDM methods to deal improved. Thus, supplier evaluation which tracks and measures current
with the practical and analytical complexity of the problems. The supplier performance based on multiple evaluation dimensions is also
following research questions will be addressed. important. Given the rapid development of data science and better data
availability in real businesses, it is possible to evaluate supplier quality
• How to evaluate supplier/item quality systematically based on the
more effectively and efficiently. Therefore, in this study, we focus on
diverse data sources? the second type of supplier evaluation and name it as supplier quality
• What evaluation dimensions and criteria should be considered for evaluation.
supplier quality evaluation? Furthermore, three streams of studies are closely related to the
• How to characterize the big-data driven supplier quality evalua- supplier evaluation: scope, dimensions, and techniques. More specifi-
tion problem? cally, scope defines where and why supplier evaluations are applied;
• How to aggregate the multiple MCDM methods for robust and dimensions define major aspects with specific criteria of which are
consistent evaluation results by employing ML techniques? considered to evaluate supplier quality performance. Techniques define
mathematical methods which are prevalent to measure supplier perfor-
To answer the research questions, we propose a DSS for the sup-
mance. We review representative and closely related studies from the
plier quality evaluation. In the system, supplier quality evaluation is
three perspectives in the following subsections.
conducted under a Purchasing Portfolio Model-Supplier Portfolio Model
(PPM-SPM) framework, in which both multiple component items and
2.1.1. Scope of supplier evaluation
multiple suppliers are taken into account. In the PPM-SPM framework,
A large number of studies work on supplier evaluation, however,
four dimensions of criteria are considered, including Supply Risk (SR),
the majority of current literature mainly focuses on evaluating sup-
Profit Impact (PI), Supplier Willingness (SW), and Supplier Quality plier overall performance, such as Ahmadi and Golbabaie (2015),
Capability (SQC). The DSS consists of three modules (or steps): (i) Lima-Junior and Carpinetti (2016), Rezaei et al. (2015) and Rezaei
decision matrix and criteria weights initialization, (ii) MCDM methods and Fallah Lajimi (2019). Recently, researchers pay more attention
selection and implementation, and (iii) aggregation. In consideration to supplier environmental (green or sustainable) performance (Akman,
of the fast learning ability and computing power of ML, we integrate 2015; Bai et al., 2017; Demir et al., 2018). Even though many studies
ML, MCDM methods, and the concept of aggregation to obtain the already found that quality is one of the most critical criteria in sup-
consistent evaluation results. At last, through an application case of plier evaluation (Rezaei & Fallah Lajimi, 2019; Rezaei & Ortt, 2013a;
a large automotive company, the application of DSS is illustrated and Rezaei et al., 2015), most extant literature only views supplier quality
the evaluation results shows that the ML-based aggregation method is performance by a rejection rate (or defect rate) (Hamdi et al., 2018;
robust and reliable. Krishnadevarajan et al., 2015; Restrepo & Villegas, 2019) or a single
This study makes original contributions on two aspects: the de- quality score (Ahmadi & Golbabaie, 2015; Al Salem et al., 2016; Galo
velopment of a DSS for supplier quality evaluation and the ML-based et al., 2018; Rezaei & Fallah Lajimi, 2019).
MCDM aggregation method. In the DSS, a conceptual framework for the As mentioned earlier, unlike the supplier overall performance eval-
supplier quality evaluation is proposed. In the framework, constructing uation, we focus on supplier evaluation for the purpose of quality
the dimension of SQC with various types of data will contribute to a improvement or development. The supplier quality evaluation on daily
deeper understanding of a data-driven and systematic supplier quality operations needs to be performed throughout the product life cycle.
evaluation in the manufacturing industry and make it possible to link It has not been studied profoundly because it often requires a large
evaluation outcomes to supplier quality development. The innovative amount of data and has an inherent dynamic feature.
2
Table 1
Summary of techniques for supplier evaluation.
Literature Techniques for supplier evaluation Category
Rezaei and Fallah Lajimi (2019) BWM Single approach
Galo et al. (2018) ELECTRE TRI & hesitant fuzzy & group decision Hybrid approach
Demir et al. (2018) VIKORSORT Single approach
Santos et al. (2017) AHP & 2-Tuple representation Hybrid approach
Bai et al. (2017) Rough Set Theory & VIKOR Hybrid approach
Lima-Junior and Carpinetti (2016) Fuzzy TOPSIS Hybrid approach
Rezaei et al. (2015) BWM Single approach
Akman (2015) VIKOR Single approach
Osiro et al. (2014) Fuzzy inference Single approach
Rezaei and Ortt (2013b) A fuzzy rule-based method Single approach
Rezaei and Ortt (2013a) Fuzzy AHP Hybrid approach
Rezaei and Ortt (2012) Scores are equally rated and averaged Single approach
2.1.2. Dimensions for supplier evaluation In order to compensate for the drawbacks of the classic MCDM
In general, suppliers are often evaluated by multiple dimensions methods, some hybrid approaches have been developed. Santos et al.
including quality management, problem resolution, and delivery, cost, (2017) use 2-tuple fuzzy linguistic representation and the AHP (An-
and short-term delivery performance, long-term potential for partner- alytic Hierarchy Process) method. Lima-Junior and Carpinetti (2016)
ship in literature (Lima-Junior & Carpinetti, 2016 and Osiro et al., use fuzzy TOPSIS model, which poses no limit on the number of
2014). Among all these dimensions, four most well-known dimensions suppliers evaluated and avoids the ranking reversal problem. Akman
are SR, PI, SW, and Supplier Capabilities (SC). Kraljic (1983) initially (2015) utilize VIKOR to evaluate supplier environmental performance
introduce SR and PI dimensions under the PPM model from the stand- for green supplier development. Osiro et al. (2014) propose a fuzzy
point of supplied items. Each dimension is divided into two levels: low logic approach based on fuzzy inference and combined with a simple
and high. Then a 2 × 2 matrix is generated to classify items into four fuzzy grid method. The approach sets no limits on the number of suppli-
categories: strategic, leverage, bottleneck, and non-critical. However, ers for evaluation and development. More importantly, the knowledge
despite being an easy to use tool, these two dimensions only focus on of the decision-makers is integrated and stored in the decision rules
supply items and neglect features of suppliers. created by the fuzzy logic approach. citerezaei2013suppliera propose a
To integrate the features of suppliers, Rezaei and Ortt (2012) pro- fuzzy rule-based method by interviewing knowledgeable managers or
pose the SPM model to include SW and SC dimensions. These two experts. Rezaei and Ortt (2013a) suggest using Fuzzy AHP to incor-
dimensions have been widely considered in recent literature (Rahayu porate the ambiguities and uncertainties that usually exist in expert
& Ardi, 2020, Rezaei & Fallah Lajimi, 2019, and Bai et al., 2017). experience. However, these methods largely depend on the decision
However, although they provide an overall evaluation from suppli- makers’ comparative and subjective judgments, which may result in
ers’ perspectives, valuable information based on the individual supply biased or inconsistent evaluation results.
items is missing. Osiro et al. (2014) show that the supplier quality
development actions depend on both items’ and suppliers’ performance. 2.2. Aggregation of MCDM methods
Therefore, in order to achieve a comprehensive supplier quality eval-
uation, it is necessary to consider the four dimensions simultaneously. MCDM applies unique mechanisms including normalization proce-
However, most extant research considers them separately except for a dures, parameter introduction and setting, and ideal solution identi-
few recent studies, such as Ahmadi and Golbabaie (2015), Osiro et al. fication and treatment, etc. Jahan et al. (2011), Lo et al. (2020) and
(2014), Rezaei and Fallah Lajimi (2019). Petrović et al. (2018). Hence, even though the same set of evaluation
criteria, the same criteria weight are given to the same problem, the
2.1.3. Techniques for supplier evaluation application of different MCDM methods may lead to different or even
Given different types of data, different types of techniques may conflicting results. This makes it difficult to trust and apply the results
be required and applied for the supplier evaluation. Historical supply of a single MCDM method (Jahan et al., 2011; Mohammadi & Rezaei,
data and decision-makers’ judgment may be used based on a set of 2020).
evaluation criteria. Various MCDM methods have been applied for To compensate for shortcomings of a single MCDM method and to
supplier evaluation in literature. We summarize the techniques for make use of the knowledge obtained by various MCDM methods, Petro-
supplier evaluation and relevant literature in Table 1. vić et al. (2018), Velasquez and Hester (2013) and Zavadskas et al.
As shown in Table 1, MCDM is the main technique for supplier (2016) advocate that a combination of two or more MCDM methods
evaluation. Rezaei and Fallah Lajimi (2019), Rezaei et al. (2015) apply may provide more robust and consistent evaluation results. Zavadskas
‘‘Best Worst Method’’ in which needs fewer data and yields robust et al. (2016) classify the combination of MCDM methods into following
results. However, the Best Worst Method is mainly dependent on pair- four types:
wise comparisons. Its main drawback is that the vagueness associated
with comparative judgments by different decision-makers may lead • Type 1: MCDM method + MCDM method(s)
to inconsistent results (Osiro et al., 2014). Rezaei and Ortt (2012) • Type 2: MCDM method + method for identifying importance of
use equally rated and averaged interview score sheets to evaluate criteria
supplier performance. Galo et al. (2018) propose a hybrid approach • Type 3: MCDM method + fuzzy and/or gray sets
that combines ELECTRE (Elimination and Choice Expressing the Re- • Type 4: MCDM method + other methods
ality) TRI and hesitant fuzzy. The approach simultaneously considers
the uncertainty of judgment, non-compensation between criteria, and In this study, we are mainly interested in the Type 1 methods in
group decisions. The ordered categories for each criterion need to which two or more MCDM methods are aggregated. Thus, they are
be predefined. Bai et al. (2017) and Akman (2015) leverage VIKOR named as the aggregated MCDM methods for short in the rest of the
(VlseKriterijumska Optimizacija I Kompromisno Resenje) method to paper. The underlying logic of the aggregated MCDM method is to
determine supplier ranks. Recently, Demir et al. (2018) present a aggregate the individual output of various MCDM methods and obtain
novel VIKORSORT (VIKOR-based multi-criteria sorting methodology) an aggregated result that is more consistent and reliable. Aggregated
to evaluate supplier environmental performance. MCDM methods have been applied for alternative selection problems
3
Table 2
Review of aggregated MCDM methods in literature.
Reference Problem solved MCDM method used Aggregation steps Pros and cons
Jahan et al. (2011) Material selection for Ordinary-TOPSIS Step 1: form a decision matrix based Pros: consider the importance of high
a statically loaded Block-TOPSIS on decision-makers’ preferences rankings.
thermal conductor VIKOR and\or empirical data. Cons: The formulation of the linear
ELECTRE Step 2: produce preliminary programming model is not
Comprehensive VIKOR evaluations with different MCDM well-founded.
AHP-Comprehensive methods.
VIKOR Step 3.1: calculate the number of
AHP-TOPSIS times each alternative is assigned to
the given result.
Step 3.2: formulate a linear
programming model in which the
objective function is a weighted
product ranking and a permutation
square matrix.
Step 3.3: solve the linear model and
obtain the final ranking.
Jha et al. (2014) Material selection for AHP Step 1 & 2: same as before. Pros: provide different weights for
knee implantations TOPSIS Step 3.1: compute Spearman’s rank different MCDM methods.
VIKOR correlation coefficient to select Cons: (1) Eliminate MCDM methods
SAW qualified MCDM methods with high which have low rank correlations.
MOORA rank correlations. (2) weights for MCDM methods are
ELECTRE I Step 3.2: decision-makers provide assigned based on experts’ knowledge
EXPROM II subjective weight for each MCDM and experiences.
PSI method and obtain the weighted sum
COPRAS final ranking.
EVAMIX
Peng (2015) Earthquake TOPSIS Step 1 & 2: same as before. Pros: the most trustworthy MCDM
vulnerability of 31 VIKOR Step 3.1: compute Spearman’s rank method is selected.
Chinese regions GRA correlation coefficient to determine Cons: Aggregation technique in step
PROMETHEE II the most trustworthy method. 3 is not clear and final ranking result
ELECTRE III Step 3.2: use the most trustworthy cannot be reproduced.
SWM method to combine the rankings of
the other considered MCDM methods
using the normalized weights of each
MCDM method and standardized
ranking scores to obtain the final
ranking.
Wang et al. (2016) Facility layout TOPSIS Step 1 & 2: same as before. Pros: aggregation of different MCDM
selection, equipment GRA Step 3: take the average of initial methods.
selection, etc. SAW rankings to obtain the final ranking. Cons: no guarantee of obtaining
optimum results when the rankings
of alternatives have large differences.
Petrović et al. (2018) Transport and TOPSIS Step 1 & 2: same as before. Pros: combine the information about
logistics policy VIKOR Step 3.1: compute Taguchi’s the mean and the variance of
selection ARAS signal-to-noise ratio for each MCDM different MCDM rankings.
COPRAS method. Cons: aggregation performance is not
MOORA Step 3.2: use the value of Taguchi’s outstanding and not stable.
WASPAS signal-to-noise ratio to obtain the
final ranking.
Mohammadi and Rezaei (2020) Ontology alignment TOPSIS Step 1 & 2: same as before. Pros: (1) provide different weights
evaluation initiative VIKOR Step 3.1: utilize the half-quadratic for different MCDM methods.
(OAEI) 2018 PROMETHEE (HQ) theory to determine a set of (2) assess the performance of the
weights for MCDM methods. aggregation ranking.
Step 3.2: use the weights obtained Cons: the approach and assessment
from the second step to get weighted index are limited to Welsch loss
sum final ranking. functions.
Lo et al. (2020) ranking of failure SAW Step 1 & 2: same as before. Pros: aggregation of different MCDM
modes in a machine VIKOR Step 3: integrate various MCDM methods.
tool manufacturing GRA methods based on TOPSIS concept to Cons: no guarantee of obtaining
company COPRAS obtain the final ranking. optimum results solely based on
TOPSIS concept.
in literature. We summarize the aggregation steps, pros, and cons of 2.3. Research gaps and our study
relevant studies in Table 2.
Table 2 shows that the existing aggregation methods are mainly We summarize the main characteristics of the supplier evaluation
used to handle the alternative selection problems. The pros and cons literature in Table 3 from three perspectives: scope, dimensions, and
indicate that they may become inefficient or ineffective for the supplier techniques.
evaluation problem with high dimensions and big data. The concept Table 3 shows, despite the extensive research on supplier eval-
and techniques of ML may be applied to overcome the drawbacks of uation, that research gaps and opportunities can still be detected.
the existing MCDM methods, especially when the large quantity of real In terms of the scope, recent research mainly focuses on evaluating
business data are available. supplier overall performance and supplier environmental performance.
4
Table 3
The main characteristics of the supplier evaluation literature and our study.
Literature Scope Dimensions Techniques
Supplier Supplier en- Supplier Supply risk Supplier Others Single Hybrid Aggregated
overall vironmental quality & profit willingness approach approach approach
performance performance performance impact & supplier
(components) capabilities
(suppliers)
Rezaei and Fallah Lajimi (2019) X X X X
Galo et al. (2018) X X X
Demir et al. (2018) X X X
Santos et al. (2017) X X X
Bai et al. (2017) X X X
Lima-Junior and Carpinetti (2016) X X X
Rezaei et al. (2015) X X X
Akman (2015) X X X
Osiro et al. (2014) X X X X
Rezaei and Ortt (2013b) X X X
Rezaei and Ortt (2013a) X X X
Rezaei and Ortt (2012) X X X
Our study X X X X
Little research offers insights into supplier quality evaluation in a big been widely employed in previous studies, we construct a new di-
data era. Concerning the dimensions, most of the previous studies mension named Supplier Quality Capability (SQC), as presented in
concentrate either on individual items or suppliers instead of both Fig. 1. The SQC dimension is an element of SC but only concentrates
items and suppliers. From the technical point of view, the limitation on the quality aspect. The SQC dimension is important as it can be
of extant studies on supplier evaluation lies in the narrow focus on a used to identify suppliers with poor quality performance. All the four
single MCDM method or a hybrid method in which MCDM methods are dimensions are benefit dimensions, that is, the higher the values are,
combined with the fuzzy set theory or the rough set theory. Moreover, the better they are. For example, a high value of the SR dimension
most methods are still based on subjective judgment or linguistic indicates that a number of items are available in the market and
information. How to aggregate the evaluation from different MCDM stockout risk is low. Three steps are involved in the system and remain
methods remains a challenging task. the same for each dimension. Details of each step are described below.
To fill the research gaps mentioned above, we focus on evaluating
supplier quality performance and aim to provide insights to decision- Step 1: Decision matrix and criteria weights initialization. We first
makers to improve or develop the quality of the suppliers’ products. construct a criteria library in which criteria related to items and suppli-
However, establishing strategies for supplier quality development is not ers are considered. Decision-makers then select critical criteria based on
only dependent on supplier quality performance but also relies on the their knowledge and experience. Furthermore, the data type, objective,
risk of supply, the effect on profit, and whether suppliers are willing to scope, and sources for each criterion are determined by the expert
cooperate. Therefore, four evaluation dimensions are considered in our team. Associated with a supplier list provided by the decision-makers,
study, including SR, PI, SW, and SC. Under each dimension, qualitative a decision matrix will be generated by evaluating all suppliers against
and quantitative data, especially historical production and supply chain the criteria. Meanwhile, expert judgments from various departments
data are considered. In addition, we aim at providing evaluation on are required to elicit criteria weights. The combination of the decision
multiple supplier–item pairs. matrix and criteria weight is used as inputs for the next step.
Step 2: MCDM methods selection and implementation. Multiple
Due to the consideration of multiple suppliers and items as well as
objective and well-known MCDM methods are collected to construct
multiple evaluation dimensions, multiple MCDM methods are needed
a pool where decision-makers can choose appropriate MCDM methods
to provide more completed view of the supplier’s quality performances.
according to their preferences. For illustration purposes, we select six
Due to the large amount of data involved, the concept of ML becomes
MCDM methods, including TOPSIS, VIKOR, PROMETHEE with Usual
an appropriate choice to aggregate the results of multiple MCDM meth-
type preference function, PROMETHEE with Gaussian type preference
ods. Therefore, we propose a new aggregation method in which utilizes
function, MOORA with the ratio system, and MOORA with the refer-
multiple loss functions to determine the optimal weights for different
ence approach. A preliminary evaluation matrix is obtained by imple-
MCDM methods, and then use the weights to generate aggregated
menting the six MCDM methods on the decision matrix and criteria
evaluation. The main characteristics of our study are shown in the last
weights.
row in Table 3
Step 3: Aggregation. We first construct an aggregation model in which
disagreements between an aggregated evaluation and the preliminary
3. A DSS for supplier quality evaluation process evaluation outcome tend to be minimized. To compute the disagree-
ments, we integrate a wide range of loss functions from ML to the
We consider a manufacturer who procure items from multiple sup- aggregation model. This study utilizes four well-known loss functions,
pliers and sells final products to customers. The manufacturer is inter- including L2 loss, L1 loss, Log-cosh (LC) loss, and M-estimator (ME)
ested in evaluating, managing, and developing the suppliers’ product loss. We then solve the model to derive aggregated evaluations covering
quality and aims to continuously improve the final product quality four loss functions. Finally, a comparison analysis is conducted to
during the product’s entire life cycle. The manufacturers commonly find the appropriate loss function and the final aggregated evaluation
take the lead in the quality-related initiatives in manufacturing fields, accordingly.
such as aerospace, automotive, and electronics (Lee & Li, 2018). In
daily business, one supplier may provide multiple items, and different 4. Methodology
suppliers may supply the same item. Thus, without loss of generality,
we consider unique item–supplier pairs as the research objects. This section presents methodologies used in each step of the pro-
As mentioned in Section 2.1, supplier quality performance is not posed DSS. To facilitate the description, notations in Table 4 are
fully addressed. Apart from the dimensions of SR, PI, SW which have used.
5
Fig. 1. Figure 1-A framework of the DSS for supplier quality evaluation.
4.1. Decision matrix and criteria weights initialization of quality, and fluctuation of quality. A flowchart of extracting SQC
criteria is displayed in Fig. 2 and a detailed explanation is provided
The decision matrix and criteria weights are two cornerstones of below.
supplier quality evaluation. To initialize the decision matrix, evaluation
• Defect Per 100 Units (𝐷𝑃 ̂ 𝑈)
dimensions and criteria should be determined. Thus, we first construct
It is one of the most prevailing quality criteria used in the
a criteria library in which four evaluation dimensions and related
manufacturing industry, which represents the number of quality
criteria are identified. To be more specific, criteria that measure the
defects for every 100 final products, as in Eq. (1). The term
SR and PI dimensions from the perspective of items are collected
‘‘defect’’ refers to the items that failed to meet specifications.
from Rezaei and Fallah Lajimi (2019). The criteria that measure the
Especially in the automotive industry, manufacturers daily record
SW dimension regarding suppliers are gathered from Rezaei and Ortt ̂ ̂
and monitor 𝐷𝑃 𝑈 performance. A high 𝐷𝑃 𝑈 value indicates
(2012).
a low-quality level and requires special attention. The concept
With respect to the SQC dimension, its related criteria are rarely ̂
of 𝐷𝑃 𝑈 is straightforward and thus used in this paper. Similar
mentioned in previous literature. However, it is common for manu- criterion may include defect rate or part per million equivalents
facturers to record and store outcome-based historical quality data in (PPMeq) (Teli et al., 2013).
their internal information systems or databases, such as ERP systems
̂ 𝐷𝑒𝑓 𝑒𝑐𝑡 𝑄𝑢𝑎𝑛𝑡𝑖𝑡𝑦
and quality audit databases (supplier databases for short in this paper). 𝐷𝑃 𝑈= ∗ 100 (1)
𝐹 𝑖𝑛𝑎𝑙 𝑝𝑟𝑜𝑑𝑢𝑐𝑡 𝑞𝑢𝑎𝑛𝑡𝑖𝑡𝑦
We thus are able to extract the SQC criteria from the existing supplier
databases. Additionally, based on the large volume and wide range of • Quality Cost Performance Index (𝑄𝐶𝑃 ̂𝐼)
historical quality data, we are capable of exploring SQC criteria from It is proposed by Teli et al. (2013) and is computed by Eq. (2).
various perspectives, i.e., quality, the economic impact of quality, risk This concept connects the influences of supplier quality problems
6
Table 4
Notations.
Indices
𝑖 the index of item–supplier pairs, 𝑖 = 1, … , 𝑚
𝑗 the index of criteria, 𝑗 = 1, … , 𝑛
𝑘 the index of MCDM methods, 𝑘 = 1, … , 𝐾
Parameters
𝐴𝑖 the 𝑖th item–supplier pair
𝐶𝑗 the 𝑗th criteria
𝑥𝑖𝑗 the value of the 𝑖th item–supplier pair on 𝑗th criterion
𝐱𝑖 the value vector of the 𝑖th item–supplier pair over 𝑛 criteria, 𝐱𝑖 = {𝑥𝑖1 , … , 𝑥𝑖𝑛 }
𝐗 the decision matrix of 𝑚 item–supplier pairs over 𝑛 criteria, 𝐗 = {𝐱1 , … , 𝐱𝑚 }𝑇
∑𝑛
𝝎 the weight vector of criteria, 𝝎 = {𝜔1 , 𝜔2 , … , 𝜔𝑛 }, where 𝜔𝑗 > 0 and 𝑗=1 𝜔𝑗 = 1
𝑦(𝑖)
𝑘
the preliminary evaluation score of the 𝑖th item–supplier pair by using the 𝑘th MCDM method
𝐲𝑘 the preliminary evaluation vector by using the 𝑘th MCDM method, 𝐲𝑘 = {𝑦(1) 𝑘
, … , 𝑦(𝑚)
𝑘
}𝑇
𝐘 the preliminary evaluation matrix by using all 𝐾 MCDM methods, 𝐘 = {𝐲1 , … , 𝐲𝐾 }
∑𝐾
𝑤𝑘 the weight of 𝐲𝑘 , where 0 ≤ 𝑤𝑘 ≤ 1 and 𝑘=1 𝑤𝑘 = 1
𝒘 the weight vector of all MCDM methods, 𝒘 = {𝑤1 , 𝑤2 , … , 𝑤𝐾 }𝑇
𝒘∗ the optimal weight vector of all MCDM methods
𝑧(𝑖) the aggregated evaluation score for the 𝑖th item–supplier pair
𝐳 the aggregated evaluation vector for all item–supplier pairs, 𝒛 = {𝑧(1) , 𝑧(2) , … , 𝑧(𝑚) }𝑇
𝒛∗ the optimal aggregated evaluation for all item–supplier pairs
𝐙∗ the optimal aggregated evaluation matrix by using all loss functions
𝒛̇ the final aggregated evaluation for all item–supplier pairs
𝑑𝑘(𝑖) the difference between 𝐲𝑘 and 𝒛 for the 𝑖th item–supplier pair
𝐿(⋅, ⋅) the loss function
𝐽 (⋅, ⋅) the interim loss function
𝐶(⋅, ⋅) the cost function
𝑓 (⋅) the potential form of loss function
𝑄(⋅, ⋅) the quadratic term
𝜓(⋅) the convex conjugate function of 𝑓 (⋅)
𝛿(⋅) the minimizer function of 𝑓 (⋅)
𝑅(⋅, ⋅) the rank function
𝑔𝑘 the gradient for the 𝑘th weight 𝑤𝑘
𝛼𝑘 the half-quadratic auxiliary variable of 𝑑𝑘(𝑖)
𝑡 the iteration counter
𝑙𝑟 the learning rate
𝜎2 hyper-parameter in the Welsch loss function
𝑟(𝑖)
𝑘
the difference between the ranks of 𝐲𝑘 and 𝒛∗ for the 𝑖th item–supplier pair
𝜌𝑘 the Spearman’s rank correlation coefficient of 𝐲𝑘 and 𝒛∗
𝐂 the consensus index of an aggregation model by using a specific loss function
𝐓 the trust level of an aggregation model by using a specific loss function
𝐅 the final assessment of an aggregation model by using a specific loss function
Fig. 2. Measurement flowchart of SQC dimension.
7
on the manufacturer’s financial performance and the strategic directly, while criteria with qualitative data should be converted into
practices to improve supplier quality performance. Supplier qual- quantitative criteria. The objectives are to maximize benefit criteria
ity cost includes any costs associated with nonconforming items, and to minimize cost criteria. The scale sets a specific range of criteria
and the value can be significant (Monczka et al., 2009). Given the values and assists in transforming qualitative criteria with linguistic
research content of our study and traceability of data, we only terms into numerical values. Two primary data sources include supplier
focus on the supplier quality cost with the following features: databases and the expert team with experts from different departments.
(i) the cost is caused by supplier quality problems, (ii) the cost Finally, decision-makers evaluate supplier quality regarding the
emerges after the manufacturer receive the items, and (iii) the criteria either by extracting data from supplier databases or based on
cost is initially borne by the manufacturer. In this condition, their knowledge and experience. As a result, decision-makers generate a
the quality costs caused by other parties or unknown to the decision matrix 𝐗 = [𝑥𝑖𝑗 ]𝑚×𝑛 whose rows correspond to 𝑚 item–supplier
manufacturer will not be considered. pairs and columns to 𝑛 criteria. In more details, item–supplier pairs are
̂𝐼 = 𝑆𝑢𝑝𝑝𝑙𝑖𝑒𝑟 𝑄𝑢𝑎𝑙𝑖𝑡𝑦 𝐶𝑜𝑠𝑡 + 𝑃 𝑢𝑟𝑐ℎ𝑎𝑠𝑒𝑑 𝐶𝑜𝑠𝑡
𝑄𝐶𝑃 (2)
referred to as 𝐴𝑖 , 𝑖 = 1, … , 𝑚, while various criteria are denoted by 𝐶𝑗
𝑃 𝑢𝑟𝑐ℎ𝑎𝑠𝑒𝑑 𝐶𝑜𝑠𝑡 where 𝑗 = 1, … , 𝑛. We use 𝐱𝑖 and 𝑥𝑖𝑗 to represent the 𝑖th item–supplier
• Risk Priority Number (𝑅𝑃 ̂ 𝑁) pair and the value of the 𝑖th item–supplier pair on the 𝑗th criterion,
respectively. Meanwhile, expert team provides criteria weights which
Practitioners highly suggest risk-based analysis for supplier qual- ∑
ity, yet few studies integrate risk perspective (Noshad & Awasthi, is denoted by 𝝎 = (𝜔1 , 𝜔2 , … , 𝜔𝑛 ), where 𝜔𝑗 > 0 and 𝑛𝑗=1 𝜔𝑗 = 1.
2015). Failure Mode Effect Analysis (FMEA) is a structured The decision matrix and criteria weights can then be used as inputs
method to identify different failure modes and understand their for implementing different MCDM methods.
effects on occurrences. The risk associated with each failure
mode is calculated as 𝑅𝑃̂ 𝑁 concerning Severity (𝑆),̂ Occurrence 4.2. MCDM methods selection and implementation
̂ and Detection (𝐷)
(𝑂), ̂ factors. A 10-point ordinal scale with a
linear increasing tendency from one to scale ten is commonly As previously mentioned in Section 2.2, there is no unique and
adopted. However, in practice, severity rating may have non- well-defined MCDM method that fits all problems. Since a variety of
linear characteristics since the highest severity rating usually MCDM methods can be deployed in the proposed DSS, a pool can be
carries greater influences. Thus, we use the transformed Severity constructed so that decision-makers can select appropriate methods
𝑆̂ = exp(𝛼 𝑆)
̂ proposed by Kim et al. (2013) and the 𝑅𝑃 ̂ 𝑁 value according to their preferences and business needs. We follow two rules
can be given by Eq. (4). For more information about the ranking to establish the pool. First, MCDM methods that can be implemented
system for the Severity, Occurrence, and Detection of failure without the participation of decision-makers are preferred. One main
modes, see Department of the Army, TM5-689-4 (2006). reason is that we want to minimize interference in the evaluation
̂ process and construct an automated DSS. Another reason is that it can
𝑅𝑃 𝑁 = 𝑆̂ ∗ 𝑂̂ ∗ 𝐷̂ (3)
avoid extra time and effort caused by the involvement of decision-
makers. However, it is feasible to choose other MCDM methods that
̂
𝑅𝑃 ̂ ∗ 𝑂̂ ∗ 𝐷̂
𝑁 = exp(𝛼 𝑆) (4) require the involvement of decision-makers, as the selection of MCDM
• Quality Fluctuation (𝑄𝐹 ), Cost Fluctuation (𝐶𝐹 ), and Risk methods has no impact on the essence of aggregation. The second rule
Fluctuation (𝑅𝐹 ) is that these methods have been proven to be efficient and widely
Even though quality problems occur every day, violent fluctuation applied in research and practices.
cannot be avoided. In other words, given the same item provided According to the two rules mentioned above, many MCDM meth-
by the same supplier, the three criteria mentioned above may ods are available, including TOPSIS, VIKOR, PROMETHEE, MOORA,
present significant inconsistency in a short period of time. The ELECTRE II, SAW, COPRAS, ARAS, and WASPAS, etc. For more details
fluctuation is a unique feature compared with criteria from other of different MCDM methods, see Alinezhad and Khalili (2019), Greco
dimensions. And this instability brings new challenges to the man- et al. (2016), and Velasquez and Hester (2013), etc. After implementing
ufacturers. On the one hand, it is difficult to predict the tendency. the selected MCDM methods, a preliminary evaluation matrix 𝐘 with
On the other hand, timely responses from the manufacturer are 𝑚 rows for item–supplier pairs and 𝐾 columns for MCDM methods
highly required. Hence, we use variance across each evaluation is obtained. We select six well-known MCDM methods for illustration
period to measure the value of 𝑄𝐹 , 𝐶𝐹 , and 𝑅𝐹 , as represented purposes. These MCDM methods have been widely adopted by re-
̂
by Eq. (5). Note that the 𝐷𝑃 𝑈 used in Eq. (1) is different from searchers (see Section 2.2). Moreover, these methods can also be easily
the 𝐷𝑃 𝑈 used in Eq. (5). The 𝐷𝑃 ̂ 𝑈 used in Eq. (1) is a single implemented in R by invoking specific packages, such as the MCDM
deterministic value for a long period (e.g. one quarter or one package and RMCriteria package.
year). The 𝐷𝑃 𝑈 used in Eq. (5) represents a set of numbers based
on short frequencies (e.g. daily or weekly). Other two criteria 𝐶𝐹 • TOPSIS is one of the most prominent MCDM methods which have
and 𝑅𝐹 have the same situation. been widely applied in different fields. Its simplicity and the abil-
ity to maintain the same steps regardless of the problem size make
𝑄𝐹 = 𝑉 𝑎𝑟(𝐷𝑃 𝑈 )
it reported as the second most popular approach after AHP (Zyoud
& Fuchs-Hanusch, 2017). The underlying idea of TOPSIS is to
𝐶𝐹 = 𝑉 𝑎𝑟(𝑄𝐶𝑃 𝐼) (5)
identify the positive-ideal and negative-ideal solutions at first
and then calculate the distances from evaluation objects to the
𝑅𝐹 = 𝑉 𝑎𝑟(𝑅𝑃 𝑁)
positive-ideal and negative-ideal solutions, respectively. Finally,
After constructing the criteria library, all criteria should be reviewed relative closeness coefficients are calculated, and a preference
and validated by experts from participating departments, e.g. pur- order is ranked based on that.
chasing and procurement department, supply chain management de- • VIKOR determines a compromise solution that is closest to the
partment, and quality management department, etc. (Aksoy & Öztürk, ideal solution based on the adopted distance measures. It has
2011). Furthermore, criteria attributes, including data type, objective, become one of the most frequently used methods (Zavadskas
scale, and data source, should be predefined by the expert team. et al., 2016). VIKOR method provides a maximum ‘‘group utility’’
These attributes have significant impacts on the following processing and is particularly helpful when decision-makers are uncertain
procedures. For example, criteria with quantitative data can be used about their preferences (Opricovic & Tzeng, 2004).
8
• PROMETHEE (Preference Ranking Organization Method for En-

richment Evaluations) is another classical method based on the
pairwise comparison. It was developed at the beginning of the
1980s and is still widely applied in the decision support do-
main (Mehmood & Khodabandeh, 2017). PROMETHEE method
has distinct advantages in terms of user-friendliness, simplicity,
and interpretability, stability. Its basic concept lies in a preference
function which is formulated on the deviation between two item–
supplier pairs. Six types of preference function are proposed:
(i) Usual criterion, (ii) U-shape criterion, (iii) V-shape criterion,
(iv) level criterion, (v) V-shape with indifference criterion, and
(vi) Gaussian criterion. We use Usual and Gaussian preference
functions so that no extra hyper-parameters are needed.
• MOORA (Multi-Objective Optimization Ratio Analysis) is intro-
duced by Brauers (2003). As the name indicates, MOORA is a
multi-objective optimization technique capable of simultaneously
optimizing two or more conflicting criteria. Even though it is
a new method, it has been successfully applied to solve vari-
ous types of complex decision-making problems (Chakraborty, Fig. 3. Illustration of the aggregation model.
2011). The advantages of the MOORA method include the in-
volvement of very few mathematical equations and fast compu-
tation. MOORA consists of three forms: (i) the ratio system, (ii) The idea of assigning a set of weights to different MCDM methods is
the reference approach, and (iii) the full multiplicative form. The not new. As mentioned in Table 2, researchers determine the weights
difference lies in how to calculate the score. The first two are in different ways, e.g. collecting subjective weights from decision-
leveraged in our study.
makers (Jha et al., 2014), taking the average (Wang et al., 2016), and
The multiple MCDM methods will be applied to evaluate the sup- combining M-estimator and HQ theory (Mohammadi & Rezaei, 2020).
plier quality individually first. If the evaluations from the selected In our study, we define the MCDM-aggregation problem as an optimiza-
methods are consistent, then the evaluation process terminated. How- tion problem in which the overall disagreements (a.k.a. residuals or
ever, it is quite unlikely the different MCDM methods will lead to the errors) between the aggregated evaluation and the actual evaluations
same evaluations results due to the different evaluation mechanisms are minimized. In this context, we aim to find an optimal weights 𝐰∗ ,
and the large number of item–supplier pairs and multiple evaluation which yields the minimum value of the overall disagreements.
dimensions. Therefore, the aggregation of multiple MCDM results is The aggregation problem has the similar structure with ML meth-
often necessary but challenge. In next section, we propose a ML-based ods: to train an optimization model using input data and obtain the
MCDM aggregation method. weights so that the loss function is minimized. Moreover, ML methods
can make a system automatically analyze and learn from the given
4.3. Aggregation data and the learning process can be improved by a measure of feed-
back (Ghatak, 2019). Taking these into consideration, we employ the
After the two steps presented above, we are ready to aggregate the concept of ML to construct the aggregation model. In more details,
evaluation results from the multiple selected MCDM methods. In this the overall disagreements are treated as the feedback and motivate the
section, we propose an aggregation model to based on the concept minimizing (learning) process.
of ML, in which we utilize various loss functions to implement the Given the item–supplier pair 𝑖, let 𝑑𝑘(𝑖) be the disagreement between
aggregation. Then, a comparative analysis is conducted to select the the aggregated evaluation 𝐳 and the evaluation from the MCDM method
appropriate loss function and thus determine the final aggregated 𝑘, 𝐲𝑘 . It is modeled as Eq. (7).
evaluation. The aggregation is conducted for dimension of SR, PI, and
SW, SQC respectively. Since the underlying logic of aggregation for 𝑑𝑘(𝑖) = 𝑧(𝑖) − 𝑦𝑘(𝑖) = 𝑤𝑘 𝑦(𝑖)
𝑘
− 𝑦𝑘(𝑖) (7)
each dimension remains the same, we omit the index of dimensions Loss functions from the ML domain are often used to measure the
for brevity. degree of disagreement. We define the loss function as a function of 𝑑𝑘(𝑖)
in Eq. (8)
4.3.1. Aggregation model
We consider an MCDM-aggregation problem: given a set of prelim- ∑
𝐾
𝑓 (𝑑𝑘(𝑖) ) = 𝐿(𝑧(𝑖) , 𝑦𝑘(𝑖) ) = 𝐿( 𝑤𝑘 𝑦𝑘(𝑖) , 𝑦(𝑖)
𝑘
) (8)
inary evaluations 𝐲𝑘 (𝑘 = 1, … , 𝐾) produced by 𝐾 MCDM methods, 𝑘=1
produce an aggregated evaluation 𝐳 that agrees as much as possible
Then we use an interim function Eq. (9) to measure the average
with the given evaluations. The aggregation problem is described in
disagreement between 𝐳 and 𝐲𝑘 over all 𝑚 item–supplier pairs.
Fig. 3.
1 ∑
𝑚
To be more specific, we attempt to assign a set of weights 𝑤𝑘
(𝑘 = 1, … , 𝐾) to the given evaluations. The weights assigned generally 𝐽 (𝐳, 𝐲𝑘 ) = 𝑓 (𝑑𝑘(𝑖) ) (9)
𝑚 𝑖=1
need to fulfill the non-negativity and unit-sum properties. It is expected
that small weights should be assigned to outliers which often cause Furthermore, the average disagreement across the entire data set
large fitting errors and contribute less to the aggregation, while large (all 𝑚 item–supplier pairs and all 𝐾 evaluations) is in Eq. (10).
weights should be generated for outliers with small errors and great
1 ∑ 1 ∑∑
𝐾 𝐾 𝑚
contributions (Du et al., 2012). Therefore, for each item–supplier pair 𝐶(𝐳, 𝐘) = 𝐽 (𝐳, 𝐲𝑘 ) = 𝑓 (𝑑𝑘(𝑖) ) (10)
𝐾 𝑘=1 𝐾𝑚 𝑘=1 𝑖=1
𝑖, 𝑖 = 1, … , 𝑚, the aggregated evaluation vector 𝐳 = [𝑧(𝑖) ]𝑚 can be viewed
as a weighted linear combination of a weight vector 𝐰 = [𝑤𝑘 ]𝐾 and all Together with the non-negativity and unit weight constraints, the
individual MCDM evaluations 𝐘 = [𝑦(𝑖) ]
𝑘 𝑚×𝐾
as in Eq. (6). aggregation model can be summarized as Eq. (11) as below.
∑
𝐾
1 ∑∑
𝐾 𝑚
𝑧(𝑖) = 𝑤𝑘 𝑦𝑘(𝑖) (6) Min 𝐶(𝐳, 𝐘) = 𝑓 (𝑑𝑘(𝑖) ) (11)
𝑘=1
𝐾𝑚 𝑘=1 𝑖=1
9
form of Eq. (13), where 𝛼𝑘 > 0 is the HQ auxiliary variable. 𝜓(⋅)

s.t. is the convex conjugate function of 𝑓 (𝑑𝑘(𝑖) ) and can be defined as
∑𝐾
𝑘=1 𝑤𝑘 = 1, Eq. (14) (Boyd & Vandenberghe, 2004).
0 ≤ 𝑤𝑘 ≤ 1, 𝑘 = 1, … , 𝐾
𝑓 (𝑑𝑘(𝑖) ) = 𝑄(𝑑𝑘(𝑖) , 𝛼𝑘 ) + 𝜓(𝛼𝑘 ) (13)
The optimal solution of the model Eq. (11) are the optimal weights
𝐰∗ of the selected MCDM methods as Eq. (12).
𝜓(𝛼𝑘 ) = max(𝛼𝑘 𝑑𝑘(𝑖) − 𝑓 (𝑑𝑘(𝑖) )) (14)
𝐰∗ = arg min 𝐶(𝐳, 𝐘) (12) 𝑑𝑘
(𝑖)
The optimal aggregated evaluation of the item–supplier pairs will be,

Where 𝑄(𝑑𝑘(𝑖) , 𝛼𝑘 ) is a quadratic term and can be modeled in an
∑
𝐾
additive form or a multiplicative form (He et al., 2014). The
𝐳∗ = 𝑤∗𝑘 𝐲𝑘
optimization procedure of multiplicative form can be interpreted
𝑘=1
meaningfully within the MCDM context (Mohammadi & Rezaei,
The optimization model in Eq. (11) can be solved by the following
2020). Thus, we define 𝑄(𝑑𝑘(𝑖) , 𝛼𝑘 ) = 12 𝛼𝑘 (𝑑𝑘(𝑖) )2 based on the
steps: (i) choosing specific loss functions, (ii) deriving the interim func-
multiplicative form.
tion Eq. (9), (iii) deriving the cost function Eq. (10); (iv) minimizing
Therefore, we have,
the cost function; and (v) updating the weight vector based on the
1 ∑ ∑{ 1
𝐾 𝑚
minimization technique until it converges or for a fixed number of }
iterations. 𝐶(𝑧(𝑖) , 𝑦𝑘(𝑖) ) == 𝛼 (𝑑 (𝑖) )2 + 𝜓(𝛼𝑘 ) (15)
𝐾𝑚 𝑘=1 𝑖=1 2 𝑘 𝑘
4.3.2. Loss functions The cost function can be minimized through a two-step alter-
The aggregation model of Eq. (11) will only be specified after the nating iteration process until convergence is reached as show in
loss function 𝑓 (𝑑𝑘(𝑖) ) is chosen. Many loss functions are available in Eq. (16)–(17), where 𝑡 indicates the iteration counter and 𝛼𝑘𝑡 is
literature, e.g., MSE (Mean Squared Error), RMSE (Root Mean Squared estimated as function 𝛿(⋅) is
Error), and MAE (Mean Absolute Error), RMSLE (Root Mean Squared
∑
𝑚 𝑚 ∑
∑ 𝐾
Logarithmic Error), etc. Different loss functions provide different un- 𝛼𝑘𝑡+1 = 𝛿( (𝑑𝑘(𝑖) )2 ) = 𝛿( ( 𝑤𝑡𝑘 𝑦𝑘(𝑖) − 𝑦𝑘(𝑖) )2 ) (16)
derstanding of the disagreements (Boehmke & Greenwell, 2019). To 𝑖=1 𝑖=1 𝑘=1
illustrate our aggregation model, we choose four well-known loss func- and
tions including L2 loss (Hastie et al., 2001), L1 loss (Nielsen, 2015), and
Log-cosh (LC) loss (Wang et al., 2020), M-estimator (ME) loss (Moham- ∑
𝐾
𝛼𝑘𝑡+1
𝐰𝑡+1 = argmin ∑𝐾 𝐲 (17)
madi & Rezaei, 2020). A brief description of the four loss functions and 𝐰 𝑡+1 𝑘
𝑘=1 𝑘=1 𝛼𝑘
their features is provided as follows.
Welsch estimator is one of the most promising and outlier-robust
• L2 loss, also known as quadratic loss or squared loss, is one of the estimators (He et al., 2013; Mohammadi & Rezaei, 2020). It only
most popular loss functions. As its name implies, it is computed has one hyper-parameter 𝜎 2 . Thus, we apply the Welsch estimator
based on the square of the disagreements (see the second column in our study. The HQ auxiliary variable, 𝛼𝑘𝑡+1 , is specified as,
of Table 5). It provides a stable and closed-form solution in which ∑
𝑚 (𝑑 (𝑖) )2
the optimal weight for each MCDM method is identical to each − 𝑖=1 2 𝑘
𝛼𝑘𝑡+1 = 𝑒 𝜎
other. Even though L2 loss is practical and straightforward, the
main drawback is that it is sensitive to outliers by squaring dis- Like other kernel methods, the selection of 𝜎 2 will affect the
agreements and may result in distorted results. Gradient descent performance of the proposed method, therefore, it should be care-
is commonly used for L2 loss. fully selected to guarantee the descending of the objective func-
• L1 loss, also known as absolute error loss, which measures the ab- tion. The value of 𝜎 2 herein is computed following the method
solute disagreements without considering directions (see the third in Du et al. (2012) as in Eq. (18).
column of Table 5). Compared with L2 loss, this loss function is ∑𝐾 ∑𝑚 (𝑖) 2
more robust to outliers. However, its derivatives are not contin- 𝑘=1 𝑖=1 (𝑑𝑘 )
𝜎2 = (18)
uous, resulting in inefficient computation and unstable solutions 2𝑚
(multiple possible solutions). Gradient descent is commonly used For more information of different estimator functions 𝑓 (⋅) and
for L1 loss. their corresponding minimizer functions 𝛿(⋅), see He et al. (2014).
• LC loss is the logarithm of the hyperbolic cosine of disagreements
(see the fourth column of Table 5). LC loss is approximately equal To determine the best weights, as indicated at the fifth step in
to L2 loss for small disagreements and to L1 loss minus log(2) Table 5, an initialization step of weight 𝑤(0) and learning rate 𝑙𝑟
𝑘
for the large disagreements. Because of this feature, LC loss is (controls how quickly a model learns a problem) is required. However,
less sensitive to outliers than L2 loss, and it is twice differen-
different initiation settings may result in different solutions. Hence, we
tiable everywhere compared with L1 loss. Gradient descent is also
run the models with various settings (initial weight, learning rate, or
commonly used for LC loss.
both) to locate the best weights 𝐰∗ and aggregated evaluation 𝐳∗ .
• ME (Welsch) loss is a family of robust estimators (e.g. L1–L2,
Among the four types of Loss functions, only the L2 loss leads
Fair, Log-cosh, and Welsch, Huber) based on maximum-likelihood
estimation and can be treated as loss functions (see the fifth to a linear programming model and can be solve to optimally. The
column of Table 5). Since ME loss is possibly non-convex, the other three loss functions leads to non-linear optimization problems and
HQ minimization technique originated from conjugate function may have multiple local minimums. However, the linear relationship
theory can be utilized to optimize these functions and obtain their is a simplification of the measurements of the disagreements among
optimum (He et al., 2014; Mohammadi & Rezaei, 2020). different MCDM methods. Therefore, we compare the solutions from
The HQ minimization technique is more complicated and quite the different loss functions in next section.
different from Gradient descent technique. More details about HQ Application steps of the loss functions in the aggregation model are
technique is presented here. The Loss function 𝑓 (𝑑𝑘(𝑖) ) takes the summarized in Table 5.
10
Table 5
Summary of the steps for each loss function.
Steps L2 loss L1 loss LC loss ME (Welsch) loss
cosh(𝑑𝑘(𝑖) )
( )2
i 𝐿= (𝑑𝑘(𝑖) )2 𝐿= |𝑑𝑘(𝑖) | 𝐿 = log 𝐿 = 21 𝛼𝑘 𝑑𝑘(𝑖) + 𝜓(𝛼𝑘 )
𝑚{ ( )2 }
1 ∑ (𝑖) 2 1 ∑ ∑ ∑
𝑚 𝑚 𝑚 (𝑖)
ii 𝐽= 𝑚
(𝑑𝑘 ) 𝐽= 𝑚
|𝑑𝑘(𝑖) | 𝐽= 1
𝑚
logcosh(𝑑𝑘 )
𝐽= 1
𝑚
1
2
𝛼𝑘 𝑑𝑘(𝑖) + 𝜓(𝛼𝑘 )
𝑖=1 𝑖=1 𝑖=1 𝑖=1
𝐾 𝑚{ ( )2 }
∑ ∑
𝐾 𝑚
∑ ∑
𝐾 𝑚
∑ ∑
𝐾 𝑚 (𝑖) ∑ ∑ 1
iii 𝐶= 1
𝐾𝑚
(𝑑𝑘(𝑖) )2 𝐶= 1
𝐾𝑚
|𝑑𝑘(𝑖) | 𝐶= 1
𝐾𝑚
logcosh(𝑑𝑘 )
𝐶= 1
𝐾𝑚 2
𝛼𝑘 𝑑𝑘(𝑖) + 𝜓(𝛼𝑘 )
𝑘=1 𝑖=1 𝑘=1 𝑖=1 𝑘=1 𝑖=1 𝑘=1 𝑖=1
iv Gradient descent Gradient descent Gradient descent HQ minimization

v Initialize 𝑤(0)
𝑘
Initialize 𝑤(0)
𝑘
Initialize 𝑤(0)
𝑘
Initialize 𝑤(0) 𝑘
∑
𝐾
∑
𝐾
∑
𝐾
∑𝐾
(𝑑𝑘(𝑖) )𝑡 = 𝑤𝑡𝑘 𝑦(𝑖)
𝑘
− 𝑦(𝑖)
𝑘
(𝑑𝑘(𝑖) )𝑡 = 𝑤𝑡𝑘 𝑦𝑘(𝑖) − 𝑦(𝑖)
𝑘
(𝑑𝑘(𝑖) )𝑡 = 𝑤𝑡𝑘 𝑦(𝑖)
𝑘
− 𝑦𝑘(𝑖) (𝑑𝑘(𝑖) )𝑡 = 𝑤𝑡𝑘 𝑦(𝑖)
𝑘
− 𝑦(𝑖)
𝑘
𝑘=1 𝑘=1 𝑘=1 √
𝑘=1
𝐾
∑∑
𝑚 𝐾
∑∑
𝑚
(𝑑𝑘(𝑖) )𝑡
𝐾
∑∑
𝑚
∑ ∑ 𝑡 (𝑖)
𝑚 𝐾
𝑔𝑘(𝑡) = 2
(𝑑𝑘(𝑖) )𝑡 𝑦(𝑖) 𝑔𝑘(𝑡) = 1
𝑦(𝑖) 𝑔𝑘 = 1
tanh(𝑑𝑘(𝑖) )𝑦𝑘(𝑖) 𝛼𝑘𝑡+1 = 𝛿( ( 𝑤𝑘 𝑌𝑘 − 𝑦(𝑖) )2 )
𝐾𝑚 𝑘 𝐾𝑚 |(𝑑𝑘(𝑖) )𝑡 | 𝑘 𝐾𝑚 𝑘
𝑘=1 𝑖=1 𝑘=1 𝑖=1 𝑘=1 𝑖=1 𝑖=1 𝑘=1
𝑤(𝑡) −𝑔𝑘(𝑡) ∗𝑙𝑟 𝑤(𝑡) −𝑔𝑘(𝑡) ∗𝑙𝑟 𝑤𝑘(𝑡) −𝑔𝑘(𝑡) ∗𝑙𝑟 𝛼𝑘𝑡+1
𝑤𝑡+1 = 𝑘
𝑤𝑡+1 = 𝑘
𝑤𝑡+1 = 𝑤𝑡+1 = ∑𝐾
𝑘 ∑
𝐾 𝑘 ∑
𝐾 𝑘 ∑
𝐾 𝑘
𝑘=1 𝛼𝑘
𝑡+1
(𝑤(𝑡)
𝑘
−𝑔𝑘(𝑡) ∗𝑙𝑟) (𝑤(𝑡)
𝑘
−𝑔𝑘(𝑡) ∗𝑙𝑟) (𝑤𝑘(𝑡) −𝑔𝑘(𝑡) ∗𝑙𝑟)
𝑘=1 𝑘=1 𝑘=1
𝑡 is the iteration counter, 𝑡 is the iteration counter, 𝑡 is the iteration counter, 𝑡 is the iteration counter,
𝑙𝑟 is the learning rate 𝑙𝑟 is the learning rate 𝑙𝑟 is the learning rate 𝛼𝑘 is the HQ auxiliary variable
4.3.3. Comparison analysis In general, we aim to select the aggregation model that simulta-
As previously mentioned, loss functions use distinctive mechanisms neously yields the highest assessment measures in Eqs. (20) and (22).
to handle disagreements and may lead to drastic differences in how Thus, we integrated the measures as,
we interpret the ‘‘optimal model’’ and thus the optimum aggregation 𝐅(𝐳∗ ) = 𝛽 ∗ 𝐂(𝐳∗ ) + (1 − 𝛽) ∗ 𝐓(𝐳∗ ) (23)
result. Thus, we have to answer two questions: (1) how to assess an
individual aggregation model embedded with a specific loss function. The higher the integrated measure 𝐅 is, the more favored the aggre-
(2) how to compare several aggregation models embedded with various gation model is. The value of 𝛽 can be chosen by the decision makers
loss functions. So far, very little study has been done about the issues. based on their expertise or estimated by the historical performances
Mohammadi and Rezaei (2020) tried to analyze the questions with of the measurements if exist. A sensitive analysis can be performed
respect to the Welsch loss function. Inspired by Mohammadi and Rezaei when decision-makers are uncertain about 𝛽 so that the impacts of the
preferences of the decision makers can be learned.
(2020), we propose to use Consensus Index and Trust Level (Kou et al.,
2012) to assess the performances of the aggregation model based on
5. Application case and results
different Loss functions.
We use an application case to illustrate the proposed DSS and

• Consensus Index shows the extent to which all MCDM meth-
methodologies. The application case is performed in an automotive
ods being used agree upon the final aggregated evaluation (Mo-
company for two reasons. First, the automotive industry has a high de-
hammadi & Rezaei, 2020). We use Spearman’s rank correlation
gree of outsourcing (Akman, 2015). Second, automobiles are complex
coefficient in Eq. (19) to compute the index.
products, and suppliers’ quality performance significantly affects their
∑ (𝑖) 2
6 𝑚 𝑖=1 (𝑟𝑘 )
quality levels. A subset with 100 item–supplier pairs is selected in our
𝜌𝑘 = 1 − (19) application case, and data is collected for one month.
𝑚(𝑚2 − 1)
where 5.1. Decision matrix and criteria weights initialization
𝑟(𝑖)
𝑘
= |𝑅(𝑦𝑘(𝑖) ) − 𝑅(𝑧∗(𝑖) )| (20)
As previously mentioned in Section 4.1, a criteria library is con-
𝑟(𝑖)
measures the difference between the rank of the evaluation of structed by a literature review and exploring supplier databases. Sub-
𝑘
the MCDM method 𝑘 and the rank of the aggregated evaluation sequently, key criteria are selected by the expert team, as provided in
regarding the item–supplier pair 𝑖. Table 6 and Table 7. Totally, 23 criteria are selected, and they are
of different types, e.g. binary, nominal, and continuous. Among all
Then the Consensus Index is calculated by,
criteria, 7 are quantitative, and 16 are qualitative. Nominal criteria are
1 ∑
𝐾
further illustrated in Table 8. Data sources are indicated in Table 9,
𝐂(𝐳∗ ) = 𝜌 (21) including supplier databases and six relevant departments.
𝐾 𝑘=1 𝑘
Then, the initial decision matrix for 100 item–supplier pairs regard-
The larger the value of 𝐂(𝐳∗ ) is, the better the aggregated evalu- ing the 23 criteria is constructed (Table 10). In this case, for the sake of
ation is. simplicity, the 23 criteria are considered equally important. Note that
• Trust Level is the degree to which one can accredit the final criteria may have different weights in other cases, but the logic and
aggregated evaluation (Mohammadi & Rezaei, 2020). As shown methodologies in the DSS remain the same. Given four dimensions, the
in Eq. (22), the index 𝐓(𝐳∗ ) is derived by using 𝐰∗ and 𝜌𝑘 so following procedure for each dimension is the same.
that the MCDM method with a higher weight can make more
contributions to the reliability of the final aggregated evaluation. 5.2. MCDM methods selection and implementation
Compared to the index 𝐂, the only difference is the MCDM with
a higher wight can make more contributions for the index 𝐓. If As described in Section 4.2, six MCDM methods are applied in-
the optimal weights are identical for each MCDM method (e.g. for cluding TOPSIS (M1), VIKOR (M2), PROMETHEE II with Usual type
the L2 loss), 𝐓(𝐳∗ ) will equal to 𝐂(𝐰∗ ). (M3), PROMETHEE II with Gaussian type (M4), MOORA with the
ratio system (M5), and MOORA with the reference approach (M6).
∑
𝐾
In order to implement the six MCDM methods, the data in Table 10
𝐓(𝐳∗ ) = 𝑤∗𝑘 𝜌𝑘 (22)
need to be normalized. Linear normalization and vector normalization
𝑘=1
11
Table 6
Criteria selected to measure the supply risk and profit impact dimensions.
Code Supply risk criteria (SR) Data Type Scale Objective Source
C01 Geographical location Binary [0 or 1] Maximize 1
C02 Number of available suppliers Continuous Z≥0 Maximize 2
C03 Product availability Nominal [0,4,6,8,10] Maximize 3
C04 Delivery time Nominal [0,4,6,8,10] Maximize 3
C05 Substitution possibilities Nominal [0,0.4,0.6,0.8,1] Maximize 2
Code Profit impact criteria (PI) Data Type Scale Objective Source
C06 Total amount purchased Continuous [0–1] Maximize 2
C07 Expected growth in company’s demand Nominal [0,0.4,0.6,0.8,1] Maximize 4
C08 Perceived bargaining power of the buyer Nominal [0,4,6,8,10] Maximize 2
C09 Product price Continuous R>0 Minimize 1
C10 The importance of the product in the project sequence Nominal [0,4,6,8,10] Maximize 5
Table 7
Criteria used to measure the supplier willingness and supplier quality capability dimensions.
Code Supplier willingness criteria (SW) Data Type Scale Objective Source
C11 Honest and frequent communications Nominal [0,4,6,8,10] Maximize 2,3,6
C12 Relationship closeness Nominal [0,4,6,8,10] Maximize 2,3,6
C13 Commitment to continuous improvement in product and process Nominal [0,4,6,8,10] Maximize 2,3,6
C14 Prior experience with supplier Nominal [0,4,6,8,10] Maximize 2,3,6
C15 Willingness to share information Nominal [0,4,6,8,10] Maximize 2,3,6
C16 Willingness to share cost savings Nominal [0,4,6,8,10] Maximize 2,3,6
C17 Willingness to invest in specific equipment Nominal [0,4,6,8,10] Maximize 2,3,6
Code Supplier quality capability criteria (SQC) Data Type Scale Objective Source
C18 ̂
Defect per 100 units (𝐷𝑃 𝑈) Continuous [0–1) Minimize 1
C19 Quality fluctuation (𝑄𝐹 ) [1–∞) Continuous Minimize 6
C20 ̂𝐼 )
Quality cost performance index (𝑄𝐶𝑃 Continuous [0–∞) Minimize 7
C21 Cost fluctuation (𝐶𝐹 ) [1–∞) Continuous Minimize 6
C22 ̂
Risk priority number (𝑅𝑃 𝑁) [1–∞) Continuous Minimize 6
C23 Risk fluctuation (𝑅𝐹 ) [1–∞) Continuous Minimize 6
Table 8
Evaluation scale for nominal criteria.
0 4/0.4 6/0.6 8/0.8 10/1
Unsatisfactory Partially meet expectations Largely meet expectations Good and satisfactory Excellent
Very low Low Medial High Very high
Table 9
Data sources.
Source 1 Source 2 Source 3 Source 4 Source 5 Source 6 Source 7
Databases Purchasing Material Steering Marketing Assembly Quality Financial
Table 10
Decision matrix for supplier quality evaluation.
ID C01 C02 C03 C04 C05 C06 C07 C08 C09 C10 C11 C12 ⋯ C23
A01 1 15 8 4 0.6 0.53 0.4 10 70 8 10 0 3679.67
A02 1 12 8 8 0.8 0.29 0 8 104 8 4 10 574.64
A03 1 7 4 4 0.6 0.77 0.8 8 224 8 6 6 23 096.47
A04 1 1 8 4 0.8 0.76 0 10 79 0 8 0 507.18
A05 1 17 0 8 0 0.71 0.6 8 130 4 4 10 594.89
A06 1 1 0 10 0.8 0.41 0.6 0 119 10 8 6 1094.58
A07 1 18 8 8 0.8 0.79 0.4 0 550 8 4 4 912.92
A08 1 8 10 8 0.6 0.96 0.4 10 72 10 10 10 2635.94
⋯
A100 1 20 6 8 1 0.34 0 0 130 4 6 4 211.34
are commonly used in literature. Linear normalization (aka min–max the MCDM method M1, M5, and M6, decision matrix is normalized by
normalization) and vector normalization are used to normalize the vector normalization, while the others use linear normalization.
decision matrix. We then implement all these six methods on the normalized decision
matrix using R (version 4.0.3) through RStudio (version 1.3.1093) on
Let 𝑥𝑖 𝑗 denote the original score of the supplier–item pair 𝑖 regard-
a 64-bit Windows 10 PC with Intel Core i5-8250U 1.60 GHz and 8 GB
ing criteria 𝑗, 𝑥𝑚𝑖𝑛 be the minimum score of the criterion, and 𝑥𝑚𝑎𝑥 be
RAM. After that, an evaluation matrix 𝐘 with 100 rows (item–supplier
the maximum score of the criterion. The linear normalized score is 𝑥∗𝑖𝑗 =
𝑥−𝑥𝑚𝑖𝑛 𝑥 −𝑥 pairs) and six columns (MCDM methods) is obtained. For comparison
𝑥 −𝑥
for beneficial criteria and 𝑥∗𝑖𝑗 = 𝑥 𝑚𝑎𝑥−𝑥 for non-beneficial
𝑚𝑎𝑥 𝑚𝑖𝑛 𝑚𝑎𝑥
𝑥𝑚𝑖𝑛 and aggregation purposes, we use linear normalization on each column
criteria. The vector normalized score is 𝑥∗𝑖𝑗 = √∑𝑚𝑖𝑗 . In this case, for
2
𝑖=1 𝑥𝑖𝑗
of the evaluation matrix, as shown in Table 11.
12
Table 11
Normalized evaluation matrix for SR, PI, SW, and SQC dimensions.
ID SR PI
M1 M2 M3 M4 M5 M6 M1 M2 M3 M4 M5 M6
A01 0.662 0.668 0.656 0.749 0.713 0.545 0.763 0.399 0.737 0.712 0.769 0.829
A02 0.835 0.837 0.845 0.864 0.852 0.712 0.594 0.212 0.449 0.458 0.553 0.660
A03 0.349 0.463 0.339 0.489 0.435 0.389 0.836 0.616 0.658 0.829 0.808 0.906
A04 0.417 0.380 0.466 0.558 0.521 0.000 0.588 0.186 0.539 0.432 0.557 0.660
A05 0.356 0.307 0.452 0.459 0.424 0.077 0.748 0.411 0.521 0.669 0.701 0.832
A06 0.361 0.339 0.464 0.479 0.453 0.000 0.616 0.237 0.498 0.523 0.584 0.674
A07 1.000 1.000 0.932 0.972 0.963 1.000 0.413 0.301 0.309 0.369 0.394 0.618
A08 0.687 0.700 0.750 0.795 0.780 0.453 0.863 0.521 0.912 0.914 0.930 0.829
⋯
A100 0.954 0.932 0.984 1.000 0.999 0.769 0.469 0.000 0.104 0.153 0.289 0.660
ID SW SQC
M1 M2 M3 M4 M5 M6 M1 M2 M3 M4 M5 M6
A01 0.361 0.275 0.394 0.425 0.425 0.016 0.626 0.762 0.145 0.401 0.706 0.649
A02 0.575 0.575 0.534 0.630 0.621 0.531 0.888 0.928 0.285 0.890 0.920 0.897
A03 0.447 0.500 0.417 0.523 0.498 0.498 0.140 0.489 0.128 0.068 0.212 0.008
A04 0.241 0.175 0.243 0.278 0.273 0.020 0.924 0.949 0.385 0.933 0.945 0.940
A05 0.216 0.150 0.232 0.216 0.226 0.016 0.896 0.938 0.328 0.905 0.926 0.899
A06 0.615 0.600 0.578 0.681 0.656 0.510 0.875 0.923 0.283 0.873 0.907 0.894
A07 0.567 0.575 0.539 0.634 0.617 0.525 0.870 0.930 0.377 0.875 0.904 0.895
A08 0.755 0.725 0.817 0.857 0.843 0.498 0.804 0.859 0.177 0.712 0.841 0.852
⋯
A100 0.513 0.550 0.518 0.598 0.577 0.498 0.994 0.997 0.799 0.996 0.995 0.996
For better illustration, we plot the evaluation values of the first that there are two extreme situations in which 𝐹10 = 𝐂 for 𝛽 = 1 and
ten item–supplier pairs in Fig. 4. Aggregated evaluations of four loss 𝐹0 = 𝐓 for 𝛽 = 0. The best course for each scenario is marked in green.
functions are plotted and marked by AG1, AG2, AG3, and AG4. Ap- Additionally, we summarize the execution time of aggregation mod-
parently, single MCDM methods present inconsistent evaluations. For els with various loss functions in Table 14.
instance, M6 significantly stands out in the dimension of SR and SW. Several interesting findings are explored in the case study. First,
M2 and M3 show considerable differences in the PI and SQC dimen- most of the trust levels are greater than 90%. It means aggregated
sions, respectively. We can draw two conclusions here: (i) given the evaluations produced by our model reach a good agreement with
same data sets, different MCDM methods may yield distinctive results; all MCDM methods and have a high accreditation. Fig. 4 gives a
(ii) given various data sets, an identical MCDM method may have justification for this conclusion. Our aggregated approach produces
different performances. These two conclusions justify our motivation more stable evaluations than single MCDM methods. Second, given a
that it is essential to aggregate multiple MCDM methods to obtain a different value of 𝛽, the best course changes. For the SR dimension,
robust and trustworthy result. ME loss has obvious advantages over the others in Scenario zero to
Scenario eight, while L1 loss is preferable in Scenario nine to Scenario
5.3. Aggregation ten. Third, ME loss and L1 loss are more competitive than L2 loss and
LC loss in producing the best courses with 66% and 28% probability.
In other words, ME loss and L1 loss perform better than the traditional
5.3.1. Aggregation on four loss functions
averaging method because the L2 loss function actually produces the
As mentioned in Sections Section 4.3.1, four loss functions (L2, L1,
average value. It indicates that decision-makers should prioritize these
LC, and ME loss) are applied in the aggregation model to obtain the
two loss functions, especially when they are not sure about the value
aggregated evaluations. Except for the L2 loss, the results of the other
of 𝛽.
three loss functions are affected by the learning rate or initial weight,
To further explore the generality of this point, testing more data
or both. In this case, we consider three learning rates (1, 0.1, and
sets is necessary. Finally, L1 and LC loss need a much longer execution
0.01) and 100 random initial weights. In total, for each loss function
time. The main reason is these two loss functions suffer from a large
embedded in the aggregation model, we have 300 (3*100) possible
number (300) of random settings of learning rate and initial weight.
settings. The setting with the minimum disagreement is selected. Take
L2 loss has no random setting and produces a closed-form solution. Its
the SR dimension using L1 loss as an example, disagreements range
execution time is negligible. ME loss has 100 random settings of initial
from 0.076 to 0.131. Moreover, the minimum disagreement is gener-
weights.
ated by the combination of the learning rate of 0.1 and the seventeenth
In accordance with the result of the comparison analysis, we choose
initial weight. Based on the loss function and a selected initial setting, ME loss function to generate the final aggregated evaluations, as shown
aggregated evaluations are produced and normalized, see Table 12. in Table 15.
The results in the column of the SQC dimension in Table 15 show
5.3.2. Comparison analysis that decision-makers are able to identify suppliers with low-quality per-
According to Table 12, different loss functions result in different formance. For example, the No. 3 item–supplier pair has a significantly
aggregated evaluation values. Which one is better? Here we conduct low SQC value, indicating a low-quality level. Hence, decision-makers
the comparison analysis to determine the appropriate loss function. should pay special attention to this item–supplier pair to reduce quality
Assessment measures (𝐂, 𝐓, 𝐅) defined in Section 4.3.3 are used. risk. Based on that, decision-makers develop supplier quality improve-
Additionally, we consider eleven scenarios with the value of 𝛽 ranging ment strategies by considering the other three dimensions. In addition,
from 0 to 1 (step size is 0.1). For each scenario, assessment measure 𝐅 is the No. 3 item–supplier pair has a low SR value, referring to high
calculated, are shown in Table 13. For traceability, we use the subscript supply risk. This means only a few potential suppliers are available, and
of 𝐅 to indicate a single scenario with specific 𝛽. For instance, 𝐹9 is the suppliers usually have more power. It can be challenging to seek new
assessment measure for the ninth scenario with 𝛽 equal to 0.9. Note qualified suppliers or obtain a good bargain with existing suppliers.
13
Fig. 4. Scores plot of the first 10 item–supplier pairs for each dimension.
Besides that, this item is expensive and affects profits (PI value). There- of the availability of the big industrial data. The DSS includes a detailed
fore, to ensure supply continuity, the suitable strategy is to develop conceptual framework for the supplier quality evaluation in which
the current supplier regardless of its willingness (SW value). On the evaluation criteria from multiple dimensions, data input and output
other side, decision-makers can dynamically monitor supplier quality are clearly defined. Constructing the dimension of SQC with various
performance to verify the strategy’s effectiveness. types of data helps to create deeper understanding of a data-driven and
systematic supplier quality evaluation in the manufacturing industry.
The evaluation dimension of SQC makes it possible to link evalua-
6. Conclusions and future research
tion outcomes to supplier quality development. The implementation of
the DSS system can be applied to dynamically monitor and evaluate
As supplier quality becomes increasingly important in the supply supplier and item quality performance. In addition, we propose an
chain network, manufacturers are forced to evaluate and monitor sup- aggregated MCDM model based on the ML concepts. It aims at pro-
plier quality performance on a regular basis for quality improvement viding more reliable evaluations across multiple suppliers and items.
or development. In this paper, we develop a DSS for supplier quality The aggregation model also contributes to MCDM literature as an
evaluation to deal with the practical complexity and to take advantage independent method which is not restricted to the purpose of supplier
14
Table 12
Aggregated evaluations for SR, PI, SW, and SQC dimensions.
ID SR PI
L2 L1 LC ME L2 L1 LC ME
A1 0.674 0.688 0.669 0.662 0.725 0.740 0.719 0.737
A2 0.840 0.851 0.839 0.835 0.472 0.508 0.481 0.449
A3 0.408 0.401 0.400 0.349 0.812 0.819 0.823 0.658
A4 0.387 0.428 0.386 0.417 0.478 0.498 0.480 0.539
A5 0.341 0.374 0.337 0.356 0.660 0.687 0.672 0.521
A6 0.344 0.379 0.343 0.361 0.512 0.554 0.520 0.498
A7 1.000 1.000 1.000 1.000 0.369 0.381 0.366 0.309
A8 0.704 0.725 0.700 0.687 0.874 0.916 0.866 0.912
⋯
A100 0.960 0.979 0.957 0.954 0.225 0.234 0.254 0.104
ID SW SQC
A1 0.316 0.362 0.317 0.424 0.548 0.577 0.557 0.411
A2 0.578 0.575 0.574 0.625 0.801 0.837 0.790 0.890
A3 0.481 0.465 0.476 0.510 0.174 0.178 0.186 0.069
A4 0.205 0.232 0.206 0.276 0.846 0.878 0.836 0.932
A5 0.176 0.204 0.179 0.221 0.815 0.849 0.804 0.905
A6 0.607 0.610 0.606 0.668 0.793 0.828 0.782 0.874
A7 0.576 0.574 0.572 0.625 0.809 0.839 0.799 0.876
A8 0.749 0.786 0.748 0.849 0.708 0.745 0.704 0.716
⋯
A100 0.542 0.540 0.538 0.587 0.963 0.974 0.960 0.996
Table 13
Comparison analysis on aggregated evaluations for SR, PI, SW, and SQC dimensions.
SR PI
Index
C(𝐹10 ) 0.952 0.953 0.952 0.947 0.904 0.899 0.905 0.854
T(𝐹0 ) 0.952 0.966 0.959 1.000 0.904 0.985 0.908 1.000
𝐹1 0.952 0.964 0.958 0.995 0.904 0.976 0.907 0.985
𝐹2 0.952 0.963 0.957 0.989 0.904 0.968 0.907 0.971
𝐹3 0.952 0.962 0.957 0.984 0.904 0.959 0.907 0.956
𝐹4 0.952 0.960 0.956 0.979 0.904 0.951 0.907 0.941
𝐹5 0.952 0.959 0.955 0.973 0.904 0.942 0.906 0.927
𝐹6 0.952 0.958 0.955 0.968 0.904 0.933 0.906 0.912
𝐹7 0.952 0.957 0.954 0.963 0.904 0.925 0.906 0.898
𝐹8 0.952 0.955 0.954 0.957 0.904 0.916 0.906 0.883
𝐹9 0.952 0.954 0.953 0.952 0.904 0.907 0.905 0.868
SW SQC
Index
C(𝐹10 ) 0.918 0.918 0.918 0.917 0.976 0.978 0.976 0.974
T(𝐹0 ) 0.918 0.945 0.933 0.998 0.976 0.980 0.975 0.999
𝐹1 0.918 0.942 0.931 0.990 0.976 0.980 0.975 0.997
𝐹2 0.918 0.939 0.930 0.982 0.976 0.980 0.975 0.994
𝐹3 0.918 0.937 0.928 0.974 0.976 0.980 0.975 0.992
𝐹4 0.918 0.934 0.927 0.966 0.976 0.979 0.975 0.989
𝐹5 0.918 0.932 0.926 0.957 0.976 0.979 0.975 0.987
𝐹6 0.918 0.929 0.924 0.949 0.976 0.979 0.975 0.984
𝐹7 0.918 0.926 0.923 0.941 0.976 0.979 0.975 0.982
𝐹8 0.918 0.924 0.921 0.933 0.976 0.979 0.976 0.979
𝐹9 0.918 0.921 0.920 0.925 0.976 0.979 0.976 0.977
Table 14
Comparison analysis on execution time of aggregation model for SR, PI, SW, and SQC dimensions.
Time SR (s) PI (s)
– 29.706 239.955 0.254 – 69.348 203.675 0.116
Time (s) SW (s) SQC (s)
– 3.913 199.001 0.117 – 494.325 165.617 0.762
quality evaluation. The aggregation method sheds lights on how to Despite the multiple contributions and implications associated with
construct and solve MCDM problems in the big-data era. Finally, a case the study, the study can be further extended and deepened on several
study shows that our DSS provide more robust and reliable evaluation perspectives. First, the individual MCDM methods are selected based
results than the traditional individual MCDM method. on rule of thumbs. It is interesting to have a thorough analysis about
15
Table 15
Final aggregated evaluations for SR, PI, SW, and SQC dimensions.
ID SR PI SW SQC ID SR PI SW SQC
A1 0.662 0.737 0.424 0.411 A11 0.203 0.948 0.352 0.689
A2 0.835 0.449 0.625 0.890 A12 0.331 0.449 0.541 0.774
A3 0.349 0.658 0.510 0.069 A13 0.811 0.591 0.586 0.942
A4 0.417 0.539 0.276 0.932 A14 0.684 0.863 0.549 0.969
A5 0.356 0.521 0.221 0.905 A15 0.291 0.191 0.540 0.920
A6 0.361 0.498 0.668 0.874 A16 0.920 0.728 0.542 0.375
A7 1.000 0.309 0.625 0.876 A17 0.627 0.630 0.552 0.945
A8 0.687 0.912 0.849 0.716 A18 0.441 0.126 0.395 0.944
A9 0.374 0.395 0.462 0.157 ⋯
A10 0.593 0.529 0.416 0.951 A100 0.954 0.104 0.587 0.996
the evaluation tendency of each MCDM method and relevance of the Boehmke, B., & Greenwell, B. (2019). Hands-on machine learning with R (1st ed.).
different methods so that more rigorous selection rules and process can Chapman and Hall/CRC.
Boyd, S. P., & Vandenberghe, L. (2004). Convex optimization. Cambridge University
be found. Second, the aggregation model only used the concept of ML
Press.
about the construction of the aggregation model and the online best-in- Brauers, W. K. (2003). Optimization methods for a stakeholder society: A revolution in
class ML packages are used to solve the model. Given the involvement economic thinking by multi-objective optimization, Vol. 73. Springer Science & Business
of large real world data, multiple evaluation dimensions, and item– Media.
supplier pairs, applying the ML algorithms to train the model more Chai, J., Liu, J. N., & Ngai, E. W. (2013). Application of decision-making techniques in
supplier selection: A systematic review of literature. Expert Systems with Applications,
efficiently will be studied in future, e.g., Neural networks, boosting,
40(10), 3872–3885.
etc. Third, we transform different types of data into quantitative mea- Chakraborty, S. (2011). Applications of the MOORA method for decision making
surements in order to aggregate the evaluation results. The vagueness in manufacturing environment. International Journal of Advanced Manufacturing
and uncertainty of linguistic commentary data are not considered. Technology, 54(9–12), 1155–1166. http://dx.doi.org/10.1007/s00170-010-2972-0.
Demir, L., Akpınar, M. E., Araz, C., & Ilgın, M. A. (2018). A green supplier evaluation
Integrating with fuzzy set theory can be a potential topic to handle
system based on a new multi-criteria sorting method: VIKORSORT. Expert Systems
and explore linguistic commentary data in future. At last, we focus with Applications, 114, 479–487.
on the supplier quality evaluation in this paper, and how to apply Department of the Army, TM5-689-4 (2006). TM 5-698-4 failure modes, effects and
the results of the supplier quality evaluation to facilitate the supplier criticality analysis (FMECA) for command, control, communications, computer,
quality development will be the next goal of our research. intelligence, surveillance, and reconnaissance (C4ISR) facilities. (p. 75).
Dogan, A., & Birant, D. (2021). Machine learning and data mining in manufacturing.
Expert Systems with Applications, 166, Article 114060.
CRediT authorship contribution statement Dong, Y., Xu, K., Xu, Y., & Wan, X. (2016). Quality management in multi-level supply
chains with outsourced manufacturing. Production and Operations Management,
Qiuping Ma: Conceptualization, Methodology, Data curation, Cod- 25(2), 290–305. http://dx.doi.org/10.1111/poms.12428.
ing, Writing original draft. Hongyan Li: Conceptualization, Methodol- Du, L., Li, X., & Shen, Y.-D. (2012). Robust nonnegative matrix factorization via half-
quadratic minimization. In 2012 IEEE 12th international conference on data mining
ogy, Writing original draft.
(pp. 201–210). Brussels, Belgium: IEEE.
Galo, N. R., Calache, L. D. D. R., & Carpinetti, L. C. R. (2018). A group decision
Declaration of competing interest approach for supplier categorization based on hesitant fuzzy and ELECTRE TRI.
International Journal of Production Economics, 202, 182–196.
The authors declare that they have no known competing finan- Ghatak, A. (2019). Deep learning with R, vol. 245. Springer Singapore.
Greco, S., Ehrgott, M., & Figueira, J. R. (Eds.), (2016). Multiple criteria decision
cial interests or personal relationships that could have appeared to
analysis: State of the art surveys. In International series in operations research &
influence the work reported in this paper. management science: vol. 233, Springer New York.
Hamdi, F., Ghorbel, A., Masmoudi, F., & Dupont, L. (2018). Optimization of a supply
Data availability portfolio in the context of supply chain risk management: Literature review. Journal
of Intelligent Manufacturing, 29(4), 763–788.
Handley, S. M., & Gray, J. V. (2013). Inter-organizational quality management:
Data will be made available on request.
The use of contractual incentives and monitoring mechanisms with outsourced
manufacturing. Production and Operations Management, 22(6), 1540–1556. http:
References //dx.doi.org/10.1111/j.1937-5956.2012.01351.x.
Hastie, T., Friedman, J., & Tibshirani, R. (2001). Springer series in statistics, The elements
Agrawal, A., Muthulingam, S., & Rajapakshe, T. (2017). How sourcing of interdependent of statistical learning. New York, NY: Springer New York, http://dx.doi.org/10.1007/
components affects quality in automotive supply chains. Production and Operations 978-0-387-21606-5.
Management, 26(8), 1512–1533. He, R., Zheng, W. S., Hu, B. G. H., & Kong, X. W. K. (2013). Two-stage nonnegative
Ahmadi, A., & Golbabaie, E. (2015). A hybrid approach to supplier performance sparse representation for large-scale face recognition. IEEE Transactions on Neural
evaluation using artificial neural network: A case study in automobile industry. Networks and Learning Systems, 24(1), 35–46.
Journal of Industrial and Systems Engineering, 8(1), 1–20. He, R., Zheng, W.-S., Tan, T., & Sun, Z. S. (2014). Half-quadratic-based iterative
Akman, G. (2015). Evaluating suppliers to include green supplier development programs minimization for robust sparse representation. IEEE Transactions on Pattern Analysis
via fuzzy C-means and VIKOR methods. Computers & Industrial Engineering, 86, and Machine Intelligence, 36(2), 261–275.
69–82. Jahan, A., Ismail, M. Y., Shuib, S., Norfazidah, D., & Edwards, K. (2011). An aggregation
Aksoy, A., & Öztürk, N. (2011). Supplier selection and performance evaluation technique for optimal decision-making in materials selection. Materials & Design,
in just-in-time production environments. Expert Systems with Applications, 38(5), 32(10), 4918–4924.
6351–6359. Jha, N., Kumar, R., Kumari, A., & Bepari, B. (2014). Design, development and
Al Salem, A., Awasthi, A., & Wang, C. (2016). A multicriteria multistep approach for implementation of a robust decision support expert system (branDEC) in multi
evaluating supplier quality in large data sets. In Quality and reliability management criteria decision making. Procedia Engineering, 97, 1853–1865.
and its applications (pp. 3–25). Springer London. Kamble, S. S., & Gunasekaran, A. (2020). Big data-driven supply chain performance
Alinezhad, A., & Khalili, J. (2019). International series in operations research & man- measurement system: A review and framework for implementation. International
agement science: vol. 277, New methods and applications in multiple attribute decision Journal of Production Research, 58(1), 65–86.
making(MADM). Springer International Publishing. Kim, K. O., Yang, Y., & Zuo, M. J. (2013). A new reliability allocation weight for
Bai, C., Rezaei, J., & Sarkis, J. (2017). Multicriteria green supplier segmentation. IEEE reducing the occurrence of severe failure effects. Reliability Engineering & System
Transactions on Engineering Management, 64(4), 515–528. Safety, 117, 81–88.
Bertolini, M., Mezzogori, D., Neroni, M., & Zammori, F. (2021). Machine learning Kou, G., Lu, Y., Peng, Y., & Shi, Y. (2012). Evaluation of classification algorithms
for industrial applications: A comprehensive literature review. Expert Systems with using MCDM and rank correlation. International Journal of Information Technology
Applications, 175, Article 114820. and Decision Making, 11(01), 197–225.
16
Kraljic, P. (1983). Purchasing must become supply management. Harvard Business Rahayu, S., & Ardi, R. (2020). A proposed supplier segmentation criteria in Indonesian
Review, 61(5), 109–117. manufacturing industry. In Proceedings of the 3rd Asia Pacific conference on research
Krishnadevarajan, P. K., Ravichandran, V., Balasubramanian, S., & Kannan, N. (2015). in industrial and systems engineering 2020 (pp. 284–289). Depok Indonesia: ACM.
Supplier management: A Framework for selection, evaluation and performance. Restrepo, R., & Villegas, J. G. (2019). Supplier evaluation and classification in
International Journal of Management, 6(9), 16–28. a Colombian motorcycle assembly company using data envelopment analysis.
Lee, H. H., & Li, C. (2018). Supplier quality management: Investment, inspection, and Academia Revista Latinoamericana de Administración, 32(2), 159–180.
incentives. Production and Operations Management, 27(2), 304–322. http://dx.doi. Rezaei, J., & Fallah Lajimi, H. (2019). Segmenting supplies and suppliers: Bring-
org/10.1111/poms.12802. ing together the purchasing portfolio matrix and the supplier potential matrix.
Li, Q., & Liu, A. (2019). Big data driven supply chain management. Procedia CIRP, 81, International Journal of Logistics Research and Applications, 22(4), 419–436.
1089–1094. Rezaei, J., & Ortt, R. (2012). A multi-variable approach to supplier segmentation.
Lima-Junior, F. R., & Carpinetti, L. C. R. (2016). Combining SCOR® model and fuzzy International Journal of Production Research, 50(16), 4593–4611.
TOPSIS for supplier evaluation and management. International Journal of Production Rezaei, J., & Ortt, R. (2013a). Multi-criteria supplier segmentation using a fuzzy
Economics, 174, 128–141. preference relations based AHP. European Journal of Operational Research, 225(1),
Lo, H. W., Shiue, W., Liou, J. J. H., & Tzeng, G. H. (2020). A hybrid MCDM-based FMEA 75–84.
model for identification of critical failure modes in manufacturing. Soft Computing, Rezaei, J., & Ortt, R. (2013b). Supplier segmentation using fuzzy logic. Industrial
24(20), 15733–15745. Marketing Management, 42(4), 507–517.
Mehmood, T., & Khodabandeh, F. (2017). Ranking suppliers by using sustainable supplier Rezaei, J., Wang, J., & Tavasszy, L. (2015). Linking supplier development to supplier
evaluation criteria (SSEC) and multi criteria decision making method (MCDM) (Ph.D. segmentation using best worst method. Expert Systems with Applications, 42(23),
thesis). 9152–9164.
Mohammadi, M., & Rezaei, J. (2020). Ensemble ranking: Aggregation of rankings Rui, H., & Lai, G. (2015). Sourcing with deferred payment and inspection under supplier
produced by different multi-criteria decision-making methods. Omega, 96, Article product adulteration risk. Production and Operations Management, 24(6), 934–946.
102254. http://dx.doi.org/10.1111/poms.12313.
Monczka, R. M., Handfield, R. B., Giunipero, L. C., & Patterson, J. L. (Eds.), (2009). Santos, L. F. d. O. M., Osiro, L., & Lima, R. H. P. (2017). A model based on 2-tuple fuzzy
Purchasing and supply chain management, (4th ed.). Cengage Learning. linguistic representation and analytic hierarchy process for supplier segmentation
Nguyen, T., Zhou, L., Spiegler, V., Ieromonachou, P., & Lin, Y. (2018). Big data analytics using qualitative and quantitative criteria. Expert Systems with Applications, 79,
in supply chain management: A state-of-the-art literature review. Computers & 53–64.
Operations Research, 98, 254–264. Teli, S. N., Majali, V. S., Bhushi, U. M., Gaikwad, L. M., & Surange, V. G. (2013).
Nielsen, M. A. (2015). Neural networks and deep learning, vol. 25. CA, USA: Cost of poor quality analysis for automobile industry: A case study. Journal of the
Determination press San Francisco. Institution of Engineers (India): Series C, 94(4), 373–384.
Noshad, K., & Awasthi, A. (2015). Supplier quality development: A review of literature Velasquez, M., & Hester, P. T. (2013). An analysis of multi-criteria decision making
and industry practices. International Journal of Production Research, 53(2), 466–487. methods. International Journal of Operations Research, 10(2), 56–66.
Opricovic, S., & Tzeng, G.-H. (2004). Compromise solution by MCDM methods: Wang, Q., Ma, Y., Zhao, K., & Tian, Y. (2020). A comprehensive survey of loss functions
A comparative analysis of VIKOR and TOPSIS. European Journal of Operational in machine learning. Annals of Data Science, 1–26.
Research, 156(2), 445–455. http://dx.doi.org/10.1016/S0377-2217(03)00020-1. Wang, P., Zhu, Z., & Wang, Y. (2016). A novel hybrid MCDM model combining the
Osiro, L., Lima-Junior, F. R., & Carpinetti, L. C. R. (2014). A fuzzy logic approach to SAW, TOPSIS and GRA methods based on experimental design. Information Sciences,
supplier evaluation for development. International Journal of Production Economics, 345, 27–45.
153, 95–112. Zavadskas, E. K., Govindan, K., Antucheviciene, J., & Turskis, Z. (2016). Hybrid multiple
Peng, Y. (2015). Regional earthquake vulnerability assessment using a combination of criteria decision-making methods: A review of applications for sustainability issues.
MCDM methods. Annals of Operations Research, 234(1), 95–110. Economic Research-Ekonomska Istraživanja, 29(1), 857–887.
Petrović, G. S., Madić, M., & Antucheviciene, J. (2018). An approach for robust decision Zyoud, S. H., & Fuchs-Hanusch, D. (2017). A bibliometric-based survey on AHP and
making rule generation: Solving transport and logistics decision making problems. TOPSIS techniques. Expert Systems with Applications, 78, 158–181. http://dx.doi.
Expert Systems with Applications, 106, 263–276. org/10.1016/j.eswa.2017.02.016.
17

A Decision Support System Based On IA

Uploaded by

Copyright:

Available Formats

A Decision Support System Based On IA

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

A Decision Support System Based On IA

Uploaded by

Copyright:

Available Formats

Expert Systems With Applications 242 (2024) 122746

Contents lists available at ScienceDirect

Expert Systems With Applications

A decision support system for supplier quality evaluation based on

ARTICLE INFO ABSTRACT

Fig. 2. Measurement flowchart of SQC dimension.

• PROMETHEE (Preference Ranking Organization Method for En-

form of Eq. (13), where 𝛼𝑘 > 0 is the HQ auxiliary variable. 𝜓(⋅)

The optimal aggregated evaluation of the item–supplier pairs will be,

iv Gradient descent Gradient descent Gradient descent HQ minimization

We use an application case to illustrate the proposed DSS and

You might also like