sustainability

Article
SmartISM: Implementation and Assessment of Interpretive
Structural Modeling
Naim Ahmad * and Ayman Qahmash

Department of Information Systems, King Khalid University, Abha 62529, Saudi Arabia; [email protected]
* Correspondence: [email protected]

Abstract: Interpretive Structural Modeling (ISM) is a technique to establish the interrelationships

between elements of interest in a specific domain through experts’ knowledge of the context of
the elements. This technique has been applied in numerous domains and the list continues to
grow due to its simplistic concept, while sustainability has taken the lead. The partially automated
or manual application of this technique has been prone to errors as witnessed in the literature
due to a series of mathematical steps of higher-order computing complexity. Therefore, this work
proposes to develop an end-to-end graphical software, SmartISM, to implement ISM technique
and MICMAC (Matrice d’Impacts Croisés Multiplication Appliquée á un Classement (cross-impact
matrix multiplication applied to classification)), generally applied along with ISM to classify variables.
Further, a scoping review has been conducted to study the applications of ISM in the previous studies
using Denyer and Tranfield’s (2009) framework and newly developed SmartISM. For the development
of SmartISM, Microsoft Excel software has been used, and relevant algorithms and VBA (Visual
Basic for Applications) functions have been illustrated. For the transitivity calculation the Warshall





 algorithm has been used and a new algorithm reduced conical matrix has been introduced to remove
edges while retaining the reachability of variables and structure of digraph in the final model.
Citation: Ahmad, N.; Qahmash, A.
The scoping review results demonstrate 21 different domains such as sustainability, supply chain
SmartISM: Implementation and
and logistics, information technology, energy, human resource, marketing, and operations among
Assessment of Interpretive Structural
Modeling. Sustainability 2021, 13, others; numerous types of constructs such as enablers, barriers, critical success factors, strategies,
8801. https://doi.org/10.3390/ practices, among others, and their numbers varied from 5 to 32; number of decision makers ranged
su13168801 between 2 to 120 with a median value of 11, and belong to academia, industry, and/or government;
and usage of multiple techniques of discourse and survey for decision making and data collection.
Academic Editors: Dragan Pamucar Furthermore, the SmartISM reproduced results show that only 29 out of 77 studies selected have a
and Marc A. Rosen correct application of ISM after discounting the generalized transitivity incorporation. The outcome of
this work will help in more informed applications of this technique in newer domains and utilization
Received: 5 June 2021
of SmartISM to efficiently model the interrelationships among variables.
Accepted: 4 August 2021
Published: 6 August 2021
Keywords: interpretive structural modeling (ISM); MICMAC (Matrice d’Impacts Croisés Multipli-
cation Appliquée á un Classement (cross-impact matrix multiplication applied to classification));
Publisher’s Note: MDPI stays neutral
Microsoft Excel; SmartISM; VBA (Visual Basic for Applications); reduced conical matrix
with regard to jurisdictional claims in
published maps and institutional affil-
iations.

1. Introduction
Every discipline is expanding its frontier and multiple disciplinary approaches have
Copyright: © 2021 by the authors.
become essential to solve complex problems. This leads to the study of a large number
Licensee MDPI, Basel, Switzerland.
of constructs of interests simultaneously. These constructs may have been identified in
This article is an open access article
theory or practice. Warfield [1–4] in the 1970’s developed a technique to establish an
distributed under the terms and interrelationship model between variables known as interpretive structural modeling
conditions of the Creative Commons (ISM). The holistic picture of important constructs in the structured form derived from ISM
Attribution (CC BY) license (https:// technique helps the practitioners to solve the problem effectively. This technique is widely
creativecommons.org/licenses/by/ used due to its simplistic procedure and profound value addition in problem solving in
4.0/). different domains.

Sustainability 2021, 13, 8801. https://doi.org/10.3390/su13168801 https://www.mdpi.com/journal/sustainability

Sustainability 2021, 13, 8801 2 of 27

ISM helps in representing partial, fragmented, and distributed knowledge into in-
tegrated, interactive, and actionable knowledge. This technique is therefore particularly
useful for the areas that are inherently multidisciplinary, such as sustainability. The dis-
cipline of sustainability ensures the performance in three areas: economic, social, and
environmental, termed as triple bottom line (TBL) [5], while the world undergoes devel-
opment. Additionally, the literature shows the maximum number of applications of this
technique in the area of sustainability.
The search with the quoted keywords of “interpretive structural modeling” on the
single database of Scopus yielded 5184 documents. There is an exponential growth in the
usage of this technique from 2007 onward; prior to this year articles are around 10 each
year starting from 1974. For the year 2007, 46 documents are listed and the numbers are
exponentially increasing each successive year to 1200 documents in the year of 2020. With
around 36% contribution in articles, India is leading the application of ISM, followed by
China, USA, UK, and Iran. Together these five countries contribute around 71% of total
articles. This technique is being used in many disciplines in decreasing order, namely
business, engineering, computer science, decision science, environmental science, social
science, and others.
ISM helps in modeling the variables and brings out the existing interrelationship
structure among them. It helps a group of people or decision makers to debate and share
their knowledge and achieve consensus on the relationships among the variables. The
participants can share their views without any knowledge of mathematical complexity
involved in the underlying steps. A computerized program may automate all the graphical
and algebraic computation and convert their inputs into a pictorial model consisting of
variables along with the relationships among them. The ISM process does not add any
information [6] but brings in structural value [7].
In the same time period of the 1970’s, another technique known as MICMAC (Matrice
d’Impacts Croisés Multiplication Appliquée á un Classement (cross-impact matrix mul-
tiplication applied to classification)) was developed by J. C. Duperrin and M. Godet [8].
MICMAC helps in classification of the variables into one of the four categories, namely
dependent, independent, linkage, and autonomous variables. ISM coupled with MIC-
MAC becomes a strong tool to visualize the structure of variables along with the inter-
relationships between them. ISM is also used in several multi criteria decision making
(MCDM) techniques such as analytical hierarchy process (AHP) [9], analytic network
process (ANP) [10–12], technique for order of preference by similarity to ideal solution (TOP-
SIS) [13,14], decision-making trial, evaluation laboratory (DEMATEL) [15,16], and others.
Implementation of this technique and conduction of brainstorming sessions with
experts in previous studies [17,18] led to identification of some key challenges such as
variables’ identification, selection of decision makers and method of decision making, and
unavailability of end-to-end software for ISM and MICMAC. Furthermore, the literature
shows erroneous applications of steps of ISM such as wrong reachability matrix [19–22],
wrong transitivity calculations [9,13,16,23–37], incorrect level partitioning and wrong
structure of the model [31,38–42], and incorrect addition [11,14,43–48] or reduction [49–52]
of edges affecting the reachability of variables. An error in an earlier step generally leads to
an error in subsequent steps. Similarly, the wrong calculation of transitivity leads to wrong
MICMAC diagrams. Therefore, there exists some important issues in implementation of
this technique, namely identification of variables, decision makers, expertise and experience
of decision makers, method of decision making, and computerization of the steps of ISM.
Previous ISM reviews [53–55] don’t critically analyze the steps of ISM applications in the
articles. Similarly, although some automation of the ISM technique has been provided
earlier [56,57], there does not exist any end-to-end graphical software that may help in
applying this technique and allow the decision makers to focus on sharing knowledge and
iterate the ISM technique until a high-confidence consensus model is arrived at. These
challenges set the objectives of this research as follows:
Sustainability 2021, 13, 8801 3 of 27

• Development of SmartISM, a software tool for ISM and MICMAC using Microsoft
Excel and VBA.
• Scoping review of applications of ISM on existing studies to identify application
domains, types and numbers of variables studied, composition of decision makers,
decision making and data collection techniques, and accuracy of ISM application
using SmartISM.
The remainder of the paper is organized into the following sections: literature review,
research methodology, development of SmartISM using Microsoft Excel, results, discussion,
and conclusion.

2. Literature Review
There are numerous studies in the literature that illustrate the ISM technique. They can
be summarized into the seven steps approach with an additional eighth step for MICMAC
analysis, as given in the following subsection. The next subsection illustrates the existing
available automation of the ISM. The last subsection presents some studies that have
reviewed the implementations of ISM.

2.1. ISM and MICMAC Techniques

The interpretive structural modelling (ISM) can be defined as constructs’ directional
structuring technique based on contextual interrelationships defined by domain experts,
utilizing computerized conversion of relations into a pictorial model using matrix algebra
and graph theory. It may be explained in the series of steps as follows, which will assist in
automating all the processes of the ISM technique.

2.1.1. Elements or Constructs or Variables

Identification of elements or constructs of the subject being studied is the most im-
portant of all activities. Similarly, the establishment of their definition along with the
theoretical boundaries or scope is very critical. Elements must be explained with the
details of their definition, objectives, and possible indications or measurements. These
elements are generally identified by literature review, expert opinions, and/or surveys.
Some of the unique approaches have been use of thematic analysis [58], upper echelon
theory [11], contingency theory [59], content analysis [52], strengths, weaknesses, opportu-
nities, and threats (SWOT) analysis [30], idea engineering workshop [40,60], and Delphi
technique [37,61]. One study [42] has defined the source, understanding, and interpretation
for each variable.

2.1.2. Decision Makers (DMs)

DMs play a very significant role in ISM as the whole process and outcome are depen-
dent upon their input. There are three important aspects for the selection of a group of
DMs such as size, expertise, and diversity. The group of DMs should be representative
of all of the stakeholders in the domain of the problem. They should have sound expe-
rience of domain and expert level knowledge of variables being studied. The literature
shows the number of DMs ranging between 2 [62,63] to 120 [64] with a median value
of 11, and very few studies [16,30,41,65] have taken DMs from academia, industry, and
government together.

2.1.3. Structural Self-Interaction Matrix (SSIM)

Elements or constructs are interrelated with one of the four relations such as x influ-
ences y, y influences x, x and y mutually influence each other, or x and y are unrelated.
These relations are almost universally represented by ‘V’, ‘A’, ‘X’, and ‘O’ characters re-
spectively in the SSIM. These relationships are assigned by DMs based on contextual
relationships during pairwise comparison on variables. The number of comparisons is nC2
(mathematical combination), where n is the number of variables in the domain of study.
Finally, an n by n matrix is formed with nC2 cells filled with A, V, X, and O symbols and
Sustainability 2021, 13, 8801 4 of 27

the remaining cells are blank. Most studies have used these standard symbols except few
such as [35]. As this is the basic matrix and required for all other steps therefore has been
documented in most of the studies except few such as [15,66].

2.1.4. Reachability Matrix (RM) and Final Reachability Matrix (FRM)

RM is the representation of SSIM in binary form. V, A, X, and O symbols of SSIM are
replaced with 1, 0, 1, and 0, digits respectively. At their transposed positions by row with
column and column with row, 0, 1, 1, and 0 digits are placed, respectively. The constructs
are assumed to influence self, so ones are placed at the diagonal positions. The resultant
RM is checked for transitive relations. Transitivity is the basic assumption in the ISM such
as if variable x influences y and y influences z then x will influence z transitively. This is
second-order or two-hop transitivity whereas generalized transitivity means x is related to
z through one or more variables. The transitive relations hence identified are represented
in the RM with 1*s to distinguish from original 1s and the resulting matrix is known as
FRM. FRM also consists of driving and dependence powers of each variable by counting
1s and 1*s in rows and columns respectively. Very few studies mention usage of some
software for transitivity calculations such as [56,57]. However, one of the most frequent
reasons for incorrect ISM calculations have been wrong transitivity calculations, such as in
studies [9,13,16,23–37]. Therefore, this study proposes the use of an established Warshall
algorithm [67] for transitivity calculations.

2.1.5. Level Partitioning

This is a very important step to develop the hierarchical directional structure among
the variables. Reachability, antecedent, and intersection sets are derived for all the variables
from the FRM. For a specific variable, a reachability set consists of itself and all the variables
it influences, and an antecedent set consists of itself and all the variables influencing it.
Thereafter, the intersection set of reachability and antecedent set is calculated. Variables
having the same reachability and intersection sets are given the top rank and are removed
for the next iteration and the process is repeated until all variables are ranked. Some studies
such as [31,38–42] in the literature had incorrect leveling for variables.

2.1.6. Conical Matrix (CM) and Digraph

CM is row and column wise ordered FRM based on ranks or levels of variables
identified in the level partitioning step. Further the levels of each variable are also recorded
at the end of row and column in CM. This matrix helps in drawing the digraph to get
the first visual output of the hierarchical directional structure of variables. Circular nodes
are drawn with variable numbers. Further they are connected with directional edges
based upon 1s or 1*s in the CM between pairs of variables. Fewer studies have mentioned
CM and digraph [12,20,27,35,65,68–70], as the digraph resembles the final model with a
lesser number of edges. The importance of the digraph further goes down in automatic
calculation of transitivity.

2.1.7. Reduced Conical Matrix (RCM) and Final ISM Model

Digraph is converted into a final model by replacing the node numbers with names
of the variables and representing nodes in the rectangular shapes. Moreover, efforts are
made to remove maximum edges from digraph while maintaining the levels and structure
of variables and reachability of variables. This is done to improve the readability of the
final model. Several studies have committed mistakes at this step either by adding extra
edges [11,14,43–48] or omitting edges [49–52] that have affected the reachability of the
variables. Therefore, a new algorithm, reduced conical matrix (RCM), has been devised to
remove maximum possible edges without affecting structure and reachability of variables,
as explained in the fourth section. This RCM is used for making the final ISM model. The
final model may further be subjected to validations by different means such as review by
DMs, interviews from different sets of participants, or statistical validations.
Sustainability 2021, 13, 8801 5 of 27

2.1.8. MICMAC
MICMAC (Matrice d’Impacts Croisés Multiplication Appliquée á un Classement
(cross-impact matrix multiplication applied to classification)) in the simplest terms is a
variable classification technique. Variables are mapped onto a two-dimensional grid based
on their dependence and driving power values, represented on horizontal and vertical
axes respectively. The range of these values is between 1 and total number of variables and
the axes are bifurcated at mid-points, resulting in four quadrants numbered anti clockwise.
These quadrants classify variables into autonomous, dependent, linkage, and independent
categories. The autonomous variables are not connected with the remaining system of
variables whereas linkage variables are sensitive and strongly connected with independent
and dependent variables. The final hierarchical ISM model coupled with the MICMAC
analysis greatly improves the understanding of variables. Therefore, most studies have
carried out MICMAC analysis except few such as [19,39,47,71,72].

2.2. Implementation of ISM

As originally proposed by Warfield [1–4], the ISM requires its steps to be executed with
the assistance of a computer [6]. Some of the more recent studies demonstrate specialized
software or routines being developed for ISM. The article [56] mentions the development
of the ISM software package in R software. This software package takes the SSIM input in
the comma separated (.csv) excel file and provides two outputs in excel file format, namely,
“ISM_Matrix” for FRM step to incorporate transitivity calculations and “ISM_Output”
for partitioning step to identify the levels of the variables. Similarly, some studies such
as [57] have used MATLAB software to calculate the FRM and partitioning steps. The
previous studies have attempted to automate FRM and partitioning steps, leading to
partial automation of ISM. As pointed out earlier in absence of automation, the final model
may introduce errors in edges regardless of correct FRM and leveling, leading to wrong
reachability of variables. Further, having all the steps being carried out automatically
shows the prompt results to researchers and decision makers for further possible iterations.
Therefore, there exists a need to develop an end-to-end graphical software to implement
ISM and MICMAC and identify the required algorithms for it.

2.3. Assessment of ISM Applications

The ISM technique is being applied in a range of domains [53–55]. The review arti-
cle [54] provides 10 different application domains for ISM. It further provides additional
parameters such as integration with other MCDM approaches. Similarly the review ar-
ticle [55] identified ISM applications in 14 domains without industry or organizations,
20 industrial sectors, and 4 other areas. Furthermore, among other characteristics, it men-
tions integration with other MCDM approaches, and the presence of constructs for cost
and/or quality. These reviews haven’t focused on operationalization of ISM technique.
Therefore, there exists a gap to identify the methodology of steps of applications of ISM in
the existing articles such as nature and number of variables, compositions of DMs, decision
making and data collection techniques, and accuracy of ISM results.

3. Research Methodology
This study addresses two objectives, firstly the development of SmartISM for the
implementation of ISM, as explained in the following section. The second objective is the
scoping review of literature to identify the scope of ISM and MICMAC applications and
the assessment of applications of ISM using SmartISM tool. For the scoping review the
five-step framework of Denyer and Tranfield (2009) [73] has been adopted as explained in
the following paragraphs, Figure 1. The review process also generated the data necessary
for the assessment of application of ISM using SmartISM.
 plementation  of  ISM, as  explained  in  the  following section.  The  second  objective  is  the 
scoping review of literature to identify the scope of ISM and MICMAC applications and 
the assessment of applications of ISM using SmartISM tool. For the scoping review the 
five‐step framework of Denyer and Tranfield (2009) [73] has been adopted as explained 
Sustainability 2021, 13, 8801 6 of 27
in the following paragraphs, Figure 1. The review process also generated the data neces‐
sary for the assessment of application of ISM using SmartISM. 

 
Figure 1. Research Methodology.
Figure 1. Research Methodology. 

Step I: Question formulation: Formulating questions requires identification of context,

Step I: Question formulation: Formulating questions requires identification of con‐
interventions, mechanisms, and outcomes. In this study, context is considered to be
text, interventions, mechanisms, and outcomes. In this study, context is considered to be 
domain, and decision makers; interventions are variables of interest in problem domain;
domain, and decision makers; interventions are variables of interest in problem domain; 
mechanisms are techniques for data collection from DMs; and outcomes are the ISM
mechanisms are techniques for data collection from DMs; and outcomes are the ISM out‐
outputs and MICMAC diagram. In essence there will be the following research questions
puts and MICMAC diagram. In essence there will be the following research questions that 
that will help in addressing the second objective of this study.
will help in addressing the second objective of this study. 
• What are the different domains and sub-domains of ISM applications?
• What are the different types and numbers of variables being studied?
• What are the compositions of DMs in different studies?
• What are the different decision making and data collection techniques?
• How accurate has been the application of ISM technique, using SmartISM?
Step II: Locating Studies: As the ISM based studies are huge, only quality sources
were considered, rather than an exhaustive search. As per the objectives, articles that had
significant discussion and documentation on ISM application as mentioned in Step I were
needed. As defined earlier, the steps of ISM are structural self-interaction matrix, initial
and final reachability matrix, level partitioning, conical matrix, digraph, and final model.
It was observed that an article going into the details of level partitioning had sufficient
demonstration of ISM. Therefore, “Interpretive Structural Modeling” + partitioning key-
words were used on ScienceDirect database and it resulted in 300 articles up to the year of
2021, of which 291 belonged to review and research articles.
Sustainability 2021, 13, 8801 7 of 27

Step III: Study selection and evaluation: These articles were further perused for the
relevance to present study and classified into different groups such as definition only,
other techniques, no partitioning, non-related, incomplete outputs, and desired study.
As the articles were growing nonlinearly each year, therefore, after the year of 2017, a
random selection of five articles was preferred to keep the dataset manageable. It resulted
in 77 articles in the desired study group that were considered further in this study.
Step IV: Analysis and Synthesis: This step has two components: first the analysis of
articles for context, interventions, and mechanisms was performed, as explained in step
one, by extracting relevant information as shown in Table 2. Second was the extraction of
SSIM from the 77 selected articles to reimplement the ISM technique using SmartISM. The
results from the SmartISM were compared with the outcomes illustrated in the article for
SSIM, RM, FRM, LP, CM, digraph, final model and edges in the final model, and MICMAC
and the variations are summarized in Table 3.
Step V: Reporting and using the results: Results of analysis and synthesis are reported
in results and discussion section. They have been provided in a fashion that will assist in
informed-adoption and application of ISM and MICMAC, and utilization of SmartISM for
academicians, practitioners, and policy makers alike.

Articles’ Details
Articles’ publication years range from 2005 to 2021. As the articles are increasing
non-linearly, therefore 2017 onwards only five articles were randomly chosen for each
year. The publication sources having two or more articles have been shown in Table 1.
Journals in the area of sustainability have the maximum number of articles. Journal of
cleaner production had published 18 articles out of 77 selected articles.

Table 1. Publication sources for two or more articles.

Publication Title 2005 2007 2008 2009 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 Total
Journal of Cleaner Production 2 2 5 3 1 3 1 1 18
Renewable & Sustainable Energy Reviews 1 1 1 2 5
Resources, Conservation and Recycling 1 1 1 1 1 5
Resources Policy 3 1 4
Procedia Social and Behavioral Sciences 3 1 4
Procedia Engineering 1 1 2
Journal of Environmental Management 1 1 2
Computers & Industrial Engineering 1 1 2
Telematics and Informatics 1 1 2
International Journal of Production Economics 1 1 2
Sustainable production and consumption 1 1 2
Technological Forecasting and Social Change 1 1 2
Miscellaneous Sources with 1 article 1 2 1 1 3 5 1 3 1 2 3 4 27
Total 1 1 2 1 2 1 5 9 11 10 14 5 5 5 5 77

4. Development of SmartISM Using Microsoft Excel

Microsoft Excel provides an excellent environment for graphical representation and
modelling of virtually any conceptual framework of any discipline. It has some important
features such as cellular addressable input sheets, interactive output, vector graphic objects,
integral atomic access of data in multiple ways, many inbuilt data processing functions,
backend VBA (Visual Basic for Applications) interface to code any logic or algorithm,
mechanisms for development of event driven interfaces, ubiquitous tool and ease of use,
and widespread ecosystem of support and training. Hence it makes a natural choice for
practitioners, decision makers, and researchers to develop their problem-solving models in
Microsoft Excel. Its applications in business statistics and decision making are widely docu-
mented. Following are some advanced applications of MS Excel in different domains such
as Genetic Analysis [74], Finite Element Analysis in Engineering [75], and Pharmacokinetic
Pharmacodynamic fields of Pharmacology [76]. On the flip side it has a drawback to sup-
port multiple real-time concurrent users. This section explains the functions and features
of VBA to develop SmartISM, an end-to-end graphical software to automate processes of
Sustainability 2021, 13, 8801 8 of 27

ISM and MICMAC with the help of pseudo codes. Additionally, the demonstration video
for SmartISM has been attached as a Supplementary Material, see Video S1.
Firstly, the SSIM matrix defined by DMs is entered in Excel, and serves as the basic
input for other steps of ISM. For n variables, the size of SSIM will be n by n. DMs will
compare n(n + 1)/2 or n C2 unique pairs of variables and assign one of the relationships
using symbols V, A, X, or O, as explained earlier. Thereafter, eight VBA macros will derive
matrices of RM, FRM, CM, and RCM; level partitioning; and draw diagrams of digraph,
final model, and MICMAC. RM is a binary form of SSIM using conversion rules for V, A,
X, and O as explained earlier and keeping 1s at the diagonal positions of the matrix, as
described in the following pseudo code. RM also contains the driving and dependence
powers for each variable.
Function RM
//copy the content of SSIM into RM
RM ← SSIM
//loops to replace V, A, X and O symbols with 1, 0, 1 and 0, digits; and putting 0,1, 1 //and 0
digits at their transposed positions; and keeping the diagonal elements as 1
For i = 1 To n //n is the total number of elements
For j = 1 To n
If i = j
RM[i][j] ← 1
If RM[i][j] = ‘V’
RM[i][j] ← 1
RM[j][i] ← 0
If RM[i][j] = ‘A’
RM[i][j] ← 0
RM[j][i] ← 1
If RM[i][j] = ‘X’
RM[i][j] ← 1
RM[j][i] ← 1
If RM[i][j] = ‘O’
RM[i][j] ← 0
RM[j][i] ← 0
//count non-zero elements in rows and columns and append to show the driving and
//dependence powers
For i = 1 To n
RM[i][n + 1] ← Countif(RM[i][] != 0)
RM[n + 1][i] ← Countif(RM[][i] != 0)

The second function FRM requires calculation of transitive relations among variables.
For manual calculation, RM can be visualized as a digraph with variables representing
nodes and 1s in the RM representing the directed edges. By tracing different paths, tran-
sitive relations can be identified. For a large number of variables the process would be
tedious and leads to errors, whereas a simple Warshall algorithm [67] for transitive closure
can be used to automate it. This algorithm results in generalized transitivity if applied
in-place, otherwise it will give second-order or two-hop transitivity. Transitive relations
are marked with 1* in FRM, see the pseudo code for main logic in the following paragraph.
Moreover, the 1s and 1*s are counted in rows and columns to calculate the driving and
dependence powers respectively for each variable.
Sustainability 2021, 13, 8801 9 of 27

Function FRM
//copy the content of RM into FRM
FRM ← RM
//block for generalized transitivity (all levels) Warshall algorithm in-place
//start three level nested loop to parse through FRM
For k = 1 To n //n is the total number of elements
For i = 1 To n
For j = 1 To n
If FRM[i][k] = 1 And FRM[k][j] = 1
FRM[i][j] ← 1
//putting 1* to differentiate between transitive links identified and links in RM
For i = 1 To n
For j = 1 To n
If FRM[i][j] != RM[i][j]
FRM [i][j] ← *1
//block for second-order transitivity (up to second level only) Warshall algorithm
//start three level nested loop to parse through FRM
For k = 1 To n
For i = 1 To n
For j = 1 To n
If RM[i][k] = 1 And RM[k][j] = 1 And FRM[i][j] = 0
FRM[i][j] ← 1*
//recount non-zero elements in rows and columns and append to show the driving and
//dependence powers
For i = 1 To n
RM[i][n + 1] ← Countif(RM[i][] != 0)
RM[n + 1][i] ← Countif(RM[][i] != 0)

The next step is to calculate the ranks of the variables through level partitioning.
A new matrix LP is defined with five columns namely elements (Mi), reachability set
R(Mi), antecedent set A(Mi), intersection set R(Mi)∩A(Mi) and level, and n rows. For a
specific variable Mi in FRM, non-zero cells in the row comprise its reachability set and
their corresponding identifiers are kept in the LP row of the same variable Mi. Similarly,
non-zero cells in the column comprise its antecedent set and their corresponding identifiers
are kept in the LP row of the same variable Mi. The intersection sets are calculated for
all variables and variables having the same reachability and intersection sets are given
first rank. In the next iteration, identifiers of all the ranked variables are removed from
reachability, antecedent, and intersection sets. Again, variables having the same reachability
and intersection sets are given the second rank and iteration continues until all the variables
are ranked. The iteration results may be copied in one Microsoft Excel Sheet.
Function LP
//initiate a matrix LP of size n by 5 to keep element number, reachability set, antecedent //set,
intersection set and levels for each of the n elements; levels will remain empty
For i = 1 To n
LP[i][1] ← i
//reachability set R for ith element
For j = 1 To n
If FRM[i][j] != 0
Append jth element to R
LP[i][2] ← R
//antecedent set A for ith element
For j = 1 To n
If FRM[j][i] != 0
Append jth element to A
LP[i][3] ← A
LP[i][4] ← R∩A
Sustainability 2021, 13, 8801 10 of 27

//iteration for level calculations, where levels j can go up to n

For j = 1 To n
//remove elements that have levels
For i = 1 To n
If LP[i][5] != Null
For k = 1 To n
Remove ith element from LP[k][2], LP[k][3], LP[k][4]
//assign jth level to elements that have equal reachability and intersection sets
For i = 1 To n
If LP[i][2] = LP[i][4] And LP[i][5] = Null
LP[i][5] ← j
//print the jth iteration results
Print LP

Once the variables are ranked, a digraph can be developed easily by positioning the
variables as per their ranks with the help of CM. CM is row and column wise sorted FRM
as per variables’ ranks or levels. Directed edges can be drawn between variables as per
non-zero cells in the CM. Two shape objects Oval and Connector are needed to automate
the drawing of digraph. Positing of ovals needs to be carefully assigned, as there can be
multiple ovals in one level. The simplest way to identify the needed objects in drawing
is to auto record a macro and draw a sample. Afterwards, the macro can be manually
edited and static names of the objects can be made dynamic for easy handling in the loop
structures of VBA. The pseudo codes for the functions for CM and digraph is as follows.
Function CM
//copy the content of FRM into CM
CM ← FRM
//add levels from LP to CM for each element at the end of rows and columns
For i = 1 To n //n is the total number of elements
CM[i][n+2] ← LP[i][5]
CM[n+2][i] ← LP[i][5]
CM.Sort key1: = Range(LP[n + 2][])
CM.Sort key1: = Range(LP[][n + 2]), Orientation: = xlLeftToRight
Function Digraph
//create ovals of size s to represent numbered nodes for each element
For i = 1 To n //n is the total number of elements
Ovals[i] ← Shapes.AddShape(msoShapeOval, 0, 0, s, s)
Ovals[i].TextFrame.Characters.Text ← i
//define and calculate the position arrays v and h of each rectangle based on
//drawing canvas size, required interspacing between elements as per number of
//elements, elements in each level and any offset needed
Ovals[i].Top ← v[i]
Ovals[i].Left ← h[i]
//add directed arrows between elements based on edges in CM
For i = 1 To n
For j = 1 To n
If CM[i][j] != 0 And i != j
Shapes.Range(Ovals[i], Ovals[j]).Select
Shapes.AddConnector(msoConnectorStraight, 0, 0, 0, 0).Select
Selection.ShapeRange.Line.EndArrowheadStyle ← msoArrowheadOpen
Selection.ShapeRange.ConnectorFormat.BeginConnect Ovals[i], 1
Selection.ShapeRange.ConnectorFormat.EndConnect Ovals[j], 5

The final model represents variable names in the rectangular boxes in place of their
identifiers in ovals and tries to remove maximum possible transitive links from the digraph.
Transitive reduction is a technique to reduce the number of transitive links. Transitive
reduction is complicated, specifically for the directed cyclic graphs, and the algorithm
may even distort the structure of the digraph. Therefore, an algorithm was designed to
develop a reduced conical matrix (RCM) that removes maximum links without changing
Sustainability 2021, 13, 8801 11 of 27

the structure of digraph and reachability of elements. The main logic is to remove incoming
links from second lower-level variables from the CM and results in RCM, see the pseudo
code for the main logic in the following paragraph. RCM was used to draw automated final
ISM model using Rectangle and Connector shape objects, as in the following pseudo code.
Function RCM
//copy the content of CM into RCM
RCM ← CM
//start loop to parse through columns
For i = 1 To n //n is the total number of elements
//start loop to parse through row of specific column for lower triangular matrix
For j = i To n
//search for first non-zero row cell whose level is greater than the level of that
//column element
If (RCM[j + 1][i] = 1 Or RCM[j + 1][i] = “1*”) And RCM[j + 1][n +2] > RCM[i][n + 2]
//set the L one higher than the level identified
L ← RCM[j + 1] [n + 2] + 1
Break For
//identify the row that has level equal to L
For j = i To n
If RCM[j][n + 2] = L
Break For
//set all the rows starting from identified row in preceding step and below up to n
//as 0
For j = j To n
RCM [j][i] ← 0
Function FinalModel
//create rectangles of size s to represent each element with variable text kept in names //array
For i = 1 To n //n is the total number of elements
Rects[i] ← Shapes.AddShape(msoShapeRectangle, 0, 0, s, s)
Rects[i].TextFrame.Characters.Text ← names[i]
//define and calculate the position arrays v and h of each rectangle based on
//drawing canvas size, required interspacing between elements as per number of
//elements, elements in each level and any offset needed
Rects [i].Top ← v[i]
Rects [i].Left ← h[i]
//add directed arrows between elements based on edges in RCM
For i = 1 To n
For j = 1 To n
If RCM[i][j] != 0 And i != j
Shapes.Range(Rects [i], Rects [j]).Select
Shapes.AddConnector(msoConnectorStraight, 0, 0, 0, 0).Select
Selection.ShapeRange.Line.EndArrowheadStyle ← msoArrowheadOpen
Selection.ShapeRange.ConnectorFormat.BeginConnect Rects [i], 1
Selection.ShapeRange.ConnectorFormat.EndConnect Rects [j], 3
Lastly, a macro was written to draw a MICMAC diagram. The basic input for this
diagram was the dependence and driving powers of variables from FRM. This was the
longest macro as it required many shape objects such as Line, Connector, Rectangle, Oval,
and Textbox. However, it didn’t require any special algorithm to be used. Nevertheless,
logic to initiate, aggregate, and draw different objects based on number of variables, and
dependence and driving powers in a specified space, required careful arrangement.
Sustainability 2021, 13, 8801 12 of 27

Function MICMAC
//draw n + 1 horizontal and vertical lines where n is total number of elements spaced at
//s as per canvas size and number of elements, offset has been skipped for simplification
//and add numbered labels for each line
For i = 1 To n + 1
Shapes.AddLine(0, i*s, n*s, i*s).Line //horizontal lines
Shapes.AddLine(i*s, 0, i*s, n*s).Line //vertical lines
With Shapes.AddTextbox(msoTextOrientationHorizontal, i*s, n*s, 30, 20)
.TextFrame.Characters.Text ← i-1
With Shapes.AddTextbox(msoTextOrientationHorizontal, 0, i*s, 30, 20)
.TextFrame.Characters.Text ← i-1
//draw middle horizontal and vertical lines that may be of higher weight
With Shapes.AddLine(0, n*s/2, n*s, n*s).Line //horizontal
.Weight ← 3
With Shapes.AddLine(n*s/2, 0, n*s/2, n*s).Line //vertical
.Weight ← 3
//draw horizontal and vertical arrows to demarcate dependence and driving powers
Shapes.AddConnector(msoConnectorStraight, 0, n*s, n*s, n*s).Select
Selection.ShapeRange.Line.EndArrowheadStyle ← msoArrowheadOpen
With Shapes.AddTextbox(msoTextOrientationHorizontal, n*s/2, n*s, 110, 20)
.TextFrame.Characters.Text ← “Dependence Power”
Shapes.AddConnector(msoConnectorStraight, 0, n*s, 0, s).Select
Selection.ShapeRange.Line.EndArrowheadStyle ← msoArrowheadOpen
With Shapes.AddTextbox(msoTextOrientationUpward, 0, n*s/2, 20, 80)
.TextFrame.Characters.Text ← “Driving Power”
//write labels for each quadrant such as autonomous (I), dependent (II), linkage (III),
//and independent (IV) variables
With Shapes.AddTextbox(msoTextOrientationHorizontal, 0, s*n, 130, 40)
.TextFrame.Characters.Text←“I-Autonomous Variables III-Linkage Variables”
With Shapes.AddTextbox(msoTextOrientationHorizontal, s*n, s*n, 135, 40)
.TextFrame.Characters.Text←“II-Dependent Variables IV-Independent Variables”
//place the I to IV (Roman[]) in quadrants in appropriate positions x[] and y[]
For i = 1 To 4
With Shapes.AddTextbox(msoTextOrientationHorizontal, x[i], y[i], 20, 20)
.TextFrame.Characters.Text ← Roman[i]
//set dependence and driving in 2-dimensional arrays x and y
For i = 1 To n
x[i][1] ← FRM[i][n + 1]
For i = 1 To n
y[1][i] ← FRM[n + 1][i]
//aggregate elements with same dependence and driving powers in a 2-dimensional
//array E
For i = 1 To n
If x[1][i], y[i][1] = x[1][i], y[i][1]
Append ith element at E[x[1][i]][y[i][1]]
//place ovals of size o and elements on the grid, offsets have been ignored
For i = 1 To nVar
For j = 1 To nVar
If E[i][j] != Null
Shapes.AddShape(msoShapeOval, i*s, j*s, o, o)
With Shapes.AddShape(msoShapeRectangle, i*s, j*s, 0, 15)
.TextFrame.Characters.Text = E[i][j]

The SmartISM software was extensively tested on studies available in the literature.
For any discrepancy between the reported results in the study and the SmartISM, steps
were manually verified to validate the results of SmartISM, as shown in Table 3. The sample
results of SmartISM for one of the previous studies [77] that had no discrepancy are shown
in Figures 2–5.
Sustainability 2021, 13, 8801 13 of 27
Sustainability 2021, 13, x FOR PEER REVIEW  14  of  33 
 
Sustainability 2021, 13, x FOR PEER REVIEW  14  of  33 
 

 
 
Figure 2. Final Reachability Matrix (FRM). 
Figure 2. Final Reachability Matrix (FRM).
Figure 2. Final Reachability Matrix (FRM). 

 
Sustainability 2021, 13, x FOR PEER REVIEW    15  of  33 
  Figure 3. Digraph. 
Figure 3. Digraph. 
Figure 3. Digraph.

 
Figure 4. Final ISM Model.
Figure 4. Final ISM Model. 
Sustainability 2021, 13, 8801 14 of 27
 
Figure 4. Final ISM Model. 

 
Figure 5. MICMAC Diagram.
Figure 5. MICMAC Diagram. 

5. Results
5. Results 
This section presents the scoping review answers to the questions described in the
This section presents the scoping review answers to the questions described in the 
research methodology section with respect to domain of ISM applications, variables of
research  methodology section  with  respect  to  domain  of  ISM  applications,  variables  of 
study, composition of decision makers, decision making, and data collection techniques, as
study, composition of decision makers, decision making, and data collection techniques, 
summarized in Table 2. Furthermore, the results of the assessment of ISM technique using
as summarized in Table 2. Furthermore, the results of the assessment of ISM technique 
SmartISM on the selected 77 papers are summarized in Table 3.
using SmartISM on the selected 77 papers are summarized in Table 3. 
Table 2. Articles’ domain, variables, decision makers and techniques, and ISM and other MCDM approaches.
Table 2. Articles’ domain, variables, decision makers and techniques, and ISM and other MCDM approaches. 
S. Domain Variables Method of Decision Number of Decision
No.
Reference
(Sub-Domain) Variables 
Description Making Makers
Techniques
Refer‐ Domain    Method of Decision  Number of Decision 
S. No.  Descrip‐ workshops, seminars,
Techniques 
ence  (Sub‐Domain)  Making  Makers 
1 [43]
Sustainability tion 
6 barriers
telephonic enquiries,
4 excluding authors ISM and MCDM
(Eco-Design) and individual and
consensus questionnaire
DEMATEL-MMDE-
two-phase
Sustainability 10 barriers and 10 (5 academic and 5 SEM-ISM
2 [15] questionnaire survey
(Manufacturing) 10 enablers industry) Separate analysis for
and online survey
academic and industry
Sustainability Including experts from
3 [49] 11 drivers decision team ISM
(GSCM) the industry
personal interview,
Sustainability Likert scale 10 (9 industry and 1
4 [78] 26 CFS ISM
(GSCM) questionnaire, decision government)
team
10 (6 industry, 1 NGO, 2
Sustainability brainstorming, and
5 [16] 10 barriers government and 1 DEMATEL-ISM
(E-Waste recycling) interviews
academic)
Sustainability industry, academic and ISM, fuzzy AHP and
6 [13] 8 factors group decision making
(Healthcare-Waste) government, 9 for AHP fuzzy TOPSIS.
Logistics (Reverse
7 [14] 7 factors expert opinion 5 for TOPSIS ISM and fuzzy TOPSIS
logistics Provider)
Energy (Smart grid idea engineering
8 [60] 12 barriers Industry and academia ISM and MICMAC
technologies) workshop
Sustainability face to face interview 15, (OEM 3, tier-1 5, tier-2
9 [38] 18 factors ÌPA, ISM, MICMAC
(GSCM) using questionnaire 4, tier-3 3)
Sustainability expert opinions,
10 [31] (Landfill 12 barriers interviews and group information not available ISM MICMAC
Community) discussions
Energy (Solar 6 (4 industry and 2
11 [79] 13 barriers workshop ISM MICMAC
Power) academic)
Sustainability 2021, 13, 8801 15 of 27

Table 2. Cont.

S. Domain Variables Method of Decision Number of Decision

Reference Techniques
No. (Sub-Domain) Description Making Makers
5 (industry and 2 Exploratory Factor
12 [32] SCM (Food-Waste) 9 challenges consultation
academic) Analysis, ISM MICMAC
SCM 7 (3 industry and 4
13 [80] 12 CSF consultation ISM MICMAC
(Humanitarian SC) academic)
Sustainability
14 [81] 13 enablers one day workshop 15 or 25 industry ISM MICMAC
(SSCM)
consultation and
Business 30 (6 each for 5 cases’
15 [82] 9 elements approval of interviewed ISM MICMAC
(Offshoring) development)
experts
Logistics (Reverse 2 (1 industry and 1
16 [62] 11 barriers consultation ISM MICMAC
logistics) academic)
4 for questionnaire and 3
questionnaire and
17 [77] Energy 13 barriers for agreement through ISM MICMAC
communication
communication (mix)
ranking order clustering
18 [83] HR (OSH) 8 factors discussion for 72 h 5 industry
ROC, ISM
26 barriers
Sustainability identification questionnaire and 2 (1 industry and 1
19 [63] ISM MICMAC
(GSCM) by more than 5 discussion academic)
experts
Sustainability 5 (3 industry and 2
20 [33] 20 barriers brainstorming ISM MICMAC
(GSCM) academic)
Logistics (3rd party
21 [84] Reverse logistics 7 attributes consultation information not available ISM MICMAC
provider)
Tourism (Tour 7 (4 industry and 3
22 [66] 12 enablers brainstorming ISM MICMAC
Value) academic)
52 respondents through
Marketing
23 [39] 17 motivations laddering interview content analysis of Content analysis ISM
(Motivation)
interviews
direct meetings, 150 firework industrial
Supplier Selection
telephonic inquiries, managers from SMEs and
24 [44] (Corporate social 9 CSR issues ISM MICMAC
and email, and relevant field experts for
responsibility)
questionnaire variable identification
45 or more survey
Sustainability 25 SSCM questionnaire and
25 [85] response, consolidation ISM MICMAC
(SSCM) practices judgement group
judgement group of 15
Sustainability
26 [10] 20 barriers workshop 23 experts ISM ANP
(Production)
Sustainability
thematic analysis,
(Green Business
27 [58] 5 barriers semi-structured 19 ISM MICMAC
Model in
interviews
Construction)
Marketing (Retail 9 (4 industry and 5
28 [34] 10 factors brainstorming sessions ISM MICMAC
Brand) academic)
15 (10 industry and 5
Sustainability academic) Divided into 2
29 [86] 19 barriers interview ISM MICMAC
(Neighborhoods) nominal groups for
validation
8 CSFs for
roads and
Sustainability idea engineering
30 [40] bridges, and 10 3 (2 industry, 1 academia) ISM MICMAC
(Construction) workshop
CSFs for
embarkment
IT (E-Commerce 13 semi structured
31 [19] 10 factors 13 industry Content analysis, ISM
Security) interview
8 Maintenance
Operations
32 [35] tools and consultation 5 ÍSM MICMAC
(Maintenance)
techniques
Operations
33 [36] 24 factors discussion information not available ISM MICMAC
(Maintenance)
face to face
communication,
ISM MICMA Fuzzy
34 [45] IT (ERP) 16 CSFs questionnaire surveys 14 industry
cognitive maps
since difficult to gather
them
Sustainability 2021, 13, 8801 16 of 27

Table 2. Cont.

S. Domain Variables Method of Decision Number of Decision

Reference Techniques
No. (Sub-Domain) Description Making Makers
11 innovation
Innovation
35 [61] enablers (com- Delphi technique industry and academia ISM Fuzzy MICMAC
(Manufacturing)
petitiveness)
Delphi technique from 14 (11 industry and 3
36 [37] IT (Cloud) 16 CSFs ISM MICMAC
20 to 16 academic)
27 survey response
Sustainability 27 industry survey based
37 [87] 10 enablers followed by the TISM MICMAC
(SSCM) on Fawcett et al. (2014)
telephonic interviews
14 (10 industry and 4
38 [88] HR (Team) 11 factors expert opinions ISM
academic)
9 factors
25 to 16 industry
39 [89] HR (OHSAS) influencing expert opinions ISM MICMAC
responded
safety
decision team rated on
Sustainability
40 [90] 10 hurdles 5-point Likert scale and 7 (4 industry, 3 academic) ISM MICMAC
(SSCM)
brainstorming for SSIM
41 [91] Energy (Solar) 16 barriers LR and expert opinion None ISM Fuzzy MICMAC
13 Indian
National (National Critical LR, brainstorming, face 30 experts (7 academic, 13
42 [41] ISM MICMAC
Infrastructure) infrastructure to face interview industry, 10 gov.)
sectors
5 risk factors expert opinions through research group of senior
43 [68] Energy (Biodiesel) ISM Bayesian network
for ISM group discussions university professors
semi structured
interviews, factors
Sustainability
selected on 4 to 3 votes,
44 [92] (Emission trading 15 factors 58 to 10 to 7 experts ISM MICMAC
same rule of majority
system)
gives way to minority
for relationship
Sustainability
45 [71] 14 obstacles consultation industry and academia ISM
(Remanufacturing)
2 brainstorming
sessions and verification
by recirculation of
46 [23] SCM (SCM) 15 factors 5 industry ISM MICMAC
model, 179 survey
response to develop
correlation matrix
questionnaire-based
340 to 480
experts from industry and survey; ISM MICMAC
47 [46] Manufacturing 15 factors self-administered
academic approach; EFA; CFA
questionnaires
and GTMA
14 Critical 13 Indian academics in
48 [24] SCM (Food SCM) Input from experts ISM MICMAC
causal factors phase 1, 12 industry
linguistic preferences
and that concurrently
applies perception 10 researchers then 5
Sustainability fuzzy set theory and
49 [64] 21 criteria judgment-pretesting of researchers for pre-testing,
(Production) ISM
survey-purposive For ISM 120 responses
sampling method-120
replies
Semi structured
16 causes of
50 [25] SCM (Food SCM) interview 6 (4 industry 2 academic) TISM fuzzy MICMAC
food wastages
Brainstorming
13 Upper Echelon Theory
Sustainability psychological (UET) Focus group
51 [11] 10 (2 industry 8 academic) Hybrid ISM -ANP
(Green IT/IS) drivers of discussions Survey for
motivation ANP
the semi-structured
IT (Building 15 capability
interviews and focus
52 [93] Information factors for BIM 7 industry ISM MICMAC
groups were conducted
Modelling) adoption
and LR
Phone consultation and
Sustainability 7 grouped
53 [69] face to face structured 4 (2 industry 2 academic) ISM MICMAC
(Reverse logistics) barriers
interview
Sustainability 2021, 13, 8801 17 of 27

Table 2. Cont.

S. Domain Variables Method of Decision Number of Decision

Reference Techniques
No. (Sub-Domain) Description Making Makers
Sustainability 12 behavioral 15 (10 industry, 3
54 [65] Opinion of experts ISM MICMAC
(GSCM) factors academic, 2 government)
Ability Motivation
Opportunity theoretical
11 experts listed 42
lens Questionnaire and
variables EFA 42 to 32 LR, expert opinion to
Sustainability personal interview Pilot
55 [70] 7 factors variables to 7 factors CFA EFA to CFA to ISM
(GHRM) study from 97
32 to 24 variables Same 11 MICMAC to ISEM
respondents to 7 factors
experts for ISM
ISM on 7 factors SEM to
validate model
Group discussion for
ANP -horizontal
9 antecedents ANP and pairwise
Operations (Lean integration of human
56 [12] and critical comparisons, Also, for 7 industry
Manufacturing) ISM- vertical integration
success factors identifying 9 factors and
of human
ISM
11 technological
57 [94] IT (SCM) Brainstorming 3 industry TISM
capabilities
library survey method- leveling the factors
Supplier questionnaire-research using ISM model
managers and the
58 [9] relationship 10 factors method in this survey is MICMAC, fuzzy
production heads
management (SRM) descriptive and TOPSIS-AHP- ranking
analytical suppliers
Five-point scale survey 8 industry for variable
SCM 14 barriers and
59 [72] followed by reduction + 4 academic ISM SEM
(E-Procurement) 15 benefits
brainstorming session for ISM
32 CSFs were identified
32 CSFs of
Sustainability from LR and opinions consultation with industry
60 [20] motivation and ISM MICMAC
(SSCM) of academicians and experts and academic
encouragement
industry practitioners
survey on 60 PG
IT (Social 12 Factors of students who have
61 [95] networking sites abandoning abandoned the SNS at 60 students as experts ISM MICMAC
(SNS)) usage of SNS least once and
validation by LR
LR, and invoking the
19 influential
contingency theory
Sustainability factors on the
62 [59] perspective to get 19 13 Industry ISM MICMAC
(GSCM) implementa-
factors group discussion
tion
for ISM (4 sessions)
Interview for
Sustainability identification of barriers
13 (12 industry and 1
63 [47] (recycling 3D 22 barriers and 3 experts consulted ISM
academic)
printing waste) for relationships
between barriers
LR, survey and expert
Sustainability
opinion for
64 [26] (Green Lean Six 12 enablers Not mentioned ISM MICMAC
identification of
Sigma)
enablers
LR and survey for
identification of
IT (Building enablers and
65 [21] Information 12 barriers semi-structured 5 industry ISM MICMAC
Modelling) interview for
relationships between
barriers
LR and expert opinion
for identification of
determinants,
Sustainability (Oil 18 (13 industry 5
66 [22] 7 determinants individual expert ISM MICMAC and SEM
and gas SSCM) academic)
opinion for
relationships among
determinants
Sustainability 2021, 13, 8801 18 of 27

Table 2. Cont.

S. Domain Variables Method of Decision Number of Decision

Reference Techniques
No. (Sub-Domain) Description Making Makers
LR and survey for
identification of
Number of participants
67 [96] IT (m-commerce) 10 barriers determinants, survey ISM MICMAC
not mentioned
for relationships among
determinants
LR and interview for
Building energy identification of factors,
68 [50] 16 factors 12 industry ISM MICMAC
performance and interview for
relationships
Report, passengers’
opinion including
Quality of authors and
69 [51] Passenger 15 enablers observations for Authors opinions ISM MICMAC
interaction process identification of
enablers, and also for
relationships
Content analysis on
interview transcripts of
IT (Cloud 58 industry people for ISM MICMAC, AHP
70 [52] 7 determinants Not mentioned
computing) identification of and TOPSIS
determinants, Group
discussion for relations
LR and 10 consultation
sessions with 35 experts
using brainstorming
35 (industry and
71 [27] SC (Lean Sis Sigma) 10 barriers and focused group ISM MICMAC
academic)
method to identify
barriers and
relationships
LR, survey and
interviews for
identification of 20 (industry and
72 [42] Sustainability (CSR) 12 enablers ISM MICMAC
enablers and experts’ academic)
opinions for
relationships
LR, expert opinion and
Automobile 10 (7 industry and 3 ISM MICMAC and Best
73 [28] 12 barriers best worst method to
(Electric Vehicle) academic) worst method
select barriers
LR and survey from
Marketing
expert for factors 33 (30 industry and 3
74 [97] (app-based 20 factors ISM MICMAC
identification and academic)
retailing)
relationships
LR and consultation
SC (Food loss and with experts for factors 11 (5 industry and 6
75 [48] 8 factors ISM MICMAC
waste) selection and academic)
relationships
LR and expert interview
New product
19 capabilities for identification of 14 (8 industry and 6
76 [29] development (time ISM fuzzy MICMAC
and 5 drivers capabilities and drivers academic)
to market)
and relationships
LR and SWOT analysis
for strategies
Sustainability 5 (2 industry, 2 academic
77 [30] 13 strategies identification and SWOT ISM MICMAC
(Renewable energy) and 1 government)
experts’ opinion for
relationships
SC: Supply chain, SCM: SC management, SSCM: Sustainable SCM, GSCM: Green SCM, SRM: Supplier relationship management, OSH:
Occupational safety and health, IT: Information technology, IS: Information Systems, ERP: Enterprise resource planning, HR: Human
resources, GHRM: Green HR Management, OHSAS: Occupational health and safety assessment series, NGO: Non-governmental organi-
zation, SME: Small and medium scale enterprises, ISM: Interpretive structural modelling, SSIM: Structural self-interaction matrix, AHP:
Analytical hierarchy process, ANP: Analytic network process, TOPSIS: Technique for order of preference by similarity to ideal solution, LR:
Literature review, CSF: Critical success factors, SEM: Structural equation modeling, EFA: Exploratory factor analysis, CFA: Confirmatory
factor analysis; CSR: Corporate social responsibility; SWOT: Strengths, weaknesses, opportunities, and threats.
Sustainability 2021, 13, 8801 19 of 27

Table 3. Assessment of application of ISM using Smart ISM.

S. No. References SSIM RM FRM Partitioning Transitivity Digraph Model Edges MICMAC Remarks
Added extra edge from variable 4
1 [43] Y Y Y Y 2nd Level N Y Varied Y
to variable 5
2 [15] N N N N N N N N N No SSIM
Some edges are missed such as
3 [49] Y Y Y Y 2nd Level N Y Varied Y
between variable 9 and variable 1
4* [78] Y Y Y Y 2nd Level N Y Y Y
5 [16] Y Y N N N N N N Y Wrong calculation for transitivity
Transitivity not incorporated
6 [13] Y Y N N No Level N N N Y though mentioned in the
methodology
7 [14] Y Y Y Y 2nd Level N Y Varied Y Added extra edge from R to I
8* [60] Y Y Y Y All Levels N Y Y Y
At 8th level one variable is wrong
9 [38] Y Y Y N 2nd Level N N Y Y
in model
Didn’t incorporate transitivity
10 [31] Y Y N N No Level N N N N
though the concept is mentioned
11 * [79] Y Y Y Y All Levels N Y Y Y
Wrong calculation for transitivity,
12 [32] Y Y N N All Levels N N N N only two transitive links but shown
more
13 * [80] Y Y Y Y 2nd Level N Y Y Y No transitive links
14 * [81] Y Y Y Y 2nd Level N Y Y Y
Retained some transitive links in
15 * [82] Y Y Y Y 2nd Level N Y Y Y
the model
16 * [62] Y Y Y Y 2nd Level N Y Y Y
17 * [77] Y Y Y Y 2nd Level N Y Y Y
18 * [83] Y Y Y Y 2nd Level N Y Y Y Model upside down
19 * [63] Y Y Y Y 2nd Level N Y Y Y
Wrong calculation for transitivity
and for generalized transitivity; all
20 [33] Y Y N N N N N N Y
will be having same dependence
and driving powers with max value
MICMAC axis interchanged and for
generalized transitivity all will be
21 * [84] Y Y Y Y 2nd Level N Y Y Y
having same dependence and
driving powers with max value
No SSIM and directly starts
22 [66] N N N N N N N N N
with RM
23 [39] Y Y Y N All Levels N N N N No MICMAC
Added extra edge from variable 6
24 [44] Y Y Y Y 2nd Level N Y Varied Y
to variable 9
25 * [85] Y Y Y Y 2nd Level N Y Y Y
26 * [10] Y Y Y Y 2nd Level N Y Y N B2 element wrong on MICMAC
27 * [58] Y Y Y Y 2nd Level N Y Y Y
RM and FRM together, one
28 [34] Y Y N N N N N N N transitive link skipped between
elements EM and BN
29 * [86] Y Y Y Y All Levels N Y Y Y
Wrong leveling, one element
30 [40] Y Y Y N 2nd Level N N N N
missing on MICMAC
No MICMAC, one extra wrong
31 [19] Y N N N 2nd Level N N N N
self-relation
Transitivity not incorporated
though mentioned in the
32 [35] Y Y N N No Level Y N N N methodology, non-standard
relation symbols in SSIM such as F,
R, FR, and X
33 [36] Y Y N N 2nd Level N N N N Wrong transitivity calculation
Added wrong edges for F12
34 [45] Y Y Y Y All Levels N Y Varied Y
and F15
35 * [61] Y Y Y Y 2nd Level N Y Y Y No transitivity relations found
36 [37] Y Y N N 2nd Level Y N N N Wrong transitivity calculation
Skipped one transitive relation
between 8 and 9, same results for
37 * [87] Y Y N Y 2nd Level N Y Y N
second-order and
generalized transitivity
Transitivity mentioned but not
38 * [88] Y Y N Y No Level N Y Y Y included; rest of all the
things correct
39 [89] Y N N N 2nd Level N N N N Wrong transitivity calculation
Same results for second-order and
40 * [90] Y Y Y Y All Levels N Y Y Y
generalized transitivity
41 * [91] Y Y Y Y All Levels N Y Y Y
42 [41] Y Y Y N 2nd Level N N N Y No transitive links
43 * [68] Y Y N Y 2nd Level Y Y Y N
44 [92] N N N N N N N N N Non-standard SSIM
Transitivity not calculated and
45 * [71] Y Y N Y No Level N Y Y N
no MICMAC
46 [23] Y Y N N 2nd Level N N N N
Sustainability 2021, 13, 8801 20 of 27

Table 3. Cont.

S. No. References SSIM RM FRM Partitioning Transitivity Digraph Model Edges MICMAC Remarks
47 [46] Y Y Y Y 2nd Level N Y Varied Y Extra edges added
48 [24] Y Y N N 2nd Level N N N N
49 [64] N N N N N N N N N Non-standard SSIM, fuzzy ISM
50 [25] Y Y N N 2nd Level N N N N TISM
One extra edge, MICMAC
51 [11] Y Y Y Y 2nd Level N Y Varied N calculation is correct but
drawn wrong
For self-relation X shown in SSIM
52 [93] Y N N N 2nd Level N N N N
whereas RM and FRM not given
53 * [69] Y Y Y Y 2nd Level Y Y Y Y
54 * [65] Y Y Y Y 2nd Level Y Y Y Y For self-relation X shown in SSIM
55 * [70] Y Y Y Y No Level Y Y Y Y Transitivity not included
56 * [12] Y Y Y Y 2nd Level Y Y Y Y
57 [94] N N N N N N N N N Non-standard SSIM
Many mistakes in
58 [9] Y Y N N 2nd Level N N N N
transitivity calculation
59 [72] Y Y Y Y 2nd Level N N N N No MICMAC, extra edges
60 [20] Y N N N 2nd Level N N N N RM onwards wrong calculations
61 * [95] Y Y Y Y All Levels N Y Y N MICMAC explained in detail
Partitioning in simulation didn’t
62 [59] Y N N N 2nd Level N N N N give levels, for self-relation X
shown in SSIM
Some extra edges drawn in final
63 [47] Y Y Y Y 2nd Level N Y Varied N
model, no MICMAC
64 [26] Y Y N N N N N N N Wrong transitivity calculations
Non-standard SSIM, wrong
65 [21] Y N N N All Levels N N N N conversion for relationship between
elements B10 and B08 in RM
in SSIM for CGLC, self-influence is
66 [22] Y N N Y All Levels N Y Y N
considered wrong and assigned 0
67 * [96] Y Y Y Y 2nd Level N Y N Y Digraph is not mentioned
Adjacency matrix to represent RM,
68 [50] N N Y Y All levels N Y Varied Y one edge F6 to F1 from adjacency
matrix is not drawn
Some edges are missed such as
69 [51] Y Y Y Y All Levels N Y Varied Y
between NWFOB and AEPE
1 in SSIM for self-relation,
transitivity not included although
70 [52] Y Y N Y No Level N Y Varied Y concept is discussed; some
reachability edges missed such as
C6 to C1
5 transitive links missing in 2nd
71 [27] Y Y N Y 2nd Level Y Y Y N level transitivity and 6 missing
from all levels
First partitioning iteration is wrong
72 [42] Y Y Y N 2nd Level N N N Y
and incorrectly assigns level 1 to E4
One transitive link between B1 and
73 [28] Y Y N Y All levels N Y N N B12 is missed out; B1 is wrongly
placed in MICMAC diagram
All 3 transitive links missing,
74 * [97] Y Y N Y No Level N Y Y N although the concept is mentioned;
Wrong MICMAC
One extra edge between F1 and F4
75 [48] Y Y Y Y All levels N Y Varied Y
that changes the reachability
X for self-relation in SSIM, missing
transitive link between T14 and T19,
76 [29] Y Y N N 2nd Level N N N N self-interaction between for T17
missing, Wrong transitive link
given between T19 and T6
Many mistakes in transitivity
77 [30] Y Y N N All Levels N N N N calculations such as between S5
and S13

* All SmartISM reproduced results similar to articles’ ISM process results with varied transitivity considerations. Y: SmartISM reproduced
results similar to articles’ ISM process results, N: SmartISM reproduced results not similar to articles’ ISM process results, Varied: edges in
the final model varied, although the hierarchical structure of variables may or may not be correct. ISM: Interpretive structural modelling,
TISM: Total ISM, SSIM: Structural self-interaction matrix, RM: Reachability Matrix, FRM: Final RM, MICMAC (Matrice d’Impacts Croisés
Multiplication Appliquée á un Classement (cross-impact matrix multiplication applied to classification)). Some variables code such as
B2, EM, and BN have been given in the table for their description, refer to the articles. Symbol X in SSIM means both variables mutually
influence each other.

5.1. Domain of Study

ISM is being applied in numerous fields such as sustainability, social sciences, manage-
ment, engineering, and information technology. The results show 21 different domains, with
highest studies in sustainability (32), supply chain and logistics (13), information technology
(9), energy (5), human resource (3), marketing (3), and operations (3) (Table 2). Within sus-
tainability, the highest studies are in the area of green supply chain management (GSCM)
Sustainability 2021, 13, 8801 21 of 27

and sustainable supply chain management (SSCM) with seven studies each followed by
two studies in construction [40,58] and several other areas such as e-waste recycling [16],
healthcare waste [13], recycling 3D printing waste [47], green IT/IS [11], among others. In
the area of supply chain and logistics, studies have focused on supplier relationship [9]
and selection [44], food SCM [24,25], e-procurement [72], and reverse logistics [14,62,84],
among others. In the field of information technology studies are conducted in the areas of
building information modeling [21,93], cloud computing [37,52], e-commerce security [19],
m-commerce [96], enterprise resource planning [45], supply chain management [94], and
social networking sites [95]. Energy domain studies were in the area of bio-diesel [68],
smart grid technologies [60], and solar energy [79]. For the human resource domain, two
studies were in the area of occupational health and safety [83,89] and one in team perfor-
mance [88]. The studies in the marketing area focused on motivation [39], retail brand [39],
and app-based retailing [97]. Furthermore, the articles in the area of operations focused
on maintenance [35,36] and lean manufacturing [12]. Some of the innovative areas were
landfill communities [31], emission trading system [92], tour value [66], and quality of
passenger interaction process [51].

5.2. Variables of Study

Most studies focus on enablers or drivers, challenges or barriers, critical success
factors, and influencing or significant elements in the domain of research (Table 2). Other
studies have tried to explore different sets of variables. For example, article [84] has studied
seven attributes of third party reverse logistics; article [39] has studied 17 motivational
factors in the marketing area; article [44] has used 9 corporate social responsibility factors
in the area supplier selection; article [85] has studied 25 SSCM practices; article [35]
interrelated maintenance tools and technique; article [11] explored 13 psychological drivers
of motivation in the area of green IT/IS; article [95] studied the interrelationships between
12 factors for abandoning social networking sites; article [29] studied capabilities and
drivers for new product development; and article [30] studied 13 strategies for renewable
energy. The number of variables being studied ranged from 5 [58] to 32 [20]. Additionally,
some studies explored two types of factors such as 10 barriers and 10 enablers [15] and
14 barriers and 15 benefits [72]. One study gave variables in two applications such as 8 CSFs
for roads and bridges and 10 CSFs for embarkment [40]. These variables are identified
mostly through literature review, experts’ opinions, and/or survey. Some of the unique
approaches used to identify variables are thematic analysis [58], upper echelon theory [11],
contingency theory [59], content analysis [52], best worst method [28], SWOT (strength,
weakness, opportunity, and threat) analysis [30], idea engineering workshop [40,60], and
Delphi technique [37,61].

5.3. Domain Experts or Decision Makers

This is the most crucial step as it provides the input for further steps. There are two
important aspects, namely, selection of decision makers (DM) and method of information
gathering from them. There are three different sets of DMs in the sample studies partic-
ipants from industry, academia, and government. Participants varied from 120 through
survey [64] to 2 [62,63] through group discussion and consultation. The median value of the
total number of participants was 10. Only four studies [16,30,41,65] had taken participants
from all three sectors: industry, academia, and government. While 56 studies had DMs
from industry including others and two studies [51,72] had only academic DMs, 17 studies
didn’t mention the number of DMs.

5.4. Decision Making and Data Collection Techniques

There are two approaches followed for decision making, namely, discourse and survey.
For discourse many techniques have adopted such as idea engineering workshops [40,60],
telephonic enquiries [43,44], group decision making [13,52], personal interview [70,78],
brainstorming [16,33,34,66], laddering interview [39], direct meetings [44], semi-structured
Sustainability 2021, 13, 8801 22 of 27

interview [21,58,93] and structured interview [69], Delphi technique [37,61], and focus
group discussion [11,27]. Similarly, for surveys different techniques are as follows: individ-
ual and consensus questionnaire [43], Likert scale questionnaire [78,90], email question-
naire [44], library survey method [9], and self-administered questionnaire [46].

5.5. Assessment of Application of ISM Technique Using SmartISM

The SSIM matrices from all the 77 articles were entered into the developed SmartISM
software and resulted in 77 excel files. Thereafter, for each article results were reproduced
in the SmartISM software buy running the macros in 77 excel files. Variations between
the reported results in the articles and corresponding SmartISM reproduced results were
studied. Due to differences in transitivity incorporation FRM was checked for firstly non-
incorporation of transitivity, followed by two-hop transitivity (second-order) and lastly
generalized transitivity (all levels). In some cases, second order and generalized transitivity
could be same. Furthermore, in case of inconsistency the digraph was manually built and
transitivity was traced before reporting the results. Similarly, the complete process was
analyzed to identify the reasons for the errors in any of the steps. As ISM technique is
sequential, error in one step will cause subsequent steps to be erroneous specially if error
exists in the steps of RM, FRM, and level partitioning.
The results of the assessments are summarized in Table 3 where ‘Y’ means the articles’
calculations match with that of SmartISM and ‘N’ means different results. For each article
SSIM, RM, FRM, level partitioning, digraph, final model, edges in the final model, and
MICMAC diagrams are verified. Three studies [64,92,94] didn’t report standard SSIM and
two studies [15,66] had no SSIM, therefore results were not reproduced for them. Out of
72 remaining studies, 29 studies came out correct on all the parameters and their serial
numbers (S. No.) are marked with stars ‘*’. Of these 29 studies, 25 had either second
level (two-hop or second-order transitivity) or all levels (generalized transitivity), and
four [70,71,88,97] had no transitivity calculations.
The remaining 43 studies with different results from the SmartISM outputs were further
analyzed starting from first the step of SSIM and moving on until the variations were identi-
fied. One study [19] had one wrong self-relation in RM. Three studies [20–22] had incorrect
RM and one article [93] did not provide RM and FRM. Two studies [59,89] didn’t provide
RM, and their FRM didn’t match with the SmartISM output due to wrong transitivity
calculations. Furthermore, eighteen studies [9,13,16,23–37] had incorrect transitivity calcu-
lations leading to variations in FRM. Five studies [38–42] had variations at the fourth step
of level partitioning. One study [31] was checked for level partitioning without transitivity
as it was not considered, and came out wrong in assigning levels to variables. Finally,
12 studies had accurate hierarchical structures of variables in the final ISM model but
variations in the edges that distorted the reachability of variables. Of these 12 studies, eight
studies [11,14,43–48] had some extra edges and four studies [49–52] had some missing
edges in the final model.
In the documentation of application of ISM, only eight studies [12,20,27,35,65,68–70] re-
ported digraph. MICMAC analysis has been used by all studies except five [19,39,47,71,72]
to explain grouping of the variables. Five studies [27,29,59,65,93] have explicitly men-
tioned X and one study [52] has mentioned 1 in the SSIM for variables to represent mutual
self-relation, although it is a basic assumption in ISM therefore, other studies have not
mentioned it.

6. Discussion
The operationalization of the ISM is best to be conducted through software, as there
are tedious calculations such as transitivity, level partitioning, and graphical displays
of digraph, final ISM model, and MICMAC diagram. Moreover, these calculations and
displays need to be iterated until the high confidence model is approved by the experts.
Therefore, this study has explained the methodology to develop MS Excel and VBA based,
end-to-end software, SmartISM for ISM and MICMAC with the help of pseudo codes. For
Sustainability 2021, 13, 8801 23 of 27

incorporating transitivity in FRM, the Warshall algorithm has been used, and a new algorithm
RCM has been introduced for removing edges from variables’ second lower level onwards
without affecting reachability and digraph structure. Further, the demonstration video of
SmartISM has been added as a Supplementary Material, and this tool has been extensively
tested on the existing studies and applied successfully in some of the studies [98–100].
Furthermore, the scoping review shows that the ISM and MICMAC techniques are
being applied in different domains of social sciences, management, engineering, and
technology such as sustainability, SCM, operations, manufacturing, human resource, infor-
mation technology, and many other innovative areas. This technique is also employed in
different multi criteria decision making techniques such as AHP, ANP, TOPSIS, DEMATEL,
etc. There are four important issues that need to be addressed such as variables and their
context, decision makers’ experience and numbers, decision making and data collection
techniques, and utilization of software tools. The nature of variables has been enablers or
drivers, challenges or barriers, critical success factors, strategies, capabilities and drivers,
and influencing or significant elements in the area of study. Their numbers have varied
from 5 to 32 and they have been identified through domain specific literature review, ex-
perts’ opinions, and/or survey. Furthermore, techniques such as thematic analysis, upper
echelon theory, contingency theory, content analysis, best worst method, SWOT analysis,
idea engineering workshop, and Delphi technique are used for variables’ identification.
Similarly, the variables have been explained well to establish the contextual meaning.
Another important aspect is the experts or decision makers, as the whole analysis is
dependent upon their knowledge and experience. There should be representation from
all stakeholders of the domain being studied. In the best-case, experts from academia,
industry, government and regulatory bodies should be selected in the panel of DMs. There
are very few studies such as four in the sample of articles that have had DMs from all the
stakeholders. The number of DMs varied from 2 to 120, whereas in most of the studies
they were 11. Two approaches have been utilized for extracting information from DMs
namely discourse and surveys. The discourse techniques are idea engineering workshops,
telephonic enquiries, group decision making, personal interview, brainstorming, laddering
interview, direct meetings, semi-structured and structured interview, Delphi technique,
and focus group discussion. Survey techniques have used individual and consensus
questionnaires, Likert scale questionnaire, email questionnaire, library survey method, and
self-administered questionnaire.
The SmartISM reproduced results, on the existing studies selected in scoping review,
show that only 29 out of 77 studies had correct calculations with varied transitivity in-
corporation such as no transitivity, second order transitivity, or generalized transitivity.
Wrong transitivity calculation has been the most frequent reason for incorrect ISM results
followed by variations in drawing edges in the final model that affects the reachability of
the variables.
Lastly, five studies didn’t report standard SSIM, which is essential to reproduce the
calculations. Therefore, as a standard practice some minimum outputs must be reported
namely SSIM, FRM, level partitioning (final after all iterations), and final ISM model.
Similarly, MICMAC analysis is also an important and indispensable part of ISM, as all
studies except five have used it for classifying variables into one of the four groups, namely,
dependent, independent, linkage, and autonomous.

7. Conclusions
Human decisions play a very important role in any social or technical system de-
velopment. Domain experts have intricate knowledge on the system and can predict the
contextual interrelationships between the variables of interest in the particular domain.
The interpretive structural modelling technique can assemble their tacit knowledge into
a tangible hierarchical model leading to an enhanced understanding of the subject. This
study has developed a software tool such as SmartISM to implement ISM in an error-free,
user-friendly, and graphical style. In addition to automation of existing routines of ISM,
Sustainability 2021, 13, 8801 24 of 27

the Warshall algorithm is used for transitivity calculations and a new algorithm, reduced
conical matrix, has been introduced to convert the digraph into a final model with lesser
edges while retaining the digraph structure and reachability of variables. Furthermore, the
scoping review of this research will guide practitioners, policy makers, and academicians
in applying this technique in different disciplines in an informed way. It will help in
managing ISM configuration settings such as variables’ selection, composition of decision
makers, decision making, and data collection techniques. The poor results of assessment
of application of ISM technique in the previous studies necessitate the utilization of an
end-to-end software, such as SmartISM, to produce a high confidence final model, explain-
ing the interrelationships between important constructs in the applied domain. To limit
the number of articles in the review process only the ScienceDirect database was used,
and for the last four years articles were randomly selected; therefore, results should be
interpreted accordingly. The future studies will focus on the development of software tools
to apply ISM in conjunction with other MCDM techniques such as AHP, ANP, TOPSIS,
and DEMATEL.

Supplementary Materials: The following is available online at https://www.mdpi.com/article/10

.3390/su13168801/s1, Video S1: The SmartISM software demonstration.
Author Contributions: Conceptualization, N.A. and A.Q.; methodology, N.A.; software, N.A.;
validation, N.A. and A.Q.; formal analysis, N.A.; investigation, A.Q.; resources, A.Q.; writing—
original draft preparation, N.A.; writing—review and editing, N.A. and A.Q. All authors have read
and agreed to the published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: The data presented in this study are available on request from the
corresponding author.
Acknowledgments: Authors are highly thankful to the King Khalid University, Kingdom of Saudi
Arabia for all the support provided for this research work.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Warfield, J.N. An Assault on Complexity; Office of Corporate Communications: Battelle, FI, USA, 1973.
2. Warfield, J.N. Developing subsystem matrices in structural modeling. IEEE Trans. Syst. Man Cybern. 1974, 4, 74–80. [CrossRef]
3. Warfield, J.N. Developing interconnection matrices in structural modeling. IEEE Trans. Syst. Man Cybern. 1974, 4, 81–87.
[CrossRef]
4. Warfield, J.N. Societal Systems: Planning, Policy and Complexity; John Wiley & Sons: New York, NY, USA, 1976.
5. Elkington, J. 25 Years Ago I Coined the Phrase “Triple Bottom Line.” Here’s Why it’s Time to Rethink it. Harvard Business Review,
25 June 2018. Available online: https://hbr.org/2018/06/25-years-ago-i-coined-the-phrase-triple-bottom-line-heres-why-im-
giving-up-on-it (accessed on 7 April 2021).
6. Farris, D.R.; Sage, A.P. On the use of interpretive structural modeling for worth assessment. Comput. Electr. Eng. 1975, 2, 149–174.
[CrossRef]
7. Warfield, J.N. Structuring Complex Systems, (A Battelle Monograph No. 4); Battelle Meml. Inst: Columbus, OH, USA, 1974.
8. Godet, M. Introduction to la prospective: Seven key ideas and one scenario method. Futures 1986, 18, 134–157. [CrossRef]
9. Beikkhakhian, Y.; Javanmardi, M.; Karbasian, M.; Khayambashi, B. The application of ISM model in evaluating agile suppliers
selection criteria and ranking suppliers using fuzzy TOPSIS-AHP methods. Expert Syst. Appl. 2015, 42, 6224–6236. [CrossRef]
10. Govindan, K.; Madan Shankar, K.; Kannan, D. Application of fuzzy analytic network process for barrier evaluation in automotive
parts remanufacturing towards cleaner production—A study in an Indian scenario. J. Clean. Prod. 2016, 114, 199–213. [CrossRef]
11. Dalvi-Esfahani, M.; Ramayah, T.; Nilashi, M. Modelling upper echelons’ behavioural drivers of Green IT/IS adoption using an
integrated Interpretive Structural Modelling—Analytic Network Process approach. Telemat. Inform. 2017, 34, 583–603. [CrossRef]
12. Wong, W.P.; Wong, K.Y. Synergizing an ecosphere of lean for sustainable operations. J. Clean. Prod. 2014, 85, 51–66. [CrossRef]
13. Chauhan, A.; Singh, A. A hybrid multi-criteria decision making method approach for selecting a sustainable location of healthcare
waste disposal facility. J. Clean. Prod. 2016, 139, 1001–1010. [CrossRef]
Sustainability 2021, 13, 8801 25 of 27

14. Kannan, G.; Pokharel, S.; Kumar, P.S. A hybrid approach using ISM and fuzzy TOPSIS for the selection of reverse logistics
provider. Resour. Conserv. Recycl. 2009, 54, 28–36. [CrossRef]
15. Bhanot, N.; Rao, P.V.; Deshmukh, S.G. An integrated approach for analysing the enablers and barriers of sustainable manufactur-
ing. J. Clean. Prod. 2017, 142, 4412–4439. [CrossRef]
16. Kumar, A.; Dixit, G. An analysis of barriers affecting the implementation of e-waste management practices in India: A novel
ISM-DEMATEL approach. Sustain. Prod. Consum. 2018, 14, 36–52. [CrossRef]
17. Ahmad, N.; Quadri, N.; Qureshi, M.; Alam, M. Relationship Modeling of Critical Success Factors for Enhancing Sustainability
and Performance in E-Learning. Sustainability 2018, 10, 4776. [CrossRef]
18. Naveed, Q.N.; Qureshi, M.R.N.; Alsayed, A.O.; Ahmad, N.; Sanober, S.; Shah, A. Assimilating E-Learning barriers using an
interpretive structural modeling (ISM). In Proceedings of the 2017 4th IEEE International Conference on Engineering Technologies
and Applied Sciences (ICETAS), Salmabad, Bahrain, 29 November–1 December 2017; pp. 1–7.
19. Tan, F.T.C.; Guo, Z.; Cahalane, M.; Cheng, D. Developing business analytic capabilities for combating e-commerce identity fraud:
A study of Trustev’s digital verification solution. Inf. Manag. 2016, 53, 878–891. [CrossRef]
20. Raut, R.D.; Narkhede, B.; Gardas, B.B. To identify the critical success factors of sustainable supply chain management practices in
the context of oil and gas industries: ISM approach. Renew. Sustain. Energy Rev. 2017, 68, 33–47. [CrossRef]
21. Tan, T.; Chen, K.; Xue, F.; Lu, W. Barriers to Building Information Modeling (BIM) implementation in China’s prefabricated
construction: An interpretive structural modeling (ISM) approach. J. Clean. Prod. 2019, 219, 949–959. [CrossRef]
22. Gardas, B.B.; Raut, R.D.; Narkhede, B. Determinants of sustainable supply chain management: A case study from the oil and gas
supply chain. Sustain. Prod. Consum. 2019, 17, 241–253. [CrossRef]
23. Chandramowli, S.; Transue, M.; Felder, F.A. Analysis of barriers to development in landfill communities using interpretive
structural modeling. Habitat Int. 2011, 35, 246–253. [CrossRef]
24. Gokarn, S.; Kuthambalayan, T.S. Analysis of challenges inhibiting the reduction of waste in food supply chain. J. Clean. Prod.
2017, 168, 595–604. [CrossRef]
25. Jayant, A.; Azhar, M. Analysis of the Barriers for Implementing Green Supply Chain Management (GSCM) Practices: An
Interpretive Structural Modeling (ISM) Approach. Procedia Eng. 2014, 97, 2157–2166. [CrossRef]
26. Khan, I.; Rahman, Z. Brand experience anatomy in retailing: An interpretive structural modeling approach. J. Retail. Consum.
Serv. 2015, 24, 60–69. [CrossRef]
27. Patidar, L.; Soni, V.K.; Kumar Soni, P. Development of a Framework for Implementation of Maintenance Tools and Techniques
Using Interpretive Structural Modeling. Mater. Today Proc. 2017, 4, 8158–8166. [CrossRef]
28. Mishra, R.P.; Kodali, R.B.; Gupta, G.; Mundra, N. Development of a Framework for Implementation of World-class Maintenance
Systems Using Interpretive Structural Modeling Approach. Procedia CIRP 2015, 26, 424–429. [CrossRef]
29. Raut, R.D.; Gardas, B.B.; Jha, M.K.; Priyadarshinee, P. Examining the critical success factors of cloud computing adoption in the
MSMEs by using ISM model. J. High Technol. Manag. Res. 2017, 28, 125–141. [CrossRef]
30. Agarwal, A.; Shankar, R.; Tiwari, M.K. Modeling agility of supply chain. Ind. Mark. Manag. 2007, 36, 443–457. [CrossRef]
31. Gardas, B.B.; Raut, R.D.; Narkhede, B. Modeling causal factors of post-harvesting losses in vegetable and fruit supply chain: An
Indian perspective. Renew. Sustain. Energy Rev. 2017, 80, 1355–1371. [CrossRef]
32. Balaji, M.; Arshinder, K. Modeling the causes of food wastage in Indian perishable food supply chain. Resour. Conserv. Recycl.
2016, 114, 153–167. [CrossRef]
33. Kaswan, M.S.; Rathi, R. Analysis and modeling the enablers of Green Lean Six Sigma implementation using Interpretive Structural
Modeling. J. Clean. Prod. 2019, 231, 1182–1191. [CrossRef]
34. Ali, S.M.; Hossen, M.A.; Mahtab, Z.; Kabir, G.; Paul, S.K.; ul Haq Adnan, Z. Barriers to lean six sigma implementation in the
supply chain: An ISM model. Comput. Ind. Eng. 2020, 149, 106843. [CrossRef]
35. Tarei, P.K.; Chand, P.; Gupta, H. Barriers to the adoption of electric vehicles: Evidence from India. J. Clean. Prod. 2021, 291, 125847.
[CrossRef]
36. de Oliveira Mota, R.; Godinho Filho, M.; Osiro, L.; Ganga, G.M.D.; de Sousa Mendes, G.H. Unveiling the relationship between
drivers and capabilities for reduced time-to-market in start-ups: A multi-method approach. Int. J. Prod. Econ. 2021, 233, 108018.
[CrossRef]
37. Mukeshimana, M.C.; Zhao, Z.-Y.; Nshimiyimana, J.P. Evaluating strategies for renewable energy development in Rwanda: An
integrated SWOT—ISM analysis. Renew. Energy 2021, 176, 402–414. [CrossRef]
38. Ruiz-Benitez, R.; López, C.; Real, J.C. Environmental benefits of lean, green and resilient supply chain management: The case of
the aerospace sector. J. Clean. Prod. 2018, 167, 850–862. [CrossRef]
39. Xiao, L. Analyzing consumer online group buying motivations: An interpretive structural modeling approach. Telemat. Inform.
2018, 35, 629–642. [CrossRef]
40. Haleem, A.; Luthra, S.; Mannan, B.; Khurana, S.; Kumar, S.; Ahmad, S. Critical factors for the successful usage of fly ash in roads
& bridges and embankments: Analyzing indian perspective. Resour. Policy 2016, 49, 334–348. [CrossRef]
41. Singh, A.N.; Gupta, M.P.; Ojha, A. Identifying critical infrastructure sectors and their dependencies: An Indian scenario. Int. J.
Crit. Infrastruct. Prot. 2014, 7, 71–85. [CrossRef]
42. Kanji, R.; Agrawal, R. Exploring the use of corporate social responsibility in building disaster resilience through sustainable
development in India: An interpretive structural modelling approach. Prog. Disaster Sci. 2020, 6, 100089. [CrossRef]
Sustainability 2021, 13, 8801 26 of 27

43. Jabbour, C.J.C.; Jugend, D.; de Sousa Jabbour, A.B.L.; Govindan, K.; Kannan, D.; Leal Filho, W. “There is no carnival without
samba”: Revealing barriers hampering biodiversity-based R&D and eco-design in Brazil. J. Environ. Manag. 2018, 206, 236–245.
[CrossRef]
44. Kumar, D.T.; Palaniappan, M.; Kannan, D.; Shankar, K.M.; Thresh Kumar, D.; Palaniappan, M.; Kannan, D.; Shankar, K.M.
Analyzing the CSR issues behind the supplier selection process using ISM approach. Resour. Conserv. Recycl. 2014, 92, 268–278.
[CrossRef]
45. Baykasoğlu, A.; Gölcük, İ. Development of a two-phase structural model for evaluating ERP critical success factors along with a
case study. Comput. Ind. Eng. 2017, 106, 256–274. [CrossRef]
46. Jain, V.; Raj, T. Modelling and analysis of FMS productivity variables by ISM, SEM and GTMA approach. Front. Mech. Eng. 2014,
9, 218–232. [CrossRef]
47. Peeters, B.; Kiratli, N.; Semeijn, J. A barrier analysis for distributed recycling of 3D printing waste: Taking the maker movement
perspective. J. Clean. Prod. 2019, 241, 118313. [CrossRef]
48. Gokarn, S.; Choudhary, A. Modeling the key factors influencing the reduction of food loss and waste in fresh produce supply
chains. J. Environ. Manag. 2021, 294, 113063. [CrossRef] [PubMed]
49. Diabat, A.; Govindan, K. An analysis of the drivers affecting the implementation of green supply chain management. Resour.
Conserv. Recycl. 2011, 55, 659–667. [CrossRef]
50. Xu, X.; Zou, P.X.W. Analysis of factors and their hierarchical relationships influencing building energy performance using
interpretive structural modelling (ISM) approach. J. Clean. Prod. 2020, 272, 122650. [CrossRef]
51. Kadam, S.; Bandyopadhyay, P.K. Modelling passenger interaction process (PIP) framework using ISM and MICMAC approach.
J. Rail Transp. Plan. Manag. 2020, 14, 100171. [CrossRef]
52. Sharma, M.; Sehrawat, R. A hybrid multi-criteria decision-making method for cloud adoption: Evidence from the healthcare
sector. Technol. Soc. 2020, 61, 101258. [CrossRef]
53. Attri, R.; Dev, N.; Sharma, V. Interpretive structural modelling (ISM) approach: An overview. Res. J. Manag. Sci. 2013, 2319, 1171.
54. Attri, R. Interpretive structural modelling: A comprehensive literature review on applications. Int. J. Six Sigma Compet. Advant.
2017, 10, 258–331. [CrossRef]
55. Gardas, B.B.; Raut, R.D.; Narkhede, B.E. A state-of the-art survey of interpretive structural modelling methodologies and
applications. Int. J. Bus. Excell. 2017, 11, 505–560. [CrossRef]
56. Anand, A.; Bansal, G. Interpretive structural modelling for attributes of software quality. J. Adv. Manag. Res. 2017, 14, 256–269.
[CrossRef]
57. Guan, L.; Abbasi, A.; Ryan, M.J. Analyzing green building project risk interdependencies using Interpretive Structural Modeling.
J. Clean. Prod. 2020, 256, 120372. [CrossRef]
58. Abuzeinab, A.; Arif, M.; Qadri, M.A. Barriers to MNEs green business models in the UK construction sector: An ISM analysis.
J. Clean. Prod. 2017, 160, 27–37. [CrossRef]
59. Agi, M.A.N.; Nishant, R. Understanding influential factors on implementing green supply chain management practices: An
interpretive structural modelling analysis. J. Environ. Manag. 2017, 188, 351–363. [CrossRef]
60. Luthra, S.; Kumar, S.; Kharb, R.; Ansari, M.F.; Shimmi, S.L. Adoption of smart grid technologies: An analysis of interactions
among barriers. Renew. Sustain. Energy Rev. 2014, 33, 554–565. [CrossRef]
61. Dewangan, D.K.; Agrawal, R.; Sharma, V. Enablers for Competitiveness of Indian Manufacturing Sector: An ISM-Fuzzy MICMAC
Analysis. Procedia Soc. Behav. Sci. 2015, 189, 416–432. [CrossRef]
62. Raci, V.; Shankar, R. Analysis of interactions among the barriers of reverse logistics. Technol. Forecast. Soc. Chang. 2005, 72,
1011–1029. [CrossRef]
63. Mathiyazhagan, K.; Govindan, K.; NoorulHaq, A.; Geng, Y. An ISM approach for the barrier analysis in implementing green
supply chain management. J. Clean. Prod. 2013, 47, 283–297. [CrossRef]
64. Tseng, M.-L. Modeling sustainable production indicators with linguistic preferences. J. Clean. Prod. 2013, 40, 46–56. [CrossRef]
65. Muduli, K.; Govindan, K.; Barve, A.; Kannan, D.; Geng, Y. Role of behavioural factors in green supply chain management
implementation in Indian mining industries. Resour. Conserv. Recycl. 2013, 76, 50–60. [CrossRef]
66. Lin, L.-Z.; Yeh, H.-R. Analysis of tour values to develop enablers using an interpretive hierarchy-based model in Taiwan. Tour.
Manag. 2013, 34, 133–144. [CrossRef]
67. Warshall, S. A theorem on boolean matrices. JACM 1962, 9, 11–12. [CrossRef]
68. Sajid, Z.; Khan, F.; Zhang, Y. Integration of interpretive structural modelling with Bayesian network for biodiesel performance
analysis. Renew. Energy 2017, 107, 194–203. [CrossRef]
69. Bouzon, M.; Govindan, K.; Rodriguez, C.M.T. Reducing the extraction of minerals: Reverse logistics in the machinery manufac-
turing industry sector in Brazil using ISM approach. Resour. Policy 2015, 46, 27–36. [CrossRef]
70. Gholami, H.; Rezaei, G.; Saman, M.Z.M.; Sharif, S.; Zakuan, N. State-of-the-art Green HRM System: Sustainability in the sports
center in Malaysia using a multi-methods approach and opportunities for future research. J. Clean. Prod. 2016, 124, 142–163.
[CrossRef]
71. Singhal, D.; Tripathy, S.; Jena, S.K.; Nayak, K.K.; Dash, A. Interpretive structural modelling (ISM) of obstacles hindering the
remanufacturing practices in India. Procedia Manuf. 2018, 20, 452–457. [CrossRef]
Sustainability 2021, 13, 8801 27 of 27

72. Toktaş-Palut, P.; Baylav, E.; Teoman, S.; Altunbey, M. The impact of barriers and benefits of e-procurement on its adoption
decision: An empirical analysis. Int. J. Prod. Econ. 2014, 158, 77–90. [CrossRef]
73. Denyer, D.; Tranfield, D. Producing a Systematic Review. SAGE Handb. Organ. Res. Methods 2009, 39, 672–688. [CrossRef]
74. Peakall, R.O.D.; Smouse, P.E. GENALEX 6: Genetic analysis in Excel. Population genetic software for teaching and research. Mol.
Ecol. Notes 2006, 6, 288–295. [CrossRef]
75. Chandrupatla, T.R.; Belegundu, A.D.; Ramesh, T.; Ray, C. Introduction to Finite Elements in Engineering; Prentice Hall: Upper
Saddle River, NJ, USA, 2002; Volume 2.
76. Zhang, Y.; Huo, M.; Zhou, J.; Xie, S. PKSolver: An add-in program for pharmacokinetic and pharmacodynamic data analysis in
Microsoft Excel. Comput. Methods Programs Biomed. 2010, 99, 306–314. [CrossRef] [PubMed]
77. Wang, G.; Wang, Y.; Zhao, T. Analysis of interactions among the barriers to energy saving in China. Energy Policy 2008, 36,
1879–1889. [CrossRef]
78. Luthra, S.; Garg, D.; Haleem, A. An analysis of interactions among critical success factors to implement green supply chain
management towards sustainability: An Indian perspective. Resour. Policy 2015, 46, 37–50. [CrossRef]
79. Ansari, M.F.; Kharb, R.K.; Luthra, S.; Shimmi, S.L.; Chatterji, S. Analysis of barriers to implement solar power installations in
India using interpretive structural modeling technique. Renew. Sustain. Energy Rev. 2013, 27, 163–174. [CrossRef]
80. Yadav, D.K.; Barve, A. Analysis of critical success factors of humanitarian supply chain: An application of Interpretive Structural
Modeling. Int. J. Disaster Risk Reduct. 2015, 12, 213–225. [CrossRef]
81. Diabat, A.; Kannan, D.; Mathiyazhagan, K. Analysis of enablers for implementation of sustainable supply chain management—A
textile case. J. Clean. Prod. 2014, 83, 391–403. [CrossRef]
82. D. Vivek, S.; Banwet, D.K.; Shankar, R. Analysis of interactions among core, transaction and relationship-specific investments:
The case of offshoring. J. Oper. Manag. 2008, 26, 180–197. [CrossRef]
83. Cagno, E.; Micheli, G.J.L.; Jacinto, C.; Masi, D. An interpretive model of occupational safety performance for Small- and
Medium-sized Enterprises. Int. J. Ind. Ergon. 2014, 44, 60–74. [CrossRef]
84. Govindan, K.; Palaniappan, M.; Zhu, Q.; Kannan, D. Analysis of third party reverse logistics provider using interpretive structural
modeling. Int. J. Prod. Econ. 2012, 140, 204–211. [CrossRef]
85. Jia, P.; Diabat, A.; Mathiyazhagan, K. Analyzing the SSCM practices in the mining and mineral industry by ISM approach. Resour.
Policy 2015, 46, 76–85. [CrossRef]
86. Shi, Q.; Yu, T.; Zuo, J.; Lai, X. Challenges of developing sustainable neighborhoods in China. J. Clean. Prod. 2016, 135, 972–983.
[CrossRef]
87. Shibin, K.T.; Gunasekaran, A.; Dubey, R. Explaining sustainable supply chain performance using a total interpretive structural
modeling approach. Sustain. Prod. Consum. 2017, 12, 104–118. [CrossRef]
88. Sağ, S.; Kaynak, R.; Sezen, B. Factors Affecting Multinational Team Performance. Procedia Soc. Behav. Sci. 2016, 235, 60–69.
[CrossRef]
89. Rajaprasad, S.V.S.; Chalapathi, P.V. Factors Influencing Implementation of OHSAS 18001 in Indian Construction Organizations:
Interpretive Structural Modeling Approach. Saf. Health Work 2015, 6, 200–205. [CrossRef] [PubMed]
90. Luthra, S.; Haleem, A. Hurdles in Implementing Sustainable Supply Chain Management: An Analysis of Indian Automobile
Sector. Procedia Soc. Behav. Sci. 2015, 189, 175–183. [CrossRef]
91. Sindhu, S.; Nehra, V.; Luthra, S. Identification and analysis of barriers in implementation of solar energy in Indian rural sector
using integrated ISM and fuzzy MICMAC approach. Renew. Sustain. Energy Rev. 2016, 62, 70–88. [CrossRef]
92. Shen, L.; Song, X.; Wu, Y.; Liao, S.; Zhang, X. Interpretive Structural Modeling based factor analysis on the implementation of
Emission Trading System in the Chinese building sector. J. Clean. Prod. 2016, 127, 214–227. [CrossRef]
93. Ahuja, R.; Sawhney, A.; Arif, M. Prioritizing BIM Capabilities of an Organization: An Interpretive Structural Modeling Analysis.
Procedia Eng. 2017, 196, 2–10. [CrossRef]
94. Rajesh, R. Technological capabilities and supply chain resilience of firms: A relational analysis using Total Interpretive Structural
Modeling (TISM). Technol. Forecast. Soc. Chang. 2017, 118, 161–169. [CrossRef]
95. Jothimani, D.; Bhadani, A.K.; Shankar, R. Towards Understanding the Cynicism of Social Networking Sites: An Operations
Management Perspective. Procedia Soc. Behav. Sci. 2015, 189, 117–132. [CrossRef]
96. Rana, N.P.; Barnard, D.J.; Baabdullah, A.M.A.; Rees, D.; Roderick, S. Exploring barriers of m-commerce adoption in SMEs in the
UK: Developing a framework using ISM. Int. J. Inf. Manag. 2019, 44, 141–153. [CrossRef]
97. Mondal, J.; Chakrabarti, S. Insights and anatomy of brand experience in app-based retailing (eRBX): Critical play of physical
evidence and enjoyment. J. Retail. Consum. Serv. 2021, 60, 102484. [CrossRef]
98. Naveed, Q.N.; Ahmad, N.; Qamar, S.; Khan, N.; Naim, A.; Hussain, M.R.; Qureshi, M.R.N.; Alsayed, A.O.; Mohiuddin, K.
Relationship modeling for OSN-based E-Learning Deployment. In Proceedings of the 2019 IEEE 6th International Conference on
Engineering Technologies and Applied Sciences (ICETAS), Kuala Lumpur, Malaysia, 20–21 December 2019; pp. 1–7.
99. Ahmad, N.; Hoda, N.; Alahmari, F. Developing a Cloud-Based Mobile Learning Adoption Model to Promote Sustainable
Education. Sustainability 2020, 12, 3126. [CrossRef]
100. Ahmad, N. The Structural Modeling of Significant Factors for Sustainable Cloud Migration. Int. J. Intell. Eng. Syst. 2021, 14, 1–10.
[CrossRef]