A Comparative Analysis of Optimization Techniques
A Comparative Analysis of Optimization Techniques
A Comparative Analysis of Optimization Techniques
net/publication/290978235
CITATIONS READS
16 11,942
2 authors, including:
Kirti Seth
INHA university in Tashkent, Uzbekistan
57 PUBLICATIONS 344 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Kirti Seth on 29 June 2016.
6
International Journal of Computer Applications (0975 – 8887)
Volume 131 – No.10, December2015
Methodology: ABC inhibits the behaviour of natural ants 2.2.Particle Swarm Intelligence
which is based on the random behaviour of ants as they
wander randomly to find path [5]. ABC aims at generating Optimization
the optimal number of test cases with fewer amounts of time Particle Swarm Intelligence Optimization (PSO) simulates
and resources. Bees tend to find out the food sources with swarming behaviour observed in herds of animals, flocks of
higher nectar amount. Depending on the experiences of birds etc., where social sharing of information takes place
themselves and their nest mates Employed and Onlookers and individuals gain from the discoveries and experiences of
bees select food sources [6]. other companions while searching for food. System is
initialized with a random population, known as particles.
Performance: ABC is a simple approach, yet it deals with During optimization particles explore a D-dimensional
complex problems efficiently. This technique uses fewer space. Each particle maintains its own current position,
parameters as compared to other search algorithms [7]. It is current velocity and best position so far [16].The iterative
efficient to hybridize ABC with other techniques to increase update of rules leads to a stochastic manipulation of
its performance. Some hybridized ABC are GABC [8], velocities.
CABC [9], P-DABC [10], hybrid ABC [11], ABC with
neural networks [12] and many more. Evolution: Kennedy and Eberhart introduced a heuristic
global optimization method known as PSO in year 1995 in a
Applications: There are many applications of ABC research paper named “Particle Swarm Optimization” [17].
algorithm to real world and benchmark optimization Since then, various modifications and improvements are
problems. It was applied to integer programming problems done. Some of them are listed in Table 2.
[13], Travelling Salesman problem [14], Bio- Informatics
applications [15], scheduling applications, clustering, image Methodology: PSO is a global optimization algorithm which
processing and many more. deals with problems which tends to find the best solution as a
2007 Karaboga et al. Modified artificial bee colony optimization (MABC) Constrained optimization
2009 P.W.Tsai et al. Interactive artificial bee colony optimization (IABC) Numerical optimization
2010 M. Sonmez. Artificial bee colony with adaptive penalty function (ABC-AP) Weight of truss structures
2011 N. Taspnar Partial transmit sequences (PTSs) based on ABC (ABC-PTS) Peak-to-average power
ratio
2011 M. El-Abd. ABC with the concept of opposition number based optimization Black box optimization
(OABC)
2014 Deepak Rai et al. Honey bee mating optimization algorithm (HBMO) Test case optimization
point in a D-dimensional space. In PSO, particles fly in the Applications: PSO have a vast range of applications. It
respective problem space by following the current optimum may be applied to optimize multi-objective problems [20],
flying particles. Each of the particle keeps a track of its T-way software testing [21], mini-max problems [22],
position in terms of coordinates in the problem space which image classification, job shop scheduling, artificial neural
are contributes to the best solution so far. Particles then networks, gesture recognition and many more.
move in solution space and they are evaluated after each
iteration through some fitness functions. 2.3.Ant Colony Optimization Algorithm:
Ant Colony Optimization (ACO) is an intuitive algorithm
Performance: PSO is a population based algorithm based influenced by the behaviour of natural ants. Ants are blind
on the cooperation of each particle. The convergence ability and have the tendency to find the shortest path from their
of PSO is faster than of other optimization techniques [18]. initial position to the food source. Pheromone, the chemical
It requires fewer parameters for the calculation of used for unintended communication between ants helps to
optimizing value. Number of particles may be decreased to search the shortest path.
increase the performance [19]. It may be hybridized to
further increase its performance. Evolution: In year 1991, Ant System was first proposed by
7
International Journal of Computer Applications (0975 – 8887)
Volume 131 – No.10, December2015
Marco Dorigo in his doctoral thesis. Afterwards, some Performance: ACO has the ability to find optimal solution
improvements were made into Ant System as the in less computational time. The performance of ACO may
introduction of Elitist ants [23], the ranking of ants [24]. be improved by introducing approaches like modification of
One of the major developments is the description of ACO transition rule, parallel ACO [26]. It may be hybridized
metahueristic in year 1999 [25]. Other modifications and with other techniques for better results.
variations of ACO are given in the Table 3.
Applications: The first combinatorial problem solved using
Methodology: Ant Colony Algorithm is inspired from the ACO was Travelling Salesman Problem (TSP) in 1991 by
behaviour of natural ants. Ants resolve their problems by Dorigo in his Phd dissertation. The nest two applications of
collaborating with each other using pheromone. Ants place ACO were the Quadratic Assignment Problem (QAP) [27]
pheromone on the ground while walking and each ant and Job Shop Scheduling [28] 1994. Then it was applied to
follow a direction where pheromone intensity is high. When network routing applications [29], vehicle routing problems
an ant recognizes an obstacle in the path it changes the [30], sequential ordering, graph colouring problems and
direction and tries to find new shortest path. design of algorithms for knowledge representation
structures.
Table 2: Evolution of PSO Algorithm
Year Author Name of algorithm Problem constrained
2001 Van Den Bergh Multi-start particle swarm optimization (MPSO) Encountering a global
minimiser
2002 Van Den Bergh Guaranteed Convergence Particle Swarm Optimization Convergence to local
(GCPSO) minimum
2005 Stefan Janson and Martin Hierarchical Particle Swarm Optimization (HPSO) Better solution
Midden Dorf
2005 Chunning Yang and Dan New Particle Swarm Optimization Better solution
Simon
2007 Hui Wang et al. Opposition based Particle Swarm Optimization (OPSO) To accelerate the
convergence
2008 Marco A. Montes Dea et al. Fully Informed Particle Swarm Optimization (FIPSO) Optimization problems
2009 George I. Evers et al. Regrouping Particle Swarm Optimization (RPSO) Premature convergence
2011 X.S. Yang, s. Fong and S. Accelerated Particle Swarm Optimization (APSO) Accelerate convergence
Deb
2.4.Genetic Algorithm: (2) Selection, (3) Crossover and (4) Mutation. Initialization
Genetic Algorithm is an adaptive heuristic search method operator creates the initial population and assigns a fitness
based on the population genetics. It is probabilistic search function which evaluates the fitness value. The selection
method inspired from the process of natural selection and operator chooses the chromosomes from population for
reproduction. It is used to generate solutions of mating. Crossover operator is used for sharing the
optimization search problems. It belongs to the class of information between two chromosomes. Mutation operator
evolutionary algorithms which uses initialization, selection, alters one or more gene values of a chromosome from its
crossover and mutation. initial state. The process of evolution is repeated until end
condition of the problem is satisfied.
Evolution: John Holland introduced Genetic Algorithm in
his book named „Adaptation in Natural and Artificial Performance: GA is a search technique used to find the
Systems‟ in 1975 [31]. Then John Koza has used genetic exact or approximate solutions to an optimization problem.
algorithm in 1992 to evolve programs to perform certain GA exhibits “inherent parallelism” as the evaluation of
tasks. He called his method as "Genetic Programming" individuals within a population is conducted
(GP). Some improvements in GA are listed below in Table simultaneously. It generally finds the global optima in
4. complex spaces, hence GA is fast [32]. GA may be
hybridized with any other search method to achieve an
Methodology: A GA starts with a set of solutions called as optimization goal.
population. It consists of four operators: (1) Initialization,
8
International Journal of Computer Applications (0975 – 8887)
Volume 131 – No.10, December2015
1996 Stutzle , Hoos Max-Min Ant System (MMAS) TSP and QAP
1997 Bullnheimer, Hartl and Rank Based Ant System(AS rank) TSP
Strauss
2007 Chunhui Zhao Bing Qi Multiple Ant Colony Optimization Resource allocation in
network sessions
2008 Z Hu, J Zhang and Y Li Continuous Orthogonal Ant Colony Continuous optimizing
problems
2009 Bin Y, Zhang-Zhen and Improved Ant Colony Optimization Vehicle routing
Baozhen Y
2010 Ku Ruhana and Ku Interacted Multiple Ant Colony Optimization Different instances of
Mahamud Alaa Aljanaby TSP
2012 Gupta DK, Arora Y et. Al Recursive Multiple Ant Colony Optimization Estimation of parameters
of a function
1995 N. Srinivas and K. Deb Non-Dominated Sorting Genetic Algorithms (NSGA) Multi-objective
optimization
9
International Journal of Computer Applications (0975 – 8887)
Volume 131 – No.10, December2015
2011 Manuel Chica et al Advanced Multi-Objective Genetic Algorithm time and space assembly
line balancing problem
2012 José Fernando Gonçalves, Parallel Multi Population Biased Random-Key Genetic Container loading
Mauricio G.C. Resende Algorithm problem
2014 Camila Silva de Magalhães Dynamic Nitching Genetic Algorithm docking highly flexible
et al ligands
Sigmoid Function
ABC x CABC(Combinatorial ABC)
DABC(Discrete ABC)
Sigmoid Function
Crossover Operator
Lagrangian Relaxation
Lagrangian Decomposition
ACO x Simulated Annealing
GA x Not Required
3. SUMMARY 5. REFERENCES
All the algorithms discussed in the above section are [1] K.K.Aggarwal & Yogesh Singh, “Software
optimization problems. These are compared with each other Engineering Programs Documentation, Operating
and every algorithm has its own advantages. Genetic Procedures,” New Age International Publishers,
Algorithm is much more popular because of its parallel Revised Second Edition – 2005.
computation. One more advantage of Genetic Algorithm is
that it may handle both continuous and discrete variable [2] Chen, Jun, and Mahdi Mahfouf. "Artificial immune
without any gradient information where as all other systems as a bio-inspired optimization technique and
techniques may give best performance for continuous its engineering applications." Handbook of Research
problems but need slight on Artificial Immune Systems and Natural Computing:
Applying Complex Adaptive Technologies: Applying
modifications to handle discrete variables [36]. The Complex Adaptive Technologies (2009): 22.
variations used are shown in the Table 5.
[3] Philip T Cox and Baoming Song, “A formal Model for
4. CONCLUSION Component-Based Software”, IEEE Computer
In this critical review, we examined various optimization Society, Document number 07695-474-4/01, 2001,
techniques by a comparative analysis based on evolution, pp.304-310.
methodology, performance and applications. In this paper, [4] Karaboga, Dervis. An idea based on honey bee swarm
we find that these algorithms may be applied in various for numerical optimization. Vol. 200. Technical
domains whether using as a direct approach or as any report-tr06, Erciyes university, engineering faculty,
modified version. In future, new improved algorithms can computer engineering department, 2005.
be found for different scope areas.
[5] Alaya, I., C. Solnon, and K. Ghedira. "Ant Colony
Optimization for Multi-Objective Optimization
10
International Journal of Computer Applications (0975 – 8887)
Volume 131 – No.10, December2015
Problems." In Tools with Artificial Intelligence, 2007. [18] Hemlata S Urade and Prof. Rahila Patel. Article: Study
ICTAI 2007. 19th IEEE International Conference on, and Analysis of Particle Swarm Optimization: A
vol. 1, pp. 450-457. IEEE, 2007. Review. IJCA Proceedings on 2nd National
Conference on Information and Communication
[6] Kulkarni, Nandakishore J., K. Venkat Naveen, Puneet Technology NCICT(4):1-5, November 2011.
Singh, and Praveen Ranjan Srivastava. "Test Case
Optimization Using Artificial Bee Colony [19] van den Bergh, Frans, and A. P. Engelbrecht. "A new
Algorithm." Advances in Computing and locally convergent particle swarm optimizer."
Communications (2011): 570-579. In Proceedings of the IEEE international conference
on systems, man, and cybernetics, vol. 7, pp. 6-9.
[7] Karaboga, Dervis, and Bahriye Akay. "A comparative 2002.
study of artificial bee colony algorithm." Applied
Mathematics and Computation 214, no. 1 (2009): 108- [20] Parsopoulos, Konstantinos E., and Michael N.
132. Vrahatis. "Particle swarm optimization method in
multiobjective problems." In Proceedings of the 2002
[8] Zhu, Guopu, and Sam Kwong. "Gbest-guided artificial ACM symposium on Applied computing, pp. 603-607.
bee colony algorithm for numerical function ACM, 2002.
optimization." Applied Mathematics and
Computation 217, no. 7 (2010): 3166-3173. [21] Ahmet, Bestoun, and Kamal Zamli. "A Greedy
Particle Swarm Optimization Strategy for T-way
[9] Karaboga, Dervis, and Beyza Gorkemli. "A Software Testing." Journal of Artificial
combinatorial artificial bee colony algorithm for Intelligence 5.2 (2012): 85-90.
traveling salesman problem." In Innovations in
Intelligent Systems and Applications (INISTA), 2011 [22] Laskari, E. C., K. E. Parsopoulos, and M. N. Vrahatis.
International Symposium on, pp. 50-53. IEEE, 2011. "Particle swarm optimization for minimax problems."
In Evolutionary Computation, 2002. CEC'02.
[10] Li, Jun-Qing, Quan-Ke Pan, and Kai-Zhou Gao. Proceedings of the 2002 Congress on, vol. 2, pp. 1576-
"Pareto-based discrete artificial bee colony algorithm 1581. IEEE, 2002.
for multi-objective flexible job shop scheduling
problems."The International Journal of Advanced [23] Dorigo, Marco, and Luca Maria Gambardella. "Ant
Manufacturing Technology 55, no. 9-12 (2011): 1159- colony system: a cooperative learning approach to the
1169. traveling salesman problem." Evolutionary
Computation, IEEE Transactions on 1, no. 1 (1997):
[11] Li, Junqing, Quanke Pan, and Shengxian Xie. 53-66.
"Flexible job shop scheduling problems by a hybrid
artificial bee colony algorithm." In Evolutionary [24] Bullnheimer B, Hartl RF, Strauß C ” A new rank based
Computation (CEC), 2011 IEEE Congress on, pp. 78- version of the ant system - a computational study.”
83. IEEE, 2011. Central European Journal for Operations Research and
Economics 7(1997): 25–38.
[12] Karaboga, Dervis, and Celal Ozturk. "Fuzzy clustering
with artificial bee colony algorithm." Scientific [25] Dorigo M, Di Caro G,” The ant colony optimization
research and Essays 5, no. 14 (2010): 1899-1902. meta-heuristic”, McGraw-Hill Ltd., UK, Maidenhead,
UK, England, (1999) pp 11–32.
[13] Akay, Bahriye, and Dervis Karaboga. "Solving integer
programming problems by using artificial bee colony [26] Hong Liu, Ping Li and Yu Wen, “Parallel Ant colony
algorithm." In AI* IA 2009: Emergent Perspectives in optimization algorithm,” in Proc. WCICA, 2006, pp
Artificial Intelligence, pp. 355-364. Springer Berlin 3222-3226.
Heidelberg, 2009.
[27] Maniezzo, Vittorio, and Alberto Colorni. "The ant
[14] Jiao, Jian, Shan Yao, and Chunehe Xia. "Application system applied to the quadratic assignment
for artificial bee colony algorithm in migration of problem." Knowledge and Data Engineering, IEEE
mobile agent." In Advanced Intelligent Computing Transactions on 11.5 (1999): 769-778.
Theories and Applications, pp. 232-238. Springer
Berlin Heidelberg, 2010. [28] Colorni, Alberto, et al. "Ant system for job-shop
scheduling." Belgian Journal of Operations Research,
[15] C. Vargas Ben´ıtez and H. Lopes. Parallel artificial bee Statistics and Computer Science 34.1 (1994): 39-53.
colony algorithm approaches for protein structure
prediction using the 3dhp-sc model. Intelligent [29] R. Schoonderwoerd, O. Holland, J. Bruten et L.
Distributed Computing IV, pages 255-264, 2010. Rothkrantz, Ant-based load balancing in
telecommunication networks, Adaptive Behaviour,
[16] J. Kennedy and R. C. Eberhart. Particle swarm volume 5, numéro 2, pages 169-207, 1997.
optimization. In Proceedings of the IEEE International
Conference on Neural Networks, volume 4, pages [30] Bullnheimer, Bernd, Richard F. Hartl, and Christine
1942–1948 vol.4. IEEE Press, 1995. Strauss. "An improved ant System algorithm for
thevehicle Routing Problem." Annals of operations
[17] Windisch, Andreas, Stefan Wappler, and Joachim research 89 (1999): 319-328.
Wegener. "Applying particle swarm optimization to
software testing." In Proceedings of the 9th annual [31] Holland, J.H., “Adaptation in natural and artificial
conference on Genetic and evolutionary computation, systems”, The university of Michigan press, 1975.
pp. 1121-1128. ACM, 2007. [32] Davis, L. (1991): Handbook of Genetic
Algorithms. Van Nostrand Reinhold. New York, NY.
11
International Journal of Computer Applications (0975 – 8887)
Volume 131 – No.10, December2015
[33] Zhang, J., Lo, W.L., and Chung, H., Problems using Genetic algorithms”, Evolutionary
"Pseudocoevolutionary Genetic Algorithms for Power Computation, 2002. CEC ‟02. Proceedings of the
Electronic Circuits Optimization", IEEE Trans 2002Congress on, 1, pp 175 –180, 2002.
Systems, Man, and Cybernetics, Part C., Vol.36, No.4,
July 2006, pp. 590–598. [36] Krause, Jonas, Jelson Cordeiro, Rafael Stubs
Parpinelli, and Heitor Silverio Lopes. "A survey of
[34] Minglun G. and Yee-Hong Y., “Multi-resolution swarm algorithms applied to discrete optimization
Stereo Matching using Genetic Algorithm”, Stereo problems." Swarm Intelligence and Bio-inspired
and Multi-Baseline Vision, 2001. (SMBV 2001). Computation: Theory and Applications. Elsevier
Proceedings. IEEE Workshop on, pp 21 –29, 2001. Science & Technology Books (2013): 169-191.
[35] Madureira, A.; Ramos, C.; do Carmo Silva, S.,”A
Coordination Mechanism for Real World Scheduling
IJCATM : www.ijcaonline.org
12