Artificial Bee Colony (ABC), Harmony Search and Bees Algorithms on Numerical Optimization Optimization D. Karaboga, B. Akay Erciyes University, The Dept. of Computer Engineering, 38039, 38039, Melikgazi, Kayseri, Turkiye
Abstract
In this paper, performances per formances of basic Artificial Bee Colony, Harmony Search and Bees algorithms and Improved Bees algorithm were compared on unimodal and multimodal well-known benchmark problems. Effect of problem dimensionality on the performance of the algorithms was investigated and the algorithms were compared to each other in terms of the number of control parameters to be tuned. Keywords: Artificial Bee Colony, Harmony Search
1. Introduction 2. Artificial Bee Colony Algorithm
Nature is inspiring inspiring researchers researchers to develop models for solving their problems. Optimization is an instance field in which these models are frequently developed and applied. Genetic algorithm simulating natural selection and genetic operators, Particle Swarm Optimization algorithm simulating flock of birds and school of fishes, Artificial Immune Immune System simulating the cell masses of immune system, Ant Colony Optimization algorithm simulating foraging behaviour of ants and Artificial Bee Colony algorithm simulating foraging behaviour of honeybees are typical examples of nature inspired optimization algorithms. Another algorithm algori thm simulating the foraging behaviour of bees by a different algorithmic model is Bees Algorithm (BA) Harmony Search is also a recent optimization algorithm simulating improvisation process of musicians. Although there are many algorithms, most of these algorithms need fine tuning of control parameters, are successful only a class of problems. Furthermore, on a kind of problem, although it shows superior performance on on low dimensions, the algorithm can not preserve its superior performance on high dimensional case. In this work, the performances of Artificial Bee Colony, Harmony Search and Bees algorithms algorithms were compared to each other on varying dimensions from low to high. Paper is organized as follows, in consequent sections Artificial Bee Colony Algorithm, Harmony Search and Bees algorithm are articulated, in Section 5, experimental results are presented and finally the paper is concluded..
Artificial Bee Colony(ABC) algorithm, proposed by Karaboga Karaboga in 2005 for real parameter parameter optimization, is a recently introduced optimization algorithm and simulates the foraging behaviour of bee colony [1] for unconstrained optimization problems [1]–[5]. For solving constrained optimization problems, a constraint handling method was incorporated incorporated with the algorithm [6]. In a real bee colony, there are some tasks performed by specialized individuals. These specialized bees try to maximize the nectar amount stored in the hive by performing efficient division of labour and self-organization. The minimal model of swarm-intelligent forage selection in a honey bee colony, that ABC algorithm adopts, consists of three kinds of bees: employed bees, onlooker bees, and scout bees. Half of the colony comprises employed bees and the other half includes the onlooker bees. Employed bees are responsible from exploiting the nectar sources explored before and giving information to the other waiting bees (onlooker bees) in the hive about the quality of the food source site which they are exploiting. Onlooker bees wait in the hive and decide a food source to exploit depending on the information shared by the employed bees. Scouts randomly search the environment in order to find a new food source depending on an internal motivation or possible external clues or randomly. Main steps of the ABC algorithm simulating these behaviour are given below:
1: Initialize the food source positions. 2: Each employed bee produces a new food source in her food source site and exploits the better source. 3: Each onlooker bee selects a source depending on the quality of her solution, produces a new food s ource in selected food source site and exploits the better source. 4: Determine the source to be abandoned and allocate its employed bee as scout for searching new food sources. 5: Memorize the best food source found so far. 6: Repeat steps 2-5 until the stopping criterion is met.
Solutions of the optimization process correspond to musicians and the harmony of the notes generated by a musician corresponds to the fitness of the solution [7]. Main steps of the algorithm are given below: 1: Initialize the problem and algorithm parameters. 2: Initialize the harmony memory. 3: Improvise a new harmony. 4: Update the harmony memory. 5: Repeat steps 3-4 until the stopping criterion is met In HS algorithm, in each improvisation, a new harmony ( x ′ ) is produced and this new harmony competes with the worst harmony of the memory. In improvisation phase is a new harmony is produced by (4) for each component (j =1, . . .,D) r
In first step of the algorithm, x i (i = 1 , . . . , SN) r
solutions are randomly produced in the range of parameters where SN is the number of the food sources. In the second step of the algorithm, for each employed bee, whose total number equals to the half of the number of food sources, a new source is produced by (1): υ ij
= x ij +
φij
(1)
(x ij - x kj )
where φ ij is a uniformly distributed real random
if (rand ≥ phcmr ) x j' = x jmin + ( x jmax − x jmin ) * rand else { x j' = x int (rand*SN)+1 ,j if ( rand < ppar )
number within the range [-1,1], k is the index of the solution chosen randomly from the colony (k = int(rand ∗ SN) + 1), j = 1, . . .,D and D is the dimension of the problem. After producing υ i , this
(4)
{ x j' = x j' ± bw * rand
r
}
r
new solution is compared to x i solution and the
}
employed bee exploits the better source. In the third step of the algorithm, an onlooker bee chooses a food source with the probability (2) and produces a new source in selected food source site by (1). As for employed bee, the better source is decided to be exploited. fit i (2) p i =
p hmcr
is the rate of choosing from memory,
p par
is the pitch adjustment rate, bw is the distance bandwidth which determines the amount of maximum change for pitch adjustment. 4. Bees Algorithm
SN
∑ fit
j
j =1
r
where fit i is the fitness of the solution xi . After all onlookers are distributed to the sources, sources are checked whether they are to be abandoned. If the number of cycles that a source can not be improved is greater than a predetermined limit, the source is considered to be exhausted. The employed bee associated with the exhausted source becomes a scout and makes a random search in problem domain by (3).
x ij = x jmin + ( x jmax − x jmin ) * rand
(3)
3. Harmony Search
Harmony search (HS) is a recent metaheuristic that simulates the improvisation process of musicians.
In Bees algorithm (BA) [8], after initializing the population of bees by random sites, bees are ranked according to the fitnesses of their sites. First “e” of the sites are classified as elite sites and the “nep” number of bees are sent to these elite sites. These bees produce new sites in the neighbourhood of the elite sites. After sending bees to the elite sites, neighbourhood of the following sites (selected sites, m-e) depending on their fitnesses are searched . The number of bees sent to the selected sites is “nsp”. Remaining bees (n-m) are classified as scout and allowed to conduct random search. Totally, the algorithm requires seven control parameters including the number of scout bees (n), number of sites selected out of n visited sites (m), number of best sites out of m selected sites (e), number of bees recruited for best e sites (nep), number of bees recruited for the other (m-e) selected sites (nsp), initial size of patches (ngh) and maximum iteration number. Main steps of the algorithm are given below [8]:
1: Initialise population with random solutions. 2: Evaluate fitness of the population. 3: Repeat 4: Select sites for neighbourhood search. 5: Recruit bees for selected sites (more bees for best e sites) and evaluate fitnesses. 6: Select the fittest bee from each patch. 7: Assign remaining bees to search randomly and evaluate their fitnesses. 8: Until stopping criterion is met. BA was improved by using shrinking process. In this paper, this model is called Improved Bees Algorithm (IBA). In IBA, the patch size (ngh) is gradually reduced by dividing it by a shrinking constant at each iteration. Therefore, a new control parameter is introduced to the algorithm, called shrinking constant. 5. Experiments
In the experiments, results of basic HS, BA and ABC algorithms are compared to each other on unimodal (Sphere, Rosenbrock) and multimodal (Griewank,Rastrigin, Ackley) functions with different dimensions. Beside, the results of IBA are provided. For HS algorithm, parameters are set to the values recommended in [9]. Size of the harmony memory was 5, p hmcr = 0.9, p par = 0.3, bw = 0.01 and the maximum improvisation number was 50000 for all test problems. For BA, n=40, e=5, m=15, nep=25, nsp=25, ngh={0.5,3} and maximum iteration number was 125. BA was implemented based on the MATLAB code given in the site dedicated to the BA. Therefore, maximum evaluation number was 50000 for all functions. For IBA, n=40, e=1, m=3, nep=50, nsp=10, maximum iteration number was 468, initial ngh was 6, and shrinking constant (sc) was 1.05. For ABC algorithm, number of food sources was 20 and maximum cycle number was 2500 in order to equalize the maximum evaluation number to 50000 as for HS and BA. Limit control parameter was 100 for all problems. Each of the experiments was repeated 30 times with different random seeds, and the mean function values of the best s olutions and the standard deviations throughout the optimization runs was recorded under different dimensions. In the experiments, mean and standard deviations of 30 runs with different random seeds for 5, 10, 30, 50 and 100 dimensions were reported on Table II for Sphere function, on Table III for Rosenbrock function, on Table IV for Rastrigin function, on Table V for Griewank function and on Table VI for Ackley function. From the tables, it can be seen that ABC algorithm outperforms HS, BA and IBA on all functions and on all dimensions. Only on unimodal Sphere function ABC algorithm and IBA produced similar results. Results reported on tables are plotted on Figures 1(a)-1(e). Both from tables and figures, it can be said that the performance of HS, BA and IBA
algorithms are more sensitive to the increment in problem dimensions as compared to ABC algorithm. ABC algorithm produced reasonable results even for high dimensions for both unimodal and multimodal functions. Another issue that is a drawback of HS, BA and IBA is that they employ much more control parameters than ABC algorithm does. Except for common control parameters (population size and maximum iteration number), ABC algorithm has only one control parameter, limit, while HS algorithm has three control parameters: p hmcr , p par , bw ; BA has five control parameters: e, m, nep, nsp, ngh and IBA has six control parameters including the parameters of BA and a shrinking constant. The performances of BA and IBA are highly dependent to the control parameters. We have tested the BA for ngh=0.5 and 3. The results were significantly different from each other as can be seen from the tables and figures. Especially on Sphere and Rosenbrock functions, different ngh values influence the superiorness of the BA over HS. IBA has better performance with respect to basic BA. 6. Conclusion
In this work, we investigated the performances of Harmony Search, Bees algorithms and Artificial Bee Colony algorithm. From the results, Artificial Bee Colony algorithm is superior over Bees algorithms and Harmony Search. Artificial Bee Colony algorithm preserves producing reasonable results even for high dimensions. Artificial Bee Colony algorithm employs less control parameter to be tuned with respect to Bees Algorithms and Harmony Search. References [1] D. Karaboga. An idea based on honey bee swarm for numerical optimization. Technical Report TR06, Erciyes University, Engineering Faculty, Computer Engineering Department, 2005. [2] B. Basturk and D. Karaboga. An artificial bee colony (abc) algorithm for numeric function optimization. In IEEE Swarm Intelligence Symposium 2006, Indianapolis, Indiana, USA, May 2006. [3] D. Karaboga and B. Basturk. A powerful and efficient algorithm for numerical function optimization: Artificial bee colony (abc) algorithm. Journal of Global Optimization, 39(3):459–471, 2007. [4] D. Karaboga and B. Basturk. On the performance of artificial bee colony (abc) algorithm. Applied Soft Computing, 8(1):687–697, 2008. [5] D. Karaboga and B. Akay. A comparative study of artificial bee colony algorithm. Applied Mathematics and Computation, In Press, 2009. [6] D. Karaboga and B. Basturk. Advances in Soft Computing: Foundations of Fuzzy Logic and Soft Computing, volume 4529/2007 of LNCS, chapter Artificial Bee Colony (ABC) Optimization Algorithm for Solving Constrained Optimization Problems, pages 789–798. Springer-Verlag, 2007. [7] K.S. Lee and Z.W. Geem. A new meta-heuristic
algorithm for continues engineering optimization: harmony search theory and practice. Comput. Meth. Appl. Mech. Eng, 194:39023933, 2004. [8] Pham D.T., Ghanbarzadeh A., Koc E., Otri S., Rahim S., and M.Zaidi "The Bees Algorithm - A Novel Tool for Complex Optimisation Problems"", Proceedings of
IPROMS 2006 Conference, pp.454-461 [9] M.G. H. Omran and M. Mahdavi. Global-best harmony search. Applied Mathematics and Computation, 198(2):643–656, 2008
Table 1:Test functions used in the experiments
Function Sphere
Formulae
f ( x ) = r
n
∑ x
2 i
Range
Minimum Value
[-100,100]
f (0) = 0
[-30,30]
f (1) = 0
[-600,600]
f (0) = 0
[-5.12,5.12]
f (0) = 0
[-32,32]
f (0) = 0
r
i =1
Rosenbrock
n −1
f ( x ) = r
∑ [100( x
i +1
− x i2 ) 2 + ( x i − 1) 2 ]
r
i =1
Griewank
f ( x ) = r
∑ x − ∏ 4000 1
2 i
i =1
Rastrigin
n
n
i =1
⎛ x ⎞ cos⎜⎜ i ⎟⎟ + 1 ⎝ i ⎠
n
f ( x ) = r
∑ [ x
2 i
− 10 cos(2π x i ) + 10]
r
r
i =1
Ackley
⎛ 1 ⎜ f ( x ) = −20 exp − 0.2 ⎜⎜ n ⎝ r
⎞ ⎛ 1 2 ⎟ x i − exp⎜ ⎟⎟ ⎜n i =1 ⎝ ⎠ n
∑
n
∑ i =1
⎞ ⎟ ⎠
cos(2π x i ) ⎟ + 20 + e
r
Table 2: Mean and standard deviations of the results obtained by HS, BA, IBA and ABC algorithms for Sphere function, D:Dimension of the problem. D 5 10 30 50 100 HS Mean 3.20E-10 6.45E-8 7.21 546.25 19014.69 Std 2.89E-10 3.07E-8 3.62 92.69 1782.61 BA(ngh=0.5) Mean 0.006568 0.097978 2.09 196.76 27673.34 Std 0.002707 0.026708 0.288142 153.02 3103.98 BA (ngh=3) Mean 0.213793 3.046319 50.19 173.03 1012.579 Std 0.078865 0.887558 7.36 12.55 61.06 IBA Mean 3.915E-17 4.95E-17 2.92E-16 5.39E-16 1.45E-15 Std 1.24E-17 2.30E-17 6.77E-17 1.07E-16 1.63E-16 ABC Mean 4.30E-17 7.36E-17 4.69E-16 1.19E-15 1.99E-6 Std 1.07E-17 4.43E-17 1.07E-16 4.68E-16 2.26E-6 Table 3: Mean and standard deviations of the results obtained by HS, BA, IBA and ABC algorithms for Rosenbrock function, D:Dimension of the problem. D 5 10 30 50 100 HS Mean 5.94 6.52 382.33 24681.85 14503117.07 Std 6.71 8.16 529.04 10212.03 2157165.22 BA(ngh=0.5) Mean 1.53 16.96 223.32 1073.41 12567.00 Std 0.59 2.09 21.93 166.35 1969.08 BA (ngh=3) Mean 17.81 458.94 31180.77 216634.16 3332501.65 Std 8.24 140.31 7652.24 38909.08 436022.59 IBA Mean 0.454522 10.99 75.72 630.28 641.55 Std 1.538558 25.53 115.95 1195.67 820.33 ABC Mean 0.232567 0.462018 0.997689 4.33 112.20 Std 0.223950 0.543544 1.520233 5.48 69.19
Table 4: Mean and standard deviations of the results obtained by HS, BA, IBA and ABC algorithms for Rastrigin function, D:Dimension of the problem.
HS BA(ngh=0.5) BA(ngh=3) IBA ABC
Mean Std Mean Std Mean Std Mean Std Mean Std
5 6.07E-8 5.52E-8 13.56 5.05 7.81 1.62 4.58 2.31 4.34E-17 1.10E-17
10 1.05E-5 5.23E-6 59.58 8.58 46.75 5.10 21.99 7.46 5.77E-17 2.98E-17
D 30 0.74 0.70 290.54 14.56 306.03 14.38 127.72 24.89 4.80E-5 0.000243
50 37.60 4.87 586.22 21.03 626.72 26.61 271.62 32.70 0.472339 0.492378
100 315.00 23.30 1376.48 30.43 1482.18 26.52 649.07 45.17 14.56 4.18
Table 5: Mean and standard deviations of the results obtained by HS, BA, IBA and ABC algorithms for Griewank function, D:Dimension of the problem. D 5 10 30 50 100 HS Mean 0.026028 0.056974 1.09 5.81 178.35 Std 0.013841 0.030178 0.039666 0.916130 19.79 BA(ngh=0.5) Mean 9.17 44.08 273.47 662.82 1812.67 Std 4.60 11.05 31.96 65.08 94.78 BA(ngh=3) Mean 4.92 0.525676 1.00 3.28 295.318 Std 1.562813 0.078877 0.017153 1.52 34.07 IBA Mean 3.14 1.04 6.68 134.05 793.00 Std 1.41 1.13 6.43 24.14 79.61 ABC Mean 4.04E-17 6.96E-17 5.82E-6 0.572165 13.12 Std 1.12E-17 4.06E-17 3.13E-5 0.921611 6.30
Table 6: Mean and standard deviations of the results obtained by HS, BA, IBA and ABC algorithms for Ackley function, D:Dimension of the problem. D 5 10 30 50 100 HS Mean 2.68E-5 0.000276 0.942645 5.28 13.20 Std 1.24E-5 7.58E-5 0.562522 0.402540 0.489560 BA(ngh=0.5) Mean 16.84 18.53 19.55 20.13 20.53 Std 2.09 1.05 0.160357 0.099509 0.060243 BA(ngh=3) Mean 1.96 3.88 6.69 6.69 20.24 Std 0.336423 0.255085 0.251600 0.251600 0.277258 IBA Mean 6.35E-10 0.067111 1.75 8.43 18.88 Std 9.77E-11 0.361401 0.931689 7.70 0.850277 ABC Mean 9.64E-17 3.51E-16 3.86E-15 4.38E-8 0.013215 Std 5.24E-17 6.13E-17 3.16E-15 4.65E-8 0.012991
(a) Sphere
(c) Rastrigin
(e) Ackley Fig. 1. Results for functions for different dimensions
(b) Rosenbrock
(d) Griewank
