Benchmarking of Heuristic Optimization Methods Alexandr Štefek * *
University of Defence, Faculty of Military Technology, Brno, Czech Republic, e-mail:
[email protected]
Abstract — It is short history when many heuristic optimization methods appear. As example Particle swarm optimization method (PSO) or Repulsive particle swarm optimization method (RPSO), Gravitational search algorithm (GSA), Central force optimization (CFO), Harmony search algorithm (HAS) etc. Those methods are working differently but all of them can optimize same problems. There is general question: Exists any standard benchmark which can be used for individual methods comparing? It is a bit hard to answer this question because it is possible to find some optimization problems which are widely used along some papers but in fact there does not exists summary which can be uses for standard evaluation of optimization process. This paper will describe set of benchmarks which can be used for evaluation of optimization process. As a part of paper some results of optimization and utilization of optimization methods will be included. Keywords — Keywords — Heuristic, Optimization, Benchamarks.
I. I NTRODUCTION The optimization problem can be defined as: ,
( () , .
(1)
(2)
Sphere function in fact is not sphere but the projection to subspace will create circles around optimal solution. This function is generalized generalized to . (4) ( () ∑ Optimum is 0 , ( () 0 [X,Y] = meshgrid(-100:5:100,-100:5:100); Z = X.^2 + Y.^2;
Code 1. Sphere function in 2D
Code 1 is presenting the code for use in Matlab. The version of function is show on figures below. 100
0 0 0 6 80 1 60
0 0 8 0
0 0 0 2 1
1 2 0 0 0
8 0 0 0 00 0
20
(3)
where is coefficient which have force solver to find optimum 0. There exist many algorithms for problem solving. Some of them are special, some are more general. Many problems cannot be solved by deterministic algorithm so heuristic algorithm are used. In the set of heuristic
0 0 0 8
1 6 0 0 0
40 40 00 8 0 0 0
4 0 0 0
40
If the problem has defined d efined constrain ( ) 0 it can be extended to () 0 ,
0 , ( () .
II. U NIMODAL PROBLEMS Unimodal functions have only one local optimum. Those functions are relatively easy to analyze for optimums. They are used for checking the speed of optimization and convergence. There are used commonly two functions. First one is Sphere and second one is Sum and Product. A. F1 – sphere
Basically the problems can be divided into unimodal and multimodal. Another aspect are constrains. Some problems have not any constrain. The best optimization method has to find optimal value in all cases. Unfortunately this is, in current state of art, impossible. Still it will be useful to be able compare different optimization methods.
() 0 , () 0 .
algorithm can be found PSO [1], GSA [2], CFO [3], HAS[4]. New methods are compared to old known solutions but there is not defined standard for solvers (algorithms) testing. At current state there is not possibility to reconstruct presented solution. Very often there is not possible to use already computed results because there does not exist standards for “measuring”. This standard “measuring” – benchmarks can be defined via functions. To measure different algorithm/solvers algorithm/solvers on different problems standard problems and interfaces to those problems must be defined.
0 0 0 4
0
-20
-40
8 0 0 0
4 0 0 0
0 0 0 4
-60
0 0 0 2 1
40 00
-80
-100 -100
0 0 0 8
1 6 0 0 0 - 80
-60
0 8 0 0
8 0 0 0 0 0
1 2 0 0 0 -40
-20
0
20
40
Fig. 1. Sphere function in 2D
0 0 0 6 1 60
80
100
III. MULTIMODAL PROBLEMS Multimodal functions have multiple local optimums. Some methods will stuck in local optimum. Main goal of those problems is to test solvers how they are able to avoid local optimum. Some problems have no single global optimum. Some of them have one global optimum and many local which are very close in the term of fitness function. A. Schefel’s function
Fig. 2. Sphere function in 3D
B. F2 – sum and product
Sum and Product function is simply to evaluate. This function has same role as Sphere function but the different shape. (5) () ∑| | ∏| |. Optimum is 0 , () 0 [X,Y] = meshgrid(-10:0.5:10,-10:0.5:10); Z = abs(X).*abs(Y) + abs(X) + abs(Y);
500 8 12 00 0 0 0 0 8 400
10
8
[X,Y] = meshgrid(-500:10:500,-500:10:500); Z = -X.*sin(sqrt(abs(X))) Y.*sin(sqrt(abs(Y)));
Code 3. Schefel’s function
Code 2. Sum and Product function in 2D
4 0
Schefel’s function is very often used problem for solvers benchmarking. This problem is one of most difficult. This problem is a bit different from others because in some point of view it is constrained. Presented optimum (see below) is not global optimum. This function has no global optimum. (6) () ∑ | |. Commonly declared optimum is 420.968743 , () 418.9829 at interval 500;500.
8 0
4 0 0
0 4
8 0
8 0 0
4 0 0
400
8 0 0 300
6
12 0 0
1 2 0 0 4
2
80 0
4 0
100
80 0
4 0 0
200
4 0
0 0 8
0 0 8
1 2 0 0 0 0
0
-2
-100
-4
4 0
4 0
1 2 0 0 -300
-6
-8
1600
-6
-4
1 2 0 0
-2
0
2
4
6
8
10
-500 -500
80 0 12 00
-400
-300
-200
0 4 0
8 0 0
80 0
800
-400
4 0
8 0
-10 -10
0 0
8 0
0 4
-8
0 0 4 8 0 0
8 0 0 -100
0 4 0
0 0 0 0 8 2
0 0 8
8 0 0
0 0 1 2
12 00 0
0 0 8
1
8 0 8 0 0 0
0 0 8
4 0 0
0 0 1 2 0 0 8
80 0
0 0 2 1
-200
0 0 8 0 4 0
8 0 0
800
8
0
0 0 2 1
4 0 0
0 0 1 2
0 0 2 1
0 0 8
0 0 8
0 4 0
8 0 0
1 2 0 0
40 0 80 0
8 0 0 100
200
Fig. 3. Sum and Product function in 2D
Fig. 5. Schefel’s function in 2D
Fig. 4. Sum and Product function in 3D
Fig. 6. Schefel’s function in 3D
300
400
500
Schefel’s function is very similar to constrained problems. This is demonstrated on next figure where is used only one dimension. This function has only local optimums. Global optimum does not exist. But usually this is not taken on mind. It is very common to test local optimums on set 500;500
Optimum is 0 , () 0 Expected interval for solving is
600;600
[X,Y] = meshgrid(-600:10:600,-600:10:600); Z = (X .* X + Y .* Y) / 4000 -cos(X) .* cos(Y / sqrt(2)) + 1;
Code 5. Grienwank’s function
B. Rastrigin’s function
Rastrigin’s function is “hills and valleys” function. To leave local optimum the mass points have to putt big effort in point of mechanical energy. () ∑( 10(2 ) 10). (7)
0 , () 0 . Interval 5.12;5.12 Optimum is
10
8
2
1 . 2
4
2
Code 4. Rastrigin’s function
-2
4 3
6 0
4 6 0 0
6 0
4 0 6 0 0 4
6 0
2
6 0
1
0 4
-2
4 2 0 0
0 2 0 20 4
40
4 2 0 0
0 2 0 2 0 4
4 0
4 0 4 0
4 0 6 0
6 0
-4
4 6 0 0
2 40 0
-4
2 0 4 0
4 0 2 40 0 0 2 4 0 4 0 4 0
6 0
6 0
6 0
-5 -5
4 2 0 4 0 2 0 0 20
2 0 40
6 0
-3
4 0
2 0 0
2 0
2 0 40
0 2
-1
4 0
2 0 40 20 4 0
-2
4
4 2 00
2 0
2 0 0 4 0 2 0 4 0 2 0 4
4 0
2 0 4 0
0 0
0 2 0 4
0 2 0 4
4 0 2 0 4 0 2 0
0 2 0 4
2 0 4 0
2 0
4 0 2 2 2 0 0
4 0
0 2
4
4 0 2 0
4 0
4 0 -1
0
1
2 0 4 0
2 0
4 0
6 0
1 .2
8 . 0
-8
2
2 8 0. 4 0 .
1 . 6 0. 8 0 .4
1 . 6
1 . 2
0.8
0 . 8
1. 2
0. 8
1 . 2 6 1.
2
1.2 1. 6
8 . 0. 4 0
1.2 1. 6 2
2 0
4 . 0
2
8 0 .
-2
4 0 0.
2
1 . 6
8 0.
-4
1 . 2
1. 2
1 . 2
0 .4
0.8
-6
2
0. 8 1.2 1. 6
0.8
0 .8
2
6
1
1 . . 6 2
8 .
1 . 6
1 .6
1 1 . . 2
1 . 2
0 . 4
2
2
4
1. 2
0 . 8
1. 2 1. 6
8 . 0 4 . 0
2
6
8
10
Fig. 9. Grienwank’s function in 3D
2 0
0 4
0 2 0 4 0 2
2 0
2 0 4 0 2 0 20
2 0 4 0
2 -10 -10
6 0
1 . 2
4 0.
-8
0 40 6 0 4 0 6 2 0 0 2 0 4 0 2 0 4 0 0 4 4 0 6
2 0
4 0 0
4 2 0
0
-3
2 4 0 0
6 0
6 0
4 0 20 0 4 4 0
2 0
4 2 0 0
2 4 0 0
0 2
0 4
6 0
0 .8
0 . 8
1 . . 1 2
0. 8 1 . 2
0 .4
0.8 1.2
1 . 2
6 1. 1 . 2
4 0.
2
0 2
-6
6 0 2 4 0 0
8 . 0
0 . 8 0 .4
-4
0
6 4 0 0
0. 8 1 . 2
1 1 . 2 . 6
1 . 6
[X,Y] = meshgrid(-5.12:0.1:5.12,-5.12:0.1:5.12); Z = 20 + X .* X + Y .* Y -10 * cos(2 * pi * X) 10 * cos(2 * pi * Y);
5
0 . 4
1. 6 1. 2 0 .8 0 . 4
6
6
2
2 0
4 0
6 0
0 40 6
0 6 0 40 6
40 6 0 0 4 0 6 0
4 0 6 2
3
4
5
Fig. 7. Rastrigin’s function in 3D
Fig. 10. Grienwank’s function in 3D
IV. PROBLEM TRANSFORMATIONS All but Schefel’s function are centered. Global optimum is at 0 . Heuristic optimization methods use random initialization. Centre of particles is at 0 where is global optimum too. If the particles will just move toward the centre they will reach global optimum. This problem can be avoided by shifting. Fig. 8. Rastrigin’s function in 3D
C. Griewank’s function
Grienwank’s function is function which has many similar local optimums but there is only one global optimum. () ∑ ∏ √ 1. (8)
() ( ).
(9)
For common solvers (PSO, GSA, CFO) the shifting is granting harder problem. This is basic reason why Schefel’s function is too hard for some problems. Experiments where the Schefel’s function was shifted so expected optimum was at 0 proved that problem was more easy to solve.
Another possibility is usage of rotation in two dimensions. Assume that then
. ()
(10)
Where is rotation angle. The rotation removes variables symmetry. V. PROBLEM STANDARDIZATION In process of optimization suite some standard interfaces was defined. This definition extends older one [5]. public interface IProblem { double Evaluate(Vector Solution); Vector RandomSolution(); }
Code 6. Interface for problem definition IProblem interface grants standard communication between solver and problem. public interface IProblemBenchmark { void ResetCounter(); int Counter(); float OptimalValue(); Vector OptimalSolution(); } IProblemBenchmark interface allows to check the test suite if global optimum was found (see OptimalValue and OptimalSolution methods). Counter and ResetCounter works with internal counter. This allows test how many times the IProblem.Evaluate was used.
VI. SOLVER EFFORT EVALUATION As indices which are evaluating solver effort can be Number of calls (see IProblemBenchmark. Counter) Distance between optimal solution and founded solution (IProblemBenchmark. OptimalSolution) Error difference between optimal value and founded optimal value (IProblemBenchmark. OptimalValue). t can be •
•
•
TABLE I. DIFFERENT SOLVERS – SCHEFEL’S FUNCTION
PSO PSO PSO PSO PSO GSA GSA GSA
Solution [404,936191082104 348,001256167839] [-320,933366236779 274,460865333133] [426,686165773184 388,597262333762] [399,988156684249 321,41296485172] [388,522666176303 439,975273703124] [401,055625861613 432,045095352901] [-285,281251836546 433,000921879768] [-271,516959516475 208,403625506034]
GSA HSA HSA HSA HSA HSA
[-296,037442490502 209,39712069017] [-279,277321172739 276,313479757881] [420,973450374312 420,94526110261] [420,986577366007 420,96051663205] [420,985058378887 420,968488287632] [420,947748199547 420,967596546266] [420,993900821076 420,953676533398]
-493,115686933
100 000
-454,497766327
100 000
-837,965702155
100 000
-837,965725879
100 000
-837,965740961
100 000
-837,965718741
100 000
-837,965666045
100 000
VII. CONCLUSION The problem of comparing different solvers for optimization problems have to be solved so new methods can be lead in already used set of solvers. The set of problems and standard interface to interaction with those problems will help noticeable. This paper describes some problems which can be used as standard benchmarks and introduced interface which have to be implemented in testing process.
R EFERENCES
Code 7. Interface for problem benchmark definition
Solv er
GSA
Value
Calls
-454,519415512
100 000
-466,012293604
100 000
-710,190054207
100 000
-621,139462792
100 000
-668,679885408
100 000
-773,8611299
100 000
-664,784988061
100 000
-388,130741666
100 000
[1] James Kennedy, Russell Eberhart, “Particle Swarm Optimization,” Proc. IEEE Int'l. Conf. on Neural Networks, pages 1942–1948, Piscataway, NJ, USA, 1995. IEEE Press. [2] Esmat Rashedi, Hossein Nezamabadi-pour *, Saeid Saryazdi, “GSA: A Gravitational Search Algorithm,” Information Sciences 179 (2009), pages 2232–2248, doi:10.1016/j.ins.2009.03.004 [3] R. A. Formato, “Central Force Optimization: A New Metaheuristic With Applications” In Applied Electromagnetics Progress In Electromagnetics Research, PIER 77, pages 425–491, 2007 [4] Kang Seok Lee, Zong Woo Geem, “A new meta-heuristic algorithm for continuous engineering optimization: harmony search theory and practice” Comput. Methods Appl. Mech. Engrg. 194 (2005), pages 3902–3933, doi:10.1016/j.cma.2004.09.007 [5] Alexandr Štefek, “Method Cooperation in Optimization Process,” In Proc. of Mechatronika 2010, ISBN 978-80-8075-394-8 [6] Wagner , J., Maga, D., Führichová, R., Guba, R., Opaterný., J. “Determination of step motor parameters with magnetic field solution,” Compel – The International Journal for Computation and Mathematics in Electrical and Electronic Engineering . 1994, vol. 13, Nr. 1, pp. 137-139. ISSN 0332-1649.
