Module benchmark 

Test functions with known minima, for checking an optimizer.

Each is a test function of CMA-ES’s authors, from N. Hansen, S. D. Müller and P. Koumoutsakos, “Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES)”, Evolutionary Computation 11(1), 1–18 (2003), https://doi.org/10.1162/106365603321828970, Table 1 (printed p. 7), with the variables numbered from 0 here. Their minima are known exactly, so a run’s best point and value can be held to them; that paper runs each to f = 10⁻¹⁰, as the tests do.

Function	What it tests	Minimum
sphere	the step size alone	0 at x = 0
ellipsoid	coefficients from 1 to 10⁶, so the distribution must grow 1,000 times longer one way than the other	0 at x = 0
rotated_ellipsoid	the same along axes that aren’t the variables’	0 at x = 0
rosenbrock	following a long, curved valley	0 at x = 1

constrained holds three problems whose minima lie on their constraints’ edges. mixed holds three whose second half of variables take only whole numbers.

Modules

constrained

Constrained test problems with known minima on their constraints’ edges, for Run::tell_constrained. Each gives an Evaluation with constraints written g(x) ≤ 0.

global

Test functions for global optimization with few evaluations, for ego: the Branin and Hartmann functions of L. C. W. Dixon and G. P. Szegö, “The global optimisation problem: an introduction”, in Towards Global Optimisation 2, North-Holland, 1–15 (1978), on which D. R. Jones, M. Schonlau and W. J. Welch, “Efficient global optimization of expensive black-box functions”, Journal of Global Optimization 13, 455–492 (1998), https://doi.org/10.1023/A:1008306431147, run EGO (their Table 1). The constants are as S. Surjanovic and D. Bingham’s Virtual Library of Simulation Experiments lists them, https://www.sfu.ca/~ssurjano/optimization.html. Branin has three minima, all of the same value; the Hartmann functions have local minima above their least value. A missing variable is taken as 0.

mixed

Test functions of continuous and integer variables together: those of R. Hamano, S. Saito, M. Nomura and S. Shirakawa, “CMA-ES with Margin: Lower-Bounding Marginal Probability for Mixed-Integer Black-Box Optimization”, GECCO 2022, https://arxiv.org/abs/2205.13482, §5.1 (p. 7). The first ⌊n/2⌋ variables are continuous and the rest integer (Variable::integer); each function is given the values the optimizer encodes, whole numbers in the integer variables.

zdt

Two-goal test problems with known Pareto fronts, for nsga2: ZDT1, ZDT2 and ZDT3 of E. Zitzler, K. Deb and L. Thiele, “Comparison of multiobjective evolutionary algorithms: empirical results”, Evolutionary Computation 8(2), 173–195 (2000), https://doi.org/10.1162/106365600568202, §4, eqs. (7)–(9) (pp. 177–178), each of n variables in [0, 1] (the paper’s n = 30):

Constants

ELLIPSOID_CONDITION

The ellipsoid’s condition number: the ratio of its largest curvature to its smallest.

Functions

ellipsoid

The ellipsoid, f(x) = Σ 10^(6 i/(n − 1)) xᵢ² for i = 0 … n − 1: its curvatures run from 1 to ELLIPSOID_CONDITION. With one variable it is the sphere.

rosenbrock

Rosenbrock’s function, f(x) = Σ [100 (xᵢ² − xᵢ₊₁)² + (1 − xᵢ)²] for i = 0 … n − 2. Its global minimum is 0 at x = 1. “For higher dimension, even function 8 f_Rosen has a local minimum near y = (−1, 1, …, 1)ᵀ” (N. Hansen and A. Ostermeier, Evolutionary Computation 9(2), 159–195, 2001, https://doi.org/10.1162/106365601750190398, footnote 18), and CMA-ES sometimes misses the global one: in 1 to 3 of 20 runs at 4 to 16 variables in S. Kern, N. Hansen and P. Koumoutsakos, “Local meta-models for optimization using evolution strategies”, PPSN IX (2006), Table 3.

rotated_ellipsoid

The ellipsoid turned so that its axes aren’t the variables’: ellipsoid(H x), with H the reflection I − 2 v vᵀ/(vᵀv), vᵢ = i + 1. H is orthogonal, so the minimum is still 0 at x = 0, but no variable can be stepped alone to reach it. CMA-ES is invariant under rotations and reflections of the variables, so it should take about as many evaluations as on ellipsoid.

sphere

The sphere, f(x) = Σ xᵢ².