Evolution Strategies From Scratch in Python

Evolution strategies is a stochastic global optimization algorithm.
It is an evolutionary algorithm related to others, such as the genetic algorithm, although it is designed specifically for continuous function optimization.
In this tutorial, you will discover how to implement the evolution strategies optimization algorithm.
After completing this tutorial, you will know:

Evolution Strategies is a stochastic global optimization algorithm inspired by the biological theory of evolution by natural selection.
There is a standard terminology for Evolution Strategies and two common versions of the algorithm referred to as (mu, lambda)-ES and (mu + lambda)-ES.
How to implement and apply the Evolution Strategies algorithm to continuous objective functions.

Let’s get started.

Evolution Strategies From Scratch in Python Photo by Alexis A. Bermúdez , some rights reserved.

Tutorial Overview
This tutorial is divided into three parts; they are:

Evolution Strategies
Develop a (mu, lambda)-ES
Develop a (mu + lambda)-ES

Evolution Strategies
Evolution Strategies , sometimes referred to as Evolution Strategy (singular) or ES, is a stochastic global optimization algorithm.
The technique was developed in the 1960s to be implemented manually by engineers for minimal drag designs in a wind tunnel.
The family of algorithms known as Evolution Strategies (ES) were developed by Ingo Rechenberg and Hans-Paul Schwefel at the Technical University of Berlin in the mid 1960s.
— Page 31, Essentials of Metaheuristics , 2011.
Evolution Strategies is a type of evolutionary algorithm and is inspired by the biological theory of evolution by means of natural selection. Unlike other evolutionary algorithms, it does not use any form of crossover; instead, modification of candidate solutions is limited to mutation operators. In this way, Evolution Strategies may be thought of as a type of parallel stochastic hill climbing.
The algorithm involves a population of candidate solutions that initially are randomly generated. Each iteration of the algorithm involves first evaluating the population of solutions, then deleting all but a subset of the best solutions, referred to as truncation selection. The remaining solutions (the parents) each are used as the basis for generating a number of new candidate solutions (mutation) that replace or compete with the parents for a position in the population for consideration in the next iteration of the algorithm (generation).
There are a number of variations of this procedure and a standard terminology to summarize the algorithm. The size of the population is referred to as lambda and the number of parents selected each iteration is referred to as mu .
The number of children created from each parent is calculated as ( lambda / mu ) and parameters...