Add tuning by genetic algorithms

[alan-turing-institute]

Add tuning by genetic algorithms

[jpsamaroo]

In order to utilize a genetic algorithm, one first need to define the structure of the genetic material and how it maps to the final individual.

Going based on the existing tuning implementation, we could imagine having a minimum of one "gene" per supplied tunable parameter, where the gene is represented as a bitstring and a mapping from parameter value to bits in the bitstring. Genes will be strung together to construct the full genetic material for an individual, and would be subject to the usual rules of mutation and recombination. Additionally, parameters with more than 2 possible states would be expanded into longer genes, whose combined bit values encode the final value of that parameter. Of course, if the number of feasible values is not a power of two, then normalization or rejection of infeasible encodings will need to be implemented. If the number of feasible solutions is mathematically unbounded (such as a range of floating point values), then we also need a strategy to deal with this.

As a comparison, the Borg MOEA algorithm (implemented in BlackBoxOptim.jl) takes the approach of first randomly sampling from (continuous) parameter values to establish an initial population, and then utilizes genetic methods to exchange genes. Importantly, the genes may only take on a value present in one of the initial individuals; arbitrary floating point values are not generated at "runtime". One important caveat is that Borg cannot naturally handle parameters with a countable number of states (such as integer ranges or countable sets). (Note that this is my understanding of the algorithm, and I could be mistaken).

I think locating a method which is able to handle both continuous and discrete parameter ranges would be worth doing, if such a beast exists, as it would be straightforward to implement in the current MLJ tuning configuration.

[jpsamaroo]

Relevant discussion: robertfeldt/BlackBoxOptim.jl#136