pastelbelem8/adopt.jl

This is the result of my master thesis on Multi-Objective Optimization. This repository is more focused towards Pareto-based optimization rather than SIngle-Objective optimization with preference articulation. We focus on time-consuming optimization routines and, as a result we focus on model-based methods to allow for faster convergence times. This is relevant for Architectural Design Optimization, which depends on time-intensive simulations (e.g. minutes, hours or even days to complete a single simulation).

adopt.jl's Issues

Refactor Domain Application

Relates to issue #21.
The proposed organization of the repository is something similar to the picture depicted below...

Rewrite the project (version 1): General Reviews

After reviewing the literature a final document was produced and presented.

The Jury provided some useful advice that I should address in the final report:

• Glossary of Acronyms/Abbreviations.
• Reinforce the impact and usefulness of this work in Introduction and Conclusion.
• Present examples during sections

Platypus changes

Deploy platypus changes

Problem with shared objective

When using shared_objective, the function evaluate is returning a matrix 3x2 instead of 2x1.
This is a problem when trying to solve a problem with a shared objective.

This example shows the problem:
facade((f_length, m_stripes_south03))=parse.(Float64, ["6.7", "11.0", "5.0"])

objs=[SharedObjective(facade, [:MIN, :MAX])]
obj1=objs[1]
evaluate(obj1, [1,2])

Fix infinte cycle in Meta Solver.solve_it method

Raise an error in the metasolver solve_it method when the number of candidate solutions is not greater than 0.

To do this, please create a branch using the following command:

$ git checkout -b 38-Fix-Infinite-Cycle-MetaSolver 

Then commit the code that is specific to this question and create a pull request

Fix Hypervolume implementation

HVI implementation only works for 2-Dimension problems.

Define the processing information necessary throughout a typical optimization process

When performing optimization, we collect different measurements. A careful review over the involved metrics and the necessary information should be considered.

This should be documented properly.

For example, important information to collect is

• the time it took an evaluation, thus allowing to monitor problems in the simulator/model or even to provide feedback that the process is running.
• the set of parameters' values that generated a specific design.
  ...

Fix constraints

Implement Mechanism to wrap information

Implement Mechanisms to wrap necessary information during the optimization process.
This information should be somewhat similar to the one collected in issue #5 .

LinearRegression should mantain track of data points...

Provide MOEAs algorithms

• SPEA2
• NSGA-II
• VEGA
• IBEA
• OMOPSO ?

Implement the MOO Metrics module

Rewrite the project (version 1): Solution Proposal

The evaluator advised us to:

• use the term hill-climbing instead of local search
• Try the application of a guided local search when using more variables!

Problems should always be minimization.

transform problems in the backend into minimization problem.

Identify MOO libraries' requirements

After a slight overview of the more reputed Multi-Objective Optimization (MOO) libraries, these seem the broadest and the most complete, providing several Evolutionary Algorithms (EA).

A more profound and careful study should be performed to determine which libraries are worth integrating with and then a collection of functional requirements should be done.

Python	Java
Platypus	MOEA Framework
Pygmo	jMetal
OTL / PyOptimization

Implement core functionality

• Variables
• Objectives
• Constraints
• Model/Problem
• SolverFactory
• Solvers
• Solution

Implement Sampling Techniques

• LHS (unit volume)
• Random (randomMC)
• Random (stratifiedMC)
• Two Factorial
• Full factorial
• Box Behken
• OA?

Implement base for SBO

Implement the base for the SBO

• Core routine to perform SBO
• Sampling Techniques
• ML Techniques
• Other techniques ?
• Improvement Measurement Techniques

Surrogate Based Optimization behaves strangely when multiple Surrogates are specified

Due to the way we implemented surrogates, i.e., by associating the sampling routines to each surrogate. We incurred a new problem: when we use these surrogates in the context of SBO (Surrogate Based Optimization), they might be using a smaller subset of variables instead of all the variables, which will yield to a mismatch in the correction, creation and validation routines. Also, the create! routine is poorly conceived as it determines the number of variables based on the number of unscalers, which might now always be the case, i.e., we might sometimes use a smaller subset of unscalers.

This issue is priority and requires all the attention.

Write the thesis (Final version)

The solution should be extensible discussed and every detail should be clearly stated.
The evaluation should be clear, simple, and reveal the results obtained.

Refactor Surrogate base files

• Different concepts and domains should be better defined
• Including MLP Regressor yields stack overflow in Matrix transposition
• Few functions should be better organized
• Error Measurement routines
• Correction mechanism will be biased if we provide the old solutions as well

Add Clustering techniques to base SBO workflow

Implement clustering methods to enhance solutions' diversity
Introduce mechanism to automatically sample a few solutions in case of the solutions divised by the optimiser were not diverse or not very revealing (e.g. when a single solution is returned, that makes linear regression methods worse...)

MetaModels - Constraint modelling?

How to model constraints and objectives? Should they be modelled together ?

Write thesis report

Add Reprs to Data Structures

We need to confirm some data structures, iit would be better to somehow provide that information for debugging...

Decrease by fails in IntegerVariables due to Platypus LImitations

Surrogate modelling is not working when coupling single and shared objectives

(have to modify objective_indices approach)

MLPRegressor - Early Stopping

Must change training-based early stopping to accumulate the batch loss, instead of the loss per sample... The currently existing implementation will work, but training and "validation" options will be different when validation_fraction = 0. Ideally, this should not happen. The difference is that MLPRegressor keeps track of the loss per batch, whereas EarlyStopping maintains the track of the loss per sample.

Implement Platypus Integration

Rewrite project (Version 1): Introduction

Rearrange the introduction: Instead of writing about generic optimization problems, one could simply introduce optimization applied to architecture and so on. It depends on the focus of the work.

Advice provided during the project evaluation:

• Change the MOO example to an example with more conflicting examples.
• Provide examples when discussing Algorithmic Design

Implement a few test functions in the benchmarks module.

Rewrite the project (version 1): Related Work

Several enhancements can be made in this section. Although the project was a more general document, it is supposed to be more focused in the final thesis report. Depending on the elaborated work, one should update the content.

Advice provided during the evaluation:

• Black-Box Optimization is a completely different term in the context of Computer Science. Either use the terminology of derivative free optimization or explain it better, i.e., why do architects call it black-box optimization?
• Clarify that even that we are do not find the best solution, if we still find a better solution than the initial one, it is good anyway.
• Correct the terminology used to refer to IBM's Watson, which is a Question-Answering system and not a game.

Int Variables yields Real-valued solutions

Advanced tests t25 are a good example of the problem.

Update unfeasible penalty for constraints

WRONG IGD INDICATOR

Should determine the minimum Euclidean distance and its considering the inverted GD

Implement DecayingLinearRegression

Keeps record of past n data entries, to avoid having such bad results after initial sampling (as it never sees those solutions again)

Add BORG-MOEA

https://github.com/robertfeldt/BlackBoxOptim.jl#multi-objective-optimization

Create an initial schematics of the optimization process and involved data flows

To provide a clearer picture of the developer's intention, a schematics of the architecture should be provided as well as the proper foundations for the schematics.

The predicted data flows between different data structures should be represented as well.

Add SamplerSolver

• Uses sampling methods to analyse different points in regions and then chooses n best performing solutions (or solutions within a range) but that are diverse.
• Use higher-order functions to decide how to divise the strategy
• Makes available methods to :
• Uniform Sampling
• Decaying Sampling (exploitation increases gradually)

Pareto Front allows to add duplicated individuals!