Comments (9)
Thanks for bringing up this issue. I'd like to help with you but I have a couple of questions in this case:
- Would you let me know which algorithm or algorithmic module combination that you are using? E.g. DGEMO or TSEMO, etc.?
- What's your problem like? Is it evaluated in a kind of real-world lab setup? Is the evaluation noisy?
- What happened after the 200th evaluation? It seems like there's an obvious turning point. Did you change any settings?
It's hard to tell without more information, unfortunately. Since people use BO for drastically different problems.
In terms of parameter tuning, actually, we don't tune a lot. For nu we mainly use 1 or 5. But it's known that Gaussian Process will sometimes predict very extreme values if the underlying dataset is somewhat noisy.
Currently, the bounds on the objectives are not supported, but I think it should be relatively easy to add that in a naive way, perhaps like post-processing on the surrogate model's evaluation result.
from autooed.
Thanks for the quick reply. Following your comments, I list more details as follows.
- I used the DGEMO algorithm. For other settings, the Surrogate model: Gaussian Process, Acquisition function: Identity, Solver: ParetoFrontDiscovery (n_gen-10, pop_size-100), Selection: Direct.
- Yes, our problem aims to the laser marking task, i.e., marking the picture on the steel using laser, we need to find the relationship between design space (laser parameters) and performance space (colors) while investigating the diversity of colors. Its evaluations are somehow noisy.
- On the evaluation, we first generated the 100 parameters using AutoOED, and then we used these parameters to mark the steel and obtained the corresponding performance space. Subsequently, the measured colors are loaded into AutoOED and we set the batch size to 100 to generate the next 100 parameters. It seems the model predicts well in the first generation (100~200 as shown in the Figure). For the second generation, I remembered I didn't do anything. Before the marking of this generation, I saw the predicted values are very large.
Thanks for your tuning and other suggestions. It's not sure if I apply AutoOED to our problem correctly and I am looking forward to your further comments.
from autooed.
PS: I found the 0.2.0 version of AutoOED may induce an inverse exploration issue. In my case, I want to maximize two objectives. However, the predicted values seem to exhibit in the minimum region as shown in the following figures (the first iteration and second iteration). Could you please give some suggestions? I have tried different initializations but occurred the same inverse exploration. For the 0.1.0 version of AutoOED, the direction of the exploration is correct.
from autooed.
Thanks for the feedback! This looks like a bug to me. I'll take a look very soon and keep you updated.
from autooed.
Thanks for the feedback! This looks like a bug to me. I'll take a look very soon and keep you updated.
Thanks a lot!
from autooed.
Thanks for the feedback! This looks like a bug to me. I'll take a look very soon and keep you updated.
Maybe the issue exists in the pareto.py. In this script, you call the function convert_minimization (Convert maximization to minimization). I don't understand the reason for this operation. Could you please explain it?
from autooed.
I've fixed that bug in the latest commit. Please pull if you're working on the source code, otherwise, if you're installing through the executable files, I will upload a new version later today.
Thanks again for pointing out this important issue! Your intuition is right, something is wrong with the minimization conversion. That is needed because our underlying optimization algorithm essentially is a minimizer, so we need to convert the maximization problem to the equivalent minimization problem. But the bug was that I converted it twice somewhere in the code. But I've fixed that now and tested on both a maximization problem and a minimization problem. It should work fine now. Let me know if you still have any other problems?
from autooed.
I've fixed that bug in the latest commit. Please pull if you're working on the source code, otherwise, if you're installing through the executable files, I will upload a new version later today.
Thanks again for pointing out this important issue! Your intuition is right, something is wrong with the minimization conversion. That is needed because our underlying optimization algorithm essentially is a minimizer, so we need to convert the maximization problem to the equivalent minimization problem. But the bug was that I converted it twice somewhere in the code. But I've fixed that now and tested on both a maximization problem and a minimization problem. It should work fine now. Let me know if you still have any other problems?
Thanks a lot for addressing this problem! I am working on the source code and I also tried to revise the part using the convert_minimization function. Now, it works for me and I have no further questions.
from autooed.
Great to hear that. Please feel free to reach out to us once you have any other issues.
from autooed.
Related Issues (7)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from autooed.