Comments (5)
In that example (https://ax.dev/tutorials/gpei_hartmann_developer.html) there are two main sources of randomness: The scrambling in the Sobol generator used for the initialization, and the randomness in the (quasi) Monte Carlo approximation used in the acquisition function.
Like @kkashin says, the first can easily be handled by making the change to
sobol = Models.SOBOL(search_space=exp.search_space, seed=1000)
And yes, this will require using the more explicit API rather than optimize
, whose purpose is to do a bunch of stuff under the hood for you.
Most of the randomness will come from that (the initialization). The actually Bayesian optimization phase will still have some randomness in that the acquisition function is evaluated with a Monte Carlo approximation. The seed for that is set here:
Ax/ax/models/torch/botorch_defaults.py
Line 162 in eebb8c7
So, you can just run
torch.manual_seed(1000)
before every call to gpei.gen
.
As a side note, for the purposes of benchmarking / performance evaluation I'd recommend setting the number of reps high enough that the seed doesn't matter. But of course there are other settings where reproducibility of a single run would be useful, and this should give that.
from ax.
I'm going to close out this issue for now, given that you're unblocked. We will look into making it possible to pass a seed directly to the optimize
function so that reproducibility is easier to achieve in the future, and will follow up with developments.
from ax.
@Irynei - what level are you looking for reproducibility at? The whole optimization loop or a specific generator?
You can set the seed for (quasi)-random generators, such as the Sobol generator (see https://ax.dev/api/models.html#ax.models.random.sobol.SobolGenerator.seed).
from ax.
I was looking for the whole optimization loop, in the optimize
method, ideally.
As far as I understand a (quasi)-random generator is chosen somehow under the hood?
So I can only set seed when I am using something similar to what explained here https://ax.dev/tutorials/gpei_hartmann_developer.html, right?
from ax.
@bletham, thank you for a great explanation! It's clear to me now.
from ax.
Related Issues (20)
- Defining a parameter as choice between a value and a range HOT 4
- How to use tensor in Ax service API HOT 3
- parameterization values are not disply on the plot HOT 5
- model from restored ax_client from JSON optimization returns None HOT 1
- [Question] How to add fixed parameter choices in GenerationStrategy? HOT 4
- Rejection Sampling Error Due to Search Space Complexity in Hardware-Aware NAS HOT 3
- Multiobjective Multifidelity BO using the Service API HOT 11
- Tutorial fails HOT 2
- BUG: MBM sums up MOO outputs when given a single objective acquisition function HOT 2
- Last trial not being recognized (in the submitit tutorial) HOT 3
- Ax pulling numpy 2.0 with breaking changes HOT 3
- How to set the beta coefficient of generation strategy? HOT 6
- Numpy 2.0 Compatibility Issue HOT 1
- save the state HOT 9
- Defining a Model class for ModelListGP HOT 2
- Error : Try again with more data HOT 26
- Using nonlinear constraints with boolean masks HOT 5
- input data is not standardized (mean = tensor([0.] HOT 9
- nonlinear constarined in evaluate paramter instead add to the experiment HOT 2
- Ax 0.4.0 Causing Segmentation Fault When Calling `.get_next_trial()` HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ax.