Bayesian optimization gives flat latent function?

[iRove108]

Bayesian optimization gives flat latent function?

[apaleyes]

Thanks for the report! No idea what's going on, but I've similar behavior recently, and the same warning too. Some googling shows that this might be related to Emukit generating some very big numbers somewhere: https://stackoverflow.com/questions/40726490/overflow-error-in-pythons-numpy-exp-function

Would you be able to get a full stack trace of this warning?

[iRove108]

For sure! I was able to get a trace of it below in colab. Seems to be an issue with GPy and paramz.optimize_restarts. The final warning is thrown here.

This thread looks related. Despite the claim that it is "just a warning," I suspect it still might be causing problems here (particularly given the high parameters reported), but I'm not positive.

  File "<ipython-input-4-46631344dbd1>", line 11, in <module>
    loop.run_loop(UserFunctionWrapper(f), FixedIterationsStoppingCondition(ITER))
  File "/usr/local/lib/python3.7/dist-packages/emukit/core/loop/outer_loop.py", line 91, in run_loop
    self._update_models()
  File "/usr/local/lib/python3.7/dist-packages/emukit/core/loop/outer_loop.py", line 104, in _update_models
    model_updater.update(self.loop_state)
  File "/usr/local/lib/python3.7/dist-packages/emukit/core/loop/model_updaters.py", line 62, in update
    self.model.optimize()
  File "/usr/local/lib/python3.7/dist-packages/emukit/model_wrappers/gpy_model_wrappers.py", line 86, in optimize
    self.model.optimize_restarts(self.n_restarts, verbose=verbose, robust=True)
  File "/usr/local/lib/python3.7/dist-packages/paramz/model.py", line 152, in optimize_restarts
    initial_parameters = self.optimizer_array.copy()
  File "/usr/local/lib/python3.7/dist-packages/paramz/core/parameter_core.py", line 87, in optimizer_array
    [np.put(self._optimizer_copy_, ind, c.finv(self.param_array[ind])) for c, ind in self.constraints.items() if c != __fixed__]
  File "/usr/local/lib/python3.7/dist-packages/paramz/core/parameter_core.py", line 87, in <listcomp>
    [np.put(self._optimizer_copy_, ind, c.finv(self.param_array[ind])) for c, ind in self.constraints.items() if c != __fixed__]
  File "/usr/local/lib/python3.7/dist-packages/paramz/transformations.py", line 111, in finv
    return np.where(f>_lim_val, f, np.log(np.expm1(f)))
 /usr/local/lib/python3.7/dist-packages/paramz/transformations.py:111: RuntimeWarning:overflow encountered in expm1

[apaleyes]

Thanks! What happens if in your second snippet the GPy model is wrapped in the Emukit wrapper? I am suspecting the issue might be because of the fact we don't jsut call optimize, and instead optimize_restarts(self.n_restarts, verbose=verbose, robust=True)

[iRove108]

At first I was thinking that too. I tried replacing the second snippet with:

# GP regression on experimental points, run separately
model_gpy2 = GPRegression(loop.loop_state.X, loop.loop_state.Y)
model_gpy2_emukit = GPyModelWrapper(model_gpy2)
model_gpy2_emukit.optimize()
model_gpy2.plot()
plt.show()

And it worked the same (fit the function properly).

I also tried creating a new GPyModelWrapper class that calls optimize instead, but the warning was still thrown with the following traceback:

  File "/var/folders/xl/nxhg5qwd3f7b8yjnc80d334h0000gn/T/ipykernel_50332/4090956357.py", line 15, in <module>
    loop.run_loop(UserFunctionWrapper(f), FixedIterationsStoppingCondition(MAX_ITER))
  File "python3.9/site-packages/emukit/core/loop/outer_loop.py", line 91, in run_loop
    self._update_models()
  File "python3.9/site-packages/emukit/core/loop/outer_loop.py", line 104, in _update_models
    model_updater.update(self.loop_state)
  File "python3.9/site-packages/emukit/core/loop/model_updaters.py", line 62, in update
    self.model.optimize()
  File "gpy_model_wrapper_norestart.py", line 93, in optimize
    self.model.optimize()
  File "python3.9/site-packages/GPy/core/gp.py", line 675, in optimize
    ret = super(GP, self).optimize(optimizer, start, messages, max_iters, ipython_notebook, clear_after_finish, **kwargs)
  File "python3.9/site-packages/paramz/model.py", line 98, in optimize
    start = self.optimizer_array
  File "python3.9/site-packages/paramz/core/parameter_core.py", line 87, in optimizer_array
    [np.put(self._optimizer_copy_, ind, c.finv(self.param_array[ind])) for c, ind in self.constraints.items() if c != __fixed__]
  File "python3.9/site-packages/paramz/core/parameter_core.py", line 87, in <listcomp>
    [np.put(self._optimizer_copy_, ind, c.finv(self.param_array[ind])) for c, ind in self.constraints.items() if c != __fixed__]
  File "python3.9/site-packages/paramz/transformations.py", line 111, in finv
    return np.where(f>_lim_val, f, np.log(np.expm1(f)))
python3.9/site-packages/paramz/transformations.py:111: RuntimeWarning: overflow encountered in expm1

My guess is that it isn't the model wrapper that's malfunctioning but rather BayesianOptimizationLoop.

[apaleyes]

THanks for all this effort. I'll try to carve out time to debug this properly and update here

[apaleyes]

Ok, I think this is just a modelling issue, and the fact that the error comes from paramz is a hint at that. When you are creating the model, it's just plain GPRegression from GPy, with all default parameters. This includes default prior values for lengthscale, variance and noise, all set to 1 in GPy by default I believe. Usually this isn't a problem, but in your case this is completely out of scale of actual inputs and outputs. When I am playing with different combinations of prior values for these, the result is very different fits.

The simplest thing to try would be to normalize your data to [0,1]. Setting priors might also be worth it, but it's much harder to pick good ones, and probably won't be necessary.

[iRove108]

Hi, thank you so much for that—I really appreciate the help! I tried passing normalizer=Standardize() to GPRegression and it worked much better. It's no longer giving any warnings as well.

I was still wondering, though: should I expect the graph of the latent function after BayesianOptimizationLoop.run_loop to match the one after doing GP separately on (loop.loop_state.X, loop.loop_state.Y)? (as in my original question)

For example, after re-running the process with some more data, the graph displayed following BayesianOptimizationLoop.run_loop looks like:

While the graph displayed after optimizing a new GPRegression model (with normalizer=Standardize() still) looks like:

[apaleyes]

No, generally they don't have to be the same. Re-training from scratch often yields different parameter values compared to iteratively adding points one by one

[iRove108]

Ah ok, thanks for the clarification! I assume differences could come from randomization while optimizing parameters.

[apaleyes]

Indeed