jungtaekkim/bayeso

Simple, but essential Bayesian optimization package

bayeso's Issues

Review

Firstly, I would like to congratulate the developers on putting together such a nice package. The package is of a very high-standard and, in my opinion, contains functionality far beyond the requirements set out by JOSS. I have a few comments and suggestions that I detail below. Note, I would only require points 5 and 7 being done for me to accept, all other points are suggestions that I feel would further improve the package.

I have opened issues in the repo with my specific comments. Some high-level comments

1. It would be nice to have more supporting text in your notebooks example. This would be helpful for new users trying to onboard into Bayeso. My acceptance is not conditional on this being done.
2. Having a real world example of BO would be nice. It's always a true test of an algorithm/implementation when it runs smoothly on real world data. My acceptance is not conditional on this being done.
3. Nice work on your visualisations - they're very helpful and clear.
4. It would be nice to decouple the need to always save figures in your examples. If I download/clone a new library, I would not want my local disk filling up with images. Maybe this could be made optional or removed altogether?
5. In line with the JOSS standards, please could you add a contributing file to the repo and documentation?
6. You may like to have some end-to-end or smoke tests in your package. With all stochasticity fixed through seeds, this ensures that your package runs end-to-end in a consistent manner. Given you already have some notebooks, I do not think this would be too hard to achieve using the approach we adopted in GPJax. My acceptance is not conditional on this being done.
7. There is no way in Bayeso for users to request features, flag bugs, or suggest improvements; one can only raise a generic issue. I would suggest having some pre-defined issue templates to more easily signpost users.

Review: Paper typo

There is a small typo on Line 154. codes -> code. For clarity, I would consider rephrasing this line as Our software is....

Supressing INFO on run_single_round

Thanks for the paper and package.
Is there a way to suppress the INFO printouts when running run_single_round?
Something like a verbose option?
Thanks

Review: State of the field

I do not currently see a section in the paper that outlines the other BO packages i.e., the state of the field. As per JOSS' guidelines, could you please add such a section?

Review: Typo

The wording of Line 164 could be improved. As a suggestion, "Our software is released via the Python Package Index (PyPi) meaning users can easily install Bayeso into their environment".

Review: Missing tqdm citation

There is no citation for tqdm. Could you consider adding on in the form

@software{jax2018github,
  author  = {James Bradbury and Roy Frostig and Peter Hawkins and Matthew James Johnson and Chris Leary and Dougal Maclaurin and George Necula and Adam Paszke and Jake Vander{P}las and Skye Wanderman-{M}ilne and Qiao Zhang},
  title   = {{JAX}: composable transformations of {P}ython+{N}um{P}y programs},
  url     = {http://github.com/google/jax},
  version = {0.2.5},
  year    = {2018}
}

Could you create an example with optimization without arrays of x and y?

I prefer not to use sklearn crossvalidation because it is damn slow.