jeffkinnison / shadho Goto Github PK
View Code? Open in Web Editor NEWScalable, structured, dynamically-scheduled hyperparameter optimization.
License: MIT License
Scalable, structured, dynamically-scheduled hyperparameter optimization.
License: MIT License
See title. Need documentation for all of this.
max_tasks
parameter in shadho object does not equate to the total number of runs.
For example, passed max_tasks = 2
but received 7 successes:
Experiment nn_mnist_test
-------------------------------------------------------------------
106 Trials 99 Pending 7 Successes 0 Errors
-------------------------------------------------------------------
Bug: shadho attempts to copy work_queue.py to the same location.
How to Reproduce:
conda create --name shadho-test
//create a new environment to reproduce this bugconda activate shadho-test
//activate this environmentconda install -c conda-forge ndcctools
//install cctools from conda channel conda-forgeconda install pip
//let conda handle pip installpython -m pip install shadho
//pip install shadho to shadho-test environmentpython -m shadho.installers.workqueue --prefix $CONDA_PREFIX
//"glue" shadho and cctoolsError:
Traceback (most recent call last):
File "/afs/crc.nd.edu/user/t/tphung/anaconda3/envs/shadho-test/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/afs/crc.nd.edu/user/t/tphung/anaconda3/envs/shadho-test/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/afs/crc.nd.edu/user/t/tphung/anaconda3/envs/shadho-test/lib/python3.9/site-packages/shadho/installers/workqueue.py", line 209, in
main()
File "/afs/crc.nd.edu/user/t/tphung/anaconda3/envs/shadho-test/lib/python3.9/site-packages/shadho/installers/workqueue.py", line 202, in main
install_shadho_files(prefix, sp_prefix)
File "/afs/crc.nd.edu/user/t/tphung/anaconda3/envs/shadho-test/lib/python3.9/site-packages/shadho/installers/workqueue.py", line 176, in install_shadho_files
shutil.copy(os.path.join(source, 'work_queue.py'), sp_prefix)
File "/afs/crc.nd.edu/user/t/tphung/anaconda3/envs/shadho-test/lib/python3.9/shutil.py", line 418, in copy
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/afs/crc.nd.edu/user/t/tphung/anaconda3/envs/shadho-test/lib/python3.9/shutil.py", line 244, in copyfile
raise SameFileError("{!r} and {!r} are the same file".format(src, dst))
shutil.SameFileError: '/afs/crc.nd.edu/user/t/tphung/anaconda3/envs/shadho-test/lib/python3.9/site-packages/work_queue.py' and '/afs/crc.nd.edu/user/t/tphung/anaconda3/envs/shadho-test/lib/python3.9/site-packages/work_queue.py' are the same file
Temporary Solution: commenting out lines 176 and 177 in file https://github.com/jeffkinnison/shadho/blob/master/shadho/installers/workqueue.py
solves the problem.
Per title. This is because shadho depends on package pyrameter
, which uses object Sequence
from library collections
, which is deprecated since python3.3
and removed since python3.9
. This issue is reproducible by running shadho-test.sh
as specified in this issue, but changing the python version from 3.8 to 3.10 when creating the conda environment. Hope this helps!
See title. What are you doing with your life, Jeff???
Installing from PyPI does not create a .shadhorc file in $HOME. This should be created automatically on first use of SHADHO if it does not exist.
The code in both of these files is waaaaay less organized than in the rest of the package. Clean it up, Jeff.
Problem: shadho module cannot be imported.
How to reproduce: (assuming that we have conda or miniconda)
conda create --name shadho-env
conda activate shadho-env
conda install pip
pip install shadho
python -c "import shadho"
The last command throws this error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "~/anaconda3/envs/shadho-env/lib/python3.9/site-packages/shadho/shadho.py", line 8, in <module>
from .configuration import ShadhoConfig
ImportError: attempted relative import with no known parent package
It looks like an import error, which hopefully is easy to fix.
Currently, trees are assigned to ccs under the assumption that the number of trees will be greater than the number of ccs. This will not always hold, and for the sake of robustness the assignment process must work in all cases.
Currently, SHADHO requires a lot of manual configuration. Much of this can be pushed behind-the-scenes (e.g. Work Queue config) or be automated (e.g. a worker type function for running the training code).
There should be a default use mode that requires no input from the user besides which function to optimize.
Create an installed executable to install/uninstall Work Queue and the .shadho directory.
Shadho seems to insist on finding its config file as there's no option for me to tell shadho to use the default configuration. It'd be nice if there's an option to do so, maybe in Shadho()
object instantiation. Also shadho looks for its working directory .shadho
when it doesn't need one. This is because I installed shadho via pip, and I installed pip via conda, so .shadho
already lives in a conda environment, instead of the home directory.
See title. Need good docs up and running.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.