ajsvb Goto Github PK

agao22_script

cefrem_oil

cl-for-ad

Semester Project in ISE Lab (ETHZ)

dirichlet

Dirichlet MLE python library

examples

A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.

files

gpbt

hydra_template_for_joao

novel-approaches-for-tuning-models-hyperparameters

Hyperparameters are crucial in machine learning, as they determine if the model will be efficient and accurate. Usually, a tuning process is used to find good combinations of hyperparameters. This tuning process used to be very time-consuming as it historically consisted of training the model with all combinations of hyperparameters among the space of all possibilities for grid-search, and of a random set of combinations for random search. Then results were compared to choose the best combination. Now notice that the space of possibilities grows exponentially with the number of hyperparameters. Furthermore, for complex deep-learning problems a single training can take days. \\ Thus new approaches are needed to find good hyperparameters faster. In this project, we would like to choose a few deep-learning problems as they are usually long to train and compare different tuning approaches \footnote{ using the \href{https://docs.ray.io/en/latest/tune/}{Ray Tune} library}. We would like to empirically quantify and qualify the speed for each method as well as the efficiency of the parameters found. \underline{Project Plan:} \begin{enumerate} \item Choose and describe optimization algorithms that are going to be compared during the project. \item Implement a benchmark tool to measure the results of various optimization algorithms. \item Choose a batch of different machine learning problems (more precisely, deep-learning problems). \item Analyze the results and determine if any approach performs better in average in terms of efficiency or in terms of speed than random search, which is the commonly used algorithm. \end{enumerate}

pycaret

An open-source, low-code machine learning library in Python

pytorch-lightning

The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.

rl_path_tracing

time-patchesoilseeps