DARTS algorithm fails to create more than one trial

[MatthiasCarnein]

DARTS algorithm fails to create more than one trial

[MatthiasCarnein]

Oh I see, I wasn't aware that DARTS is designed to run as a single trial. Thank you for the explanation!

[andreyvelich]

Thank you for testing DARTS algorithm in Katib @MatthiasCarnein!

If you take a closer look at the implementation it is intentionally that Katib creates just a single Trial for DARTS.

As described in this doc: https://github.com/kubeflow/katib/tree/master/pkg/suggestion/v1beta1/nas/darts#katib-implementation,
Suggestion generates set of primitives from Experiment NAS Config, number of layers, and algorithm settings.
After that, single Trial evaluates DARTS algorithm using this script: https://github.com/kubeflow/katib/tree/master/examples/v1beta1/trial-images/darts-cnn-cifar10.
Finally, Metrics Collector saves best genotype in the Experiment metrics.

Katib Hyperparameter tuning architecture with Experiment, Suggestion, Trials is not ideally designed for various architecture search algorithms.
That is why we have to follow this model to run DARTS with Katib controllers.