liamcli/randomnas_release

Code release for paper "Random Search and Reproducibility for NAS"

randomnas_release's Issues

Confusion between probabillity in [0, 1] and percentage

evaluate() in searchers/random_weight_share.py has return 1-top1.avg on line 219. Since it is a percentage, it should be return 100-top1.avg

Questions about the accuracy of the supernet?

Hi, thanks for your great job! When I try to reproduce your results, I train the supernet using he default parameters but the accuracy of supernet is only about 50%. Is it right?
Best!

[request] Avoid hard code

https://github.com/liamcli/randomNAS_release/blob/master/searchers/random_weight_share.py#L2 hard coded the path of the dep, which caused an error when I run it.

About arch saving

I ran the code "random_weight_share.py" ,
in the final, it shows
"File "random_weight_share.py", line 163, in main
with open('./tmp/arch', 'w') as f:
IsADirectoryError: [Errno 21] Is a directory: './tmp/arch'"

Is ti need to add a txt file in the path like "./tmp/arch/arch.txt" ?
Or I miss some details?

Code for search & train WITHOUT weight sharing

The authors report the results for the random search baseline without any weight sharing but do not provide the code to reproduce the baseline's result. This would be helpful in fully reproducing their tables in the results section and not just their weight-shared versions.

input argument is unclear to me

Hi,

While trying to run python parse_cnn_arch.py "[arch_str]", I am confused about what should be the argument? What is arch_str?

Could you provide the searching configuration of the CNN model?

As title.

RuntimeError: CUDNN_STATUS_EXECUTION_FAILED

Got the RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED when executing the CNN model.

Traceback

(torch) baijin@pku:~/Code/randomNAS/searchers$ python random_weight_share.py
07/19 12:41:24 PM Namespace(batch_size=64, benchmark='cnn', config='search', epochs=100, eval_only=0, grad_clip=0.25, init_channels=16, save_dir=None, seed=100)
[Errno 2] No such file or directory: './result/cnn/random/trial100/weights.pt'
07/19 12:41:31 PM param size = 1.930618MB
07/19 12:41:31 PM param size: 1930618
07/19 12:41:31 PM Args: {'data': '/home/baijin/Dataset/cifar10/', 'epochs': 100, 'learning_rate': 0.025, 'batch_size': 64, 'learning_rate_min': 0.001, 'momentum': 0.9, 'weight_decay': 0.0003, 'init_channels': 16, 'layers': 8, 'drop_path_prob': 0.3, 'grad_clip': 0.25, 'train_portion': 0.5, 'seed': 100, 'log_interval': 50, 'save': './result/cnn/random/trial100', 'gpu': 0, 'cuda': True, 'cutout': False, 'cutout_length': 16, 'report_freq': 50}
07/19 12:41:31 PM Model total parameters: 1930618
07/19 12:41:31 PM budget: 39062
Traceback (most recent call last):
  File "random_weight_share.py", line 184, in <module>
    main(args)
  File "random_weight_share.py", line 155, in main
    searcher.run()
  File "random_weight_share.py", line 75, in run
    self.model.train_batch(arch)
  File "/home/baijin/Code/randomNAS/benchmarks/cnn/darts/darts_wrapper_discrete.py", line 157, in train_batch
    logits = self.model(input, discrete=True)
  File "/home/baijin/Lib/anaconda3/envs/torch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/baijin/Code/l-darts/cnn/model_search.py", line 104, in forward
    s0 = s1 = self.stem(input)
  File "/home/baijin/Lib/anaconda3/envs/torch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/baijin/Lib/anaconda3/envs/torch/lib/python3.6/site-packages/torch/nn/modules/container.py", line 67, in forward
    input = module(input)
  File "/home/baijin/Lib/anaconda3/envs/torch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/baijin/Lib/anaconda3/envs/torch/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 282, in forward
    self.padding, self.dilation, self.groups)
  File "/home/baijin/Lib/anaconda3/envs/torch/lib/python3.6/site-packages/torch/nn/functional.py", line 90, in conv2d
    return f(input, weight, bias)
RuntimeError: CUDNN_STATUS_EXECUTION_FAILED

Environment

Collecting environment information...
PyTorch version: 0.3.1.post2
Is debug build: No
CUDA used to build PyTorch: 9.0.176

OS: Ubuntu 18.04.2 LTS
GCC version: (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
CMake version: version 3.10.2

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 9.1.85
GPU models and configuration:
GPU 0: GeForce RTX 2080 Ti
GPU 1: GeForce RTX 2080 Ti

Nvidia driver version: 418.56
cuDNN version: Could not collect

Versions of relevant libraries:
[pip3] numpy==1.16.3
[conda] blas                      1.0                         mkl    https://mirrors.ustc.edu.cn/anaconda/pkgs/free
[conda] cuda90                    1.0                  h6433d27_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch
[conda] mkl                       2018.0.3                      1    https://mirrors.ustc.edu.cn/anaconda/pkgs/main
[conda] pytorch                   0.3.1           py36_cuda9.0.176_cudnn7.0.5_2  [cuda90]  https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch
[conda] torchvision               0.2.0                    py36_0    https://mirrors.ustc.edu.cn/anaconda/pkgs/main

I tried python train.py --auxiliary --cutout in liamcli/darts/cnn and I encounted the same RuntimeError: CUDNN_STATUS_EXECUTION_FAILED error.

Could you please help me to check what's the problem with the environments and the DARTS? Thank you very much.

NAS for custom dataset

Hi there, thanks for providing your codes! I am trying to use your repo for my thesis, I was wondering if you could please give some clue, what should I do if I wanna search architecture for my own dataset and in a classification task? I really appreciate your help.

Best