sg-nm/cgp-cnn

A Genetic Programming Approach to Designing CNN Architectures, In GECCO 2017 (oral presentation, Best Paper Award)

Designing Convolutional Neural Network Architectures Based on Cartegian Genetic Programming

This repository contains the code for the following paper:

Masanori Suganuma, Shinichi Shirakawa, and Tomoharu Nagao, "A Genetic Programming Approach to Designing Convolutional Neural Network Architectures," Proceedings of the Genetic and Evolutionary Computation Conference (GECCO '17), pp. 497-504 (2017) [DOI] [arXiv]

Requirement

We use the Chainer framework for neural networks and tested on the following environment:

• Chainer version 1.16.0
• GPU: GTX 1080 or 1070
• Python version 3.5.2 (anaconda3-4.1.1)

PyTorch version is here

Usage

Run the architecture search

This code can reproduce the experiment for CIFAR-10 dataset with the same setting of the GECCO 2017 paper (by default scenario). The (training) data are split into the training and validation data. The validation data are used for assigning the fitness to the generated architectures. We use the maximum validation accuracy in the last 10 epochs as the fitness value.

If you run with the ResSet described in the paper as the function set:

python exp_main.py -f ResSet

Or if you run with the ConvSet described in the paper:

python exp_main.py -f ConvSet

When you use the multiple GPUs, please specify the -g option:

python exp_main.py -f ConvSet -g 2

After the execution, the files, network_info.pickle and log_cgp.txt will be generated. The file network_info.pickle contains the information for Cartegian genetic programming (CGP) and log_cgp.txt contains the log of the optimization and discovered CNN architecture's genotype lists.

Some parameters (e.g., # rows and columns of CGP, and # epochs) can easily change by modifying the arguments in the script exp_main.py.

Re-training

The discovered architecture is re-trained by the different training scheme (500 epoch training with momentum SGD) to polish up the network parameters. All training data are used for re-training, and the accuracy for the test data set is reported.

python exp_main.py -m retrain

cgp-cnn's Issues

module 'chainer.functions' has no attribute 'ReLU

There are a couple of issues that I am facing while running the main file
module 'chainer.functions' has no attribute 'ReLU
module 'chainer.functions' has no attribute 'Concat'
relu() missing 1 required positional argument: 'x'

RuntimeError: The size of tensor a (4) must match the size of tensor b (8) at non-singleton dimension 3 multiprocessing.pool.RemoteTraceback:

gpu_id: 0 , [['input', 0, 0], ['S_ResBlock_64_1', 0, 0], ['S_ConvBlock_128_5', 0, 0], ['S_ConvBlock_64_3', 0, 0], ['S_ResBlock_64_5', 2, 3], ['S_ConvBlock_128_3', 1, 1], ['S_ResBlock_32_1', 3, 1], ['Max_Pool', 4, 6], ['Avg_Pool', 7, 6], ['Sum', 5, 6], ['S_ResBlock_32_1', 8, 7], ['Sum', 9, 10], ['Max_Pool', 11, 10], ['S_ConvBlock_64_1', 12, 10], ['S_ConvBlock_128_3', 13, 13], ['S_ConvBlock_32_3', 14, 14], ['full', 15, 15]]

in_data[small_ch_id] = torch.cat([in_data[small_ch_id], tmp * 0], 1) RuntimeError: Sizes of tensors must match except in dimension 2. Got 4 and 8

in_data[small_ch_id] = torch.cat([in_data[small_ch_id], tmp * 0], 1)
RuntimeError: Sizes of tensors must match except in dimension 2. Got 4 and 8

I would like to ask what is the problem? This problem always occurs when the algorithm runs halfway

Error when running on Ubuntu 16.04, chainer 3.1.0

I tried to run the code, error is as follow:

Traceback (most recent call last):
File "/home/data/luopengting/workplace/python/cgp-cnn-master/cnn_train.py", line 118, in call
optimizer.update(model, x, t)
File "/home/b3434/anaconda3/envs/chainer/lib/python3.5/site-packages/chainer/optimizer.py", line 534, in update
loss = lossfun(*args, **kwds)
File "/home/data/luopengting/workplace/python/cgp-cnn-master/cnn_model.py", line 160, in call
outputs[nodeID], tmp_num = getattr(self, name)(outputs[self.cgp[nodeID][1]], outputs[self.cgp[nodeID][1]], self.train)
File "/home/data/luopengting/workplace/python/cgp-cnn-master/cnn_model.py", line 58, in call
x = getattr(self, name)(x, not train)
TypeError: call() takes 2 positional arguments but 3 were given
gpu_id: 0 , eval: 0.0

---

can u help me, plz ? thx a lot.

How to set up multi-gpu training

Hello, first of all, I would like to thank you for your excellent work. In addition, I am not very familiar with evolutionary algorithms. I want to know how can I count the number of all parameters in the network.

I have made some changes to your work. I changed the single label to multi-label recognition. The loss uses multilablemarginloss, but the problem is that the loss is only available at epoch1, and the rest are all 0. Please ask me how to change Change this status quo

Hello author, what's going on here with the evaluation function? The goal has been changed to multi-target, but I don't know how to do it. Forgive me if I'm a beginner.

Hello author, what's going on here with the evaluation function? The goal has been changed to multi-target, but I don't know how to do it. Forgive me if I'm a beginner.

Hello, can you generate multiple parent networks at the same time when the network is initialized

Hello author, first of all, thank you very much for your work, how do I change the code if I want to recognize the string

Hello author, first of all, thank you very much for your work, how do I change the code if I want to recognize the string

Error when running on Ubuntu 16.04

Traceback (most recent call last):
File "/development/cgp-cnn/cnn_train.py", line 118, in call
optimizer.update(model, x, t)
File "/usr/local/lib/python2.7/dist-packages/chainer/optimizer.py", line 528, in update
loss = lossfun(*args, **kwds)
File "/development/cgp-cnn/cnn_model.py", line 160, in call
outputs[nodeID], tmp_num = getattr(self, name)(outputs[self.cgp[nodeID][1]], outputs[self.cgp[nodeID][1]], self.train)
File "~/development/cgp-cnn/cnn_model.py", line 58, in call
x = getattr(self, name)(x, not train)
TypeError: call() takes exactly 2 arguments (3 given)

I get the previous error when running the example given in the README.md I am happy to provide any other information required.

The script continues running but outputs this error every time and restarts.

TypeError: concat() missing 1 required positional argument: 'xs'

        elif name == 'sum':
            links += [('_'+name+'_'+str(i), F.concat())]