jiaminren / randwirenn Goto Github PK
View Code? Open in Web Editor NEWPytorch Implementation of: "Exploring Randomly Wired Neural Networks for Image Recognition"
Pytorch Implementation of: "Exploring Randomly Wired Neural Networks for Image Recognition"
Hi, I have a question about weight initialization of node's additive aggregation here.
At model.py#L41, weights are initialized with torch.ones
, not torch.zeros
. This will make actual weights to be initialized with 0.7311
, not 0.5000
. Was it intended? (Though it won't affect much on training, I'm just curious about it.)
I try to run your code on 4 1080Ti GPUs, and I found that there is not enough memory under your default setting (256 batchsize, regular model). So I want to know how could you run the model on 8 Xp GPUs.
Thanks a lot.
Hi, dear author,
thanks for your great work and open source code. I wonder is this repo. the official implementation of original paper? Thank you.
PS. It would be better if you can post your implementation performance benchmark in readme file.
Hi, I’m author of https://github.com/seungwonpark/RandWireNN.
One SO question brought me here. At first, I was excited to see another implementation of RandWireNN. However, I’ve observed that much of your code resembles my implementation(plus, you even emailed me last Saturday), and was disappointed to see that there are no statement referencing my repo.
I’m not saying that this repo is just copy-and-paste of mine. There are several useful features that are not included in mine: multi-GPU support and tool for ImageNet data loading, and so on.
I would like to kindly ask you to add reference to my repository, if you have referred it.
In your implementation line 71~73 in model.py
, nodes are divided into input and first-order ones. This may lead that second(or higher)-order(from input node) ones are ignored. Your implementation only consider the situation that the max distance between input and output node is 2. Am I right?
Hi,
thanks for your contribution of NAS, it is very nice work.
but, the pytorch's data loader is very slow when you use a big dataset likes ImageNet2012.
Fortunately, I found mxnet's dataloader is a nice work, you can package ImageNet2012
into a rec file, then call mxnet's dataloader interface in your code, and convert mxnet data
to pytorch data.
Hi @JiaminRen Thanks for you work.
Use FP16 training on the basis of your code to achieve 75% accuracy
Have you realized the faster rcnn + FPN detection training of RandWireNN as backbone?
If you implemented it, is it implemented in mmdetection? Can you publish the training code? Thanks.
Hi, Jiamin. Many thanks for sharing your code.
I run the train.sh and found that
broadcast_params(model)
makes code stuck.
When I remove this code, the training is still stuck by
losses.update(reduced_loss.item(), input.size(0)) top1.update(reduced_prec1.item(), input.size(0)) top5.update(reduced_prec5.item(), input.size(0))
.
I used single node 8 GPUs for training. The training procedure is the same as you described.
Do you know why this happens?
Hello, I have an error message, I don't know if this file is missing. I hope to get your reply. Thank you。
FileNotFoundError: [Errno 2] No such file or directory: 'checkpoint/regular_c109_n32/conv2.yaml'
请问这个代码里面是复现了论文的神经架构搜索吗?还是只训练了最好的架构。训练过程中网络的结构会改变吗?
Is this code retrieving the neural architecture search for the paper? Still only trained the best architecture. Will the structure of the network change during the training process?
what's the meaning of rand and world_size? They cannot be obtained from "os.environ". The program throws a error when it comes to here.
Hi, I am excited to run your code. I run it on 4 TITAN Xp GPU. And yet, each epoch roughly takes 2 hours. Is that normal? What about your time to train an epoch?
Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.