Comments (9)
Hi, I have uploaded the training log from scratch with the prefix "From_Scratch". You can find them in the corresponding folders now.
The ResNet result is almost the same, but the ResNeXt result is around 0.6%~0.8% lower than training on pretrained models. I guess the reason is that training ResNeXt is more difficult than ResNet. Therefore, a pretrained model for ResNeXt helps a lot for a higher accuracy.
By the way, the accuracy on the paper is the accuracy training from scratch. I update the accuracy on paperswithcode website with the highest accuracy.
from open-aff.
Thanks a lot for the quick reply!
Again - well done!
from open-aff.
Can you provide the training script used for training from scratch ? It seems that the current provided training log from scratch does not use 'train_cifar.py' in this repo.
from open-aff.
Sorry. Can you explain a bit more? I do not understand what "the current provided training log from scratch does not use 'train_cifar.py' in this repo." means?
What I have in mind is that I just renamed train_cifar_mixup.py
to train_cifar.py
when I create this repo. Everything else is the same. It is because my private repo is very redundant and has many codes unrelated with this paper. Therefore, I created this new public repo when I release the code.
from open-aff.
As shown in open-aff/params/cifar100/AFF-ResNet-32/From_Scratch_Log_train_cifar100_cifar100-ASKCFuse-resnet-32-c-4-s-1.log
, the log line is "INFO:root:[Epoch 0] train=0.079624 val=0.107400 loss=3.099419 time: 20.976282". However, in train_cifar.py
, the log format is [Epoch %d] train=%f val=%f loss=%f lr: %f time: %f
. They are different.
from open-aff.
In train_cifar.py
, you can choose cosine or step, not cosine only. It depends whether you add --cosine
in the training script.
I started with step at first, the training from scratch log version. When I tried to use the pretrained model to have a higher accuracy, I added the cosine code because people say that cosine lr can improve the performance so I had a try. I guess it is the reason why you think my log format mismatch, because I change it when I add the cosine code. The current log format is the version after I add the cosine code.
Unfortunately, it seems that cosine didn't perform better than step on my cifar experiment. However, as long as the network is not changed, the network can use the stored params produced by both step and cosine.
from open-aff.
Thank you, I have noticed --cosine option. Except for printing lr, are there any other differences between the training script you used for training from scratch and train_cifar.py
?
from open-aff.
I think I only have made two changes on the code train_cifar.py
when I started to run the very first experiment. The first is adding the label smoothing choice, the second is the adding cosine choice.
What I have in mind is that the training log for AFF-ResNeXt-38-32x4d
was not trained with label smoothing because it is trained before I added the label smoothing choice. But the rest models are all with label smoothing.
Sorry for the inconvenience about the updates in my code. The ideal case is that I trained all these models after everything is fixed with the final version of code. But I have very limited access to GPUs, so I basically presented the training logs and params on the fly.
from open-aff.
Thank you very much for your detailed response.
from open-aff.
Related Issues (20)
- AFF-FPN HOT 3
- ASKCResNetFPN
- 您好,请问AFF模块中的最后一步为什么要乘以2?有什么意义呢?希望收到您的回复 HOT 2
- TypeError: forward() missing 1 required positional argument: 'residual' HOT 2
- resnet50预训练模型 HOT 3
- module -> self.global_att2
- __init__() takes from 1 to 3 positional arguments but 6 were given
- 代码中AFF模块中,为什么要用2×呢? HOT 1
- 请教 这种特征融合思路能否应用到 NLP 中?
- 可否进行实现三种特征图的融合 HOT 1
- Concat and fusion instead of add in iAFF and AFF
- iAFF类的forward函数中, xg2 = self.global_att(xi) 这一行是不是笔误 HOT 1
- 运行模块时报错请教 HOT 1
- aff使用于fpn或者Unet结构 HOT 2
- When the website will be ready?
- 为何将iaff加入到resnet第一层会报如下错误:TypeError: forward() missing 1 required positional argument: 'residual'
- 您好可以发一下iAFF-ResNeXt-50-32x4d训练ImageNet的权重文件吗
- ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 256, 1, 1]) HOT 2
- 关于BN层 HOT 2
- 请问怎么在pytorch直接使用AFF呀? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from open-aff.