Comments (4)
@clw5180 I've run some experiments with YOLOv5s, which seem to show some early improvement. I have not tried larger models, and improvements from one size to another sometimes does not extrapolate well, so I can't draw any conclusions across the entire v5 lineup, but so for small models it appears to help.
One warning is that FReLU increases training requirements greatly, similar to Swish perhaps, not quite as bad as Mish, so it's probably not a viable replacement for all activation functions in a model, but might be suitable for partial replacement.
from funnelact.
Hi @glenn-jocher ,
(1) you should use torch.max(), which is different from MegEngine API.
(2) yes you could try to use FReLU only in the backbone first, then try it in head. you could also replace all the LeakyReLU activations to FReLU because there is only one activation in each BottleneckCSP which are not originally many.
from funnelact.
@nmaac great, thanks for the feedback! We'll try to experiment with this and see if it helps our results!
from funnelact.
We'll try to experiment with this and see if it helps our results!
Hi glenn, is there any improment on COCO or VOC dataset about FReLU ? Or how does it perform on the verification set? Thanks !
from funnelact.
Related Issues (13)
- how to apply in PSPNet HOT 9
- Is there any special consideration of initialization ? HOT 2
- how to apply in InceptionV3 HOT 1
- Funnel Activation implementation question HOT 2
- 应用到cifar-10小型数据集
- Why do you apply relu instead of frelu to first activation function? HOT 2
- Apply FReLU to MobileNetV3 HOT 2
- Strategies to avoid overfitting HOT 6
- replace frelu with depth-wise conv + relu HOT 3
- Difference between Parametric Pooling and Depthwise (Separable) Conv? HOT 7
- What's the difference between funnelact and maxout? HOT 2
- base_width报错 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from funnelact.