Comments (1)
DropOut and AlphaDropOut on SELU
Thanks for the comments.
As you mentioned from the paper of the scaled exponential linear units https://arxiv.org/abs/1706.02515, on page 6, they recommend not use dropout as the extra variance hinders the convergence of the algorithm when using normalization.
We observed some convergence issues when exploring the hyperparameter space. Although with optimal model configurations, the training procedure was stable.
One thing to keep in mind is that the two best regularization techniques we found in our experiments are early stopping and second ensembling. Since ensembling boosts accuracy from the diversity and variance of models, the interaction of AlphaDropOut with the ensemble might be something interesting to explore. Still, we will try the AlphaDropOut regularization to test the SELU paper recommendation on this regression setting.
from nbeatsx.
Related Issues (16)
- [Question] dropout_prob_exogenous
- Non-EPF Example HOT 4
- Number of stacks confusion HOT 4
- Number of stacks confusion
- Forecasting with long output dimension HOT 1
- Hyperparameter files HOT 5
- Any advice, please. HOT 1
- FCNN realization doesn't take exogenous variables as arguments HOT 2
- nothing
- n_val_weeks in the paper and code do not match HOT 3
- `run_test_nbeatsx` confusion HOT 2
- Missing import statements in nbeats.py HOT 1
- insample_mask and outsample_mask HOT 1
- Can NbeatsX be used for multiple-variate datasets?
- datasets of nbeatsx can not download HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nbeatsx.