Comments (4)
Hi @DelinQu, thank you for raising this issue.
It seems that get_logger
is problematic as you pointed out and I'm currently working on this. I'll let you know if there is any progress.
Also, if you are interested in using hydra_DDP
branch of this repo, I would recommend using my clone version of that. I made a few commits which solves serious bug(It was not using DDP at all) a while ago on that branch, but forgot to make a PR to apply that fix to this repo. I will make PR soon, but it could take some time.
from pytorch-template.
Hi @DelinQu, thank you for raising this issue.
It seems that
get_logger
is problematic as you pointed out and I'm currently working on this. I'll let you know if there is any progress.Also, if you are interested in using
hydra_DDP
branch of this repo, I would recommend using my clone version of that. I made a few commits which solves serious bug(It was not using DDP at all) a while ago on that branch, but forgot to make a PR to apply that fix to this repo. I will make PR soon, but it could take some time.
Thanks for your replying SunQpark, I will follow your DDP! 😃
from pytorch-template.
Oh thanks, but I'll let this issue open yet!
from pytorch-template.
Oh thanks, but I'll let this issue open yet!
Hi SunQpark,
your repo has really helped me tremendously, but I got another problem when the training process is early stopped:
Although it doesn't affect my models much, the error persists. My configuration file is as follows:
n_cpu: 8
n_gpu: 8
batch_size: 4
learning_rate: 0.0001
weight_decay: 0
scheduler_step_size: 50
scheduler_gamma: 0.1
status: train
trainer:
epochs: 500
logging_step: 100
monitor: min loss/valid
save_topk: 5
early_stop: 10
tensorboard: true
from pytorch-template.
Related Issues (20)
- Loss function HOT 1
- Any plans to support Wandb Hyperparameter Searching?
- Strange bugs occur when the number of Gpus trained and tested is inconsistent
- usage of shuffle=True when using SubsetRandomSampler HOT 1
- DataLoader example
- python train.py --resume path/to/checkpoint HOT 1
- Save best model HOT 1
- Model is moved to GPU after the optimizer is instatiated, resulting in a performance hit. HOT 4
- Setting 'early_stop: 0' does not disable it HOT 2
- Passing an iterable from config.json HOT 2
- Add custom flag & override from CLI HOT 1
- Some features I have implemented.
- ReduceLROnPlateau lr_scheduler HOT 3
- Only `data_loader.data_loaders` is plural HOT 5
- Any support for multiple loss functions? HOT 5
- Latest checkpoint HOT 8
- Some thing wrong with add_histogram function HOT 2
- Adding mesh to Tensorboard not working via TensorboardWriter HOT 2
- Any plans to support DistributedDataParallel? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pytorch-template.