Comments (6)
Hi, thanks for your great work!
I'm trying to run the first command in 'prepare.sh', CUDA_VISIBLE_DEVICES=6,7 ./tools/dist_train.sh configs/lvis/detpro_ens_20e.py 2 --work-dir workdirs/collect_data --cfg-options model.roi_head.load_feature=False totol_epochs=1 which is used to generate the CLIP embeddings for precomputed proposals. However, this process will take about 30 days with 8 16g-V100s. And in this issue #4, you already claim that it only takes one day. So I was wondering if I missed any details?
Hi, do you modify the command the use 8 gpus? And make sure the total schedule is 1 epoch which should take about one day.
from detpro.
Yes, I have already changed the GPU numbers to 8, and the total_epochs is also reset to 1 in the command.
And it looks like the running time is still 30+ days after printing some float tensors.
0.20857863751051303
0.19100091827364554
0.1633187772925764
0.12694300518134716
0.2342857142857143
0.23985572587917042
2022-04-20 09:35:50,993 - mmdet - INFO - Epoch [1][50/7665] lr: 1.978e-03, eta: 36 days, 13:22:29, time: 20.610, data_time: 2.224, memory: 8942, loss_rpn_cls: 0.6659, loss_rpn_bbox: 0.1318, loss_bbox: 0.0274, text_cl
s_loss: 2.6543, kd_loss: 7.1762, loss_mask: 1.5504, loss: 12.2060
from detpro.
@cailk totol -> total
from detpro.
@cailk totol -> total
Alright, I'm an idiot. Thank you for the reminder~
from detpro.
@cailk totol -> total
BTW, I'm still wondering why this embeddings generation process produces training loss. Shouldn't only forward computation is required for this?
from detpro.
@cailk I guess it's an implementation issue. Personally speaking I prefer to generating the feature in test mode.
from detpro.
Related Issues (20)
- question about current_mmdetection_Head.pth HOT 1
- Background embedding HOT 1
- Question about the prompt HOT 1
- The prompts for COCO, VOC and Objects365 HOT 1
- Training Time
- query about the order of proposals generated by rpn101
- FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints/exp/fg_bg_5_10_end_ens.pth' HOT 1
- Having issues reproducing DetPro HOT 1
- RPN Training Code
- Training RPN based on the provided code. Rather low performance. HOT 2
- about reprodcution HOT 2
- python: can't open file 'gen_cls_embedding.py': [Errno 2] No such file or directory HOT 1
- Installation of mmdet HOT 1
- Question of prompt
- question about RPN training HOT 1
- some issues about RPN
- Transfer need coco proposals?
- Which config file to use HOT 1
- Training time for DetPro?
- running error
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from detpro.