Error pretrain

[shuangli59]

Error pretrain

[Cysu]

It seems that the lmdb is corrupted. What are the sizes of the lmdbs (by du -sh data/psdb/pretrain_db/*_lmdb)? Are there any errors in executing the data generation script?
scripts/make_db.sh /path/to/the/downloaded/dataset.zip

[nhquanst]

Hi @ShuangLI59
I am using GPU K1100M and VGG16_iter_50000.caffemodel

When i trained with "experiments/scripts/train.sh 0" command. I got following error:

Solving...
F0808 16:14:54.009300 23672 syncedmem.cpp:56] Check failed: error == cudaSuccess (2 vs. 0) out of memory
*** Check failure stack trace: ***
experiments/scripts/train.sh: line 46: 23672 Aborted (core dumped) python2 tools/train_net.py --gpu ${GPU_ID} --solver models/${PT_DIR}/${NET}/solver.prototxt --weights output/${DATASET}_pretrain/${NET}_iter_50000.caffemodel --imdb ${TRAIN_IMDB} --iters ${ITERS} --cfg experiments/cfgs/train.yml ${EXTRA_ARGS}

---

I think my GPU memory not enough. May i use another pretrain model?

[Cysu]

Hi, @nhquanst. Sorry but we currently only use VGG-16 as the pretrained model, which requires at least 6GB memory I think.

[nhquanst]

@Cysu, thanks a lot! I will try with another GPU.

[duanLH]

@nhquanst @pribadihcr @ShuangLI59 Do you run the code successfully now?

[nhquanst]

sorry, not yet. Because of limitation of my server hardware

On Wed, Aug 17, 2016 at 5:06 PM, duanLH [email protected] wrote:

@nhquanst https://github.com/nhquanst @pribadihcr
https://github.com/pribadihcr @ShuangLI59
https://github.com/ShuangLI59 Do you run the code successfully now?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#3 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/APiUHJskH2UH6rKoNiKr1s_sCLihy8YMks5qgt0UgaJpZM4JTT4m
.

[Cysu]

We have improved our method significantly and now we use a better version of caffe that provides memory optimization. Maybe you could give it a try again. And please feel free to reopen this issue if there are further problems.

[nhquanst]

thank you very much, i will try it as soon as possible

…

On Tue, Feb 28, 2017 at 6:32 PM, Tong Xiao ***@***.***> wrote: Closed #3 <#3>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/APiUHGa5Rj1sv6WbZXEOMhsg8UukqY5xks5rhAXngaJpZM4JTT4m> .