Is there an estimate of parallel/thread-safe Neural Net classifier?

[haifengl]

Is there an estimate of parallel/thread-safe Neural Net classifier?

[haifengl]

What's your use case? Do you want it mt-safe for training or testing/prediction? Thanks!

[owlmsj]

My case: I am looking for a NeuralNet capable to train a corpus of 20GB data in a 8-thread server. Already tried @deeplearning4j and other java impl., but didn't like any of it.

Predicting not mt-safe could be bypassed implementing a resource manager (such as priority queue or duplicating models).

Since Smile was clean, source independent and intuitive (got running mt-thread SVM and Random Forest in a couple minutes), thought to ask about project priorities :)

[haifengl]

I doubt how much we can gain by making training of neural net mt-safe. There are a lot of shared data (weights in the net). To make it mt-safe, locking/mutex will make it slow. Can you have a try with your data in single thread to see how long it takes? Thanks!

BTW, Neural Net of smile is traditional back propagation algorithm. It is different deep learning and is suitable for different problems.

[owlmsj]

Wow, it makes sense. I will give a try and and im closing this issue. Thanks for the answer!

[haifengl]

Any updates? Is it too slow for you? Thanks!

[owlmsj]

Found some bugs in my code at "processing features" phase. After validating this, I am going to test and post some results here!

[haifengl]

Thanks! If you have any questions about Smile's NeuralNetwork, please feel free open a ticket.

[owlmsj]

Hi, finally got my first results.

Training 10k instances (14k attributes) got 33 seconds to learn.

Train size:10000
Training in: 33.127 seconds
Test size: 1000
Finishing Cross Validation
Final precision: 0.91607128289355
Final recall: 0.908765520895934

Next step: 100k

[haifengl]

Given the sample size, the number of attributes, this looks very good to you. How do you feel? Especially compared to your experience with other packages? Thank you very much for reporting the results!

Btw, what's neural net settings?

[haifengl]

Can you please write a post on our wiki describing what you are doing? It doesn't have to be long. Simply describe your data and your problem, why you choose neural net, how you train and test the model (and some code snippets). This will be extremely valuable for others. Thanks a lot in advance!

[haifengl]

BTW, our SVM training algorithm is also an online algorithm. You may want to give it a try and compare it to neural net. Thanks!

[owlmsj]

My experience has been awesome. Please, keep this great job, elegant and effective :)

And sure, it would be great! I am going to test everything before and then I will write the post.

About SVM, I tried it in first time. But got results on 0.7 / 0.9 in prec/recall. And it got time to train (like 250 seconds). But I will give a second try, right after implement a naive grid search.

Thanks!

[haifengl]

How's going with 100k?

[owlmsj]

Hi! Really sorry for the delay. I was "stormed" by my Master's thesis qualification (writing and evaluating a lot what already did).

Getting back to the track tomorrow and hope to give good news (results and blog post) soon.

[haifengl]

Thanks for updates! Hope everything is fine with you.