szilard/benchm-dl

Playing with various deep learning tools and network architectures

benchm-dl's Issues

H2O's GPU Deep Learning

Sneak preview for H2O's Deep Water project:
https://github.com/h2oai/h2o-3/blob/master/examples/deeplearning/notebooks/deeplearning_benchmark_mnist.ipynb

To try this script yourself on Ubuntu 16 with CUDA8, CUDNN5, simply pip install this package: https://slack-files.com/T0329MHH6-F2TN01TUN-b0577b68da

You will also want to install the matching mxnet egg: https://slack-files.com/T0329MHH6-F2PU85GEN-2f4fee68e2

More instructions and links provided here: https://github.com/h2oai/deepwater

Please note that this is not a final version for benchmark purposes, but rather a community preview among friends.

g2.2xlarge: 47s
p2.2xlarge: TBD
2x 10-core Xeon with GTX1080: 26s

Cheers,
Arno

You should remove or validate the accuracy benchmarks

Assuming accurate model creation and sufficient benchmarking there should be no accuracy delta between different libraries in creating different models. The results you've presented are all far below the threshold of showing any real difference between models, and I'd assert that they're well within noise parameters.

I'd suggest either removing them or confirming them, but given the size of the differences you're reporting on the chance of them being statistically significant is very close to zero.

Theano backend?

Hi, I found your analysis very interesting. May I suggest to complete the section "GPU vs CPU" using Theano backend? I heard that for CPU, Theano is faster than Tensorflow.