philferriere/dlwin

GPU-accelerated Deep Learning on Windows 10 native

dlwin's Issues

Is there any reason for choosing 10.0.10240 over 10.0.10584?

Is there any reason why you're showing the older version of Windows 10 SDK in the install list?

cuDNN 5.1 - August 10, 2016 build no longer available

The docs specify to get cuDNN Library for Windows10 dated August 10, 2016 but that doesn't seem to be available anymore...

https://developer.nvidia.com/rdp/cudnn-download

I'm going to try cuDNN v5.1 (Jan 20, 2017), for CUDA 8.0...

Many thanks to the author. Everything works great!!!

I am a very long time (2 weeks) tried solve this problem. I could not. However, this manual helped me. Now everything works perfectly. Many thanks to the author. I used:

• Window 10
• Anaconda with python 3.5
• Tneano 0.9.0rc1 (I made: pip install --upgrade --no-deps git+git://github.com/Theano/Theano.git)
• keras 1.2.2 (I made: pip install keras)

Everything is great!!! Thanks a lot, Phil!!!

ERROR (theano.gpuarray): Could not initialize pygpu, support disabled

Followed the instructions on Windows 10 with a Nvidia 1080 & 1070 GPU and the four backend checks ran fine. I moved to the next step to validate the Theano install.

set THEANO_FLAGS=%THEANO_FLAGS_CPU%
and
set THEANO_FLAGS=%THEANO_FLAGS_GPU%

ran fine. However, when I tried:
set THEANO_FLAGS=%THEANO_FLAGS_GPU_DNN%

I get this error
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled

Haven't found a good answer on Google that help me correct this. Any ideas?

Thanks!

THEANO_FLAGS_CPU=
floatX=float32,device=cpu

THEANO_FLAGS_GPU=
floatX=float32,device=cuda0,dnn.enabled=False,gpuarray.preallocate=0.8

THEANO_FLAGS_GPU_DNN=
floatX=float32,device=cuda0,optimizer_including=cudnn,gpuarray.preallocate=0.8,dnn.conv.algo_bwd_filter=deterministic,dnn.conv.algo_bwd_data=deterministic,dnn.include_path=c:/toolkits.win/cuda-8.0.61/include,dnn.library_path=c:/toolkits.win/cuda-8.0.61/lib/x64

TheanoConfigWarning: Config key 'THEANO_FLAGS_GPU' has no value, ignoring it

Hello,
Thanks for the clear and helpful instruction. I'm still having problems running Theano on GPU and receive this error when I run the test file:

**> ...\Anaconda2\lib\site-packages\theano\configdefaults.py:16: TheanoConfigWarning: Config key 'THEANO_FLAGS_GPU' has no value, ignoring it

from theano.configparser import (AddConfigVar, BoolParam, ConfigParam, EnumStr,
[Elemwise{exp,no_inplace}(<TensorType(float64, vector)>)]
Looping 1000 times took 12.192000 seconds
Result is [ 1.23178032 1.61879341 1.52278065 ..., 2.20771815 2.29967753
1.62323285]
Used the cpu**

Thanks!

A few suggestions

Python 2.7

Is Python 2.7 really necessary? I install Theano+Keras pretty much the same way as you do (without OpenBLAS and cuDNN though), but I use Anaconda with Python 3 and everything works fine.

MinGW

The installation of MinGW can be simplified, because there's a conda package for it. Just conda install mingw.

pip + git

Instead of git clone and python setup.py you can just use a single line of pip to install Theano and Keras:

pip install git+https://github.com/Theano/[email protected]
pip install git+https://github.com/fchollet/[email protected]

Thanks for the guide!

Command prompt different

Your version of command prompt is different than me. How can I get yours?

Fix for "vcvars64.bat" error

If you get the error:

nvcc fatal   : Microsoft Visual Studio configuration file 'vcvars64.bat' could not be found
for installation at 'C:/Program Files (x86)/Microsoft Visual Studio 12.0/VC/bin/../..'

The solution is:

1. Copy $VS12/VC/bin/x86_amd64 to $VS12/VC/bin/amd64
2. Rename $VS12/VC/bin/amd64/vcvarsx86_amd64.bat to vcvars64.bat

CUDA Installer Failed

The Cuda installer failed every time with a "CUDA Installer Failed" message, just like the one explained here:

https://devtalk.nvidia.com/default/topic/1024458/cuda-setup-and-installation/nvidia-cuda-version-9-0-installer-failed/

I solved it by unchecking the "Visual Studio Integration" part in the Custom Installation Options, as the link suggested.

Can anyone be kind and share CUDA 8 for win10 for me?

I install the CUDA but there is only 9.0 version on Nvida website,and it doesn't work, at lest in my test.
So could you please share a 8.0 version for win10 ?
Thanks!

I follow you tutorial and finish installing theano but can't run your test code

C:\Users\Administrator\PycharmProjects\test_image_blending>python cpu_gpu_test.py
('blas.ldflags=', '')
Traceback (most recent call last):
File "cpu_gpu_test.py", line 13, in
mf = theano.function([X, Y], X.dot(Y))
File "C:\Users\Administrator\Anaconda2\lib\site-packages\theano-0.8.2-py2.7.egg\theano\compile\function.py", line 320, in function
output_keys=output_keys)
File "C:\Users\Administrator\Anaconda2\lib\site-packages\theano-0.8.2-py2.7.egg\theano\compile\pfunc.py", line 479, in pfunc
output_keys=output_keys)
File "C:\Users\Administrator\Anaconda2\lib\site-packages\theano-0.8.2-py2.7.egg\theano\compile\function_module.py", line 1777, in orig_function
defaults)
File "C:\Users\Administrator\Anaconda2\lib\site-packages\theano-0.8.2-py2.7.egg\theano\compile\function_module.py", line 1641, in create
input_storage=input_storage_lists, storage_map=storage_map)
File "C:\Users\Administrator\Anaconda2\lib\site-packages\theano-0.8.2-py2.7.egg\theano\gof\link.py", line 690, in make_thunk
storage_map=storage_map)[:3]
File "C:\Users\Administrator\Anaconda2\lib\site-packages\theano-0.8.2-py2.7.egg\theano\gof\vm.py", line 1047, in make_all
self.updated_vars,
File "C:\Users\Administrator\Anaconda2\lib\site-packages\theano-0.8.2-py2.7.egg\theano\gof\vm.py", line 913, in make_vm
vm = CVM(
NameError: global name 'CVM' is not defined

Is it possible to uninstall all the software installed with VS 2015 Community Edition at the end?

Hello,
as by title I wonder if it is possible to uninstall the very large VS 2015 packages, once Theano and everything is installed and performs properly.
Thank you.

Missing cudnn.h file?

Can not use cuDNN on context None: cannot compile with cuDNN. We got this error:
b'C:\Users\DanMo\AppData\Local\Temp\try_flags_jfukzb6c.c:4:19: fatal error: cudnn.h: No such file or directory\r\ncompilation terminated.\r\n'

When I set my THEANO_FLAGS to CPU and GPU they worked perfectly, but when I type

set THEANO_FLAGS=%THEANO_FLAGS_GPU_DNN%

I get the error about cudnn.h is missing.

The thing is, when I go to my sysvariables

this is what I have for THEANO_FLAGS_GPU_DNN

floatX=float32,device=cuda0,optimizer_including=cudnn,gpuarray.preallocate=0.8,dnn.conv.algo_bwd_filter=deterministic,dnn.conv.algo_bwd_data=deterministic,dnn.include_path=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\include,dnn.library_path=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\lib\x64

And I'm able to clearly see cudnn.h file inside

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\include

Any ideas?

Regarding Visual Studio

Hi,
The latest update says 'doesn't require installing Visual Studio', but the first step entails:
'Download Visual Studio Community 2015 with Update 3 (x86)'
Any clarifications?

Windows 8.1 Compatibility

Can I follow the instruction on windows 8.1 too? Will it be compatible to windows 8.1?

TensorFlow -> Theano for Keras

Hi Phil,

You are my hero - I am finally able to use GPU with Windows 10 thanks to your instruction! I suggest one improvement - Keras should be switched from TensorFlow (default) to Theano in keras.json file (or alternatively via env var) - see https://keras.io/backend/. Without this step Keras GPU example tries to use TF and fails to run if it is not installed.

DEBUG: nvcc STDOUT mod.cu

I am getting lots of error message like this -

DEBUG: nvcc STDOUT mod.cu
Creating library C:/Users/Prithi/AppData/Local/Theano/compiledir_Windows-10-10.0.14393-Intel64_Family_6_Model_42_Stepping_7_GenuineIntel-2.7.12-64/tmppkp9hb/b7b90bf8ebe2fac761d67bb8406d5516.lib and object C:/Users/Prithi/AppData/Local/Theano/compiledir_Windows-10-10.0.14393-Intel64_Family_6_Model_42_Stepping_7_GenuineIntel-2.7.12-64/tmppkp9hb/b7b90bf8ebe2fac761d67bb8406d5516.exp

Openblas Dependency

When I completed the process, the test python program 'mnist_cnn.py‘ worked well in GPU0 but failed with CPU only mode. In this case, the problem lies in the incorrect configuration of OPENBLAS.

If we wanna use cpu-only keras, the openblas should be configured as follows:

1. Download the precompiled libopenblas.dll from openblas/v0.2.14/
2. Download mingw64_dll.zip as well
   Mine is OpenBLAS-v0.2.14-Win64-int32.zip.

Extract the dll in bin folder of the zip to C:\openblas. Also extract all dlls in mingw64 to the same location

Set blas.ldflags=-LC:\openblas -lopenblas in THEANO_FLAGS (or .theanorc.txt)

error: GPU device not available

Getting the following error, any help would be greatly appreciated!

ERROR (theano.sandbox.cuda): Failed to compile cuda_ndarray.cu: ('nvcc return st atus', 1, 'for cmd', 'nvcc -shared -O3 --use-local-env --cl-version=2008 -Xlinke r /DEBUG -D HAVE_ROUND -m64 -Xcompiler -DCUDA_NDARRAY_CUH=18715462c72ed6afcd7ca5 d52813ce90,-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,/Zi,/MD -IC:\toolkits\a naconda2-4.2.0\lib\site-packages\theano-0.8.2-py2.7.egg\theano\sandbox\cud a -IC:\toolkits\anaconda2-4.2.0\lib\site-packages\numpy\core\include -IC: \toolkits\anaconda2-4.2.0\include -IC:\toolkits\anaconda2-4.2.0\lib\site- packages\theano-0.8.2-py2.7.egg\theano\gof -o C:\Users\viksit\AppData\Loc al\Theano\compiledir_Windows-10-10.0.14393-Intel64_Family_6_Model_60_Stepping_ 3_GenuineIntel-2.7.12-64\cuda_ndarray\cuda_ndarray.pyd mod.cu -LC:\toolkits\ anaconda2-4.2.0\libs -LC:\toolkits\anaconda2-4.2.0 -lcublas -lpython27 -lcuda rt')
WARNING (theano.sandbox.cuda): CUDA is installed, but device gpu is not availabl e (error: cuda unavailable)
nvcc fatal : Value '2008' is not defined for option 'cl-version'

['nvcc', '-shared', '-O3', '--use-local-env', '--cl-version=2008', '-Xlinker', '/DEBUG', '-D HAVE_ROUND', '-m64', '-Xcompiler', '-DCUDA_NDARRAY_CUH=18715462c72ed6afcd7ca5d52813ce90,-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,/Zi,/MD', '-IC:\toolkits\anaconda2-4.2.0\lib\site-packages\theano-0.8.2-py2.7.egg\theano\sandbox\cuda', '-IC:\toolkits\anaconda2-4.2.0\lib\site-packages\numpy\core\include', '-IC:\toolkits\anaconda2-4.2.0\include', '-IC:\toolkits\anaconda2-4.2.0\lib\site-packages\theano-0.8.2-py2.7.egg\theano\gof', '-o', 'C:\Users\viksit\AppData\Local\Theano\compiledir_Windows-10-10.0.14393-Intel64_Family_6_Model_60_Stepping_3_GenuineIntel-2.7.12-64\cuda_ndarray\cuda_ndarray.pyd', 'mod.cu', '-LC:\toolkits\anaconda2-4.2.0\libs', '-LC:\toolkits\anaconda2-4.2.0', '-lcublas', '-lpython27', '-lcudart']
[Elemwise{exp,no_inplace}(<TensorType(float32, vector)>)]
Looping 1000 times took 11.654000 seconds
Result is [ 1.23178029 1.61879337 1.52278066 ..., 2.20771813 2.29967761
1.62323284]
Used the cpu

Not able to import the theano

I have followed the guides step by step, but not able to import theano. When import theano,I got the following error

File "d:\Users\myname\Anaconda3\envs\py36\lib\site-packages\theano\gof\cmodule.py", line 1797, in _try_compile_tmp
os.remove(exe_path + ".exe")
PermissionError: [WinError 32]

My system info:

1. Win 10 64bit
2. Anconda 3 4.4, Python 3.6

Have tried google but not able to find a solution.

Thanks!

powershell, vs2015/2017

Thanks for this detailed set of instructions.
There is a lot of old and now incorrect instructions trying to get this tech stack going on windows - yours is the most complete!

I'm trying to run through them now and have some suggestions. Happy to PR them once I get this all going.

1. Replace some manual steps with powershell where possibe.
   eg: This is what I have for setting mingw paths. There are some other spots that can be scripted too

# Powershell - run elevated
Write-Host "Define the sysenv variable MINGW_HOME"
$mingwHome = "c:\toolkits\mingw-w64-5.4.0"
[Environment]::SetEnvironmentVariable( "MINGW_HOME", $mingwHome, "Machine")
Write-Host "Appending mingw bin's to PATH"
[Environment]::SetEnvironmentVariable( "Path", $env:Path + ";$mingwHome\mingw64\bin", "Machine")

2. VS2015 Community is no longer available for download except for MSDN subscribers. I thought I would try my luck with VS2017.
   If so, could we split the VS2015/VS2017 instructions into separate readme's (vs2015.md, vs2017.md) with a pointer from the main readme.md.
   The main readme.md could have something like

Stable: Install Visual Studio 2015 (hyperlinked)
Experimental: Install Visual Studio 2017 (hyperlinked)

Splitting the instructions down allows a set of updated instructions to be in the pipeline that may not always be the easy happy path but available for those who want to try and allows you to get feedback / PRs for refinement.

fyi I'm coming at this as a student of fast.ai
Thoughts?

CUDA_ERROR_OUT_OF_MEMORY

It works fine with THEANO_FLAGS_GPU:

set THEANO_FLAGS=%THEANO_FLAGS_GPU%

C:\Users\resuly\Desktop>python cpu_gpu_test.py
Can not use cuDNN on context None: Disabled by dnn.enabled flag
Preallocating 1638/2048 Mb (0.800000) on cuda0
Mapped name None to device cuda0: GeForce GTX 960M (0000:01:00.0)
[GpuElemwise{exp,no_inplace}(<GpuArrayType<None>(float32, (False,))>), HostFromGpu(gpuarray)(GpuElemwise{exp,no_inplace}.0)]
Looping 1000 times took 0.274695 seconds
Result is [ 1.23178029  1.61879349  1.52278066 ...,  2.20771813  2.29967761
  1.62323296]
Used the gpu

However, it showed me a memory error when I used GPU with CUDA. ( pygpu is 0.6.9 version )

set THEANO_FLAGS=%THEANO_FLAGS_GPU_DNN%

C:\Users\resuly\Desktop>python cpu_gpu_test.py
Using cuDNN version 5110 on context None
WARNING: Preallocating too much memory can prevent cudnn and cublas from working properly
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
Traceback (most recent call last):
  File "D:\Anaconda3\lib\site-packages\theano\gpuarray\__init__.py", line 164, in <module>
    use(config.device)
  File "D:\Anaconda3\lib\site-packages\theano\gpuarray\__init__.py", line 151, in use
    init_dev(device)
  File "D:\Anaconda3\lib\site-packages\theano\gpuarray\__init__.py", line 90, in init_dev
    pygpu.empty((gmem,), dtype='int8', context=context)
  File "pygpu\gpuarray.pyx", line 749, in pygpu.gpuarray.empty
  File "pygpu\gpuarray.pyx", line 676, in pygpu.gpuarray.pygpu_empty
  File "pygpu\gpuarray.pyx", line 290, in pygpu.gpuarray.array_empty
pygpu.gpuarray.GpuArrayException: b'cuMemAlloc: CUDA_ERROR_OUT_OF_MEMORY: out of memory'
[Elemwise{exp,no_inplace}(<TensorType(float32, vector)>)]

I don't think this is a real memory problem because THEANO_FLAGS_GPU works fine with the same script.
My environment variables:

THEANO_FLAGS_CPU
floatX=float32,device=cpu

THEANO_FLAGS_GPU
floatX=float32,device=cuda0,dnn.enabled=False,gpuarray.preallocate=0.8

THEANO_FLAGS_GPU_DNN
floatX=float32,device=cuda0,optimizer_including=cudnn,gpuarray.preallocate=0.8,dnn.conv.algo_bwd_filter=deterministic,dnn.conv.algo_bwd_data=deterministic,dnn.include_path=c:/CUDA/v8.0/include,dnn.library_path=c:/CUDA/v8.0/lib/x64

When I changed the pygpu back to 0.6.2 version. It shows:

C:\Users\resuly\Desktop>python cpu_gpu_test.py
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
Traceback (most recent call last):
  File "D:\Anaconda3\lib\site-packages\theano\gpuarray\__init__.py", line 164, in <module>
    use(config.device)
  File "D:\Anaconda3\lib\site-packages\theano\gpuarray\__init__.py", line 151, in use
    init_dev(device)
  File "D:\Anaconda3\lib\site-packages\theano\gpuarray\__init__.py", line 60, in init_dev
    sched=config.gpuarray.sched)
  File "pygpu\gpuarray.pyx", line 614, in pygpu.gpuarray.init (pygpu/gpuarray.c:9415)
  File "pygpu\gpuarray.pyx", line 566, in pygpu.gpuarray.pygpu_init (pygpu/gpuarray.c:9106)
  File "pygpu\gpuarray.pyx", line 1021, in pygpu.gpuarray.GpuContext.__cinit__ (pygpu/gpuarray.c:13468)
pygpu.gpuarray.GpuArrayException: Error loading library: 0
[Elemwise{exp,no_inplace}(<TensorType(float32, vector)>)]

BTW, tensorflow-gpu works well with CUDA. I don't know how to solve this problem.

Update flags for the next Theano release to use new cuda backend (instead of gpu backend)

As noticed when running Theano 0.9 and described in the documentation the GPU-Backend is deprecated and the Cuda-Backend should be used. When the next Theano version will be release (probably 0.10), the documentation should be updated to use the correct flags, including:

• device=cuda instead of device=gpu
• config.gpuarray.preallocate instead of config.lib.cnmem

I've briefly tested those two flags, but then I ended up getting other errors. The official documentation currently does not seem helpful.

Specify which version of Visual Studio 2015 to download

I saw from your pictures that you're download it to Program Files (x86) path which made me realize you want the 32bit version.

But I feel like it would be good to explicitly write that.

Couple additions

Hi Phil,
Nice tutorial. Here is something to add.

1. I noticed MinGW isn't required at all as long as libpython installed:
   conda install libpython
2. As of now libpython can not be installed with Python 3.5. You would need 3.4 environment or 2.7

How to update Keras 1.1.1

New version of Keras 1.1.1 has arrived at github. How do I update Keras?

How long does it take to start training after the model is compiled?

Thank you a lot for your guide. I followed your step and it works!

I got two questions hopefully you can help me

I use exactly the environment as you use, except mine OS is Win10 Home
Computer: i7 6820HK, GTX 1060

When I compared with my old linux computer (ubuntu 14.04 and cudnn v4 with GTX765m and i7 4702HQ), my new computer with takes much longer time to build cuda code.

Here is the detailed discription.

running output (useless info deleted)

(c:\toolkits\anaconda2-4.2.0\envs\py34) C:\toolkits\keras-1.1.2\examples>python mnist_cnn.py
Using Theano backend.
Using gpu device 0: GeForce GTX 1060 (CNMeM is enabled with initial size: 82.0% of memory, cuDNN 5105)
X_train shape: (60000, 28, 28, 1)
60000 train samples
10000 test samples
---Compiling time is 0.008022069931030273 seconds ---
start training

Train on 60000 samples, validate on 10000 samples
Epoch 1/12
60000/60000 [==============================] - 10s - loss: 0.3910 - acc: 0.8784 - val_loss: 0.0977 - val_acc: 0.9707

After model.compile() is done. My windows PC takes about 470 seconds before it start the next stage , which displays "Train on 60000 samples, validate on 10000 samples" and so on.

On the other hand, my old linux computer only takes about 2s.

I try to find out what is wrong with my settings. After some tests, I find that my theano 0.8.2 outputs lots of debug info, which looks like:

Using Theano backend.

DEBUG: nvcc STDOUT nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
mod.cu
Creating library C:/Users/chaoj/AppData/Local/Theano/compiledir_Windows-10-10.0.14393-Intel64_Family_6_Model_94_Stepping_3_GenuineIntel-2.7.12-64/cuda_ndarray/cuda_ndarray.lib and object C:/Users/chaoj/AppData/Local/Theano/compiledir_Windows-10-10.0.14393-Intel64_Family_6_Model_94_Stepping_3_GenuineIntel-2.7.12-64/cuda_ndarray/cuda_ndarray.exp

DEBUG: nvcc STDOUT nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
mod.cu
Creating library C:/Users/chaoj/AppData/Local/Theano/compiledir_Windows-10-10.0.14393-Intel64_Family_6_Model_94_Stepping_3_GenuineIntel-2.7.12-64/tmpul7fbj/265abc51f7c376c224983485238ff1a5.lib and object C:/Users/chaoj/AppData/Local/Theano/compiledir_Windows-10-10.0.14393-Intel64_Family_6_Model_94_Stepping_3_GenuineIntel-2.7.12-64/tmpul7fbj/265abc51f7c376c224983485238ff1a5.exp

Using gpu device 0: GeForce GTX 1060 (CNMeM is enabled with initial size: 82.0% of memory, cuDNN 5105)

c:\toolkits\anaconda2-4.2.0\lib\site-packages\theano-0.8.2-py2.7.egg\theano\sandbox\cuda_init_.py:600: UserWarning: Your cuDNN version is more recent than the one Theano officially supports. If you see any problems, try updating Theano or downgrading cuDNN to version 5.
warnings.warn(warn)
DEBUG: nvcc STDOUT mod.cu
Creating library C:/Users/chaoj/AppData/Local/Theano/compiledir_Windows-10-10.0.14393-Intel64_Family_6_Model_94_Stepping_3_GenuineIntel-2.7.12-64/tmpchxavp/97496c4d3cf9a06dc4082cc141f918d2.lib and object C:/Users/chaoj/AppData/Local/Theano/compiledir_Windows-10-10.0.14393-Intel64_Family_6_Model_94_Stepping_3_GenuineIntel-2.7.12-64/tmpchxavp/97496c4d3cf9a06dc4082cc141f918d2.exp

DEBUG: nvcc STDOUT mod.cu
Creating library C:/Users/chaoj/AppData/Local/Theano/compiledir_Windows-10-10.0.14393-Intel64_Family_6_Model_94_Stepping_3_GenuineIntel-2.7.12-64/tmpk8ce6m/6174b19f8005a60d6a2faaae7ff1c9a7.lib and object C:/Users/chaoj/AppData/Local/Theano/compiledir_Windows-10-10.0.14393-Intel64_Family_6_Model_94_Stepping_3_GenuineIntel-2.7.12-64/tmpk8ce6m/6174b19f8005a60d6a2faaae7ff1c9a7.exp
...........much more

Therefore I think during the waiting time, nvcc is compiling and optimize for GPU code. I did not see you have such output. I googled it and find that it is a bug for 0.8.2 and when I install 0.9.0dev4, the problem is fixed (Theano/Theano#4579)

Do you take such long time for nvcc to build program? I think it is quite wired.

Thank you in advance!

If Visual Studios was already installed on the machine "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.10240.0\um\x64" will be missing

The solution is to:
"To get this (C:\Program Files (x86)\Windows Kits\10\Lib\10.0.10586.0\um\x64\WindowsApp.lib) file use the Windows Control Panel, go to Programs and Features and double click Microsoft Visual Studio Community 2015. Then select Customize (or "Adjust", I see "Anpassen" as I'm from Germany) and select Tools (1.2) and Windows 10 SDK (10.0.10586) under Development Tools for Universal Windows Apps under Windows and Web Development (again: translations may differ). Then hit UPDATE. If that doesn't work try Emulators for Windows 10 Mobile [phone]) and/or Windows 10 SDK (10.0.10240) too"
microsoft/FFmpegInterop#48

thanks for the great guide!

ERROR (theano.gpuarray): Could not initialize pygpu, support disabled

Hi,

I followed your tutorial and everything went fine. The CPU test went perfectly but when I set THEANO_FLAGS=%THEANO_FLAGS_GPU%, I got the following error:

(dlwin36) C:\Users\Gautam>python cpu_gpu_test.py
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
Traceback (most recent call last):
  File "C:\Users\Gautam\Anaconda2\envs\dlwin36\lib\site-packages\theano\gpuarray\__init__.py", line 164, in <module>
    use(config.device)
  File "C:\Users\Gautam\Anaconda2\envs\dlwin36\lib\site-packages\theano\gpuarray\__init__.py", line 151, in use
    init_dev(device)
  File "C:\Users\Gautam\Anaconda2\envs\dlwin36\lib\site-packages\theano\gpuarray\__init__.py", line 60, in init_dev
    sched=config.gpuarray.sched)
  File "pygpu\gpuarray.pyx", line 614, in pygpu.gpuarray.init (pygpu/gpuarray.c:9415)
  File "pygpu\gpuarray.pyx", line 566, in pygpu.gpuarray.pygpu_init (pygpu/gpuarray.c:9106)
  File "pygpu\gpuarray.pyx", line 1021, in pygpu.gpuarray.GpuContext.__cinit__ (pygpu/gpuarray.c:13468)
GpuArrayException: Error loading library: 0
[Elemwise{exp,no_inplace}(<TensorType(float32, vector)>)]
Looping 1000 times took 20.002000 seconds
Result is [ 1.23178029  1.61879337  1.52278066 ...,  2.20771813  2.29967761
  1.62323284]
Used the cpu

Add apacha as maintainer

Hi @philferriere,
I would like to support you in updating and maintaining this tutorial. Please give me the appropriate rights, to close issues and gain write-access to the repo (I will still issue pull-requests for you to review, if you wish to do so).

Cheers
Alex

ddeghdgh

git branch -m git-branch--m-git-branch--m-main--git-fetch-origin-git-branch--u-origin/---git-fetch-origin-git-branch--u-origin/-
git fetch origin
git branch -u origin/

Validating GPU install with Keras

Hi Phil,

thanks for this amazing tutorial. I followed your steps and the Theano GPU validation works perfectly. However, validating GPU install with Keras is a bit strange.

Running the example script, it first executes DEBUG: .... Creating library ... several times (which takes quite a while) before starting the training. Is this normal or have I made a mistake during installation?

Thanks for your help!
Best,
Johannes

VS2013 link rot

It's no longer possible to link directly to VS2013. Instead, one has to:

1. Go to https://www.visualstudio.com/vs/older-downloads/
2. "Already a member? Sign in"
3. Use http://bugmenot.com/view/microsoftonline.com if you don't have an account
4. Go to https://my.visualstudio.com/downloads
5. Ctrl+F "Visual Studio Express 2013 for Windows with Update 4"

The 2015 Visual Studio link doesn't work

Once I click the link that takes me to older downloads, I click

https://my.visualstudio.com/Downloads?q=visual%20studio%202015

And I then get sent to this page

http://i.imgur.com/sWtgPfJ.png

any ideas?

Build gpu_nms file: cannot find -lcublas

Hi.
I have follow your steps build DL env on windows.
But when i run python setup_gpu.py build script from py-faster-rcnn-windows i got this problems, in my system, there was no D:\toolkits\Anaconda3\envs\dlwin36\PCbuild\amd64 directory, so g++ can not found cublas lib.
How can i solve this problem? i have searched a lot from google, but not found the good way to solve this.

Hope get some help from you.
THANKS.

D:\toolkits\Anaconda3\envs\dlwin36\Library\mingw-w64\bin\g++.exe --shared -s build\temp.win-amd64-3.5\Release\nms\gpu_nms.o build\temp.win-amd64-3.5\Release\nms\gpu_nms.cp35-win_amd64.def -LD:\toolkits\Anaconda3\envs\dlwin36\libs -LD:\toolkits\Anaconda3\envs\dlwin36\PCbuild\amd64 -lcublas -lpython35 -lmsvcr140 -o build\lib.win-amd64-3.5\nms\gpu_nms.cp35-win_amd64.pyd
D:/toolkits/Anaconda3/envs/dlwin36/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lcublas
collect2.exe: error: ld returned 1 exit status
error: command 'D:\\toolkits\\Anaconda3\\envs\\dlwin36\\Library\\mingw-w64\\bin\\g++.exe' failed with exit status 1

Keras 2.0.6 works perfectly fine!

Hey! I just upgraded to keras 2.0.6 and everything seems to be working perfectly fine

conda install theano

There's a theano package for conda since a while. Makes installation much easier because it already comes with all the dependencies - like gcc and libpython.

How I install theano lately:

conda create -p pyenv python=3.5
conda install -p pyenv theano
activate pyenv
pip install keras

done 👍

Simplified Keras + Tensorflow-GPU installation

Yesterday, upon setting up Keras+TF on another Windows 7 machine, I figured, that the TF+Keras installation can be simplified:

• No need of OpenBLAS
• No need of MinGW
• No need of setting up any Flags

but

• cuDNN is mandatory, or otherwise you will get a DLL load failed exception as described in this questions on Stackoverflow - took me quite a while to figure out that cuDNN was the reason.

I wonder, if Visual Studio was required after all, but I didn't try it, since VS was already installed. Next time, I get the chance to set up a fresh system, I will try to only install the following:

• Anaconda 3 - 4.2.0
• Cuda 8.0
• cuDNN 5.1
• pip keras==2.0.4
• pip tensorflow-gpu==1.1.0

If someone get's the chance of trying this out, it would be great. We should also add an appropriate notice to the Tensorflow section, that cuDNN is mandatory.

Problems with theano and mingw

Theano is not working.

===============================
Problem occurred during compilation with the command line below:
C:\toolkits\mingw-w64-6.3.0\mingw64\bin\g++.exe -shared -g -march=skylake -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mmovbe -maes -mno-sha -mpclmul -mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mbmi2 -mno-tbm -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mrtm -mhle -mrdrnd -mf16c -mfsgsbase -mrdseed -mprfchw -madx -mfxsr -mxsave -mxsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mclflushopt -mxsavec -mxsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-mwaitx -mno-clzero -mno-pku --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=6144 -mtune=skylake -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -DMS_WIN64 -IC:\toolkits\lib\site-packages\numpy\core\include -IC:\toolkits\include -IC:\toolkits\lib\site-packages\theano\gof -o C:\Users\Rodrigo\AppData\Local\Theano\compiledir_Windows-10-10.0.10586-Intel64_Family_6_Model_94_Stepping_3_GenuineIntel-2.7.13-64\lazylinker_ext\lazylinker_ext.pyd C:\Users\Rodrigo\AppData\Local\Theano\compiledir_Windows-10-10.0.10586-Intel64_Family_6_Model_94_Stepping_3_GenuineIntel-2.7.13-64\lazylinker_ext\mod.cpp -LC:\toolkits\libs -LC:\toolkits -lpython27
In file included from C:/toolkits/mingw-w64-6.3.0/mingw64/lib/gcc/x86_64-w64-mingw32/6.3.0/include/c++/math.h:36:0,
from C:\toolkits\include/pyport.h:325,
from C:\toolkits\include/Python.h:58,
from C:\Users\Rodrigo\AppData\Local\Theano\compiledir_Windows-10-10.0.10586-Intel64_Family_6_Model_94_Stepping_3_GenuineIntel-2.7.13-64\lazylinker_ext\mod.cpp:1:
C:/toolkits/mingw-w64-6.3.0/mingw64/lib/gcc/x86_64-w64-mingw32/6.3.0/include/c++/cmath:1157:11: error: '::hypot' has not been declared
using ::hypot;
^~~~~

Traceback (most recent call last):
File "", line 1, in
File "C:\toolkits\lib\site-packages\theano_init_.py", line 63, in
from theano.compile import (
File "C:\toolkits\lib\site-packages\theano\compile_init_.py", line 9, in
from theano.compile.function_module import *
File "C:\toolkits\lib\site-packages\theano\compile\function_module.py", line 22, in
import theano.compile.mode
File "C:\toolkits\lib\site-packages\theano\compile\mode.py", line 12, in
import theano.gof.vm
File "C:\toolkits\lib\site-packages\theano\gof\vm.py", line 638, in
from . import lazylinker_c
File "C:\toolkits\lib\site-packages\theano\gof\lazylinker_c.py", line 126, in
preargs=args)
File "C:\toolkits\lib\site-packages\theano\gof\cmodule.py", line 2204, in compile_str
(status, compile_stderr.replace('\n', '. ')))
Exception: Compilation failed (return status=1): In file included from C:/toolkits/mingw-w64-6.3.0/mingw64/lib/gcc/x86_6. from C:\Users\Rodrigo\AppData\Local\Theano\compiledir_Windows-10-10.0.10586-Intel64_Family_6_Model_94. C:/toolkits/mingw-w64-6.3.0/mingw64/lib/gcc/x86_64-w64-mingw32/6.3.0/include/c++/cmath:1157:11: error: '::hypot' has n. ^~~~~;