Kernel not compiled with GPU support

[Angzz]

Kernel not compiled with GPU support

[dale1202]

i run python3 -c 'import torch; from torch.utils.cpp_extension import CUDA_HOME; print(torch.cuda.is_available(), CUDA_HOME)', it return True。
and i run detectron2, it still return "RuntimeError: Not compiled with GPU support", Have you solved it? @ppwwyyxx

[dale1202]

i have solved it, reinstall detectron2 : remove bulid file , execute "python setup.py build develop" again，make sure your cuda version is match with pytorch version @ppwwyyxx

[MarkAlive]

remember to remove 'build' dir before reinstall detectron2.

[viven12138]

i have solved it, reinstall detectron2 : remove bulid file , execute "python setup.py build develop" again，make sure your cuda version is match with pytorch version @ppwwyyxx

@dale1202 I did it as what u said and ran into the same problem"RuntimeError: Not compiled with GPU support ". :(

[viven12138]

If you can't use nvcc, then you can't install detectron2.
nvcc never requires root permission.

I can't use nvcc, it returns "apt install nvidia-cuda-toolkit", which needs root permission.
I actually see detectron2 in my conda list.

I ask root to install cuda. And I met the issue "vcc fatal : Unsupported gpu architecture 'compute_75'", then i add "export TORCH_CUDA_ARCH_LIST="7.0"" to the environment to solve it!
reference

[marsggbo]

check two things:

1. whether pytorch-gpu version matches your nvcc version
2. whether your set CUDA_HOME correctly, e.g. export CUDA_HOME=/usr/local/cuda in ~/.bashrc

reference: https://zhuanlan.zhihu.com/p/93278639

[ppwwyyxx]

It will compile with CPU only, only if pytorch cannot find gpus/cuda at the time you compile it.
It can be checked by

python -c 'import torch; from torch.utils.cpp_extension import CUDA_HOME; print(torch.cuda.is_available(), CUDA_HOME)'

[Angzz]

it' s very strange that:

print(torch.version.cuda)
print(torch.__version__)

and the outputs:

'9.2.148'
'1.3.0+cu92'

my cuda version：

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Tue_Jun_12_23:07:04_CDT_2018
Cuda compilation tools, release 9.2, V9.2.148

but when run commond above:

python -c 'import torch; from torch.utils.cpp_extension import CUDA_HOME; print(torch.cuda.is_available(), CUDA_HOME)'

it still return false.

[ppwwyyxx]

Since it is an pytorch/cuda installation issue, please ask in pytorch forum or issues instead. Thanks!

[zhudd-hub]

I have the same issue when I run demo in jupyter notebook:
RuntimeError: Not compiled with GPU support (ROIAlign_forward at /home/workspace/zhudd/detectron2/detectron2/layers/csrc/ROIAlign/ROIAlign.h:71) frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x47 (0x7ff68cc04687 in /home/workspace/zhudd/anaconda3/envs/detectron/lib/python3.7/site-packages/torch/lib/libc10.so) frame #1: ROIAlign_forward(at::Tensor const&, at::Tensor const&, float, int, int, int, bool) + 0x149 (0x7ff67aaf62b9 in /home/workspace/zhudd/detectron2/detectron2/_C.cpython-37m-x86_64-linux-gnu.so) frame #2: <unknown function> + 0x2097f (0x7ff67ab0597f in /home/workspace/zhudd/detectron2/detectron2/_C.cpython-37m-x86_64-linux-gnu.so) frame #3: <unknown function> + 0x20a6e (0x7ff67ab05a6e in /home/workspace/zhudd/detectron2/detectron2/_C.cpython-37m-x86_64-linux-gnu.so) frame #4: <unknown function> + 0x1b358 (0x7ff67ab00358 in /home/workspace/zhudd/detectron2/detectron2/_C.cpython-37m-x86_64-linux-gnu.so) frame #5: _PyMethodDef_RawFastCallKeywords + 0x264 (0x562d6d8cd774 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #6: _PyCFunction_FastCallKeywords + 0x21 (0x562d6d8cd891 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #7: _PyEval_EvalFrameDefault + 0x4ede (0x562d6d93afce in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #8: _PyFunction_FastCallDict + 0x10b (0x562d6d87c92b in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #9: THPFunction_apply(_object*, _object*) + 0x8d6 (0x7ff6bf3ade96 in /home/workspace/zhudd/anaconda3/envs/detectron/lib/python3.7/site-packages/torch/lib/libtorch_python.so) frame #10: _PyMethodDef_RawFastCallKeywords + 0x1e0 (0x562d6d8cd6f0 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #11: _PyCFunction_FastCallKeywords + 0x21 (0x562d6d8cd891 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #12: _PyEval_EvalFrameDefault + 0x47d4 (0x562d6d93a8c4 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #13: _PyFunction_FastCallDict + 0x10b (0x562d6d87c92b in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #14: _PyObject_Call_Prepend + 0x63 (0x562d6d89be23 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #15: PyObject_Call + 0x6e (0x562d6d88e51e in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #16: _PyEval_EvalFrameDefault + 0x1f4c (0x562d6d93803c in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #17: _PyEval_EvalCodeWithName + 0x2f9 (0x562d6d87b929 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #18: _PyFunction_FastCallDict + 0x1d5 (0x562d6d87c9f5 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #19: _PyObject_Call_Prepend + 0x63 (0x562d6d89be23 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #20: <unknown function> + 0x1702ea (0x562d6d8d52ea in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #21: _PyObject_FastCallKeywords + 0x3fb (0x562d6d8d616b in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #22: _PyEval_EvalFrameDefault + 0x4ac6 (0x562d6d93abb6 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #23: _PyFunction_FastCallDict + 0x10b (0x562d6d87c92b in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #24: _PyObject_Call_Prepend + 0x63 (0x562d6d89be23 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #25: PyObject_Call + 0x6e (0x562d6d88e51e in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #26: _PyEval_EvalFrameDefault + 0x1f4c (0x562d6d93803c in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #27: _PyEval_EvalCodeWithName + 0x2f9 (0x562d6d87b929 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #28: _PyFunction_FastCallDict + 0x1d5 (0x562d6d87c9f5 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #29: _PyObject_Call_Prepend + 0x63 (0x562d6d89be23 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #30: <unknown function> + 0x1702ea (0x562d6d8d52ea in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #31: _PyObject_FastCallKeywords + 0x3fb (0x562d6d8d616b in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #32: _PyEval_EvalFrameDefault + 0x53ae (0x562d6d93b49e in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #33: _PyFunction_FastCallKeywords + 0xfb (0x562d6d8cccfb in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #34: _PyEval_EvalFrameDefault + 0x4b69 (0x562d6d93ac59 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #35: _PyEval_EvalCodeWithName + 0xba9 (0x562d6d87c1d9 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #36: _PyFunction_FastCallDict + 0x1d5 (0x562d6d87c9f5 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #37: _PyObject_Call_Prepend + 0x63 (0x562d6d89be23 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #38: PyObject_Call + 0x6e (0x562d6d88e51e in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #39: _PyEval_EvalFrameDefault + 0x1f4c (0x562d6d93803c in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #40: _PyEval_EvalCodeWithName + 0x2f9 (0x562d6d87b929 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #41: _PyFunction_FastCallDict + 0x1d5 (0x562d6d87c9f5 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #42: _PyObject_Call_Prepend + 0x63 (0x562d6d89be23 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #43: <unknown function> + 0x1702ea (0x562d6d8d52ea in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #44: _PyObject_FastCallKeywords + 0x3fb (0x562d6d8d616b in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #45: _PyEval_EvalFrameDefault + 0x53ae (0x562d6d93b49e in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #46: _PyEval_EvalCodeWithName + 0xba9 (0x562d6d87c1d9 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #47: _PyFunction_FastCallDict + 0x1d5 (0x562d6d87c9f5 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #48: _PyObject_Call_Prepend + 0x63 (0x562d6d89be23 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #49: PyObject_Call + 0x6e (0x562d6d88e51e in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #50: _PyEval_EvalFrameDefault + 0x1f4c (0x562d6d93803c in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #51: _PyEval_EvalCodeWithName + 0x2f9 (0x562d6d87b929 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #52: _PyFunction_FastCallDict + 0x1d5 (0x562d6d87c9f5 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #53: _PyObject_Call_Prepend + 0x63 (0x562d6d89be23 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #54: <unknown function> + 0x1702ea (0x562d6d8d52ea in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #55: _PyObject_FastCallKeywords + 0x3fb (0x562d6d8d616b in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #56: _PyEval_EvalFrameDefault + 0x53ae (0x562d6d93b49e in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #57: _PyFunction_FastCallDict + 0x10b (0x562d6d87c92b in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #58: _PyEval_EvalFrameDefault + 0x1f4c (0x562d6d93803c in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #59: _PyEval_EvalCodeWithName + 0xba9 (0x562d6d87c1d9 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #60: _PyFunction_FastCallDict + 0x1d5 (0x562d6d87c9f5 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #61: _PyObject_Call_Prepend + 0x63 (0x562d6d89be23 in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #62: <unknown function> + 0x1702ea (0x562d6d8d52ea in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3) frame #63: _PyObject_FastCallKeywords + 0x3fb (0x562d6d8d616b in /home/workspace/zhudd/anaconda3/envs/detectron/bin/python3)
### whatsmore,when I run:
python -c 'import torch; from torch.utils.cpp_extension import CUDA_HOME; print(torch.cuda.is_available(), CUDA_HOME)'
### it return :
True /usr/local/cuda
the version of cuda
`+-----------------------------------------------------------------------------+
| NVIDIA-SMI 410.104 Driver Version: 410.104 CUDA Version: 10.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|=====================+=============+==============|
| 0 Tesla P100-PCIE... Off | 00000000:3B:00.0 Off | 0 |
| N/A 40C P0 29W / 250W | 4836MiB / 12198MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 1 Tesla P100-PCIE... Off | 00000000:AF:00.0 Off | 0 |
| N/A 37C P0 27W / 250W | 10MiB / 16280MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 2 Tesla P100-PCIE... Off | 00000000:D8:00.0 Off | 0 |
| N/A 39C P0 28W / 250W | 10MiB / 16280MiB | 0% Default |
+-------------------------------+----------------------+----------------------+

my pytorch is 1.3
what should I do to solve the problem

[ppwwyyxx]

You should uninstall and reinstall detectron2, when python -c 'import torch; from torch.utils.cpp_extension import CUDA_HOME; print(torch.cuda.is_available(), CUDA_HOME)' is working

[SkeletonOne]

@zhudd-hub I meet a similar problem as same as yours. Have you solved it?

[viven12138]

i run python3 -c 'import torch; from torch.utils.cpp_extension import CUDA_HOME; print(torch.cuda.is_available(), CUDA_HOME)', it return True。
and i run detectron2, it still return "RuntimeError: Not compiled with GPU support", Have you solved it? @ppwwyyxx

The same question! Have u solved it?

[dale1202]

use cuda9.2，use "pip3 install torch==1.3.0+cu92 torchvision==0.4.1+cu92 -f https://download.pytorch.org/whl/torch_stable.html" to install pytorch。execute "nvcc --version" check cuda version

[viven12138]

use cuda9.2，use "pip3 install torch==1.3.0+cu92 torchvision==0.4.1+cu92 -f https://download.pytorch.org/whl/torch_stable.html" to install pytorch。execute "nvcc --version" check cuda version
@dale1202 I install pytorch, cuda and cudnn in a conda environment. I can't use nvcc, cause I don't have root permission. What's more, the command python -c 'import torch; from torch.utils.cpp_extension import CUDA_HOME; print(torch.cuda.is_available(), CUDA_HOME)' returns "True None". Is it related to the conda environment?
My environment info:

------------------------  --------------------------------------------------
sys.platform              linux
Python                    3.7.3 (default, Mar 27 2019, 22:11:17) [GCC 7.3.0]
Numpy                     1.17.2
Detectron2 Compiler       GCC 7.4
Detectron2 CUDA Compiler  not available
DETECTRON2_ENV_MODULE     <not set>
PyTorch                   1.3.0+cu92
PyTorch Debug Build       False
torchvision               0.4.1+cu92
CUDA available            True
GPU 0,1,2,3               GeForce RTX 2080 Ti
CUDA_HOME                 None
Pillow                    6.2.0
cv2                       4.1.1
------------------------  --------------------------------------------------
PyTorch built with:
  - GCC 7.3
  - Intel(R) Math Kernel Library Version 2019.0.4 Product Build 20190411 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v0.20.5 (Git Hash 0125f28c61c1f822fd48570b4c1066f96fcb9b2e)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - NNPACK is enabled
  - CUDA Runtime 9.2
  - NVCC architecture flags: -gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_50,code=compute_50
  - CuDNN 7.6.3
  - Magma 2.5.1
  - Build settings: BLAS=MKL, BUILD_NAMEDTENSOR=OFF, BUILD_TYPE=Release, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -fopenmp -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Wno-stringop-overflow, DISABLE_NUMA=1, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, USE_CUDA=True, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_STATIC_DISPATCH=OFF,

[ppwwyyxx]

If you can't use nvcc, then you can't install detectron2.
nvcc never requires root permission.

[viven12138]

If you can't use nvcc, then you can't install detectron2.
nvcc never requires root permission.

I can't use nvcc, it returns "apt install nvidia-cuda-toolkit", which needs root permission.
I actually see detectron2 in my conda list.

[fatalfeel]

#if meet RuntimeError('Not compiled with GPU support#
(i have same problem and do it ok now)
#make sure /usr/local/cuda link to /usr/local/cuda-10.1#
ln -sf /usr/local/cuda-10.1 /usr/local/cuda
rm -R /root/projects/detectron2/build
python3 setup.py build develop

[KunalMGupta]

In case someone is still wondering how to solve it. This is what I tried:

1. Check the cuda version using: python -c 'import torch; print(torch.version.cuda)'
2. remove detectron2
3. reinstall with the correct cuda: python -m pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cuABC/index.html

where ABC is the cuda version found in step 1) i.e. ABC = 101 (cuda 10.1) and 100 (for cuda 10.0)

Following this I was able to solve this error. Hope it helps!