Pip install fails

[parlance]

Pip install fails

[andres-fr]

In my case (Ubuntu 20.04, Python3.7, torch 1.8), pip install also failed but cloning and calling the pip install . inside the repo worked.

To add something to the discussion, you may be interested in checking the setup.py file in this repo, which is basically the installation script.

In it, several compressed libraries are downloaded from here and installed.
A possible fix for your problems could be to replace those with other versions more compatible with your setup.

Then the libraries are compiled using a g++ command that you can also find inside setup.py. Another possible fix could be to adapt that command to your setup.

Sorry for the rather vague comment. Hopefully it helps someone!
Cheers,
Andres

[francois-wellenreiter]

I do have a similar problem but at a different level :

Step 13/15 : RUN git clone --recursive https://github.com/parlance/ctcdecode.git
---> Using cache
---> 0631f9daab06
Step 14/15 : RUN cd ctcdecode; pip install .
---> Running in 64e727750519
Processing /tmp/ctcdecode
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-req-build-fUCpdH/setup.py", line 55, in
os.path.join(this_file, "build.py:ffi")
File "/usr/lib/python2.7/distutils/core.py", line 111, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 272, in init
_Distribution.init(self,attrs)
File "/usr/lib/python2.7/distutils/dist.py", line 287, in init
self.finalize_options()
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 327, in finalize_options
ep.load()(self, ep.name, value)
File "/usr/local/lib/python2.7/dist-packages/cffi/setuptools_ext.py", line 204, in cffi_modules
add_cffi_module(dist, cffi_module)
File "/usr/local/lib/python2.7/dist-packages/cffi/setuptools_ext.py", line 49, in add_cffi_module
execfile(build_file_name, mod_vars)
File "/usr/local/lib/python2.7/dist-packages/cffi/setuptools_ext.py", line 25, in execfile
exec(code, glob, glob)
File "/tmp/pip-req-build-fUCpdH/build.py", line 28, in
'third_party/boost_1_67_0.tar.gz')
File "/tmp/pip-req-build-fUCpdH/build.py", line 19, in download_extract
tar = tarfile.open(dl_path)
File "/usr/lib/python2.7/tarfile.py", line 1678, in open
raise ReadError("file could not be opened successfully")
tarfile.ReadError: file could not be opened successfully

[francois-wellenreiter]

A warning at compilation time shows that the libstdc++ library version and the compilation may not have compatible ABI versions.
That could be the source of the issue.

[sailordiary]

Well as it says torch.utils.ffi has become deprecated in the latest version of PyTorch. So probably try writing your own C++ extension before the author takes this into his own hands...

[CuriousDeepLearner]

has anyone have a look on that? what is your problem? i have a similar issue here.
It suppose to be installed in CPU right?

Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-req-build-vlrfq4zp/setup.py", line 43, in <module>
        import build
      File "/tmp/pip-req-build-vlrfq4zp/build.py", line 27, in <module>
        'third_party/openfst-1.6.7.tar.gz')
      File "/tmp/pip-req-build-vlrfq4zp/build.py", line 16, in download_extract
        wget.download(url, out=dl_path)
      File "/opt/conda/lib/python3.6/site-packages/wget.py", line 526, in download
        (tmpfile, headers) = ulib.urlretrieve(binurl, tmpfile, callback)
      File "/opt/conda/lib/python3.6/urllib/request.py", line 248, in urlretrieve
        with contextlib.closing(urlopen(url, data)) as fp:
      File "/opt/conda/lib/python3.6/urllib/request.py", line 223, in urlopen
        return opener.open(url, data, timeout)
      File "/opt/conda/lib/python3.6/urllib/request.py", line 526, in open
        response = self._open(req, data)
      File "/opt/conda/lib/python3.6/urllib/request.py", line 544, in _open
        '_open', req)
      File "/opt/conda/lib/python3.6/urllib/request.py", line 504, in _call_chain
        result = func(*args)
      File "/opt/conda/lib/python3.6/urllib/request.py", line 1361, in https_open
        context=self._context, check_hostname=self._check_hostname)
      File "/opt/conda/lib/python3.6/urllib/request.py", line 1320, in do_open
        raise URLError(err)
    urllib.error.URLError: <urlopen error [Errno -3] Temporary failure in name resolution>
    No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'

[francois-wellenreiter]

you may have a proxy, it seems that downloading fails

[yuyangw]

Same issue. Does anyone get the idea how to fix it?

[jiefeixia]

Any suggestion on solving this problem? It seems there's some internal error.
I am using win10 with gcc (MinGW.org GCC-8.2.0-3) 8.2.0, It can installed on my Ubuntu server but not on my local laptop. Thanks!

C:\Users\Jeffy\Desktop\ctcdecode (master -> origin)
$ pip install .
Processing c:\users\jeffy\desktop\ctcdecode
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\Jeffy\AppData\Local\Temp\pip-req-build-qwlmcohg\setup.py", line 43, in <module>
        import build
      File "C:\Users\Jeffy\AppData\Local\Temp\pip-req-build-qwlmcohg\build.py", line 29, in <module>
        'third_party/boost_1_67_0.tar.gz')
      File "C:\Users\Jeffy\AppData\Local\Temp\pip-req-build-qwlmcohg\build.py", line 21, in download_extract
        tar.extractall('third_party/')
      File "C:\ProgramData\Anaconda3\envs\pytorch\lib\tarfile.py", line 2002, in extractall
        numeric_owner=numeric_owner)
      File "C:\ProgramData\Anaconda3\envs\pytorch\lib\tarfile.py", line 2044, in extract
        numeric_owner=numeric_owner)
      File "C:\ProgramData\Anaconda3\envs\pytorch\lib\tarfile.py", line 2114, in _extract_member
        self.makefile(tarinfo, targetpath)
      File "C:\ProgramData\Anaconda3\envs\pytorch\lib\tarfile.py", line 2155, in makefile
        with bltn_open(targetpath, "wb") as target:
    FileNotFoundError: [Errno 2] No such file or directory: 'third_party\\boost_1_67_0\\libs\\geometry\\doc\\html\\geometry\\reference\\spatial_indexes\\boost__geometry__index__rtree\\rtree_parameters_type_const____indexable_getter_const____value_equal_const____allocator_type_const___.html'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\Jeffy\AppData\Local\Temp\pip-req-build-qwlmcohg\

[francois-wellenreiter]

at a first glance, the script tries to integrate a non-existing file into a tar file. Can you check if it does exist, and if it is generated or supposed to be in the sources ?

[binhna]

In my case, there might be the problem with g++ and the compiler Pytorch was built. Seems like Pytorch was built with clang++ by default, and ctcdecode extension was supposed to build by g++. So it caused an error. Anyone has any idea how to fix it?
Your compiler (g++) is not compatible with the compiler Pytorch was built with for this platform, which is clang++ on darwin. Please use clang++ to to compile your extension. Alternatively, you may compile PyTorch from source using g++, and then you can also use g++ to compile your extension.

Command "/Users/NAME USER/anaconda3/envs/NAME PRJ/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/w0/778gynln48761v5fh716qs580000gn/T/pip-req-build-_qk4uw2d/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/w0/778gynln48761v5fh716qs580000gn/T/pip-record-wfu_b1ud/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/w0/778gynln48761v5fh716qs580000gn/T/pip-req-build-_qk4uw2d/

[Jogool]

Any suggestion on solving this problem? It seems there's some internal error.
I am using win10 with gcc (MinGW.org GCC-8.2.0-3) 8.2.0, It can installed on my Ubuntu server but not on my local laptop. Thanks!

C:\Users\Jeffy\Desktop\ctcdecode (master -> origin)
$ pip install .
Processing c:\users\jeffy\desktop\ctcdecode
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\Jeffy\AppData\Local\Temp\pip-req-build-qwlmcohg\setup.py", line 43, in <module>
        import build
      File "C:\Users\Jeffy\AppData\Local\Temp\pip-req-build-qwlmcohg\build.py", line 29, in <module>
        'third_party/boost_1_67_0.tar.gz')
      File "C:\Users\Jeffy\AppData\Local\Temp\pip-req-build-qwlmcohg\build.py", line 21, in download_extract
        tar.extractall('third_party/')
      File "C:\ProgramData\Anaconda3\envs\pytorch\lib\tarfile.py", line 2002, in extractall
        numeric_owner=numeric_owner)
      File "C:\ProgramData\Anaconda3\envs\pytorch\lib\tarfile.py", line 2044, in extract
        numeric_owner=numeric_owner)
      File "C:\ProgramData\Anaconda3\envs\pytorch\lib\tarfile.py", line 2114, in _extract_member
        self.makefile(tarinfo, targetpath)
      File "C:\ProgramData\Anaconda3\envs\pytorch\lib\tarfile.py", line 2155, in makefile
        with bltn_open(targetpath, "wb") as target:
    FileNotFoundError: [Errno 2] No such file or directory: 'third_party\\boost_1_67_0\\libs\\geometry\\doc\\html\\geometry\\reference\\spatial_indexes\\boost__geometry__index__rtree\\rtree_parameters_type_const____indexable_getter_const____value_equal_const____allocator_type_const___.html'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\Jeffy\AppData\Local\Temp\pip-req-build-qwlmcohg\

You can actually no longer access boost_1_67_0.tar.gz, it is forbidden. Find an alternative mirror or unpack yourself and remove that line of code.

download_extract('https://sourceforge.net/projects/boost/files/boost/1.67.0/boost_1_67_0.tar.gz/download, 'third_party/boost_1_67_0.tar.gz')``

[ZixuanNi]

In my case, there might be the problem with g++ and the compiler Pytorch was built. Seems like Pytorch was built with clang++ by default, and ctcdecode extension was supposed to build by g++. So it caused an error. Anyone has any idea how to fix it?
Your compiler (g++) is not compatible with the compiler Pytorch was built with for this platform, which is clang++ on darwin. Please use clang++ to to compile your extension. Alternatively, you may compile PyTorch from source using g++, and then you can also use g++ to compile your extension.

Command "/Users/NAME USER/anaconda3/envs/NAME PRJ/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/w0/778gynln48761v5fh716qs580000gn/T/pip-req-build-_qk4uw2d/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/w0/778gynln48761v5fh716qs580000gn/T/pip-record-wfu_b1ud/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/w0/778gynln48761v5fh716qs580000gn/T/pip-req-build-_qk4uw2d/

me too, do you have solved this problem now?

[CracKCatZ]

@ZixuanNi did you solved your problem?

[pranav-chandrode]

I am on windows 10 and I am getting this error

F:\ctcdecode\third_party\openfst-1.6.7\src\include\fst/log.h(48): warning C4005: 'LOG': macro redefinition
D:\Anaconda\envs\env38\lib\site-packages\torch\include\c10/util/logging_is_not_google_glog.h(94): note: see previous definition of 'LOG'
F:\ctcdecode\third_party\openfst-1.6.7\src\include\fst/log.h(49): warning C4005: 'VLOG': macro redefinition
D:\Anaconda\envs\env38\lib\site-packages\torch\include\c10/util/logging_is_not_google_glog.h(97): note: see previous definition of 'VLOG'
F:\ctcdecode\third_party\openfst-1.6.7\src\include\fst/log.h(61): warning C4005: 'CHECK': macro redefinition
D:\Anaconda\envs\env38\lib\site-packages\torch\include\c10/util/logging_is_not_google_glog.h(121): note: see previous definition of 'CHECK'
F:\ctcdecode\third_party\openfst-1.6.7\src\include\fst/log.h(70): warning C4005: 'DCHECK': macro redefinition
D:\Anaconda\envs\env38\lib\site-packages\torch\include\c10/util/logging_is_not_google_glog.h(131): note: see previous definition of 'DCHECK'
F:\ctcdecode\third_party\openfst-1.6.7\src\include\fst/generic-register.h(8): fatal error C1083: Cannot open include file: 'dlfcn.h': No such file or directory
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\bin\HostX86\x64\cl.exe' failed with exit code 2

Can anyone suggest what's happening here.
Thank you!