Comments (13)
I think the InvalidVersionSpecError comes from using an old version of conda itself. I am using the 2020.11 anaconda module on devfair, which is conda 4.9.2, and which I think is the recommended one.
from vissl.
Ok after working with a more recent conda it worked (with python =3.7). Thanks for the support and closing the issue
from vissl.
ok willdo, thanks Priya!
from vissl.
OK let me try with a better conda version then
from vissl.
@pclucas14 , just an update here: @bottler uploaded new v0.1.5 conda packages that should now also work with python=3.8.
You should still have the conda 4.9.2 or latest conda otherwise all good to go! :)
from vissl.
I updated conda version with just conda update conda
, and it worked for me.
from vissl.
I also tried on another server, and it's still failing on the same step
(vissl) lpagec@dp-gpu1[pytorch]$conda install -c vissl -c iopath -c conda-forge -c pytorch -c defaults apex vissl
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: |
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
Examining conflict for olefile ca-certificates typing_extensions mkl-service Examining conflict for olefile typing_extensions mkl-service numpy ninja six Examining conflict for olefile typing_extensions mkl-service numpy ninja six Examining conflict for olefile typing_extensions mkl-service numpy ninja six Examining conflict for olefile typing_extensions mkl-service numpy ninja six Examining conflict for olefile typing_extensions mkl-service numpy ninja six Examining conflict for olefile typing_extensions mkl-service numpy ninja six Examining conflict for olefile typing_extensions mkl-service numpy ninja six Examining conflict for olefile typing_extensions mkl-service numpy ninja six Examining conflict for olefile typing_extensions mkl-service numpy freetype nExamining conflict forExamining conflict for olefile typing_extensions mkl-service numpy freetype ninja six libstdcxx-ng nump-
from vissl.
Same step and same issue on my Ubuntu 18.04 server.
from vissl.
thank you for reporting this @pclucas14 and @962086838. I am able to reproduce this too myself now which is strange as we haven't updated VISSL packages recently the uploaded packages worked seamlessly few days ago. If you are immediately blocked, you can install vissl in conda from source https://github.com/facebookresearch/vissl/blob/master/INSTALL.md#Install-from-source-in-Conda-environment which works semalessly
Assigning to @bottler to take a look at our conda packages. It seems like apex installs fine and something is not right with the vissl package only.
Looking a bit to identify what package is causing this issue:
(vissl-test2) prigoyal@devfair0209:~/.conda/envs/vissl-test2/lib/python3.8/site-packages$ grep -r "=2.7"
wheel-0.36.2-py3.6.egg-info/PKG-INFO:Requires-Python: !=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7
setuptools/ssl_support.py: # http://bugs.python.org/issue7776: Python>=3.4.1 and >=2.7.7
olefile-0.46.dist-info/METADATA:Requires-Python: >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*
six-1.15.0.dist-info/METADATA:Requires-Python: >=2.7, !=3.0.*, !=3.1.*, !=3.2.*
mkl_service-2.3.0-py3.8.egg-info/PKG-INFO:Requires-Python: >=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*
mkl_random-1.2.0-py3.8.egg-info/PKG-INFO:Requires-Python: >=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*
numpy-1.19.2.dist-info/RECORD:numpy/random/_sfc64.cpython-38-x86_64-linux-gnu.so,sha256=2L7eIJmqQaR5oMpQc0Y6fJxU9UNMznLXIIVWuWey8N8,75600
caffe2/python/test/executor_test_util.py: image_size=227,
Binary file caffe2/python/caffe2_pybind11_state_gpu.cpython-38-x86_64-linux-gnu.so matches
Binary file torch/bin/protoc matches
Binary file torch/bin/protoc-3.11.4.0 matches
Binary file torch/lib/libtorch_cpu.so matches
Binary file torch/lib/libtorch_cuda.so matches
Binary file torchvision/_C.so matches
torchvision-0.8.2-py3.8.egg-info/PKG-INFO: | ``1.4.0`` | ``0.5.0`` | ``==2.7``, ``>=3.5``, ``<=3.8`` |
torchvision-0.8.2-py3.8.egg-info/PKG-INFO: | ``1.3.1`` | ``0.4.2`` | ``==2.7``, ``>=3.5``, ``<=3.7`` |
torchvision-0.8.2-py3.8.egg-info/PKG-INFO: | ``1.3.0`` | ``0.4.1`` | ``==2.7``, ``>=3.5``, ``<=3.7`` |
torchvision-0.8.2-py3.8.egg-info/PKG-INFO: | ``1.2.0`` | ``0.4.0`` | ``==2.7``, ``>=3.5``, ``<=3.7`` |
torchvision-0.8.2-py3.8.egg-info/PKG-INFO: | ``1.1.0`` | ``0.3.0`` | ``==2.7``, ``>=3.5``, ``<=3.7`` |
torchvision-0.8.2-py3.8.egg-info/PKG-INFO: | ``<=1.0.1`` | ``0.2.2`` | ``==2.7``, ``>=3.5``, ``<=3.7`` |
seems like the error is coming from torchvision which is strange as well.
from vissl.
This is an opencv weirdness.
First observation: The command works fine if I replace python=3.8
with python=3.7
. The version of python shouldn't matter much. This is helpful, because we have a baseline. (If you don't actually need Python3.8 and are just trying to use vissl, the rest of this comment can be ignored!)
There are several packages which exist in more than one place on anaconda cloud. I have separated out all the dependencies. The following goes straight through, with the last line only pulling in apex and vissl. When I say "straight through", I mean the conda solver doesn't seem to need to work hard.
conda create -p /raid/reizenstein/jeremyconda/v4_p37 python=3.7 scikit-learn scipy parameterized
conda activate /raid/reizenstein/jeremyconda/v4_p37
conda install -c pytorch pytorch=1.7.1 torchvision cudatoolkit=10.2
conda install -c conda-forge opencv hydra-core pycocotools tensorboard faiss
conda install -c iopath -c fvcore -c conda-forge fvcore
conda install -v -c vissl -c fvcore -c iopath -c conda-forge -c pytorch -c defaults vissl=0.1.5
There are some background things to be aware of, but none are the primary problem here:
- The recommended way to install faiss is with
faiss-gpu
from thepytorch
channel, but we depend onfaiss
which is in conda-forge. They have different dependencies, but the conda-forge one depends on pytorch and we need to avoid pulling in apytorch
from a default channel, which would be an unofficial build. Hence installing pytorch before any conda-forge packages. Ideally we would switch to depend on faiss-gpu. After that change, I would hope that the first twoconda install
lines could be swapped. - There is a tensorboard package on both the
anaconda
andconda-forge
channels and they have different dependencies. - I have an initial build of vissl (0.1.0) sitting on anaconda cloud. This is from before we decided to put the classy vision code inside the vissl package instead of its own package. I specify the version of vissl in the above to avoid touching the old one. When the next release happens we should delete all the previous ones.
- The three packages
scikit-learn
,scipy
, andparameterized
exist both inanaconda
andconda-forge
and it doesn't seem to matter which ones you use.
When I try the same recipe with python 3.8, it chokes at the conda-forge stage. Just conda install -c conda-forge opencv
alone is problematic for conda. The opencv package for Python 3.8 on conda-forge seems to be quite unfriendly to install. If you start an environment with conda create -p /raid/reizenstein/jeremyconda/foo python=3.8
and immediately try conda install -c conda-forge opencv
, it works but insists on a downgrade from Python 3.8.8 to Python 3.8.3. If I try conda install -c defaults -c conda-forge opencv
it works more smoothly. There may be a way forward here to make things work.
Basically opencv is a bit "special". It would be better to not explicitly depend on opencv, and tell the user to install it via pip. Similar to how our pip packages work. Or even, could we use PIL for the functionality we currently use in opencv, and therefore not need it at all.
from vissl.
thank you @bottler ! It seems like our immediate fix is to just recommend python=3.7
in the conda installation to the user. This is probably something I should update the website and our install instructions immediately.
For our next release, I agree with all the suggestions. I think opencv is a dependency but we can make it optional as we only have 0.1% of the code that requires opencv and otherwise we use PIL.
For the faiss, we should probably just switch to faiss-gpu but again , this is also a dependency that can become optional. I'll add a follow-up for both of these tasks!
from vissl.
Hi,
I tried the following commands with python 3.7
conda create -n vissl2 python=3.7
conda activate vissl2
conda install -c pytorch pytorch=1.7.1 torchvision cudatoolkit=10.2
conda install -c vissl -c iopath -c conda-forge -c pytorch -c defaults apex vissl
And still I'm getting
(vissl2) lucaspc@devfair0224:~/repos/swav$ conda install -c vissl -c iopath -c conda-forge -c pytorch -c defaults apex vissl
Solving environment: failed
InvalidVersionSpecError: Invalid version spec: =2.7
from vissl.
thank you @pclucas14 , we are working on building new packages with the dependencies reduced! In the meantime, if you are blocked, I recommend 2 options:
- If conda is must, build vissl from scratch following the instructions https://github.com/facebookresearch/vissl/blob/master/INSTALL.md#Install-from-source-in-Conda-environment
- Use PIP packages of VISSL instead
from vissl.
Related Issues (20)
- Example of using ZeRO and LARC
- Incosistent results when running inference on trained resnet model HOT 1
- Low performance of Supervised training
- MLP_DIM in ViT is not adjustable through the config. HOT 2
- [Feature] Online linear probing
- Linear Image classification weights not loading properly: vissl pretrained models simclr or Dcv2 HOT 3
- "on_forward"-hook operations and back-propagation
- EMA does not work on fp16 and does not copy weights?
- First-class support for timm models
- Can't load pretrained seer/swav checkpoints (RG_Y_128GF) HOT 1
- Is it possible to get full imagenet pretrained weights of MoCo v2 ? HOT 2
- NPID + MoCoV2 weights are the same? HOT 1
- Potential confusion over licensing comment
- Does the MoCo implementation do shuffled Batch Norm when run on a single GPU? HOT 3
- Loading TRAINED models with VISSL HOT 9
- Use pretrain outside of vissl
- can someone share/has anyone shared a checkpoint ( with the class head) of a ssl model on imagenet?
- What config relate to learning rate warm up, weight decay, and momentum in 1 node n GPUs (n > 1 && n < 8) config?
- AttributeError: module 'importlib_resources' has no attribute 'is_resource'
- all vissl.ai/tutorials links broken
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vissl.