ukcheolshin / crm_rgbtseg Goto Github PK
View Code? Open in Web Editor NEWOfficial implementation of the paper "Complementary Random Masking for RGB-T Semantic Segmentation."
License: MIT License
crm_rgbtseg's People
Contributors
Stargazers
Watchers
Forkers
jackzhousz aust-hansen cv-seg whuhxb takizzl muzaffersaylan danielmorewang linaom1214 farshidrayhancvcrm_rgbtseg's Issues
How can I convert onnx file from MFNet-SWIN-B checkpoint?
I want to convert the ckpt file to onnx file.
But I can't convert because torch.onnx.export's input.
When I convert the model using input=torch.randn(1,4,480,640), the error message (images = [x["image"].to(self.device) for x in batched_inputs]
IndexError: too many indices for tensor of dimension 3) is happend.
So, if i want to convert onnx file, how can i set the model's input?
MF dataset config files not loading the thermal images
The Dataloader does not load the thermal image anywhere!
CRM_RGBTSeg/dataloaders
/MF_dataset.py
Was a pre-modifed dataset used for this model where the input image was already concatenated with the thermal image?
CUDA version and PyTorch mismatch
Hey there,
I'm currently trying to get the repo going but can't get passed compling the CUDA kernel for MSDeformAttn.
I followed the README and used the provided environment.yaml to set up my environment.
I get following error message when trying to execute:
sh make.sh
(CRM) chris@Workstation:~/repos/CRM_RGBTSeg/models/mask2former/pixel_decoder/ops$ sh make.sh
running build
running build_py
running build_ext
/home/chris/programs/miniconda3/envs/CRM/lib/python3.9/site-packages/torch/utils/cpp_extension.py:381: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
warnings.warn(msg.format('we could not find ninja.'))
Traceback (most recent call last):
File "/home/chris/repos/CRM_RGBTSeg/models/mask2former/pixel_decoder/ops/setup.py", line 69, in <module>
setup(
File "/home/chris/programs/miniconda3/envs/CRM/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/home/chris/programs/miniconda3/envs/CRM/lib/python3.9/distutils/core.py", line 148, in setup
dist.run_commands()
File "/home/chris/programs/miniconda3/envs/CRM/lib/python3.9/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/home/chris/programs/miniconda3/envs/CRM/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/chris/programs/miniconda3/envs/CRM/lib/python3.9/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/home/chris/programs/miniconda3/envs/CRM/lib/python3.9/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/chris/programs/miniconda3/envs/CRM/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/chris/programs/miniconda3/envs/CRM/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/home/chris/programs/miniconda3/envs/CRM/lib/python3.9/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/home/chris/programs/miniconda3/envs/CRM/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 404, in build_extensions
self._check_cuda_version()
File "/home/chris/programs/miniconda3/envs/CRM/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 781, in _check_cuda_version
raise RuntimeError(CUDA_MISMATCH_MESSAGE.format(cuda_str_version, torch.version.cuda))
RuntimeError:
The detected CUDA version (10.1) mismatches the version that was used to compile
PyTorch (11.3). Please make sure to use the same CUDA versions.
My environment looks like this:
(CRM) chris@Workstation:~/repos/CRM_RGBTSeg/models/mask2former/pixel_decoder/ops$ conda list
# packages in environment at /home/chris/programs/miniconda3/envs/CRM:
#
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_kmp_llvm conda-forge
absl-py 2.0.0 pypi_0 pypi
addict 2.4.0 pypi_0 pypi
aiohttp 3.8.6 pypi_0 pypi
aiosignal 1.3.1 pypi_0 pypi
antlr4-python3-runtime 4.9.3 pypi_0 pypi
appdirs 1.4.4 pypi_0 pypi
async-timeout 4.0.3 pypi_0 pypi
attrs 23.1.0 pypi_0 pypi
black 21.4b2 pypi_0 pypi
blas 1.0 mkl conda-forge
bzip2 1.0.8 h7f98852_4 conda-forge
ca-certificates 2022.12.7 ha878542_0 conda-forge
cachetools 5.3.1 pypi_0 pypi
certifi 2023.7.22 pypi_0 pypi
charset-normalizer 3.3.0 pypi_0 pypi
click 8.1.7 pypi_0 pypi
cloudpickle 2.2.1 pypi_0 pypi
contourpy 1.1.1 pypi_0 pypi
cudatoolkit 11.3.1 h9edb442_11 conda-forge
cycler 0.12.1 pypi_0 pypi
detectron2 0.6+cu113 pypi_0 pypi
ffmpeg 4.3 hf484d3e_0 pytorch
filelock 3.12.4 pypi_0 pypi
fonttools 4.43.1 pypi_0 pypi
freetype 2.12.1 hca18f0e_1 conda-forge
frozenlist 1.4.0 pypi_0 pypi
fsspec 2023.9.2 pypi_0 pypi
future 0.18.3 pypi_0 pypi
fvcore 0.1.5.post20221221 pypi_0 pypi
gmp 6.2.1 h58526e2_0 conda-forge
gnutls 3.6.13 h85f3911_1 conda-forge
google-auth 2.23.3 pypi_0 pypi
google-auth-oauthlib 1.0.0 pypi_0 pypi
grpcio 1.59.0 pypi_0 pypi
huggingface-hub 0.18.0 pypi_0 pypi
hydra-core 1.3.2 pypi_0 pypi
icu 72.1 hcb278e6_0 conda-forge
idna 3.4 pypi_0 pypi
imageio 2.27.0 pypi_0 pypi
importlib-metadata 6.8.0 pypi_0 pypi
importlib-resources 6.1.0 pypi_0 pypi
iopath 0.1.9 pypi_0 pypi
joblib 1.3.2 pypi_0 pypi
jpeg 9e h0b41bf4_3 conda-forge
kiwisolver 1.4.5 pypi_0 pypi
lame 3.100 h166bdaf_1003 conda-forge
lcms2 2.15 hfd0df8a_0 conda-forge
ld_impl_linux-64 2.40 h41732ed_0 conda-forge
lerc 4.0.0 h27087fc_0 conda-forge
libdeflate 1.17 h0b41bf4_0 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libgcc-ng 12.2.0 h65d4601_19 conda-forge
libhwloc 2.9.0 hd6dc26d_0 conda-forge
libiconv 1.17 h166bdaf_0 conda-forge
libnsl 2.0.0 h7f98852_0 conda-forge
libpng 1.6.39 h753d276_0 conda-forge
libsqlite 3.40.0 h753d276_0 conda-forge
libstdcxx-ng 12.2.0 h46fd767_19 conda-forge
libtiff 4.5.0 h6adf6a1_2 conda-forge
libuuid 2.32.1 h7f98852_1000 conda-forge
libuv 1.44.2 h166bdaf_0 conda-forge
libwebp-base 1.3.0 h0b41bf4_0 conda-forge
libxcb 1.13 h7f98852_1004 conda-forge
libxml2 2.10.3 hfdac1af_6 conda-forge
libzlib 1.2.13 h166bdaf_4 conda-forge
lightning-utilities 0.9.0 pypi_0 pypi
llvm-openmp 16.0.0 h417c0b6_0 conda-forge
markdown 3.5 pypi_0 pypi
markupsafe 2.1.3 pypi_0 pypi
matplotlib 3.8.0 pypi_0 pypi
mkl 2021.4.0 h8d4b97c_729 conda-forge
mkl-service 2.4.0 py39h7e14d7c_0 conda-forge
mkl_fft 1.3.1 py39h0c7bc48_1 conda-forge
mkl_random 1.2.2 py39hde0f152_0 conda-forge
mmcv 1.7.1 pypi_0 pypi
multidict 6.0.4 pypi_0 pypi
mypy-extensions 1.0.0 pypi_0 pypi
ncurses 6.3 h27087fc_1 conda-forge
nettle 3.6 he412f7d_0 conda-forge
numpy 1.23.5 py39h14f4228_0
numpy-base 1.23.5 py39h31eccc5_0
oauthlib 3.2.2 pypi_0 pypi
omegaconf 2.3.0 pypi_0 pypi
opencv-python 4.8.1.78 pypi_0 pypi
openh264 2.1.1 h780b84a_0 conda-forge
openjpeg 2.5.0 hfec8fc6_2 conda-forge
openssl 3.1.0 h0b41bf4_0 conda-forge
packaging 23.2 pypi_0 pypi
pathspec 0.11.2 pypi_0 pypi
pillow 9.4.0 py39h2320bf1_1 conda-forge
pip 23.0.1 pyhd8ed1ab_0 conda-forge
platformdirs 3.11.0 pypi_0 pypi
portalocker 2.8.2 pypi_0 pypi
protobuf 4.24.4 pypi_0 pypi
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
pyasn1 0.5.0 pypi_0 pypi
pyasn1-modules 0.3.0 pypi_0 pypi
pycocotools 2.0.7 pypi_0 pypi
pydot 1.4.2 pypi_0 pypi
pyparsing 3.1.1 pypi_0 pypi
python 3.9.16 h2782a2a_0_cpython conda-forge
python-dateutil 2.8.2 pypi_0 pypi
python_abi 3.9 3_cp39 conda-forge
pytorch 1.10.1 py3.9_cuda11.3_cudnn8.2.0_0 pytorch
pytorch-lightning 1.9.2 pypi_0 pypi
pytorch-mutex 1.0 cuda pytorch
pyyaml 6.0.1 pypi_0 pypi
readline 8.2 h8228510_1 conda-forge
regex 2023.10.3 pypi_0 pypi
requests 2.31.0 pypi_0 pypi
requests-oauthlib 1.3.1 pypi_0 pypi
rsa 4.9 pypi_0 pypi
scikit-learn 1.2.2 pypi_0 pypi
scipy 1.11.3 pypi_0 pypi
setuptools 59.5.0 pypi_0 pypi
six 1.16.0 pyh6c4a22f_0 conda-forge
tabulate 0.9.0 pypi_0 pypi
tbb 2021.8.0 hf52228f_0 conda-forge
tensorboard 2.14.1 pypi_0 pypi
tensorboard-data-server 0.7.1 pypi_0 pypi
termcolor 2.3.0 pypi_0 pypi
threadpoolctl 3.2.0 pypi_0 pypi
timm 0.6.13 pypi_0 pypi
tk 8.6.12 h27826a3_0 conda-forge
toml 0.10.2 pypi_0 pypi
tomli 2.0.1 pypi_0 pypi
torchmetrics 1.2.0 pypi_0 pypi
torchvision 0.11.2 py39_cu113 pytorch
tqdm 4.66.1 pypi_0 pypi
typing_extensions 4.5.0 pyha770c72_0 conda-forge
tzdata 2023b h71feb2d_0 conda-forge
urllib3 2.0.6 pypi_0 pypi
werkzeug 3.0.0 pypi_0 pypi
wheel 0.40.0 pyhd8ed1ab_0 conda-forge
xorg-libxau 1.0.9 h7f98852_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
yacs 0.1.8 pypi_0 pypi
yapf 0.40.2 pypi_0 pypi
yarl 1.9.2 pypi_0 pypi
zipp 3.17.0 pypi_0 pypi
zlib 1.2.13 h166bdaf_4 conda-forge
zstd 1.5.2 h3eb15da_6 conda-forge
and Nvidia-smi shows up like this:
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.113.01 Driver Version: 535.113.01 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 3090 Off | 00000000:08:00.0 On | N/A |
| 0% 41C P8 44W / 350W | 620MiB / 24576MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 1176 G /usr/lib/xorg/Xorg 53MiB |
| 0 N/A N/A 1750 G /usr/lib/xorg/Xorg 287MiB |
| 0 N/A N/A 1878 G /usr/bin/gnome-shell 52MiB |
| 0 N/A N/A 2562 G /usr/lib/firefox/firefox 144MiB |
| 0 N/A N/A 22848 G ...sion,SpareRendererForSitePerProcess 47MiB |
+---------------------------------------------------------------------------------------+
Any help would be highly appreciated!
when can we use train.py and download the download_backbone_pt_weight
First,please let me show my respect to your.CRM is a great work for RGB-T segmentation. Then,could you tell us when we can use train.py and download the download_backbone_pt_weight。
I am looking for your reply.
How to download and handle pretraining weights?
During the installation process, we did not find download_backbone_pt_weight.sh and convert_pth_to_pkl.sh file. Sincerely hope you provide them.
download_backbone_pt_weight.sh and convert_pth_to_pkl.sh
when I run
cd pretrained
sh download_backbone_pt_weight.sh
sh convert_pth_to_pkl.sh
In your repo, I don't find the file:download_backbone_pt_weight.sh and convert_pth_to_pkl.sh. Could you provide it ?
How many categories does KPdataset_concise have?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.