Comments (15)
Sorry for the bugs. I have uploaded the corresponding __init__.py
. GT of 'trainaug' data can be downloaded at here. label
can be downloaded at here.
from clims.
Thanks for helping me to fix the bugs in the code and pointing out the typo.
from clims.
I have copied the file __init__.py
from here and put it under the folder segmentation/deeplabv2/libs/datasets/
, and the above error is solved. But I met a new error below:
Traceback (most recent call last):
File "main.py", line 26, in <module>
from libs.datasets import get_dataset
File "/home/wensheng/code/for_dev/CLIMS/segmentation/deeplabv2/libs/datasets/__init__.py", line 1, in <module>
from .voc import VOC, VOCAug
File "/home/wensheng/code/for_dev/CLIMS/segmentation/deeplabv2/libs/datasets/voc.py", line 19, in <module>
from .base import _BaseDataset, _BaseDatasetTest, _BaseDatasetCotrain
ImportError: cannot import name '_BaseDatasetCotrain' from 'libs.datasets.base' (/home/wensheng/code/for_dev/CLIMS/segmentation/deeplabv2/libs/datasets/base.py)
Could you check as well?
from clims.
Try to comment _BaseDatasetCotrain
.
from clims.
Thanks! I have solved the _BaseDatasetCotrain
error by removing _BaseDatasetCotrain
from this line.
Then I met other import related errors, and I solved them by copying another two __init__.py
files. Specifically,
- copy the
__init__.py
from here and put it under the foldersegmentation/deeplabv2/libs/models
. This solves the error below:
Traceback (most recent call last):
File "main.py", line 27, in <module>
from libs.models import DeepLabV2_ResNet101_MSC
ImportError: cannot import name 'DeepLabV2_ResNet101_MSC' from 'libs.models' (unknown location)
- copy the
__init__.py
from this place and put it under the foldersegmentation/deeplabv2/libs/utils
. This solves the error below:
Traceback (most recent call last):
File "main.py", line 28, in <module>
from libs.utils import DenseCRF, PolynomialLR, scores
ImportError: cannot import name 'DenseCRF' from 'libs.utils' (unknown location)
from clims.
After these copy things, I think no .py
files are missing.
However, I met another error about the dataset:
FileNotFoundError: [Errno 2] No such file or directory: '../../data/VOC2012/ImageSets/Segmentation/train_aug.txt'
I tried to compare the segmentation/deeplabv2/libs/datasets/voc.py
with this file, and found the latter uses the path ImageSets/SegmentationAug
. So I guess this tutorial for making augmented PASCAL VOC dataset may not be applicable here, and I'm not sure how the augmented PASCAL VOC dataset for CLIMS is formed.
Could you explain how to make the augmented dataset in detail (like this), and how it is organized (including all subdirectories)? This tree structure in the readme does not include subdirectories/specific txt files, and does not specify the way to make the augmented dataset. This would be a key help, thanks!
from clims.
Thanks for the update! Could you specify in detail how the augmented PASCAL VOC dataset is organized? This issue seems unresolved.
After these copy things, I think no
.py
files are missing.However, I met another error about the dataset:
FileNotFoundError: [Errno 2] No such file or directory: '../../data/VOC2012/ImageSets/Segmentation/train_aug.txt'
I tried to compare the
segmentation/deeplabv2/libs/datasets/voc.py
with this file, and found the latter uses the pathImageSets/SegmentationAug
. So I guess this tutorial for making augmented PASCAL VOC dataset may not be applicable here, and I'm not sure how the augmented PASCAL VOC dataset for CLIMS is formed.Could you explain how to make the augmented dataset in detail (like this), and how it is organized (including all subdirectories)? This tree structure in the readme does not include subdirectories/specific txt files, and does not specify the way to make the augmented dataset. This would be a key help, thanks!
from clims.
Hi, just unzip the segmentationaug.rar
and put .txt files into your data root ../../data/VOC2012/ImageSets/Segmentation/
.
from clims.
Thanks for the guidance! I have put these txt files into the data root folder, but then I got the following error.
Traceback (most recent call last):
File "main.py", line 506, in <module>
main()
File "/home/wensheng/anaconda3/envs/clims/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/home/wensheng/anaconda3/envs/clims/lib/python3.8/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/wensheng/anaconda3/envs/clims/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/wensheng/anaconda3/envs/clims/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/wensheng/anaconda3/envs/clims/lib/python3.8/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "main.py", line 226, in train
_, images, labels = next(loader_iter)
File "/home/wensheng/anaconda3/envs/clims/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 517, in __next__
data = self._next_data()
File "/home/wensheng/anaconda3/envs/clims/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 557, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/home/wensheng/anaconda3/envs/clims/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/wensheng/anaconda3/envs/clims/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/wensheng/code/for_dev/CLIMS/segmentation/deeplabv2/libs/datasets/base.py", line 107, in __getitem__
image_id, image, label = self._load_data(index)
File "/home/wensheng/code/for_dev/CLIMS/segmentation/deeplabv2/libs/datasets/voc.py", line 161, in _load_data
image = cv2.imread(image_path, cv2.IMREAD_COLOR).astype(np.float32)
AttributeError: 'NoneType' object has no attribute 'astype'
I found the problem comes from below function (copied from here)
def _load_data(self, index):
# Set paths
image_id = self.files[index]
image_path = osp.join(self.image_dir, image_id + ".jpg")
label_path = osp.join(self.label_dir, image_id + ".png")
# Load an image
image = cv2.imread(image_path, cv2.IMREAD_COLOR).astype(np.float32)
label = np.asarray(Image.open(label_path), dtype=np.int32)
return image_id, image, label
image_dir
has value ../../data/VOC2012/JPEGImages
, and image_id
has value /JPEGImages/2010_003067.jpg /SegmentationClassAug/2010_003067.png
(one line from the train_aug.txt
I think). This makes the image_path
wrong, and hence cv2.imread
cannot work. I guess the _load_data
function is not compatible with the train_aug.txt
file. It seems that the image_id
should be someting like 2007_000032
only. Could you please have a look?
from clims.
I have downloaded the file list from my personal server. Try this one
from clims.
I have downloaded the file list from my personal server. Try this one
Eh, it seems that I do not have the access permission.
from clims.
Try this one
from clims.
Try this one
Thanks for sharing! Now I can run the training process for DeepLab v2 with the file above.
It looks like the first 2 lines shown below in this shell can run normally,
OMP_NUM_THREADS=32 CUDA_VISIBLE_DEVICES=6 python main.py train --config-path configs/voc12_coco_pretrained.yaml
OMP_NUM_THREADS=32 CUDA_VISIBLE_DEVICES=6 python main.py test --config-path configs/voc12_coco_pretrained.yaml --model-path data/models/voc12_coco_pretrained/deeplabv2_resnet101_msc/train_aug/checkpoint_final.pth
But when it goes to the crf command OMP_NUM_THREADS=32 CUDA_VISIBLE_DEVICES=6 python main.py crf --config-path configs/voc12_coco_pretrained.yaml
, I met the following error:
Mode: crf
# jobs: 64
Dataset: VOCAug
# data: 1449
Split: val
Root: ../../data/VOC2012
Logit src: data/features/voc12_coco_pretrained/deeplabv2_resnet101_msc/val/logit
Score dst: data/scores/voc12_coco_pretrained/deeplabv2_resnet101_msc/val/scores_crf.json
[Parallel(n_jobs=64)]: Using backend LokyBackend with 64 concurrent workers.
[Parallel(n_jobs=64)]: Done 145 out of 1449 | elapsed: 22.0s remaining: 3.3min
[Parallel(n_jobs=64)]: Done 290 out of 1449 | elapsed: 39.1s remaining: 2.6min
[Parallel(n_jobs=64)]: Done 435 out of 1449 | elapsed: 57.0s remaining: 2.2min
[Parallel(n_jobs=64)]: Done 580 out of 1449 | elapsed: 1.2min remaining: 1.9min
[Parallel(n_jobs=64)]: Done 725 out of 1449 | elapsed: 1.6min remaining: 1.6min
[Parallel(n_jobs=64)]: Done 870 out of 1449 | elapsed: 1.9min remaining: 1.2min
[Parallel(n_jobs=64)]: Done 1015 out of 1449 | elapsed: 2.2min remaining: 56.1s
[Parallel(n_jobs=64)]: Done 1160 out of 1449 | elapsed: 2.5min remaining: 37.2s
[Parallel(n_jobs=64)]: Done 1305 out of 1449 | elapsed: 2.8min remaining: 18.7s
[Parallel(n_jobs=64)]: Done 1449 out of 1449 | elapsed: 3.1min finished
mIoU: 0.6634496325417438
Device:
0: NVIDIA RTX A5000
Traceback (most recent call last):
File "eval.py", line 341, in <module>
evaluation(config_path=args.config_path, model_path=args.model_path)
File "eval.py", line 52, in evaluation
dataset = get_dataset('voc_test')(
File "/home/wensheng/code/for_dev/CLIMS/segmentation/deeplabv2/libs/datasets/__init__.py", line 6, in get_dataset
return {
KeyError: 'voc_test'
Could you please have a look?
from clims.
Hi, as you copy __init__.py
from deeplab-pytorch repository, voc_test
was not imported. Adding the code below to libs/datasets/__init__.py
can work.
from .voc import VOC, VOCAug, VOC_test
from .cocostuff import CocoStuff10k, CocoStuff164k
def get_dataset(name):
return {
"cocostuff10k": CocoStuff10k,
"cocostuff164k": CocoStuff164k,
"voc": VOC,
"vocaug": VOCAug,
"voc_test": VOC_test, # for test set
}[name]
from clims.
Thanks! I have updated the libs/datasets/__init__.py
file, and the crf command OMP_NUM_THREADS=32 CUDA_VISIBLE_DEVICES=6 python main.py crf --config-path configs/voc12_coco_pretrained.yaml
works.
Then when I run the eval command below:
OMP_NUM_THREADS=32 CUDA_VISIBLE_DEVICES=0 python eval.py --config_path configs/voc12_coco_pretrained.yaml --model_path data/models/voc12_coco_pretrained/deeplabv2_resnet101_msc/train_aug/checkpoint_final.pth
I met the following error:
Device:
0: NVIDIA RTX A5000
Dataset: VOC_test
# data: 1456
Split: test
Root: ../../data/VOC2012
Logit dst: data/features/voc12_coco_pretrained/deeplabv2_resnet101_msc/test/logit
0%| | 0/1456 [00:00<?, ?it/s][ WARN:[email protected]] global loadsave.cpp:244 findDecoder imread_('../../data/VOC
2012/JPEGImages/2008_000006.jpg'): can't open/read file: check file path/integrity
0%| | 0/1456 [00:00<?, ?it/s]
Traceback (most recent call last):
File "eval.py", line 341, in <module>
evaluation(config_path=args.config_path, model_path=args.model_path)
File "eval.py", line 102, in evaluation
for image_ids, images in tqdm(
File "/home/wensheng/anaconda3/envs/clims/lib/python3.8/site-packages/tqdm/std.py", line 1195, in __iter__
for obj in iterable:
File "/home/wensheng/anaconda3/envs/clims/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 517, in __next__
data = self._next_data()
File "/home/wensheng/anaconda3/envs/clims/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 557, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/home/wensheng/anaconda3/envs/clims/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/wensheng/anaconda3/envs/clims/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/wensheng/code/for_dev/CLIMS/segmentation/deeplabv2/libs/datasets/base.py", line 215, in __getitem__
image_id, image = self._load_data(index)
File "/home/wensheng/code/for_dev/CLIMS/segmentation/deeplabv2/libs/datasets/voc.py", line 195, in _load_data
image = cv2.imread(image_path, cv2.IMREAD_COLOR).astype(np.float32)
AttributeError: 'NoneType' object has no attribute 'astype'
Then I realized I need to copy those .jpg
images in test_set
of PASCAL VOC 2012 to the VOC2012/JPEGImages
. After this copy, then the eval
command runs well. Then I zip the results
and submit it to the PASCAL VOC server, and get the 71.69720 mIoU (Object Segmentation (comp5))
BTW, there is a typo in this place, where the command should be changed from cd data/features/voc12_imagenet_pretrained/deeplabv2_resnet101_msc/test/
to cd data/features/voc12_coco_pretrained/deeplabv2_resnet101_msc/test/
.
Thanks for your all help!
from clims.
Related Issues (20)
- About The quality of initial CAMs HOT 5
- The difference between previous version with new version. HOT 3
- about deeplab setting HOT 2
- When will the code of COCO be released? HOT 25
- 请问如何Finetune CLIP模型? HOT 1
- Ran out of input HOT 1
- 是否可提供训练好的权重档作复现? HOT 1
- Error on load_img_name_list function HOT 5
- How to obtain pre-trained baseline CAM HOT 14
- Need Coco baseline scores HOT 3
- Please check the Pascal VOC train_aug. HOT 2
- 读取数据集出现错误 HOT 2
- Creation of sem-seg HOT 2
- Problem Solve
- How to extract background image features HOT 4
- How to train DeepLabV1-R38 ? HOT 1
- irnet on coco HOT 6
- test time HOT 2
- train_aug ground-truth Link is broken
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from clims.