dddraxxx/weakly-supervised-camouflaged-object-detection-with-scribble-annotations

Code for the AAAI 2023 paper "Weakly-Supervised Camouflaged Object Detection with Scribble Annotations"

Weakly-Supervised Camouflaged Object Detection with Scribble Annotations (AAAI23, ORAL)

Authors: Ruozhen He*, Qihua Dong*, Jiaying Lin, and Rynson Lau (* joint first authors)

Paper Link: arxiv

Dataset

• We relabeled 4,040 images (3,040 from COD10K, 1,000 from CAMO) with scribbles and proposed the S-COD dataset (Download) for training. In our annotations, "1" stands for foregrounds, "2" for backgrounds, and "0" for unlabeled regions. (The image is viewed as black because its range is 0-255)
• Download the training dataset (COD10K-train) at here.
• Download the testing dataset (COD10K-test + CAMO-test + CHAMELEON) at here.

Experimental Results

• We provide CRNet testing maps and training weights presented in the papers.

Code

Requirements

git clone --recurse-submodules https://github.com/dddraxxx/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations.git
pip install -r requirements.txt

Pretrained weights

The pretrained weight can be found here: ResNet-50.

Train

• Download the dataset and pretrained model. (examples of train.txt and test.txt are in the path ./CodDataset)
• Modify the path in train.py.
• Run python train.py.

Test and Evaluate

• The evaluation is done using the submodule PySODEvalToolKit. Add the json files according to its instruction. (examples of json files are in the path ./CodDataset)
• Modify the path and filename.
• Run python test.py.

Credit

The code is partly based on SCWSSOD, GCPANet and GatedCRFLoss.

weakly-supervised-camouflaged-object-detection-with-scribble-annotations's Issues

Question about annotation

hello. while I was reading the paper , I got your proposed architecture gets scribble annotation instead of pixel wise annotation. so why when I test code on my custom datasets , it gets GT annotation error. in code I comment GT lines for metrics , but again I get error of GT. can you help me to solve this problem that can test images without ground truth of pixel wise?

question about configuration

hello
first of all, I admire your perfect idea. I have tried to get run this code, but I got troubled by training.
when tried to run training dataset , the code worked. but when epochs arrived to evaluation part it stopped and did not work.
the problem is to build configuration of test data. i have problem how to make directories of dataset and paths of them. please help me soon .

this is one of the error :
!python train.py

Parameters...
datapath : /content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset
savepath : ./out/trained/
mode : train
batch : 16
lr : 0.001
verbose : 1
momen : 0.9
decay : 0.0005
epoch : 1
label_dir : Scribble
/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py:563: UserWarning: This DataLoader will create 8 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
warnings.warn(_create_warning_msg(

Parameters...
datapath : /content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test
mode : test
label_dir : Scribble
/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py:563: UserWarning: This DataLoader will create 4 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
warnings.warn(create_warning_msg(
initialize net
********/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/Image/CAMO/Image/camourflage_00012.jpg
******/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/GT/CAMO/Image/camourflage_00012.png
********/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/Image/CAMO/Image/camourflage_00018.jpg
******/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/GT/CAMO/Image/camourflage_00018.png
[ WARN:[email protected]] global /io/opencv/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread('/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/Image/CAMO/Image/camourflage_00012.jpg'): can't open/read file: check file path/integrity
[ WARN:[email protected]] global /io/opencv/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/Image/CAMO/Image/camourflage_00018.jpg'): can't open/read file: check file path/integrity
********/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/Image/CAMO/Image/camourflage_00061.jpg
******/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/GT/CAMO/Image/camourflage_00061.png
********/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/Image/CAMO/Image/camourflage_00064.jpg
******/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/GT/CAMO/Image/camourflage_00064.png
[ WARN:[email protected]] global /io/opencv/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/Image/CAMO/Image/camourflage_00061.jpg'): can't open/read file: check file path/integrity
[ WARN:[email protected]] global /io/opencv/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/Image/CAMO/Image/camourflage_00064.jpg'): can't open/read file: check file path/integrity
********/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/Image/CAMO/Image/camourflage_00079.jpg
******/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/GT/CAMO/Image/camourflage_00079.png
[ WARN:[email protected]] global /io/opencv/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/Image/CAMO/Image/camourflage_00079.jpg'): can't open/read file: check file path/integrity
********/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/Image/CAMO/Image/camourflage_00071.jpg
******/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/GT/CAMO/Image/camourflage_00071.png
[ WARN:[email protected]] global /io/opencv/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/Image/CAMO/Image/camourflage_00071.jpg'): can't open/read file: check file path/integrity
Traceback (most recent call last):
File "/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/train.py", line 180, in
********/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/Image/CAMO/Image/camourflage_00090.jpg
******/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/GT/CAMO/Image/camourflage_00090.png
[ WARN:[email protected]] global /io/opencv/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/Image/CAMO/Image/camourflage_00090.jpg'): can't open/read file: check file path/integrity
train(dataset, Net, cfg, tm, start_from=0)
File "/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/train.py", line 123, in train
********/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/Image/CAMO/Image/camourflage_00087.jpg
******/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/GT/CAMO/Image/camourflage_00087.png
[ WARN:[email protected]] global /io/opencv/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/Image/CAMO/Image/camourflage_00087.jpg'): can't open/read file: check file path/integrity
mae = validate_multiloader(net, val_loaders)
File "/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/train.py", line 77, in validate_multiloader
mae = validate(model, v)
File "/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/train.py", line 61, in validate
for image, mask, shape, name in val_loader:
File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py", line 681, in next
data = self._next_data()
File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py", line 1376, in _next_data
return self._process_data(data)
File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py", line 1402, in _process_data
data.reraise()
File "/usr/local/lib/python3.10/dist-packages/torch/_utils.py", line 461, in reraise
raise exception
AttributeError: Caught AttributeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
data = fetcher.fetch(index)
File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/data/dataset.py", line 108, in getitem
image = cv2.imread(imagepath).astype(np.float32)[:,:,::-1]
AttributeError: 'NoneType' object has no attribute 'astype'

********/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/Image/CAMO/Image/camourflage_00088.jpg
******/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/GT/CAMO/Image/camourflage_00088.png
[ WARN:[email protected]] global /io/opencv/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('/content/drive/MyDrive/camouflaged/scribble/Weakly-Supervised-Camouflaged-Object-Detection-with-Scribble-Annotations/CodDataset/test/Image/CAMO/Image/camourflage_00088.jpg'): can't open/read file: check file path/integrity

关于文章标题的疑惑

您好，在拜读您的文章时，我发现这篇文章最后输出的是分割的结果，但文章标题中所提及的任务是object detection。所以想向您请教一下，COD方法是否是做的分割任务，为什么要定义为目标检测呢？
期待您的回复，不甚感激。

Here are some questions, why the weakly supervised dataset Scribble you provided is all black?

Thanks for the great work! Here are some questions, why the weakly supervised dataset Scribble you provided is all black?

Scribble Dataset

I have some questions to ask you about the scribble dataset. First, can I know which software did you use to generate the scribble dataset. Second, I assume that the scribble picture at first is in RGB image, how do you convert it to 0-255 range with png extension which you uploaded your scribble dataset here.

train.txt

FileNotFoundError: [Errno 2] No such file or directory: '../CodDataset/train.txt'

关于测试的问题

没有见到测试过程代码中用到的cod_method.json和cod_dataset.json，请问可以提供一下吗

Thanks for the great work! I would like to ask if you can publish the code and dataset of the paper?

Hi! I read your paper "Weakly-Supervised Camouflaged Object Detection with Scribble Annotations", Thanks for the great work! I would like to ask if you can publish the code and dataset of the paper?

S-COD 数据集为什么都是黑的图片

S-COD 数据集为什么都是黑的图片

References

hi. I have a request for update of your paper please use number for references. Its easy to find by numbers.

again thank you for your powerful work...

Fatima

Thanks for the great work! Here are some questions. Which version of python corresponds to this paper?

Thanks for the great work! Here are some questions. Which version of python corresponds to this paper?

ask for optimization

hello. I want to work on your paper and add a module for texture segmentation, a different module, can you help me how can I do it?

LSR Module中激活函数的相关问题

您好，最新拜读您的论文时，在Supplementary Material中的Logical Semantic Relation (LSR) Module发现您在论文中提及最后的输出时用GeLU函数激活，但在net.py文件中，LSR模块最后依旧是使用ReLU模块激活。另外，我想请问一下您是根据什么来决定使用GeLU还是ReLU激活的呢？


静候您的回复，不甚感激。

Question about Results on NC4K

Thank you very much for your work. May I ask if it is possible to release the prediction results of DUSD (Zhang et al 2018), USPS (Nguyen et al 2019), SS (Zhang et al 2020b), and SCWSSOD (Yu et al 2021) on the NC4K benchmark?

数据集中图片全部为黑色，没有期待中的涂鸦注释

非常感谢您的工作！在对您的代码进行复现时，我发现项目中提供的数据集S-COD中的图片似乎被损坏了。如果方便的话，您能否检查并重新上传S-COD数据集呢？万分感谢！

涂鸦标注数据预处理

您好！非常感谢您出色的工作！在拜读您的代码的过程中，对data文件夹下dataset.py中的第110-112行代码，mask[mask == 0.] = 255. mask[mask == 2.] = 0. ，论文中说0代表未标注，1代表目标部分标注，2代表背景部分标注，请问这里为什么只处理了0和2的情况，而没有对mask == 1.进行处理呢？

关于Fss特征图的一些问题

您好，最近在拜读您的论文时，对论文中提及的Fss特征图有一些问题想向您请教：
（1）Fss(semantic feature map/intermediate feature map)是在CRNet的倒数第二层得到的吗？(根据文中给出的CRNet结构图，我看到Fss是由最后提取出的Fout0特征和Fout1特征进行fusion操作得到的，所以我对文中后续提到的Fss is extracted before the final prediction layer比较困解)

（2）文中提到，需要停止Fss的梯度，我不太理解这里的含义和这样做的意义(从文中可以知道，提取Fss是用来计算SS loss)
（3）根据文章可以知道，Pi表示的是像素i为前景的概率，为什么Pi的计算是用Fss的每个通道对应的像素位置的值做累和来计算得到(除了Fss，可不可以用其他的feature map来这样做呢)？

期待您的回复，不甚感激。

Results on NC4K

Hi, @dddraxxx

Could you please provide the prediction results on NC4K dataset?

Thanks.