yang-bob/pmms

Prototype Mixture Models for Few-shot Semantic Segmentation

pmms's Issues

Performance on COCO

Hi Bob,

I'm trying to reproduce results on COCO, but the results I obtain stand almost 3% below the ones announced in the paper. Here is what I got :

Coco 1-shot	Fold-0	Fold-1	Fold-2	Fold-3	Mean
Paper	29.5	36.8	28.9	27.0	30.6
1-shot	27.1	31.6	26.5	26.8	28.0

For info, I used this repo and followed scrupulously the same training procedure as the one mentioned in the paper. I got the best models respectively at epochs : 55k, 115k, 60k, 110k
Would it be possible to release your pre-trained models for COCO to assess whether I have an error in my reproduction ? Thank you very much.

Malik

binary_mask question

Hi Bob
I integrated the binary maps in accordance with the requirements of the comments, then canceled all the comments related to the binary maps, commented the relevant codes, and wanted to try the effects of the binary map, but I got an error. Excuse me, The comment is cancelled, do other operations need to be performed?

Sorry my English is not very well, My idea is that I used the code related to the binary maps to run the entire experiment, but the code reported an error. Can I use all the codes related to the binary maps to successfully run the code?

Pre-trained weights

Hi Bob,

Do you provide any pre-trained weights so that we can run as a demo code?

Could you provide a link to download such pre-trained weights?

Thanks!

How to get predicted mask

Can you please let me know how to get the predicted mask during validation? (after through ASPP)

I used this code(in test_frame.py) to get the predicted mask, but this code always gives GT for me.

for data in val_dataloader:
            begin_time = time.time()
            it = it+1
            query_img, query_mask, support_img, support_mask, idx, size = data

            query_img, query_mask, support_img, support_mask, idx \
                = query_img.cuda(), query_mask.cuda(), support_img.cuda(), support_mask.cuda(), idx.cuda()

            with torch.no_grad():
                logits = model(query_img, support_img, support_mask)

                query_img = F.upsample(query_img, size=(size[0], size[1]), mode='bilinear')
                query_mask = F.upsample(query_mask, size=(size[0], size[1]), mode='nearest')
                print(query_mask.size())

                values, pred = model.get_pred(logits, query_img)
                evaluations.update_evl(idx, query_mask, pred, 0)

                plt.figure()
                plt.subplot(2,2,1)
                plt.imshow(np.array(query_mask.squeeze().cpu()), cmap=cm.tab10_r)
                plt.subplot(2,2,2)
                plt.imshow(np.array(query_img.squeeze().permute(1,2,0).cpu()), cmap=cm.tab10_r)
                plt.axis('off')
                # plt.show()
                print(cnt)
                cnt = cnt + 1
                plt.savefig("result/"+str(cnt)+".png")
                time.sleep(0.1)

About some details in PMMs

Hi, I have read this great work.
However I have some questions especially in the PMMs module code. First, What is the purpose of L2 normalization in Line 61 of PMMs.py ? Dose it affect to remove it? Second, the input to EM algorithm is the masked feature representation, however, zero value positions are not removed after mask operation. The masked feature will have many zero elements. Does this affect the result of EM algorithm?

Found error in computing performance (iou)

Hello,
I found an error in your evaluation code (in function "update_evl" of NoteEvaluation.py).

Given N sample episodes, your code computes average IoU as follows:
(tp_1 + tp_2 + ... + tp_N) / (uni_1 + uni_2 + ... + uni_N)
where uni = tp + fp + fn (union) and tp (intersection)

I believe correct way of measuring IoU is following:
1/N * sum_{i=1}^{N} (tp_i) / (uni_i)

Is there anything that I'm missing from your code?
If I'm correct how would the resulting mIoU change on PASCAL-5i?

Thank you.

PASCAL VOC data_list

Hi, thanks for sharing your work!

It seems that you train and test in VOC using ids from your 'data_list' directory. However, if ids are generated from the train+aug set from VOC and SBD, there should be more ids than those in 'data_list' directory(for example, in my experiment, there are 8829 images containing objects from classes excluding split_0). So would you mind describing the process of generating those ids in each split?

Model selection

Hi,
Thank you for your great work and for releasing the code. I am currently trying to train my own models from scratch, but I can't see from the code how the "best" model is selected during training, as they are all just saved as step_x.pth.tar, hence my question : based on which criterion did you select the "best" model for inference.
Thank you in advance :)

Malik

question about image size of testing

I found the image size is set [481,481] in your val_loader. Does it mean in your testing phase, the miou received in size 481 not the original size as canet?

multi-GPUs training

lost function

Hello, thanks for your paper and code. But, I think your lost some code when release this project.

In test_all_fame.py file, I can find NoteEvaluation.note_best()

from utils import NoteEvaluation
......

Best_Note = NoteEvaluation.note_best()
......

Can your upload the file again, thanks.

About the performence based on our dataset

Hi Yang-Bob,
I train PMM and RPMM model based on our dateset, but the miou is low about 0.1. One example(two defect) of our dataset as follow:

Do you any good ideas? Thanks

TypeError: 'NoneType' object is not subscriptable

I downloaded the Pascal VOC2012 dataset from (http://host.robots.ox.ac.uk/pascal/VOC/voc2012/index.html.
Then, I modified the dataset path in data/voc_train.py and config/setting.py and printed the dataset image paths, that proves my dataset path is correct.
Then, I found that the mask path cannot match the path in data_list/train/splitx_train.txt. Hence, can you release the correct dataset Download URL？Thank you！

When I run your code, the mean IOU value is higher than the mean IOU value in your article.

Hello author, thank you for sharing. When I run your code, the mean IOU value is higher than the mean IOU value in your article. For example, pascal-50 in 1-way 1-shot has reached 53.24% in 5500 steps, while 51.98% in the article. Why is this?

test_5_shot.py

Hello,I have a question.Why merge on dim=2 not dim=1 in the def forward_5shot () function in test_5_shot.py?
support_feature_all = torch.cat([support_feature_all, support_feature], dim=2) #torch.Size([1, 256, 305, 61]) support_mask_all = torch.cat([support_mask_all, support_mask_temp], dim=2)#torch.Size([1, 1, 305, 61]

the resume question

The training phase is very time-cosuming, if the training phase is stopped in the middle time, could we resume from the last saved model?

VOC2012 and Pascal-5i

Hello! I'm sorry I am a newbie in deep learning, I want to ask how to get the Dataset Pascal-5i, I used VOC2012 directly, but there will be some label and picture mismatch problems. Thanks!

TypeError: 'NoneType' object is not subscriptable

I downloaded the Pascal VOC2012 datase and I make sure that the path of the dataset file and the location of the segmentationclassaug file are correct and printed correctly

The code in line #80 under the folder voc_train.py does not runing

    #78    mask_path = self.mask_dir + name + '.png'
   #79      mask = cv2.imread(mask_path)
   #80      labels = np.unique(mask[:,:,0])

Where is the paper link?

'NoneType' object is not subscriptable

import pyupbit
import numpy as np

df = pyupbit.get_ohlcv("BTC")
df['range'] = (df['high'] - df['low']) * 0.5
df['target'] = df['open'] + df['range'].shift(1)

fee = 0.0032
df['ror'] = np.where(df['high'] > df['target'],
df['close'] / df['target'] - fee,
1)

df['hpr'] = df['ror'].cumprod()
df['dd'] = (df['hpr'].cummax() - df['hpr']) / df['hpr'].cummax() * 100
print("MDD(%): ", df['dd'].max())
df.to_excel("dd.xlsx")

---

코인 자동매매를 공부하고 있는 파이썬 초보 입니다.
유튜브 보면서 따라하고 있는데 위와 같이 코드를 입력했는데 터미널로 실행 시키면
TypeError: 'NoneType' object is not subscriptable <--이와 같은 에러 메시지가 뜨는데 뜨는 이유는 뭐고 어떻게 해야 해결이 되는건가요?
현재 아이맥을 사용하고 있으며, 아나콘다랑, 스파이더를 사용하고 있어요 초보자의 문제 해결에 도움 부탁드립니다. 고수님들

running error

Hi, I am new to python. Can any one help to make this code sample work?
Thank you for providing the code! I always report errors when running the main program. Are there some parameter files missing in your code? Can you help me see what errors are? Thank you
error:
File "home/chenjNewisk/PHMs-naster/train.py", Line 124, in train
train(args)
File "home/chenjNewisk/PHMs-naster/train.py", Line 65, in train
train_loader = data. _Loader(args)

question about K-shot setting

Hi!
I found no settings for K-shot in voc_train.py or coco_train.py, but they appear in voc_val.py and coco_val.
So how do I experiment with K-shot settings?

the permance on Pascal-5i

I'm trying to reproduce results on Pascal-5i with the pred-trained weights that you provided, but the results I obtain stand is below the ones announced in the paper.

Dataset Setting

Hi,

In voc_tarin.py, you filter out the image that include noval categories using function 'filte_multi_class'. Is that right?

What is the reason for filtering? And what are the effects if you don't filter out some images?

Can other paper such as CANet also use this setting?

jachymchen

Where download your paper

The paper not in Googlescholar, where can download the paper? Thank you！

COCO2014 or COCO2017 ?

Hi Bob,

Good work on few-shot segmentation. But I have a question about the COCO dataset.

I think PANet and FWB are both evaluated on COCO 2014(~83K training images), but it seems that PMMs are evaluated on COCO 2017(~118K training images). See the code coco_train.py.

If true, I think the comparison in Table 6 (in the paper) is not fair. If I missed some details, please let me know.

Thank you .

question about FPMMs.py

        nn.Conv2d(256+2,256,kernel_size=3,stride=1,padding=1,bias=True),

why the in_channel = 256+2?