seoungwugoh/rgmp

Fast Video Object Segmentation by Reference-Guided Mask Propagation

rgmp's Issues

Accuracy without pre-training

Hi, I would like to ask about the ablation study without pre-training(-PT).
How many epoch did you train and how to set the learning rate for the fine-tuning part to get the accuracy in Table 5? (J mean is 68.6 and F mean is 68.9)
Thanks for your attention and hope to hear your reply soon.

Hello，I want check the method that we calculate FPS.

It's only a trivial problem.
Maybe, we should calculate FPS by using
num_frames /(time.time()-tt)

Thank you for your researches. ^_^

How about the one-stage training strategy?

Hi, I note that all your models are trained by using two-stage training strategy, which trains models on a larger additional dataset for the first stage. However, in Table 1 from your paper, models you compared with are trained just on DAVIS-2016. THis is not fair for other models, so I wonder how about J mean and F mean of your models trained just on DAVIS-2016?

The weight path is missing?

Why use softmax instead of sigmoid?

line 66: msv_E2[sc] = upsample(F.softmax(e2[0], dim=1)[:,1].data.cpu(), (h,w))

line 71: msv_E2[sc] = F.softmax(e2[0], dim=1)[:,1].data.cpu()

If you run these lines for each object (Propagate_MS is inside the loop for o in range(num_objects) ) , why use softmax instead of sigmoid?

Implement three measures

Hello! I am a newcomer to learning video segmentation. Do you implement three measures:region similarity,contour accuracy and temporal instability? Thank you !

why we us额

Train with youtube-vos

Sorry to bother you.Have you tried the youtube-vos dataset. Can it work? I have tried the youtube-vos dataset. But I failed to make it work.

Propagate_MS() missing 1 required positional argument: 'P2'

Hello.
I tried the RGMP code to understand it.
And i use the code made by xanderchf.
but there is missing argument in the middle of code.
How can i solve it.

Error was occurred in here
for f in range(0, num_bptt - 1):
output, ms = Propagate_MS(ms, all_F[:,:,f+1], all_E[:,0,f])

TypeError: Propagate_MS() missing 1 required positional argument: 'P2'

Thanks you in advance.

Real time video Analysis.

Thank you for the implementation.

Can you provide any idea how I can test this on real time video ?

Fine-Tuning input size

Hi,
In the paper, input size is stated to be 256x512. However, in the code patch input size is 512x864. Please correct me what I am missing in the code.
Thanks

In train.py, There is no 'P2' argument in Propagate_MS made by xanderchf

Hello.
I tried the RGMP code to understand it.
And i use the code made by xanderchf.
but there is missing argument in the middle of code.
How can i solve it.

Error was occurred in here
for f in range(0, num_bptt - 1):
output, ms = Propagate_MS(ms, all_F[:,:,f+1], all_E[:,0,f])

TypeError: Propagate_MS() missing 1 required positional argument: 'P2'

Thanks you in advance.

why should we use "softmax" twice in "run.py"?

1. in 71 line: msv_E2[sc] = F.softmax(e2[0], dim=1)[:,1].data.cpu()

2. in 116 line: all_E[:,:,f+1] = F.softmax(Variable(all_E[:,:,f+1]), dim=1).data

Why should we use "softmax" twice?

the problem of train.py

why do not you use bptt_hsm in train.py? what is the meaning of ntokens in line 134 of train.py? looking forward your replay!Thank you very much

Training & data generation code?

Thanks for releasing the demo code! I'm wondering if you are planning to release training and the code to generate simulated data?

Questions about the logit function

In your RGMP paper Sec 3.3, you write the soft logit function. But in your code logit, I am a little confused about it.

Questions:

1. In your training, you view the background as one instance and then propagate it? If so, the probability (Mask) of the background is the sum of other instance labels, which really confuses me.
2. Could you please provide more details in training? (How to logit function with softmax is being used, is it just like the inference stage?)

Thanks in advance!

Sorry for my mistake.

Global Convolution Block

Thanks for your work. but there is no "Residual Block" in your code.

Davis annotation

Hi ,How do you handle the multiple instances tag for Davis 2017 dataset? Is it converted into two categories? How do you visualize Davis 2017 data sets?Because Davis is not a different category with different colors. For example, the label color of bear is the same as that of bicycle.Thank you!

Would like to provide a small size pth?

I want to test it in mobile system to see if the speed is good enough.