DIfference in the results from HF demo and COlab

[syscv]

DIfference in the results from HF demo and COlab

[lkeab]

when you install detectron2 first and using "from detectron2 import xxx", the files of original detectron2 package is invoked instead of our modified mask head file.

[lkeab]

"The checkpoint state_dict contains keys that are not used by the model:..." This suggests that the pretrained model and your mask head file has a dismatch. Can you make sure you install transfiner correctly? I guess you are using the default mask head by mrcnn provided by detectron2.

[vanga]

Thanks.
I installed detectron2 using pip first and then installed transfiner by doing python3 setup.py build develop..
Let me follow the instructions in this repo fully and try again (I remember I had some installation issues on the colab which is why I installed detectron2 separately.). Are you aware of any compatability issues with different detectron2 versions?

[vanga]

It indeed seems to be an issue with some dependency compatability issue.
Earlier I was installing detectron2 first which is installing a bunch of dependecies unlike the instructions in the readme of this repo..
that somehow led to the issue of model not getting loaded properly.
The reason that I was donig that is that the package "omegaconf" is getting installed by the transfiner installation script at a place that is not visible in colab.

Things are working as expected if If I dont' install detectron2 first. And the output masks are much much better :)

I would still like to understand where the problem is exactly, but please feel free to close this issue.

[vanga]

Hi @lkeab

While the outputs are much better, I still see these logs now while trying to use the demo.py in Colab.
I suppose that this is also not supposed to happen, any idea what might be wrong (note the lines about roi_heads.mask_head*)?

fpn input shapes: {'res2': ShapeSpec(channels=256, height=None, width=None, stride=4), 'res3': ShapeSpec(channels=512, height=None, width=None, stride=8), 'res4': ShapeSpec(channels=1024, height=None, width=None, stride=16), 'res5': ShapeSpec(channels=2048, height=None, width=None, stride=32)}
[06/15 07:28:33 fvcore.common.checkpoint]: [Checkpointer] Loading from ./pretrained_models/output_3x_transfiner_r101_deform.pth ...
WARNING [06/15 07:28:33 fvcore.common.checkpoint]: Some model parameters or buffers are not found in the checkpoint:
roi_heads.mask_head.deconv_bo.{bias, weight}
roi_heads.mask_head.predictor_bo.{bias, weight}
  0% 0/1 [00:00<?, ?it/s]/content/transfiner/detectron2/modeling/roi_heads/fast_rcnn.py:154: UserWarning: This overload of nonzero is deprecated:
	nonzero()
Consider using one of the following signatures instead:
	nonzero(*, bool as_tuple) (Triggered internally at  /pytorch/torch/csrc/utils/python_arg_parser.cpp:882.)
  filter_inds = filter_mask.nonzero()
/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py:3063: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  "See the documentation of nn.Upsample for details.".format(mode))

configs/transfiner/mask_rcnn_R_101_FPN_3x_deform.yaml is the config being used.
I have followed the instructions from the readme and the python version is also 3.7. Could this still be because of some installation issue?

[lkeab]

Are you using the correct model? "Some model parameters or buffers are not found in the checkpoint:" indicates a mismatch between config file and the model weights.

[vanga]

I downloaded the latest checkpoint from drive and I don't see these warnings any longer. Thanks.

I was using a checkpoint that I downloaded a couple of months back (April).. I am not sure if something changed or If I downloaded it wrong.