Issue in using the generated Fine-tuned weight file

[asad-ismail]

Issue in using the generated Fine-tuned weight file

[Asad-Ismail]

Hi @tarunsk1998 thank you for pointing it out should be fixed now see this commit
757928c#r138678714

[tarunsk1998]

Hi @Asad-Ismail,

I tried the latest code, but faced a follow up issue while running the test.py file.

The below is the error,
C:\venv\lib\site-packages\transformers\modeling_utils.py:993: FutureWarning: The device argument is deprecated and will be removed in v5 of Transformers.
warnings.warn(
Traceback (most recent call last):
File "C:\Grounding-Dino-FineTuning\test.py", line 124, in
process_video(model_weights=model_weights, model_config=model_config,
File "C:\Grounding-Dino-FineTuning\test.py", line 90, in process_video
boxes, logits, phrases = predict(
File "C:\Grounding-Dino-FineTuning\groundingdino\util\inference.py", line 76, in predict
outputs = model(image[None], captions=[caption])
File "C:\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Grounding-Dino-FineTuning\groundingdino\models\GroundingDINO\groundingdino.py", line 303, in forward
self.set_image_tensor(samples)
File "C:\Grounding-Dino-FineTuning\groundingdino\models\GroundingDINO\groundingdino.py", line 212, in set_image_tensor
self.features, self.poss = self.backbone(samples)
File "C:\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Grounding-Dino-FineTuning\groundingdino\models\GroundingDINO\backbone\backbone.py", line 151, in forward
xs = self0
File "C:\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Grounding-Dino-FineTuning\groundingdino\models\GroundingDINO\backbone\swin_transformer.py", line 716, in forward
x = self.patch_embed(x)
File "C:\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Grounding-Dino-FineTuning\groundingdino\models\GroundingDINO\backbone\swin_transformer.py", line 491, in forward
x = self.proj(x) # B C Wh Ww
File "C:\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\venv\lib\site-packages\torch\nn\modules\conv.py", line 463, in forward
return self._conv_forward(input, self.weight, self.bias)
File "C:\venv\lib\site-packages\torch\nn\modules\conv.py", line 459, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Input type (unsigned char) and bias type (float) should be the same

Let me know if this is a known issue and how I can rectified this issue.