asad-ismail/grounding-dino-finetuning

Fine tuning grounding Dino

👋 My name is Asad I like to write code, train vision and multimodal ML models and deploy them in production

🚀 Feel free to contibute to one of my existing projects or reach out for colaboration on a project.

grounding-dino-finetuning's Issues

Issue in using the generated Fine-tuned weight file

I followed the document to set the library code. I generated my custom dataset as provided in the sample CSV file.

I used the groundingdino_swint_ogc.pth pre-trained model for the finetuning purpose and its corresponding config file. I ran the train.py provided in this code for training and the new weights were saved in my machine.

However, the issue appeared when I tried to test the model using the test.py file provided in this code. I configured the model path to the generated fine-tuned model while trying to predict a test image.

The Error is as below:
File "Grounding-Dino-FineTuning\groundingdino\util\inference.py" in Line number 37:
model.load_state_dict(clean_state_dict(checkpoint["model"]), strict=False)
KeyError: 'model'

Let me know how this issue can be rectified. Could the train.py script be missing something?

test the trained model

hello!thanks for your code, that's useful! However, I have several problems during the training process. 1) How much data did you use in fine-tuning？ 2) When I used the fine-tuned model to test, I got more than 800 detection boxes, but there were only 3 objects in the image actually. Have you met this situation? I hope you can help with this. Thank you again and best wishes!

Fine-tune the code

hello,
Is the fine-tuning code (train.py) not complete enough for normal training?

error: UnpicklingError: invalid load key, 'v'.

/content/Grounding-Dino-FineTuning/groundingdino/models/GroundingDINO/ms_deform_attn.py:31: UserWarning: Failed to load custom C++ ops. Running on CPU mode Only!
warnings.warn("Failed to load custom C++ ops. Running on CPU mode Only!")
/usr/local/lib/python3.10/dist-packages/torch/functional.py:507: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3549.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
final text_encoder_type: bert-base-uncased
Traceback (most recent call last):
File "/content/Grounding-Dino-FineTuning/train.py", line 11, in
model = load_model("groundingdino/config/GroundingDINO_SwinT_OGC.py", "weights/groundingdino_swint_ogc.pth")
File "/content/Grounding-Dino-FineTuning/groundingdino/util/train.py", line 40, in load_model
checkpoint = torch.load(model_checkpoint_path, map_location="cpu")
File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1040, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1258, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'v'.

请问训练了很多轮生生成了很多pth文件，应该使用哪个是最好的？