kamata1729/qatm_pytorch

Pytorch Implementation of QATM:Quality-Aware Template Matching For Deep Learning

Pytorch Non-Official Implementation of QATM:Quality-Aware Template Matching For Deep Learning

arxiv: https://arxiv.org/abs/1903.07254
original code (tensorflow+keras): https://github.com/cplusx/QATM
Qiita(Japanese): https://qiita.com/kamata1729/items/11fd55992c740526f6fc

Dependencies

• torch(1.0.0)
• torchvision(0.2.1)
• cv2
• seaborn
• sklearn
• pathlib

Usage

See qatm_pytorch.ipynb

or

python qatm.py -s sample/sample1.jpg -t template --cuda

• Add --cuda option to use GPU
• Add -s/--sample_image to specify sample image
  only single sample image can be specified in this present implementation
• Add -t/--template_images_dir to specify template image dir

[notice] If neither -s nor -t is specified, the demo program will be executed, which is the same as:

python qatm.py -s sample/sample1.jpg -t template

• --thresh_csv and --alpha option can also be added

Result of Demo

template1_1.png to template1_4.png are contained in sample1.jpg, however, template1_dummy.png is a dummy and not contained

template1_1.png	template1_2.png	template1_3.png	template1_4.png	template1_dummy.png

qatm_pytorch's Issues

score is nan

ソースコードを共有していただきありがとうございます。
お伺いしたいことがあります。
デモのテンプレートマッチングはおこなうことができました。
そこで自分の画像とテンプレートでマッチングを行ったところ以下のようになりました。

詳しく見るとrun_one_sample関数下の
val = model(template, image, image_name)でvalがnanと返ってきました。
どのような原因が考えられるでしょうか。
アドバイスいただけるとありがたいです。

追加情報として
テンプレートとマッチング画像の拡張子はdemoのときと同じですが、サイズはテンプレートしかおなじではありません。
また、テンプレートはマッチング画像から切り取ったものではなく、人間の目で見てもテンプレートがマッチング画像に含まれているか判断が困難な画像を使っています。

Having seen your excellent code implementation, I wonder if you could share the experimental code of CoTM,DDIS and BBS

Having seen your excellent code implementation, I wonder if you could share the experimental code of CoTM,DDIS and BBS？
thank you！

Threshold function always got one match?

I think below function may be always got at least one matching.

def nms(score, w_ini, h_ini, thresh=0.7):
    dots = np.array(np.where(score > thresh*score.max()))

may be need change "score.max()" to a constant (0.1~0.2).

About dos_indices -> dots_indices variable name

Hello, isn't this a misspelling?
qatm_pytorch.py
Line 244 dos_indices = None
dos_indices -> dots_indices used in variable name

Poor performance on BBS dataset

I have tested this implementation on BBS dataset, but get poor performance, specifically, nearly zero accuracy. I wonder if you could test it and post the testing code. Thank you.

Slow and out of memory

As I tested, it's much slower than NCC, and always out of memory for big images (4000*4000), any suggestion?

TypeError: einsum() takes 2 positional arguments but 3 were given

when I run
python qatm.py -s sample/sample1.jpg -t template --cuda

Result
import qatm_pytorch.py...
define model...
calculate score...
Traceback (most recent call last):
File "qatm.py", line 46, in
scores, w_array, h_array, thresh_list = run_multi_sample(model, dataset)
File "mod.py", line 333, in run_multi_sample
File "mod.py", line 306, in run_one_sample
File "mod.py", line 150, in call
TypeError: einsum() takes 2 positional arguments but 3 were given

I would appreciate if you give me any advice~

About License

Dear @kamata1729 ,

I find this OSS useful and I want to use it in my software and include it in my software distribution.

In this case, can I comply with the MIT License for this?

The README states that the original code is QATM(https://github.com/cplusx/QATM) and the license can only be used by non-commercial or academic.
Is QATM_pytorch only required to comply with MIT apart from QATM?

ResNet101 instead of VGG16?

Can we use a pretrained resnet101 instead of vgg? If yes, which two layers' outputs do we use with 'register_forward_hook'?

Update README

Create README and add docker-compose

Runs out of memory

Hi,

My template image is

And my sample image is

My system is running out of memory during the calculate scores phase.

Do you have any suggestions for this?

Thanks,
Shamik

TypeError: pic should be PIL Image or ndarray. Got <class 'NoneType'>

Dear authors,
Really appreciate your wonderful contributions, but unfortunately, I did encounter some problem while I am trying to use the project. When I tried to match my own template with my own sample picture, I get this error: "TypeError: pic should be PIL Image or ndarray. Got <class 'NoneType'>". Are there any formatting problems that might cause this issue?
Thank you so much

no matching!!!

Hi,
Thank you for the code. But please test it and then post. There seems to be no matching whatsoever with any template and any image. I even tried matching template with the same image from where I made the template out....Still no matching.

How to use multiple images instead of just one sample image

How to use multiple images for the template matching?
I see that you have a function called run_multi_sample, but from thereafter, the plot doesn't seem to be the plot of each image that's processed for the templates - matching. Please clarify. I am trying to use this for multiple images not just one.