Performance differences between 2 versions of SuperPoint with pretrained weights

[nobobobo]

Performance differences between 2 versions of SuperPoint with pretrained weights

[nobobobo]

uploading two images I used for testing

[sarlinpe]

Can you share the code that yields the first results? Here's what I get with your images and SuperPoint-Open:

import torch
from gluefactory.utils.image import ImagePreprocessor
from gluefactory.visualization.viz2d import plot_images, plot_keypoints, plot_matches
from gluefactory.models.extractors.superpoint_open import SuperPoint
from gluefactory.models.matchers.nearest_neighbor_matcher import NearestNeighborMatcher

improc = ImagePreprocessor({'resize': 1024})
im0 = improc.load_image('test_data/receipt1.jpg')['image']
im1 = improc.load_image('test_data/receipt2.jpg')['image']

sp = SuperPoint({'nms_radius': 4}).cuda().eval()
with torch.no_grad():
    pred0 = sp({'image': im0.cuda()[None]})
    pred1 = sp({'image': im1.cuda()[None]})

pts0, pts1 = [p['keypoints'].squeeze(0).cpu() for p in (pred0, pred1)]
plot_images([im.permute(1,2,0) for im in (im0, im1)],
            titles=[f'{len(p)} keypoints' for p in (pts0, pts1)])
plot_keypoints([pts0, pts1])


matcher = NearestNeighborMatcher({"distance_thresh": 0.5})
matches = matcher({'descriptors0': pred0['descriptors'], 'descriptors1': pred1['descriptors']})['matches0'].squeeze(0)

plot_images([im.permute(1,2,0) for im in (im0, im1)])
plot_matches(pts0[matches.cpu()!=-1], pts1[matches[matches!=-1].cpu()], a=0.5, lw=1)

[nobobobo]

Thanks for trying!

I just ran your code snippet on my end, it seems SuperPoint-Open generates different results with two versions of torch: 2.0.1 and 2.1.0. The difference might be because of some changes implemented in the new PyTorch, which was just relased couple days ago. So for now, I will use torch 2.0.1 for workaround.

import torch 
print(torch.__version__)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

from gluefactory.utils.image import ImagePreprocessor
from gluefactory.models.extractors.superpoint_open import SuperPoint
from gluefactory.visualization.viz2d import plot_images, plot_keypoints, plot_matches

improc = ImagePreprocessor({'resize': 1024})
im0 = improc.load_image('../../test_images/01.jpeg')['image']
im1 = improc.load_image('../../test_images/02.jpeg')['image']

sp = SuperPoint({'nms_radius': 4}).cuda().eval()
with torch.no_grad():
    pred0 = sp({'image': im0.cuda()[None]})
    pred1 = sp({'image': im1.cuda()[None]})

pts0, pts1 = [p['keypoints'].squeeze(0).cpu() for p in (pred0, pred1)]
plot_images([im.permute(1,2,0) for im in (im0, im1)],
            titles=[f'{len(p)} keypoints' for p in (pts0, pts1)])
plot_keypoints([pts0, pts1])

[sarlinpe]

I obtain the exact same results with torch 2.1.0+cu121, 2.0.1+cu117, and 2.0.0+cu117.

[nobobobo]

Okay, thank you for checking. I will test again with a clean environment here as well. Close this issue for the time being.