About inference speed

[xuebinqin]

About inference speed

[DBraun]

With the large U2Net model on my RTX 2080, I'm getting about 20ms per frame in my C++ app, making sure the module and input data are on the GPU. Using release (not debug) libtorch DLLS:

// in loop for every frame
auto start = high_resolution_clock::now();
auto result = module.forward({ rgb });
auto stop = high_resolution_clock::now();
myDuration = duration_cast<milliseconds>(stop - start);
myDuration.count(); // about 20ms avg when displayed via HUD

[xuebinqin]

Both are tested on a NVIDIA gtx 1080 ti GPU.

[clelouch]

I've also tried to test the inference speed of U2Net. However, the larger U2Net running at a speed of 16 FPS on RTX TITAN Xp when processing 352X352X3 image.

with torch.no_grad():
    start = time.time()
    res = model(img)
    torch.cuda.synchronize()
    end = time.time()
    # print(res.shape)
    print(end - start)