jiexiong2016/gcnv2_slam

Real-time SLAM system with deep features

GCNv2 SLAM

Introduction

GCNv2 is a high-throughput variant of the Geometric Correspondence Network for performing RGB-D SLAM online on embedded platforms. We trained the binary descriptor in the same format with ORB (32 bytes) for the convenience of integration. In this implementation, we evaluate the motion estimation using a system built on top the [ORB-SLAM2], (https://github.com/raulmur/ORB_SLAM2). Thanks to the robustness of ORB-SLAM2, our system is able to achive reliable tracking perfomance on our drone platform in real-time.

Example

Online running performance with ORB and GCNv2 features:

ORB:

GCNv2:

Related Publications

• GCNv2: Efficient Correspondence Prediction for Real-Time SLAM, J. Tang, L. Ericson, J. Folkesson and P. Jensfelt, in arXiv:1902.11046, 2019
• Geometric Correspondence Network for Camera Motion Estimation, J. Tang, J. Folkesson and P. Jensfelt, RA-L and ICRA 2018

Dependencies

C++11 or C++0x Compiler

We use the new thread and chrono functionalities of C++11.

Pytorch

We use Pytorch C++ api(libtorch) for deloying the GCNv2. The libtorch can be built as follows:

git clone --recursive -b v1.0.1 https://github.com/pytorch/pytorch
cd pytorch && mkdir build && cd build
python ../tools/build_libtorch.py

The built libtorch library is located at pytorch/torch/lib/tmp_install/ in default.

Update: Have added support for master branch of pytorch or version larger than 1.0.1. For newer version, set TORCH_PATH to pytorch/torch/share/cmake/Torch

Required at least 1.0.1. Lower version of pytorch has cuDNN linking issue:pytorch/pytorch#14033 (comment).

Plese avoid using the pre-built version of libtorch since it will cause linking errors (due to CXX11 ABI issue).

Pangolin

We use Pangolin for visualization and user interface. Dowload and install instructions can be found at: https://github.com/stevenlovegrove/Pangolin.

OpenCV

We use OpenCV to manipulate images and features. Dowload and install instructions can be found at: http://opencv.org.

Required at least 2.4.3. Tested with OpenCV 2.4.11 and OpenCV 3.2.

Eigen3

Required by g2o (see below). Download and install instructions can be found at: http://eigen.tuxfamily.org.

Required at least 3.1.0.

DBoW2 and g2o (Included in Thirdparty folder)

We use modified versions of the DBoW2 library to perform place recognition and g2o library to perform non-linear optimizations. Both modified libraries (which are BSD) are included in the Thirdparty folder.

Preparation

Clone the code

git clone https://github.com/jiexiong2016/GCNv2_SLAM.git

Then build the project

cd GCNv2_SLAM
./build.sh

Make sure to edit build.sh pointing to your local libtorch installation. Edit run.sh to check out how to run with GCNv2 or vanilla ORB. Check the Network.md for the network structure and link for trained models.

Image resolution

Update Set "FULL_RESOLUTION=1" and use "gcn2_640x480.pt" to test with image resolution "640x480" intead. The input image size should be consitent with the model to be used.

Demonstration video

gcnv2_slam's Issues

if matcher SearchByNN bad map point too many

GCNv2 matcher SearchByNN bad map point too many, if GCNv2 matcher SearchByBoW look map point too little

how to get association file?

I can not found such a file in downloaded data:

\./Datasets/rgbd_dataset_freiburg3_long_office_household/associations.txt

the features are too few

Hello! I applied your gcn2.320*240.pt into the monocular and the stereo system, and I ran the KITTI00 dataset . But the number of the features is too small. Is it the reason caused by the training data?

I think there is something wrong with GCNextractor.cc? I hope you can solve this problem

[ 17%] Building CXX object CMakeFiles/ORB_SLAM2.dir/src/Optimizer.cc.o
/home/tang/GCNv2_SLAM/src/GCNextractor.cc:262:11: error: #elif with no expression
/home/tang/GCNv2_SLAM/src/GCNextractor.cc: In function ‘void ORB_SLAM2::nms(cv::Mat, cv::Mat, std::vectorcv::KeyPoint&, cv::Mat&, int, int, int, int, float, float)’:
/home/tang/GCNv2_SLAM/src/GCNextractor.cc:96:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (int i = 0; i < pts_raw.size(); i++)
^
/home/tang/GCNv2_SLAM/src/GCNextractor.cc:107:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (int i = 0; i < pts_raw.size(); i++)
^
/home/tang/GCNv2_SLAM/src/GCNextractor.cc:148:20: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (int i=0; i<select_indice.size(); i++)
^
/home/tang/GCNv2_SLAM/src/GCNextractor.cc: In member function ‘void ORB_SLAM2::GCNextractor::operator()(cv::InputArray, cv::InputArray, std::vectorcv::KeyPoint&, cv::OutputArray)’:
/home/tang/GCNv2_SLAM/src/GCNextractor.cc:269:22: error: ‘img_var’ was not declared in this scope
inputs.push_back(img_var);
^
CMakeFiles/ORB_SLAM2.dir/build.make:127: recipe for target 'CMakeFiles/ORB_SLAM2.dir/src/GCNextractor.cc.o' failed
make[2]: *** [CMakeFiles/ORB_SLAM2.dir/src/GCNextractor.cc.o] Error 1
make[2]: *** 正在等待未完成的任务....
CMakeFiles/Makefile2:109: recipe for target 'CMakeFiles/ORB_SLAM2.dir/all' failed
make[1]: *** [CMakeFiles/ORB_SLAM2.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

GCNextractor.txt

Using the GCN features independantly

Hi, thank you for making this code available. I am trying to build a test project that uses the deep features outside of OrbSlam, and I am running:

void main()
{
	GCNextractor* mpGCNextractor;

	int nFeatures = 1200;
	float fScaleFactor = 1.2;
	int nLevels = 8;
	int fIniThFAST = 20;
	int fMinThFAST = 7;

	mpGCNextractor = new GCNextractor(nFeatures, fScaleFactor, nLevels, fIniThFAST, fMinThFAST);


	std::cout << "loaded deep model" << std::endl;

	cv::Mat frame1 = cv::imread("frames/frame_0Left.jpg", 1);
	cv::Mat frame2 = cv::imread("frames/frame_60Left.jpg", 1);


	cv::resize(frame1, frame1, cv::Size(640, 480));
	cv::resize(frame2, frame2, cv::Size(640, 480));


	std::vector<cv::KeyPoint> mvKeys, mvKeys2;
	cv::Mat mDescriptors, mDescriptors2;

	(*mpGCNextractor)(frame1, cv::Mat(), mvKeys, mDescriptors);
	(*mpGCNextractor)(frame2, cv::Mat(), mvKeys2, mDescriptors2);


	cv::Mat outImg;
	cv::drawKeypoints(frame1, mvKeys, outImg, cv::Scalar(255, 0, 0),cv::DrawMatchesFlags::DRAW_RICH_KEYPOINTS);

	std::vector<cv::DMatch> matches;
	std::vector<cv::Mat> descMatch; // matched desc to return
	find_feature_matches(mvKeys, mvKeys2, mDescriptors, mDescriptors2, matches);


	cv::Mat matchesImage2;
	cv::drawMatches(frame1, mvKeys, frame2, mvKeys2, matches, matchesImage2);
	cv::imshow("curr-prev", matchesImage2);
	cv::waitKey(1);
	cv::imshow("kp", outImg);
	cv::waitKey(0);

}

This gives me keypoints, but they don't make any sense, and none are matched.

I have switched to:

 const char *net_fn = "GCN/gcn2_640x480.pt";
  //  net_fn = (net_fn == nullptr) ? "gcn2.pt" : net_fn;
    module = torch::jit::load(net_fn);

in GCNextractor.cpp.

What am I doing wrong?

Thanks!

When will you release the training code?

Hi,
Your paper has been published now and I am very interested in your work.
Do you have any plan to release your training code now?

特征获取不错，但误差和ORB对比还很大

特征获取不错，但误差和ORB对比还很大。你可以插入一个opegl 3d 物体和orb的数据运行效果对比。

Failed to load image at

ORB-SLAM2 Copyright (C) 2014-2016 Raul Mur-Artal, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: RGB-D

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Camera Parameters:

• fx: 267.7
• fy: 269.6
• cx: 160.05
• cy: 123.8
• k1: 0
• k2: 0
• p1: 0
• p2: 0
• fps: 30
• color order: RGB (ignored if grayscale)

ORB Extractor Parameters:

• Number of Features: 1000
• Scale Levels: 8
• Scale Factor: 1.2
• Initial Fast Threshold: 20
• Minimum Fast Threshold: 7

Depth Threshold (Close/Far Points): 5.97684

---

Start processing sequence ...
Images in the sequence: 2588

Failed to load image at: /home/mujinzhen/Workspace/Datasets/TUM/freiburg3/rgbd_dataset_freiburg3_long_office_household/
./run.sh: 行 10: 7248 段错误 NN_ONLY=1 GCN_PATH=gcn2_320x240.pt ./rgbd_gcn ../Vocabulary/GCNvoc.bin TUM3_small.yaml ~/Workspace/Datasets/TUM/freiburg3/rgbd_dataset_freiburg3_long_office_household ~/Workspace/Datasets/TUM/freiburg3/rgbd_dataset_freiburg3_long_office_household/rgb.txt

PYtorch GPU version or CPU version?

Dear author ,I wanna ask a question that can I use your slam version by means of pytorch cpu version,or it will fail in compiling?

linking errors

this question was raised before(#2 ,according to the previous method,it is not solved on my computer,so i want to ask others ,did you encounter the similar question?and how to solve it?

../lib/libORB_SLAM2.so: undefined reference to pangolin::Split(std::string const&, char)' ../lib/libORB_SLAM2.so: undefined reference to pangolin::BindToContext(std::string)' ../lib/libORB_SLAM2.so: undefined reference to DBoW2::FORB::toString(cv::Mat const&)' ../lib/libORB_SLAM2.so: undefined reference to pangolin::CreateWindowAndBind(std::string, int, int, pangolin::Params const&)' ../lib/libORB_SLAM2.so: undefined reference to DBoW2::FORB::fromString(cv::Mat&, std::string const&)' ../lib/libORB_SLAM2.so: undefined reference to pangolin::CreatePanel(std::string const&)'
my setup:
ubuntu 16.04
gcc 5.4.0
g++ 5.4.0
cmake 3.11.3
pytorch 1.1.0
cuda 9.2

Error compiling pytorch under windows

c:\users\administrator\pytorch\aten\src\aten\cpu\vec256\vec256_int.h(428): error C3861: “_mm256_extract_epi64”: 找不到标识符 (编译源文件C:\Users\Administrator\pytorch\build\aten\src\ATen\native\cpu\ReduceOpsKernel.cpp.AVX2.cpp)

When I use the compiled libtorch, I get an error when using it under Windows:
"std":Unclear symbol(不明确的符号)，ie.std::vector
When I looked up the std statement, I found that pytorch defined many std namespaces, which caused the name to be duplicated.

descriptor is how to form?

Hello,can you tell me ,after upsample,and binary neural networ,form an 256x320x240 feature map/heatmap ,and now,how to operation?

Build Fail

I've been trying to install this project onto a JetsonTX2 and keep getting build errors.

I have openCV 4.1.1, EIgen and Pangolin suscesfully installed and the build.sh file is correctly pointing to my build of Torch.

In python2 and python3, I can suscesfully import cv2 but for somereason in these logs it shows that g2o fails to build due to "fatal error: opencv/cv.h: No such file or directory."

Here is the exact output of build.sh

Configuring and building Thirdparty/DBoW2 ...
mkdir: cannot create directory ‘build’: File exists
CMake Error at CMakeLists.txt:31 (message):
  OpenCV > 2.4.3 not found.


-- Configuring incomplete, errors occurred!
See also "/home/nvidia/GCNv2_SLAM/Thirdparty/DBoW2/build/CMakeFiles/CMakeOutput.log".
make: *** No targets specified and no makefile found.  Stop.
Configuring and building Thirdparty/g2o ...
mkdir: cannot create directory ‘build’: File exists
-- BUILD TYPE:Release
-- Compiling on Unix
CMake Warning (dev) at /usr/share/cmake-3.10/Modules/FindOpenMP.cmake:310 (if):
  if given arguments:

    "TRUE"

  An argument named "TRUE" appears in a conditional statement.  Policy
  CMP0012 is not set: if() recognizes numbers and boolean constants.  Run
  "cmake --help-policy CMP0012" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.
Call Stack (most recent call first):
  /usr/share/cmake-3.10/Modules/FindOpenMP.cmake:425 (_OPENMP_GET_SPEC_DATE)
  CMakeLists.txt:47 (FIND_PACKAGE)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /usr/share/cmake-3.10/Modules/FindOpenMP.cmake:310 (if):
  if given arguments:

    "TRUE"

  An argument named "TRUE" appears in a conditional statement.  Policy
  CMP0012 is not set: if() recognizes numbers and boolean constants.  Run
  "cmake --help-policy CMP0012" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.
Call Stack (most recent call first):
  /usr/share/cmake-3.10/Modules/FindOpenMP.cmake:425 (_OPENMP_GET_SPEC_DATE)
  CMakeLists.txt:47 (FIND_PACKAGE)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring done
-- Generating done
-- Build files have been written to: /home/nvidia/GCNv2_SLAM/Thirdparty/g2o/build
[100%] Built target g2o
Configuring and building ORB_SLAM2 ...
mkdir: cannot create directory ‘build’: File exists
Build type: Release
-- Using flag -std=c++11.
TORCH_PATH set to: /home/nvidia/pytorch/torch/lib/tmp_install/share/cmake/Torch
-- Caffe2: CUDA detected: 10.0
-- Caffe2: CUDA nvcc is: /usr/local/cuda-10.0/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda-10.0
-- Caffe2: Header version is: 10.0
-- Found cuDNN: v7.6.3  (include: /usr/include, library: /usr/lib/aarch64-linux-gnu/libcudnn.so)
-- Autodetected CUDA architecture(s): 6.2
-- Added CUDA NVCC flags for: -gencode;arch=compute_62,code=sm_62
-- Torch version is: 1.0.0
-- Configuring done
-- Generating done
-- Build files have been written to: /home/nvidia/GCNv2_SLAM/build
Scanning dependencies of target ORB_SLAM2
[  8%] Building CXX object CMakeFiles/ORB_SLAM2.dir/src/Tracking.cc.o
[  8%] Building CXX object CMakeFiles/ORB_SLAM2.dir/src/System.cc.o
[ 13%] Building CXX object CMakeFiles/ORB_SLAM2.dir/src/LoopClosing.cc.o
[ 17%] Building CXX object CMakeFiles/ORB_SLAM2.dir/src/LocalMapping.cc.o
In file included from /home/nvidia/GCNv2_SLAM/include/Frame.h:32:0,
                 from /home/nvidia/GCNv2_SLAM/include/KeyFrame.h:29,
                 from /home/nvidia/GCNv2_SLAM/include/MapPoint.h:24,
                 from /home/nvidia/GCNv2_SLAM/include/FrameDrawer.h:25,
                 from /home/nvidia/GCNv2_SLAM/include/Viewer.h:25,
                 from /home/nvidia/GCNv2_SLAM/include/Tracking.h:28,
                 from /home/nvidia/GCNv2_SLAM/include/System.h:30,
                 from /home/nvidia/GCNv2_SLAM/src/System.cc:23:
/home/nvidia/GCNv2_SLAM/include/GCNextractor.h:34:10: fatal error: opencv/cv.h: No such file or directory
 #include <opencv/cv.h>
          ^~~~~~~~~~~~~
compilation terminated.
CMakeFiles/ORB_SLAM2.dir/build.make:62: recipe for target 'CMakeFiles/ORB_SLAM2.dir/src/System.cc.o' failed
make[2]: *** [CMakeFiles/ORB_SLAM2.dir/src/System.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from /home/nvidia/GCNv2_SLAM/include/Frame.h:32:0,
                 from /home/nvidia/GCNv2_SLAM/include/KeyFrame.h:29,
                 from /home/nvidia/GCNv2_SLAM/include/MapPoint.h:24,
                 from /home/nvidia/GCNv2_SLAM/include/FrameDrawer.h:25,
                 from /home/nvidia/GCNv2_SLAM/include/Viewer.h:25,
                 from /home/nvidia/GCNv2_SLAM/include/Tracking.h:28,
                 from /home/nvidia/GCNv2_SLAM/src/Tracking.cc:22:
/home/nvidia/GCNv2_SLAM/include/GCNextractor.h:34:10: fatal error: opencv/cv.h: No such file or directory
 #include <opencv/cv.h>
          ^~~~~~~~~~~~~
compilation terminated.
In file included from /home/nvidia/GCNv2_SLAM/include/Frame.h:32:0,
                 from /home/nvidia/GCNv2_SLAM/include/MapPoint.h:25,
                 from /home/nvidia/GCNv2_SLAM/include/KeyFrame.h:24,
                 from /home/nvidia/GCNv2_SLAM/include/LoopClosing.h:24,
                 from /home/nvidia/GCNv2_SLAM/src/LoopClosing.cc:21:
/home/nvidia/GCNv2_SLAM/include/GCNextractor.h:34:10: fatal error: opencv/cv.h: No such file or directory
 #include <opencv/cv.h>
          ^~~~~~~~~~~~~
In file included from /home/nvidia/GCNv2_SLAM/include/Frame.h:32:0,
                 from /home/nvidia/GCNv2_SLAM/include/MapPoint.h:25,
                 from /home/nvidia/GCNv2_SLAM/include/KeyFrame.h:24,
                 from /home/nvidia/GCNv2_SLAM/include/LocalMapping.h:24,
                 from /home/nvidia/GCNv2_SLAM/src/LocalMapping.cc:21:
/home/nvidia/GCNv2_SLAM/include/GCNextractor.h:34:10: fatal error: opencv/cv.h: No such file or directory
 #include <opencv/cv.h>
          ^~~~~~~~~~~~~
compilation terminated.
compilation terminated.
CMakeFiles/ORB_SLAM2.dir/build.make:86: recipe for target 'CMakeFiles/ORB_SLAM2.dir/src/Tracking.cc.o' failed
make[2]: *** [CMakeFiles/ORB_SLAM2.dir/src/Tracking.cc.o] Error 1
CMakeFiles/ORB_SLAM2.dir/build.make:134: recipe for target 'CMakeFiles/ORB_SLAM2.dir/src/LoopClosing.cc.o' failed
make[2]: *** [CMakeFiles/ORB_SLAM2.dir/src/LoopClosing.cc.o] Error 1
CMakeFiles/ORB_SLAM2.dir/build.make:110: recipe for target 'CMakeFiles/ORB_SLAM2.dir/src/LocalMapping.cc.o' failed
make[2]: *** [CMakeFiles/ORB_SLAM2.dir/src/LocalMapping.cc.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/ORB_SLAM2.dir/all' failed
make[1]: *** [CMakeFiles/ORB_SLAM2.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

Perhaps is a linking problem since I installed openCV with JetPack 4.2?

Can gcnv2 extract feature points on pictures with resolution greater than 640 * 480?

If my image resolution is large, can I extract features without reducing the resolution? How will it work?

Linking issue

Hi @jiexiong2016,

I have tried to compile your code but I got error in the final linking step:

../lib/libORB_SLAM2.so: undefined reference to pangolin::Split(std::string const&, char)' ../lib/libORB_SLAM2.so: undefined reference to pangolin::BindToContext(std::string)'
../lib/libORB_SLAM2.so: undefined reference to DBoW2::FORB::toString(cv::Mat const&)' ../lib/libORB_SLAM2.so: undefined reference to pangolin::CreateWindowAndBind(std::string, int, int, pangolin::Params const&)'
../lib/libORB_SLAM2.so: undefined reference to DBoW2::FORB::fromString(cv::Mat&, std::string const&)' ../lib/libORB_SLAM2.so: undefined reference to pangolin::CreatePanel(std::string const&)'

Do you have any idea on how to fix it? I also tried to compile ORB-SLAM2 and that works fine. So I assume both DBOW2 and Pangolin are installed correctly.

Thank you for your help!

When will it be publish traking GCN codes？

When will it be publish traking GCN codes？

virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()

virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()

Support MobileNet to make it faster?

Does the author try the Mobilenet design? How much speedup and will performance change much?

A small question about paper

Thanks for sharing this nice work!

I have a question about the expression "if abs(xnj − xi) < c " at line 4 in the Algorithm 1 in paper "GCNv2: Efficient Correspondence Prediction for Real-Time SLAM, J. Tang, L. Ericson, J. Folkesson and P. Jensfelt, in arXiv:1902.11046, 2019".
I think the "<" should be ">", because we want to tolerate the noise caused by imperfect camera calibration and ground truth. So we will not use points within "c" threshold (e.g c = 5 pixel) as negative samples, which are very likely to be the bias caused by noise.
Am I correct? Is this a typo mistake or am I getting it wrong? Waiting for your reply! Thanks!

question about CMAKE_PREFIX_PATH for pytorch

Hi, @jiexiong2016 ,

According to build.sh, CMAKE_PREFIX_PATH should be set pointing to pytorch lib location. Would you provide an example for it?
For example, my pytorch is installed at /root/anaconda3/envs/pytorch1.0/lib/python3.6/site-packages/torch/lib.

THX!

terminate called after throwing an instance of 'torch::jit::script::ErrorReport'

Torch version is: 1.1.0
in the past ,i used v1.0.1,every thing is ok.However,when i change pytorch to v1.1.0,when it comes to torch::jit::script::Module module = torch::jit::load("../gcn2_320x240.pt");
it returns :
`terminate called after throwing an instance of 'torch::jit::script::ErrorReport'
what():
Arguments for call are not valid.
The following operator variants are available:

aten::grid_sampler(Tensor input, Tensor grid, int interpolation_mode, int padding_mode, bool align_corners) -> (Tensor):
Argument align_corners not provided.

The original call is:
at code/gcn.py:67:22
_25 = torch.slice(vu, 0, 0, 9223372036854775807, 1)
_26 = torch.select(_25, 1, 0)
_27 = torch.slice(vu, 0, 0, 9223372036854775807, 1)
_28 = torch.select(_27, 1, 1)
_29 = torch.to(_26, dtype=4, layout=0, device=torch.device("cuda:0"), non_blocking=False, copy=False)
_30 = torch.to(_28, dtype=4, layout=0, device=torch.device("cuda:0"), non_blocking=False, copy=False)
_31 = torch.unsqueeze(torch.index(CONSTANTS.c1, [_29, _30]), 1)
ref_points = torch.cat([_24, _31], 1)
grid = torch.view(ref_points, [1, 1, -1, 2])
_32 = torch.squeeze(torch.grid_sampler(input, grid, 0, 0))
~~~~~~~~~~~~~~~~~~ <--- HERE
desc_1 = torch.t(_32)
desc_2 = torch.to(torch.gt(desc_1, 0), 6, False, False)
_33 = ops.prim.NumToTensor(torch.size(desc_2, 0))
desc_3 = torch.view(desc_2, [int(_33), 32, 8])
desc_4 = torch.mul(desc_3, CONSTANTS.c2)
desc = torch.sum(desc_4, [2], False)
_34 = (_17, torch.to(desc, 0, False, False))
return _34`
who can help me?

Is the matching the same as original orbslam2?

Hi, and thank you for making this code available. I am digging into it, and it looks like the features / descriptors are detected and created here:

https://github.com/jiexiong2016/GCNv2_SLAM/blob/master/src/GCNextractor.cc#L222

But the matching is done in the same class as the original orbslam2?
IS the matching step unchanged?

Thanks again!

Some operational problems

Excuse me , i have some problems.

1. Do you have NVIDIA VGA on your plane?
2. If I don't have nvidia, can I finish gcnv2_slam?

is there some index like response of ORB feature?

Hi, I am very grateful for your work, and the result of feature extraction made my eyes shine. But I would like to ask whether the feature points extracted by GCNV2 contain indicators similar to corner response, so that I can eliminate some unstable points, for example: the features on the ground

GCN-SLAM runing only use RGB?

The GCN-SLAM using the RGB-D. And it can runing only use RGB?

some differences between model and network

excuse me, I find there might be some differences between your network and model,in your model, there is a drop function, could you tell me about the content in this function?thanks

你好，不知道是我的pytorch装错了还是路径有问题？

CMake Error at CMakeLists.txt:48 (find_package):
By not providing "FindTorch.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "Torch", but
CMake did not find one.

Could not find a package configuration file provided by "Torch" with any of
the following names:

TorchConfig.cmake
torch-config.cmake

Add the installation prefix of "Torch" to CMAKE_PREFIX_PATH or set
"Torch_DIR" to a directory containing one of the above files. If "Torch"
provides a separate development package or SDK, be sure it has been
installed.

-- Configuring incomplete, errors occurred!
See also "/home/tang/GCNv2_SLAM/build/CMakeFiles/CMakeOutput.log".
make: *** 没有指明目标并且找不到 makefile。 停止。

build error

how to get the visual correspondence training data?

Hi, I am also studying the method of using deep learning to obtain visual associations. I would like to ask how your annotation data is obtained? Kitti optical flow data or automatic labeling?

Could the GCNv2 network forward with a single RGB frame?

Thanks for sharing this nice work!

In the paper, the only place you mentioned depth information is in eq. 3, to find correct correspondence. I wonder could I just passed a single RGB frame to get keypoints and descriptions into network while in inference time？

Thanks！

GCNExtraction breaks after frame #18

Program execution breaks at module = torch::jit::load(net_fn); in GCNExtractor.cc.

net_fn has the value gcn2.pt.
Why is it breaking at frame #18 and what is the program doing with previous frames that is different at frame #18? I am performing Monocular tracking.

Supporting Stereo Euroc

Will the author consider supporting Stereo cameras in the future?

LoadFromBinaryFile function need to be fixed

The program breaks at the loadFromBinary file funtion. How to fix it?

can you tell me,what's this,the number below the "New map created with 287 points"

create_voc error

Thank you for your good work, I want to follow your create_voc.cc to train vocabulary, but I encounter an error.

[ 91%] Built target ORB_SLAM2
[ 95%] Linking CXX executable ../GCN2/create_voc
CMakeFiles/create_voc.dir/GCN2/create_voc.cc.o: In function void std::__once_call_impl<std::_Bind_simple<torch::jit::script::Method::get_executor()::{lambda()#1} ()> >()': create_voc.cc:(.text._ZSt16__once_call_implISt12_Bind_simpleIFZN5torch3jit6script6Method12get_executorEvEUlvE_vEEEvv[_ZSt16__once_call_implISt12_Bind_simpleIFZN5torch3jit6script6Method12get_executorEvEUlvE_vEEEvv]+0xb6): undefined reference to torch::jit::GraphExecutor::GraphExecutor(std::shared_ptrtorch::jit::Graph, bool)'
CMakeFiles/create_voc.dir/GCN2/create_voc.cc.o: In function torch::jit::script::Method::defaultSchemaFor(torch::jit::script::Method const&)': create_voc.cc:(.text._ZN5torch3jit6script6Method16defaultSchemaForERKS2_[_ZN5torch3jit6script6Method16defaultSchemaForERKS2_]+0x1b2): undefined reference to torch::jit::Value::uniqueNameBaseabi:cxx11 const'
CMakeFiles/create_voc.dir/GCN2/create_voc.cc.o: In function main': create_voc.cc:(.text.startup+0xdd): undefined reference to torch::jit::load(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, c10::optionalc10::Device)'
create_voc.cc:(.text.startup+0x56e): undefined reference to torch::autograd::Variable::Impl::Impl(at::Tensor, bool, torch::autograd::Edge)' create_voc.cc:(.text.startup+0xf5c): undefined reference to torch::jit::GraphExecutor::run(std::vector<c10::IValue, std::allocatorc10::IValue >&)'
collect2: error: ld returned 1 exit status
CMakeFiles/create_voc.dir/build.make:166: recipe for target '../GCN2/create_voc' failed
make[2]: *** [../GCN2/create_voc] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/create_voc.dir/all' failed
make[1]: *** [CMakeFiles/create_voc.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2
Thank you, sincerely.

是真的看不懂哪错了？

[ 34%] Building CXX object CMakeFiles/ORB_SLAM2.dir/src/GCNextractor.cc.o
/home/tang/GCNv2_SLAM/src/GCNextractor.cc:262:11: error: #elif with no expression
[ 39%] Building CXX object CMakeFiles/ORB_SLAM2.dir/src/Converter.cc.o
/home/tang/GCNv2_SLAM/src/GCNextractor.cc: In function ‘void ORB_SLAM2::nms(cv::Mat, cv::Mat, std::vectorcv::KeyPoint&, cv::Mat&, int, int, int, int, float, float)’:
/home/tang/GCNv2_SLAM/src/GCNextractor.cc:96:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (int i = 0; i < pts_raw.size(); i++)
^
/home/tang/GCNv2_SLAM/src/GCNextractor.cc:107:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (int i = 0; i < pts_raw.size(); i++)
^
/home/tang/GCNv2_SLAM/src/GCNextractor.cc:148:20: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (int i=0; i<select_indice.size(); i++)
^
/home/tang/GCNv2_SLAM/src/GCNextractor.cc: In member function ‘void ORB_SLAM2::GCNextractor::operator()(cv::InputArray, cv::InputArray, std::vectorcv::KeyPoint&, cv::OutputArray)’:
/home/tang/GCNv2_SLAM/src/GCNextractor.cc:269:22: error: ‘img_var’ was not declared in this scope
inputs.push_back(img_var);
^
CMakeFiles/ORB_SLAM2.dir/build.make:127: recipe for target 'CMakeFiles/ORB_SLAM2.dir/src/GCNextractor.cc.o' failed
make[2]: *** [CMakeFiles/ORB_SLAM2.dir/src/GCNextractor.cc.o] Error 1
make[2]: *** 正在等待未完成的任务....
CMakeFiles/Makefile2:109: recipe for target 'CMakeFiles/ORB_SLAM2.dir/all' failed
make[1]: *** [CMakeFiles/ORB_SLAM2.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

The trajectory results are different with them in paper

We use CUDA10 & opencv4 & torch1.0 to build you project.
Then the run.sh with gnc2_aug.pt is used. The ATE on TUM fr1_desk, fr1_desk2 are nearly 0.4 m.
I'm not sure where the problem is. Is there any suggestion?

can it put any arbitrary resolution image like 1024*1024？

when I test my own dataset ，it doesn't work， the resolution is 10241024， does the resolution must be 320240 and 640*480？

Need some advice on network training losses

Hi. I'm working on a program to extract keypoints and descriptors with pytorch. I'm really interested in your work and try to reimplement it in python. But I'm struggling with the loss function. May I ask for some advice or demo codes to help me complete the loss function for keypoints and descriptors? Thanks~

GCN training code

@jiexiong2016 Excuse me, will you release the training code of GCN or .v2 any more?

GCN training code

I am doing the same work with you,Could you show the training code

some problem when operating

How can I solve the problem like this？

Trying to build for Raspberry Pi 4. Is this code CUDA only?

Hi,
I was excited to find this as I'm trying to get a slam system to run 30fps on a Raspberry Pi 4!
I just tried to build pytorch following your instructions, on my Raspian-32bit gcc8.3 raspberry.

If fails after ~10 minutes with the error messages,

Found a library with LAPACK API. (generic)
disabling CUDA because NOT USE_CUDA is set
-- CuDNN not found. Compiling without CuDNN support
disabling ROCM because NOT USE_ROCM is set
-- MIOpen not found. Compiling without MIOpen support

---

CMake Error at third_party/ideep/mkl-dnn/CMakeLists.txt:55 (message):
Intel(R) MKL-DNN supports 64 bit platforms only

Does anyone know if I should be able to build and run this project?

Cheers
Fred

Problems with model.

Thanks for this project. I use Windows 10 and have some problems with the model. I described it in detail in the pytorch repository (pytorch/pytorch#28276). For the problem with loading the model, a solution was proposed. It was necessary to slightly fix the model. At the moment, I am faced with the fact that the forward() method does not work correctly. Do you have any thoughts about this error? Could this be due to the fact that I'm using version 1.3 of the pytorch? Or maybe this is a model problem?

Currently I have the following error:

The above operation failed in interpreter, with the following stack trace:
at code/gcn2_480x640.py:43:8
  _6 = torch.unsqueeze(torch.select(_5, 1, 0), 1)
  _7 = torch.to(_6, 6, False, False)
  _8 = torch.slice(vu_1, 0, 0, 9223372036854775807, 1)
  _9 = torch.select(_8, 1, 0)
  _10 = torch.slice(vu_1, 0, 0, 9223372036854775807, 1)
  _11 = torch.select(_10, 1, 1)
  _12 = torch.to(_9, dtype=4, layout=0, device=torch.device("cuda:0"), non_blocking=False, copy=False)
  _13 = torch.to(_11, dtype=4, layout=0, device=torch.device("cuda:0"), non_blocking=False, copy=False)
  _14 = torch.unsqueeze(torch.index(det, [_12, _13]), 1)
  pts = torch.cat([_4, _7, _14], 1)
        ~~~~~~~~~ <--- HERE
  _15 = torch.slice(pts, 0, 0, 9223372036854775807, 1)
  input_20 = torch.neg(torch.select(_15, 1, 2))
  _16, inds_1 = torch.sort(input_20, -1, False)
  inds_2 = torch.to(inds_1, dtype=4, layout=0, device=torch.device("cuda:0"), non_blocking=False, copy=False)
  _17 = torch.index(pts, [inds_2])
  inds = torch.to(inds_2, dtype=4, layout=0, device=torch.device("cuda:0"), non_blocking=False, copy=False)
  vu = torch.index(vu_1, [inds])
  _18 = torch.slice(vu, 0, 0, 9223372036854775807, 1)
  _19 = torch.select(_18, 1, 0)

can it input 640*480 image?

I can run it with a 320240 image,but when i use 640480, it failed. how can i get a 640*480 model?

#error C++14 or later compatible compiler is required to use ATen

Dear colleagues, please tell, how to solve this problem?
...
-- Build files have been written to: /home/gleb/GCNv2_SLAM/build
[ 4%] Building CXX object CMakeFiles/ORB_SLAM2.dir/src/System.cc.o
[ 8%] Building CXX object CMakeFiles/ORB_SLAM2.dir/src/Tracking.cc.o
[ 13%] Building CXX object CMakeFiles/ORB_SLAM2.dir/src/LocalMapping.cc.o
[ 17%] Building CXX object CMakeFiles/ORB_SLAM2.dir/src/LoopClosing.cc.o
In file included from /home/gleb/pytorch/torch/include/torch/csrc/api/include/torch/types.h:3:0,
from /home/gleb/pytorch/torch/include/torch/script.h:3,
from /home/gleb/GCNv2_SLAM/include/GCNextractor.h:24,
from /home/gleb/GCNv2_SLAM/include/Frame.h:32,
from /home/gleb/GCNv2_SLAM/include/MapPoint.h:25,
from /home/gleb/GCNv2_SLAM/include/KeyFrame.h:24,
from /home/gleb/GCNv2_SLAM/include/LocalMapping.h:24,
from /home/gleb/GCNv2_SLAM/src/LocalMapping.cc:21:
/home/gleb/pytorch/torch/include/ATen/ATen.h:4:2: error: #error C++14 or later compatible compiler is required to use ATen.
#error C++14 or later compatible compiler is required to use ATen.
^~~~~
In file included from /home/gleb/pytorch/torch/include/torch/csrc/api/include/torch/types.h:3:0,
from /home/gleb/pytorch/torch/include/torch/script.h:3,
from /home/gleb/GCNv2_SLAM/include/GCNextractor.h:24,
from /home/gleb/GCNv2_SLAM/include/Frame.h:32,
from /home/gleb/GCNv2_SLAM/include/MapPoint.h:25,
from /home/gleb/GCNv2_SLAM/include/KeyFrame.h:24,
from /home/gleb/GCNv2_SLAM/include/LoopClosing.h:24,
from /home/gleb/GCNv2_SLAM/src/LoopClosing.cc:21:
/home/gleb/pytorch/torch/include/ATen/ATen.h:4:2: error: #error C++14 or later compatible compiler is required to use ATen.
#error C++14 or later compatible compiler is required to use ATen.
^~~~~
In file included from /home/gleb/pytorch/torch/include/c10/util/ArrayRef.h:19:0,
from /home/gleb/pytorch/torch/include/c10/core/MemoryFormat.h:5,
from /home/gleb/pytorch/torch/include/ATen/core/TensorBody.h:5,
from /home/gleb/pytorch/torch/include/ATen/Tensor.h:3,
from /home/gleb/pytorch/torch/include/ATen/Context.h:4,
from /home/gleb/pytorch/torch/include/ATen/ATen.h:9,
from /home/gleb/pytorch/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/gleb/pytorch/torch/include/torch/script.h:3,
from /home/gleb/GCNv2_SLAM/include/GCNextractor.h:24,
from /home/gleb/GCNv2_SLAM/include/Frame.h:32,
from /home/gleb/GCNv2_SLAM/include/MapPoint.h:25,
from /home/gleb/GCNv2_SLAM/include/KeyFrame.h:24,
from /home/gleb/GCNv2_SLAM/include/LocalMapping.h:24,
from /home/gleb/GCNv2_SLAM/src/LocalMapping.cc:21:
/home/gleb/pytorch/torch/include/c10/util/C++17.h:24:2: error: #error You need C++14 to compile PyTorch
#error You need C++14 to compile PyTorch
^~~~~
In file included from /home/gleb/pytorch/torch/include/c10/util/ArrayRef.h:19:0,
from /home/gleb/pytorch/torch/include/c10/core/MemoryFormat.h:5,
from /home/gleb/pytorch/torch/include/ATen/core/TensorBody.h:5,
from /home/gleb/pytorch/torch/include/ATen/Tensor.h:3,
from /home/gleb/pytorch/torch/include/ATen/Context.h:4,
from /home/gleb/pytorch/torch/include/ATen/ATen.h:9,
from /home/gleb/pytorch/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/gleb/pytorch/torch/include/torch/script.h:3,
from /home/gleb/GCNv2_SLAM/include/GCNextractor.h:24,
from /home/gleb/GCNv2_SLAM/include/Frame.h:32,
from /home/gleb/GCNv2_SLAM/include/MapPoint.h:25,
from /home/gleb/GCNv2_SLAM/include/KeyFrame.h:24,
from /home/gleb/GCNv2_SLAM/include/LoopClosing.h:24,
from /home/gleb/GCNv2_SLAM/src/LoopClosing.cc:21:
/home/gleb/pytorch/torch/include/c10/util/C++17.h:24:2: error: #error You need C++14 to compile PyTorch
#error You need C++14 to compile PyTorch
^~~~~
In file included from /home/gleb/pytorch/torch/include/torch/csrc/api/include/torch/types.h:3:0,
from /home/gleb/pytorch/torch/include/torch/script.h:3,
from /home/gleb/GCNv2_SLAM/include/GCNextractor.h:24,
from /home/gleb/GCNv2_SLAM/include/Frame.h:32,
from /home/gleb/GCNv2_SLAM/include/KeyFrame.h:29,
from /home/gleb/GCNv2_SLAM/include/MapPoint.h:24,
from /home/gleb/GCNv2_SLAM/include/FrameDrawer.h:25,
from /home/gleb/GCNv2_SLAM/include/Viewer.h:25,
from /home/gleb/GCNv2_SLAM/include/Tracking.h:28,
from /home/gleb/GCNv2_SLAM/src/Tracking.cc:22:
/home/gleb/pytorch/torch/include/ATen/ATen.h:4:2: error: #error C++14 or later compatible compiler is required to use ATen.
#error C++14 or later compatible compiler is required to use ATen.
^~~~~
In file included from /home/gleb/pytorch/torch/include/torch/csrc/api/include/torch/types.h:3:0,
from /home/gleb/pytorch/torch/include/torch/script.h:3,
from /home/gleb/GCNv2_SLAM/include/GCNextractor.h:24,
from /home/gleb/GCNv2_SLAM/include/Frame.h:32,
from /home/gleb/GCNv2_SLAM/include/KeyFrame.h:29,
from /home/gleb/GCNv2_SLAM/include/MapPoint.h:24,
from /home/gleb/GCNv2_SLAM/include/FrameDrawer.h:25,
from /home/gleb/GCNv2_SLAM/include/Viewer.h:25,
from /home/gleb/GCNv2_SLAM/include/Tracking.h:28,
from /home/gleb/GCNv2_SLAM/include/System.h:30,
from /home/gleb/GCNv2_SLAM/src/System.cc:23:
/home/gleb/pytorch/torch/include/ATen/ATen.h:4:2: error: #error C++14 or later compatible compiler is required to use ATen.
#error C++14 or later compatible compiler is required to use ATen.
^~~~~
In file included from /home/gleb/pytorch/torch/include/c10/util/ArrayRef.h:19:0,
from /home/gleb/pytorch/torch/include/c10/core/MemoryFormat.h:5,
from /home/gleb/pytorch/torch/include/ATen/core/TensorBody.h:5,
from /home/gleb/pytorch/torch/include/ATen/Tensor.h:3,
from /home/gleb/pytorch/torch/include/ATen/Context.h:4,
from /home/gleb/pytorch/torch/include/ATen/ATen.h:9,
from /home/gleb/pytorch/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/gleb/pytorch/torch/include/torch/script.h:3,
from /home/gleb/GCNv2_SLAM/include/GCNextractor.h:24,
from /home/gleb/GCNv2_SLAM/include/Frame.h:32,
from /home/gleb/GCNv2_SLAM/include/KeyFrame.h:29,
from /home/gleb/GCNv2_SLAM/include/MapPoint.h:24,
from /home/gleb/GCNv2_SLAM/include/FrameDrawer.h:25,
from /home/gleb/GCNv2_SLAM/include/Viewer.h:25,
from /home/gleb/GCNv2_SLAM/include/Tracking.h:28,
from /home/gleb/GCNv2_SLAM/src/Tracking.cc:22:
/home/gleb/pytorch/torch/include/c10/util/C++17.h:24:2: error: #error You need C++14 to compile PyTorch
#error You need C++14 to compile PyTorch
^~~~~
In file included from /home/gleb/pytorch/torch/include/c10/util/ArrayRef.h:19:0,
from /home/gleb/pytorch/torch/include/c10/core/MemoryFormat.h:5,
from /home/gleb/pytorch/torch/include/ATen/core/TensorBody.h:5,
from /home/gleb/pytorch/torch/include/ATen/Tensor.h:3,
from /home/gleb/pytorch/torch/include/ATen/Context.h:4,
from /home/gleb/pytorch/torch/include/ATen/ATen.h:9,
from /home/gleb/pytorch/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/gleb/pytorch/torch/include/torch/script.h:3,
from /home/gleb/GCNv2_SLAM/include/GCNextractor.h:24,
from /home/gleb/GCNv2_SLAM/include/Frame.h:32,
from /home/gleb/GCNv2_SLAM/include/KeyFrame.h:29,
from /home/gleb/GCNv2_SLAM/include/MapPoint.h:24,
from /home/gleb/GCNv2_SLAM/include/FrameDrawer.h:25,
from /home/gleb/GCNv2_SLAM/include/Viewer.h:25,
from /home/gleb/GCNv2_SLAM/include/Tracking.h:28,
from /home/gleb/GCNv2_SLAM/include/System.h:30,
from /home/gleb/GCNv2_SLAM/src/System.cc:23:
/home/gleb/pytorch/torch/include/c10/util/C++17.h:24:2: error: #error You need C++14 to compile PyTorch
#error You need C++14 to compile PyTorch
^~~~~
In file included from /home/gleb/pytorch/torch/include/torch/csrc/api/include/torch/torch.h:3:0,
from /home/gleb/GCNv2_SLAM/include/GCNextractor.h:25,
from /home/gleb/GCNv2_SLAM/include/Frame.h:32,
from /home/gleb/GCNv2_SLAM/include/MapPoint.h:25,
from /home/gleb/GCNv2_SLAM/include/KeyFrame.h:24,
from /home/gleb/GCNv2_SLAM/include/LoopClosing.h:24,
from /home/gleb/GCNv2_SLAM/src/LoopClosing.cc:21:
/home/gleb/pytorch/torch/include/torch/csrc/api/include/torch/all.h:4:2: error: #error C++14 or later compatible compiler is required to use PyTorch.
#error C++14 or later compatible compiler is required to use PyTorch.
^~~~~
In file included from /home/gleb/pytorch/torch/include/torch/csrc/api/include/torch/torch.h:3:0,
from /home/gleb/GCNv2_SLAM/include/GCNextractor.h:25,
from /home/gleb/GCNv2_SLAM/include/Frame.h:32,
from /home/gleb/GCNv2_SLAM/include/MapPoint.h:25,
from /home/gleb/GCNv2_SLAM/include/KeyFrame.h:24,
from /home/gleb/GCNv2_SLAM/include/LocalMapping.h:24,
from /home/gleb/GCNv2_SLAM/src/LocalMapping.cc:21:
/home/gleb/pytorch/torch/include/torch/csrc/api/include/torch/all.h:4:2: error: #error C++14 or later compatible compiler is required to use PyTorch.
#error C++14 or later compatible compiler is required to use PyTorch.
^~~~~
In file included from /home/gleb/pytorch/torch/include/torch/csrc/api/include/torch/torch.h:3:0,
from /home/gleb/GCNv2_SLAM/include/GCNextractor.h:25,
from /home/gleb/GCNv2_SLAM/include/Frame.h:32,
from /home/gleb/GCNv2_SLAM/include/KeyFrame.h:29,
from /home/gleb/GCNv2_SLAM/include/MapPoint.h:24,
from /home/gleb/GCNv2_SLAM/include/FrameDrawer.h:25,
from /home/gleb/GCNv2_SLAM/include/Viewer.h:25,
from /home/gleb/GCNv2_SLAM/include/Tracking.h:28,
from /home/gleb/GCNv2_SLAM/src/Tracking.cc:22:
/home/gleb/pytorch/torch/include/torch/csrc/api/include/torch/all.h:4:2: error: #error C++14 or later compatible compiler is required to use PyTorch.
#error C++14 or later compatible compiler is required to use PyTorch.
^~~~~
In file included from /home/gleb/pytorch/torch/include/torch/csrc/api/include/torch/torch.h:3:0,
from /home/gleb/GCNv2_SLAM/include/GCNextractor.h:25,
from /home/gleb/GCNv2_SLAM/include/Frame.h:32,
from /home/gleb/GCNv2_SLAM/include/KeyFrame.h:29,
from /home/gleb/GCNv2_SLAM/include/MapPoint.h:24,
from /home/gleb/GCNv2_SLAM/include/FrameDrawer.h:25,
from /home/gleb/GCNv2_SLAM/include/Viewer.h:25,
from /home/gleb/GCNv2_SLAM/include/Tracking.h:28,
from /home/gleb/GCNv2_SLAM/include/System.h:30,
from /home/gleb/GCNv2_SLAM/src/System.cc:23:
/home/gleb/pytorch/torch/include/torch/csrc/api/include/torch/all.h:4:2: error: #error C++14 or later compatible compiler is required to use PyTorch.
#error C++14 or later compatible compiler is required to use PyTorch.
^~~~~
In file included from /home/gleb/pytorch/torch/include/c10/util/Exception.h:5:0,
from /home/gleb/pytorch/torch/include/c10/core/Device.h:5,
from /home/gleb/pytorch/torch/include/c10/core/Allocator.h:6,
from /home/gleb/pytorch/torch/include/ATen/ATen.h:7,
from /home/gleb/pytorch/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/gleb/pytorch/torch/include/torch/script.h:3,
from /home/gleb/GCNv2_SLAM/include/GCNextractor.h:24,
from /home/gleb/GCNv2_SLAM/include/Frame.h:32,
from /home/gleb/GCNv2_SLAM/include/MapPoint.h:25,
from /home/gleb/GCNv2_SLAM/include/KeyFrame.h:24,
from /home/gleb/GCNv2_SLAM/include/LoopClosing.h:24,
from /home/gleb/GCNv2_SLAM/src/LoopClosing.cc:21:

...

GCNv2 hanging after a couple of seconds running.

GCNv2 look promising when looking at the videos online. However when trying to run it on a quite strong pc it freezes after 3~5 seconds. It sometimes even hard locks the pc. The setup is:
PC:

• i9 7920x
• GTX 1080ti
• 32gb ram
  Host os is:
• Ubuntu 18.04
• NVIDIA 440 driver
• Cuda 9.1.85
  For easy distribution we running everything in docker containers.
  Docker:
• Ubuntu 18.04
• Nvidia docker with gpu passthrough
• Cuda 9.2.148
• Pytorch version 1.0.1

Are there any recommendations for what version to run cuda etc?

run error

launchx-500-10802@launchx50010802:~/github/GCNv2_SLAM/GCN2$ ./rgbd_gcn ../Vocabulary/GCNvoc.bin TUM3.yaml /home/launchx-500-10802/github/rgbd_dataset_freiburg3_long_office_household_validation /home/launchx-500-10802/github/rgbd_dataset_freiburg3_long_office_household_validation/associations.txt
./rgbd_gcn: /home/launchx-500-10802/anaconda3/lib/libgomp.so.1: version GOMP_4.0' not found (required by /home/launchx-500-10802/github/pytorch/torch/lib/libcaffe2.so) ./rgbd_gcn: /home/launchx-500-10802/anaconda3/lib/libgomp.so.1: version GOMP_4.0' not found (required by /opt/OpenBLAS/lib/libopenblas.so.0)
./rgbd_gcn: /home/launchx-500-10802/anaconda3/lib/libgomp.so.1: version `GOMP_4.0' not found (required by /usr/lib/x86_64-linux-gnu/libsoxr.so.0)

Use Monocular

Hello! Thank you very much for sharing your source code.
I want to know what to watch out for when running programs with a monocular dataset.
Thank you!

build.sh error

/home/launchx-500-10802/github/GCNv2_SLAM/src/ORBmatcher.cc:32:34: fatal error: opencv2/features2d.hpp: 没有那个文件或目录
compilation terminated.
CMakeFiles/ORB_SLAM2.dir/build.make:206: recipe for target 'CMakeFiles/ORB_SLAM2.dir/src/ORBmatcher.cc.o' failed
make[2]: *** [CMakeFiles/ORB_SLAM2.dir/src/ORBmatcher.cc.o] Error 1
make[2]: *** 正在等待未完成的任务....
/home/launchx-500-10802/github/GCNv2_SLAM/src/GCNextractor.cc: In function ‘void ORB_SLAM2::nms(cv::Mat, cv::Mat, std::vectorcv::KeyPoint&, cv::Mat&, int, int, int, int)’:
/home/launchx-500-10802/github/GCNv2_SLAM/src/GCNextractor.cc:96:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (int i = 0; i < pts_raw.size(); i++)
^
/home/launchx-500-10802/github/GCNv2_SLAM/src/GCNextractor.cc:107:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (int i = 0; i < pts_raw.size(); i++)
^
/home/launchx-500-10802/github/GCNv2_SLAM/src/GCNextractor.cc:148:20: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (int i=0; i<select_indice.size(); i++)
^
CMakeFiles/Makefile2:104: recipe for target 'CMakeFiles/ORB_SLAM2.dir/all' failed
make[1]: *** [CMakeFiles/ORB_SLAM2.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2