bertabescos/dynaslam

DynaSLAM is a SLAM system robust in dynamic environments for monocular, stereo and RGB-D setups

DynaSLAM

[Project] [arXiv] [Journal]

DynaSLAM is a visual SLAM system that is robust in dynamic scenarios for monocular, stereo and RGB-D configurations. Having a static map of the scene allows inpainting the frame background that has been occluded by such dynamic objects.

DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes
Berta Bescos, José M. Fácil, Javier Civera and José Neira
RA-L and IROS, 2018

We provide examples to run the SLAM system in the TUM dataset as RGB-D or monocular, and in the KITTI dataset as stereo or monocular.

News

• DynaSLAM supports now both OpenCV 2.X and OpenCV 3.X.

Getting Started

• Install ORB-SLAM2 prerequisites: C++11 or C++0x Compiler, Pangolin, OpenCV and Eigen3 (https://github.com/raulmur/ORB_SLAM2).
• Install boost libraries with the command sudo apt-get install libboost-all-dev.
• Install python 2.7, keras and tensorflow, and download the mask_rcnn_coco.h5 model from this GitHub repository: https://github.com/matterport/Mask_RCNN/releases.
• Clone this repo:

git clone https://github.com/BertaBescos/DynaSLAM.git
cd DynaSLAM

cd DynaSLAM
chmod +x build.sh
./build.sh

• Place the mask_rcnn_coco.h5 model in the folder DynaSLAM/src/python/.

RGB-D Example on TUM Dataset

• Download a sequence from http://vision.in.tum.de/data/datasets/rgbd-dataset/download and uncompress it.

• Associate RGB images and depth images executing the python script associate.py:

  python associate.py PATH_TO_SEQUENCE/rgb.txt PATH_TO_SEQUENCE/depth.txt > associations.txt
  

These associations files are given in the folder ./Examples/RGB-D/associations/ for the TUM dynamic sequences.

• Execute the following command. Change TUMX.yaml to TUM1.yaml,TUM2.yaml or TUM3.yaml for freiburg1, freiburg2 and freiburg3 sequences respectively. Change PATH_TO_SEQUENCE_FOLDER to the uncompressed sequence folder. Change ASSOCIATIONS_FILE to the path to the corresponding associations file. PATH_TO_MASKS and PATH_TO_OUTPUT are optional parameters.

  ./Examples/RGB-D/rgbd_tum Vocabulary/ORBvoc.txt Examples/RGB-D/TUMX.yaml PATH_TO_SEQUENCE_FOLDER ASSOCIATIONS_FILE (PATH_TO_MASKS) (PATH_TO_OUTPUT)
  

If PATH_TO_MASKS and PATH_TO_OUTPUT are not provided, only the geometrical approach is used to detect dynamic objects.

If PATH_TO_MASKS is provided, Mask R-CNN is used to segment the potential dynamic content of every frame. These masks are saved in the provided folder PATH_TO_MASKS. If this argument is no_save, the masks are used but not saved. If it finds the Mask R-CNN computed dynamic masks in PATH_TO_MASKS, it uses them but does not compute them again.

If PATH_TO_OUTPUT is provided, the inpainted frames are computed and saved in PATH_TO_OUTPUT.

Stereo Example on KITTI Dataset

• Download the dataset (grayscale images) from http://www.cvlibs.net/datasets/kitti/eval_odometry.php

• Execute the following command. Change KITTIX.yamlto KITTI00-02.yaml, KITTI03.yaml or KITTI04-12.yaml for sequence 0 to 2, 3, and 4 to 12 respectively. Change PATH_TO_DATASET_FOLDER to the uncompressed dataset folder. Change SEQUENCE_NUMBER to 00, 01, 02,.., 11. By providing the last argument PATH_TO_MASKS, dynamic objects are detected with Mask R-CNN.

./Examples/Stereo/stereo_kitti Vocabulary/ORBvoc.txt Examples/Stereo/KITTIX.yaml PATH_TO_DATASET_FOLDER/dataset/sequences/SEQUENCE_NUMBER (PATH_TO_MASKS)

Monocular Example on TUM Dataset

• Download a sequence from http://vision.in.tum.de/data/datasets/rgbd-dataset/download and uncompress it.

• Execute the following command. Change TUMX.yaml to TUM1.yaml,TUM2.yaml or TUM3.yaml for freiburg1, freiburg2 and freiburg3 sequences respectively. Change PATH_TO_SEQUENCE_FOLDERto the uncompressed sequence folder. By providing the last argument PATH_TO_MASKS, dynamic objects are detected with Mask R-CNN.

./Examples/Monocular/mono_tum Vocabulary/ORBvoc.txt Examples/Monocular/TUMX.yaml PATH_TO_SEQUENCE_FOLDER (PATH_TO_MASKS)

Monocular Example on KITTI Dataset

• Download the dataset (grayscale images) from http://www.cvlibs.net/datasets/kitti/eval_odometry.php

• Execute the following command. Change KITTIX.yamlby KITTI00-02.yaml, KITTI03.yaml or KITTI04-12.yaml for sequence 0 to 2, 3, and 4 to 12 respectively. Change PATH_TO_DATASET_FOLDER to the uncompressed dataset folder. Change SEQUENCE_NUMBER to 00, 01, 02,.., 11. By providing the last argument PATH_TO_MASKS, dynamic objects are detected with Mask R-CNN.

./Examples/Monocular/mono_kitti Vocabulary/ORBvoc.txt Examples/Monocular/KITTIX.yaml PATH_TO_DATASET_FOLDER/dataset/sequences/SEQUENCE_NUMBER (PATH_TO_MASKS)

Citation

If you use DynaSLAM in an academic work, please cite:

@article{bescos2018dynaslam,
  title={{DynaSLAM}: Tracking, Mapping and Inpainting in Dynamic Environments},
  author={Bescos, Berta, F\'acil, JM., Civera, Javier and Neira, Jos\'e},
  journal={IEEE RA-L},
  year={2018}
 }

Acknowledgements

Our code builds on ORB-SLAM2.

DynaSLAM

dynaslam's Issues

OpenCV Error: Assertion failed (a_size.width == len) in gemm

I've installed the environment of DynaSlam on two servers. Because not only me use the servers, I use docker and anaconda separately on the two servers to prepare the environment. I've check the maskrcnn's environment and compiled the DynaSlam code successfully both on the two. However, when I ran the rgbd_tum by same command, the one on docker successfully completed, while the one using anaconda(install the python packages by conda, install the c++ dependence by apt-get or compile the source code) got error:
OpenCV Error: Assertion failed (a_size.width == len) in gemm, file /home/tensorflow03/mxw_slam/opencv-2.4.11/modules/core/src/matmul.cpp, line 728 terminate called after throwing an instance of 'cv::Exception' what(): /home/tensorflow03/mxw_slam/opencv-2.4.11/modules/core/src/matmul.cpp:728: error: (-215) a_size.width == len in function gemm Aborted (core dumped)

The data is Tum-RGBD, and I've tried the rgbd_dataset_freiburg3_walking_static and rgbd_dataset_freiburg3_walking_xyz. Let the program print the count of passing image to locate the error. In sequence rgbd_dataset_freiburg3_walking_xyz, error appeared at 775th/828(sum of images), while in rgbd_dataset_freiburg3_walking_static appeared at 599th/718.
I've set the opencv directory in CMakeLists.txt, and the error seems show it was using opencv-2.4.11. Why the same code shows different results? How should I fix it?

Importing Mask R-CNN Settings...Segmentation fault (core dumped)

Sorry to bothering you.
When I run the DynaSLAM with the command "./Examples/RGB-D/rgbd_tum Vocabulary/ORBvoc.txt Examples/RGB-D/TUM3.yaml Data/rgbd_dataset_freiburg3_walking_xyz/ Data/rgbd_dataset_freiburg3_walking_xyz/associate.txt Data/Mask/ Data/Output/"

My terminal's output is :
nImages: 827
Loading Mask R-CNN. This could take a while...
Importing Mask R-CNN Settings...
Segmentation fault (core dumped)

And I find the interrupt occurs at the MaskNet.cc "this->py_class = PyObject_GetAttrString(this->py_module, this->class_name.c_str());"
And I also find after the "this->py_module = PyImport_ImportModule(this->module_name.c_str());" been called, the py_module is NULL.

Could you help me to solve the problem?

Odometry Only Output

Is it possible to run your code with odometry function only (without full SLAM system, e.g. by disabling loop-closure)?

About the speed of segmenting image using Mask-rcnn

Hello! This work is amazing! I want to measure the speed of image semantic segmentation. I use the chrono to measure the speed in the rgbd_tum.cc as follows:

std::chrono::steady_clock::time_point t3 = std::chrono::steady_clock::now();
maskRCNN=MaskNet->GetSegmentation(imRGB,string(argv[5]),vstrImageFilenamesRGB[ni].replace(0,4,""));
std::chrono::steady_clock::time_point t4 = std::chrono::steady_clock::now();
double seg_time= std::chrono::duration_cast<std::chrono::duration<double> >(t4 - t3).count();

It seems that the time of segmentation is about 1ms for each frame. However, I also measure the time of segmentation in the MaskRCNN.py as follows:

tic = time.time()
results = self.model.detect([image], verbose=0)
seg_time = time.time() - tic
print (seg_time)

However, the seg_time only shows once and is 1.23s. Is the GetDynSeg(self, image, image2=None) function in this python file only called once? How long does it take to segment an image? Thank you very much!

The evaluation on the TUM dataset is not as good as ORB_SLAM, Why?

I use the EVO tools to evalute the trajectory, but it shows that the trajectory of ORB-SLAM is better, why?

make build.sh error

Thanks for your work! I try to run the build.sh ,but I meet a problems,could you help me?

error: cannot declare variable ‘DynaSLAM::g_numpyAllocator’ to be of abstract type ‘DynaSLAM::NumpyAllocator’ NumpyAllocator g_numpyAllocator;

DynaSLAM build error

Hello everyone,
I compiled everything just like mentioned steps in readme.md. but I've got build error according below:

[ 62%] Building CXX object CMakeFiles/DynaSLAM.dir/src/MapDrawer.cc.o
[ 62%] Building CXX object CMakeFiles/DynaSLAM.dir/src/KeyFrameDatabase.cc.o
[ 65%] Building CXX object CMakeFiles/DynaSLAM.dir/src/PnPsolver.cc.o
[ 68%] Building CXX object CMakeFiles/DynaSLAM.dir/src/Geometry.cc.o
In file included from /usr/lib/python2.7/dist-packages/numpy/core/include/numpy/ndarraytypes.h:1777:0,
from /usr/lib/python2.7/dist-packages/numpy/core/include/numpy/ndarrayobject.h:18,
from /home/navid/DynaSLAM/include/Conversion.h:17,
from /home/navid/DynaSLAM/src/Conversion.cc:9:
/usr/lib/python2.7/dist-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#warning "Using deprecated NumPy API, disable it by "
^
In file included from /usr/lib/python2.7/dist-packages/numpy/core/include/numpy/ndarraytypes.h:1777:0,
from /usr/lib/python2.7/dist-packages/numpy/core/include/numpy/ndarrayobject.h:18,
from /home/navid/DynaSLAM/include/Conversion.h:17,
from /home/navid/DynaSLAM/include/MaskNet.h:26,
from /home/navid/DynaSLAM/src/MaskNet.cc:9:
/usr/lib/python2.7/dist-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#warning "Using deprecated NumPy API, disable it by "
^
/home/navid/DynaSLAM/src/Conversion.cc: In member function ‘cv::UMatData* DynaSLAM::NumpyAllocator::allocate(int, const int*, int, void*, size_t*, int, cv::UMatUsageFlags) const’:
/home/navid/DynaSLAM/src/Conversion.cc:158:25: error: ‘CV_MAX_DIM’ was not declared in this scope
npy_intp _sizes[CV_MAX_DIM+1];
^
/home/navid/DynaSLAM/src/Conversion.cc:161:13: error: ‘_sizes’ was not declared in this scope
_sizes[i] = sizes[i];
^
/home/navid/DynaSLAM/src/Conversion.cc:166:13: error: ‘_sizes’ was not declared in this scope
_sizes[dims++] = cn;
^
In file included from /home/navid/DynaSLAM/include/Conversion.h:17:0,
from /home/navid/DynaSLAM/src/Conversion.cc:9:
/home/navid/DynaSLAM/src/Conversion.cc:168:47: error: ‘_sizes’ was not declared in this scope
PyObject* o = PyArray_SimpleNew(dims, sizes, typenum);
^
/usr/lib/python2.7/dist-packages/numpy/core/include/numpy/ndarrayobject.h:135:40: note: in definition of macro ‘PyArray_SimpleNew’
PyArray_New(&PyArray_Type, nd, dims, typenum, NULL, NULL, 0, 0, NULL)
^
In file included from /usr/local/include/opencv4/opencv2/core.hpp:54:0,
from /usr/local/include/opencv4/opencv2/imgproc.hpp:46,
from /usr/local/include/opencv4/opencv2/imgproc/imgproc.hpp:48,
from /home/navid/DynaSLAM/include/Conversion.h:14,
from /home/navid/DynaSLAM/src/Conversion.cc:9:
/home/navid/DynaSLAM/src/Conversion.cc:172:23: error: ‘CV_StsError’ was not declared in this scope
CV_Error(CV_StsError, ("The numpy array of typenum=%d, ndims=%d can not be created", typenum, dims));
^
/home/navid/DynaSLAM/src/Conversion.cc: At global scope:
/home/navid/DynaSLAM/src/Conversion.cc:205:16: error: cannot declare variable ‘DynaSLAM::g_numpyAllocator’ to be of abstract type ‘DynaSLAM::NumpyAllocator’
NumpyAllocator g_numpyAllocator;
^
/home/navid/DynaSLAM/src/Conversion.cc:74:7: note: because the following virtual functions are pure within ‘DynaSLAM::NumpyAllocator’:
class NumpyAllocator : public MatAllocator
^
In file included from /usr/local/include/opencv4/opencv2/core.hpp:59:0,
from /usr/local/include/opencv4/opencv2/imgproc.hpp:46,
from /usr/local/include/opencv4/opencv2/imgproc/imgproc.hpp:48,
from /home/navid/DynaSLAM/include/Conversion.h:14,
from /home/navid/DynaSLAM/src/Conversion.cc:9:
/usr/local/include/opencv4/opencv2/core/mat.hpp:475:23: note: virtual cv::UMatData* cv::MatAllocator::allocate(int, const int*, int, void*, size_t*, cv::AccessFlag, cv::UMatUsageFlags) const
virtual UMatData* allocate(int dims, const int* sizes, int type,
^
/usr/local/include/opencv4/opencv2/core/mat.hpp:477:18: note: virtual bool cv::MatAllocator::allocate(cv::UMatData*, cv::AccessFlag, cv::UMatUsageFlags) const
virtual bool allocate(UMatData* data, AccessFlag accessflags, UMatUsageFlags usageFlags) const = 0;
^
/home/navid/DynaSLAM/src/Conversion.cc: In member function ‘cv::Mat DynaSLAM::NDArrayConverter::toMat(const PyObject*)’:
/home/navid/DynaSLAM/src/Conversion.cc:244:17: error: ‘CV_MAX_DIM’ was not declared in this scope
if(ndims >= CV_MAX_DIM)
^
/home/navid/DynaSLAM/src/Conversion.cc:249:14: error: ‘CV_MAX_DIM’ was not declared in this scope
int size[CV_MAX_DIM+1];
^
/home/navid/DynaSLAM/src/Conversion.cc:257:9: error: ‘size’ was not declared in this scope
size[i] = (int)_sizes[i];
^
/home/navid/DynaSLAM/src/Conversion.cc:258:9: error: ‘step’ was not declared in this scope
step[i] = (size_t)_strides[i];
^
/home/navid/DynaSLAM/src/Conversion.cc:261:23: error: ‘step’ was not declared in this scope
if( ndims == 0 || step[ndims-1] > elemsize ) {
^
/home/navid/DynaSLAM/src/Conversion.cc:262:9: error: ‘size’ was not declared in this scope
size[ndims] = 1;
^
/home/navid/DynaSLAM/src/Conversion.cc:267:23: error: ‘step’ was not declared in this scope
if( ndims >= 2 && step[0] < step[1] )
^
/home/navid/DynaSLAM/src/Conversion.cc:269:19: error: ‘size’ was not declared in this scope
std::swap(size[0], size[1]);
^
/home/navid/DynaSLAM/src/Conversion.cc:274:23: error: ‘size’ was not declared in this scope
if( ndims == 3 && size[2] <= CV_CN_MAX && step[1] == elemsizesize[2] )
^
/home/navid/DynaSLAM/src/Conversion.cc:274:47: error: ‘step’ was not declared in this scope
if( ndims == 3 && size[2] <= CV_CN_MAX && step[1] == elemsizesize[2] )
^
/home/navid/DynaSLAM/src/Conversion.cc:285:20: error: ‘size’ was not declared in this scope
m = Mat(ndims, size, type, PyArray_DATA(o), step);
^
/home/navid/DynaSLAM/src/Conversion.cc:285:49: error: ‘step’ was not declared in this scope
m = Mat(ndims, size, type, PyArray_DATA(o), step);
^
In file included from /home/navid/DynaSLAM/src/Conversion.cc:9:0:
/home/navid/DynaSLAM/include/Conversion.h: At global scope:
/home/navid/DynaSLAM/include/Conversion.h:25:18: warning: ‘DynaSLAM::opencv_error’ defined but not used [-Wunused-variable]
static PyObject* opencv_error = 0;
^
/home/navid/DynaSLAM/src/Conversion.cc:61:18: warning: ‘PyObject* DynaSLAM::failmsgp(const char*, ...)’ defined but not used [-Wunused-function]
static PyObject* failmsgp(const char fmt, ...)
^
/home/navid/DynaSLAM/src/Conversion.cc:16:13: warning: ‘void DynaSLAM::init()’ defined but not used [-Wunused-function]
static void init()
^
CMakeFiles/DynaSLAM.dir/build.make:518: recipe for target 'CMakeFiles/DynaSLAM.dir/src/Conversion.cc.o' failed
make[2]: *** [CMakeFiles/DynaSLAM.dir/src/Conversion.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
/home/navid/DynaSLAM/src/MaskNet.cc: In member function ‘cv::Mat DynaSLAM::SegmentDynObject::GetSegmentation(cv::Mat&, std::__cxx11::string, std::__cxx11::string)’:
/home/navid/DynaSLAM/src/MaskNet.cc:58:43: error: ‘CV_LOAD_IMAGE_UNCHANGED’ was not declared in this scope
cv::Mat seg = cv::imread(dir+"/"+name,CV_LOAD_IMAGE_UNCHANGED);
^
/home/navid/DynaSLAM/src/MaskNet.cc:62:125: warning: ISO C++ forbids converting a string constant to ‘char’ [-Wwrite-strings]
PyObject* py_mask_image = PyObject_CallMethod(this->net, const_cast<char*>(this->get_dyn_seg.c_str()),"(O)",py_image);
^
In file included from /usr/lib/python2.7/dist-packages/numpy/core/include/numpy/ndarrayobject.h:27:0,
from /home/navid/DynaSLAM/include/Conversion.h:17,
from /home/navid/DynaSLAM/include/MaskNet.h:26,
from /home/navid/DynaSLAM/src/MaskNet.cc:9:
/usr/lib/python2.7/dist-packages/numpy/core/include/numpy/__multiarray_api.h: At global scope:
/usr/lib/python2.7/dist-packages/numpy/core/include/numpy/__multiarray_api.h:1448:1: warning: ‘int _import_array()’ defined but not used [-Wunused-function]
_import_array(void)
^
In file included from /home/navid/DynaSLAM/include/MaskNet.h:26:0,
from /home/navid/DynaSLAM/src/MaskNet.cc:9:
/home/navid/DynaSLAM/include/Conversion.h:25:18: warning: ‘DynaSLAM::opencv_error’ defined but not used [-Wunused-variable]
static PyObject* opencv_error = 0;
^
/home/navid/DynaSLAM/include/Conversion.h:27:12: warning: ‘int DynaSLAM::failmsg(const char*, ...)’ declared ‘static’ but never defined [-Wunused-function]
static int failmsg(const char fmt, ...);
^
/home/navid/DynaSLAM/include/Conversion.h:45:18: warning: ‘PyObject DynaSLAM::failmsgp(const char*, ...)’ declared ‘static’ but never defined [-Wunused-function]
static PyObject* failmsgp(const char fmt, ...);
^
CMakeFiles/DynaSLAM.dir/build.make:542: recipe for target 'CMakeFiles/DynaSLAM.dir/src/MaskNet.cc.o' failed
make[2]: *** [CMakeFiles/DynaSLAM.dir/src/MaskNet.cc.o] Error 1
In file included from /home/navid/DynaSLAM/src/Frame.cc:10:0:
/home/navid/DynaSLAM/include/Frame.h: In copy constructor ‘ORB_SLAM2::Frame::Frame(const ORB_SLAM2::Frame&)’:
/home/navid/DynaSLAM/include/Frame.h:173:9: warning: ‘ORB_SLAM2::Frame::mnScaleLevels’ will be initialized after [-Wreorder]
int mnScaleLevels;
^
/home/navid/DynaSLAM/include/Frame.h:131:13: warning: ‘cv::Mat ORB_SLAM2::Frame::mImRGB’ [-Wreorder]
cv::Mat mImRGB;
^
/home/navid/DynaSLAM/src/Frame.cc:28:1: warning: when initialized here [-Wreorder]
Frame::Frame(const Frame &frame)
^
In file included from /home/navid/DynaSLAM/src/Frame.cc:10:0:
/home/navid/DynaSLAM/include/Frame.h:175:11: warning: ‘ORB_SLAM2::Frame::mfLogScaleFactor’ will be initialized after [-Wreorder]
float mfLogScaleFactor;
^
/home/navid/DynaSLAM/include/Frame.h:130:13: warning: ‘cv::Mat ORB_SLAM2::Frame::mImGray’ [-Wreorder]
cv::Mat mImGray;
^
/home/navid/DynaSLAM/src/Frame.cc:28:1: warning: when initialized here [-Wreorder]
Frame::Frame(const Frame &frame)
^
In file included from /home/navid/DynaSLAM/src/Frame.cc:10:0:
/home/navid/DynaSLAM/include/Frame.h:177:19: warning: ‘ORB_SLAM2::Frame::mvInvScaleFactors’ will be initialized after [-Wreorder]
vector mvInvScaleFactors;
^
/home/navid/DynaSLAM/include/Frame.h:129:13: warning: ‘cv::Mat ORB_SLAM2::Frame::mImMask’ [-Wreorder]
cv::Mat mImMask;
^
/home/navid/DynaSLAM/src/Frame.cc:28:1: warning: when initialized here [-Wreorder]
Frame::Frame(const Frame &frame)
^
In file included from /home/navid/DynaSLAM/src/Frame.cc:10:0:
/home/navid/DynaSLAM/include/Frame.h:179:19: warning: ‘ORB_SLAM2::Frame::mvInvLevelSigma2’ will be initialized after [-Wreorder]
vector mvInvLevelSigma2;
^
/home/navid/DynaSLAM/include/Frame.h:128:10: warning: ‘bool ORB_SLAM2::Frame::mIsKeyFrame’ [-Wreorder]
bool mIsKeyFrame;
^
/home/navid/DynaSLAM/src/Frame.cc:28:1: warning: when initialized here [-Wreorder]
Frame::Frame(const Frame &frame)
^
In file included from /home/navid/DynaSLAM/src/Frame.cc:10:0:
/home/navid/DynaSLAM/include/Frame.h:128:10: warning: ‘ORB_SLAM2::Frame::mIsKeyFrame’ will be initialized after [-Wreorder]
bool mIsKeyFrame;
^
/home/navid/DynaSLAM/include/Frame.h:127:13: warning: ‘cv::Mat ORB_SLAM2::Frame::mImDepth’ [-Wreorder]
cv::Mat mImDepth;
^
/home/navid/DynaSLAM/src/Frame.cc:28:1: warning: when initialized here [-Wreorder]
Frame::Frame(const Frame &frame)
^
In file included from /home/navid/DynaSLAM/src/Frame.cc:10:0:
/home/navid/DynaSLAM/include/Frame.h: In constructor ‘ORB_SLAM2::Frame::Frame(const cv::Mat&, const cv::Mat&, const cv::Mat&, const double&, ORB_SLAM2::ORBextractor, ORB_SLAM2::ORBVocabulary*, cv::Mat&, cv::Mat&, const float&, const float&)’:
/home/navid/DynaSLAM/include/Frame.h:129:13: warning: ‘ORB_SLAM2::Frame::mImMask’ will be initialized after [-Wreorder]
cv::Mat mImMask;
^
/home/navid/DynaSLAM/include/Frame.h:95:20: warning: ‘ORB_SLAM2::ORBVocabulary* ORB_SLAM2::Frame::mpORBvocabulary’ [-Wreorder]
ORBVocabulary* mpORBvocabulary;
^
/home/navid/DynaSLAM/src/Frame.cc:154:1: warning: when initialized here [-Wreorder]
Frame::Frame(const cv::Mat &imGray, const cv::Mat &imDepth, const cv::Mat &imMask, const double &timeStamp, ORBextractor* extractor, ORBVocabulary* voc,
^
In file included from /home/navid/DynaSLAM/src/Frame.cc:10:0:
/home/navid/DynaSLAM/include/Frame.h:130:13: warning: ‘ORB_SLAM2::Frame::mImGray’ will be initialized after [-Wreorder]
cv::Mat mImGray;
^
/home/navid/DynaSLAM/include/Frame.h:101:12: warning: ‘double ORB_SLAM2::Frame::mTimeStamp’ [-Wreorder]
double mTimeStamp;
^
/home/navid/DynaSLAM/src/Frame.cc:154:1: warning: when initialized here [-Wreorder]
Frame::Frame(const cv::Mat &imGray, const cv::Mat &imDepth, const cv::Mat &imMask, const double &timeStamp, ORBextractor* extractor, ORBVocabulary* voc,
^
In file included from /home/navid/DynaSLAM/src/Frame.cc:10:0:
/home/navid/DynaSLAM/include/Frame.h:128:10: warning: ‘ORB_SLAM2::Frame::mIsKeyFrame’ will be initialized after [-Wreorder]
bool mIsKeyFrame;
^
/home/navid/DynaSLAM/include/Frame.h:127:13: warning: ‘cv::Mat ORB_SLAM2::Frame::mImDepth’ [-Wreorder]
cv::Mat mImDepth;
^
/home/navid/DynaSLAM/src/Frame.cc:154:1: warning: when initialized here [-Wreorder]
Frame::Frame(const cv::Mat &imGray, const cv::Mat &imDepth, const cv::Mat &imMask, const double &timeStamp, ORBextractor* extractor, ORBVocabulary* voc,
^
In file included from /home/navid/DynaSLAM/src/Frame.cc:10:0:
/home/navid/DynaSLAM/include/Frame.h: In constructor ‘ORB_SLAM2::Frame::Frame(const cv::Mat&, const cv::Mat&, const cv::Mat&, const cv::Mat&, const double&, ORB_SLAM2::ORBextractor*, ORB_SLAM2::ORBVocabulary*, cv::Mat&, cv::Mat&, const float&, const float&)’:
/home/navid/DynaSLAM/include/Frame.h:129:13: warning: ‘ORB_SLAM2::Frame::mImMask’ will be initialized after [-Wreorder]
cv::Mat mImMask;
^
/home/navid/DynaSLAM/include/Frame.h:95:20: warning: ‘ORB_SLAM2::ORBVocabulary* ORB_SLAM2::Frame::mpORBvocabulary’ [-Wreorder]
ORBVocabulary* mpORBvocabulary;
^
/home/navid/DynaSLAM/src/Frame.cc:236:1: warning: when initialized here [-Wreorder]
Frame::Frame(const cv::Mat &imGray, const cv::Mat &imDepth, const cv::Mat &imMask, const cv::Mat &imRGB,
^
In file included from /home/navid/DynaSLAM/src/Frame.cc:10:0:
/home/navid/DynaSLAM/include/Frame.h:130:13: warning: ‘ORB_SLAM2::Frame::mImGray’ will be initialized after [-Wreorder]
cv::Mat mImGray;
^
/home/navid/DynaSLAM/include/Frame.h:101:12: warning: ‘double ORB_SLAM2::Frame::mTimeStamp’ [-Wreorder]
double mTimeStamp;
^
/home/navid/DynaSLAM/src/Frame.cc:236:1: warning: when initialized here [-Wreorder]
Frame::Frame(const cv::Mat &imGray, const cv::Mat &imDepth, const cv::Mat &imMask, const cv::Mat &imRGB,
^
In file included from /home/navid/DynaSLAM/src/Frame.cc:10:0:
/home/navid/DynaSLAM/include/Frame.h:128:10: warning: ‘ORB_SLAM2::Frame::mIsKeyFrame’ will be initialized after [-Wreorder]
bool mIsKeyFrame;
^
/home/navid/DynaSLAM/include/Frame.h:127:13: warning: ‘cv::Mat ORB_SLAM2::Frame::mImDepth’ [-Wreorder]
cv::Mat mImDepth;
^
/home/navid/DynaSLAM/src/Frame.cc:236:1: warning: when initialized here [-Wreorder]
Frame::Frame(const cv::Mat &imGray, const cv::Mat &imDepth, const cv::Mat &imMask, const cv::Mat &imRGB,
^
/home/navid/DynaSLAM/src/Geometry.cc: In member function ‘std::vectorDynaSLAM::Geometry::DynKeyPoint DynaSLAM::Geometry::ExtractDynPoints(const std::vector<ORB_SLAM2::Frame>&, const ORB_SLAM2::Frame&)’:
/home/navid/DynaSLAM/src/Geometry.cc:337:21: warning: unused variable ‘xIndex’ [-Wunused-variable]
int xIndex = minIdx.x;
^
/home/navid/DynaSLAM/src/Geometry.cc: In member function ‘cv::Mat DynaSLAM::Geometry::RegionGrowing(const cv::Mat&, int&, int&, const float&)’:
/home/navid/DynaSLAM/src/Geometry.cc:1045:45: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
while(pixdist < reg_maxdist && reg_size < im.total())
^
CMakeFiles/Makefile2:178: recipe for target 'CMakeFiles/DynaSLAM.dir/all' failed
make[1]: *** [CMakeFiles/DynaSLAM.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

Please help me to solve....

Which is the TUM dataset fr3/nst fr3/office?

I want to evaluate the rgbd tum dataset, could you tell which one is fr3/nst and fr3/office in tum dataset? Thanks.

hard initialize and easy lose in mono style

in mono style, i find there are two difficulty

1. it is too hard to initialize
   
2. it is too easy to lose localization
   

in question #12 , advise to uprise the detect num, i try it , but has nothing different. its anything i can done to solve this problem, like imu ????

how to evaluate the kitti dateset with rpe rre and ate as your paper

could you supply the evaluate code, or tell how to evaluate rpe rre and ate. looking forward to u reply

make error

thanks for your work! I try to run ,but I meet a problems,could you help me???
when I run your code! I meet a error which is ：
Segment error (core dumped)
then I find the error occur at the function this->py_module = PyImport_ImportModule(this->module_name.c_str());this->class_name.c_str()) in the file of MaskNet.cc !and it return the null, I can not solve it ! can you help me?thanks!

problem in running the code

Hello! I have some problem in running the code. Could you please give me some suggustion?
1.When I run the DynaSLAM with the command " ./Examples/RGB-D/rgbd_tum Vocabulary/ORBvoc.txt Examples/RGB-D/TUMX3.yaml data/rgbd_dataset_freiburg3_walking_xyz/ data/rgbd_dataset_freiburg3_walking_xyz/associations.txt data/mask data/output”
My terminal's output is :
nImages: 827
Loading Mask R-CNN. This could take a while...
Importing Mask R-CNN Settings...
./Examples/RGB-D/MaskSettings.yaml
RuntimeError: _ARRAY_API is not PyCapsule object
Segmentation fault (core dumped)

2.When I don't add the PATH_TO_MASKS and PATH_TO_OUTPUT
, My terminal's output is :
nImages: 3983
DynaSLAM Copyright (C) 2018 Berta Bescos, University of Zaragoza.
Input sensor was set to: RGB-D
Failed to open settings file at: Examples/RGB-D/TUMX2.yaml

something wrong about the output of the depth

Hi，I'm afraid I need to disturb you again. When I test the rgbd-slam， I found that the folder of the depth output image is empty. Maybe there is some wrong in the code。

no ndarrayobject.h

May I ask you a question? When I build the code, there is a fatal error:
"DynaSLAM-master/include/Conversion.h:17:27: fatal error: ndarrayobject.h: No such file or directory".
How can I solve it? Thanks !

Issue with tensorflow

Hello everyone,
I follow steps to install DynaSLAM on my laptop but got following error:

navid@navid-X541UVK:~/DynaSLAM$ python src/python/Check.py
Traceback (most recent call last):
File "src/python/Check.py", line 10, in
import coco
File "/home/navid/DynaSLAM/src/python/coco.py", line 45, in
import utils
File "/home/navid/DynaSLAM/src/python/utils.py", line 15, in
import tensorflow as tf
ImportError: No module named tensorflow

Please help me.
Thanks alot.

Segmentation fault (core dumped)

Something run when I run the code

I'm sorry to bother you about the Compiling, but here is my problem.
The Check.py shows that it's good

/usr/local/lib/python2.7/dist-packages/pycocotools/coco.py:49: UserWarning: 
This call to matplotlib.use() has no effect because the backend has already
been chosen; matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
or matplotlib.backends is imported for the first time.

The backend was *originally* set to 'TkAgg' by the following code:
  File "src/python/Check.py", line 8, in <module>
    import matplotlib.pyplot as plt
  File "/usr/local/lib/python2.7/dist-packages/matplotlib/pyplot.py", line 71, in <module>
    from matplotlib.backends import pylab_setup
  File "/usr/local/lib/python2.7/dist-packages/matplotlib/backends/__init__.py", line 16, in <module>
    line for line in traceback.format_stack()


  import matplotlib; matplotlib.use('Agg')
/usr/local/lib/python2.7/dist-packages/requests/__init__.py:83: RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown.
  warnings.warn(warning, RequestsDependencyWarning)
Using TensorFlow backend.
Initializing Mask RCNN network...
./src/python
./src/python/logs
./src/python/mask_rcnn_coco.h5
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py:1154: calling reduce_max (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py:1188: calling reduce_sum (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
2019-02-18 21:42:14.370602: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
Initialated Mask RCNN network...
Mask R-CNN is correctly working

and I correct the CMakeLists.txt to use the opencv2.4.11. It's wired it always shows that I haven't give the mode:

 ./mono_tum ~/DynaSLAM/Vocabulary/ORBvoc.txt TUM1.yaml ~/Dataset/rgbd_dataset_freiburg2_desk_with_person /home/robotics/DynaSLAM/src/python/mask_rcnn_coco.h5 
Loading Mask R-CNN. This could take a while...
Importing Mask R-CNN Settings...
mono_tum: /home/robotics/DynaSLAM/src/MaskNet.cc:39: DynaSLAM::SegmentDynObject::SegmentDynObject(): Assertion `this->py_module != NULL' failed.
Aborted (core dumped)

I also use the python2.7
Could please help me, It makes me crazy.

Assertion `this->net != NULL' (again..!)

After encountering many issues running DynaSLAM on my machine (tensorlfow/cuda/protobuf versions...), I tried running it with Docker

Here's the backtrace when I compile in Debug

nImages: 1021
Loading Mask R-CNN. This could take a while...
Importing Mask R-CNN Settings...
    py_path: ./src/python/
    module_name: MaskRCNN
    class_name: Mask
    get_dyn_seg: GetDynSeg
[New Thread 0x7fffd288f700 (LWP 1037)]
[New Thread 0x7fffd208e700 (LWP 1038)]
[New Thread 0x7fffcd88d700 (LWP 1039)]
[New Thread 0x7fffcd08c700 (LWP 1040)]
[New Thread 0x7fffc888b700 (LWP 1041)]
[New Thread 0x7fffc808a700 (LWP 1042)]
[New Thread 0x7fffc5889700 (LWP 1043)]
[New Thread 0x7fffbcf42700 (LWP 1044)]
[New Thread 0x7fffbc741700 (LWP 1045)]
[New Thread 0x7fffb9f40700 (LWP 1046)]
[New Thread 0x7fffb573f700 (LWP 1047)]
[New Thread 0x7fffb2f3e700 (LWP 1048)]
[New Thread 0x7fffb073d700 (LWP 1049)]
[New Thread 0x7fffaff3c700 (LWP 1050)]
[Thread 0x7fffaff3c700 (LWP 1050) exited]
[Thread 0x7fffb073d700 (LWP 1049) exited]
[Thread 0x7fffb2f3e700 (LWP 1048) exited]
[Thread 0x7fffb573f700 (LWP 1047) exited]
[Thread 0x7fffb9f40700 (LWP 1046) exited]
[Thread 0x7fffbc741700 (LWP 1045) exited]
[Thread 0x7fffbcf42700 (LWP 1044) exited]
[Thread 0x7fffc5889700 (LWP 1043) exited]
[Thread 0x7fffc808a700 (LWP 1042) exited]
[Thread 0x7fffc888b700 (LWP 1041) exited]
[Thread 0x7fffcd08c700 (LWP 1040) exited]
[Thread 0x7fffcd88d700 (LWP 1039) exited]
[Thread 0x7fffd208e700 (LWP 1038) exited]
[Thread 0x7fffd288f700 (LWP 1037) exited]
/usr/local/lib/python2.7/dist-packages/h5py/__init__.py:40: UserWarning: h5py is running against HDF5 1.10.0 when it was built against 1.10.4, this may cause problems
  '{0}.{1}.{2}'.format(*version.hdf5_built_version_tuple)
Using TensorFlow backend.
Initializing Mask RCNN network...
./src/python
[New Thread 0x7fffc5889700 (LWP 1054)]
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py:4070: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.

WARNING:tensorflow:From /DynaSLAM/src/python/model.py:366: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /DynaSLAM/src/python/model.py:390: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version.
Instructions for updating:
box_ind is deprecated, use box_indices instead
WARNING:tensorflow:From /DynaSLAM/src/python/model.py:746: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, there are two
    options available in V2.
    - tf.py_function takes a python function which manipulates tf eager
    tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
    an ndarray (just call tensor.numpy()) but having access to eager tensors
    means `tf.py_function`s can use accelerators such as GPUs as well as
    being differentiable using a gradient tape.
    - tf.numpy_function maintains the semantics of the deprecated tf.py_func
    (it is not differentiable, and manipulates numpy arrays). It drops the
    stateful argument making all functions stateful.
    
rgbd_tum: /DynaSLAM/src/MaskNet.cc:43: DynaSLAM::SegmentDynObject::SegmentDynObject(): Assertion `this->net != NULL' failed.

Thread 1 "rgbd_tum" received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51	../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) ny
Undefined command: "ny".  Try "help".
(gdb) bt
#0  0x00007ffff59eae97 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007ffff59ec801 in __GI_abort () at abort.c:79
#2  0x00007ffff59dc39a in __assert_fail_base (fmt=0x7ffff5b637d8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x7ffff7bb4b72 "this->net != NULL", file=file@entry=0x7ffff7bb4a7e "/DynaSLAM/src/MaskNet.cc", line=line@entry=43, function=function@entry=0x7ffff7bb51a0 <DynaSLAM::SegmentDynObject::SegmentDynObject()::__PRETTY_FUNCTION__> "DynaSLAM::SegmentDynObject::SegmentDynObject()") at assert.c:92
#3  0x00007ffff59dc412 in __GI___assert_fail (assertion=assertion@entry=0x7ffff7bb4b72 "this->net != NULL", file=file@entry=0x7ffff7bb4a7e "/DynaSLAM/src/MaskNet.cc", line=line@entry=43, function=function@entry=0x7ffff7bb51a0 <DynaSLAM::SegmentDynObject::SegmentDynObject()::__PRETTY_FUNCTION__> "DynaSLAM::SegmentDynObject::SegmentDynObject()") at assert.c:101
#4  0x00007ffff7b877a7 in DynaSLAM::SegmentDynObject::SegmentDynObject() (this=0x555555cccd80) at /DynaSLAM/src/MaskNet.cc:43
#5  0x00005555555635da in main(int, char**) (argc=6, argv=0x7fffffffe438) at /DynaSLAM/Examples/RGB-D/rgbd_tum.cc:62

When I run in RelWithDebInfo, here's the backtrace (don't get why it's different, probably doesn't matter):

/usr/local/lib/python2.7/dist-packages/h5py/__init__.py:40: UserWarning: h5py is running against HDF5 1.10.0 when it was built against 1.10.4, this may cause problems
  '{0}.{1}.{2}'.format(*version.hdf5_built_version_tuple)
Using TensorFlow backend.
Initializing Mask RCNN network...
./src/python
[New Thread 0x7fffa766a700 (LWP 115)]
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py:4070: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.

WARNING:tensorflow:From /DynaSLAM/src/python/model.py:315: The name tf.log is deprecated. Please use tf.math.log instead.

WARNING:tensorflow:From /DynaSLAM/src/python/model.py:366: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /DynaSLAM/src/python/model.py:390: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version.
Instructions for updating:
box_ind is deprecated, use box_indices instead
WARNING:tensorflow:From /DynaSLAM/src/python/model.py:746: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, there are two
    options available in V2.
    - tf.py_function takes a python function which manipulates tf eager
    tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
    an ndarray (just call tensor.numpy()) but having access to eager tensors
    means `tf.py_function`s can use accelerators such as GPUs as well as
    being differentiable using a gradient tape.
    - tf.numpy_function maintains the semantics of the deprecated tf.py_func
    (it is not differentiable, and manipulates numpy arrays). It drops the
    stateful argument making all functions stateful.
    
Creating net instance...
Loading net parameters...

Thread 1 "rgbd_tum" received signal SIGSEGV, Segmentation fault.
0x00007ffff7b9a05b in DynaSLAM::NDArrayConverter::toMat (this=0x555555d15920, o=0x0) at /DynaSLAM/src/Conversion.cc:225
225	   if( !PyArray_Check(o) )
(gdb) bt
#0  0x00007ffff7b9a05b in DynaSLAM::NDArrayConverter::toMat(_object const*) (this=0x555555d15920, o=0x0) at /DynaSLAM/src/Conversion.cc:225
#1  0x00007ffff7b9b2b2 in DynaSLAM::SegmentDynObject::GetSegmentation(cv::Mat&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (this=this@entry=0x555555ccbd80, image=..., dir="no_save", name="no_file") at /DynaSLAM/src/MaskNet.cc:63
#2  0x00007ffff7b9c751 in DynaSLAM::SegmentDynObject::SegmentDynObject() (this=0x555555ccbd80) at /DynaSLAM/src/MaskNet.cc:47
#3  0x00005555555636b4 in main(int, char**) (argc=6, argv=0x7fffffffe438) at /DynaSLAM/Examples/RGB-D/rgbd_tum.cc:62

The only thing that looks wrong is the h5py / hdf5 warning, but I have no idea if that could be the problem...

src/python/Check.py fails because of deprecated APIs

I updated models.py to use the new APIs, and the output seems fine:

root@9ad5cf137e22:/DynaSLAM# python src/python/Check.py Using TensorFlow backend.
Initializing Mask RCNN network...
./src/python
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py:4070: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.

WARNING:tensorflow:From /DynaSLAM/src/python/model.py:366: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /DynaSLAM/src/python/model.py:390: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version.
Instructions for updating:
box_ind is deprecated, use box_indices instead
WARNING:tensorflow:From /DynaSLAM/src/python/model.py:746: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, there are two
    options available in V2.
    - tf.py_function takes a python function which manipulates tf eager
    tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
    an ndarray (just call tensor.numpy()) but having access to eager tensors
    means `tf.py_function`s can use accelerators such as GPUs as well as
    being differentiable using a gradient tape.
    - tf.numpy_function maintains the semantics of the deprecated tf.py_func
    (it is not differentiable, and manipulates numpy arrays). It drops the
    stateful argument making all functions stateful.
    
2020-03-19 18:06:27.355507: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-03-19 18:06:27.360083: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcuda.so.1
2020-03-19 18:06:27.424248: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-03-19 18:06:27.424896: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5640caf5cab0 executing computations on platform CUDA. Devices:
2020-03-19 18:06:27.424920: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): GeForce GTX 1050 Ti, Compute Capability 6.1
2020-03-19 18:06:27.426470: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3696330000 Hz
2020-03-19 18:06:27.426780: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5640cbc2b980 executing computations on platform Host. Devices:
2020-03-19 18:06:27.426811: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): <undefined>, <undefined>
2020-03-19 18:06:27.426955: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-03-19 18:06:27.427406: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties: 
name: GeForce GTX 1050 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.392
pciBusID: 0000:01:00.0
2020-03-19 18:06:27.427613: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2020-03-19 18:06:27.428651: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
2020-03-19 18:06:27.429485: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0
2020-03-19 18:06:27.429705: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0
2020-03-19 18:06:27.430804: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0
2020-03-19 18:06:27.431657: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0
2020-03-19 18:06:27.434450: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2020-03-19 18:06:27.434631: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-03-19 18:06:27.435172: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-03-19 18:06:27.435636: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0
2020-03-19 18:06:27.435707: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2020-03-19 18:06:27.436585: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-03-19 18:06:27.436600: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187]      0 
2020-03-19 18:06:27.436610: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0:   N 
2020-03-19 18:06:27.436711: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-03-19 18:06:27.437166: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-03-19 18:06:27.437596: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2965 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1050 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)
Initialated Mask RCNN network...
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py:422: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

2020-03-19 18:06:31.597647: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set.  If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU.  To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2020-03-19 18:06:31.668133: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
2020-03-19 18:06:31.816810: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
Mask R-CNN is correctly working

You can find the Dockerfile as well as the API updates on my fork

I saw issues #24, but my Check.py works fine
#11 as well, but I run everything from the root directory:
./Examples/RGB-D/rgbd_tum Vocabulary/ORBvoc.txt Examples/RGB-D/TUM3.yaml /data/tum/rgbd_dataset_freiburg3_walking_halfsphere /data/tum/rgbd_dataset_freiburg3_walking_halfsphere/associations.txt no_save

I'm seeing comments about opencv versions (my Dockerfile uses opencv3), I will update this issue whenever I find a solution...

segment picture size

cv::Mat image  = cv::Mat::zeros(480,640,CV_8UC3); //Be careful with size!!

@BertaBescos @jmfacil Can i change the pucture size ,such as 960*540and it will case the lower
velocity of segment??

check.py failed

Hi there~! I have alrerady downloaded the file 'mask_rcnn_coco.h5' and palced it in /src/python. However, when I use terminal with command 'python Check.py' to run check, it failed and show that 'No such file or Directory'. I'm confused and don't know why....

Importing Mask R-CNN Settings... Segmentation fault (core dumped)

The build process could be finished.
But under the Mask R-CNN detecting model, the Segment fault occurred.

:~/DynaSLAM-master$ ./Examples/RGB-D/rgbd_tum Vocabulary/ORBvoc.txt Examples/RGB-D/TUM3.yaml rgbd_dataset_freiburg3_walking_xyz/ associate.txt mask/ output/
nImages: 827
Loading Mask R-CNN. This could take a while...
Importing Mask R-CNN Settings...
Segmentation fault (core dumped)

After that I checked the Check.py file for Mask R-CNN, the result as bellowed:

~/DynaSLAM-master/src/python$ python Check.py
Traceback (most recent call last):
File "Check.py", line 10, in
import coco
File "/home/sunleyuan/DynaSLAM-master/src/python/coco.py", line 40, in
from pycocotools.coco import COCO
File "/usr/local/lib/python2.7/dist-packages/pycocotools/coco.py", line 55, in
from . import mask as maskUtils
File "/usr/local/lib/python2.7/dist-packages/pycocotools/mask.py", line 3, in
import pycocotools._mask as _mask
File "init.pxd", line 918, in init pycocotools._mask
ValueError: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216 from C header, got 192 from PyObject

plz, give me some instructions.
Sorry for open an issue again.

why tracking is not initialzed？

how to evaluate KITTI results？

there is a way to evaluate the TUM result.but how can we evaluate the result which running in KITTI dataset? thank you .

The optimizer runs slowly

Hi, i found that the project runs so slowly in optimizer module.
For example, when the tracking thread calls function "Optimizer::PoseOptimization()" in function "TrackLocalMap()", it uses about 1.2s. But in ORB_SLAM2, this part only uses 0.003s.

I did not edit optimizer related code, have you guys met the same problem? What should i do to fix it?
Thanks in advance.

A problem of the system run unsmoothly.

Sorry to bother. When I run the rgbd_tum examples, although I just input 5 parameters which means the system runs without MASK_RCNN, the system runs unsmoothly. It can just load 2 or 3 images for every 10 seconds.
I can run ORBSLAM2 very smoothly, and my cpu is i7.
Is there anyone can help me?

getting seg-faults when importing mask r-cnn

I ran the command to run dyna slam for mono_kitti adding at the end the path to the mask rcnn, which for me was at "./src/python/mask_rcnn_coco.h5", but I am getting seg faults.

Anyone with solution to this?

Thank you in advance!

Could not run.....

Hi Guys... I installed successfully according to readme.md. but when running "./Examples/RGB-D/rgbd_tum Vocabulary/ORBvoc.txt Examples/RGB-D/TUMX.yaml PATH_TO_SEQUENCE_FOLDER ASSOCIATIONS_FILE (PATH_TO_MASKS) (PATH_TO_OUTPUT)" i've got following error:

Loading Mask R-CNN. This could take a while...
Importing Mask R-CNN Settings...
2020-03-20 19:59:37.952620: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer.so.6
2020-03-20 19:59:37.957395: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer_plugin.so.6
/home/navid/.local/lib/python2.7/site-packages/h5py/init.py:40: UserWarning: h5py is running against HDF5 1.10.0 when it was built against 1.10.4, this may cause problems
'{0}.{1}.{2}'.format(*version.hdf5_built_version_tuple)
Using TensorFlow backend.
Initializing Mask RCNN network...
./src/python
2020-03-20 19:59:38.466275: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-03-20 19:59:38.479826: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-03-20 19:59:38.480240: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce 920MX computeCapability: 5.0
coreClock: 0.993GHz coreCount: 2 deviceMemorySize: 1.96GiB deviceMemoryBandwidth: 13.41GiB/s
2020-03-20 19:59:38.480308: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-03-20 19:59:38.480373: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-03-20 19:59:38.481469: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-03-20 19:59:38.481860: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-03-20 19:59:38.483331: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-03-20 19:59:38.484447: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-03-20 19:59:38.484543: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-03-20 19:59:38.484635: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-03-20 19:59:38.485293: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-03-20 19:59:38.485733: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
2020-03-20 19:59:38.485969: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-03-20 19:59:38.509175: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2899885000 Hz
2020-03-20 19:59:38.509382: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55ba360323d0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-03-20 19:59:38.509398: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2020-03-20 19:59:38.549730: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-03-20 19:59:38.550075: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55ba360a8410 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-03-20 19:59:38.550093: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): GeForce 920MX, Compute Capability 5.0
2020-03-20 19:59:38.550260: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-03-20 19:59:38.550499: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce 920MX computeCapability: 5.0
coreClock: 0.993GHz coreCount: 2 deviceMemorySize: 1.96GiB deviceMemoryBandwidth: 13.41GiB/s
2020-03-20 19:59:38.550599: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-03-20 19:59:38.550648: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-03-20 19:59:38.550697: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-03-20 19:59:38.550743: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-03-20 19:59:38.550787: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-03-20 19:59:38.550831: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-03-20 19:59:38.550874: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-03-20 19:59:38.550940: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-03-20 19:59:38.551200: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-03-20 19:59:38.551415: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
2020-03-20 19:59:38.551471: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-03-20 19:59:38.825272: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1096] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-03-20 19:59:38.825305: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] 0
2020-03-20 19:59:38.825311: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] 0: N
2020-03-20 19:59:38.825556: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-03-20 19:59:38.825862: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-03-20 19:59:38.826104: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1241] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1506 MB memory) -> physical GPU (device: 0, name: GeForce 920MX, pci bus id: 0000:01:00.0, compute capability: 5.0)
Creating net instance...
Loading net parameters...
Segmentation fault

Please help me.
Thanks alot

some wrong about the ndarrayobject.h

When I build the project, I meet the error that I couldn't find ndarrayobject.h, I don't know what is it, and how can I solve the problem?

Issues with protobuf

while running im getting this issues.

Loading Mask R-CNN. This could take a while...
Importing Mask R-CNN Settings...
[libprotobuf FATAL google/protobuf/stubs/common.cc:61] This program requires version 3.7.0 of the Protocol Buffer runtime library, but the installed version is 2.6.1. Please update your library. If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library. (Version verification failed in "bazel-out/k8-opt/genfiles/tensorflow/core/framework/tensor_shape.pb.cc".)

I installed protobuf version correctly but still this issues come

Bus error (core dumped)

Hi! I met an error when running the code. I have tried this both with cpu and gpu.

Initialated Mask RCNN network...
Creating net instance...
Loading net parameters...
Bus error (core dumped)

I run the Check.py successfully and can run the code without [PATH TO MASK] [PATH TO OUTPUT] parameters. Could anyone help me? Thanks!!!

Can we use the GPU to run the code?

I'm sorry to bothering you. In my computer, I figure out that the tensorflow is using the CPU to run the code. How to make it use the GPU？

build error about -- no member named ‘mvpMatchedLoop’

---

[ 6%] Building CXX object CMakeFiles/DynaSLAM.dir/src/System.cc.o
/home/sly/DynaSLAM/src/System.cc: In member function ‘void ORB_SLAM2::System::SaveDetectedLoops(const string&)’:
/home/sly/DynaSLAM/src/System.cc:467:72: error: ‘class ORB_SLAM2::LoopClosing’ has no member named ‘mvpMatchedLoop’
vector<pair<KeyFrame*, KeyFrame*> > mvpMatchedLoop = mpLoopCloser->mvpMatchedL
^
CMakeFiles/DynaSLAM.dir/build.make:62: recipe for target 'CMakeFiles/DynaSLAM.dir/src/System.cc.o' failed

Please give me some guidance, thanks!

I have same problem

Importing Mask R-CNN Settings...
ImportError: numpy.core.multiarray failed to import
Segmentation fault (core dumped)

tracking stuck and lost

hi ! when I run the system in only geometry mode ,program will dead and tracking lost after tracking few frames.

Light Tracking not working because Tracking is not initialized...

There is a problem for me. When I run the command" ./Examples/RGB-D/rgbd_tum Vocabulary/ORBvoc.txt Examples/RGB-D/TUM1.yaml ........", there will always be a problem:"Light Tracking not working because Tracking is not initialized...". Could you tell me the reason for that? Thank you very much.

cannot declare variable ‘DynaSLAM::g_numpyAllocator’

/home/xh/Projects/SLAM/DynaSLAM/src/Conversion.cc:205:16: error: cannot declare variable ‘DynaSLAM::g_numpyAllocator’ to be of abstract type ‘DynaSLAM::NumpyAllocator’

NumpyAllocator g_numpyAllocator;

/home/xh/Projects/SLAM/DynaSLAM/src/Conversion.cc:74:7: note: because the following virtual functions are pure within ‘DynaSLAM::NumpyAllocator’:

class NumpyAllocator : public MatAllocator In file included from /usr/local/include/opencv2/core.hpp:59:0, from/usr/local/include/opencv2/imgproc.hpp:46, from/home/xh/Projects/SLAM/DynaSLAM/include/Conversion.h:14, from/home/xh/Projects/SLAM/DynaSLAM/src/Conversion.cc:9:

I got this issue when I run make -j12 in ~/Projects/SLAM/DynaSLAM/build.
But I don't find any pure virtual functions within ‘DynaSLAM::NumpyAllocator’.
Do you know why this occurs? Looking forward to your answer.

about the output of the octo map

Hi, this is a excellent work. I have saw you build the octomap in the video, but I don't find some code about building the map in your open source

run error segment fault

hi, every big god!
when i run the code with R-CNN, it show the following error

nImages: 795
Loading Mask R-CNN. This could take a while...
Importing Mask R-CNN Settings...
[1]    29733 segmentation fault (core dumped)  ./Examples/RGB-D/rgbd_tum Vocabulary/ORBvoc.txt Examples/RGB-D/TUM3.yaml

and when i run Check.py , it show the following error

Traceback (most recent call last):
  File "Check.py", line 6, in <module>
    import skimage.io
ImportError: No module named skimage.io

python version is: 2.7.12, opencv version is: 3.3.1

I have some problem when i runing the code

when i runing the code with Mask_RCNN, some error occur as follow:

nImages: 1352
Loading Mask R-CNN. This could take a while...
Importing Mask R-CNN Settings...
Segmentation fault (core dumped)

and i have check Mask_RCNN using Check.py:

Initialated Mask RCNN network...
Mask R-CNN is correctly working

when i running without MASK_RCNN:

Start processing sequence ...
Images in the sequence: 1352

Light Tracking not working because Tracking is not initialized...
Geometry not working.
New map created with 855 points

I don't konw why, could you tell me what should i do ? thank you!

Some questions about running the code

Hello! I have some trouble in running the code. Could you please give me some guidance?
1.When I add the argument PATH_TO_MASKS, it always shows segmentation fault. I don't know how to add the argument correctly and what dose the argument mean?
2.When I don't add the argument, it can run but it always can not initialize. I have tried several sequences and it always shows "Trying to Initialize" .
3.I find the MaskSettingsStereo.yaml and should I modify any parameter in the file?

about the opencv version?

hello ,your project is amazing.but when i build your project,it makes error.
Does your program have to use opencv 2.4.11?it is not be opencv 3.4.1?

about the "this->py_module "error

hello, @BertaBescos,i met the issue as the #11 (comment):
/home/robotics/DynaSLAM/src/MaskNet.cc:39: DynaSLAM::SegmentDynObject::SegmentDynObject(): Assertion `this->py_module != NULL' failed.
Aborted (core dumped)

And i run the python Check.py,it runs successfully.

but i can't understand the solution :
"Depending the directory you are running the program from (e.g. ./Examples/Monocular or ./ ) this may change (e.g. ../../src/python or ./src/python). Make sure that the path the program is using is the correct one."
I run the command in my computer:
./Example/Monocular/mono_tum Vocabulary/ORBvoc.txt Examples/Monocular/TUM3.yaml /home/skydddive/datasets/tum/rgb_freiburg3_walking_xyz mask
[where do i need to change "./src/python"?Could you please help me ..

Light Tracking not working because Tracking is not initialized...

~/DynaSLAM-master$ ./Examples/RGB-D/rgbd_tum Vocabulary/ORBvoc.txt Examples/RGB-D/TUM3.yaml rgbd_dataset_freiburg3_walking_xyz/ associate.txt mask/ output/

---

Initialated Mask RCNN network...
Creating net instance...
Loading net parameters...
Mask R-CNN loaded!

DynaSLAM Copyright (C) 2018 Berta Bescos, University of Zaragoza.

Input sensor was set to: RGB-D

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Camera Parameters:

• fx: 535.4
• fy: 539.2
• cx: 320.1
• cy: 247.6
• k1: 0
• k2: 0
• p1: 0
• p2: 0
• fps: 30
• color order: RGB (ignored if grayscale)

ORB Extractor Parameters:

• Number of Features: 1000
• Scale Levels: 8
• Scale Factor: 1.2
• Initial Fast Threshold: 20
• Minimum Fast Threshold: 7

Depth Threshold (Close/Far Points): 2.98842

---

Start processing sequence ...
Images in the sequence: 827

Framebuffer with requested attributes not available. Using available framebuffer. You may see visual artifacts.Light Tracking not working because Tracking is not initialized...
Light Tracking not working because Tracking is not initialized...
Light Tracking not working because Tracking is not initialized...
Light Tracking not working because Tracking is not initialized...
Light Tracking not working because Tracking is not initialized...

Quick questions?

Is it possible to use a webcam, instead of the datasets?
I want to use this in a Head mounted display and use the SLAM for position tracking is it possible?

DynaSLAM::SegmentDynObject::SegmentDynObject(): Assertion `this->py_module != NULL' failed.

OK! I really appreciate for your help! It can work now, but another question also bothering me that I can't see the point cloud? Have you publish the code about that ?

Originally posted by @Gatsby23 in #11 (comment)

Hi, I also encounter the same problem,could you help me to solve it in details?

the git instruction has no authority

git clone [email protected]:BertaBescos/DynaSLAM.git
Cloning into 'DynaSLAM'...
The authenticity of host 'github.com (xxx.xxx.xxx.xxx)' can't be established.
RSA key fingerprint is .
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '***' (RSA) to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

(while I can use git clone --recursive to clone)

Conversion.h

in file included from /root/DynaSlam/src/Conversion.cc:9:0: /root/DynaSlam/include/Conversion.h:17:27: fatal error: **ndarrayobject.h: No such file or directory compilation terminated.** CMakeFiles/DynaSLAM.dir/build.make:518: recipe for target 'CMakeFiles/DynaSLAM.dir/src/Conversion.cc.o' failed make[2]: *** [CMakeFiles/DynaSLAM.dir/src/Conversion.cc.o] Error 1 make[2]: *** Waiting for unfinished jobs.... In file included from /root/DynaSlam/include/MaskNet.h:26:0, from /root/DynaSlam/src/MaskNet.cc:9: /root/DynaSlam/include/Conversion.h:17:27: fatal error: ndarrayobject.h: No such file or directory compilation terminated. CMakeFiles/DynaSLAM.dir/build.make:542: recipe for target 'CMakeFiles/DynaSLAM.dir/src/MaskNet.cc.o' failed make[2]: *** [CMakeFiles/DynaSLAM.dir/src/MaskNet.cc.o] Error 1

Software requirement....

Hello,
Would you please help me to install and run Dynaslam on ubuntu by introducing software version:
Ubuntu version...
Cuda cersion...
Tensorflow Version...
Keras version...

Thanks a lot.

Light Tracking not working because Tracking is not initialized... ？

why the error will cause ? if you have solved this problem ,please tell me Thanks !!

How to save the segmentation results using geometry and deep learning?

Hi, your work is amazing! How to save the segmentation results using geometry and deep learning? Thank you very much!