hlkyss/modified_yolo_orb_slam3

原开源代码只给了RGBD的，补充了一下自己使用的双目的，后面再补其他的

来源：https://github.com/YWL0720/YOLO_ORB_SLAM3
原开源代码只给了RGBD的，补充了一下自己使用的双目的，后面再补其他的

编译：
cmakelists只修改了opencv和cv_bridge相关部分，需要根据自己的环境修改。此外，YoloDetect.cpp里有一句加载模型的语句，我改成了我的绝对路径，需要修改成自己的。

双目：
./Examples/Stereo/stereo_tum_vi /home/hl/project/ORB_SLAM3_detailed_comments-master/Vocabulary/ORBvoc.txt /home/hl/project/YOLO_ORB_SLAM3-master/Examples/Stereo/ue_pin.yaml /media/hl/Stuff/ubuntu_share_2/Dataset/ue_pin_fov100/theta0/cam0 /media/hl/Stuff/ubuntu_share_2/Dataset/ue_pin_fov100/theta0/cam1 /media/hl/Stuff/ubuntu_share_2/Dataset/ue_180/time.txt /media/hl/Stuff/ubuntu_share_2/Dataset/ue_pin_fov100/theta0/result/dataset-hik_stereo-traj

单目：
./Examples/Monocular/mono_euroc ./Vocabulary/ORBvoc.txt ./Examples/Monocular/EuRoC.yaml /media/hl/Stuff/ubuntu_share_2/Dataset/EuRoc/V1_02 ./Examples/Monocular/EuRoC_TimeStamps/V102.txt

./Examples/Monocular/mono_tum_vi /home/hl/project/YOLO_ORB_SLAM3-master/Vocabulary/ORBvoc.txt /home/hl/project/YOLO_ORB_SLAM3-master/Examples/Monocular/ue_pin.yaml /media/hl/Stuff/ubuntu_share_2/Dataset/ue_pin_fov100/theta0/cam0 /media/hl/Stuff/ubuntu_share_2/Dataset/ue_180/time.txt

---

YOLO_ORB_SLAM3

This is an improved version of ORB-SLAM3 that adds an object detection module implemented with YOLOv5 to achieve SLAM in dynamic environments.

• Object Detection
• Dynamic SLAM

Fig 1 : Test with TUM dataset

Getting Started

0. Prerequisites

We have tested on:

OS = Ubuntu 20.04

OpenCV = 4.2

Eigen3 = 3.3.9

Pangolin = 0.5

ROS = Noetic

1. Install libtorch

Recommended way

You can download the compatible version of libtorch from Baidu Netdisk code: 8y4k, then

unzip libtorch.zip
mv libtorch/ PATH/YOLO_ORB_SLAM3/Thirdparty/

Or you can

wget https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-1.11.0%2Bcpu.zip
unzip libtorch-cxx11-abi-shared-with-deps-1.11.0%2Bcpu.zip
mv libtorch/ PATH/YOLO_ORB_SLAM3/Thirdparty/

2. Build

cd YOLO_ORB_SLAM3
chmod +x build.sh
./build.sh

Only the rgbd_tum target will be build.

3. Build ROS Examples

Add the path including Examples/ROS/YOLO_ORB_SLAM3 to the ROS_PACKAGE_PATH environment variable. Open .bashrc file:

gedit ~/.bashrc

and add at the end the following line. Replace PATH by the folder where you cloned YOLO_ORB_SLAM3:

export ROS_PACKAGE_PATH=${ROS_PACKAGE_PATH}:PATH/YOLO_ORB_SLAM3/Examples/ROS

Then build

chmod +x build_ros.sh
./build_ros.sh

Only the RGBD target has been improved.

The frequency of camera topic must be lower than 15 Hz.

You can run this command to change the frequency of topic which published by the camera driver.

roslaunch YOLO_ORB_SLAM3 camera_topic_remap.launch

4. Try

TUM Dataset

./Examples/RGB-D/rgbd_tum Vocabulary/ORBvoc.txt Examples/RGB-D/TUMX.yaml PATH_TO_SEQUENCE_FOLDER ASSOCIATIONS_FILE

ROS

roslaunch YOLO_ORB_SLAM3 camera_topic_remap.launch
rosrun YOLO_ORB_SLAM3 RGBD PATH_TO_VOCABULARY PATH_TO_SETTINGS_FILE

modified_yolo_orb_slam3's Issues

build_ros.sh error

单目

请问大佬有单目版本吗

双目数据集

请问大佬你跑的双目数据集图像是RGB三通道的吗？如果方便的话可否分享下你的双目数据集（[email protected]）。
我用的是euroc数据集，他提供的是灰度双目图像，跑ros下的双目模式就会报错。我查了查报错原因是因为输入图像的通道数没有大于1，也就是说不是RGB图像代码处理不了。
另外我用了原版orb_slam3试了下是没问题的，看了下YOLO检测的代码应该是还没有写怎处理灰度图像，所以我想要跑灰度数据集是不是还要重新训练一个针对灰度图像检测的权重？下面是我报错的信息：

terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(4.2.0) ../modules/imgproc/src/color.simd_helpers.hpp:92: error: (-2:Unspecified error) in function 'cv::impl::{anonymous}::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::impl::{anonymous}::Set<3, 4>; VDcn = cv::impl::{anonymous}::Set<1>; VDepth = cv::impl::{anonymous}::Set<0, 2, 5>; cv::impl::{anonymous}::SizePolicy sizePolicy = cv::impl::<unnamed>::NONE; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'
> Invalid number of channels in input image:
>     'VScn::contains(scn)'
> where
>     'scn' is 1

Aborted (core dumped)

环境和README中一样

编译错误

CMake Error at CMakeLists.txt:91 (ADD_LIBRARY):
No SOURCES given to target: g2o

Cannot find source file:

g2o/stuff/timeutil.cpp

CMake Error at CMakeLists.txt:43 (add_library):

Cannot find source file:

DUtils/Timestamp.cpp

这两个位置各少了四个文件
还有cmakelists 里面调用opencv那里我是按照原版本的yolo slam 来的
这样编译成功了

但是跑V102这个数据集的时候报错

Initialization of Atlas from scratch
Creation of new map with id: 0
Creation of new map with last KF id: 0
已放弃 (核心已转储)