petworm/larvio

A lightweight, accurate and robust monocular visual inertial odometry based on Multi-State Constraint Kalman Filter.

larvio's Issues

Using LARVIO with realsense d435i

Hello,

I would like to know if it is possible to run this algorithm with the realsense d435i.
I tried it but the results are far from being good. May be is something wrong with the configuration that I'm using...

Do you have any configuration for this camera like you have for the mynteye?

Best,
Filipe

Covariance for dynamic initialization

thanks for your great job on LARVIO！in dynamic initialization，the covariance is not considered for imu_state in LARVIO？ could you have any ideas about the obtian of covariance from your dynamic initialization？

ZUPT algorithm

Hi,
Are there any reference papers for the ZUPT algorithm applied to MSCKF?

License?

What is the license of your code?

"Are the Numerical Effects of Zero-Velocity Update (ZUPT) Constraints on Relative Rotation the Same When Using Quaternions and Rotation Matrices?

Hi, 邱博您好

请问zupt相对旋转上约束, 采用四元数与旋转矩阵数值效果一样嘛？
可以替换吗?

Error while trying to run the ROS example

@PetWorm
Dear Sir,

Thank you for publishing this work !!
While trying to run the ROS example we receive the following error:

user@user-HP-ProDesk-600-G4-DM:~/LARVIO/ros_wrapper$ roslaunch larvio larvio_euroc.launch
... logging to /home/user/.ros/log/4efc391c-0333-11eb-bbb7-3822e23158ce/roslaunch-user-HP-ProDesk-600-G4-DM-27946.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://user-HP-ProDesk-600-G4-DM:46333/

SUMMARY

PARAMETERS

• /qxc_robot/system/child_frame_id: odom
• /qxc_robot/system/config_file: /home/user/LARVIO...
• /qxc_robot/system/fixed_frame_id: world
• /rosdistro: melodic
• /rosversion: 1.14.7

NODES
/qxc_robot/
system (nodelet/nodelet)

ROS_MASTER_URI=http://localhost:11311

process[qxc_robot/system-1]: started with pid [27961]
type is larvio/SystemNodelet
[ INFO] [1601535646.772076998]: System: Finish loading ROS parameters...

===========================================
using FEJ...
estimating td... initial td = 0
estimating extrinsic...
not calibrating imu instrinsic online...
Hybrid MSCKF...Maximum number of feature in state is 30
features augmented into state will use 1d idp

[ INFO] [1601535646.795484764]: System Manager: Finish creating ROS IO...
Images from now on will be utilized...

Dynamic initialization success !

[qxc_robot/system-1] process has died [pid 27961, exit code -11, cmd /opt/ros/melodic/lib/nodelet/nodelet standalone larvio/SystemNodelet ~imu:=/imu0 ~cam0_image:=/cam0/image_raw __name:=system __log:=/home/user/.ros/log/4efc391c-0333-11eb-bbb7-3822e23158ce/qxc_robot-system-1.log].
log file: /home/user/.ros/log/4efc391c-0333-11eb-bbb7-3822e23158ce/qxc_robot-system-1*.log
all processes on machine have died, roslaunch will exit
shutting down processing monitor...
... shutting down processing monitor complete
done

Please advise.
Thank you

請問如何將軌跡以tum格式輸出到文件

ubuntu16.04 run euroc example error

1. gcc 7
2. ubuntu16.04

===========================================
using FEJ...
estimating td... initial td = 0
estimating extrinsic...
not calibrating imu instrinsic online...
Hybrid MSCKF...Maximum number of feature in state is 30
features augmented into state will use 1d idp
===========================================

Images from now on will be utilized...

Inclinometer-initializer completed by using 181 imu data !!!

*** stack smashing detected ***: ./larvio terminated
Aborted (core dumped)

Should I add someting else ?

Learning materials

Hi! I'm trying to learn LARVIO implementration, but i have poor knowledge in the field of inertial sensors. Could You tell me what books can help in understanding the work?

Running LARVIO on Jetson Nano XS

Hi,

Great work!

Do you by any chance have results regarding running time stats for both the frontend and backend part and CPU consumption for when you run LARVIO on the TX2 or whether it has the same running time as the pc? At the moment I am able to run it on a Jetson Nano XS at 30 Hz.

Also, I just tried to run LARVIO on my pc, but the best I get for the backend it 11ms. What did you do to get it to approx. 8 ms (i.e how many features, and whether you disabled some options on the yaml file)?

I would like to reach the 50 Hz on the Jetson or at least try to optimize as much as possible (I am thinking to offload the frontend to the GPU maybe by using OpenCV with GPU).

If you could give me any hint on which direction I should take that would be very helpful.

Thanks a lot :)

Best,
Ilyass

零速修正对精度的提升大吗，零速修正只能对当前时刻和前一时刻进行约束，无法对累计误差进行修正。

TUM corridor1 的参数能共享一份嘛,邱博

TUM corridor1 的参数能共享一份嘛,邱博

Trying to get LARVIO to run in "realtime"

I have been attempting to modify the provided example codes to run "Real-time" with an imu and raspberry pi cam on a NVIDIA Jetson Nano dev board. So far I have managed to fill the imu buffer and gather imaged properly but after some digging in the code it is returning an error at "not enough features; move device around". The imu and camera are attached to the same device so their movement is correlated but it is not able to track features through the frames.

As there are no examples of how to get the algorithm running in "real time" I was wondering if anyone could help with how to structure the information and feed it to the algorithm correctly!

Thank you for any help you can provide, I can provide any information that would be helpful.

连续状态转移矩阵的离散化

您好 邱博！请教您一个公式推导问题。在连续的状态转移矩阵Phi中，式38,41,42,46,47是如何推导进行离散化的啊？能对其中某个进行讲解一下吗？十分感谢！

如何用larvio跑kaist urban dataset？

亲爱的邱博：
您试过用larvio跑kaist 数据集吗？我尝试了下，没有成功，一上来就发散得很快。

ZUPT

in zupt-update. the residual is about "prev - cur", but the jacobian is about " cur - prev", is there a mistake?

Hi， when I make the code, there is errors,

/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp: In member function ‘bool larvio::StaticInitializer::tryIncInit(const std::vectorlarvio::ImuData&, larvio::MonoCameraMeasurementPtr)’:
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:20:22: error: ISO C++ forbids declaration of ‘feature’ with no type [-fpermissive]
for (const auto& feature : img_msg->features)
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:20:32: warning: range-based ‘for’ loops only available with -std=c++11 or -std=gnu++11
for (const auto& feature : img_msg->features)
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:21:29: error: request for member ‘id’ in ‘feature’, which is of non-class type ‘const int’
init_features[feature.id] = Vector2d(feature.u, feature.v);
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:21:52: error: request for member ‘u’ in ‘feature’, which is of non-class type ‘const int’
init_features[feature.id] = Vector2d(feature.u, feature.v);
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:21:63: error: request for member ‘v’ in ‘feature’, which is of non-class type ‘const int’
init_features[feature.id] = Vector2d(feature.u, feature.v);
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:30:20: error: ISO C++ forbids declaration of ‘feature’ with no type [-fpermissive]
for (const auto& feature : img_msg->features) {
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:30:30: warning: range-based ‘for’ loops only available with -std=c++11 or -std=gnu++11
for (const auto& feature : img_msg->features) {
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:31:27: error: request for member ‘id’ in ‘feature’, which is of non-class type ‘const int’
curr_features[feature.id] = Vector2d(feature.u, feature.v);
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:31:50: error: request for member ‘u’ in ‘feature’, which is of non-class type ‘const int’
curr_features[feature.id] = Vector2d(feature.u, feature.v);
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:31:61: error: request for member ‘v’ in ‘feature’, which is of non-class type ‘const int’
curr_features[feature.id] = Vector2d(feature.u, feature.v);
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:32:36: error: request for member ‘id’ in ‘feature’, which is of non-class type ‘const int’
if (init_features.find(feature.id) != init_features.end()) {
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:33:32: error: request for member ‘u’ in ‘feature’, which is of non-class type ‘const int’
Vector2d vec2d_c(feature.u, feature.v);
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:33:43: error: request for member ‘v’ in ‘feature’, which is of non-class type ‘const int’
Vector2d vec2d_c(feature.u, feature.v);
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:34:48: error: request for member ‘id’ in ‘feature’, which is of non-class type ‘const int’

Question

邱博士：
您好！我有一个问题想要请教你，原谅我英文不好，所以选择中文交流。
我看到你在imu和图像的时间戳处理上，选择以imu时间作为基准，然后通过特征点坐标预测的方式得到当前时刻的视觉观测，我看到很多其他工作是选择以图像时间戳为基准，通过插值的方式得到图像时间对应的imu测量。我想问一下这两种方式有什么优缺点或者异同吗？
祝安

Question about measurement update

Hello,

Thanks for your great work!
I saw you update MSCKF feature/SLAM feature/new SLAM feature in one step. Will it perform better if we update these features step by step?

Best,
Oliver