uzh-rpg/flightmare

An Open Flexible Quadrotor Simulator

Flightmare - 左青龙

Flightmare is a flexible modular quadrotor simulator. Flightmare is composed of two main components: a configurable rendering engine built on Unity and a flexible physics engine for dynamics simulation. Those two components are totally decoupled and can run independently from each other. Flightmare comes with several desirable features: (i) a large multi-modal sensor suite, including an interface to extract the 3D point-cloud of the scene; (ii) an API for reinforcement learning which can simulate hundreds of quadrotors in parallel; and (iii) an integration with a virtual-reality headset for interaction with the simulated environment. Flightmare can be used for various applications, including path-planning, reinforcement learning, visual-inertial odometry, deep learning, human-robot interaction, etc.

Website & Documentation

Installation

Installation instructions can be found in our Wiki.

Updates

• 17.11.2020 Spotlight Talk at CoRL 2020
• 04.09.2020 Release Flightmare

Publication

If you use this code in a publication, please cite the following paper PDF

@inproceedings{song2020flightmare,
    title={Flightmare: A Flexible Quadrotor Simulator},
    author={Song, Yunlong and Naji, Selim and Kaufmann, Elia and Loquercio, Antonio and Scaramuzza, Davide},
    booktitle={Conference on Robot Learning},
    year={2020}
}

License

This project is released under the MIT License. Please review the License file for more details.

flightmare's Issues

no module flightgym (Unity Failed running in Virtual machine)

Hi Yunlong,

thanks for your excellent work with the Flightmare simulator which would be a good contribution to the realm of robotics.

btw, when I setting up with python after pip install, and try to run an example by 'python3 run_drone_control.py --train 1', it always reports there is no module 'flightgym'.

do you know the possible reason for that? thanks.

Failed building wheel for flightgym

 CMake Error: The current CMakeCache.txt directory /tmp/pip-req-build-or5x44y6/externals/pybind11/CMakeCache.txt is different than the directory /home/sysadmin/GitHub/flightmare/flightlib/externals/pybind11 where CMakeCache.txt was created. This may result in binaries being created in the wrong place. If you are not sure, reedit the CMakeCache.txt
  CMake Error at cmake/pybind11.cmake:16 (message):
    Download of Pybind11 failed: 1
  Call Stack (most recent call first):
    CMakeLists.txt:45 (include)

callibration matrix for unity

How would be able to extract callibration matrix from unity ?

Extract the 3D point-cloud of the scene

I would like to know how to perform the extraction of 3D point-cloud.

And thank you very much for sharing your simulator.

Unity Connection time out!

Thanks for your excellent work with the Flightmare simulator which would be a good contribution to the realm of robotics.
btw,I can't find the file rgb_flightmare_unity to make a connection when I installed unityhub

Non needed settings.json

Hi, can you remove https://github.com/uzh-rpg/flightmare/blob/master/flightrl/.vscode/settings.json? It seems to be catered to the machine of whoever committed this file. Thanks!

Where is documentation?

Hi,

The flightmare paper makes all sorts of very nice feature claims, most notably for me an API that allows access to ground truth point cloud and a collision-free motion planner. However I cannot find any documentation on how to use these two, or anything else beyond the basic usage with ROS. Can you point me to more extended information on how to get started there?

Documentation on Specifics of Env

Hi, for the RL example, would you be able to provide some documentation on the specifications of the observation space and action space (of the env)? Thanks!

Failed to build flightgym

Command errored out with exit status 1: /home/jiaping001/anaconda3/envs/Flightmare/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-vlc9823y/setup.py'"'"'; file='"'"'/tmp/pip-req-build-vlc9823y/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-r94opyvc/install-record.txt --single-version-externally-managed --compile --install-headers /home/jiaping001/anaconda3/envs/Flightmare/include/python3.6m/flightgym Check the logs for full command output.

error: ‘CV_RGB2BGR’ was not declared in this scope

I tried to build the new commit, the above error is showing.

I included the #include <opencv2/imgproc/types_c.h> in the unity_bridge.hpp and it builds without any error. Is it the right way to fix the error?

Flightmare Gym

Is it possible to use Flightmare's Gym interface without installing FlightRl (we would like to avoid the dependency to stable-baselines and specific tensorflow version you are using in this project)?

Is there any example how to do this?

Connect with custom trajectory planner results in emergency landing protocol.

Been racking my head around it for quite some time.
Just want to understand whats the logic behind the emergency landing protocol in flightmare so that I can edit it for my own use case.

Depth Image from unity causing the unity bridge connection to crash! EDIT: Need camera Calibration Matrix for depth image to point cloud generation

As per the code given in test/unity_bridge :

`
TEST(UnityBridge, HandleOutputDepth) {
  Logger logger{"Test HandleOutputDepth"};
  UnityBridge unity_bridge;
  QuadrotorDynamics dyn = QuadrotorDynamics(1.0, 0.2);
  std::shared_ptr<Quadrotor> quad = std::make_shared<Quadrotor>(dyn);
  std::shared_ptr<RGBCamera> rgb = std::make_shared<RGBCamera>();
  rgb->setPostProcesscing(std::vector<bool>{true, false, false});
  quad->addRGBCamera(rgb);
  unity_bridge.addQuadrotor(quad);

  EXPECT_TRUE(unity_bridge.connectUnity(UnityScene::GARAGE));

  FrameID frame_id = 1;
  unity_bridge.getRender(frame_id);
  bool handle_output = unity_bridge.handleOutput();

  EXPECT_TRUE(handle_output);

  cv::Mat test_img;

  EXPECT_TRUE(rgb->getRGBImage(test_img));
  EXPECT_TRUE(rgb->getDepthMap(test_img));
  EXPECT_FALSE(rgb->getSegmentation(test_img));
  EXPECT_FALSE(rgb->getOpticalFlow(test_img));

  // timeout flightmare
  usleep(5 * 1e6);
}
`

I followed a similar logic to build a publisher from my flight_pilot_node. However the unity bridge connection seems to crash after only 4-5 published messages. I even used image transporter to compress my messages and tried giving a rate with sufficient sleep duration to ensure that my data overflow was not causing issues. However, I haven't achieved much success till now.

Sharing my code edits here:

flight_pilot_node.cpp:

#include <ros/ros.h>
#include <sensor_msgs/Image.h>

#include "flightros/flight_pilot.hpp"

int main(int argc, char** argv) {
  ros::init(argc, argv, "flight_pilot");
  flightros::FlightPilot pilot(ros::NodeHandle(), ros::NodeHandle("~"));
  
  ros::Publisher pub_depth_image; // Added by Stavya bhatia

  // Initiate publisher for depth Image - Added by Stavya bhatia
  ros::NodeHandle n_;
  // image_transport::ImageTransport it(n_);
  pub_depth_image = n_.advertise<sensor_msgs::Image>("/sdepth_image_raw",1);

  while(ros::ok()){
    pilot.depthCamera(pub_depth_image);
    // spin the ros
    ros::spinOnce();
  }
  return 0;
}

flight_pilot.hpp

1. Added callback function
   void depthCamera(ros::Publisher msg);
2. Added depth camera variable in private:
   // Depth camera variables - cv::Mat depth_image; bool handle_output;

flight_pilot.cpp:
Function depthCamera added

void FlightPilot::depthCamera(ros::Publisher pub_depth_image){
    cv_bridge::CvImage img_bridge;

    handle_output = unity_bridge_ptr_->handleOutput();
    rgb_camera_->getDepthMap(depth_image);
    sensor_msgs::Image msg ;
    // img_bridge = cv_bridge::
    // msg = cv_bridge::CvImage(std_msgs::Header(), "bgr8", depth_image).toImageMsg();
    std_msgs::Header header; // empty header
    header.seq = 1; // user defined counter
    header.stamp = ros::Time::now(); // time
    img_bridge = cv_bridge::CvImage(header, sensor_msgs::image_encodings::RGB8, depth_image);
    img_bridge.toImageMsg(msg); // from cv_bridge to sensor_msgs::Image
    // pub_img.publish(msg); // ros::Publisher pub_img = node.advertise<sensor_msgs::Image>("topic", queuesize);
    pub_depth_image.publish(msg);
}

This is an extra time - sensitive task and would really appreciate if I can get some help in understanding if I am doing something wrong or is this a unity-ros issue that I cannot account for.

Regards

Use in own enviroment

I think the documentation is missing how we can implement this into our own environment.
Or is it even possible to have your own unity environment or are we set on the ones released from you?

Airsim instead of Unity

How about using Airsim instead of Unity?

Cannot connect the Unity when run python3 run_drone_control.py --train 0 --render 1

Loading a model without an environment, this model cannot be trained until it has a valid environment.

Question About PyTorch

I was curious if I could submit an example PR for using this environment with a PyTorch model, since there are some strong PyTorch RL libraries out there as well.

High level controller interface

Hi,
Is there a higher level abstraction to interact with the quad in the flight gym env? Rather than learning a policy that outputs concrete motor thrusts, I would like to produce a policy that outputs velocities or waypoints that can be flown by the quad model.

On a side note, I am quite new to RL and am worried that I will end up learning the dynamics of the quad rather than the policy that fulfils the concrete objective I have in mind. Is this a reasonable assumption?

Ubuntu 20.04 and OpenCV 4.2 compilation issues

To install the flightmare simulator I precisely followed the steps mentioned on https://github.com/uzh-rpg/flightmare/wiki/Install-with-ROS on a fresh Ubuntu 20.04 install. However, I ran into a four problems which I'll detail below along with my fixes for them.

First Issue: ROS Dependencies
In Ubuntu 20.04 there is no python-vcstool to install. It needs to be `python3-vcstool

Second Issue: OpenCV 4.2
The current version of OpenCV is 4.2. In the flightmare/flightlib/src/bridges/unity_bridge.cpp file, line 237 the call to cvtColor gives a compile-time error because CV_RGB2BGR is not defined in OpenCV 4.2. According to https://docs.opencv.org/4.2.0/d8/d01/group__imgproc__color__conversions.html it needs to be cv::COLOR_RGB2BGR.

Third Issue: Python Syntax Error
In line 53 of the file rotors_simulator/rotors_evaluation/src/rosbag_tools/analyze_bag.py the python syntax except Exception, e is not recognized when building the package. It works if the line is changed to except Exception as e

Fourth Issue: OpenMP
Errors << flightros:make /home/bfd/catkin_ws/logs/flightros/build.make.000.log /usr/bin/ld: /home/bfd/catkin_ws/install/lib/libflightlib.so: undefined reference to 'omp_set_num_threads' collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/flight_pilot_node.dir/build.make:181: /home/bfd/catkin_ws/devel/.private/flightros/lib/flightros/flight_pilot_node] Error 1
make[1]: *** [CMakeFiles/Makefile2:241: CMakeFiles/flight_pilot_node.dir/all] Error 2 make: *** [Makefile:141: all] Error 2

It seems to like the -fopenmp flag is missing in the flightmare/flightros/CMakeLists.txt. Adding a -fopenmp flag to line 29 resolved the issue and catkin build runs without any errors.

Building components into the warehouse environment of unity

I believe the rpg_flightmare_unity repository is responsible for this. Can i get some idea on if its still possible to do it without it and how I can do it. Alternatively, can I get an idea when the repository should be up again?

How to add gates to Flightmare?

Hello,

I would like to add gates to Flightmare just like to ones in the youtube video. Could you please tell me how to do that? I don't have any experience in Unity but I can learn.

Thank you,

How to get data from flightmare sensors?

Hello,

First of all, Thank you for sharing this simulator. We really need it!

How to get sensors data from ROS? I have seen flight_pilot.cpp where an RGB camera is added. However, there are no topics in ROS to subscribe to.

Thank you,

Few wiki typos

This is very minor but I can't make a PR for it so I'll just list them here

In Home.md

• Uasge (2x)

In Basic-Usage-with-Python.md

• exectuable

In Known-Issues.md

• Soluation

How to fix the camera on the quadrotor？

Hello,

Yunlong, thank you for sharing this simulator.
I have seen flight_pilot.cpp where an RGB camera is added, and I tried to publish the RGB_image like the example in the test folders. But the camera can not move with the quadrotor, it seems like the pose of the camera is stationary but not fixed on the quadrotor. I would like to know how to fix the camera on the quadrotor？
I run Flightmare according to the Basic Usage with ROS in the wiki.

Thank you

Dynamic gates?

Hi!

Is it possible to spawn dynamic gates (or dynamic obstacles in general) in Flightmare? I've only found this

Thanks!

Detect collision between drones during Reinforcement Learning

Hello
I am trying to train a controller using flightrl for multiple drones by setting n_envs = x in the configuration for getting x drones. I was wondering if there is a way for the RL environment to be able to detect collisions between these drones since right now they can all overlap each other.
Is there some place in the code I can make this change so that the episode can terminate when there is a collision between 2 drones?

Thank you

Depth image problem in the forest environment

Hello,
In the forest environment, the ground seems to be removed in the depth image as we can see on this image :

Is it possible to correct this problem (in Unity I suppose) ?

Catkin build failed during flightros:make

System: Ubuntu 18.04
Installation with ROS failed.

Error 1:

...src/flightmare/flightlib/src/dynamics/quadrotor_dynamics.cpp:95:10: error: ‘clamp’ is not a member of ‘std’
   return std::clamp(thrust, thrust_min_, thrust_max_);

Replaced it std::clamp(thrust, thrust_min_, thrust_max_) with std::min(thrust_max_, std::max(thrust_min_, thrust)) temporarily to fix this issue.

Error 2:

Errors     << flightros:make .../logs/flightros/build.make.003.log
.../devel/lib/libflightlib.so: undefined reference to `flightlib::Logger::YELLOW'
.../devel/lib/libflightlib.so: undefined reference to `flightlib::Logger::WARN'
.../devel/lib/libflightlib.so: undefined reference to `flightlib::Logger::INFO'
.../devel/lib/libflightlib.so: undefined reference to `flightlib::Logger::RED'
.../devel/lib/libflightlib.so: undefined reference to `flightlib::Logger::ERROR'
.../devel/lib/libflightlib.so: undefined reference to `flightlib::Logger::RESET'
collect2: error: ld returned 1 exit status

Any help is appreciated. Thank you.

Import error when python3 run_drone_control.py --train 1 in Ubuntu 20.04

Hi.

I am trying to run the fligtmare with python3 run_drone_control.py --train 1 in flightmare/flightrl/examples. However, I got the import error. The detail of the error is here:

I am trying to look for cuda-9 in my /usr/local but it is nowhere.

May I know how to solve this problem?

Unable to vcs-import < flightros/dependencies.yaml

I am fairly new to git. I read this could be related to SSH key authentication. If that is the case how can I add my SSH key for vs import

Not able to Catkin build

virtualpullu@virtual-PulluBox:~/catkin_ws/src$ catkin build

Profile: default
Extending: [explicit] /opt/ros/melodic
Workspace: /home/virtualpullu/catkin_ws

Build Space: [exists] /home/virtualpullu/catkin_ws/build
Devel Space: [exists] /home/virtualpullu/catkin_ws/devel
Install Space: [unused] /home/virtualpullu/catkin_ws/install
Log Space: [missing] /home/virtualpullu/catkin_ws/logs
Source Space: [exists] /home/virtualpullu/catkin_ws/src
DESTDIR: [unused] None

Devel Space Layout: merged
Install Space Layout: None

Additional CMake Args: -DCMAKE_BUILD_TYPE=Release
Additional Make Args: None
Additional catkin Make Args: None
Internal Make Job Server: True
Cache Job Environments: False

Whitelisted Packages: None
Blacklisted Packages: None

Workspace configuration appears valid.

NOTE: Forcing CMake to run for each package.

[build] Found '35' packages in 0.0 seconds.
[build] Updating package table.
Starting >>> catkin_simple
Finished <<< catkin_simple [ 2.4 seconds ]
Starting >>> mav_msgs

---

Errors << mav_msgs:cmake /home/virtualpullu/catkin_ws/logs/mav_msgs/build.cmake.000.log
CMake Error at /opt/ros/melodic/share/catkin/cmake/catkinConfig.cmake:83 (find_package):
Could not find a package configuration file provided by "trajectory_msgs"
with any of the following names:

trajectory_msgsConfig.cmake
trajectory_msgs-config.cmake

Add the installation prefix of "trajectory_msgs" to CMAKE_PREFIX_PATH or
set "trajectory_msgs_DIR" to a directory containing one of the above files.
If "trajectory_msgs" provides a separate development package or SDK, be
sure it has been installed.
Call Stack (most recent call first):
CMakeLists.txt:4 (find_package)

cd /home/virtualpullu/catkin_ws/build/mav_msgs; catkin build --get-env mav_msgs | catkin env -si /usr/bin/cmake /home/virtualpullu/catkin_ws/src/mav_comm/mav_msgs --no-warn-unused-cli -DCATKIN_DEVEL_PREFIX=/home/virtualpullu/catkin_ws/devel -DCMAKE_INSTALL_PREFIX=/home/virtualpullu/catkin_ws/install -DCMAKE_BUILD_TYPE=Release; cd -
....................................................................................................
Failed << mav_msgs:cmake [ Exited with code 1 ]
Failed <<< mav_msgs [ 2.3 seconds ]
Abandoned <<< mav_state_machine_msgs [ Unrelated job failed ]
Abandoned <<< mav_system_msgs [ Unrelated job failed ]
Abandoned <<< rotors_comm [ Unrelated job failed ]
Abandoned <<< rotors_description [ Unrelated job failed ]
Abandoned <<< rotors_evaluation [ Unrelated job failed ]
Abandoned <<< rqt_rotors [ Unrelated job failed ]
Abandoned <<< eigen_catkin [ Unrelated job failed ]
Abandoned <<< flightlib [ Unrelated job failed ]
Abandoned <<< flightrender [ Unrelated job failed ]
Abandoned <<< quad_launch_files [ Unrelated job failed ]
Abandoned <<< quadrotor_msgs [ Unrelated job failed ]
Abandoned <<< rpg_single_board_io [ Unrelated job failed ]
Abandoned <<< mav_planning_msgs [ Unrelated job failed ]
Abandoned <<< quadrotor_common [ Unrelated job failed ]
Abandoned <<< polynomial_trajectories [ Unrelated job failed ]
Abandoned <<< position_controller [ Unrelated job failed ]
Abandoned <<< rotors_control [ Unrelated job failed ]
Abandoned <<< rotors_gazebo_plugins [ Unrelated job failed ]
Abandoned <<< rotors_gazebo [ Unrelated job failed ]
Abandoned <<< rotors_hil_interface [ Unrelated job failed ]
Abandoned <<< rotors_joy_interface [ Unrelated job failed ]
Abandoned <<< rpg_rotors_interface [ Unrelated job failed ]
Abandoned <<< rqt_quad_gui [ Unrelated job failed ]
Abandoned <<< sbus_bridge [ Unrelated job failed ]
Abandoned <<< manual_flight_assistant [ Unrelated job failed ]
Abandoned <<< state_predictor [ Unrelated job failed ]
Abandoned <<< trajectory_generation_helper [ Unrelated job failed ]
Abandoned <<< autopilot [ Unrelated job failed ]
Abandoned <<< flightros [ Unrelated job failed ]
Abandoned <<< rpg_quadrotor_integration_test [ Unrelated job failed ]
Abandoned <<< vbat_thrust_calibration [ Unrelated job failed ]
[build] Summary: 1 of 33 packages succeeded.
[build] Ignored: 2 packages were skipped or are blacklisted.
[build] Warnings: None.
[build] Abandoned: 31 packages were abandoned.
[build] Failed: 1 packages failed.
[build] Runtime: 5.0 seconds total.
[build] Note: Workspace packages have changed, please re-source setup files to use them.

How to get depth images for gym-style RL

I'm trying to train a policy to control a quadcopter in parallel using FlightEnvVec similarly to run_drone_control. But I want to add depth image pixels as inputs (observations) to a policy.

How can I add depth images to observations and get observations with depth images from the python side for training? (I'm not using Gazebo, for parallelism.)

Thanks!

Segmentation fault (core dumped) error in Ubuntu 20.04

If you compile flightlib using default gcc version 9.3 provided with Ubuntu 20.04 it gives this error. To make it work it needs to be compiled using gcc 7 that needs to be manually selected in 20.04. An example procedure is given here.

Multiple Quadrotors

I am using flightmare with ROS and gazebo. What changes are to be made to launch multiple quadrotors in the flightmare environment and extract pointcloud and camera data?

Also,I have a few questions regarding the flightmare simulator:

1. How to added Lidar and camera capabilities to the rpg quadrotor and extract the data in the flightmare environment? Also how to add obstacles to the flightmare scene?
2. What is the purpose of using flightmare with ROS and gazebo, why can't we directly use ROS with flightmare unity?
3. Is it possible to launch mobile wheeled robots( eg. Clearpath Husky robots) in the flightmare simulator?

Thanks!

How many cameras it can support?

Problem completing Basic usage with ROS

Hi!

I've done all the installation steps as suggested by the guide "Basic usage with ROS", however I couldn't launch the file rotors_gazebo.launch (the launch was successful but the Flightmare gui wasn't able to connect to anything).. I've tried to launch rotors_gazebo_test.launch and it worked.

By commenting piece by piece of the rotors_gazebo.launch launch file, I've figured out that the problem was about the Gazebo nodes and the line https://github.com/uzh-rpg/flightmare/blob/master/flightros/launch/rotors_gazebo.launch#L27.

Is this line of code necessary for any purpose?

Build FlightGym error

I got this error (Ubuntu 16.04):
/tmp/pip-req-build-t9aaza3_/include/flightlib/bridges/unity_bridge.hpp:12:27: fatal error: zmqpp/zmqpp.hpp: No such file or directory.

What I should install to solve this?

Will Unity files be provided?

I want to use make some changes in the scene using Unity Editor, but it seems like only binary files are provided. Can you also share Unity files & scripts so that we can change the scene depending on our application? Thanks.

subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j4']' returned non-zero exit status 2.

I'm getting this error when calling pip install . in the flightlib directory (after installing all prerequisites). Any ideas?
Full log:

ERROR: Command errored out with exit status 1:
command: /home/guy/Projects/venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/guy/Projects/flightmare/flightlib/setup.py'"'"'; file='"'"'/home/guy/Projects/flightmare/flightlib/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps
cwd: /home/guy/Projects/flightmare/flightlib/
Complete output (162 lines):
running develop
running egg_info
writing flightgym.egg-info/PKG-INFO
writing dependency_links to flightgym.egg-info/dependency_links.txt
writing requirements to flightgym.egg-info/requires.txt
writing top-level names to flightgym.egg-info/top_level.txt
reading manifest file 'flightgym.egg-info/SOURCES.txt'
writing manifest file 'flightgym.egg-info/SOURCES.txt'
running build_ext
-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- ====================== !Flightmare! ======================
-- ======> Setup Dependencies
-- No sufficient Eigen version (3.3.4) found.
-- Restoring to download Eigen sources.
-- Getting Eigen...
Cloning into 'eigen3'...
Already on 'master'
-- Eigen downloaded!
-- Eigen3 include dir:
-- Getting Pybind11...
Cloning into 'pybind11-src'...
Already on 'master'
-- Pybind11 downloaded!
-- pybind11 v2.6.1 dev1
-- Found PythonInterp: /home/guy/Projects/venv/bin/python3 (found version "3.8.2")
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.8.so
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Getting yaml-cpp...
-- Yaml downloaded!
-- Found OpenCV: /usr (found version "4.2.0")
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
INFOBuild time could be improved with ccache!
INFO sudo apt install ccache
-- ======> Setup Compilation
-- Enabling fast optimization flags!
-- The activated CXX RELEASE configuration is:
-DVERSION_INFO="0.0.1" -fPIC -Wall -DNDEBUG -fopenmp -Ofast -march=native
-- The activated CXX DEBUG configuration is:
-DVERSION_INFO="0.0.1" -fPIC -Wall -g -fopenmp
-- ======> Setup Build
-- Getting gtests...
Cloning into 'googletest-src'...
Already on 'master'
-- gtests downloaded!
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- ================ !Done. No more nightmare! ================
-- Configuring done
-- Generating done
-- Build files have been written to: /home/guy/Projects/flightmare/flightlib/build/temp.linux-x86_64-3.8
Scanning dependencies of target gtest
Scanning dependencies of target yaml-cpp
[ 1%] Building CXX object ../../externals/googletest-build/googletest/CMakeFiles/gtest.dir/src/gtest-all.cc.o
[ 2%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/contrib/graphbuilderadapter.cpp.o
[ 5%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/contrib/graphbuilder.cpp.o
[ 5%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/binary.cpp.o
[ 6%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/convert.cpp.o
[ 8%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/depthguard.cpp.o
[ 9%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/directives.cpp.o
[ 10%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/emit.cpp.o
[ 12%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/emitfromevents.cpp.o
[ 13%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/emitter.cpp.o
[ 14%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/emitterstate.cpp.o
[ 16%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/emitterutils.cpp.o
[ 17%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/exceptions.cpp.o
[ 18%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/exp.cpp.o
[ 20%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/memory.cpp.o
[ 21%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/node.cpp.o
[ 22%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/node_data.cpp.o
[ 24%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/nodebuilder.cpp.o
[ 25%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/nodeevents.cpp.o
[ 27%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/null.cpp.o
[ 28%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/ostream_wrapper.cpp.o
[ 29%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/parse.cpp.o
[ 31%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/parser.cpp.o
[ 32%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/regex_yaml.cpp.o
[ 33%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/scanner.cpp.o
[ 35%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/scanscalar.cpp.o
[ 36%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/scantag.cpp.o
[ 37%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/scantoken.cpp.o
[ 39%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/simplekey.cpp.o
[ 40%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/singledocparser.cpp.o
[ 41%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/stream.cpp.o
[ 43%] Building CXX object ../../externals/yaml-build/CMakeFiles/yaml-cpp.dir/src/tag.cpp.o
[ 44%] Linking CXX static library libyaml-cpp.a
[ 44%] Built target yaml-cpp
Scanning dependencies of target flightlib
[ 45%] Building CXX object CMakeFiles/flightlib.dir/src/common/command.cpp.o
[ 47%] Building CXX object CMakeFiles/flightlib.dir/src/common/integrator_base.cpp.o
[ 48%] Building CXX object CMakeFiles/flightlib.dir/src/bridges/unity_bridge.cpp.o
[ 50%] Linking CXX static library ../../../build/temp.linux-x86_64-3.8/lib/libgtest.a
[ 50%] Built target gtest
[ 51%] Building CXX object CMakeFiles/flightlib.dir/src/common/integrator_euler.cpp.o
[ 52%] Building CXX object CMakeFiles/flightlib.dir/src/common/integrator_rk4.cpp.o
[ 54%] Building CXX object CMakeFiles/flightlib.dir/src/common/logger.cpp.o
[ 55%] Building CXX object CMakeFiles/flightlib.dir/src/common/math.cpp.o
[ 56%] Building CXX object CMakeFiles/flightlib.dir/src/common/parameter_base.cpp.o
/home/guy/Projects/flightmare/flightlib/src/bridges/unity_bridge.cpp: In member function ‘bool flightlib::UnityBridge::handleOutput()’:
/home/guy/Projects/flightmare/flightlib/src/bridges/unity_bridge.cpp:237:46: error: ‘CV_RGB2BGR’ was not declared in this scope; did you mean ‘CV_RGB’?
237 | cv::cvtColor(new_image, new_image, CV_RGB2BGR);
| ^~~~~~~~~~
| CV_RGB
[ 58%] Building CXX object CMakeFiles/flightlib.dir/src/common/pend_state.cpp.o
[ 59%] Building CXX object CMakeFiles/flightlib.dir/src/common/quad_state.cpp.o
make[2]: *** [CMakeFiles/flightlib.dir/build.make:63: CMakeFiles/flightlib.dir/src/bridges/unity_bridge.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
Scanning dependencies of target gtest_main
[ 60%] Building CXX object ../../externals/googletest-build/googletest/CMakeFiles/gtest_main.dir/src/gtest_main.cc.o
[ 62%] Linking CXX static library ../../../build/temp.linux-x86_64-3.8/lib/libgtest_main.a
[ 62%] Built target gtest_main
make[1]: *** [CMakeFiles/Makefile2:321: CMakeFiles/flightlib.dir/all] Error 2
make: *** [Makefile:141: all] Error 2
Traceback (most recent call last):
File "", line 1, in
File "/home/guy/Projects/flightmare/flightlib/setup.py", line 94, in
setup(
File "/home/guy/Projects/venv/lib/python3.8/site-packages/setuptools/init.py", line 145, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/guy/Projects/venv/lib/python3.8/site-packages/setuptools/command/develop.py", line 38, in run
self.install_for_development()
File "/home/guy/Projects/venv/lib/python3.8/site-packages/setuptools/command/develop.py", line 140, in install_for_development
self.run_command('build_ext')
File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/guy/Projects/flightmare/flightlib/setup.py", line 59, in run
self.build_extension(ext)
File "/home/guy/Projects/flightmare/flightlib/setup.py", line 90, in build_extension
subprocess.check_call(['cmake', '--build', '.'] +
File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j4']' returned non-zero exit status 2.
----------------------------------------
ERROR: Can't roll back flightgym; was not uninstalled
ERROR: Command errored out with exit status 1: /home/guy/Projects/venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/guy/Projects/flightmare/flightlib/setup.py'"'"'; file='"'"'/home/guy/Projects/flightmare/flightlib/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output

Abandoned: 15 packages were abandoned.

I followed the installation instructions https://github.com/uzh-rpg/flightmare/wiki/Install-with-ROS
but during the catkin build i had the attached errors.

I have to point out that 15 packages were abandoned and 1 package failed.
What are the problems? What is missing?
Also i run: rosdep install --from-paths src --ignore-src -r -y
and #All required rosdeps installed succesfully

Thank you very much for the help.

How get the pointcloud

Hello,

I could see that you update the new version but I couldn't find how get the pointcloud in the documentation.

Regards

Is there a way to replace the hummingbird with a custom urdf quadrotor ?

Hi,

On first looks, I noticed that the entire quadrotor dynamics is built on c++ files. Is it also possible to do the same with a hexacopter URDF?

In a nutshell Id like to use flightmare with a DJI M600 pro (hexacopter)

Apologies in advance, I am fairly new and still only just figuring the nuances of ROS.

Windows Support

Hi! Is a Windows relase of the UI (RPG_Flightmare.x86_64) on the roadmap?

Also, looking at the Unity Developer Guide on the Wiki suggests that any Unity-related code (e.g. the rendering engine and also the C# scripts mentioned in the paper) will soon be made available.

Do you have any prediction of when rpg_flightmare_unity should be released then? As this would most likely allow us to build a Windows executable ourselves.

catkin build fails

I was following the instructions on the wiki to install fligthmare with ROS, but when I try to build it catkin build fails.
I get the following error message:

I've tried to edit the PYTHONPATH, but then it messes up my pip installation.

After trying couple of different solutions I have ended up removing my build and devel folder and I ran catkin_make.
catkin_make was able to build my packages without any problems.

I think this tip might be handy for other users as well. Would you consider adding it to your documentation?
Thanks!

Unable to understand PCL data extraction

Hi, I followed the example code - unity_bridge in tests on other issues, but am still not clear on how can I create a node that subscribes and receives pcl data. I am trying to build a global path planner system. Any direction would be much appreciated. Thanks and sorry for multiple new issues!

Depth image rendering format results in information loss

Hello,

In using the simulators depth image rendering pipeline. I noticed that by default the images are stored in 8UC3 format. Working on the pipeline to control using depth images, I realised most depth cameras work with 16 bit images. There also seems to some sense of information loss with 8 bits when I integrate it with depth cloud to point cloud nodes. So I wanted to understand if there was any particular reason why the images rendered are in 8 bit format, or if there is any way we can already get them in 16 bit format from unity.

Regards

ROS Tutorials

Could you please create some tutorials on creating new environments in flightmare e.g. urban areas, creating python scripts to control the movement of the UAV, etc?

I get the following error when I do catkin_make in catkin_ws file after saving flightmare to the ruined catkin_ws / src directory. Can you help me?

CMake Error at flightmare/flightlib/cmake/catkin.cmake:2 (find_package):
By not providing "Findcatkin_simple.cmake" in CMAKE_MODULE_PATH this
project has asked CMake to find a package configuration file provided by
"catkin_simple", but CMake did not find one.

Could not find a package configuration file provided by "catkin_simple"
with any of the following names:

catkin_simpleConfig.cmake
catkin_simple-config.cmake

Add the installation prefix of "catkin_simple" to CMAKE_PREFIX_PATH or set
"catkin_simple_DIR" to a directory containing one of the above files. If
"catkin_simple" provides a separate development package or SDK, be sure it
has been installed.
Call Stack (most recent call first):
flightmare/flightlib/CMakeLists.txt:176 (include)

-- Configuring incomplete, errors occurred!
See also "/home/cansel/catkin_ws/build/CMakeFiles/CMakeOutput.log".
See also "/home/cansel/catkin_ws/build/CMakeFiles/CMakeError.log".
Makefile:4324: recipe for target 'cmake_check_build_system' failed
make: *** [cmake_check_build_system] Error 1
Invoking "make cmake_check_build_system" failed

Asking for a help about manual control

I'm new to this stuffs. Can you send me a sample shortcode to control of this drone in python please?

Problem completing the Basic Usage with ROS

I have successfully made catkin build my workspace (only one workspace for the flightmare simulator) and i try to complete the Basic Usage with ROS.
I downloaded RPG_Flightmare, i extracted it into the /path/to/flightmare/flightrender and run roslaunch flightros rotors_gazebo.launch as indicated.
Therefore, i have and Error in REST request and also the Unity Connection time out!

Is there a proper way of extracting the contents of RPG_Flightmare? Should i change something or the extracted just has to be inside the flightrender directory?
Is there anothe problem?
Also if i run: roslaunch flightros rotors_gazebo_test.launch
then i have the error below:

Looking forward for any help.
Thank you very much.