microsoft/airsim-drone-racing-lab

A framework for drone racing research, built on Microsoft AirSim.

AirSim Drone Racing Lab

Quickstart

• Linux and Windows Binaries
• Python API:
  • pip install airsimdroneracinglab
  • Documentation

Note: If you use this repository in your research, please cite our pre-print, AirSim Drone Racing Lab.

@article{madaan2020airsim,
  title={AirSim Drone Racing Lab},
  author={Madaan, Ratnesh and Gyde, Nicholas and Vemprala, Sai and Brown, Matthew and Nagami, Keiko and Taubner, Tim and Cristofalo, Eric and Scaramuzza, Davide and Schwager, Mac and Kapoor, Ashish},
  journal={arXiv preprint arXiv:2003.05654},
  year={2020}
}

Getting Binaries

• Linux
• Windows
• Docker on Linux

Using AirSim Drone Racing Lab

• AirSim Drone Racing Lab Overview
• API Overview
• Baselines
• Race Monitoring
• Documentation

Questions

Please open a Github Issue on this repository (not AirSim) for any technical questions associated with AirSim Drone Racing Lab.

airsim-drone-racing-lab's Issues

Why won't it generate json file when I run "generate_settings_file.py" as instructed ?

where can I enable OpenGL of the airsim

Hi, I'm trying to run the ADRL in a remote Linux Server.

According to microsoft/AirSim#2139 I have to use the virtualGL and TurtleVNC since the Vulkan will not work over X11 forwarding

After configuration, I start the ADRL by command
DISPLAY=:8 vglrun -d :7.0 ./ADRL.sh -opengl4

The program can start but with a pop-up error:

  Trying to force OpenGL RHI but the project does not have it in TargetedRHIs list.

All searching asks me to modify the engine setting to enable TargetedRHIs=GLSL_430:
https://answers.unrealengine.com/questions/920624/view.html
https://answers.unrealengine.com/questions/936174/view.html

but I cannot find a BaseEngine.ini in the folder ADRL I wgeted.

Is there any way to force ADRL to run with OpenGL?

Cannot execute through docker

Hello,

I follow the instruction trying to execute ADRL through docker, but don't get success.

my environment

• Ubuntu 20.04
• Docker 19.03
• nvidia-container-toolkit installed
• GPU : RTX2070

I modified run_docker_image.sh to fit my docker version

docker run -it \
    --gpus all \
   ....
   ....

When I execute run_docker_image.sh, I will get the error

Trying to force OpenGL RHI but the project does not have it in TargetedRHIs list.

So I try to change the base image to nvidia/vulkan, remove the parameter -opengl in run_docker_image.sh.

When I execute run_docker_image.sh, a black window will pop and terminate immediately.

And the following is the log

LogVulkanRHI: Error: VulkanRHI::vkGetPhysicalDeviceSurfaceFormatsKHR(Device.GetPhysicalHandle(), Surface, &NumFormats, nullptr) failed, VkResult=-3
 at /home/ue4/ue_build/Engine/Source/Runtime/VulkanRHI/Private/VulkanSwapChain.cpp:165 
 with error VK_ERROR_INITIALIZATION_FAILED
Failed to find symbol file, expected location:
"/home/airsim_user/ADRL/ADRL/ADRL/Binaries/Linux/ADRL.sym"
Fatal error: [File:/home/ue4/ue_build/Engine/Source/Runtime/VulkanRHI/Private/VulkanUtil.cpp] [Line: 786] 
VulkanRHI::vkGetPhysicalDeviceSurfaceFormatsKHR(Device.GetPhysicalHandle(), Surface, &NumFormats, nullptr) failed, VkResult=-3
 at /home/ue4/ue_build/Engine/Source/Runtime/VulkanRHI/Private/VulkanSwapChain.cpp:165 
 with error VK_ERROR_INITIALIZATION_FAILED
[2020.09.03-03.01.16:267][  0]LogCore: Error: appError called: Fatal error: [File:/home/ue4/ue_build/Engine/Source/Runtime/VulkanRHI/Private/VulkanUtil.cpp] [Line: 786] 
VulkanRHI::vkGetPhysicalDeviceSurfaceFormatsKHR(Device.GetPhysicalHandle(), Surface, &NumFormats, nullptr) failed, VkResult=-3
 at /home/ue4/ue_build/Engine/Source/Runtime/VulkanRHI/Private/VulkanSwapChain.cpp:165 
 with error VK_ERROR_INITIALIZATION_FAILED
0x00000000056703ab ADRL!UnknownFunction(0x54703aa)
0x0000000005641adc ADRL!UnknownFunction(0x5441adb)
0x0000000005671d78 ADRL!UnknownFunction(0x5471d77)
0x0000000005671b8c ADRL!UnknownFunction(0x5471b8b)
0x0000000005676ee8 ADRL!UnknownFunction(0x5476ee7)
0x0000000005b17c0d ADRL!UnknownFunction(0x5917c0c)
0x00000000040237c6 ADRL!UnknownFunction(0x3e237c5)
0x0000000006558700 ADRL!UnknownFunction(0x63586ff)
0x000000000606403f ADRL!UnknownFunction(0x5e6403e)
0x0000000002d44012 ADRL!UnknownFunction(0x2b44011)
0x0000000002d3a3aa ADRL!UnknownFunction(0x2b3a3a9)
0x000000000765e11b ADRL!UnknownFunction(0x745e11a)
0x00007f20538a4b97 libc.so.6!__libc_start_main(+0xe6)
0x0000000002d2e029 ADRL!UnknownFunction(0x2b2e028)


Signal 11 caught.
Malloc Size=65538 LargeMemoryPoolOffset=65554

I have no idea how to handle this situation.

moveOnSpline and moveOnSplineAsync with custom environments

Hi,

Please let me know if this isn't the appropriate place for this issue. I'm trying to use moveOnSpline with custom environments but I think that API is only available in binaries available in this repository. Is that correct and if so, how can I get that API functionality to custom environments?

Drones spawn in white box for race...

Hello I can't seem to overcome 'the white box' when a drone race starts. Finicking around with basic commands I realized the multirotor would sometimes spawn in a white box when on the soccer field map. I had to backspace to reset the drones position and I was able to play with it. Whenever I replay the sample race script, baseline_racer.py, the environment resets and the level reloads putting me back in the white box. How do I deal with this white box? Is there any way I can delete it? :)

MultiRotorClient has no attribute race_tier

Hello,

I am upgrading the Gym environment that I made during the 2019 competition to ADRL, and I found this bug.

When calling simGetObjectPause() before simStartRace(), there is an error because race_tier is not in initialized:

File "C:\Users\Yann\Desktop\git\gym_game_of_drones\gym_game_of_drones\envs\multi_agent\gym_airsimdroneracinglab\rewardfunction.py", line 192, in get_ground_truth_gate_poses_and_half_dims
    p = self.airsim_client.simGetObjectPose(object_name=gate_name)
  File "C:\Users\Yann\anaconda3\envs\adrl\lib\site-packages\airsimdroneracinglab\client.py", line 411, in simGetObjectPose
    if self.race_tier is None:
AttributeError: 'MultirotorClient' object has no attribute 'race_tier'
Exception ignored in: <function GameOfDatasets.__del__ at 0x000001962260B3A0>
Traceback (most recent call last):
  File "C:\Users\Yann\Desktop\git\gym_game_of_drones\gym_game_of_drones\dataset_collector_game.py", line 495, in __del__
    self.env.stop()
AttributeError: 'GameOfDatasets' object has no attribute 'env'

Process finished with exit code 1

Edit the order of the gates

Hello,

My teammates and I are trying to implement our planning algorithm and we're wondering if there is a way for us to edit the order that we can pass through the gates?

Thank you!

Where can I find the baseline gate detection solution for Tier II?

Hi ADRL team,

I would like to know where can I find the baseline gate detection solution for Tier II race?

I have read through the preprint paper at https://arxiv.org/abs/2003.05654, and it mentions there is a baseline implementation for gate detection using recursive filtering algorithm in Section 4, but I cannot find the code in this repository. We are interested in the safety analysis of the closed loop system w.r.t the accuracy of the vision-based perception system, and it will very helpful if we can access the reference implementation.

Thanks,
Chiao Hsieh
PhD student, University of Illinois at Urbana-Champaign

Implementation on Pixhawk

Hello,

I am doing a project on drone racing. I want to setup the following combination:
AirSim + PX4 SITL + MAVROS + Drone Racing LAB.

So the idea is to use:

Airsim as the simulator;
Drone racing lab as the environment (ADRL unreal environment);
PX4 as the flight controller;
MAVROS to read sensor data from Airsim to PX4; publish attitude setpoints to PX4.
I already setup PX4, Airsim and MAVROS. I can tell the drone to land, arm and takeoff via MAVROS and see the results on AirSim using the default blocks environment. Now i want to do it on the actual drone racing environment. I tried to combine the settings.json of both PX4 and Drone racing lab, but when i launch the ADRL unreal environment, keeps saying fatal error.

How can i do this? Is it even possible?

This is the log file.
ADRL.log

Is the API and/or binary sources available somewhere?

Hello,

I am fairly new to AirSim, sorry if this is the wrong place to ask this question. This project looks outstanding, and I would like to do some research that involves some aspects of what appears to be demonstrated here.

Particularly I have my own method of generating spline trajectories in the form, x(t) = ... y(t) = ... z(t) = ... and also for higher derivatives such as velocity.

I would like to implement something like the moveOnSplineAsync function but with my own trajectory generator, is the source for the environments or the API that relates to this function available somewhere? I was hoping to use it as a kind of template. I would of course credit this project if my research were ever to be published

To clarify what I meant by API, I understand where pip has installed 'client.py' that contains the API calls, but for example the moveOnSpline function simply returns this:

return self.client.call_async(
            "moveOnSpline",
            waypoints,
            add_position_constraint,
            add_velocity_constraint,
            add_acceleration_constraint,
            vel_max,
            acc_max,
            viz_traj,
            viz_traj_color_rgba,
            replan_from_lookahead,
            replan_lookahead_sec,
            vehicle_name,
        )

But I am unsure where that call is actually being implemented/how it is implemented

Thank you

*Edited to clarify

How to add dynamic objects to the existing environment?

Hi Guys,

I want to add some dynamic objects, such as moving birds or people, to the existing ADRL environment? Can you provide some suggestions for me about how to modify the current environment?

Thanks and regards!

Is there anyway to get rotor state??

Hello, I'm trying to implement Reinforcement learning algorithm in this project.

I want to get rotor states. I know there is getRotorStates() in airsim. However, I can't find in this project.

def getRotorStates(self, vehicle_name =""):
    """
    Used to obtain the current state of all a multirotor's rotors. The state includes the speeds,
    thrusts and torques for all rotors.

    Args:
        vehicle_name (str, optional): Vehicle to get the rotor state of

    Returns:
        RotorStates: Containing a timestamp and the speed, thrust and torque of all rotors.
    """
    return RotorStates.from_msgpack(self.client.call('getRotorStates', vehicle_name))
getRotorStates.__annotations__ = {'return': RotorStates}

I copied above code in airsim package's client.py to airsimdroneracing package's client.py.
Then, under log comes out.
File "C:\Users\KSJ\anaconda3\envs\new_airsim\lib\site-packages\airsimdroneracinglab\client.py", line 1911, in getRotorStates
return RotorStates.from_msgpack(self.client.call('getRotorStates', vehicle_name))
File "C:\Users\KSJ\anaconda3\envs\new_airsim\lib\site-packages\msgpackrpc\session.py", line 41, in call
return self.send_request(method, args).get()
File "C:\Users\KSJ\anaconda3\envs\new_airsim\lib\site-packages\msgpackrpc\future.py", line 45, in get
raise error.RPCError(self._error)
msgpackrpc.error.RPCError: rpclib: server could not find function 'getRotorStates' with argument count 1.

I read the comment that it is not available to approach to source code. so I wonder that is there any other way to get rotor's states?

Error running test_reset.py

Hello,
my environment
win10
ADRL
When I ran the code, there was an error:

msgpackrpc.error.RPCError: rpclib: function 'enableApiControl' (called with 2 arg(s)) threw an exception. The exception contained this information: Vehicle API for 'drone_1' is not available. This could either because this is simulation-only API or this vehicle does not exist.

How can I correct it

where is the racing map?

there just Blocks?

Tro browsers

Add gate object in customized environment

Hi
Is there any way to add gates in our own environments (through UE editor?) just for other training purposes?

Connect Error with WSAECONNFUSED

Hello, I try to connect ADRL which is running in a remote computer and run the script of baseline_racer in another computer. I come with connection errors as following
"

"
The two computers are in local network hub via WiFi and I have set "LocalHostIp" and "ApiServerPort" in settings.json file in the remote computer and also revised to define the airsim_client as self.airsim_client = airsim.MultirotorClient(ip="192.168.31.247") in the baselien_racer.py instead of the default ones.

The baseline_racer can run successfully in a single computer with ADRL and encounter the connection error in two separated computers.

I did not figure out and fixed it for several days? Could anyone help me fix it? thanks a lot