microsoft/airsim-neurips2019-drone-racing

Drone Racing @ NeurIPS 2019, built on Microsoft AirSim

Game of Drones: A NeurIPS 2019 Competition

Note: This repository is not being maintained any more. Please use AirSim Drone Racing Lab.

Quickstart

• Website
• Register
• Competition guidelines
• Linux and Windows Binaries
• Python API, airsimneurips PyPI package

Note: If you use this repository in your research, please cite our pre-print, AirSim Drone Racing Lab.

@article{madaan2020airsim,
  title={AirSim Drone Racing Lab},
  author={Madaan, Ratnesh and Gyde, Nicholas and Vemprala, Sai and Brown, Matthew and Nagami, Keiko and Taubner, Tim and Cristofalo, Eric and Scaramuzza, Davide and Schwager, Mac and Kapoor, Ashish},
  journal={arXiv preprint arXiv:2003.05654},
  year={2020}
}

Downloading and running AirSim Binaries

Downloading

• Final round binaries and environments (v1.1)

  • tl;dr:
    • [Linux] Use the download_final_round_binaries.sh script
  • Long version:
    • Download the v1.1 Linux or Windows AirSim.zip, and unzip it.
    • Download your qualifier environments (shipped in pakfiles) - Final_Tier_1_and_Tier_2.pak and Final_Tier_3.pak.
    • Move the environment pakfiles into AirSim/AirSimExe/Content/Paks.
    • Download and move the settings.json file to ~/Documents/AirSim/settings.json.
    • Use airsimneurips >= 1.2.0

• Qualifier binaries and environments (v1.0)

  • tl;dr:
    • [Linux] Use the download_qualification_binaries.sh script
  • Long version:
    • Download the v1.0 Linux or Windows AirSim.zip, and unzip it.
    • Download your qualifier environments (shipped in pakfiles) - Qual_Tier_1_and_Tier_3.pak and Qual_Tier_2.pak.
    • Move the environment pakfiles into AirSim/AirSimExe/Content/Paks.
    • Download and move the settings.json file to ~/Documents/AirSim/settings.json.

• Training binaries and environments (v0.3):

  • tl;dr:
    • [Linux] Use the download_training_binaries.sh script
  • Long version:
    • Download the v0.3 Linux or Windows AirSim.zip, and unzip it.
    • Download training environments (shipped in pakfiles) - Soccer_Field.pak, ZhangJiaJie.pak, and Building99.pak.
    • Move the environment pakfiles into AirSim/AirSimExe/Content/Paks.
    • Download and move the settings.json file to ~/Documents/AirSim/settings.json.

Notes:

• Source code (zip) or Source code (tar.gz) might not be up-to-date with the master branch of this repository. It can be lagging by n commits to master since this release, specified on the released page.
  For the code on this repository, it's best to just git clone.

• List of disabled APIs in qualification binaries: The following APIs on the server side in the qualification binaries. You should see an error message pop up in the terminal message when you call these. They do work in the training binaries:

  • simSetObjectPose
  • simSetVehiclePose
  • simSetObjectScale
  • simGetObjectScale
  • simSetSegmentationObjectID
  • simGetSegmentationObjectID
  • simPause
  • simContinueForTime

Running

• Linux

  • Open a terminal window, cd to AirSim_Training/ or AirSim_Qualification directory, and enter the following command:

     ./AirSimExe.sh -windowed -opengl4
    

  • Running headless (with rendering of images enabled):

     DISPLAY= ./AirSimExe.sh -opengl4
    

  • To disable rendering completely for training planning and / or control policies, you can use:

     -./AirSimExe.sh -nullrhi
    

    Note that simGetImages will not work with this option.
  • To increase speed of simGetImages / increase speed of Unreal Engine's game thread;
    • Add the "ViewMode": "NoDisplay" to your settings.json file, or use this file directly.
      This disables rendering in the main viewport camera.
      Then run the binary with the following options.

     ./AirSimExe.sh -windowed -NoVSync -BENCHMARK
    

    You can also use the Unreal console commands Stat FPS, Stat UnitGraph, r.VSync, t.maxFPS. See Issue #111 for more details.

• Windows

  • Navigate to the AirSim/ directory, and double-click run.bat (or AirSimExe.exe -windowed)

Docker

• Prerequisites:

  • Install docker-ce.
  • Complete the desired post-installation steps for linux after installing docker.
    At the minimum, the page tells you how torun docker without root, and other useful setup options.
  • Install nvidia-docker2.

• Dockerfile:
  We provide a sample dockerfile you can modify.
  It downloads the training and qualification binaries automatically, and installs the python client.
  By default, it uses Ubuntu 18.04 and CUDA 10.0 with OpenGL, and is build on top of nvidia/cudagl:10.0-devel-ubuntu18.04.
  This can be changed of course, as explained in the following section.

• Building the docker image:
  You can use build_docker_image.py to build the dockerfile above (or your own custom one)
  Usage (with default arguments)

   cd docker/;
   python3 build_docker_image.py \
   	--dockerfile Dockerfile \
   	--base_image nvidia/cudagl:10.0-devel-ubuntu18.04 \
   	-- target_image airsim_neurips:10.0-devel-ubuntu18.04

• Running the docker image: See docker/run_docker_image.sh to run the docker image:
  Usage

  • for running default image, training binaries, in windowed mode:
    $ ./run_docker_image.sh "" training
  • for running default image, qualification binaries, in windowed mode:
    $ ./run_docker_image.sh "" qualification
  • for running default image, training binaries, in headless mode:
    $ ./run_docker_image.sh "" training headless
  • for running default image, qualification binaries, in headless mode:
    $ ./run_docker_image.sh "" qualification headless
  • for running a custom image in windowed mode, pass in you image name and tag:
    $ ./run_docker_image.sh DOCKER_IMAGE_NAME:TAG
  • for running a custom image in headless mode, pass in you image name and tag, followed by "headless":
    $ ./run_docker_image.sh DOCKER_IMAGE_NAME:TAG headless

AirSim API

• To control your drone and get information from the environment, you will need the airsimneurips API, which is accessible via Python.
  We recommend you used python >= 3.6. Python 2.7 will go out of support soon

• To install the Python API, do a :

   pip install airsimneurips
  

• See quick overview of the API below

• The API is documented at airsimneurips API doc

• Resources

  • Going through both open and closed issues in this repository might answer some of your questions. The search bar on top left can prove useful.
  • AirSim upstream API and examples can also be of use. However, please note that the main AirSim repo's API is not used in the competition (there's some overlap and some differences), however is a good learning resource.

Submitting Results and Leaderboard - Qualification Round

• For the qualification round, we have one race track for each tier. The relevant binaries (v1.0) are available for linux and windows

  • Tier 1: This is in the Soccer Field environment.
    THe race track is in the Qual_Tier_1_and_Tier_3.pak pakfile
  • Tier 2: This is in the ZhangJiaJie environment.
    The race track is in the Qual_Tier_2.pak pakfile.
  • Tier 3: This is again in the Soccer Field environment.
    The race track is in the Qual_Tier_1_and_Tier_3.pak pakfile.

• How to generate logfiles for each tier:

  • Loading level and starting race:

    • Please update your airsimneurips pythonclient (should be >=1.0.0).

    • Calling simStartRace(race_tier=1, 2, or 3) generates the appropriate log files.

    • Tier 1:

       	airsim_client.simLoadLevel('Qualifier_Tier_1')
       	airsim_client.simStartRace(1)

    • Tier 2:

       	airsim_client.simLoadLevel('Qualifier_Tier_2')
       	airsim_client.simStartRace(2)

    • Tier 3:

       	airsim_client.simLoadLevel('Qualifier_Tier_3')
       	airsim_client.simStartRace(3)

  • As Tier 2 focuses on perception and Tier 3 focuses on both perception and planning, note that simGetObjectPose returns noisy gate poses, after simStartRace(2) and simStartRace(3) is called.

  • As soon as simStartRace(1) or simStartRace(3) is called, drone_2 (MSR opponent racer) will start flying.

  • See baseline_racer.py for sample code. The previous bullet points are being called in wrapper functions in the following snippet in baseline_racer.py:

   	baseline_racer.load_level(args.level_name)
   	if args.level_name == "Qualifier_Tier_1":
   	    args.race_tier = 1
   	if args.level_name == "Qualifier_Tier_2":
   	    args.race_tier = 2
   	if args.level_name == "Qualifier_Tier_3":
   	    args.race_tier = 3
   	baseline_racer.start_race(args.race_tier)

• To submit your results to the leaderboard:

  • Navigate to the submission site, enter your team name in the proper field, and upload any number of race logs.
    It's ok to make a submission for as little as a single track and/or a single tier.
    You can find race logs inside of AirSimExe/Saved/Logs/RaceLogs in your downloaded binary folder.
    Please read the race monitoring section in the competition guidelines for more details.
  • The leaderboard will publish the results of a drone that is named drone_1 (call generate_settings_file.py to generate an AirSim settings file, as done for the baseline_racer below.
  • Please submit a PDF file in the report section to help us verify the honesty of your submission for the Nov 21st deadline. Please summarize your approach for all tiers you make a submission for, with appropriate citations. The report PDF size should not exceed 10 MB, and should be a maximum of 4 pages in length. We leave the exact format of the report to your descrition, but the IEEE template is a good choice.
  • We have emailed you a private key, which should be entered in the Team ID field. This helps us verify it was your team who indeed made the submission.
  • The leaderboard is updated once per day at 2100 PST.
    If you do not see your results after 24 hours, please email us with your team name and submitted log files.

Submitting Results and Leaderboard - Final Round

• For the final round, we have one race track for each tier. The relevant binaries (v1.1) are available for linux and windows

  • Tier 1: This is in the Soccer Field environment.
    THe race track is in the Final_Tier_1_and_Tier_2.pak pakfile
  • Tier 2: This is in the Soccer Field environment.
    The race track is in the Final_Tier_1_and_Tier_2.pak pakfile.
  • Tier 3: This is again in the ZhangJiaJie environment.
    The race track is in the Final_Tier_3.pak pakfile.

• How to generate logfiles for each tier:

  • Loading level and starting race:

    • Please update your airsimneurips pythonclient (should be >=1.2.0).

    • Calling simStartRace(race_tier=1, 2, or 3) generates the appropriate log files. You can only run tier N races in Final_Tier_N levels.

    • Tier 1:

       	airsim_client.simLoadLevel('Final_Tier_1')
       	airsim_client.simStartRace(tier=1)

    • Tier 2:

       	airsim_client.simLoadLevel('Final_Tier_2')
       	airsim_client.simStartRace(tier=2)

    • Tier 3:

       	airsim_client.simLoadLevel('Final_Tier_3')
       	airsim_client.simStartRace(tier=3)

  • As Tier 2 focuses on perception and Tier 3 focuses on both perception and planning, note that simGetObjectPose returns noisy gate poses.

  • As soon as simStartRace(tier=1) or simStartRace(tier=3) is called, drone_2 (MSR opponent racer) will start flying.

  • See baseline_racer.py for sample code. The previous bullet points are being called in wrapper functions in the following snippet in baseline_racer.py:

   	baseline_racer.load_level(args.level_name)
   	baseline_racer.start_race(args.race_tier)

• To submit your results to the final leaderboard:

  • Navigate to the submission site, enter your team name in the proper field, and upload any number of race logs.
    It's ok to make a submission for as little as a single track and/or a single tier.
    You can find race logs inside of AirSimExe/Saved/Logs/RaceLogs in your downloaded binary folder.
    Please read the race monitoring section in the competition guidelines for more details.
  • The leaderboard will publish the results of a drone that is named drone_1 (call generate_settings_file.py to generate an AirSim settings file, as done for the baseline_racer below.
  • Please submit a PDF file in the report section to help us verify the honesty of your submission by the Dec 5th, 2359 PST deadline. Please summarize your approach for all tiers you make a submission for, with appropriate citations. The report PDF size should not exceed 10 MB, and should be a maximum of 6 pages in length. We leave the exact format of the report to your descrition, but the IEEE template is a good choice.
  • We have emailed you a private key, which should be entered in the Team ID field. This helps us verify it was your team who indeed made the submission.
  • The final leaderboard is updated once per day at 2100 PST.
    If you do not see your results after 24 hours, please email us with your team name and submitted log files.

Sample code

• Plan and move on a minimum jerk trajectory using ground truth poses of gates:

  • Generate an AirSim settings.json file (same as the one provided in releases)

  $ cd baselines;
  $ python generate_settings_file.py

  • Start the AirSim Neurips binary, as explained above
  • Run the code!

  $ python baseline_racer.py \
  	--enable_viz_traj \
  	--enable_viz_image_cv2 \
  	--planning_baseline_type all_gates_at_once \
  	--planning_and_control_api moveOnSpline \
  	--level_name ZhangJiaJie_Medium \
  	--race_tier 1 

  Usage is:

  $ python baselines/baseline_racer.py -h
  usage: baseline_racer.py [-h]
  	[--level_name {Soccer_Field_Easy,Soccer_Field_Medium,ZhangJiaJie_Medium,Building99_Hard,Qualifier_Tier_1,Qualifier_Tier_2,Qualifier_Tier_3,Final_Tier_1,Final_Tier_2,Final_Tier_3}]
  	[--planning_baseline_type {all_gates_at_once,all_gates_one_by_one}]
  	[--planning_and_control_api {moveOnSpline,moveOnSplineVelConstraints}]
  	[--enable_viz_traj] [--enable_viz_image_cv2]
  	[--race_tier {1,2,3}]

• Plan a Game Theoretic Plan (GTP) trajectory for an ego drone based on an estimate of the opponent drone's behavior.

  • Generate an AirSim settings.json file

  $ cd baselines;
  $ python generate_settings_file.py

  • Start the AirSim Neurips binary, as explained above
  • Run the GTP code!

  $ python baseline_racer_gtp.py \
  	--blocking_behavior \
  	--plot_gtp \
  	--enable_viz_traj \
  	--level_name Qualifier_Tier_1

  • This method is an Iterative Best Response (IBR) trajectory planning technique. In IBR, first the trajectories of both drones are initialized as straight down the track at maximum speed (to win the game!). The opponent trajectory is then held constant while we solve for the ego trajectory via Model Predictive Control (MPC) optimization (details in gtp.py). Then, we hold the ego trajectory constant and solve for a guess of the opponent's trajectory in the same fashion. If after some iterations, the solution convereges (i.e., the resulting trajectories stop changing), we have reached a Nash equilibrium over the space of trajectories. That is to say, either agents can not unilaterally change their trajectory to increase their own performance. This implementation is a heuristic based on the original method proposed in the paper below (PDF here).
    • R. Spica, D. Falanga, E. Cristofalo, E. Montijano, D. Scaramuzza, and M. Schwager, "A Real-Time Game Theoretic Planner for Autonomous Two-Player Drone Racing", in the Proccedings of Robotics: Science and Systems (RSS), 2018.

Quick API overview

We added some new APIs (marked with 💚) to AirSim for the NeurIPS competition binaries.

Loading Unreal Engine environments

• simLoadLevel(level_name) 💚
  Possible values for level_name are:

  • "Soccer_Field_Easy", "Soccer_Field_Medium", "ZhangJiaJie_Medium", "Building99_Hard" in the training binaries (v0.3).
  • "Qualification_Tier_1", "Qualification_Tier_2", "Qualification_Tier_3" in the qualification binaries (v1.0).
  • "Final_Tier_1", "Final_Tier_2", "Final_Tier_3" in the final round binaries (v1.1). Before trying this, please ensure you've downloaded the corresponding training (v0.3) / qualifier (v1.0) / final round (v1.0) binaries, as described above

• UI Menu

  • Press F10 to toggle the level menu
  • Click your desired level. (Note: the UI lists all the pakfiles in the AirSim/AirSimExe/Content/Paks directory. Ensure you downloaded the pakfile, if you are not able to see a particular environment)

Race APIs:

• Start a race: simStartRace(tier=1/2/3) 💚

• Reset race: simResetRace() 💚

• Check if racer is disqualified: simIsRacerDisqualified() 💚

• Get index of last gate passed: simGetLastGatePassed() 💚

• Disable generation of logfiles by race APIs: simDisableRaceLog 💚

Lower level control APIs:

• FPV like Angle rate setpoint APIs:

  • moveByAngleRatesThrottleAsync 💚
  • moveByAngleRatesZAsync 💚 (stabilizes altitude)

• Angle setpoint APIs:

  • moveByRollPitchYawThrottleAsync 💚
  • moveByRollPitchYawZAsync 💚 (stabilizes altitude)

• RollPitchYawrate setpoint APIs:

  • moveByRollPitchYawrateThrottleAsync 💚
  • moveByRollPitchYawrateZAsync 💚 (stabilizes altitude)

Medium level control APIs:

• Velocity setpoints

  • moveByVelocityAsync
  • moveByVelocityZAsync (stabilizes altitude)

• Position setpoints

  • moveToPosition
  • moveOnPath
  • moveToZAsync

High level control APIs:

• Minimum jerk trajectory planning (using ethz-asl/mav_trajectory_generation), and trajectory tracking (using a pure pursuit like controller minimizing position and velocity errors), with position setpoints. Optionally use the *lookahead* parameters to start new trajectory from a point sampled n seconds ahead for trajectory being tracked currently.

  • moveOnSplineAsync 💚

• Minimum jerk trajectory planning (using ethz-asl/mav_trajectory_generation), and trajectory tracking (using a pure pursuit like controller minimizing position and velocity errors), with position setpoints and corresponding velocity constraints. Useful for making a drone go through a gate waypoint, while obeying speed and direction constraints. Optionally use the *lookahead* parameters to start new trajectory from a point sampled n seconds ahead for trajectory being tracked currently.

  • moveOnSplineVelConstraintsAsync 💚

• Clear and stop following current trajectory.

  • clearTrajectory 💚

Gain setter APIs:

• setAngleRateControllerGains 💚
• setAngleLevelControllerGains 💚
• setVelocityControllerGains 💚
• setPositionControllerGains 💚
• setTrajectoryTrackerGains 💚

APIs to help generate gate detection datasets:

• Object pose setter and getter:

  • simSetObjectPose
  • simGetObjectPose

• Object scale setter and getter:

  • simSetObjectScale 💚
  • simGetObjectScale 💚

• Object segmentation ID setter and getter:

  • simGetSegmentationObjectID
  • simSetSegmentationObjectID

• Listing all the objects in the scene:

  • simListSceneObjects 💚

• Gate specific APIs:

  • simGetNominalGateInnerDimensions 💚
  • simGetNominalGateOuterDimensions 💚

Questions

Please open a Github Issue on this repository (not AirSim) for any technical questions w.r.t. the Neurips competition.

airsim-neurips2019-drone-racing's Issues

moveOnSplineAsync is not work

I created my waypoints to navigate drone from a to b, I command moveOnSplineAsync(waypoints)
but drone did not move at all, please check it and let me know if this profound function is working or not ???

AttributeError: 'MultirotorClient' object has no attribute 'getServerVersion'

Hi

this is my settings.json

{
  "SeeDocsAt": "https://github.com/Microsoft/AirSim/blob/master/docs/settings.md",
  "SettingsVersion": 1.2,
  "SimMode": "Multirotor"
}

and im getting this error when using the example code given

[(base) C:\Users\wings>python
Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
KeyboardInterrupt
>>> import air
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'air'
>>> import airsimneurips as airsim
>>> client = airsim.MultirotorClient()
>>> client.confirmConnection()
Connected!
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\wings\Miniconda3\lib\site-packages\airsimneurips\client.py", line 128, in confirmConnection
    server_ver = self.getServerVersion()
AttributeError: 'MultirotorClient' object has no attribute 'getServerVersion'
>>> import airsim
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'airsim'
>>>
KeyboardInterrupt
>>>](url)

also to note that this works fine in another python env so its related to just the competition module

import airsim

# connect to the AirSim simulator 
client = airsim.MultirotorClient()

Cannot run AirSimExe on Windows 7 with NVIDIA Quadro 400

Hello,
I tried to run AirSimExe on Windows 7 64 bit with NVIDIA Quadro 400 driver, installed version 340.52.
I got a warning to upgrade to version 398.36. Unfortunately it was not possible to install this version.
Runnung AirSimExe I've got the following Assertion error:
Assrtion failed: Resource
[File:D\Build++UE4\Sync\Engine\Source\Runtime\Windows\D3D11RHI\Private\D3D11UniformBuffer.cpp] [Line: 257]
Invalid resource entry creating uniform buffer,
FViewUniformShaderParameters.Resources[48],ResourceType 0x6.

and the application crashed.

Is there any way to short-cut this assertion?

Different Training and Testing environments?

Will the environments used in the final competition be the exact same as the training environments? In case the maps of the final competition environment are the same but the gate placement would be different, would there be a way to randomize gate placement during training to help with generalization?

How to run several instances of AirSim in parallel?

Hello,

I am having issues on trying to launch several instances of AirSim in parallel on my machine (say, one per GPU for example) and to communicate with them through the Python API as I will eventually need to do this on a cluster to train RL algorithms. Is there any easy way of doing this, please?

Regards,
Yann.

Random Airsim crash with segmentation fault after many iterations

Hello,

I am testing reinforcement learning approaches in Airsim, and during training Airsim randomly crashes with a segmentation fault. I don't really understand the error, what is happening here please?

The full Airsim terminal output is available at https://pastebin.com/Mrxk3dZy

The actual error output starts at line 750 I believe. It says:

[2019.09.18-21.47.57:359][549]LogOutputDevice: Warning:

Script Stack (2 frames):
CompManager_BP_C.ExecuteUbergraph_CompManager_BP
CompManager_BP_C.Reset

Failed to find symbol file, expected location:
"/home/yann/Desktop/GoD/AirSim/AirSimExe/Binaries/Linux/AirSimExe.sym"
Assertion failed: (Index >= 0) & (Index < ArrayNum) [File:/home/airsim/repos/UnrealEngine/Engine/Source/Runtime/Core/Public/Containers/Array.h] [Line: 611]
Array index out of bounds: 0 from an array of size 0
[2019.09.18-21.47.57:364][549]LogCore: Error: appError called: Assertion failed: (Index >= 0) & (Index < ArrayNum) [File:/home/airsim/repos/UnrealEngine/Engine/Source/Runtime/Core/Public/Containers/Array.h] [Line: 611]

Signal 11 caught.
Malloc Size=65538 LargeMemoryPoolOffset=65554
CommonUnixCrashHandler: Signal=11
Malloc Size=65535 LargeMemoryPoolOffset=131119
Malloc Size=118144 LargeMemoryPoolOffset=249280
[2019.09.18-21.47.57:375][549]LogCore: === Critical error: ===
Unhandled Exception: SIGSEGV: invalid attempt to read memory at address 0x00000000000042e3

[2019.09.18-21.47.57:375][549]LogCore: Assertion failed: (Index >= 0) & (Index < ArrayNum) [File:/home/airsim/repos/UnrealEngine/Engine/Source/Runtime/Core/Public/Containers/Array.h] [Line: 611]

0x0000000002ecc314 AirSimExe!UnknownFunction(0x2ccc313)
0x0000000002d4e66b AirSimExe!UnknownFunction(0x2b4e66a)
0x0000000002cee6d7 AirSimExe!UnknownFunction(0x2aee6d6)
0x0000000002cee4fd AirSimExe!UnknownFunction(0x2aee4fc)
0x0000000002730f93 AirSimExe!UnknownFunction(0x2530f92)
0x0000000002730abd AirSimExe!UnknownFunction(0x2530abc)
0x00000000027324ee AirSimExe!UnknownFunction(0x25324ed)
0x0000000002fd9ed7 AirSimExe!UnknownFunction(0x2dd9ed6)
0x0000000003102333 AirSimExe!UnknownFunction(0x2f02332)
0x0000000003102dfd AirSimExe!UnknownFunction(0x2f02dfc)
0x0000000003110ef9 AirSimExe!UnknownFunction(0x2f10ef8)
0x0000000003102dfd AirSimExe!UnknownFunction(0x2f02dfc)
0x0000000002fd9ed7 AirSimExe!UnknownFunction(0x2dd9ed6)
0x000000000310557e AirSimExe!UnknownFunction(0x2f0557d)
0x0000000004a19855 AirSimExe!UnknownFunction(0x4819854)
0x0000000002702d0b AirSimExe!UnknownFunction(0x2502d0a)
0x0000000002702bc1 AirSimExe!UnknownFunction(0x2502bc0)
0x00000000026a835b AirSimExe!UnknownFunction(0x24a835a)
0x0000000002b9229d AirSimExe!UnknownFunction(0x299229c)
0x0000000002b90f74 AirSimExe!UnknownFunction(0x2990f73)
0x00000000056a2201 AirSimExe!UnknownFunction(0x54a2200)
0x000000000569d1bb AirSimExe!UnknownFunction(0x549d1ba)
0x0000000004ffd538 AirSimExe!UnknownFunction(0x4dfd537)
0x0000000004e8756f AirSimExe!UnknownFunction(0x4c8756e)
0x000000000257664b AirSimExe!UnknownFunction(0x237664a)
0x000000000257cf5a AirSimExe!UnknownFunction(0x237cf59)
0x0000000005d9995f AirSimExe!UnknownFunction(0x5b9995e)
0x00007fbd18ca4b97 libc.so.6!__libc_start_main(+0xe6)
0x000000000256c029 AirSimExe!UnknownFunction(0x236c028)

[2019.09.18-21.47.57:387][549]LogExit: Executing StaticShutdownAfterError
[2019.09.18-21.47.57:403][549]LogCore: Warning: Unable to statfs('/home/yann/Desktop/GoD/AirSim/AirSimExe/Saved/Crashes/crashinfo-AirSimExe-pid-17123-027EF2E008D73C60573494863D049F27/AirSimExe.log'): errno=2 (No such file or directory)
Malloc Size=75426 LargeMemoryPoolOffset=324722
Engine crash handling finished; re-raising signal 11 for the default handler. Good bye.
Segmentation fault (core dumped)

How to save and set the simulator state?

Hello,

I don't think this is actually possible right now because the closest things I find in the API are getMultirotorState(), getVehiclePose() and setVehiclePose() (I don't think these two last ones allow to save and set kinematics), but it would be useful for some Reinforcement Learning approaches if we could save a previous state and set it back.

Regards,
Yann.

What subset of the API are we allowed to use?

The API offers many functions. Which ones will be allowed in the competition?

>>> client.sim
client.simCharGetAvailableFaceExpressions(  client.simCharResetBonePose(                client.simCharSetSkinAgeing(                client.simGetGroundTruthEnvironment(        client.simGetVehiclePose(                   client.simSetObjectPose(
client.simCharGetBonePose(                  client.simCharSetBonePose(                  client.simCharSetSkinDarkness(              client.simGetGroundTruthKinematics(         client.simIsPause(                          client.simSetPose(
client.simCharGetBonePoses(                 client.simCharSetBonePoses(                 client.simContinueForTime(                  client.simGetImage(                         client.simListSceneObjects(                 client.simSetSegmentationObjectID(
client.simCharGetFaceExpression(            client.simCharSetFaceExpression(            client.simDestroyObject(                    client.simGetImages(                        client.simLoadLevel(                        client.simSetTimeOfDay(
client.simCharGetHeadRotation(              client.simCharSetFacePreset(                client.simEnableWeather(                    client.simGetObjectPose(                    client.simPause(                            client.simSetVehiclePose(
client.simCharGetSkinAgeing(                client.simCharSetFacePresets(               client.simGetCameraInfo(                    client.simGetPose(                          client.simPrintLogMessage(                  client.simSetWeatherParameter(
client.simCharGetSkinDarkness(              client.simCharSetHeadRotation(              client.simGetCollisionInfo(                 client.simGetSegmentationObjectID(          client.simSetCameraOrientation(   

Guidelines for dev packages include Tensorflow?

Hi,
From the guidelines:

The participants are expected to develop everything in Python using typical packages (numpy, scipy, etc.) and the provided AirSim API.

Does this include Tensorflow?

Thanks!

client.confirmConnection() fails

Hi, I get the following error

>>> import airsimneurips as airsim
>>> client = airsim.MultirotorClient()
>>> client.confirmConnection()
WARNING:tornado.general:Connect error on fd 7: ECONNREFUSED
WARNING:tornado.general:Connect error on fd 7: ECONNREFUSED
WARNING:tornado.general:Connect error on fd 7: ECONNREFUSED
WARNING:tornado.general:Connect error on fd 7: ECONNREFUSED
WARNING:tornado.general:Connect error on fd 7: ECONNREFUSED
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/app/miniconda/lib/python3.7/site-packages/airsimneurips/client.py", line 54, in confirmConnection
    if self.ping():
  File "/app/miniconda/lib/python3.7/site-packages/airsimneurips/client.py", line 24, in ping
    return self.client.call('ping')
  File "/app/miniconda/lib/python3.7/site-packages/msgpackrpc/session.py", line 41, in call
    return self.send_request(method, args).get()
  File "/app/miniconda/lib/python3.7/site-packages/msgpackrpc/future.py", line 43, in get
    raise self._error
msgpackrpc.error.TransportError: Retry connection over the limit

I'm running AirSim headless with a GPU accelerated X-server. When I take a screenshot it looks like this:

I can't interact with the UI to see what happens, though.

msgpackrpc.error.RPCError: rpclib: server could not find function 'simStartRace' with argument count 1.

Hi, I am trying to run baseline_racer.py, but I got an error.

The version of the api is the latest one after performing upgrade with pip as follows sudo pip3 install --upgrade airsimneurips and the repository's version is the master branch and the latest version.

My settings.json:

{
  "ClockSpeed": 1,
  "SeeDocsAt": "https://github.com/Microsoft/AirSim/blob/master/docs/settings.md",
  "SettingsVersion": 1.2,
  "SimMode": "Multirotor",
  "Vehicles": {
    "drone_1": {
      "Cameras": {
        "fpv_cam": {
          "CaptureSettings": [
            {
              "FOV_Degrees": 90,
              "Height": 240,
              "ImageType": 0,
              "Width": 320
            }
          ],
          "Pitch": 0.0,
          "Roll": 0.0,
          "X": 0.25,
          "Y": 0.0,
          "Yaw": 0.0,
          "Z": 0.0
        }
      },
      "Pitch": 0.0,
      "Roll": 0.0,
      "VehicleType": "SimpleFlight",
      "X": 0.0,
      "Y": 0.0,
      "Yaw": 0.0,
      "Z": 0.0
    },
    "drone_2": {
      "Pitch": 0.0,
      "Roll": 0.0,
      "VehicleType": "SimpleFlight",
      "X": 0.0,
      "Y": 1.0,
      "Yaw": 0.0,
      "Z": 0
    }
  }
}

I got the following error:

  File "baseline_racer.py", line 283, in <module>
    main(args)
  File "baseline_racer.py", line 250, in main
    baseline_racer.start_race(args.race_tier)
  File "baseline_racer.py", line 41, in start_race
    self.airsim_client.simStartRace(tier)
  File "/usr/local/lib/python3.6/dist-packages/airsimneurips/client.py", line 132, in simStartRace
    self.client.call('simStartRace', tier)
  File "/usr/local/lib/python3.6/dist-packages/msgpackrpc/session.py", line 41, in call
    return self.send_request(method, args).get()
  File "/usr/local/lib/python3.6/dist-packages/msgpackrpc/future.py", line 45, in get
    raise error.RPCError(self._error)
msgpackrpc.error.RPCError: rpclib: server could not find function 'simStartRace' with argument count 1.

Also, I tried an older version of the api 0.0.14 and an older version of the repository v0.1.2, and it worked after some slight modifications in the code and there were no errors.

Thanks in advance.

Methods Results

I was trying some methods in documentation but python shell responds Object has no attribute , it seems that it does not exist ?? any solution for that guys.

Thanks in advance

simGetObjectPose(gate_name) randomly returns nan

Hello,

I have been working quite a while on debugging an algorithm where 'nan' appeared randomly to finally find that this snippet from the baseline is responsible:

def get_ground_truth_gate_poses(self):
        gate_names_sorted_bad = sorted(self.airsim_client.simListSceneObjects("Gate.*"))
        gate_indices_bad = [int(gate_name.split('_')[0][4:]) for gate_name in gate_names_sorted_bad]
        gate_indices_correct = sorted(range(len(gate_indices_bad)), key=lambda k: gate_indices_bad[k])
        gate_names_sorted = [gate_names_sorted_bad[gate_idx] for gate_idx in gate_indices_correct]
        for gate_name in gate_names_sorted:
            p = self.airsim_client.simGetObjectPose(gate_name)
            assert not math.isnan(p.position.x_val), f"DEBUG: {gate_name} p.position.x_val is {p.position.x_val}"
            assert not math.isnan(p.position.y_val), f"DEBUG: {gate_name} p.position.y_val is {p.position.y_val}"
            assert not math.isnan(p.position.z_val), f"DEBUG: {gate_name} p.position.z_val is {p.position.z_val}"
        return [self.airsim_client.simGetObjectPose(gate_name) for gate_name in gate_names_sorted]

This randomly returns 'nan's on random gates after calling airsim_client.reset() repeatedly.

Regards,
Yann.

How to change API port

Hi!

I tried to change ApiServerPort via settings.json

    "drone_1": {
      "VehicleType": "SimpleFlight",
      "ApiServerPort": 41457,
      "Yaw": 0.0, 
      "Pitch": 0.0, 
      "Y": 0.0, 
      "X": 0.0, 
      "Z": 0.0, 
      "Roll": 0.0
    }

and create client like this
self.airsim_client = airsim.MultirotorClient(ip = "my_ip", port=41457)

But connection only works with port=41451, so it looks like "ApiServerPort": 41457 in settings.json doesn't work.

Thanks!

How is disqualification defined?

Hello,

I am having a hard time understanding the new rule for disqualification. For what I understood so far, the leading drone (closest to the next gate center) would disqualify the other drone if he got within the collision radius, but this seems not to be the case in the released binary. I need this information for my reward function to work accordingly.

simGetObjectPose() crashes

Hello,

Since Friday, the simulator has become unusable for me because simGetObjectPose() crashes, and therefore I cannot get the gate poses.

What I get is basically:

import airsimneurips as airsim
client = airsim.MultirotorClient()
client.confirmConnection()
Connected!
client.simLoadLevel('Soccer_Field_Easy')
True
client.simGetObjectPose('gate_01')
Traceback (most recent call last):
File "", line 1, in
File "/home/yann/anaconda3/lib/python3.7/site-packages/airsimneurips/client.py", line 221, in simGetObjectPose
pose = self.client.call('simGetObjectPose', object_name)
File "/home/yann/anaconda3/lib/python3.7/site-packages/msgpackrpc/session.py", line 41, in call
return self.send_request(method, args).get()
File "/home/yann/anaconda3/lib/python3.7/site-packages/msgpackrpc/future.py", line 45, in get
raise error.RPCError(self._error)
msgpackrpc.error.RPCError: rpclib: function 'simGetObjectPose' (called with 1 arg(s)) threw an exception. The exception is not derived from std::exception. No further information available.

Open .pak files with UE4

Hi,
Is it possible to open the .pak files with Unreal Engine 4 in order to get ground-truth poses of gates and obstacles? I tried UnrealPak and other third party unpackers but they didn't work.
Cheers,
CT

AirSim Executable Crashing Randomly After Several Baseline Racer Runs

Hello,

AirSim executable is randomly crashing after several baseline racer runs with the following error, anyone else having this issue? Is it system related?

F0806 22:04:04.803827 18871 polynomial_optimization_linear_impl.h:297] Check failed: segment_time > 0 (-nan vs. 0) Segment times need to be greater than zero *** Check failure stack trace: *** Signal 6 caught. Malloc Size=65538 LargeMemoryPoolOffset=65554 CommonUnixCrashHandler: Signal=6 Malloc Size=65535 LargeMemoryPoolOffset=131119 Malloc Size=118144 LargeMemoryPoolOffset=249280 Failed to find symbol file, expected location: "/home/${USER}/AirSimChallenge/AirSim/AirSimExe/Binaries/Linux/AirSimExe.sym" Malloc Size=66886 LargeMemoryPoolOffset=316182 Engine crash handling finished; re-raising signal 6 for the default handler. Good bye. Aborted (core dumped)

I'm running on Ubuntu18.04 with 64GB of RAM and a TitanXp 12GB VRAM GPU. Please advise. Thanks!

RGB or RGBD cameras, what resolution?

Hi!

At competition_guidelines.md, Tier II: Perception Only, two RGB cameras mentioned ("Equipped with two RGB cameras"). So, no depth information or 'D' should be added?

At Tier III: Full Autonomy there is "Given forward and downward facing RBD cameras". Do you mean RGBD cameras? Is it a typo?

Thanks!

enableApiControl() issue

Hi,
After I updated to version 0.0.8 everything was working fine-ish (my controller didn't work anymore with moveByRollPitchYawrateThrottleAsync() - previously worked well with moveByAngleThrottle()). Before I tried to debug my code I updated to 0.0.11. Now, with version 0.0.11 I get the following error when doing client.enableApiControl(True, vehicle_name='drone_1') - or when running the baseline script :

Traceback (most recent call last):
  File "baseline_racer.py", line 190, in <module>
    main(args)
  File "baseline_racer.py", line 166, in main
    baseline_racer.initialize_drone()
  File "baseline_racer.py", line 25, in initialize_drone
    self.airsim_client.enableApiControl(True, vehicle_name=self.drone_name)
TypeError: enableApiControl() got multiple values for argument 'vehicle_name'

Any ideas?
Cheers,
CT

cancelLastTask() issue

Hello,

I've got following error using client.cancelLastTask() method without any arguments

TypeError: cancelLastTask() takes 0 positional arguments but 1 was given

I'm using latest v0.1.2 relises

What's the proper way to capture images for the competition?

Following Airsim Image API docs like the one here, when I run

# reshape array to 4 channel image array H X W X 4
img_rgba = img1d.reshape(response.height, response.width, 4)

I get this error:

img_rgba = img1d.reshape(response.height, response.width, 4)  
    ValueError: cannot reshape array of size 110592 into shape (144,256,4)  

It seems that the dimensions being returned are actually (144,256,3), and the image is also not inverted. I can capture and view images with:

responses = client.simGetImages([
        airsim.ImageRequest("0", airsim.ImageType.DepthVis),  #depth visualization image
        airsim.ImageRequest("1", airsim.ImageType.DepthPerspective, True), #depth in perspective projection
        airsim.ImageRequest("1", airsim.ImageType.Scene), #scene vision image in png format
        airsim.ImageRequest("1", airsim.ImageType.Scene, False, False)])  #scene vision image in uncompressed RGB array
response = responses[3]
img1d = np.fromstring(response.image_data_uint8, dtype=np.uint8)  # get numpy array
img_rgb = img1d.reshape(response.height, response.width, 3)  # reshape array to 3 channel image array H X W X 3
cv2.imshow('RGB',img_rgb)

but I wonder if you could point to updated Image API docs or give an example of how best to capture images in the context of the competition.
Thanks!

Need to pass -nosound to avoid crash.

Seems to be the same issue reported here: carla-simulator/carla#1619
Passing -nosound to AirSimExe.sh seems to fix it.

Failure log here: https://gist.github.com/chisophugis/d45d41c68b3e14c419487b2d14b9b4e4

I'm on ubuntu 18.04 with nvidia.

How to control (or at least speed up) time in the simulator?

Hello everyone,

So, I am experimenting with Reinforcement Learning strategies to develop a controller for the competition. Since this is very sample-intensive, I need to speed Airsim up as much as possible to generate as many samples as possible (for now I understand it only runs real-time?).

Is it possible to basically control time in airsimneurips, please?

This is an obvious requirement for making the engine more useful than the real world in training Machine Learning algorithms in general, and it seems people tried this in microsoft/AirSim#901 modifying the setting.json file, but I don't think we have access to this file?

Regards,
Yann.

Tier 1 waypoints for gates.

For Tier 1, are the gates/levels going to be fixed with what is provided in the example levels ?
If not is there going to be an API call that returns the pose of the next gate/all the gates ?

Neurips Lottery Questions

Hi,

As I understand it, we have to win the lottery to be able to attend NeurIPS and compete in the live event. This means that even if one qualifies in first place, there is no garantee that he will be able to compete in the live event.

1. What are the odds of the lottery?
2. Is there a way to make competing in the live event without being physically present at the conference possible?

Cheers,
CT

Offline trajectory generation

Hi,

I was wondering if we are allowed to generate trajectories offline (which may take an hour) well before the start of the race, especially for tier 1 where gate positions are known. Granted, overtaking maneuvers will be added real-time.

Cheers,
CT

Which drone do I control for the validation leaderboard?

Hi,

When we upload the files for the validation leaderboard, which drone does it choose to publish its results?
Also, It seems as if the world reference frame for drone_2 is different then that of drone_1 and the one used to get gate poses. While testing, according to simGetVehiclePose drone_2 should be in the middle of the gate whereas it is not (translated by a constant vector). If you try baseline_racer with drone_2, you can see the issue.

Cheers,
CT

Error while trying to run baseline_racer.py

Hello, I just recently upgraded to airsimneurips v-0.20 in python. I am now getting this error while trying to run baseline_racer. Did I miss some pre-configuration?

Traceback (most recent call last): File "baseline_racer.py", line 284, in <module> main(args) File "baseline_racer.py", line 252, in main baseline_racer.initialize_drone() File "baseline_racer.py", line 63, in initialize_drone self.airsim_client.setTrajectoryTrackerGains(traj_tracker_gains.to_list(), vehicle_name=self.drone_name) File "D:\Softwares\Anaconda\envs\AirSimComp\lib\site-packages\airsimneurips\client.py", line 701, in setTrajectoryTrackerGains self.client.call('setTrajectoryTrackerGains', *((gains.to_list(),)+(vehicle_name,))) AttributeError: 'list' object has no attribute 'to_list'

[Linux] AirSimExe only working if NOT windowed

Hi,

When running ./AirSimExe.sh in window mode in Ubuntu 18.04, once I choose the vehicle at the start, it crashes.
This does not happen if in full screen mode. However, if I am in full screen mode and try switching workplaces or alt-tab to the terminal it also crashes.
I have a Dell XPS 9560 (GTX1050 - latest Nvidia driver and Vulkan version).
The errors that appeared in the terminal:

[2019.07.26-18.38.32:291][  0]LogPawn: Error: EnableInput can only be specified on a Pawn for its Controller
[2019.07.26-18.38.32:557][  0]LogScript: Warning: Script Msg: Attempted to access index 0 from array 'CallFunc_GetAllActorsOfClass_OutActors' of length 0 in '/AirSim/Weather/WeatherFX/WeatherActor.WeatherActor_C:ExecuteUbergraph_WeatherActor'!
[2019.07.26-18.38.32:557][  0]LogStreaming: Error: Couldn't find file for package /AirSim/HUDAssets/MenuActor requested by async loading code. NameToLoad: /AirSim/HUDAssets/MenuActor
[2019.07.26-18.38.32:557][  0]LogStreaming: Error: Found 0 dependent packages...
[2019.07.26-18.38.32:557][  0]LogTemp: Warning: Warning, WeatherAPI got invalid menu actor class!
[2019.07.26-18.38.32:559][  0]LogTemp: PhysXCar
PhysXCar

And:

[2019.07.26-18.38.34:921][  0]LogCore: Error: appError called: Assertion failed:  [File:/home/airsim/repos/UnrealEngine/Engine/Source/Runtime/VulkanRHI/Private/VulkanMemory.cpp] [Line: 243] 
Out of Device Memory, Requested=31552.00Kb MemTypeIndex=7

[2019.07.26-18.38.34:932][  0]LogCore: === Critical error: ===
Unhandled Exception: SIGSEGV: invalid attempt to read memory at address 0x00000000000034e7

[2019.07.26-18.38.34:932][  0]LogCore: Assertion failed:  [File:/home/airsim/repos/UnrealEngine/Engine/Source/Runtime/VulkanRHI/Private/VulkanMemory.cpp] [Line: 243] 
Out of Device Memory, Requested=31552.00Kb MemTypeIndex=7

You can find the full crash report in the zip below,

Cheers,
CT

crash-info.zip

baseline_racer.py problem

Hi,

I am getting the following error when running the baseline script:

Traceback (most recent call last):
  File "baseline_racer.py", line 190, in <module>
    main(args)
  File "baseline_racer.py", line 166, in main
    baseline_racer.initialize_drone()
  File "baseline_racer.py", line 37, in initialize_drone
    self.airsim_client.setTrajectoryTrackerGains(traj_tracker_gains.to_list(), vehicle_name=self.drone_name)
  File "/home/charbel/.local/lib/python3.6/site-packages/airsimneurips/client.py", line 224, in setTrajectoryTrackerGains
    self.client.call('setTrajectoryTrackerGains', gains, vehicle_name)
  File "/home/charbel/.local/lib/python3.6/site-packages/msgpackrpc/session.py", line 41, in call
    return self.send_request(method, args).get()
  File "/home/charbel/.local/lib/python3.6/site-packages/msgpackrpc/future.py", line 45, in get
    raise error.RPCError(self._error)
msgpackrpc.error.RPCError: rpclib: server could not find function 'setTrajectoryTrackerGains' with argument count 2.

I git pulled all repos to make sure I have the latest versions.
Any Ideas?

Cheers,
CT

Running AirSim headless

HI,

I'm trying to run AirSim headless using VirtualGL, but I'm having some problems because I need to use the flag -opengl3 and this generates some errors related to the shaders ('uncooked' stuff... I'm not used to Unreal engine).
Error using the flag for OpenGL3:

[2019.07.18-02.54.35:620][  0]LogShaders: Error: The global shader cache file '/home/test/Airsim-Linux/Engine/GlobalShaderCache-GLSL_150.bin' is missing.

Your application is built to load COOKED content. No COOKED content was found; This usually means you did not cook content for this build.
It also may indicate missing cooked data for a shader platform(e.g., OpenGL under Windows): Make sure your platform's packaging settings include this Targeted RHI.

Alternatively build and run the UNCOOKED version instead.

... and this seems to be related to the unchecked project packaging option Share Material Shader Code.

Error without the flag:

[2019.07.18-02.57.33:109][  0]LogRHI: Error: OpenGL 4.3 not supported by driver

BTW, the env variables:

export SDL_VIDEODRIVER_VALUE=offscreen
export SDL_HINT_CUDA_DEVICE=0

...don't make any difference on my system.

Would you have any hints on how to run AirSim headless (or a free-of-charge way to use the Azure AirSim headless option)?

Thanks in advance!

Cheers,
Ricardo

no `run.bat` in the zip file

For Windows users, there seems no run.bat file in the downloaded zip file.

AirSim Windows Binary won't open (missing nlopt-0.dll)

Attempting to open the AirSim binary on my machine gives the following System Error:
"The code execution cannot proceed because nlopt-0.dll was not found. Reinstalling the program may fix this problem."

Using a RoG Zephyrus gx701 with Windows 10 Pro v1809 and haven't had issues with other UE4 games.

Baseline enableApiControl error

I am getting the following error while running the baseline, I have already updated the binaries and packages.

Connected!
Client Ver:1 (Min Req: 1), Server Ver:1 (Min Req: 1)

False
Connected!
Client Ver:1 (Min Req: 1), Server Ver:1 (Min Req: 1)

Traceback (most recent call last):
File "baseline_racer.py", line 190, in
main(args)
File "baseline_racer.py", line 166, in main
baseline_racer.initialize_drone()
File "baseline_racer.py", line 25, in initialize_drone
self.airsim_client.enableApiControl(vehicle_name=self.drone_name)
File "/usr/local/lib/python3.5/dist-packages/airsimneurips/client.py", line 85, in enableApiControl
return self.client.call('enableApiControl', True, vehicle_name)
File "/usr/local/lib/python3.5/dist-packages/msgpackrpc/session.py", line 41, in call
return self.send_request(method, args).get()
File "/usr/local/lib/python3.5/dist-packages/msgpackrpc/future.py", line 45, in get
raise error.RPCError(self._error)
msgpackrpc.error.RPCError: rpclib: function 'enableApiControl' (called with 2 arg(s)) threw an exception. The exception is not derived from std::exception. No further information available.

Is it possible to use C++ API and C++ code for competition or Python only?

Is it possible to use C++ API and C++ code for competition or Python only?

How to disable log streaming?

Hello,

How can we disable log streaming please?

How to get/set the ground truth for gates sizes?

Hello everyone,

According to the competition guidelines, for difficulties medium and hard, the sizes of the gates will not be constant. How do we get the ground truth for these, please? I am basically simply trying to detect when the drone passes through a gate.

Also, is it possible to change these dimensions algorithmically?

Thanks,
Yann.

Plot Transform

Hi, sorry for opening many issues, I am trying to get some help since this is my first experience in both Python and Airsim. After defining a Pose I called function plot_transform it did not work , please call this function and tell me whether it works or not

how to run aisrim using root user

Hello, i figured out how to use airsim without visualization. Now i was trying to run airsim in floydhub cloud . But it uses root user. How can i run airsim despite knowing only root user?

is it possible to run the game without visualization?

Hello , is it possible to run the game without visualization? That will be helpful because, in that case i can run the game and codes in the clound environment where i only have access to jupyter notebooks

jupyter lab doesnt run properly with airsim

Hello, I wanted to run jupyter lab and /or jupyter notebook with airsim. But there are some problems with these two.. When I install airsim, i cannot run jupyter notebook properly. and also vice versa..can you tell me the proper version of jupyter for airsim so that i can run both well?

What API would be available to control drone?

https://github.com/microsoft/AirSim-NeurIPS2019-Drone-Racing/blob/master/docs/competition_guidelines.md#how-to-participate-in-the-challenge

The simulation environment will later contain a low-level trajectory planner and feedback controller to control the dynamics of the drone to achieve the specified waypoint and velocity, thereby simplifying the interface as much as possible.

Would it be possible to control by drone velocities and attitude also?

Maximum acceleration of drone

Hi,
While testing moveByAngleThrottle(), it appears that the maximum acceleration of the drone is ~1.7g which is low for a racing drone. Are you planing on modifying it or will it be the same throughout the competition? Or is the drag coefficient too high?
Cheers,
CT

airsimneurips==0.0.14 doesn't work with corresponding AirSim-Linux.tar.gz

Repro instructions:

I'm looking at: https://github.com/microsoft/AirSim-NeurIPS2019-Drone-Racing/releases/tag/v0.1.2

1. Download and unpack https://github.com/microsoft/AirSim-NeurIPS2019-Drone-Racing/releases/download/v0.1.2/AirSim-Linux.tar.gz and ZhangJiaJie.pak
2. pip install airsimneurips==0.0.14
3. python baseline_racer.py

I get the error:

sean:~/pg/game-of-drones/AirSim-NeurIPS2019-Drone-Racing/baselines % python baseline_racer.py
Connected!
Connected!
Traceback (most recent call last):
  File "baseline_racer.py", line 188, in <module>
    main(args)
  File "baseline_racer.py", line 164, in main
    baseline_racer.initialize_drone()
  File "baseline_racer.py", line 24, in initialize_drone
    self.airsim_client.enableApiControl(vehicle_name=self.drone_name)
  File "/home/sean/.local/lib/python2.7/site-packages/airsimneurips/client.py", line 49, in enableApiControl
    return self.client.call('enableApiControl', True, vehicle_name)
  File "/home/sean/.local/lib/python2.7/site-packages/msgpackrpc/session.py", line 41, in call
    return self.send_request(method, args).get()
  File "/home/sean/.local/lib/python2.7/site-packages/msgpackrpc/future.py", line 45, in get
    raise error.RPCError(self._error)
msgpackrpc.error.RPCError: rpclib: function 'enableApiControl' (called with 2 arg(s)) threw an exception. The exception is not derived from std::exception. No further information available.
zsh: exit 1     python baseline_racer.py

Doing pip install airsimneurips==0.0.11 fixes the issue.

AirSim API environment navigation

After downloading paks of the environments, i followed tutorial in github. but I am facing some problems in changing the environments.
First one , after requesting a specific environment unreal starts loading but eventually it does not change it grey screen and drone only.
in python shell client.simLoadLevel('Building99').
Second one , is hard levels once I request specific hard levels python shell return an error says that you enter incorrect level names as follows.
raise ValueError("Allowed level names are: 'Building99', 'Soccer_Field', 'ZhangJiaJie', or 'MainMenu'")

thanks in advance

Which sim prefix functions are OK to use in which tiers?

I got confused with reading closed issues from July to here on. On here

#14

It says

For planning only tier, you'd get access to the ground truth poses of gates, but that would not be the case of tiers using perception

Which itself calls simListSceneObjects method/function. A prefix sim function. As far i know only sim related functions were simLoadLevel and simGetImage

#19

Stay tuned for an official release of whitelisted APIs for the competition, but as a general rule of thumb, APIs with the sim prefix will be blocked. The exceptions to this being simGetImage(s) and simLoadLevel

Can you post what is OK to use and what's not for each tier
thank you

Using AirSim Images with NumPy example fails

I was trying out the example "Using AirSim Images with NumPy" (https://microsoft.github.io/AirSim/docs/image_apis/) but it failed. What's the problem here?

Python 3.7.0 (default, Jun 28 2018, 13:15:42) 
[GCC 7.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> from airsimneurips import ImageRequest
>>> import airsimneurips as airsim
>>> client = airsim.MultirotorClient()
>>> client.confirmConnection()
Connected!
Client Ver:1 (Min Req: 1), Server Ver:1 (Min Req: 1)

>>> # client.simLoadLevel('Soccer_Field_Easy')  
... # client.simLoadLevel('Soccer_Field_Medium')        
... # client.simLoadLevel('ZhangJiaJie_Medium')
... client.simLoadLevel('Building99_Hard')
True
>>> 
>>> responses = client.simGetImages([ImageRequest("0", airsim.ImageType.Scene, False, False)])
>>> response = responses[0]
>>> 
>>> # get numpy array
... img1d = np.fromstring(response.image_data_uint8, dtype=np.uint8) 
__main__:2: DeprecationWarning: The binary mode of fromstring is deprecated, as it behaves surprisingly on unicode inputs. Use frombuffer instead
>>> 
>>> # reshape array to 4 channel image array H X W X 4
... img_rgba = img1d.reshape(response.height, response.width, 4)  
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
ValueError: cannot reshape array of size 110592 into shape (144,256,4)

bug report and how to change graphic settings?

Bug = --enable_plot_transform parameter is not in latest code yet still in github page. Please remove it

Also how to change graphic settings , my laptop having hard time with current settings would love to change graphic quality and render quality before train

Mysterious exception thrown with `enableApiControl`

Trace:

Traceback (most recent call last):
  File "baseline_racer.py", line 188, in <module>
    main(args)
  File "baseline_racer.py", line 164, in main
    baseline_racer.initialize_drone()
  File "baseline_racer.py", line 24, in initialize_drone
    self.airsim_client.enableApiControl(vehicle_name=self.drone_name)
  File "env/lib/python3.7/site-packages/airsimneurips/client.py", line 49, in enableApiControl
    return self.client.call('enableApiControl', True, vehicle_name)
  File "env/lib/python3.7/site-packages/msgpackrpc/session.py", line 41, in call
    return self.send_request(method, args).get()
  File "env/lib/python3.7/site-packages/msgpackrpc/future.py", line 45, in get
    raise error.RPCError(self._error)
msgpackrpc.error.RPCError: rpclib: function 'enableApiControl' (called with 2 arg(s)) threw an exception. 
The exception is not derived from std::exception. No further information available.

Env:

• Ubuntu
• Neurips binary version: v0.1.2
• airsimneurips version (from PyPI): 0.0.14