faizansana/intersection-carla-gym

Intersection Gym environment in CARLA Town 3

Fancy seeing you here!

• 🔭 I’m currently doing my Masters @ UWaterloo
• 🌱 I'm currently working on Deep Reinforcement Learning for Autonomous Vehicles 🚙
• 📫 How best to reach me: [email protected]

intersection-carla-gym's Issues

Scale the rewards between -1 and 1

Based on current research, it was seen that scaling the rewards between -1 and 1 allowed for more stable training.

Incorporate Vx for other vehicles and pedestrians into observation space

Currently, it seems like the observation space only contains v_y. Having v_x or tangential velocity might be better.

Add Discrete space to environment

Currently the environment can only take Box (continuous) action space. Upgrade env to also allow for discrete action spaces.

Destroy vehicles and other sensors in a batch format

Based on Carla documentaion this is more efficient and could potentially speed up execution.

Upgrade reward function to incorporate pedestrian penalty

At the moment, the model, even after learning for 1.5 million steps, still crashes into pedestrians.

• Incorporating penalties for getting close to pedestrians in dense rewards might help.
• Larger negative reward for crashing into pedestrian.

Look into setting no render mode for CARLA

In this case, since we are not depending on sensor states (LiDAR, camera etc.) and only care about other vehicle states, pedestrians etc. using no render mode makes sense. This would also reduce resource consumption and increase FPS. Documentation is here

Send info over UDP to Plotjuggler to plot control, acceleration and velocity curves

PlotJuggler is a graphing tool. Send information from the ego vehicle to plotJuggler over UDP as shown by @Mafumaful.

Potentially add negative reward for distance to other vehicles

Update reward function to give negative reward if too close to other vehicle. This might improve collision percentage.

Installation of carla library in Docker container for 0.9.10.1 does not work

The egg file created only contains 0.9.10 and not the minor 0.1 version. Since we are passing the version directly, this causes the egg file path to be incorrect.

Need to think of a better way of adding this to path.

Create functional test for overall repo

There is a test_env.py script that runs the environment and checks the returns are compatible with the gym API. Use this as a CI test.

Remove port field in config file

Currently, the host is defined within code whereas port is called from the config file. This is confusing.

Unify `desired_speed` and `target_speeds` within config file

Currently target_speeds is of type List[floats] and desired_speed is of type float. Since the latter is for continuous while earlier for discrete, unify into a single config desired_speed which can be of type List[floats] or float.

Enhance logging system

Currently the logs are created using the experiment name and output dir params in the config file. This doesn't seem too useful so think of a better way of enhancing the logging and reduce the tree structure of config yaml file.

Have multiple destinations to turn left, right and straight

Currently, the vehicle only takes left turns since the global route sets the final destination as the left area. Enhance this logic to make it go straight and right too.

Improve reward function for vehicle and pedestrian closeness

At the moment a negative reward is given if the vehicle is within certain vicinity of pedestrian or vehicle. Improve this to make it more robust and incorporate the distance to these obstacles.

Update state space to only include vehicles within specific observation area

Currently, the ego vehicle technically has a "Gods-eye" view of the environment. In order to make it more realistic, it should only contain info within specific surroundings.

Look into integration with Foxglove for analysis

A sample project is shown here: https://github.com/collabora/carlafox/tree/main

By doing this, we can potentially get the velocity, acceleration and other details of adversarial vehicles without having to explicitly code them.

However, this will require significant effort as it will require ROS integration through CARLA ROS Bridge

Remove local_carla_agents folder and directly import from carla folder

Currently, there is a local_carla_agents folder which is a copy of that found in the carla Python API folder.

Since containers are being used, we can just refer it directly by adding it to PythonPath

Segmentation fault occurring after n random timesteps

The environment exits with a segmentation fault that occurs after n timesteps. This is an indeterministic number failing within even 2000 timesteps and as late as never as seen.

The following has been observed.

• It is seen to fail faster in python 3.8.16 (environment.yml) as compared to 3.7
• It was thought that this might be due to the collision sensor. However, the collision sensor was completely removed from the script and it stil continued to fail.
• This seg fault might be occurring due to failure of destroying the actor. It seems that the seg fault is occurring due to the carla library. However since the carla python library is a wrapper for C code, it is hard to debug exactly what is going wrong.
• After the traffic manager instantiation was moved to after setting it to synchronous mode, it seems these seg faults occur less. However, this could be unrelated.