Giter Site home page Giter Site logo

zyunfeii / uav_obstacle_avoiding_drl Goto Github PK

View Code? Open in Web Editor NEW
345.0 2.0 63.0 85.21 MB

This is a project about deep reinforcement learning autonomous obstacle avoidance algorithm for UAV.

Python 66.08% M 0.13% MATLAB 31.34% C++ 2.45%
reinforcement-learning reinforcement-learning-algorithms uav

uav_obstacle_avoiding_drl's Introduction

Typing SVG

README

This is a project about deep reinforcement learning autonomous obstacle avoidance algorithm for UAV. The whole project includes obstacle avoidance in static environment and obstacle avoidance in dynamic environment. In the static environment, Multi-Agent Reinforcement Learning and artificial potential field algorithm are combined. In the dynamic environment, the project adopts the combination of disturbed flow field algorithm and single agent reinforcement learning algorithm.

As this is my undergraduate graduation project and my academic level is limited, I wrote a conference paper at that time. The address of the paper is: Adaptive Interfered Fluid Dynamic System Algorithm Based on Deep Reinforcement Learning Framework

Static environment

There are four methods to solve:

  1. MADDPG
  2. Fully Centralized DDPG
  3. Fully Decentralized DDPG
  4. Fully Centralized TD3

The third and the fourth methods perform better than others.

Dynamic environment

There are four methods to solve:

  1. PPO+GAE(with multi-processing )
  2. TD3
  3. DDPG
  4. SAC

The first three methods perform just the same. PPO convergence needs less episodes. TD3 and DDPG converge fast. Though Soft Actor-Critic is an outstanding algorithm in DRL, it has no obvious effect in my environment.

Traditional methods for UAV path planning

Three traditional methods are written with MATLAB:

  1. A * search algorithm*
  2. RRT algorithm
  3. Ant colony algorithm

C++:

  1. D star algorithm

The experiments show that A* search algorithm is much better than others but it is less effective than reinforcement learning path planning.

Artificial potential field algorithm

This project provides the MATLAB and Python realization of artificial potential field algorithm.

Python realization: ./APF/APFPy2.py ./APF/APFPy3.py ./APF/ApfAlgorithm.py (two-dimensional and three-dimensional)

Matlab realization: ./APF/APF_matlab (two-dimensional)

IFDS and IIFDS algorithm

This is an obstacle avoidance planning algorithm based on flow field. I realize it with matlab. The code is in folder IIFDS_and_IFDS.

How to begin trainning

For example, you want to train the agent in dynamic environment with TD3, what you need to do is just running the main.py, then test.py, finally open matlab and run the test.m to draw.

If you want to test the model in the environment with 4 obstacles, you just need to run Multi_obstacle_environment_test.py.

Requirements

numpy

torch

matplotlib

seaborn==0.11.1

Files to illustrate

calGs.m: calculate the index Gs which shows the performance of the route.

calLs.m: calculate the index Ls which shows the performance of the route.

draw.py: this file includes the Painter class which can draw the reward curve of various methods.

config.py: this file give the setting of the parameters in trainning process of the algorithm such as the MAX_EPISODE, batch_size and so on.

Method.py: this file concludes many important methods such as how to calculate the reward of the agents.

static_obstacle_environment.py: there are many static obstacle environments' parameters in this file.

dynamic_obstacle_environment.py: there are many dynamic obstacle environments' parameters in this file.

Multi_obstacle_environment_test.py: this file test the dynamic model in the environment in dynamic_obstacle_environment.py.

data_csv: this file save some data such as the trace of UAV and the reward in trainning.

AntColonybenchmark.m: ACO algorithm realized by MATLAB.

Astarbenchmark.m: A* algorithm realized by MATLAB.

RRTbenchmark.m: RRT algorithm realized by MATLAB.

A simple simulation example

  • avatar

Star history

  • avatar

all rights reserved.

uav_obstacle_avoiding_drl's People

Contributors

zyunfeii avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

uav_obstacle_avoiding_drl's Issues

Unable to run

Hello sir,I was unable to run this please look into it sir Here I am attaching the images.please help sir as it was relating to my project.
IMG-20220305-WA0014
IMG-20220305-WA0013

what‘s ’the meaning of three actions?

Hi, this is a wonderful job!
But, i can't understand the meaning of row0, theta, sigma0. Great thanks if u reply this issue!

def getqNext(self, uavPos, obsCenter, vObs, row0, sigma0, theta, qBefore): u = self.initField(uavPos, self.V0, self.goal) repulsiveMatrix = self.calRepulsiveMatrix(uavPos, obsCenter, self.obsR, row0) tangentialMatrix = self.calTangentialMatrix(uavPos, obsCenter, self.obsR, theta, sigma0) T = self.calculateT(obsCenter, uavPos, self.obsR) vp = np.exp(-T / self.lam) * vObs M = np.eye(3) + repulsiveMatrix + tangentialMatrix ubar = (M.dot(u - vp.reshape(-1, 1)).T + vp.reshape(1, -1)).squeeze() # 限制ubar的模长,避免进入障碍内部后轨迹突变 if self.calVecLen(ubar) > 5: ubar = ubar/self.calVecLen(ubar)*5 if qBefore[0] is None: uavNextPos = uavPos + ubar * self.stepSize else: uavNextPos = uavPos + ubar * self.stepSize _, _, _, _, qNext = self.kinematicConstrant(uavPos, qBefore, uavNextPos) return uavNextPos

运行不了

我在pycharm上提示没有相关模块,但是在matlab却能运行,为什么呢?
微信图片_20240711111219

2維

我想請教一下,如果我把DDPG想改成二維的路徑規劃,並且全部程式都是用matlab執行,請問應該要怎麼改呢? 謝謝
因為我使用了python,但目前遇到許多問題,已經沒有時間慢慢調整。所以打算使用matlab較為簡單地處理

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.