mljejucamp2017 / drl_based_selfdrivingcarcontrol Goto Github PK
View Code? Open in Web Editor NEWDeep Reinforcement Learning (DQN) based Self Driving Car Control with Vehicle Simulator
Deep Reinforcement Learning (DQN) based Self Driving Car Control with Vehicle Simulator
I run the program and simulator according to your method, set it to training mode, but the trained model cannot be saved. Num_training is 1M.But on my computer, when the number of steps is 10K, the program runs slowly.If I need to reset Num_training?The problem is that it cannot generate ckpt file.The file type it produces is different from ckpt file, such as "events.out.tfevents.1527528894" .I would like to ask if you have any good suggestions or have a trained network that i can use.Thank you so much for helping me.
Hi @Kyushik,
I'm using Ubuntu 16.04 to train the sensor version of the DRL Vehicle simulator, I need to remove the lane input based vehicle movement using sensor. How can I compile/build VehicleAgent.cs in linux?
Thanks in Advance
Good Day!
Can you please clarify my questionns?
I am looking for training the following scenario in Indian Road(Please review the screen shots attached) - the scenario is similar to application of ADAS in Cars like Volvo XC90
Scenario 1 - Car A and Car B are in the same side of the road(lane concept is not included), Car B is stopped in front of Car A
Scenario 2 - Car A moves to a particular distance(behind Car B), alerted by Radar / LIDAR(Cube's color changed from black to red) - Car A is manually driven by a human driver
Scenario 3 - Realizing the Radar alert, Car A switches to autonomous mode, takes control from the driver and steers the Car A sideways to avoid crash
Can you please let me know, if I can simulate a training scenario in DRL, I am confused about how to drive the car manually during the DRL training.
Can you please help me.
Thanks
Guru
How can you plot the lidar data in xy coordinate? i mean how to convert it
Hello, I encountered a strange problem during the development process, the simulation screen is stuck, my operating system is win7 64 bit, may I ask where the problem may occur? If I load the emulator of the previous version or use emv_name = "../environment/Planning/Windows/Planning", the picture will not freeze
UnityTimeOutException Traceback (most recent call last)
in
----> 1 env = UnityEnvironment(file_name=env_name)
2
3 # Examine environment parameters
4 print(str(env))
5
~\anaconda3\lib\site-packages\mlagents_envs\environment.py in init(self, file_name, worker_id, base_port, seed, no_graphics, timeout_wait, additional_args, side_channels, log_folder)
215 )
216 try:
--> 217 aca_output = self._send_academy_parameters(rl_init_parameters_in)
218 aca_params = aca_output.rl_initialization_output
219 except UnityTimeOutException:
~\anaconda3\lib\site-packages\mlagents_envs\environment.py in _send_academy_parameters(self, init_parameters)
459 inputs = UnityInputProto()
460 inputs.rl_initialization_input.CopyFrom(init_parameters)
--> 461 return self._communicator.initialize(inputs)
462
463 @staticmethod
~\anaconda3\lib\site-packages\mlagents_envs\rpc_communicator.py in initialize(self, inputs)
102
103 def initialize(self, inputs: UnityInputProto) -> UnityOutputProto:
--> 104 self.poll_for_timeout()
105 aca_param = self.unity_to_external.parent_conn.recv().unity_output
106 message = UnityMessageProto()
~\anaconda3\lib\site-packages\mlagents_envs\rpc_communicator.py in poll_for_timeout(self)
94 """
95 if not self.unity_to_external.parent_conn.poll(self.timeout_wait):
---> 96 raise UnityTimeOutException(
97 "The Unity environment took too long to respond. Make sure that :\n"
98 "\t The environment does not need user interaction to launch\n"
How can I fix this problem?
thanks for providing the DRL algorithm and the unity environment. i doubt how the agent(self-driving car) in the unity environment communicate with other vehicles, how the self-driving car' state transform, and how the reward define in the the code of unity environment. Because unity environment is a executable program, we can't know the detail information, is there some reference material for the detail information about the internal code of unity environment .thanks very much!
I would like to ask that, is this skip frame necessary? since the agent does not act according to pixel(frame) but act according to sensor and camera features?
Environment details
Windows 10
Unity version - 2017.3.0f3
Steps
downloaded the simulator from here:
https://www.dropbox.com/s/33iscinb81n2uue/ADAS_Windows.zip?dl=0)
Copied UnityPlayer.dll in the exe folder. (After it complained could not load UnityPLayer.dll)
Ran the code Basic.py
got the output:
(7628) wsgi starting up on http://0.0.0.0:4567
Tried running the simulator exe.
Got the error failed to load mono
Hi,
In the DRL Simulator distance between Agent and other obstacles calculated in float values.
Is this float values equal to Meters ? If not, then what this float value represent?
Please help me to understand this?
Thanks in Advance
Malathi K
HI, PPT link and simulator link can't open
thank for your code.when i run the ipynb, it remember me init() got an unexpected keyword argument 'file' ,can you tell me how to resolve
I'm sorry for disturbing you.I'm a beginner for DRL and interested in your project.I download your project and the simulator.But I don't know how to run it successfully.Hope to get your help.Thank you.
Hi,
I am new to this project.I would like to know did you use ML Agents of unity in this project. If so, can you please guide me with the tutorials/links to learn more about it.
I would like to recreate this for my college project on my own can you please guide me.
With curiosity,
Ranjani N
Hi, congratulate won the camp, and the paper was accepted to iv 2018!
In the README.md
, you have mentioned " I used lots of vehicle sensors(e.g. RADAR, LIDAR, ...) to perceive environments around host vehicle. Also, There are a lot of Advanced Driver Assistant Systems (ADAS) which are already commercialized."
I'm wondering if you can tell us something about where I can find those sensors and how should i use them.
And because of the environment made by purchased models, if you can upload the agents, brain and academy scripts?
Thanks.
Hey I plan to use this simulator for my class project and I was wondering if the Mac version is going to be released any time soon?
Thanks for this!
Hi, can we get the speed of each vehicles?
Hi ,
ModuleNotFoundError Traceback (most recent call last)
in ()
8 import time
9
---> 10 from unityagents import UnityEnvironment
11
12 get_ipython().run_line_magic('matplotlib', 'inline')
ModuleNotFoundError: No module named 'unityagents'
Name Error: name " UnityEnvironment " is not defined
NameError Traceback (most recent call last)
in ()
----> 1 env = UnityEnvironment(file_name=env_name)
2
3 # Examine environment parameters
4 print(str(env))
5
NameError: name 'UnityEnvironment' is not defined
error Traceback (most recent call last)
in ()
17
18 # Get information for update
---> 19 env_info = env.step(action_in)[default_brain]
20
21 next_observation_stack, observation_set, next_state_stack, state_set = resize_input(env_info, observation_set, state_set)
C:\Users\Q\Desktop\JejuCamp_ML_Agents\unityagents\environment.py in step(self, vector_action, memory, text_action)
469 self._conn.send(b"STEP")
470 self._send_action(vector_action, memory, text_action)
--> 471 return self._get_state()
472 elif not self._loaded:
473 raise UnityEnvironmentException("No Unity environment is loaded.")
C:\Users\Q\Desktop\JejuCamp_ML_Agents\unityagents\environment.py in _get_state(self)
285 self._data = {}
286 while True:
--> 287 state_dict, end_of_message = self._get_state_dict()
288 if end_of_message is not None:
289 self._global_done = end_of_message
C:\Users\Q\Desktop\JejuCamp_ML_Agents\unityagents\environment.py in _get_state_dict(self)
241 :return:
242 """
--> 243 state = self._recv_bytes().decode('utf-8')
244 if state[:14] == "END_OF_MESSAGE":
245 return {}, state[15:] == 'True'
C:\Users\Q\Desktop\JejuCamp_ML_Agents\unityagents\environment.py in _recv_bytes(self)
217 try:
218 s = self._conn.recv(self._buffer_size)
--> 219 message_length = struct.unpack("I", bytearray(s[:4]))[0]
220 s = s[4:]
221 while len(s) != message_length:
error: unpack requires a bytes object of length 4
once i open the simulator app, I get following exception found at end in log file. I cannot file .cs file. please help me in this regard.
Mono path[0] = 'C:/DL/adas/jas/DRL_based_SelfDrivingCarControl-master (1)/DRL_based_SelfDrivingCarControl-master/environment/jeju_camp_Data/Managed'
Mono config path = 'C:/DL/adas/jas/DRL_based_SelfDrivingCarControl-master (1)/DRL_based_SelfDrivingCarControl-master/environment/jeju_camp_Data/MonoBleedingEdge/etc'
PlayerConnection initialized from C:/DL/adas/jas/DRL_based_SelfDrivingCarControl-master (1)/DRL_based_SelfDrivingCarControl-master/environment/jeju_camp_Data (debug = 0)
PlayerConnection initialized network socket : 0.0.0.0 55335
Multi-casting "[IP] 10.0.3.178 [Port] 55335 [Flags] 2 [Guid] 3442019743 [EditorId] 2618453742 [Version] 1048832 [Id] WindowsPlayer(LAPTOP-MDH1861V) [Debug] 0" to [225.0.0.222:54997]...
Started listening to [0.0.0.0:55335]
PlayerConnection already initialized - listening to [0.0.0.0:55335]
Initialize engine version: 2017.3.1f1 (fc1d3344e6ea)
GfxDevice: creating device client; threaded=1
Direct3D:
Version: Direct3D 11.0 [level 11.1]
Renderer: Intel(R) HD Graphics 5500 (ID=0x1616)
Vendor: Intel
VRAM: 1130 MB
Driver: 20.19.15.4642
Begin MonoManager ReloadAssembly
XInput1_3.dll not found. Trying XInput9_1_0.dll instead...
Input initialized.
Initialized touch support.
Setting up 2 worker threads for Enlighten.
Thread -> id: 3714 -> priority: 1
Thread -> id: 342c -> priority: 1
UnloadTime: 4.280462 ms
UnityAgentsException: The brain Brain was set to External mode but Unity was unable to read the arguments passed at launch.
at CoreBrainExternal.InitializeCoreBrain (Communicator communicator) [0x00059] in D:\UnityGames\ML_Agent_Jejucamp2017\Assets\Scripts\CoreBrainExternal.cs:37
at Brain.InitializeBrain (Academy aca, Communicator communicator) [0x0000e] in D:\UnityGames\ML_Agent_Jejucamp2017\Assets\Scripts\Brain.cs:209
at Academy.InitializeEnvironment () [0x00056] in D:\UnityGames\ML_Agent_Jejucamp2017\Assets\Scripts\Academy.cs:230
at Academy.Awake () [0x00002] in D:\UnityGames\ML_Agent_Jejucamp2017\Assets\Scripts\Academy.cs:208
(Filename: D:/UnityGames/ML_Agent_Jejucamp2017/Assets/Scripts/CoreBrainExternal.cs Line: 37)
Can you please suggest me step by step procedure to execute as i am beginer to RL concepts... Thank you.
Could you tell me the code execution process for each ipynb files in RL algorithms folder.
Hi @Kyushik
My system specifications are,
Ubuntu 18.04(64 bit)
Processor Intel® Core™ i3-8109U CPU @ 3.00GHz × 4
Graphics Intel® HD Graphics (Coffeelake 3x8 GT3)
Memory 7.7 GiB
I would like to modify your code using Unity hub, after opening the Unity SDK, while using play option I got the following error stating that "The communicator was unable to connect. Please make sure the external process is ready to accept communication with unity" as below.
As I am new to unity development, can you please suggest me how to run your car simulator as like any other games in unity in order to check and modify as I wish.
Thanks in advance.
Hello,
Is this an academic project? If so, were the results published elsewhere and where can I find the corresponding paper?
All the best,
Eduardo
Can I customize the reward function ? Which code file should I modify if I can ?
Thank you very much !
Hi @Kyushik
Can you please clarify me the following questions
Thanks in advance
Hi @Kyushik , I'm using Ubuntu 16.04. I have run the file Double_Dueling_image.ipynb. The Num_training = 1000000 is done but still the training is not stopped.
When will the training stop or how can I stop the training?
Thanks in Advance
I am glad to work on your codes. While working, the simulator is not loading. Can you describe the simulator connections? Is it uses Socket? or it starts automatically with the python code that you've provided?
Thanks,
Fayjie
@Kyushik I'm using Ubuntu 16.04. While I run the Double_Dueling_image.ipynb file I got the permission error.
env = UnityEnvironment(file_name=env_name)
# Examine environment parameters
print(str(env))
# Set the default brain to work with
default_brain = env.brain_names[0]
brain = env.brains[default_brain]
can you please help me with this
Thanks in Advance
hello,Can you help me solve this problem?Thanks very much.
I really don't know where this path(D:\Github\ML_Agent_Jeju_Simulator\ML_Agent_Jejucamp2017\Assets\Scripts) comes from.
SocketException: Unable to connect because the target computer actively refused.
at System.Net.Sockets.Socket.Connect (System.Net.IPAddress[] addresses, System.Int32 port) [0x000c3] in <4b9f316768174388be8ae5baf2e6cc02>:0
at System.Net.Sockets.Socket.Connect (System.String host, System.Int32 port) [0x00007] in <4b9f316768174388be8ae5baf2e6cc02>:0
at ExternalCommunicator.InitializeCommunicator () [0x000b2] in D:\Github\ML_Agent_Jeju_Simulator\ML_Agent_Jejucamp2017\Assets\Scripts\ExternalCommunicator.cs:128
at Academy.InitializeEnvironment () [0x00094] in D:\Github\ML_Agent_Jeju_Simulator\ML_Agent_Jejucamp2017\Assets\Scripts\Academy.cs:235
at Academy.Awake () [0x00002] in D:\Github\ML_Agent_Jeju_Simulator\ML_Agent_Jejucamp2017\Assets\Scripts\Academy.cs:208
(Filename: <4b9f316768174388be8ae5baf2e6cc02> Line: 0)
Hi,
I have downloaded the github repo and started the training process. May i know where the models are getting dumped?
I could not find any ".ckpt" files in that directory while the training was going on.
The unity simulator opens correctly manually the first time. But when I run it with the python code (DQN_image) it shows the unity splash screen logo and crashes. After that when I try to open it manually it crashes too. I have tried this with windows 10 and ubuntu 20.04.
Additionally, I am not able to run the .app of the simulator in mac Catalina.
Thanks,
Alex
thank for your code.when i run the ipynb, it remember me init() got an unexpected keyword argument 'file' ,can you tell me how to resolve
do i need to install other dependencies, like unity3d, to run this simulator? because the simulator always got crash and not responding when the progress is still observing. And also do you have any idea to make the simulator running smoothly on my PC?
btw I run this simulator with:
CPU : AMD Ryzen 5 4600H @3.00 GHz
GPU : NVIDIA Geforce GTX 1650
Memory : 8GB
Which ipynb file shoud we run in ml algorithms folder??
sry.. i am a beginner
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.