Impala agent Code Issue

[praveen-palanisamy]

Impala agent Code Issue

[praveen-palanisamy]

No. The Reward functions aren't expected to be modified by/from the Agent algorithm/library.

Reward functions are defined in/by the RL environment. In this case, in the Macad-Gym environments.

[praveen-palanisamy]

Hi @AizazSharif ,
Could you post your training logs to see the status of your training runs?

Once the agents are trained, you can test their learned policies in a single instance of the environment (with rendering enabled) to visualize how the cars are controlled by the Agents.

[AizazSharif]

Please find the logs below

output.txt
Also, I used env_config["env"]["render"] = False to disable the rendering but its still enabled.

[AizazSharif]

And below are the logs maintained by Macad-agents.

macad-gym.log
server_36779.log
server_46275.log
server_51703.log
server_52455.log

[praveen-palanisamy]

Thanks for the logs. Could you post this file?: /home/aizaz/ray_results/MA-Inde-IMPALA-SSUI3CCARLA/IMPALA_HomoNcomIndePOIntrxMASS3CTWN3-v0_0_2021-02-24_08-37-426o165tlh/error_2021-02-24_08-40-09.txt
That's where the errors seemed to be logged to.

[AizazSharif]

Please find the attached logs you mentioned.

error_2021-02-24_08-40-09.txt

[AizazSharif]

@praveen-palanisamy
From what I have observed, impala code within macad agents stops in each epoch due to shape tensor mismatch even when I run the code on default parameters.

[praveen-palanisamy]

It looks like a proper observation pre-processor is not registered with RLLib to resize the observation images (shape: (168, 168, 3)) to RLLib's default Conv layer input shape (84, 84, 3). I can push a fix tomorrow.
Meanwhile, you can just change the MACAD-Gym env config (In this file for example) listed below:

            "x_res": 168,
            "y_res": 168,

To 84 for all the Actors.

[AizazSharif]

Thanks, I will try this change.
Also, I wanted to ask is it possible to add your own reward function within macad-agents?

[AizazSharif]

Makes sense. Thanks for clarifying.
I will try out the change and let you know.

[AizazSharif]

I have tried changing Actors x and y dimensions to 84 but it's still giving an error.

ValueError: Cannot feed value of shape (1, 168, 168, 3) for Tensor 'car1/Placeholder:0', which has shape '(?, 84, 84, 3)'

error_2021-02-25_08-39-10.txt

[praveen-palanisamy]

Hi @AizazSharif ,

I pushed an update that should fix the issue you are facing. Could you try running on your side and confirm?

[AizazSharif]

Hi @praveen-palanisamy,

I reinstalled the whole environment and github repo but the error is still there. The logs are from today's testing.
The command is: python -m macad_agents.rllib.impala_agent_independent

error_2021-02-27_11-38-21.txt

[AizazSharif]

I ran python -m macad_agents.rllib.ppo_multiagent_shared_weights and its working fine without errors. But in impala_agent_independent.py the bug might still persist.