aechpro/rlgym-ppo

A vectorized implementation of PPO for use with RLGym

rlgym-ppo's Issues

Multi-Discrete Policy NaNs on CPU

Very niche problem, when using multi-discrete, and apparently only while running without a GPU, at some point a NaN pops up in probabilities. I caught in ppo_learner on line 90, old_probs had a single NaN, if left to go another cycle, all policy weights become NaNs. No non-finite values caught in obs to cause this.

Error after finishing the Learning process

I am getting this error once the Learning process finishes:

Traceback (most recent call last):
  File "c:\Users\x\Desktop\GetEmRLGym\venv\lib\site-packages\rlgym_ppo\batched_agents\batched_agent.py", line 43, in batched_agent_process
    pipe.send(("env_step_data", (obs, rew, done)))
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.3568.0_x64__qbz5n2kfra8p0\lib\multiprocessing\connection.py", line 211, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.3568.0_x64__qbz5n2kfra8p0\lib\multiprocessing\connection.py", line 295, in _send_bytes
    nwritten, err = ov.GetOverlappedResult(True)
BrokenPipeError: [WinError 109] Ha terminado la canalización 
Failed to send stop signal to child process!```

It repeats many times. I guess once for each subprocess.

Doesn't seem to affect anything tho.

Render error

Getting this error:

Traceback (most recent call last):
  File "C:\Users\x\Desktop\BrutusRL\.venv\lib\site-packages\rlgym_ppo\batched_agents\batched_agent.py", line 45, in batched_agent_process
    env.render()
  File "C:\Users\x\Desktop\BrutusRL\.venv\lib\site-packages\rlgym_sim\gym.py", line 91, in render
    rlviser.render_rlgym(self._prev_state)
TypeError: render_rlgym() missing 2 required positional arguments: 'tick_rate' and 'gym_state'