[Bug Report] Getting Error from "D:\PongDQN_RL\venv\lib\site-packages\gym\wrappers\compatibility.py"

[creativekids11]

[Bug Report] Getting Error from "D:\PongDQN_RL\venv\lib\site-packages\gym\wrappers\compatibility.py"

[pseudo-rnd-thoughts]

Could you provide a minimal working example to test with?
It is unclear from your stack trace what the issue is

[creativekids11]

PongDQN_RL.zip
This is the code (02_dqn_pong.py) to train (where the error occured)

[creativekids11]

BTW, pytorch with version 2.2.0 cuda 11.8 is also installed in the same env. IDK if it can cause any problem.

[pseudo-rnd-thoughts]

If you using gym v0.26.2 and the latest atari version then you shouldn't be using apply_api_compatibility

[creativekids11]

If you using gym v0.26.2 and the latest atari version then you shouldn't be using apply_api_compatibility

Didn't work after removing it

[creativekids11]

The error changed to:

E:\PongDQN_RL\venv\lib\site-packages\gym\utils\passive_env_checker.py:233: DeprecationWarning: np.bool8 is a deprecated alias for np.bool_. (Deprecated NumPy 1.24)
if not isinstance(terminated, (bool, np.bool8)):
Traceback (most recent call last):
File "E:\PongDQN_RL\02_dqn_pong.py", line 141, in
agent = Agent(env, buffer)
File "E:\PongDQN_RL\02_dqn_pong.py", line 63, in init
self._reset()
File "E:\PongDQN_RL\02_dqn_pong.py", line 66, in _reset
self.state, _ = self.env.reset()
File "E:\PongDQN_RL\venv\lib\site-packages\gym\core.py", line 379, in reset
obs, info = self.env.reset(**kwargs)
File "E:\PongDQN_RL\lib\wrappers.py", line 117, in reset
return self.observation(self.env.reset())
File "E:\PongDQN_RL\venv\lib\site-packages\gym\core.py", line 379, in reset
obs, info = self.env.reset(**kwargs)
File "E:\PongDQN_RL\venv\lib\site-packages\gym\core.py", line 379, in reset
obs, info = self.env.reset(**kwargs)
File "E:\PongDQN_RL\lib\wrappers.py", line 20, in reset
(obs, reward, terminated, truncated, info) = self.env.step(1)
ValueError: not enough values to unpack (expected 5, got 4)

[creativekids11]

Have you got the error replicated with the code. Or else i can share name and versions of all the libraries for you to try in an venv.

[pseudo-rnd-thoughts]

Can you make a minimal working example of the code with the bug, there is too much for me to look through.
Try testing commenting out a wrapper each time to see which one causes the issue

[creativekids11]

I managed to fix this error but now the input format is not approving, to the model.

[creativekids11]

Could you kindly provide me with an latest code with dqn which can solve pong. As there are no updates online, plus no available latest code.

[pseudo-rnd-thoughts]

There are a host of projects that implement DQN for you to see: CleanRL, Stablebaselines3, Tianshou, etc

[creativekids11]

CleanRL docs was very helpful, thanks: https://docs.cleanrl.dev/rl-algorithms/dqn/