nuno-faria/tetris-ai

A deep reinforcement learning bot that plays tetris

tetris-ai's Issues

Unable to reproduce the reported results

I retrained your model using the default hyperparameters in run.py, but my results are not similar to the reported results, the score is still too low after 2000 episodes. Could you please give me any advice to reproduce your results?

Missing license

Hi @nuno-faria,

I am currently working with some other students on our own tetris rl project. Can we use your project / results for our project? There is currently no license in this repository, thus we do not know whether we can use your project. Could you please add one?

Thanks a lot :)

help for students ?

Hello, I am an engineer student, as a school project in reinforcement learning, we tried to give a look at your code and see if there were any way to improve even more the agent. So far, we'v etried changing some parameters and implemented behavioral cloning but haven't had much result. I also tried to train the AI with different iteration number and noticed that it doesn't affect the result as much as I thought. For example, sometimes I had better results with 2000 episodes than 10000.
Do you have any idea for us to work with ?
Thanks you !

The code committed has CRLF as line endings.

It is discouraging to have the code with CRLF line endings in the git repostiory, because there might be a lot of unnecessary merge conflicts, when users work under different OSes.

To avoid commit files with CRLF, the git should be configured properly (autocrlf = input) and IDE.
E.g. for PyCharm https://www.jetbrains.com/help/pycharm/configuring-line-endings-and-line-separators.html

Thanks

Bellman Equation is not correct

Hey man,

I used your train function in my project because of its optimization. It runs the fit function in one batch an accelerates training quite a bit, thx for that.

Problem is that your Bellman equation is slightly wrong. The original Bellman equation states that the best policy is the one that leads to the next state that yields the highest possible return.

Check out this blog or their sources: Deeplizard

Basically what you need to do is instead of adding reward and next_qs[i] you want to add reward and max(next_qs)

next_states = np.array([memory[3] for memory in batch])
next_q_values = self.model.predict(next_states, verbose=0)
max_next_q_value = np.max(next_q_values, axis=1)

x = []
y = []

for index, (state, action, reward, next_state, done) in enumerate(batch):
    if done:
        new_q = reward
    else:
        new_q = float(reward + self.gamma * max_next_q_value)

    x.append(state)
    y.append(new_q)

self.model.fit(np.array(x), np.array(y), batch_size=len(
    x), use_multiprocessing=True, workers=16, verbose=0)

I copy and pasted this from my code, so the variable names are different, but I think you get the point.

https://github.com/nuno-faria/tetris-ai/blob/4d01877100870e2a6a1ef84dc955354e534589ae/dqn_agent.py#L132C64-L132C64

Again thanks for this cool optimization!
Keep up the good work.

Problem with new version of libs

There is a problem in log.py, in new version of libs, its not possible to call 'from tensorflow.summary import FileWriter' and to call function self._write_logs(stats, step)

User error

I am trying to build a similar model using Tensorflow 2 with a slightly different neural network. The only error I continue to encounter is in the agent's "best_move" function. It continues to provide an error stating I am comparing "float" values to "None" values. I am unsure if this is a result of my improperly building the network, and as a result I am incorrectly feeding values to the network or if this is due to me incorrectly building the agent.