Exception resulting from illegal action is not propagated in obs.simulate

[rte-france]

Exception resulting from illegal action is not propagated in obs.simulate

[jhmenke]

Furthermore, using "change_line_status" instead of "set_line_status" seems also to be causing some issues.

Status before the action is line0 is connected and line5 is disconnected. This action should therefore connect line5 while disconnecting line0:

This action will:

• NOT change anything to the injections
• NOT perform any redispatching action
• NOT force any line status
• switch status of 2 powerlines ([0 5])
• NOT switch anything in the topology
• NOT force any particular bus configuration

Again, an exception was thrown but hidden from the user:

                    if bus_ex == 0 or bus_or == 0:
                        raise InvalidLineStatus("Line {} was disconnected. The action switched its status, "
                                                "without providing buses to connect it on both ends.".format(i))

However, i don't quite understand why i should define a bus for a line that was previously disconnected by using the change_line_status operator. Reconnecting it with the change_line_status should just restore the previous connection state?

[BDonnot]

Can you please send me a piece of code that does just that ? I'm asking because I think the reason of why the action was not implemented has been returned in the "info" part of the env.step method.

There is however a problem for "obs.simulate" that should return these information but does not at the moment obs.simulate is a bit behind env.step and I will work on this in the next release.

[BDonnot]

By design, i made the choice not to propagate the error because i think it's common in "video game".

In pacman for example, when you face a wall and you ask to go ahead, the game don't crash, and nothing is done. This behaviour is pretty common in RL environment: at the beginning of the training of an agent, they will do this kind of "mistake" a lot.

I agree this should be more explicit for non RL agent. I will set a flag in the "info" dictionnary when an exception was raised in the backend.

[jhmenke]

Can you please send me a piece of code that does just that ? I'm asking because I think the reason of why the action was not implemented has been returned in the "info" part of the env.step method.

Ah that may be it! I was looking at another flag returned by simulate, but it could have been the wrong one or it's not yet implemented as you wrote! I can send you the relevant code tomorrow if that's still an issue after you helpful comments.

By design, i made the choice not to propagate the error because i think it's common in "video game".

Sure, it makes total sense for reinforcement learning, sorry! I have to get used to the different approach of not requiring any reasons for why something does not work and just using the reward as the signal.

It seems to be personal confusion for most of these issues and not actually bugs in your software :) Again, thanks for the quick support!

[BDonnot]

I will be more explicit, and especially in the "obs.simulate" function that need to be reworked :-)

You can look in the "env.step" last returned value, which is called "info":

observation, reward, done, info = env.step(action)

This should give you more information. However, in the current version, obs.simulate does not reflect all of this information.

Thanks again for the valuable feedback :-)

[BDonnot]

Hello,

This should be fixed in the new 0.5.7 version that i just released today.

Benjamin