stheid/constraint_water_treatment_gym

Simple OpenAI Gym implementation of a simple water treatment plant. The gym is created with constrained reinforcement learning in mind.

constraint_water_treatment_gym's Issues

Extend stablebaseline 3 PPO implementation by with penalties

In each policy update step, the penalty function is called with the policy and the current state. this results in a gradient.

Currently i have two ideas for such a penalty function, both need a (computationally) differentiable system approximation function f:

• calculate action a = policy(state), calculate succ-state = f(state,action), calculate penalty gradient on succ-state
• instead of taking the current state, the policy is evaluated on a predefined set of states. the rest simmilar

In order to use Pytorch autograd all functions need to be formulated in pytorch tensors, such that the policy can backpropagate from the loss

Investigate Lagrangian implementation