The `Problem Formulation.ipynb` notebook has weird Policy data retention issues, potentially corrupting results

[quaquel]

The `Problem Formulation.ipynb` notebook has weird Policy data retention issues, potentially corrupting results

[quaquel]

The current implementation is designed to be flexible with respect to the number of planning steps you want to consider. Given that for each planning step you need levers, there is no easy way to specify default values. If you look at the modified notebook, you see my quick trick to work around this. In essence, create a dict with all levers set to 0 and only update those levers you want to change.

[quaquel]

Not sure I fully understand the issue without trying to reproduce it. However, you must make sure that custom policies have values for all levers. This value can of course be 0.

The model.policy attribute is generic (part of the inner workings of the workbench) and is updated when you are evaluating a new policy. See experiment_runner.py in the ema_workbench and AbstractModel.model_init.

[EwoutH]

However, you must make sure that custom policies have values for all levers.

Right, that really is counter intuitive in the Problem Formulations notebook, since in the second example with custom policies only a few specific levers are set, which implies the other values are either random or a default value and thus don’t have to be explicitly set. However, it looks like the values are copied over from the previous run in cell 8.

[quaquel]

I figured out what is causing this and it is indeed not desired behavior. In short, in DikeNetwork.__call__, the code sets G to self.G (see line 120 in dike_model_funciton.py). However, dike raising options are set as attributes on the node in the network (see line 157 in dike_model_function.py). This attribute persists for a next policy unless explicitly overwritten. The easiest fix is to replace G = self.G with G = copy.deepcopy(self.G). This will cause the notebook to throw an error in cell 9 because some levers are missing.

Still have to check the knock on consequences of this in other parts of the example code and fix those before pushing this fix.

[EwoutH]

Thanks for getting back, and implementing the fix above.

Could we prevent manually having to define all Policy levers by assigning a default value to the RfR, EWS and dike levers (in problem_formulation.py)? If so, how do we parse this when defining policies? Are there build in functions to use the default levers for in a Policy and modify only a few lever values?

https://github.com/quaquel/epa1361_open/blob/91d2b63942c576aaf617bddedd55286fb3c72a04/final%20assignment/problem_formulation.py#L56-L65

https://github.com/quaquel/epa1361_open/blob/91d2b63942c576aaf617bddedd55286fb3c72a04/final%20assignment/problem_formulation.py#L79-L84

[EwoutH]

It seems we have a comparable issue in the Optimization Moro.ipynb notebook. Could you rerun it with the current version of the EMAworkbench and confirm that it works?

Unfortunately this time I also the SequentialEvaluator keeps running endlessly.

[EwoutH]

When I talked to a few other students after the debate today (it was nice btw!), I noticed how they were also unclear about why how the Policy levers were influenced by previous runs in this notebook. I explained shortly that you have to set all the levers explicitly, otherwise they are copied, but maybe this is something worth explaining a bit more extensively in the notebook and mentioning in an announcement of such.