Total params is twice as large

[andreped]

Total params is twice as large

[andreped]

This is likely related to #25

[andreped]

Update: This does not happen after loading. It happens after model wrapping. It is likely due to the usage of tf.Variable for creating the gradients, which should be accumulated (see here).

Also note that by calling the reinit_grad_accum method, externally, further increases the nontrainable weights count (see here).

Both behaviours are wrong and needs proper solutions.

[andreped]

We may have to do something like this.

[tno123]

Unfortunately, double parameters during gradient accumulation is expected behavior due to the accumulation of gradients for each parameter separately using the gradient_accumulation variables.