bug: restore_checkpoint_path doesn't seem to work.

[awslabs]

bug: restore_checkpoint_path doesn't seem to work.

[gianlucadetommaso]

Hi Paul,
I could be wrong, but I suspect the issue is not the checkpoint here. The way we obtain the mean of the predictive distribution is by sampling from the posterior. The posterior approximation given by the Laplace approximation is a Gaussian, namely

$N(\theta^*, diag(\sigma^2))$

where $\theta^*$ is the MAP and $\sigma$ is a vector of standard deviations for each parameter.

Now, if the estimate $\sigma$ is very bad - for instance, it is too large - the samples that you will obtain from this distribution might be too far from $\theta^*$ to provide good predictive estimates, and therefore a good accuracy.

I have noticed myself that the Laplace approximation may suffer quite a bit of this issue. As a possible solution, try doing only last-layer Laplace by enabling the freeze_fun in the FitOptimizer. This might help.

Let me know!

[PaulScemama]

@gianlucadetommaso okay! Coincidentally I just tried that myself and yes you're right -- it yields more reasonable results now, thank you :) That's good to know about the Laplace approx.