Giter Site home page Giter Site logo

Comments (5)

devmotion avatar devmotion commented on June 16, 2024

I guess the problem is that the the samples in vi are saved as Real, and hence rerunning leads to the error since the evaluation of the logpdf in Bijectors calls eps for Real which is not defined. It seems reasonable that it doesn't affect the likelihood context since we don't evaluate the logpdf there. I guess this issue should be transferred to Bijectors, maybe it's possible to avoid eps or define some _eps instead that falls back to eps(Float64) for Real.

from bijectors.jl.

phipsgabler avatar phipsgabler commented on June 16, 2024

Hm, the funny thing is that eps is only used with Dirichlet: https://github.com/TuringLang/Bijectors.jl/blob/master/src/Bijectors.jl#L124.

I guess this is the reason:

julia> logpdf(Dirichlet(2, 1.0), [1, 0])
NaN

julia> logpdf(Dirichlet(2, 1.0), [1 + eps(Float64), eps(Float64)])
0.0

Probably either of the following should be fine:

julia> logpdf(Dirichlet(2, 1.0), nextfloat.(Real[1.0, 0.0]))
0.0

julia> logpdf(Dirichlet(2, 1.0), Real[1.0, 0.0] .+ eps.(Real[1.0, 0.0]))
0.0

(and eps is defined internally through nextfloat, so I'd prefer the first).

from bijectors.jl.

mohamed82008 avatar mohamed82008 commented on June 16, 2024

_eps should be used here. I will make a PR.

from bijectors.jl.

devmotion avatar devmotion commented on June 16, 2024

l = logpdf(d, x .+ eps(eltype(x)))

is actually not defined for x >= 1 - eps mathematically even though it works with Distributions (and incorrect for any x > 0). Maybe one should apply the same fix as in the SimplexBijector and rescale x to x * (1 - 2 * eps) + eps, which leads to values in [eps, 1-eps] if x in [0, 1] before and would be consistent with the calculation in SimplexBijector. Of course, that still doesn't work if x < 0 or x > 1 due to numerical issues, so probably the only numerically stable way would be to work with the logarithm of the unnormalized Gamma random variates instead and apply the softmax function later on if needed (e.g., for parameterizing a categorical distribution).

from bijectors.jl.

devmotion avatar devmotion commented on June 16, 2024

More generally speaking, I'm wondering if for sampling and optimization in the unconstrained space we could use a rand_trans function that generates samples in the transformed space directly to avoid these issues altogether. E.g., there exist algorithms for sampling X with exp(X) \sim Gamma(a, 1) directly in log-space, which avoids the issue of getting zero values for small shape parameters a. It could always fall back to sampling in the original space and applying the transformation afterwards, but a more sophisticated implementation could avoid numerical issues whenever possible.

from bijectors.jl.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.