Comments (4)
I think the counter idea is probably a good idea in theory, but I wonder how well it'll hold up in practice. Given our push towards more community inclusion in terms of package design, is it possible to make the VarInfo
stuff extremely robust given an externally produced sampler? The fixed access pattern may cause troubles for some people unaware of what might be viewed as strange behavior.
As for the fields of the VarInfo
, how does using a static array provide a performance improvement? Aren't the fields themselves fairly quick to read/write? Regardless, I think the dictionary access has to go. It's kind of a band-aid that really shouldn't be in the middle of a performant system.
from dynamicppl.jl.
Well, the fixed access pattern is a property of the models not the sampler. So ideally the sampler choice should not interfere with that. This can be triggered with an opt-in strategy. For instance, we can allow the passing of different keyword arguments to the @model
macro, e.g. @model .... static = true, det = true
. We can also make some diagnostic test function to make sure the the model is truly static and deterministic. This can compare the run of the model with the static VarInfo
and counter with that of a traditional VarInfo
. We can do this by running the model multiple times, but in reality we can only be as sure as the user is since the possibility of an untriggered randomness is hard to eliminate. That said, for the advanced users, which we seem to have a fair number of, this is a nice feature to have for small and medium sized models.
As for the fields of the VarInfo, how does using a static array provide a performance improvement?
In general, the Julia compiler isn't too liberal with optimizations involving Base.Array
as it is with StaticArrays
. The power of static arrays comes from the compiler being able to more heavily optimize it. This includes loop unrolling, eliding allocations, faster linear algebra, among others. If you let this propagate through the logpdf
, AD and the entire model, those improvements can build up and become significant. So it is not about the read and write per se, it is about what we do with what we read after, e.g. https://github.com/TuringLang/Turing.jl/blob/master/src/inference/hmc.jl#L598.
The dictionary access is hard to get rid of in the most general case where variables can pop up at any order and any point in the sampling. You need a way to keep track of where everything is, and idcs
plays this critical role.
from dynamicppl.jl.
I think we might not be able to use order only, as we need to access by name for the Gibbs interface. But if we aim a specialized VarInfo for HMC and PG only, this proposal makes a lot of sense (and I also think this is the right direction).
Also, another specialized VarInfo can be created for PG that stores r.v.s with Julia types instead of vectorized version. Then we can create different types of VarInfo that this is necessary for different inference algorithm, and get the best performance while keeps the flexibility! (Ofc this could also be triggered by something like @model .... static = true, det = true
or @staticmodel
as you mentioned.)
from dynamicppl.jl.
Closed in favour of #309
from dynamicppl.jl.
Related Issues (20)
- Derived variables from data on the LHS of tilde HOT 10
- Decouple from Distributions.jl HOT 8
- ThreadSafeVarInfo(::SimpleVarInfo) bug with ForwardDiff HOT 1
- Need alternative to `NamedTuple` for `SimpleVarInfo` HOT 12
- Storing returned values in `VarInfo`.
- `BangBang.setindex!!(vi::SimpleVarInfo{<:AbstractDict}, val, vn::VarName)` returns eltype Any arrays when `vn` is multidimensional HOT 1
- Remove `tonamedtuple`
- Can't use `@info` in models HOT 1
- Move TestUtils to an extension
- `SimpleVarInfo` sometimes returns a `SimpleVarInfo` with `Int`-type `logp` and creates down-stream errors HOT 1
- Roadmap for removal of `Selector` shenanigans HOT 2
- change defaults: @addlogprob! not included in PriorContext() HOT 5
- Type-instability when using type-parameters HOT 1
- Type instability in unflatten for medium / large tuples HOT 6
- DynamicPPL..jl tests 23 Broken. HOT 3
- Introduce a convenience function, `AutoType`, to hide `TypeWrap`.
- Issue with ReverseDiff and undef on Julia v1.7 HOT 1
- Add an exported function to get variable names for a model
- Replace uage of `typeof` with `Core.Typeof` HOT 2
- `minibatch` / `stochastic_gradient` operation on models HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dynamicppl.jl.