Showing Input-Output dimensions

[turinglang]

Showing Input-Output dimensions

[torfjelde]

Thanks!

So this is heavily related to issue #35 and thus PR #44. In PR #44 we've introduced dimensionality into the Bijector type, so it's now Bijector{D}. But this is slightly different from what you're suggesting:

1. In #44 D refers to the dimensionality rather than the size of the array, e.g. SimplexBijector acts on 1-dim input (vector) while Logit acts on 0-dim input (real).
2. We assume input- and output-dimensions D to be equal.

With that being said, as mentioned in the comment-section of #35, we want to introduce a fixed size StaticBijector or something which also has the input- and output size in the type similar to StaticArrays.jl. This seems closer in nature to what you're referring to, right?

The only difference is that you want the "flattened" structure. I'm also not sure if I'm comfortable with the notion of using the flattened number of components. I see why you'd do that though, e.g. need to consider a matrix as a vector to talk about integration, but seems like an unnecessary detail to expose to the user. Seems better to give the size of the matrix, e.g. (3, 3) rather than 9. Is there a particular reason why you'd want it "flattened" like this?

Finally, I'd ideally like to preserve the notion of the mapping being one-to-one and onto, i.e. personally I don't like reducing the number of components from the input to the output. I do agree it's not always obvious that we'd like this, e.g. SimplexBijector is really acting on a K - 1 subspace of the K vector you pass into it. This is a result of the underlying constraint on the length K input vector. I'm honestly not sure how to handle this properly at the moment, so for now we just set one output component to 0 (this is also done in other packages, e.g. Stan or PyTorch for the same transformation).

I'm honestly not 100% sure what the best approach for all of this is. We were also briefly considering actually specifying the input- and output types. This way you could potentially specialize for doing work on GPU, etc. Again, not entirely sure what the best approach is yet. Feel free to join the discussion over at #35 though!