Proposal: NEW MPI splitting

[cobayasampler]

Proposal: NEW MPI splitting

[cmbant]

I think this would increase significantly the total numerical cost as you’d be evaluating locations that actually end up being rejected? The sequential ordering of vectors just determines the proposals, but the evaluation order is still Markovian.

The best way to test slow variants is usually to importance sample, that way you only need to evaluate the likelihood on final accepted (and possibly thinned) samples. Using importance sampling can also avoid Monte Carlo noise when comparing different theory calculations on the same samples.

[vivianmiranda]

Hi

Thank you.

"The sequential ordering of vectors just determines the proposals, but the evaluation order is still Markovian." -> Not sure if I understood that. But in anycase, while it is true that IS can be quite useful, when testing shifts in scales you don't know if a new effect (such as including non-limber or RSD or baryonic effect) can shift things more than a sigma or two, IS can fail. We see that in DES when we include for example RSD and baryonic effects in specific scale cuts. Having a higher numerical cost but smaller convergence time is a good deal for us. But anycase, just a proposal based on a sampler I am testing that do that.

Best
Vivian

[cmbant]

That’s true, though if IS doe not work that tells you already there is a problem (result not numerically stable).

By markovian I meant the next step depends on the previous point. So while you can propose N points at once, typically it will accept one of the first few, at which point you have to throw away the calculations for the remaining points and generate a new set of proposals based at the new point. You can increase the step distance to decrease the acceptance rate so less are wasted, but overall efficiency will go down. Usually one is running a bunch of runs in parallel anyway, so overall it’s more efficient to run each run on a modest number of CPU’s and the different runs in parallel. ThIngs like nonLimber should parallelize well with openmp (eg calculate using Camb) on each mpi process.

[JesusTorrado]

Hi both,

I am afraid I agree with Antony in that what you are proposing is likely not to increase performance significantly, and that you would have some trouble making it behave in a markovian way, and with both that IS can miss something new.

I remember we discussed at some point about internal caching of different parts of your calculation, which could very significantly improve performance now that Cobaya has manual parameter blocking. Did you get to implement that sort of caching where possible?

Another tip would be computing the initial covariant matrix from the Limber case (even if it's a little different, will be faster that guessing a full one from a slower non-Limber likelihood). (In terms of increasing acceptance rate, maybe emcee can also be a good approach? we can try to do a quick implementation)

Or even using PolyChord with a small number of live points and small n_repeats, which would not provide a good approximation of the evidence, but the Monte Carlo sample would be as good as MCMC, and it would take better advantage of MPI parallelisation for slow likeihoods (since at least to me it looks like you should worry more about that at this point than about the dimensionality, but maybe I am wrong).

When is this particular project due for? I have something in the pipeline that may be useful for it, using machine learning), but it's a more long term approach. We can discuss privately about it, if you want.

[vivianmiranda]

Thank you for the explanation. Unfortunately none of DES pipelines (CosmoSiS and Cosmolike) are OpenMP threaded (or can be easily threaded) - so that is why I am trying ways to make MH faster with MPI. The project will be based on the CosmoLike-Cobaya glued I have done so far for my current project - but more general. CosmoLike is already one of the official pipelines but your awesome Cobaya allows the use of MH and Polychord and also use of external data easily (and allowed me to use CAMB with Cosmolike). We may also use that for WFIRST. I will also try to include EMCEE as an additional sampler in Cobaya as you said. We can talk more in private. Project will start quick in fast - we will try to use for DES-Y3. Of course - when you have a paper about Cobaya - we will cite it everywhere we use the Cobaya-Cosmolike glue. Did you get to implement that sort of caching where possible Not yet :( - CosmoLike is a big code. Best Vivian On Sep 6, 2019, at 11:58 PM, Jesús Torrado <[email protected]<mailto:[email protected]>> wrote: Hi both, I am afraid I agree with Antony in that what you are proposing is likely not to increase performance significantly, and that you would have some trouble making it behave in a markovian way, and with both that IS can miss something new. I remember we discussed at some point about internal caching of different parts of your calculation, which could very significantly improve performance now that Cobaya has manual parameter blocking. Did you get to implement that sort of caching where possible? Another tip would be computing the initial covariant matrix from the Limber case (even if it's different, will be faster that guessing a full one from a slow likelihood). (In terms of increasing acceptance rate, maybe emcee can be a good approach? we can try to do a quick implementation) Or even using PolyChord with a small number of live points and small n_repeats, which would not provide a good approximation of the evidence, but the Monte Carlo sample would be as good as MCMC, and it would take better advantage of MPI parallelisation for slow likeihoods (since at least to me it looks like you should worry more about that at this point than about the dimensionality, but maybe I am wrong). When is this particular project due for? I have something in the pipeline that may be useful for it, using machine learning), but it's a more long term approach. We can discuss privately about it, if you want. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#38?email_source=notifications&email_token=AAYP32B6D3XMIJVVZHZN7ULQINGH7A5CNFSM4IUM4HZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6ESBWA#issuecomment-529080536>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAYP32FYKIN4ZLVMZGXUUOLQINGH7ANCNFSM4IUM4HZQ>.

[JesusTorrado]

Hi @vivianmiranda

Closing this for book-keeping reasons (there is nothing clearly defined to be done), but feel free to write us privately to discuss how to approach that problem.