h3jia/bayesfast

Next generation Bayesian analysis tools for efficient posterior sampling and evidence estimation.

bayesfast's Issues

loky does not print any messages if used in Jupyter

see https://stackoverflow.com/questions/55955330/printed-output-not-displayed-when-using-joblib-in-jupyter-notebook

parallelization within the SIT transformer

Now the SIT transformer uses use_parallel=False by default. Need to review this at some point.

Standalone use of GBS

Hi, first of all, really nice package!

I'm running MCMC chains in Stan, since computing the gradient of my distribution is not trivial and Stan's autodiff does it for me.
I wanted then to use the Gaussianized Bridge Sampler starting from the MCMC samples obtained through Stan, but the GBS class offers little documentation, and in the examples you provide it is never called directly.

So is there a way to use the Gaussianized Bridge Sampler as a standalone module?
I can also code in python the (unnormalized) log density function if needed, but not its gradient

NUTS sampling seems to become slower for the DES example

After updating to 0.1.0dev2, NUTS sampling of the surrogate density seems to become slower for the DES Y1 wCDM example, although the samples are fine. Should look into this at some point. Maybe it's due to some changes of the hyper-parameter default values.

optimize the import time

BF needs ~1s to import at the moment. Should study why it takes such a long time to import at some point.

can't install correctly

I installed the package from your command on Windows 10:
git clone https://github.com/HerculesJack/bayesfast
cd bayesfast
pip install -e .
but I got the error:
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Obtaining file:///C:/Users/10654/bayesfast
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Requirement already satisfied: cython in c:\users\10654\anaconda3\lib\site-packages (from bayesfast==0.1.0.dev2+7796b7c) (0.29.21)
Requirement already satisfied: numpy>=1.17 in c:\users\10654\anaconda3\lib\site-packages (from bayesfast==0.1.0.dev2+7796b7c) (1.19.2)
Collecting multiprocess
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/6b/d1/43fd2b01aebd1294777051dadc87b6326dccc1b89f82981f261eaa92d0e8/multiprocess-0.70.11.1-py38-none-any.whl (126 kB)
|████████████████████████████████| 126 kB 1.7 MB/s
Requirement already satisfied: scipy in c:\users\10654\anaconda3\lib\site-packages (from bayesfast==0.1.0.dev2+7796b7c) (1.5.2)
Requirement already satisfied: numdifftools in c:\users\10654\anaconda3\lib\site-packages (from bayesfast==0.1.0.dev2+7796b7c) (0.9.39)
Requirement already satisfied: scikit-learn in c:\users\10654\anaconda3\lib\site-packages (from bayesfast==0.1.0.dev2+7796b7c) (0.23.2)
Requirement already satisfied: matplotlib in c:\users\10654\anaconda3\lib\site-packages (from bayesfast==0.1.0.dev2+7796b7c) (3.3.2)
Requirement already satisfied: threadpoolctl in c:\users\10654\anaconda3\lib\site-packages (from bayesfast==0.1.0.dev2+7796b7c) (2.1.0)
Requirement already satisfied: extension-helpers in c:\users\10654\anaconda3\lib\site-packages (from bayesfast==0.1.0.dev2+7796b7c) (0.1)
Requirement already satisfied: dill>=0.3.3 in c:\users\10654\anaconda3\lib\site-packages (from multiprocess->bayesfast==0.1.0.dev2+7796b7c) (0.3.3)
Requirement already satisfied: joblib>=0.11 in c:\users\10654\anaconda3\lib\site-packages (from scikit-learn->bayesfast==0.1.0.dev2+7796b7c) (0.17.0)
Requirement already satisfied: python-dateutil>=2.1 in c:\users\10654\anaconda3\lib\site-packages (from matplotlib->bayesfast==0.1.0.dev2+7796b7c) (2.8.1)
Requirement already satisfied: cycler>=0.10 in c:\users\10654\anaconda3\lib\site-packages (from matplotlib->bayesfast==0.1.0.dev2+7796b7c) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in c:\users\10654\anaconda3\lib\site-packages (from matplotlib->bayesfast==0.1.0.dev2+7796b7c) (1.3.0)
Requirement already satisfied: certifi>=2020.06.20 in c:\users\10654\anaconda3\lib\site-packages (from matplotlib->bayesfast==0.1.0.dev2+7796b7c) (2020.6.20)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in c:\users\10654\anaconda3\lib\site-packages (from matplotlib->bayesfast==0.1.0.dev2+7796b7c) (2.4.7)
Requirement already satisfied: pillow>=6.2.0 in c:\users\10654\anaconda3\lib\site-packages (from matplotlib->bayesfast==0.1.0.dev2+7796b7c) (8.0.1)
Requirement already satisfied: six>=1.5 in c:\users\10654\appdata\roaming\python\python38\site-packages (from python-dateutil>=2.1->matplotlib->bayesfast==0.1.0.dev2+7796b7c) (1.15.0)
Installing collected packages: multiprocess, bayesfast
Running setup.py develop for bayesfast
ERROR: Command errored out with exit status 1:
command: 'C:\Users\10654\anaconda3\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\10654\bayesfast\setup.py'"'"'; file='"'"'C:\Users\10654\bayesfast\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps
cwd: C:\Users\10654\bayesfast
Complete output (50 lines):
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\include" "-IE:\Windows Kits\10\include\10.0.19041.0\ucrt" "-IE:\Windows Kits\10\include\10.0.19041.0\shared" "-IE:\Windows Kits\10\include\10.0.19041.0\um" "-IE:\Windows Kits\10\include\10.0.19041.0\winrt" "-IE:\Windows Kits\10\include\10.0.19041.0\cppwinrt" /Tctest_openmp.c /Foobjects\test_openmp.obj -openmp
test_openmp.c
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /MANIFEST:EMBED,ID=1 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\lib\x64" "/LIBPATH:E:\Windows Kits\10\lib\10.0.19041.0\ucrt\x64" "/LIBPATH:E:\Windows Kits\10\lib\10.0.19041.0\um\x64" objects\test_openmp.obj /OUT:test_openmp.exe
正在生成代码
已完成代码的生成
Compiling Cython/C/C++ extension with OpenMP support
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\include" "-IE:\Windows Kits\10\include\10.0.19041.0\ucrt" "-IE:\Windows Kits\10\include\10.0.19041.0\shared" "-IE:\Windows Kits\10\include\10.0.19041.0\um" "-IE:\Windows Kits\10\include\10.0.19041.0\winrt" "-IE:\Windows Kits\10\include\10.0.19041.0\cppwinrt" /Tctest_openmp.c /Foobjects\test_openmp.obj -openmp
test_openmp.c
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /MANIFEST:EMBED,ID=1 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\lib\x64" "/LIBPATH:E:\Windows Kits\10\lib\10.0.19041.0\ucrt\x64" "/LIBPATH:E:\Windows Kits\10\lib\10.0.19041.0\um\x64" objects\test_openmp.obj /OUT:test_openmp.exe
正在生成代码
已完成代码的生成
Compiling Cython/C/C++ extension with OpenMP support
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\include" "-IE:\Windows Kits\10\include\10.0.19041.0\ucrt" "-IE:\Windows Kits\10\include\10.0.19041.0\shared" "-IE:\Windows Kits\10\include\10.0.19041.0\um" "-IE:\Windows Kits\10\include\10.0.19041.0\winrt" "-IE:\Windows Kits\10\include\10.0.19041.0\cppwinrt" /Tctest_openmp.c /Foobjects\test_openmp.obj -openmp
test_openmp.c
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /MANIFEST:EMBED,ID=1 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\lib\x64" "/LIBPATH:E:\Windows Kits\10\lib\10.0.19041.0\ucrt\x64" "/LIBPATH:E:\Windows Kits\10\lib\10.0.19041.0\um\x64" objects\test_openmp.obj /OUT:test_openmp.exe
正在生成代码
已完成代码的生成
Compiling Cython/C/C++ extension with OpenMP support
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\include" "-IE:\Windows Kits\10\include\10.0.19041.0\ucrt" "-IE:\Windows Kits\10\include\10.0.19041.0\shared" "-IE:\Windows Kits\10\include\10.0.19041.0\um" "-IE:\Windows Kits\10\include\10.0.19041.0\winrt" "-IE:\Windows Kits\10\include\10.0.19041.0\cppwinrt" /Tctest_openmp.c /Foobjects\test_openmp.obj -openmp
test_openmp.c
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /MANIFEST:EMBED,ID=1 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\lib\x64" "/LIBPATH:E:\Windows Kits\10\lib\10.0.19041.0\ucrt\x64" "/LIBPATH:E:\Windows Kits\10\lib\10.0.19041.0\um\x64" objects\test_openmp.obj /OUT:test_openmp.exe
正在生成代码
已完成代码的生成
Compiling Cython/C/C++ extension with OpenMP support
running develop
running egg_info
writing bayesfast.egg-info\PKG-INFO
writing dependency_links to bayesfast.egg-info\dependency_links.txt
writing requirements to bayesfast.egg-info\requires.txt
writing top-level names to bayesfast.egg-info\top_level.txt
adding license file 'LICENSE' (matched pattern 'LICEN[CS]E*')
reading manifest file 'bayesfast.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'bayesfast.egg-info\SOURCES.txt'
running build_ext
building 'bayesfast.transforms._constraint' extension
creating build
creating build\temp.win-amd64-3.8
creating build\temp.win-amd64-3.8\Release
creating build\temp.win-amd64-3.8\Release\bayesfast
creating build\temp.win-amd64-3.8\Release\bayesfast\transforms
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Users\10654\AppData\Local\Temp\pip-build-env-lp306cxm\overlay\Lib\site-packages\numpy\core\include -IC:\Users\10654\anaconda3\include -IC:\Users\10654\anaconda3\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\include" "-IE:\Windows Kits\10\include\10.0.19041.0\ucrt" "-IE:\Windows Kits\10\include\10.0.19041.0\shared" "-IE:\Windows Kits\10\include\10.0.19041.0\um" "-IE:\Windows Kits\10\include\10.0.19041.0\winrt" "-IE:\Windows Kits\10\include\10.0.19041.0\cppwinrt" /Tcbayesfast/transforms/_constraint.c /Fobuild\temp.win-amd64-3.8\Release\bayesfast/transforms/_constraint.obj -openmp
_constraint.c
C:\Users\10654\AppData\Local\Temp\pip-build-env-lp306cxm\overlay\Lib\site-packages\numpy\core\include\numpy\npy_1_7_deprecated_api.h(14) : Warning Msg: Using deprecated NumPy API, disable it with #define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
creating C:\Users\10654\bayesfast\build\lib.win-amd64-3.8
creating C:\Users\10654\bayesfast\build\lib.win-amd64-3.8\bayesfast
creating C:\Users\10654\bayesfast\build\lib.win-amd64-3.8\bayesfast\transforms
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\10654\anaconda3\libs /LIBPATH:C:\Users\10654\anaconda3\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\lib\x64" "/LIBPATH:E:\Windows Kits\10\lib\10.0.19041.0\ucrt\x64" "/LIBPATH:E:\Windows Kits\10\lib\10.0.19041.0\um\x64" m.lib /EXPORT:PyInit__constraint build\temp.win-amd64-3.8\Release\bayesfast/transforms/_constraint.obj /OUT:build\lib.win-amd64-3.8\bayesfast\transforms_constraint.cp38-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.8\Release\bayesfast/transforms_constraint.cp38-win_amd64.lib
LINK : fatal error LNK1181: 无法打开输入文件“m.lib”
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\link.exe' failed with exit status 1181
----------------------------------------
ERROR: Command errored out with exit status 1: 'C:\Users\10654\anaconda3\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\10654\bayesfast\setup.py'"'"'; file='"'"'C:\Users\10654\bayesfast\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

review & merge the HMC & THMC samplers

See #5

set up the docs

implement checkpoint facility

This should be easy, as we have already separated the pickle-able and (possible) non-pickle-able components.

implement the global threadpool controller

We should implement a global BF threadpool controller, instead of always using one single thread.

Changing chain lengths on user side causes sampling to hang

When I trying to shorten the chain in the first sample step like this:

sam_0 = bf.recipe.SampleStep(s_1, alpha_n=2, reuse_steps=1,
fit_options={'use_mu_f': True},
sample_options={"sampler_options":{
"n_iter":1000,
"n_warmup":250
}})

bayesfast just hangs after completing the optimize step. A similar thing happens for the sampling that occurs in the optimize step when you try to change n_iter, n_warmup through "hmc_options"

I traced the issue to here:
https://github.com/HerculesJack/bayesfast/blob/52e9f405e6c80232ab523165e54406449ac4d0e1/bayesfast/core/recipe.py#L649
where it seems like for the surrogate model, a dictionary that has two separate elements (both dictionaries called "sampler_options" are being passed to the sampler at the same time. I don't know exactly why this would cause it to hang but it does.

I can get the result I want (shortened chains) by directly editing bayesfast here (https://github.com/HerculesJack/bayesfast/blob/52e9f405e6c80232ab523165e54406449ac4d0e1/bayesfast/core/recipe.py#L654), here (https://github.com/HerculesJack/bayesfast/blob/52e9f405e6c80232ab523165e54406449ac4d0e1/bayesfast/core/sample.py#L45), and here (https://github.com/HerculesJack/bayesfast/blob/52e9f405e6c80232ab523165e54406449ac4d0e1/bayesfast/core/sample.py#L53)

dask and loky backends do not respect the global random seed

compatibility issues regarding numpy and numdifftools

Seems that for now we'd better use numpy>=1.20 and numdifftools<=0.9.39.

Should check what happened with the latest numdifftools.

sharedmem backend may display multiple messages in the same line

due to lack of a multiprocessing-type ProcessLock

additional features for the importance sampling stage

1. to compute & extract additional parameters, like sigma8, using the true model
2. to allow additional importance weights from some other function (like the post_BAO in Planck), but maybe we don't need to have this in the Recipe: we can just add a utility function to transform the PostResult using the additional weights

Thanks!

cythonize the whole poly model and/or HMC sampler

Currently only the core evaluation of the poly model is cythonized. It might be useful to move the whole poly model to cython.

Also, some parts of the HMC sampler can be cythonized, although it's not clear how much this can speed up the sampling.

set up the continuous integration tests

cannot build correctly if not installed in editable mode

pytest would fail, saying that it cannot from . import _cubic, if the package is installed with just pip install . (instead of pip install -e .). My initial guess is that, without the editable mode, it will make a copy of the package under site-packages and compile the extensions there. However, pytest somehow calls the original directory, instead of the one under site-packages, so that it cannot find the compiled libraries.

set logp=-inf if the evaluation fails

Sometimes the target density is not defined in some regions of the parameter space. Maybe we should let BF automatically return logp=-inf for such cases?