nki-ccb/sobolev_alignment

Sobolev alignment of deep probabilistic models for comparing single cell profiles

Sobolev Alignment

This GitHub repository contains the implementation of Sobolev Alignment, a computational framework designed to align pre-clinical and tumor scRNA-seq data. Sobolev Alignment combines a deep generative model with a kernel method to detect non-linear processes that are shared by a source (e.g., cell line) and a target (e.g., tumor) dataset.

Getting started

Please refer to the documentation. In particular, the

• API documentation.

Installation

You need to have Python 3.8 or newer installed on your system. If you don't have Python installed, we recommend installing Mambaforge.

The installation can be done in two steps.

1. Install Sobolev Alignment

You can install Sobolev Alignment and (almost) all dependencies using the following command:

pip install sobolev-alignment

The resulting package is ready to use, but will use scikit-learn instead of Falkon, resulting in largely sub-optimal performances.

2. Install Falkon

To employ large-scale GPU-accelerated kernel methods, we turn to Falkon. The installation notice for Falkon is available on the FalkonML documentation. The previous installation procedure has already taken care of the various dependencies required for Falkon (i.e., cython, scipy and torch.)

Tutorial

In the folder tutorial, you will find different tutorial in the form of Jupyter notebooks examplifying how to use the package. Specifically:

• process_data.ipynb: example on how to pre-process the data prior to use Sobolev Alignment. If you data has already been processed, the main step consists in adding the counts as layers, e.g., an.layers['counts'] = an.X.
• tutorial_simple.ipynb: basic example on how to run Sobolev Alignment with basic parameters.

Additional packages need to be installed to use MNN on top of Sobolev Alignment:

• rpy2, which can be installed using pip (pip install rpy2).
• The R package batchelor. To do so, after activating your conda environment, get into R and enter the following commands: ''' if (!require("BiocManager", quietly = TRUE)) install.packages("BiocManager")

BiocManager::install("batchelor") '''

Frequent issues

Issues with the compiler.

Due to incompatibilities between g++, gcc and cuda, the installation of FalkonML sometimes fails. The following elements can help alleviate potential issues:

• Prior to installing Falkon, re-install torch 1.11.
• Check compatibility between your cuda version and the one installed with torch.
• Using cxx-compiler=1.2.0 (available on conda-forge) is compatible with cuda 11.3.

Issues with Jaxlib (MacOS)

For Mac users, the jaxlib version installed from PyPI sometimes returns issues. We then advise to re-install jaxlib from condo, and subsequently re-install dcvi-tools:

mamba install jaxlib
mamba install scvi-tools

Incompatibilities with numba

Errors are sometimes raised due to numba inconsistencies. The errors raised were due to clashes between different packages. Re-installing numba seem to have fixed the issues:

pip install numba --force-reinstall

Please feel free to contact the development team by e-mail or by creating an issue.

Workflow presentation

Release notes

See the changelog.

Contact

For questions and help requests, you can reach out at the following e-mail address: s [dot] mourragui [at] hubrecht [dot] eu.

Citation

If you find this package useful, please cite our publication:

Identifying commonalities between cell lines and tumors at the single cell level using Sobolev Alignment of deep generative models, Mourragui et al, 2022, Biorxiv

sobolev_alignment's Issues

Broken link to API / tutorials

Report

Hello! Very interesting method, I was wondering if there was a plan to release the notebooks used to generate the figures in the manuscript? The link to the API also appears to be broken, would it be possible to update this? thank you so much for your help!

Version information

No response

error in sobolev_alignment_clf.krr_model_selection()

Report

Hi Soufiane, the scVI training runs no problem, but the KRR gives the following error:

---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
Cell In[27], line 3
      1 #KRR approximation
      2 # Select best model for KRR
----> 3 sobolev_alignment_clf.krr_model_selection(
      4     X_source=source_an,
      5     X_target=target_an,
      6     M=500,
      7     same_model_alignment_thresh=0.95
      8 )

File /mambaforge/envs/sobolev/lib/python3.9/site-packages/sobolev_alignment/sobolev_alignment.py:1018, in SobolevAlignment.krr_model_selection(self, X_source, X_target, M, same_model_alignment_thresh)
   1016 X_input = {"source": X_source.X, "target": X_target.X}
   1017 if self._fit_params["log_input"]:
-> 1018     X_input = {k: np.log10(X_input[k] + 1) for k in X_input}
   1020 # Compute sigma after re-scaling data (if required)
   1021 if self._fit_params["frob_norm_source"]:

File /mambaforge/envs/sobolev/lib/python3.9/site-packages/sobolev_alignment/sobolev_alignment.py:1018, in <dictcomp>(.0)
   1016 X_input = {"source": X_source.X, "target": X_target.X}
   1017 if self._fit_params["log_input"]:
-> 1018     X_input = {k: np.log10(X_input[k] + 1) for k in X_input}
   1020 # Compute sigma after re-scaling data (if required)
   1021 if self._fit_params["frob_norm_source"]:

File /mambaforge/envs/sobolev/lib/python3.9/site-packages/scipy/sparse/_base.py:462, in _spbase.__add__(self, other)
    460         return self.copy()
    461     # Now we would add this scalar to every element.
--> 462     raise NotImplementedError('adding a nonzero scalar to a '
    463                               'sparse array is not supported')
    464 elif issparse(other):
    465     if other.shape != self.shape:

NotImplementedError: adding a nonzero scalar to a sparse array is not supported

There are raw counts in both .X and .layers['counts'] for both source and target

Version information

-----
anndata             0.9.2
numpy               1.24.4
pandas              2.0.3
scanpy              1.9.3
scvi                1.0.3
session_info        1.0.0
sobolev_alignment   0.4.1
torch               2.0.0
tqdm                4.66.1
-----
Cython              3.0.0
PIL                 9.4.0
absl                NA
aiohttp             3.8.5
aiosignal           1.3.1
annotated_types     0.5.0
anyio               NA
arrow               1.2.3
asttokens           NA
async_timeout       4.0.3
attr                23.1.0
babel               2.12.1
backcall            0.2.0
backoff             2.2.1
brotli              1.0.9
bs4                 4.12.2
certifi             2023.07.22
cffi                1.15.1
charset_normalizer  3.2.0
chex                0.1.7
click               8.1.7
cloudpickle         2.2.1
colorama            0.4.6
comm                0.1.4
contextlib2         NA
croniter            NA
cycler              0.10.0
cython              3.0.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.8
decorator           5.1.1
deepdiff            6.3.1
defusedxml          0.7.1
docrep              0.3.2
entrypoints         0.4
executing           1.2.0
falkon              0.8.3
fastapi             0.101.1
fastjsonschema      NA
flax                0.6.1
fqdn                NA
frozenlist          1.4.0
fsspec              2023.6.0
future              0.18.3
gmpy2               2.1.2
h5py                3.9.0
hyperopt            0.2.7
idna                3.4
igraph              0.10.6
importlib_metadata  NA
importlib_resources NA
ipykernel           6.25.1
ipython_genutils    0.2.0
ipywidgets          8.1.0
isoduration         NA
jax                 0.4.14
jaxlib              0.4.14
jedi                0.19.0
jinja2              3.1.2
joblib              1.3.2
json5               NA
jsonpointer         2.0
jsonschema          4.17.3
jupyter_events      0.6.3
jupyter_server      2.7.1
jupyterlab_server   2.24.0
keopscore           2.1.2
kiwisolver          1.4.4
leidenalg           0.10.1
lightning           2.0.7
lightning_cloud     NA
lightning_fabric    2.0.7
lightning_utilities 0.9.0
llvmlite            0.40.1
markupsafe          2.1.3
matplotlib          3.7.2
ml_collections      NA
ml_dtypes           0.2.0
mpl_toolkits        NA
mpmath              1.3.0
msgpack             1.0.5
mudata              0.2.3
multidict           6.0.4
multipart           0.0.6
multipledispatch    0.6.0
natsort             8.4.0
nbformat            5.9.2
networkx            3.1
numba               0.57.1
numpyro             0.13.0
nvfuser             NA
opt_einsum          v3.3.0
optax               0.1.7
ordered_set         4.1.0
orjson              3.9.5
overrides           NA
packaging           23.1
parso               0.8.3
past                0.18.3
patsy               0.5.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.10.0
prometheus_client   NA
prompt_toolkit      3.0.39
psutil              5.9.5
ptyprocess          0.7.0
pure_eval           0.2.2
pvectorc            NA
py4j                0.10.9.7
pydantic            2.1.1
pydantic_core       2.4.0
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.16.1
pykeops             2.1.2
pyparsing           3.0.9
pyro                1.8.6+4be5c2e
pyrsistent          NA
pythonjsonlogger    NA
pytorch_lightning   2.0.7
pytz                2023.3
requests            2.31.0
rfc3339_validator   0.1.4
rfc3986_validator   0.1.1
rich                NA
scipy               1.11.2
seaborn             0.12.2
send2trash          NA
setuptools          68.1.2
six                 1.16.0
sklearn             1.3.0
sniffio             1.3.0
socks               1.7.1
soupsieve           2.3.2.post1
sparse              0.14.0
stack_data          0.6.2
starlette           0.27.0
statsmodels         0.14.0
sympy               1.12
texttable           1.6.7
threadpoolctl       3.2.0
tomli               2.0.1
toolz               0.12.0
torchaudio          2.0.0
torchmetrics        1.0.3
torchvision         0.15.0
tornado             6.3.3
traitlets           5.9.0
tree                0.1.8
typing_extensions   NA
uri_template        NA
urllib3             2.0.4
uvicorn             0.23.2
wcwidth             0.2.6
webcolors           1.13
websocket           1.6.2
websockets          11.0.3
xarray              2023.8.0
yaml                6.0.1
yarl                1.9.2
zipp                NA
zmq                 24.0.1
zoneinfo            NA
-----
IPython             8.14.0
jupyter_client      7.4.9
jupyter_core        5.3.1
jupyterlab          3.6.5
notebook            6.5.5
-----
Python 3.9.17 | packaged by conda-forge | (main, Aug 10 2023, 07:02:31) [GCC 12.3.0]
Linux-3.10.0-1160.90.1.el7.x86_64-x86_64-with-glibc2.17

sobolev_alignment_clf.fit - AttributeError: module 'scvi' has no attribute '__version__'

Report

Dear,

I am following the sobolev tutorial with standard settings and am encountering an issue with sobolev_alignment_clf.fit where I get the error message that "AttributeError: module 'scvi' has no attribute 'version'."
Reinstalling or updating scvi-tools (as suggested online for similar problems, eg scverse/scvi-tools#824) does not resolve the problem.
The output of import scvi; print(scvi.version) is 1.0.3
pip list shows me scvi-tools 1.0.3
Do you have an idea about how I can solve this?
Thanks for your time!

Julie

Version information

---

anndata 0.9.2
numpy 1.24.4
scanpy 1.9.4
scvi 1.0.3
session_info 1.0.0
sobolev_alignment 0.4.1

Cython 3.0.2
PIL 10.0.0
absl NA
aiohttp 3.8.5
aiosignal 1.3.1
annotated_types 0.5.0
anyio NA
asttokens NA
async_timeout 4.0.3
attr 23.1.0
babel 2.12.1
backcall 0.2.0
backoff 2.2.1
brotli 1.0.9
bs4 4.12.2
certifi 2023.07.22
cffi 1.15.1
charset_normalizer 3.2.0
chex 0.1.7
click 8.1.7
cloudpickle 2.2.1
comm 0.1.4
contextlib2 NA
croniter NA
cycler 0.10.0
cython 3.0.2
cython_runtime NA
dateutil 2.8.2
debugpy 1.6.7
decorator 5.1.1
deepdiff 6.4.1
defusedxml 0.7.1
docrep 0.3.2
etils 1.4.1
exceptiongroup 1.1.3
executing 1.2.0
fastapi 0.103.1
fastjsonschema NA
flax 0.7.2
frozenlist 1.4.0
fsspec 2023.9.0
future 0.18.3
h5py 3.9.0
hyperopt 0.2.7
idna 3.4
importlib_metadata NA
importlib_resources NA
ipykernel 6.25.1
jax 0.4.14
jaxlib 0.4.14
jedi 0.19.0
jinja2 3.1.2
joblib 1.3.2
json5 NA
jsonschema 4.17.3
jupyter_events 0.6.3
jupyter_server 2.7.3
jupyterlab_server 2.24.0
kiwisolver 1.4.5
lightning 2.0.8
lightning_cloud NA
lightning_fabric 2.0.8
lightning_utilities 0.9.0
llvmlite 0.40.1
markupsafe 2.1.1
matplotlib 3.7.2
ml_collections NA
ml_dtypes 0.2.0
mpl_toolkits NA
mpmath 1.3.0
msgpack 1.0.5
mudata 0.2.3
multidict 6.0.4
multipart 0.0.6
multipledispatch 0.6.0
natsort 8.4.0
nbformat 5.9.2
networkx 3.1
numba 0.57.1
numpyro 0.13.0
nvfuser NA
opt_einsum v3.3.0
optax 0.1.7
ordered_set 4.1.0
overrides NA
packaging 23.1
pandas 2.1.0
parso 0.8.3
past 0.18.3
patsy 0.5.3
pexpect 4.8.0
pickleshare 0.7.5
pkg_resources NA
platformdirs 3.10.0
prometheus_client NA
prompt_toolkit 3.0.39
psutil 5.9.0
ptyprocess 0.7.0
pure_eval 0.2.2
pvectorc NA
py4j 0.10.9.7
pycparser 2.21
pydantic 2.1.1
pydantic_core 2.4.0
pydev_ipython NA
pydevconsole NA
pydevd 2.9.5
pydevd_file_utils NA
pydevd_plugins NA
pydevd_tracing NA
pygments 2.16.1
pyparsing 3.0.9
pyro 1.8.6
pyrsistent NA
pythonjsonlogger NA
pytorch_lightning 2.0.8
pytz 2023.3
requests 2.31.0
rfc3339_validator 0.1.4
rfc3986_validator 0.1.1
rich NA
scipy 1.11.2
seaborn 0.12.2
send2trash NA
setuptools 68.1.2
six 1.16.0
sklearn 1.3.0
skmisc 0.3.0
sniffio 1.3.0
socks 1.7.1
soupsieve 2.5
sparse 0.14.0
stack_data 0.6.2
starlette 0.27.0
statsmodels 0.14.0
sympy 1.12
threadpoolctl 3.2.0
tomli 2.0.1
toolz 0.12.0
torch 2.0.1+cu117
torchmetrics 1.1.1
tornado 6.3.2
tqdm 4.66.1
traitlets 5.9.0
tree 0.1.8
typing_extensions NA
urllib3 2.0.4
uvicorn 0.23.2
wcwidth 0.2.6
websocket 1.6.2
websockets 11.0.3
xarray 2023.8.0
yaml 6.0
yarl 1.9.2
zipp NA
zmq 25.1.0
zoneinfo NA

IPython 8.15.0
jupyter_client 8.3.1
jupyter_core 5.3.0
jupyterlab 4.0.5
notebook 7.0.3

Python 3.9.18 | packaged by conda-forge | (main, Aug 30 2023, 03:49:32) [GCC 12.3.0]
Linux-4.18.0-477.21.1.el8_8.x86_64-x86_64-with-glibc2.28

Session information updated at 2023-09-05 16:25