deepchem/deeplearninglifesciences

Example code from the book "Deep Learning for the Life Sciences"

DeepChem

Website | Documentation | Colab Tutorial | Discussion Forum | Discord | Model Wishlist | Tutorial Wishlist

DeepChem aims to provide a high quality open-source toolchain that democratizes the use of deep-learning in drug discovery, materials science, quantum chemistry, and biology.

Table of contents:

• Requirements
• Installation
  • Stable version
  • Nightly build version
  • Docker
  • From source
  • From source lightweight
• Getting Started
  • Discord
• About Us
• Contributing to DeepChem
• Citing DeepChem

Requirements

DeepChem currently supports Python 3.7 through 3.10 and requires these packages on any condition.

• joblib
• NumPy
• pandas
• scikit-learn
• SciPy
• rdkit

Soft Requirements

DeepChem has a number of "soft" requirements. If you face some errors like ImportError: This class requires XXXX, you may need to install some packages.

Please check the document about soft requirements.

Installation

Stable version

DeepChem stable version can be installed using pip or conda as

pip install deepchem

or

conda install -c conda-forge deepchem

Deepchem provides support for tensorflow, pytorch, jax and each require a individual pip Installation.

For using models with tensorflow dependencies, you install using

pip install deepchem[tensorflow]

For using models with torch dependencies, you install using

pip install deepchem[torch]

For using models with jax dependencies, you install using

pip install deepchem[jax]

If GPU support is required, then make sure CUDA is installed and then install the desired deep learning framework using the links below before installing deepchem

1. tensorflow - just cuda installed
2. pytorch - https://pytorch.org/get-started/locally/#start-locally
3. jax - https://github.com/google/jax#pip-installation-gpu-cuda

In zsh square brackets are used for globbing/pattern matching. This means you need to escape the square brackets in the above installation. You can do so by including the dependencies in quotes like pip install --pre 'deepchem[jax]'

Nightly build version

The nightly version is built by the HEAD of DeepChem. It can be installed using

pip install --pre deepchem

Docker

If you want to install deepchem using a docker, you can pull two kinds of images.
DockerHub : https://hub.docker.com/repository/docker/deepchemio/deepchem

• deepchemio/deepchem:x.x.x
  • Image built by using a conda (x.x.x is a version of deepchem)
  • The x.x.x image is built when we push x.x.x. tag
  • Dockerfile is put in docker/tag directory
• deepchemio/deepchem:latest
  • Image built from source codes
  • The latest image is built every time we commit to the master branch
  • Dockerfile is put in docker/nightly directory

You pull the image like this.

docker pull deepchemio/deepchem:2.4.0

If you want to know docker usages with deepchem in more detail, please check the document.

From source

If you try install all soft dependencies at once or contribute to deepchem, we recommend you should install deepchem from source.

Please check this introduction.

Getting Started

The DeepChem project maintains an extensive collection of tutorials. All tutorials are designed to be run on Google colab (or locally if you prefer). Tutorials are arranged in a suggested learning sequence which will take you from beginner to proficient at molecular machine learning and computational biology more broadly.

After working through the tutorials, you can also go through other examples. To apply deepchem to a new problem, try starting from one of the existing examples or tutorials and modifying it step by step to work with your new use-case. If you have questions or comments you can raise them on our gitter.

Supported Integrations

• Weights & Biases: Track your DeepChem model's training and evaluation metrics.

Discord

The DeepChem Discord hosts a number of scientists, developers, and enthusiasts interested in deep learning for the life sciences. Probably the easiest place to ask simple questions or float requests for new features.

About Us

DeepChem is managed by a team of open source contributors. Anyone is free to join and contribute!

Citing DeepChem

If you have used DeepChem in the course of your research, we ask that you cite the "Deep Learning for the Life Sciences" book by the DeepChem core team.

To cite this book, please use this bibtex entry:

@book{Ramsundar-et-al-2019,
    title={Deep Learning for the Life Sciences},
    author={Bharath Ramsundar and Peter Eastman and Patrick Walters and Vijay Pande and Karl Leswing and Zhenqin Wu},
    publisher={O'Reilly Media},
    note={\url{https://www.amazon.com/Deep-Learning-Life-Sciences-Microscopy/dp/1492039837}},
    year={2019}
}

deeplearninglifesciences's Issues

Unable to run chapter05 (and probably others) as unable to download pdbbind_v2015.tar.gz

Running deepchem 2.3.0 in a conda environment on ubuntu.

From looking at the error by hand, it seems the url http://deepchem.io.s3-website-us-west-1.amazonaws.com/datasets/pdbbind_v2015.tar.gz is 403ing.

Stack Trace

$ python3 pdbbind_nn.py
Traceback (most recent call last):
  File "pdbbind_nn.py", line 4, in <module>
    pdbbind_tasks, pdbbind_datasets, transformers = dc.molnet.load_pdbbind(featurizer="grid", split="random", subset="core")
  File "/home/ella/anaconda3/envs/deepchem/lib/python3.7/site-packages/deepchem/molnet/load_function/pdbbind_datasets.py", line 230, in load_pdbbind
    dest_dir=data_dir)
  File "/home/ella/anaconda3/envs/deepchem/lib/python3.7/site-packages/deepchem/utils/__init__.py", line 83, in download_url
    urlretrieve(url, os.path.join(dest_dir, name))
  File "/home/ella/anaconda3/envs/deepchem/lib/python3.7/urllib/request.py", line 247, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
  File "/home/ella/anaconda3/envs/deepchem/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/home/ella/anaconda3/envs/deepchem/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/home/ella/anaconda3/envs/deepchem/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/home/ella/anaconda3/envs/deepchem/lib/python3.7/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/home/ella/anaconda3/envs/deepchem/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/home/ella/anaconda3/envs/deepchem/lib/python3.7/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

Chapter 8 - data.py

Hi there,
I was trying to run the code but it does not run, in line 41 you are looking for

image_names = [p for p in os.listdir(images_path)if p.startswith('cut_') and p.endswith('.png')]

But there are no png images in the rep that I downloaded from Kaggle, all the images are in jpeg format.
and in the list you build in :

for im in image_names:
if im.endswith('.jpeg') and not im.startswith('cut_') and not 'cut_' + im in image_names:
raw_images.append(im)

Does not get used at all the raw_images ....

I am trying to understand why you are looking for 'cut_' there is no image that starts or ends with 'cuts_'

Can you please help me get a working version.

Thanks.
Oscar.

Chapter 11, model.save() returned NotImplementedError

This notebook worked well except the last line:

https://github.com/deepchem/DeepLearningLifeSciences/blob/master/Chapter11/chapter_11_02_erk2_graph_conv.ipynb

model.save() returned as below. Any suggestion on this?

---

NotImplementedError Traceback (most recent call last)
in ()
----> 1 model.save()

~/.conda/envs/deepchem/lib/python3.5/site-packages/deepchem/models/models.py in save(self)
107 Each subclass is responsible for overriding this method.
108 """
--> 109 raise NotImplementedError
110
111 def fit(self, dataset, nb_epoch=10, batch_size=50, **kwargs):

NotImplementedError:

Chapter 5 scripts: no load_pdbbind() attribute in version 2.1.0 of deepchem with python2.7

The two sample scripts for chapter 5 are not working with deepchem version 2.1.0 running with python2.7 on a Mac Mojave.

Traceback (most recent call last):
File "./pdbbind_nn.py", line 5, in
pdbbind_tasks, pdbbind_datasets, transformers = dc.molnet.load_pdbbind(featurizer="grid", split="random", subset="core")
AttributeError: 'module' object has no attribute 'load_pdbbind'

I get the same error with the other file.
There is an attribute load_pdbbind_grid().

pdbbind_tasks, pdbbind_datasets, transformers = dc.molnet.load_pdbbind_grid(split='random', featurizer='grid', subset='core', reload=True)
Loading dataset from disk.
TIMING: dataset construction took 0.031 s
Loading dataset from disk.
TIMING: dataset construction took 0.191 s
Loading dataset from disk.
TIMING: dataset construction took 0.022 s
Loading dataset from disk.
Traceback (most recent call last):
File "", line 1, in
TypeError: 'NoneType' object is not iterable

I am stuck.

problems with installation.....

It has been a real challenge to get this to work, besides the bugs... and still trying.... I write these lines to other people that are trying to get it to work

1.- conda Linux 4.7 has problems and cant install many packages, you have to downgrade to conda 4.6 ( In my case it was deepchem-2.2.0-py37_0.tar.bz2 )

2.- first you need to install conda 4.7 then run this command to downgrade.
conda install conda-4.6.14-py37_0.tar.bz2

3.- then you need to isntall these packages in order.
conda install -c rdkit rdkit
conda install -c conda-forge opencv
conda install -c conda-forge joblib

4.- in my case I isntalled keras-gpu which installs keras and tensorflow-gpu
conda install -c anaconda keras-gpu

5.- then none of the link in https://anaconda.org/deepchem/ works
conda install -c deepchem deepchem-gpu
Collecting package metadata: done
Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:
------------------------------------
conda install -c deepchem deepchem
Collecting package metadata: done
Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:

6.- does not work either....
conda install -c deepchem -c rdkit -c conda-forge -c omnia deepchem=2.1.0

7.- conda install -c deepchem -c rdkit -c conda-forge -c omnia deepchem=2.2.0 doesn't work for me either !!!!!! this is a real pain in the .....

8.- The only way I got it to work is downaloading this file conda-4.6.14-py37_0.tar.bz2 from https://anaconda.org/deepchem/deepchem/files with this command.
conda install deepchem-2.2.0-py37_0.tar.

the only problem with this approach is that once you do this you break conda environment and cant install many other things.....

9.- I will try to install from source and see.... before I break the conda environment again. I had to install conda like 6 times..... did not worked either....

can't find 2D3U pdb file.

can't find 2D3U pdb file.

Preparing a Dataset for Model Prediction

Hello, I'm working on the predictive model in Chapter 11, but could not find the rd_filters.py and associated data files here (suggested by the book). Does anyone have idea where I can find them? Thank you!

opencv need to fix IndexError at Chapter08 code

num of images to be processed: 824
OpenCV required for image preprocessing
Number of images: 0
Traceback (most recent call last):
  File "/home/fkt/Downloads/deepchem/notebook/DeepLearningLifeSciences-master/Chapter08/run.py", line 15, in <module>
    train, valid, test = load_images_DR(split='random', seed=123)
  File "/home/fkt/Downloads/deepchem/notebook/DeepLearningLifeSciences-master/Chapter08/data.py", line 54, in load_images_DR
    dat = deepchem.data.ImageDataset(
  File "/home/fkt/mambaforge/envs/deepchem/lib/python3.9/site-packages/deepchem/data/datasets.py", line 2713, in __init__
    self._X_shape = self._find_array_shape(X)
  File "/home/fkt/mambaforge/envs/deepchem/lib/python3.9/site-packages/deepchem/data/datasets.py", line 2740, in _find_array_shape
    image_shape = load_image_files([array[0]]).shape[1:]
IndexError: list index out of range

after installing opencv,

Num of images to be processed: 824
on image 0
on image 100
(skip)

error of Classification metric can't handle a mix of binary and continous metric mean-matthews_corrcoef

Hi,
While using the dude_erk2_mk01.csv as input on jupyer notebook code for (chapter_11_02_erk2_graph_conv.ipynb), I am experiencing the following error..

/home/surendra/anaconda3/lib/python3.6/site-packages/deepchem/metrics/init.py:312: UserWarning: Error calculating metric mean-matthews_corrcoef: Classification metrics can't handle a mix of binary and continuous-multioutput targets
warnings.warn("Error calculating metric %s: %s" % (self.name, e))
/home/surendra/anaconda3/lib/python3.6/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/home/surendra/anaconda3/lib/python3.6/site-packages/numpy/core/_methods.py:85: RuntimeWarning: invalid value encountered in double_scalars
ret = ret.dtype.type(ret / rcount)

the output i am getting is

computed_metrics: [nan]
computed_metrics: [nan]
[nan]
[nan]
computed_metrics: [nan]
computed_metrics: [nan]
[nan, nan]
[nan, nan]
computed_metrics: [nan]
computed_metrics: [nan]
[nan, nan, nan]
[nan, nan, nan]
computed_metrics: [nan]
computed_metrics: [nan]
[nan, nan, nan, nan]
[nan, nan, nan, nan]
computed_metrics: [nan]
computed_metrics: [nan]
[nan, nan, nan, nan, nan]
[nan, nan, nan, nan, nan]
computed_metrics: [nan]
computed_metrics: [nan]
[nan, nan, nan, nan, nan, nan]
[nan, nan, nan, nan, nan, nan]
computed_metrics: [nan]
computed_metrics: [nan]
[nan, nan, nan, nan, nan, nan, nan]
[nan, nan, nan, nan, nan, nan, nan]
computed_metrics: [nan]
computed_metrics: [nan]
[nan, nan, nan, nan, nan, nan, nan, nan]
[nan, nan, nan, nan, nan, nan, nan, nan]
computed_metrics: [nan]
computed_metrics: [nan]
[nan, nan, nan, nan, nan, nan, nan, nan, nan]
[nan, nan, nan, nan, nan, nan, nan, nan, nan]
computed_metrics: [nan]
computed_metrics: [nan]
[nan, nan, nan, nan, nan, nan, nan, nan, nan, nan]
[nan, nan, nan, nan, nan, nan, nan, nan, nan, nan]

Can you please help me out, why this kind of problem is happening.

Thanks

Surendra

Unable to load and featurize the pdbbind data (Chapter05/pdbbind_rf.py)

Hi,

I have been trying to run the revised code from the tutorial (Chapter05/pdbbind_rf.py). However, I could not load and featurize the pdbbind data from Molnet for some reason.

Please, refer to the line of code (from the tutorial) and the subsequent error message shown in the snapshot.

tasks, datasets,transformers = dc.molnet.load_pdbbind(featurizer=grid, splitter="random", subset="core")

Thank you in advance.

Woo-Jae

Chapter 3 has not been uploaded

Can you check this code and add to the repo if valid? I have added the Jupyter notebook for code in Chapter 3: Machine Learning with Deep Chem.

[https://colab.research.google.com/drive/15U1awatK0a1VbkWhElgJyBtX4yTc8R-N?usp=sharing]

Chapter 7- issue in loading pre trained model

While following the textbook and the steps on Github on the cell counting task, when I run

model.restore()

it throws the following error:

INFO:tensorflow:Restoring parameters from model/model-7700
and
NotFoundError: Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:

2 root error(s) found.
(0) Not found: Key Conv2D_3/conv2d_4/bias not found in checkpoint
[[node save_3/RestoreV2 (defined at /tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py:1748) ]]
(1) Not found: Key Conv2D_3/conv2d_4/bias not found in checkpoint
[[node save_3/RestoreV2 (defined at /tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py:1748) ]]
[[save_3/RestoreV2/_101]]
0 successful operations.
0 derived errors ignored.

problem in data_loader.py

in the static function load_img() you are using SciPy.misc.imread() which is deprecated in SciPy 1.0.0, and removed in 1.2.0... and the lattes version is 1.2.2
in order for the code to work, I had to do this.

@staticmethod
def load_img(image_files):

images = []

for image_file in image_files:
  _, extension = os.path.splitext(image_file)
  extension = extension.lower()
  
  if extension == ".png":
    im = Image.open(image_file)
    imarray = np.array(im)
    images.append(imarray)
    #image = misc.imread(image_file)
    #images.append(image)
  elif extension == ".tif":
    im = Image.open(image_file)
    imarray = np.array(im)
    images.append(imarray)
  else:
    raise ValueError("Unsupported image filetype for %s" % image_file)
return np.array(images)

Besides I don't understand why you are using to read images functions from to different libraries.....

Chapter7 issue

this model can be used to make predictions out of the box. There are directions on downloading the pretrained model in the code repository associated with the book (https://github.com/deepchem/DeepLearningLifeSciences).

there is no pretrained model in code repository

Chapter 11, leftover active decoys

Did you forget to drop the leftover active decoy molecules or did you left them as is deliberately?

After this ratio check, there is no further dropping for the 12 leftover active decoys:

charged = revised_decoy_df[revised_decoy_df["charge"] != 0]
charged.shape[0]/revised_decoy_df.shape[0] # 0.00263

charged.shape[0] # 12

Updating Book Code to DeepChem 2.4.0

We need to work to update the book code to DeepChem 2.4.0. I've gotten a report on gitter than the chapter 6 code isn't working. What else do we need to update?

CC @peastman

Error in solubility.py in Chapter 4

I just copied and paste the code provided but after executing "model.fit(train_dataset, nb_epoch=100)", the error was shown to be below.
Please fix it and I really appreciate it.

import deepchem as dc
tasks, datasets, transformers = dc.molnet.load_delaney(featurizer='GraphConv')
train_dataset, valid_dataset, test_dataset = datasets

model = dc.models.GraphConvModel(n_tasks=1, mode='regression', dropout=0.2)
model.fit(train_dataset, nb_epoch=100)

TypeError Traceback (most recent call last)
in
----> 1 model.fit(train_dataset, nb_epoch=100)

~/opt/anaconda3/envs/home-dd/lib/python3.7/site-packages/deepchem/models/keras_model.py in fit(self, dataset, nb_epoch, max_checkpoints_to_keep, checkpoint_interval, deterministic, restore, variables, loss, callbacks, all_losses)
328 dataset, epochs=nb_epoch,
329 deterministic=deterministic), max_checkpoints_to_keep,
--> 330 checkpoint_interval, restore, variables, loss, callbacks, all_losses)
331
332 def fit_generator(self,

~/opt/anaconda3/envs/home-dd/lib/python3.7/site-packages/deepchem/models/keras_model.py in fit_generator(self, generator, max_checkpoints_to_keep, checkpoint_interval, restore, variables, loss, callbacks, all_losses)
414 inputs = inputs[0]
415
--> 416 batch_loss = apply_gradient_for_batch(inputs, labels, weights, loss)
417 current_step = self._global_step.numpy()
418

~/opt/anaconda3/envs/home-dd/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py in call(self, *args, **kwds)
455
456 tracing_count = self._get_tracing_count()
--> 457 result = self._call(*args, **kwds)
458 if tracing_count == self._get_tracing_count():
459 self._call_counter.called_without_tracing()

~/opt/anaconda3/envs/home-dd/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py in _call(self, *args, **kwds)
501 # This is the first call of call, so we have to initialize.
502 initializer_map = object_identity.ObjectIdentityDictionary()
--> 503 self._initialize(args, kwds, add_initializers_to=initializer_map)
504 finally:
505 # At this point we know that the initialization is complete (or less

~/opt/anaconda3/envs/home-dd/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py in _initialize(self, args, kwds, add_initializers_to)
406 self._concrete_stateful_fn = (
407 self._stateful_fn._get_concrete_function_internal_garbage_collected( # pylint: disable=protected-access
--> 408 *args, **kwds))
409
410 def invalid_creator_scope(*unused_args, **unused_kwds):

~/opt/anaconda3/envs/home-dd/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py in _get_concrete_function_internal_garbage_collected(self, *args, **kwargs)
1846 if self.input_signature:
1847 args, kwargs = None, None
-> 1848 graph_function, _, _ = self._maybe_define_function(args, kwargs)
1849 return graph_function
1850

~/opt/anaconda3/envs/home-dd/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py in _maybe_define_function(self, args, kwargs)
2148 graph_function = self._function_cache.primary.get(cache_key, None)
2149 if graph_function is None:
-> 2150 graph_function = self._create_graph_function(args, kwargs)
2151 self._function_cache.primary[cache_key] = graph_function
2152 return graph_function, args, kwargs

~/opt/anaconda3/envs/home-dd/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py in _create_graph_function(self, args, kwargs, override_flat_arg_shapes)
2039 arg_names=arg_names,
2040 override_flat_arg_shapes=override_flat_arg_shapes,
-> 2041 capture_by_value=self._capture_by_value),
2042 self._function_attributes,
2043 # Tell the ConcreteFunction to clean up its graph once it goes out of

~/opt/anaconda3/envs/home-dd/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py in func_graph_from_py_func(name, python_func, args, kwargs, signature, func_graph, autograph, autograph_options, add_control_dependencies, arg_names, op_return_value, collections, capture_by_value, override_flat_arg_shapes)
913 converted_func)
914
--> 915 func_outputs = python_func(*func_args, **func_kwargs)
916
917 # invariant: func_outputs contains only Tensors, CompositeTensors,

~/opt/anaconda3/envs/home-dd/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py in wrapped_fn(*args, **kwds)
356 # wrapped allows AutoGraph to swap in a converted function. We give
357 # the function a weak reference to itself to avoid a reference cycle.
--> 358 return weak_wrapped_fn().wrapped(*args, **kwds)
359 weak_wrapped_fn = weakref.ref(wrapped_fn)
360

~/opt/anaconda3/envs/home-dd/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py in wrapper(*args, **kwargs)
903 except Exception as e: # pylint:disable=broad-except
904 if hasattr(e, "ag_error_metadata"):
--> 905 raise e.ag_error_metadata.to_exception(e)
906 else:
907 raise

TypeError: in converted code:
relative to /Users/hankilee/opt/anaconda3/envs/home-dd/lib/python3.7/site-packages:

deepchem/models/keras_model.py:482 apply_gradient_for_batch  *
    grads = tape.gradient(batch_loss, vars)
tensorflow_core/python/eager/backprop.py:1014 gradient
    unconnected_gradients=unconnected_gradients)
tensorflow_core/python/eager/imperative_grad.py:76 imperative_grad
    compat.as_str(unconnected_gradients.value))
tensorflow_core/python/eager/backprop.py:138 _gradient_function
    return grad_fn(mock_op, *out_grads)
tensorflow_core/python/ops/math_grad.py:455 _UnsortedSegmentMaxGrad
    return _UnsortedSegmentMinOrMaxGrad(op, grad)
tensorflow_core/python/ops/math_grad.py:432 _UnsortedSegmentMinOrMaxGrad
    _GatherDropNegatives(op.outputs[0], op.inputs[1])

TypeError: 'NoneType' object is not subscriptable

Chapter 5 Error pdbbind_rf

I am getting the following error from pdbbind_rf:

ValueError: Found array with 0 feature(s) (shape=(3881, 0)) while a minimum of 1 is required.
the first of the traceback is from
model.fit(train_dataset)

deepchem version = 2.4.0
tensorflow version = 2.3.0

pdbbind_nn.py no longer working.

https://github.com/deepchem/DeepLearningLifeSciences/blob/master/Chapter05/pdbbind_nn.py throws this error

RuntimeError: expected scalar type Float but found Char