muammar/ml4chem

ML4Chem: Machine Learning for Chemistry and Materials

---

About

ML4Chem is a package to deploy machine learning for chemistry and materials science. It is written in Python 3, and intends to offer modern and rich features to perform machine learning (ML) workflows for chemical physics.

A list of features and ML algorithms are shown below.

• PyTorch backend.
• Completely modular. You can use any part of this package in your project.
• Free software <3. No secrets! Pull requests and additions are more than welcome!
• Documentation (work in progress).
• Explicit and idiomatic: ml4chem.get_me_a_coffee().
• Distributed training in a data parallel paradigm aka mini-batches.
• Scalability and distributed computations are powered by Dask.
• Real-time tools to track status of your computations.
• Easy scaling up/down.
• Easy access to intermediate quantities: NeuralNetwork.get_activations(X, numpy=True) or VAE.get_latent_space(X).
• Messagepack serialization.

Notes

This package is under heavy development and might break at some points until it gets stabilized. It is in its infancy, so if you find there is an error, you might want to report it so that it can be improved. We also welcome pull requests if you find any part of ML4Chem should be improved. That would be very nice.

Citing

If you find this software useful, please use this bibtex to cite it:

@article{El_Khatib2020,
author = "Muammar El Khatib and Wibe de Jong",
title = "{ML4Chem: A Machine Learning Package for Chemistry and Materials Science}",
year = "2020",
month = "3",
url = "https://chemrxiv.org/articles/ML4Chem_A_Machine_Learning_Package_for_Chemistry_and_Materials_Science/11952516",
doi = "10.26434/chemrxiv.11952516.v1"
}

Documentation

To get started, read the documentation at https://ml4chem.dev. It is arranged in a way that you can go through the theory as well as some code snippets to understand how to use this software. Additionally, you can dive through the module index to get more information about different classes and functions of ML4Chem. If you think the documentation has to be improved do not hesistate to state so in the bug reports and help out if you feel like it.

Visualizations

Copyright

License: BSD 3-clause "New" or "Revised" License.

ML4Chem: Machine Learning for Chemistry and Materials (ML4Chem) Copyright (c)
2019, The Regents of the University of California, through Lawrence Berkeley
National Laboratory (subject to receipt of any required approvals from the U.S.
Dept. of Energy).  All rights reserved.

If you have questions about your rights to use or distribute this software,
please contact Berkeley Lab's Intellectual Property Office at
[email protected].

NOTICE.  This Software was developed under funding from the U.S. Department
of Energy and the U.S. Government consequently retains certain rights.  As
such, the U.S. Government has been granted for itself and others acting on
its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the
Software to reproduce, distribute copies to the public, prepare derivative
works, and perform publicly and display publicly, and to permit other to do
so.

ml4chem's Issues

Transfer Learning

1. Load in pre-trained weights from a network trained on a large dataset
2. Freeze all the weights in the lower (convolutional) layers: the layers to freeze are adjusted depending on similarity of new task to original dataset
3. Replace the upper layers of the network with a custom classifier: the number of outputs must be set equal to the number of classes
4. Train only the custom classifier layers for the task thereby optimizing the model for smaller dataset.

Loading in a pre-trained model in PyTorch is simple:

from torchvision import models
model = model.vgg16(pretrained=True)

Source: https://towardsdatascience.com/transfer-learning-with-convolutional-neural-networks-in-pytorch-dd09190245ce

I think the pretrained keyword argument is something that has to do with the vgg16 model but should not be complicated to implement.

Methods to save() and load() models

The NNs are being trained correctly. Now, it is time to add methods to save() and load() models.

Make Pytorch train a Neural Network for mono and multi element training sets

Use the Gaussian fingerprints to train a pytorch neural network with energy target vector.

Interatomic potentials for ionic systems with density functional accuracy based on charge densities obtained by a neural network

Implement model shown in Ghasemi, S. A., Hofstetter, A., Saha, S. & Goedecker, S. Interatomic potentials for ionic systems with density functional accuracy based on charge densities obtained by a neural network. Phys. Rev. B 92, 045131 (2015).

Add a CJSON reader

This function will take as an argument the path of a cjson file and will return an ASE trajectory file that is going to be compatible with ml4chem.

For more information see https://github.com/OpenChemistry/chemicaljson

Create a custom loss function to train energies

The reason for this is because we have the energy of an image and within the atom-centered model we have to guess atomic energies that when summed up they return the target -- in this case the total energy. This same loss function can be then adjusted to train against forces if wanted.

Add Gaussian Process Model

Another one to the zoo.

Pure Python gaussian local chemical envirnoment

Make the code to create feature vectors correctly using pure python.

Add Kernel Ridge Regression

I already did this before. I have to add this model and modify the data class so that the code is not slow. This is related to issue #2.

• Build kernel matrix per atom.
• Train model with Cholesky.
• Compute just triangular kernel matrix to save computational time.
• Logging different parts of the tasks.
• Save/load parameters to file.
• Predictions.
• Clean up the code.
• Parallelization.
• Make it possible to do a calc.get_potential_energy(atoms) in memory.
• Support sigmaper atom.

Implement Weighted ACSFs

https://aip.scitation.org/doi/10.1063/1.5019667

Create some rule to initialize models

The biases are not correctly initialized. I should create some kind of trick to have them starting to values that make sense for the problem in question.

Add ModelMerger class

• Model merger.
• Loss function weighted sum vs independently.
• Printing of training evolution.
• Save model (I think code already available in ml4chem could be used for that).
• Set convergence stop mechanism.
• Parallelization.
• Add more compatibility with different available components.
• Check for consistency of the class (independent vs dependent loss).

That class would help to merge models like this:

Make the Data class universal

Data structure is quite important in machine-learning and in fact, it can represent a bottleneck as one has to iterate over the data points.

The Data class in mlchemistry, at least I hope, has to be done in a way that can interoperate with other backends and has to be designed to build data for more than neural networks (I support vector machines).