muammar/ml4chem

ML4Chem: Machine Learning for Chemistry and Materials

ml4chem's Issues

Transfer Learning

1. Load in pre-trained weights from a network trained on a large dataset
2. Freeze all the weights in the lower (convolutional) layers: the layers to freeze are adjusted depending on similarity of new task to original dataset
3. Replace the upper layers of the network with a custom classifier: the number of outputs must be set equal to the number of classes
4. Train only the custom classifier layers for the task thereby optimizing the model for smaller dataset.

Loading in a pre-trained model in PyTorch is simple:

from torchvision import models
model = model.vgg16(pretrained=True)

Source: https://towardsdatascience.com/transfer-learning-with-convolutional-neural-networks-in-pytorch-dd09190245ce

I think the pretrained keyword argument is something that has to do with the vgg16 model but should not be complicated to implement.

Make the Data class universal

Data structure is quite important in machine-learning and in fact, it can represent a bottleneck as one has to iterate over the data points.

The Data class in mlchemistry, at least I hope, has to be done in a way that can interoperate with other backends and has to be designed to build data for more than neural networks (I support vector machines).

Add a CJSON reader

This function will take as an argument the path of a cjson file and will return an ASE trajectory file that is going to be compatible with ml4chem.

For more information see https://github.com/OpenChemistry/chemicaljson

Pure Python gaussian local chemical envirnoment

Make the code to create feature vectors correctly using pure python.

Create a custom loss function to train energies

The reason for this is because we have the energy of an image and within the atom-centered model we have to guess atomic energies that when summed up they return the target -- in this case the total energy. This same loss function can be then adjusted to train against forces if wanted.

Add Gaussian Process Model

Another one to the zoo.

Implement Weighted ACSFs

https://aip.scitation.org/doi/10.1063/1.5019667

Methods to save() and load() models

The NNs are being trained correctly. Now, it is time to add methods to save() and load() models.

Create some rule to initialize models

The biases are not correctly initialized. I should create some kind of trick to have them starting to values that make sense for the problem in question.

Add Kernel Ridge Regression

I already did this before. I have to add this model and modify the data class so that the code is not slow. This is related to issue #2.

• Build kernel matrix per atom.
• Train model with Cholesky.
• Compute just triangular kernel matrix to save computational time.
• Logging different parts of the tasks.
• Save/load parameters to file.
• Predictions.
• Clean up the code.
• Parallelization.
• Make it possible to do a calc.get_potential_energy(atoms) in memory.
• Support sigmaper atom.

Add ModelMerger class

• Model merger.
• Loss function weighted sum vs independently.
• Printing of training evolution.
• Save model (I think code already available in ml4chem could be used for that).
• Set convergence stop mechanism.
• Parallelization.
• Add more compatibility with different available components.
• Check for consistency of the class (independent vs dependent loss).

That class would help to merge models like this:

Interatomic potentials for ionic systems with density functional accuracy based on charge densities obtained by a neural network

Implement model shown in Ghasemi, S. A., Hofstetter, A., Saha, S. & Goedecker, S. Interatomic potentials for ionic systems with density functional accuracy based on charge densities obtained by a neural network. Phys. Rev. B 92, 045131 (2015).

Make Pytorch train a Neural Network for mono and multi element training sets

Use the Gaussian fingerprints to train a pytorch neural network with energy target vector.