elttaes/revisiting-plms

Exploring Evolution-aware & free protein language models as protein function predictors

Exploring Evolution-aware & free protein language models as protein function predictors

Env:

Jax(Alphafold2):

https://github.com/kalininalab/alphafold_non_docker

Pytorch(ESM-1b,MSA-Transformer):

1. As a prerequisite, you must have PyTorch installed(https://pytorch.org/get-started/locally/).

2. pip install fair-esm # latest release, OR:

   pip install git+https://github.com/facebookresearch/esm.git

data:

For SSP & Contact map:

ESMStructuralSplitDataset:

Name	Description	URL
splits	train/valid splits	https://dl.fbaipublicfiles.com/fair-esm/structural-data/splits.tar.gz
pkl	pkl objects containing sequence, SSP labels, distance map, and 3d coordinates	https://dl.fbaipublicfiles.com/fair-esm/structural-data/pkl.tar.gz
msas	a3m files containing MSA for each domain	https://dl.fbaipublicfiles.com/fair-esm/structural-data/msas.tar.gz

from https://github.com/facebookresearch/esm

For Contact map Test:

CAMEO(https://www.cameo3d.org/)

For Fitness dataset:

Tape(https://github.com/songlab-cal/tape)

For pretrain ESM-1b & MSA-Transformer

Alphafold2 training data:

https://registry.opendata.aws/openfold/

from Openfold(https://github.com/aqlaboratory/openfold)

@article{hu2022exploring,
  title={Exploring evolution-aware \&-free protein language models as protein function predictors},
  author={Hu, Mingyang and Yuan, Fajie and Yang, Kevin K and Ju, Fusong and Su, Jin and Wang, Hui and Yang, Fei and Ding, Qiuyang},
  journal={arXiv preprint arXiv:2206.06583},
  year={2022}
}

revisiting-plms's Issues

About datasets with AlphaFold on contact map prediction

Hello, quite great work you do. Recently I wanted to try your code on contact map prediction. I saw the link of ESMStructuralSplitDataset, but didn't find AlphaFold data. Can you release this data? Hope to get your reply. Thanks so much.

Does the padding affect the training of SSP?

Hi. Thanks for your interesting work!

I noticed that you used the pad token to align sequences of different lengths and MSAs. I'm curious if the resulting representations with the padding token are consistent with the representations without the padding token. Additionally, I would like to know if applying a convolution network on the representations with the padding token would affect training and prediction. Looking forward to your response!

Remote homology?

Hi, thanks for sharing the code! I saw a remote homology folder in your repo. Since I didn't notice you reported, what is that for?

Metal ion binding dataset

Hi there,

Nice work!
I have a question about the metal ion binding dataset used in your paper.
Could you let me know where do you get the original dataset?

Thanks!

AF2 structure/embedding for Stability (Fitness) task is not available.

Hello,
Interesting work. Could you provide the AF2 embedding for the 60,000+ sequences in the stability task? (the dataset from TAPE's github only contains the sequences)
It would be even better if the corresponding AF2 structures for those sequences are also available.
generating those again will be a waste of resources and error prone if we are using different database/setting.

Thanks.

seq_length less than 256

Dear authors,
Thank you for sharing your work.
I have one question is that why in your dataset preparation codes, you limit the length of the protein to 256 amino acid? Doesn't it impact the performance of the training with different length datasets?

https://github.com/elttaes/Revisiting-PLMs/blob/78919610a6bf9908ed1eb841bba3d4210840099c/Fitness/flu/esm/dataset/datasets_downstream_tape.py#L217-L224

Thank you.
Ai