Low validity on moses3 dataset

[devalab]

Low validity on moses3 dataset

[charlesxu90]

BTW, why do you use LSTM for scaffold embedding, rather than a linear embedding with position embedding?

For me, the usage of LSTM here feels very strange. If you can use LSTM for scaffold, why not use for all the inputs?

[RishalAggarwal]

I'm not sure we can point out the problem straight up like this, have you tried using the pretrained weights for generation? LSTM was used to represent the scaffold with a single vector, more for convenience than anything. To take advantage of the transformer architecture we need to use linear and positional embeddings for the main molecule.