For the Flax/JAX Community Week, I worked on the T5-VAE project that combines a T5 transformer model with a variational autoencoder to learn smooth latent spaces for texts.
- Model card for the T5-VAE trained on Wikipedia sentences.
- Model card for the T5-VAE trained on python code.
- Fraser Greenlee's article for making a Transformer-VAE with JAX
- Fraser Greenlee's article about Transformers as Variational Autoencoders