vkasera/zerotohero

My implementation of modules from Andrej Karapathy's Zero to Hero course

zerotohero

My implementation of modules from Andrej Karapathy's Zero to Hero course

File descriptions:

• names_bigrams.ipynb: The first module: simple bigram character model that learns to generate names. Minimal usage of Pytorch APIs.
• names_ngrams*.ipynb: Multiple experiments with n-grams model. Changes to param initialization (e.g. kaiming init, zero init) as well as batch normalization. Minimal usage of pytorch APIs.
• nn_test.ipynb: simple tutorial from pytorch for using pytorch APIs.
• pytorch-api-name-ngram-batchnorm.ipynb: ngram model that uses Pytorch APIs.
• wavenet-name-ngrams.ipynb: wavenet implementation of name generator.
• transformer.ipynb: The final module: transformer in pytorch that implements a character level language model. Given a dataset of shakespeare texts can generate shakespeare-like language.

Backprop from scratch: See this colab notebook: https://colab.research.google.com/drive/147MlhhT1RfAwE0JWLzp3JYtv7Z9YLryc?usp=sharing