shantanunandan/anlp

Applied Natural Language Processing course code(s)

Jupyter Notebook Viewer (nbviewer) link to browse the notebooks

1. Applied Natural Language Processing Course by NPTEL

• 1. Applied Natural Language Processing Course by NPTEL
  • 1.1. COURSE LAYOUT
    • 1.1.1. COURSE TYPE
    • 1.1.2. COURSE LEVEL
  • 1.2. BOOKS AND REFERENCES
    • 1.2.1. Books
    • 1.2.2. Articles
    • 1.2.3. Proceedings
  • 1.3. Code
    • 1.3.1. Week 1
    • 1.3.2. Week 2

https://hackersandslackers.com/

Applied Natural Language Processing course code(s)

ANLP Course by Mr.Ramaseshan R, Chennai Mathematical Institute

Natural Language Processing (NLP) is an important area of Artificial Intelligence concerned with the processing and understanding (NLU) of a human language. The goal of NLP and NLU is to process and harness information from a large corpus of text with very little manual intervention.

This course will introduce various techniques to find similar words using the context of surrounding words, build a Language model to predict the next word and generate sentences, encode every word in the vocabulary of the corpus into a vector form that represents its context and similar words and encode a sentence for machine translation and conversation purposes.

The course will help learners to gather sufficient knowledge and proficiency in probabilistic, Artificial Neural Network (ANN) and deep learning techniques for NLP.

INTENDED AUDIENCE: Any interested learners

PER-REQUISITES: Essential – Algorithms, Python proficiency, elementary probability and statistics, Linear Algebra, basic understanding of machine learning

NOTE: Only English corpus is considered throughout this course.

1.1. COURSE LAYOUT

• WEEK 1: Introduction, terminologies, empirical rules
• WEEK 2: Word to Vectors
• WEEK 3: Probability and Language Model
• WEEK 4: Neural Networks for NLP
• WEEK 5: Distributed word vectors (word embeddings)
• WEEK 6: Recurrent Neural Network, Language Model
• WEEK 7: Statistical Machine Translation
• WEEK 8: Statistical Machine Translation, Neural Machine Translation
• WEEK 9: Neural Machine Translation
• WEEK 10: Conversation Modeling, Chat-bots, dialog agents, Question Processing
• WEEK 11: Information Retrieval tasks using Neural Networks- Learn to Rank, Understanding Phrases, analogies
• WEEK 12: Spelling Correction using traditional and Neural networks, end notes

1.1.1. COURSE TYPE

• Elective

1.1.2. COURSE LEVEL

• Postgraduate

YouTube Video Link

• https://www.youtube.com/playlist?list=PLyqSpQzTE6M_EcNgdZ2qOtTZe7YI4Eedb
  • Video Playlist

1.2. BOOKS AND REFERENCES

1.2.1. Books

1. Niladri Sekhar Dash and S. Arulmozi, Features of a Corpus. Singapore: Springer Singapore, 2018, pp. 17–34. isbn: 978-981-10-7458-5. doi: 10.1007/978- 981- 10- 7458- 5_2, url: https://doi.org/10.1007/978981-10-7458-5_2.
2. Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Deep Learning, http://www.deeplearningbook.org. MIT Press, 2016.
3. Nitin Indurkhya and Fred J Damerau, "Handbook of natural language processing," Chapman and Hall/CRC, 2010.
4. Daniel Jurafsky and James H. Martin "Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition," 1st. Upper Saddle River, NJ, USA: Prentice Hall PTR, 2000. isbn: 0130950696.
5. C.D. Manning et al, "Foundations of Statistical Natural Language Processing," Mit Press. MIT Press, 1999. isbn: 9780262133609. url: https://books.google.co.in/books?id=YiFDxbEX3SUC.
6. Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schutze, "An Introduction to Information Retrieval," Cambridge UP, 2009. Chap. 6,pp. 109–133.
7. Jacob Perkins, "Python 3 text processing with NLTK 3 cookbook," Packt Publishing Ltd, 2014.
8. Noah A. Smith, "Linguistic Structure Prediction. Synthesis Lectures on Human Language Technologies," Morgan and Claypool, May 2011.

1.2.2. Articles

1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. “Neural machine translation by jointly learning to align and translate”. English (US). In: arXiv (2014).
2. Yoshua Bengio et al. “A Neural Probabilistic Language Model”. In: Journal of Machine Learning Research 3 (Mar. 2003), pp. 1137–1155. issn:1532-4435.
3. Peter F. Brown et al. “Class-based N-gram Models of Natural Language”. In: Comput. Linguist. 18.4 (Dec. 1992), pp. 467–479. issn: 0891-2017.
4. Peter F. Brown et al. “The Mathematics of Statistical Machine Translation: Parameter Estimation”. In: Comput. Linguist. 19.2 (June 1993), pp. 263–311. issn: 0891-2017.
5. KyungHyun Cho et al. “On the Properties of Neural Machine Translation:Encoder-Decoder Approaches”. In: CoRR abs/1409.1259 (2014). arXiv:1409.1259.
6. Scott Deerwester et al. “Indexing by latent semantic analysis”. In: JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE 41.6 (1990), pp. 391–407.
7. Chris Dyer. “Notes on Noise Contrastive Estimation and Negative Sampling”. In: CoRR abs/1410.8251 (2014). arXiv: 1410.8251.
8. Yoav Goldberg. “A Primer on Neural Network Models for Natural Language Processing”. In: CoRR abs/1510.00726 (2015). arXiv: 1510.00726.
9. Nils Hadziselimovic et al. “Forgetting Is Regulated via Musashi-Mediated Transnational Control of the Arp2/3 Complex.” In: Cell 156.6 (Mar. 2014),pp. 1153–1166. issn: 1097-4172.
10. Sepp Hochreiter and Jürgen Schmidhuber. “Long Short-Term Memory”. In: Neural Comput. 9.8 (Nov. 1997), pp. 1735–1780. issn: 0899-7667.
11. Chiori Hori and Takaaki Hori. “End-to-end Conversation Modeling Track in DSTC6”. In: CoRR abs/1706.07440 (2017). arXiv: 1706.07440.
12. Andrej Karpathy, Justin Johnson, and Fei-Fei Li. “Visualizing and Understanding Recurrent Networks.” In: CoRR abs/1506.02078 (2015).
13. Minh-Thang Luong, Hieu Pham, and Christopher D. Manning. “Effective Approaches to Attention-based Neural Machine Translation”. In: CoRR abs/1508.04025 (2015). arXiv:1508.04025.
14. Tomas Mikolov et al. “Efficient Estimation of Word Representations in Vector Space”. In: CoRR abs/1301.3781 (2013).
15. Franz Josef Och and Hermann Ney. “The Alignment Template Approach to Statistical Machine Translation”. In: Computational Linguistics 30.4 (Dec. 2004), pp. 417–449. issn: 0891-2017.
16. F. Pedregosa et al. “Scikit-learn: Machine Learning in Python”. In: Journal of Machine Learning Research 12 (2011), pp. 2825–2830.
17. Xin Rong. “word2vec Parameter Learning Explained”. In: CoRR abs/1411.2738 (2014). arXiv: 1411.2738. url: http://arxiv.org/abs/1411.2738.
18. Fraser W. Smith and Lars Muckli. “Nonstimulated early visual areas carry information about surrounding context”. In: Proceedings of the National Academy of Sciences 107.46 (2010), pp. 20099–20103.

1.2.3. Proceedings

1. Kyunghyun Cho et al. “Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation”. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar: Association for Computational Linguistics, Oct. 2014, pp. 1724–1734.
2. Rafal Jozefowicz, Wojciech Zaremba, and Ilya Sutskever. “An Empirical Exploration of Recurrent Network Architectures”. In: Proceedings of the 32Nd International Conference on International Conference on Machine Learning - Volume 37. ICML’15. Lille, France: JMLR.org, 2015, pp. 2342–2350.
3. Quoc Le and Tomas Mikolov. “Distributed representations of sentences and documents”. In: International conference on machine learning. 2014,pp. 1188–1196.
4. Edward Loper and Steven Bird. “NLTK: The Natural Language Toolkit”, In: Proceedings of the ACL-02 Workshop on Effective Tools and Methodologies for Teaching Natural Language Processing and Computational Linguistics - Volume 1. ETMTNLP ’02. Philadelphia, Pennsylvania: Association for Computational Linguistics, 2002, pp. 63–70.
5. Tomas Mikolov et al. “Distributed Representations of Words and Phrase and Their Compositionality”. In: Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2. NIPS’13.Lake Tahoe, Nevada: Curran Associates Inc., 2013, pp. 3111–3119.
6. Andriy Mnih and Geoffrey Hinton. “A Scalable Hierarchical Distributed Language Model”. In: Proceedings of the 21st International Conference on Neural Information Processing Systems. NIPS’08. Vancouver, British Columbia, Canada: Curran Associates Inc., 2008, pp. 1081–1088. isbn:978-1-6056-0-949-2.
7. Frederic Morin and Yoshua Bengio. “Hierarchical probabilistic neural network language model.” In: Aistats. Vol. 5. Citeseer. 2005, pp. 246–252.
8. Kishore Papineni et al. “Bleu: a Method for Automatic Evaluation of Machine Translation”. In: Proceedings of 40th Annual Meeting of the Association for Computational Linguistics. Philadelphia, Pennsylvania, USA: Association for Computational Linguistics, July 2002, pp. 311–318.

---

1.3. Code

1.3.1. Week 1

S.I	File	Details
1	Week1_Assignment.ipynb, Week1_Assignment.pdf	Week 1 Self-Assignments

---

1.3.2. Week 2

S.I	File	Details
1	Week2_Exercise_Shakespear Play.ipynb	Week 2 Lecture 2 Exercise

---