Deep Learning for NLP – Part 3



What you’ll learn

  • Deep Learning for Natural Language Processing

  • Sentence Embeddings: Bag of words, Doc2Vec, SkipThought, InferSent, DSSM, USE, MTDNN, SentenceBERT

  • Generative Transformer Models: UniLM, Transformer-XL and XLNet, MASS, BART, CTRL, T5, ProphetNet


  • Basics of machine learning

  • Recurrent Models: RNNs, LSTMs, GRUs and variants

  • Basic understanding of Transformer based models and word embeddings

This course is a part of “Deep Learning for NLP” Series. In this course, I will introduce concepts like Sentence embeddings and Generative Transformer Models. These concepts form the base for good understanding of advanced deep learning models for modern Natural Language Generation.

The course consists of two main sections as follows.

In the first section, I will talk about sentence embeddings. We will start with basic bag of words methods where sentence embedddings are obtained using an aggregation over word embeddings of constituent words. We will talk about averaged bag of words, word mover’s distance, SIF and Power means method. Then we will discuss two unsupervised methods: Doc2Vec and SkipThought. Further, we will discuss about supervised sentence embedding methods like recursive neural networks, deep averaging networks and InferSent. CNNs can also be used for computing semantic similarity between two text strings; we will talk about DSSMs for the same. We will also discuss 3 multi-task learning methods including Universal Sentence Encodings and MT-DNN. Lastly, I will talk about SentenceBERT.

In the second section, I will talk about multiple Generative Transformer Models. We will start with UniLM. Then we will talk about segment recurrence and relative position embeddings in Transformer-XL. Then get to XLNets which use Transformer-XL along with permutation language modeling. Next we will understand span masking in MASS and also discuss various noising methods on BART. We will then discuss about controlled natural language generation using CTRL. We will discuss how T5 models every learning task as a text-to-text task. Finally, we will discuss how ProphetNet extends 2-stream attention modeling from XLNet to n-stream attention modeling, thereby enabling n-gram predictions.

Who this course is for:

  • Beginners in deep learning
  • Python developers interested in data science concepts
  • Masters or PhD students who wish to learn deep learning concepts quickly
  • Supervised method: RecNNs and Deep Averaging Networks


  • CNNs for semantic similarity: DSSM

    04:40Multi-Task Learning: MTDNN

  • 06:21

  • Multi-Task Learning: MILA/MSR Sentence Embeddings


Principa Applied Researcher

Manish Gupta is a Principal Applied Researcher at Microsoft India R&D Private Limited at Hyderabad, India. He is also an Adjunct Faculty at International Institute of Information Technology, Hyderabad and a visiting faculty at Indian School of Business, Hyderabad. He received his Masters in Computer Science from IIT Bombay in 2007 and his Ph.D. from the University of Illinois at Urbana-Champaign in 2013. Before this, he worked for Yahoo! Bangalore for two years. His research interests are in the areas of web mining, data mining and information retrieval. He has published more than 100 research papers in reputed refereed journals and conferences. He has also co-authored two books: one on Outlier Detection for Temporal Data and another one on Information Retrieval with Verbose Queries.

Deep Learning for NLP – Part 3, Free Tutorials Download

Download Deep Learning for NLP – Part 3 Free Tutorials Direct Links

Go to Download Tutorials Page Go to HomePage Tutorials

Password :

Author: Ho Quang Dai

I am Ho Quang Dai, from Vietnam – A country that loves peace. I share completely free courses from major academic websites around the world. Hope to bring free knowledge to everyone who can’t afford to buy

Related Courses

Notify of
Inline Feedbacks
View all comments

Report Link Die

Please provide the most detailed information, we will re-upload as soon as possible