PALS0039 Introduction to Deep Learning for Speech and Language Processing
UCL Division of Psychology and Language Sciences

Week 8 - Sequence to Sequence Models

In which we discuss networks that transform between sequences, with particular interest in machine translation.

Learning Objectives

By the end of the session the student will be able to:


  1. History of machine translation
  2. We briefly describe the long history of machine translation of human languages, contrasting rule-based, exemplar-based, statistical and neural machine translation approaches. We also describe some commonly-used metrics for measuring the performance of translation systems, including the BLEU and METEOR scores.

  3. Sequence to sequence models
  4. We describe how deep learning can be applied to the problem of sequence-to-sequence conversion. We present the basic architecture, then discuss how the introduction of attention and transformer mechanisms enhance the performance and practicality of the approach.

  5. Multilingual neural machine translation
  6. We discuss the multi-lingual translation problem in which there are large number of source and target languages. We describe one approach that claims to be able to translate bteewen pairs of languages for which no paired corpus is available.

  7. Other Applications of seq2seq Models
  8. We briefly introduce some other Speech and Language problems that have been addressed by the seq2seq approach.

Research Paper of the Week

Web Resources


Be sure to read one or more of these discussions of neural machine translation:


Implement answers to the problems described in the notebooks below. Save your completed notebooks into your personal Google Drive account.

    1. Sequence-to-Sequence processing
    2. Grapheme to phoneme conversion

Word count: . Last modified: 21:31 13-Mar-2022.