PALS0039 Introduction to Deep Learning for Speech and Language Processing
UCL Division of Psychology and Language Sciences

Week 2 - Principles of Machine Learning

In which we present the essential process of using machine learning for simple regression and classification tasks, including data preparation, algorithm selection and hyperparameter selection, with particular emphasis on the problem of overfitting and the need for cross-validation.

Learning Objectives

By the end of the session the student will be able to:


  1. When to use Machine Learning
  2. Fundamentally Machine Learning is about learning rules and patterns from data and then applying them to new samples of the problem. ML is preferred over hand-generated rules when the list of rules is long and complex, where no good solutions exist, if the problem is so hard that there are no insights into how it might be addressed, or if the data fluctuate over time. ML is not appropriate when causal models exist (except to approximate these with less computation), when only a small number of examples exist, where there is a need to test an explicit hypothesis about the problem, or where we cannot state the objective of learning in mathematical form.

  3. The process of Machine Learning
  4. For ML to be useful it must generalise from the given data to new data.So it is essential to evaluate ML using separate or held-out data. This is called "cross-validation". When planning a machine learning solution to a problem there are common steps to follow in collecting the data, designing the representations of the data, choosing an ML method and training a model, and evaluating the performance using a metric for success.

    • A. Geron, "Hands on machine learning with Scikit-learn and Tensorflow", O'Reilly, 2019, Chapter 2.

  5. Machine Learning methods (not deep learning)
  6. We give an example of supervised learning of a regression task using vowel formant F1 to predict vowel formant F2. We give an example of supervised learning of a classification task using prediction of gender from vowel formant frequencies. We give an example of unsupervised learning using a clustering problem. We give an example of reinforcement learning by training an agent to collect food but ignore poison when moving around a virtual 2D world.

  7. Machine Learning step-by-step
  8. We look in more detail at the steps necessary to implement a ML application: (1) Prepare the data, (2) Generate and select features, (3) Choose ML strategy, (4) Choose ML method, (5) Choose hyper-parameters, (6) Train model on data, (7) Final evaluation. We use the development of an application that recognises a speaker's emotional state from their speech as an example. We consider some of the problems in ML that can lead to poor performance.

Research Paper of the Week

Web Resources


Be sure to read one or more of these descriptions of the practice of Machine Learning:

Tutorial Notebooks


Implement answers to the problems described in the notebooks below. Save your completed notebooks into your personal Google Drive account.

    1. Pandas practice
    2. Regression task
    3. Classification task

Word count: . Last modified: 22:45 11-Mar-2022.