Week 1 - What is Deep Learning?
In which we discuss the characteristics of deep learning and consider its relationship with other approaches to machine learning and its position within the field of Artificial Intelligence.
Learning Objectives
By the end of the session the student will be able to:
- describe where Deep Learning fits within Artificial Intelligence and Machine Learning generally,
- explain the difference between machine learning and rule induction
- give examples of applications of machine learning
- create and run Python programs within the Jupyter notebook environment
- use basic elements of the Python language to solve simple computing problems.
Outline
- Motivation for the course
Some demonstrations of how deep learning is creating radically new applications of computer science. We look at IBM's Watson Text-to-Speech system, the use of deep learning in autonomous vehicles, deep reinforcement learning for playing games, generation of images from textual descriptions, neural machine translation, and spoken dialogue systems.
- Deep learning, machine learning and artificial intelligence
We position deep learning within the field of machine learning and within the area of artificial intelligence (AI). We give four different ways of thinking about the goals of AI. We contrast AI with the emerging field of computational cognitive science.
- S. Russell, P. Norvig, "Artificial Intelligence, a Modern Approach", Pearson, 2016, Chapter 1.
- A. Powers, "A critical time in computational cognitive science", J. Computational Cognitive Science, 1 2015.
- Machine learning compared to rule induction and statistical analysis
We introduce the field of machine learning and contrast it with conventional approaches to building intelligent applications. We consider the overlap between machine learning and conventional statistical analysis.
- A. Samuel, "Some studies in machine learning using the game of checkers", IBM Journal of Research and Development 3 1959.
- T. Hastie, R. Tibshirani, J.Friedman, "Elements of Statistical Learning", Springer, 2017.
- Types of machine learning
We differentiate Supervised, Reinforcement, and Unsupervised approaches to machine learning with some example problems. We also contrast a classification problem from a regression problem.
- D. Fumo, Types of machine learning algorithms you should know
- A. Geron, "Hands on machine learning with Scikit-learn and Tensorflow", O'Reilly, 2019, Chapter 1.
- The deep learning revolution
We introduce the particular characteristics of the "deep" approach to machine learning. We highlight the use of neural networks with large numbers of layers, the application to real-world problems, the reduced reliance on human expertise, the desire for "end-to-end" processing that combines feature estimation with problem learning, and the creation of new abstract data representations. We discuss why deep learning has become important at this point in the history of computer science.
- Deep Learning, Wikipedia.
- Kaggle Machine Learning Competitions
- The computational framework for the course
We introduce the Colaboratory environment and the Python programming language that we will be using for demonstrations and exercises through the course.
Web Learning Resources
- LinkedIn Learning Video Courses: Python Essential Training, and Numpy Data Science Essential Training: Introduction to Jupyter Notebooks. Accessible to all UCL staff and students through this sign on. Enter the video course name in the search box.
- Python 3 language reference and Python tutorial introduction.
- Jupyter notebook documentation and examples
Readings
Be sure to read one or more of these introductions to Machine Learning:
- Chapter 1 - The machine learning landscape, in Hands-On Machine Learning with Scikit-Learn and TensorFlow, by Aurélien Géron.
- Introduction to Machine Learning for Beginners
- Timeline of machine learning.
Tutorial Notebooks
Exercises
Implement answers to the problems described in the notebooks below. Save your completed notebooks into your personal Google Drive account.
Word count: . Last modified: 22:45 11-Mar-2022.