Week 10 - Deep Learning and Artificial Intelligence

In which discuss the current deficiencies of deep learning for NLP, and the challenges to the creation of a human-level artificial intelligence.

Learning Objectives

By the end of the session the student will be able to:

• describe how artificially intelligent systems may add to human problem-solving abilities
• describe the limitations of machine learning for understanding human cognition
• discuss some current deficiencies of deep learning with respect to speech and language processing
• outline the philosophical argument against the possibility that artefacts can understand language
• discuss how embodied learning agents might acquire semantics
• outline key ethical concerns about AI applications in the modern world

Outline

Goals of artificial intelligence

We revisit the goals of artificial intelligence, and suggest the most sensible objective is to create artefacts that exhibit intelligent behaviours rather than attempting to think like a human or to duplicate human intelligence. Such systems might allow us to become super-intelligent rather than being intelligent themselves. Even though AI systems are not models of human cognition, they can still influence our understanding of cognition, particularly if the emergent behaviours of AI systems parallel those of humans. This might show, for example, that certain human behaviours are consequences of solving a problem in an efficient way, and so contigent on the problem rather than on the structure of human brains.

• N. Bostrom, SuperIntelligence: Paths, Dangers Strategies, 2014.

Deficiencies of deep learning for speech and language processing

We discuss some clear weaknesses in the deep learning approach to processing speech and language: (i) that DL systems simulate the use of language rather than knowingly use language, (ii) that they appear to understand more than they actually do, (iii) that the systems have no models of interlocutors in the way that people do, (iv) that the systems treat speech as if it were spoken text, (iv) that chatbot systems treat conversations as stimulus response, (v) that systems have no meta-awareness of language, and cannot talk about language itself.

• M. Huckvale, Speech Synthesis, Speech Simulation and Speech Science, Proc. International Conference on Speech and Language Processing, Denver, 2002, pp1261-1264.

Philosophical limitations of deep learning

We discuss what it would mean for an AI system to "understand" language, through the presentation of the Chinese Room thought experiment. This returns us to the problem of symbol grounding and the need for robots to experience how language relates to the physical world rather than just to language itself. We comment that building models to predict future events in the world, whether that be of physical objects or words in a sentence, could provide an underlying model of how to acquire knowledge and understanding. In turn this could create a unified model of human learning and cognition.

• The chinese room argument, Stanford Encyclopedia of Philosophy.
• K. Craik, The nature of explanation, 1943.
• J. Hawkins, On Intelligence, 2004.
• A. Clark, Surfing Uncertainty, 2016.
• D. Mareschal, R. French, TRACX2: a connectionist autoencoder using graded chunks to model infant visual statistical learning, 2017.

Ethical considerations

We discuss the essential amorality of machine learning. Models trained from data containing the inequalities, biases and prejudices of the human world will reflect those baises, but without being aware of them. This means we should not use deep learning to make moral judgements, and we should acknowledge this when we use deep learning to make life and death decisions.

• Why we should ban lethal autonomous weapons, Future of life institute, YouTube.

Research Paper of the Week

• R. Cicky, D. Kaiser, Deep Neural Networks as Scientific Models, Trends in Cognitive Sciences, Opinion, 23 (2019).

Web Resources

• John Searle: Minds, Brains and Science 1984 Reith Lectures. The Week 2 Lecture on "Beer cans and meat machines" is particularly relevant (and very entertaining).

Readings

Be sure to read one or more of these discussions about the role of deep learning in artificial intelligence and NLP.

• T. Sejnowski, The unreasonable effectiveness of deep learning in artificial intelligence, PNAS, 2020.
• T. Young, D. Hazarika, S. Poria, E. Cambria, Recent Trends in Deep Learning Based Natural Language Processing. Arxiv, 2018.

Word count: . Last modified: 09:42 23-Dec-2021.