Department of Phonetics and Linguistics


Michael ASHBY, John MAIDMENT and Evelyn ABBERTON

In this paper we outline a new method for training and assessing auditory skills in students of phonetics, which has now been introduced into all the undergraduate programmes involving phonetics at UCL. Conventional ear-training uses dictated or recorded material of approximately phrase length, and requires students to transcribe their answers in full starting from a blank sheet of paper. The new technique, called Analytic Listening, involves multiple-choice testing. Each question focuses the students' attention upon one relevant phonetic parameter at a time and their perception is then tested with a short battery of varied stimuli. They are not asked to transcribe what they hear, but to choose from a fixed range of responses. Analytic Listening offers rapid and simple marking and feedback, leads to statistically reliable profiles of students' auditory skills, and reduces the emphasis on symbols and transcription. Because it focuses upon and tests an explicit range of relevant phonetic parameters, it permits auditory-skill benchmarks to be incorporated into syllabus design. At the same time, it encourages a realistic view of the power and limitations of humanly-made phonetic judgements alongside instrumental measurements of speech, which increasingly form a part of classroom training.

Traditional ear-training and its limitations
Traditionally, the training and testing of auditory skills has been done by Ear-Training Dictation. The teacher repeats aloud certain speech sequences (alternatively, of course, recorded materials may be used). The students write down what they hear in phonetic symbols. The students' versions are compared with what was intended, and a score is worked out which is essentially a measure of how far what the student has written corresponds with the teacher's original version. Two types of material have generally been used. (1) connected speech in a specified accent of the principal language being studied. For this a systematic (more-or-less phonemic) transcription is usually required; (2) invented or `nonsense' words, either drawing sounds from the phonological system of a specific language or else from a more general repertoire. For items drawing on a wide range of sounds, an impressionistic (non-systematic) transcription is appropriate. The obvious advantage of nonsense materials is to test sound identification and discrimination without assistance from context.

The marking of dictation scripts is labour-intensive. A single script can easily contain scores - even hundreds - of "errors". Each error must be found, and its severity ranked. Commonly, elaborate schemes are worked out for costing anticipated types of error. But answers can depart from what is intended in numerous unforseen ways, creating difficulties for those marking and moderating the test. Similarly, feedback to students needs to be tailored to their individual patterns of error. Detailed comments must be given in one-to-one tuition, or written individually on submitted work. To a limited extent, students may correct their own (or each others') work against a correct answer, but this encourages the belief that there is one correct answer. However, since transcription is to some extent subjective, more than one answer may in fact have some merit, once the most elementary level of transcription has been passed. For instance, transcribers can differ - quite reasonably - over such apparently basic issues as how many segments a given sequence seems to contain.

Analytic Listening
We have attempted to devise an approach to auditory training and assessment which will both reduce the labour involved in marking and assessment and also place more emphasis on critical, analytic listening rather than the process of getting down the "right" symbols. We call the technique Analytic Listening. It seems to go a considerable way towards overcoming both the theoretical and practical difficulties associated with the traditional approach, and is now running as part of training and assessment in all UCL undergraduate programmes involving basic practical phonetics.

Analytic listening is done in ear-training class in the regular way, and assessed as before with a dictation examination. What is different is that the students do not sit down with a blank sheet of paper, but with a multiple-choice form. Each question in the multiple choice test explicitly focuses attention upon some relevant property or parameter, and the students' ability to detect that property, or make discriminations along the parameter, is then tested several times over with changing material.

Basic form of an Analytic Listening question
A short sample of Analytic Listening material accompanies this paper. A teaching or examination session would involve a number of questions (perhaps 10 or 20) taking two to three minutes each in a live presentation. Each question deals with a different phonetic point. In Q1 of the sample given here, the focus is upon the presence or absence of a glottal stop at the beginning of a short sequence - whether or not there is a "hard attack". (This is, in fact, a realistic example of the sort of phonetic judgement that has to be made routinely by a Speech and Language Therapist in a voice clinic). Each question in turn comprises a number of items - we have settled on five in the exercises we have designed so far. The five items within the question all test the same perceptual judgement, but with a different dictated sequence. Each item is repeated typically three times before the students mark their responses and move on to the next. The dictated material may be of various types - sometimes English-like, sometimes nonsense - and can be constructed so as to contain potential distractors (e.g. in this case a glottal stop at the end instead of the beginning). The similarity of the technique with forced-choice perception testing will be obvious. Of course, the listener may sometimes guess, but the overall scoring is arranged to take account of this (as with any Multiple-Choice test). Each question alone has some statistical utility anyway: a listener who makes the right choice five times out of five has only a three percent chance of doing this by guessing. When all five items have been dealt with, the students move on to the next question, which will generally focus upon a different phonetic distinction.

For training, rather than testing, the teacher can of course use a question of this sort to introduce the discrimination in question. One can begin by giving feedback after each item and one can repeat contrasting items in sequence. In a way this merely gives some structure to the kind of improvised repetition and comparison that has always formed part of every good ear-training class in response to students' questions, but it is worth emphasising some differences from the traditional method which this simple example illustrates. First a single clearly-defined phonetic difference is at issue and the expectations for the task clearly set out. Second, the discrimination is tested both thoroughly and reliably (it is unclear what statistical reliability - if any - may attach to results obtained in conventional ear-training dictation). Thirdly, although a ready-made answer form is required (either on paper, or computer-presented), this may be re-used indefinitely with changing dictation material, leading to long-term economies in preparation time for teachers.

Cueing the listeners in
We believe that a very important feature of Analytic Listening is the way in which the question first cues the listener in to the point under consideration. When students are asked to transcribe items in full in the traditional manner, they must generally do so against a background of essentially arbitrary conventions, established for the course or even by individual teachers. For instance, it may be established in advance that some property (aspiration, nasality, length, etc) is/is not required to be marked, that all vowel qualities are to be shown as Cardinal Vowels but without diacritics, and so on. Without such conventions, students' versions would differ unpredictably and chaotically so that marking and feedback would be virtually impossible. Learning to do well in an ear-training class is thus partly a matter of acquiring and falling-in with conventions. But because Analytic Listening directs attention to a specific phonetic property in each question, and gives a direct indication of the type of answer expected, these arbitrary conventions are largely eliminated. Q2 gives another illustration of how the cueing-in process works. The wording of the question tells students to make the assumption that these items are VCV, then make a judgement about the C part. This is a very different matter from simply expecting the students to assume that the sequences are VCV. The orientation suggested in the rubric is to be taken a sort of working hypothesis. It permits the use of quite ambiguous sequences as dictation items, for instance a sequence such as [awa], (where it is questionable whether the portion represented [w] is a "consonant") or items containing affricates (where the number of segments is problematic). Further, the question does not ask "is the consonant a voiced fricative?" but rather "does it have voiced friction?" That friction might indeed be within a fricative proper, but it might be in an affricate, or might be added to a type generally familiar only as a sonorant, such as a nasal. We are thus able to elicit fine phonetic judgements about specific parts of sequences without reinforcing simple assumptions about the nature of those sequences.

Applicability of the technique
The two examples given so far have been based on single segments of consonantal type, but of course the technique is applicable to a wide range of segmental and non-segmental effects. Q3 is an illustration of the technique applied to vocalic sounds.. We can ask specifically about frontness, rounding, and so on, in particular vowels, or ask questions which require the comparison of two vowel tokens (same/different, longer/shorter, etc). Equally, stress or pitch may be what is focused on; for instance, listeners may be asked which of two identified syllables within a phrase has the higher pitch, where the highest pitch within a sequence is located, and so on. The technique thus offers a way to ask quite specific questions about prosodic properties, without requiring that students approach the material with a ready-made ("phonological") framework for the analysis of intonation.

Since in general there are no phonetic symbols used in the questions or required in the answers, we can employ sound-types which the students have not explicitly met before, or for which there are no generally agreed symbols. Learning of symbols is not infrequently seen as a laborious and unattractive task. Some speech professionals (eg Speech and Language Therapists) who have qualified in phonetics and still require to use phonetics tell us that the forgetting of symbols, or the failure to keep up with changes in symbolisation, are major embarrassments to them. Analytic Listening will therefore have an immediate application in refresher courses, and should help to get across the message that forgetting symbols is not the same as forgetting one's phonetic skills.

Eventually, of course, most of our students do learn a wide range of phonetic symbols, and as they progress students begin to include symbols alongside their answer choices and to attempt here and there complete transcription of the dictated items. This in our view puts symbols in their rightful place: very much secondary to the essential phonetic judgements. Our hope is that we can wean students from the idea that there is a fixed repertoire of sounds, and that everything they hear must be an example of one of them. We do not want our students to be able to cope only with what they have "done" previously. We would prefer them to think of speech as a complex of numerous parameters, that can combine in new and possibly surprising ways. Human attempts to represent the complexity of speech (by transcription) are limited and relatively unreliable; where human listeners excel is in making judgements concerning specific features - even tiny details - within that complexity.

Levels of analysis
It may perhaps be objected that by cueing the listeners in, and giving a fixed range of choices for responses we are producing a task that is too easy compared with unrestricted transcription. Our response to this is that the technique removes largely irrelevant distractions and pressures, but does not remove or in any way dilute the essential auditory discriminations to be acquired. It is certain that if students cannot handle the important sound discriminations under the controlled conditions of Analytic Listening, they will not be able to do so in the rough-and-tumble of unrestricted transcription.

A further advantage of cueing the listeners in, is that the expected level of `narrowness' can be varied from question to question. Most introductory practical phonetics courses, for instance, would not expect as routine the marking of aspiration differences, or of a wide range of secondary articulations. But specific Analytic Listening questions can focus on such matters, as Q4 and Q5 illustrate. We can thus teach and test finer discriminations than is easily possible with conventional dictation material.

Materials - development and future
The first Analytic Listening materials were constructed in 1991 and introduced into course P101 (Introduction to Phonetics and Phonology A) which forms part of the BA programme in Linguistics at UCL. In 1994 the technique was introduced into course B104 of the BSc (Speech Sciences) programme, and in summer 1995 assessment by means of Analytic Listening replaced the nonsense-word component of the examination for that course. Samples of Analytic Listening materials were distributed nationally to accompany presentations at a NetPhon workshop in 1995 and to the British Association of Academic Phoneticians at the 1996 Colloquium in York.

Quite extensive recordings of Analytic Listening material were completed in 1995 for use in the Department's Listening Centre and for sale to students. For these materials, students record their responses in printed answer books. Printed answer sheets and books are also used in regular classes and for examinations. This simple method of presentation has the advantage that students can listen to the material whenever they wish, and the printed answer books give a permanent record for feedback. But there will be real advantages also in linking the Analytic Listening technique to a system for Computer-Aided Learning. Such a system could deliver training and test materials, give rapid feedback, keep long-term records of progress, and perhaps work adaptively to match the different needs of users.

One promising system is LAPT (London Agreed Protocol for Teaching), developed originally at UCL with input from other London Medical Schools. This is a flexible PC-based system for delivering computer-based teaching. It has so far been used principally by medical and biomedical science students. As the exercises are simply written on a wordprocessor, new subject matter from a range of contributing colleagues is easily incorporated. A unique feature of LAPT is the introduction of confidence assessment into multiple-choice testing. After each answer, students are asked to declare a confidence level of 1, 2, or 3. For a correct answer, the marks gained are 1, 2, or 3. For an incorrect answer, the marks are 0, -2 or -6. The intention is encourage students to think realistically about the confidence they feel in their answers. We believe this scheme will be especially valuable when extended to auditory judgements. In our view, it is very important that those trained in phonetics should be able to form some estimate of the reliability of their perceptions (in terms of repeatability, likely consensus with other observers, and agreement with what instrumental analyses might be likely to show).

At present LAPT does not have facilities for outputting sound, and our intention is to marry it with the system developed by our colleague M. Huckvale which he has called CD-Mike. In this, a PC becomes a sophisticated player for an audio CD, any part of which can be randomly accessed using information in a stored `script' giving identification and timing of the contents of the disk. Advantages of this system are that the audio CD can be a relatively straightforward transfer from existing recorded material, while any new CD will also be usable as a stand-alone recording.

We have outlined the background to the development of Analytic Listening, described some of its main features and indicated one line of future work. It remains to put the technique into a broader context.

Because it involves questions each with a clear perceptual focus, Analytic Listening opens the way to designing the practical phonetics syllabus around an explicit hierarchy of phonetic properties and parameters. Instead of being based upon a certain type of material (nonsense, RP, etc), or upon a fixed number of "sounds" to be covered , our syllabus can be based directly upon auditory skills (ability to make judgements concerning voicing, nasality, pitch, aspiration, and so on). For each programme, and for each course within the programmes, we must specify just what phonetic properties it is relevant to teach and to test at each stage. The Analytic Listening technique than gives us the means to train awareness of these properties, and to test them reliably and directly. There are possibilities for national and international comparisons between courses and examinations and even for standardisation of recognised tests.


Examples of Analytic Listening material
The first few questions in this short sample are referred to in the text by question number (Q1, etc) and illustrate specific points mentioned there. The remaining questions are included as additional illustrations. The set of ten questions is not intended to illustrate a balanced lesson plan or assessment test, but merely to exemplify the technique. The phonetic transcription given at right shows suitable dictation material to accompany the items and would not of course be visible on students' copies of answer books.

© 1996 Michael Ashby, John Maidment and Evelyn Abberton

Back to SHL 9 Contents

Back to Publications

Back to Phonetics and Linguistics Home Page

These pages were created by: Martyn Holland.
Comments to: