Traditional ear-training and its limitations
Traditionally, the training and testing of auditory skills has
been done by Ear-Training Dictation. The teacher repeats aloud
certain speech sequences (alternatively, of course, recorded materials
may be used). The students write down what they hear in phonetic
symbols. The students' versions are compared with what was intended,
and a score is worked out which is essentially a measure of how
far what the student has written corresponds with the teacher's
original version. Two types of material have generally been used.
(1) connected speech in a specified accent of the principal language
being studied. For this a systematic (more-or-less phonemic) transcription
is usually required; (2) invented or `nonsense' words, either
drawing sounds from the phonological system of a specific language
or else from a more general repertoire. For items drawing on a
wide range of sounds, an impressionistic (non-systematic) transcription
is appropriate. The obvious advantage of nonsense materials is
to test sound identification and discrimination without assistance
from context.
The marking of dictation scripts is labour-intensive. A single script can easily contain scores - even hundreds - of "errors". Each error must be found, and its severity ranked. Commonly, elaborate schemes are worked out for costing anticipated types of error. But answers can depart from what is intended in numerous unforseen ways, creating difficulties for those marking and moderating the test. Similarly, feedback to students needs to be tailored to their individual patterns of error. Detailed comments must be given in one-to-one tuition, or written individually on submitted work. To a limited extent, students may correct their own (or each others') work against a correct answer, but this encourages the belief that there is one correct answer. However, since transcription is to some extent subjective, more than one answer may in fact have some merit, once the most elementary level of transcription has been passed. For instance, transcribers can differ - quite reasonably - over such apparently basic issues as how many segments a given sequence seems to contain.
Analytic Listening
We have attempted to devise an approach to auditory training and
assessment which will both reduce the labour involved in marking
and assessment and also place more emphasis on critical, analytic
listening rather than the process of getting down the "right"
symbols. We call the technique Analytic Listening. It seems to
go a considerable way towards overcoming both the theoretical
and practical difficulties associated with the traditional approach,
and is now running as part of training and assessment in all UCL
undergraduate programmes involving basic practical phonetics.
Analytic listening is done in ear-training class in the regular way, and assessed as before with a dictation examination. What is different is that the students do not sit down with a blank sheet of paper, but with a multiple-choice form. Each question in the multiple choice test explicitly focuses attention upon some relevant property or parameter, and the students' ability to detect that property, or make discriminations along the parameter, is then tested several times over with changing material.
Basic form of an Analytic Listening question
A short sample of Analytic Listening material accompanies this
paper. A teaching or examination session would involve a number
of questions (perhaps 10 or 20) taking two to three minutes each
in a live presentation. Each question deals with a different
phonetic point. In Q1 of the sample given here, the focus is
upon the presence or absence of a glottal stop at the beginning
of a short sequence - whether or not there is a "hard attack".
(This is, in fact, a realistic example of the sort of phonetic
judgement that has to be made routinely by a Speech and Language
Therapist in a voice clinic). Each question in turn comprises
a number of items - we have settled on five in the exercises we
have designed so far. The five items within the question all test
the same perceptual judgement, but with a different dictated sequence.
Each item is repeated typically three times before the students
mark their responses and move on to the next. The dictated material
may be of various types - sometimes English-like, sometimes nonsense
- and can be constructed so as to contain potential distractors
(e.g. in this case a glottal stop at the end instead of the beginning).
The similarity of the technique with forced-choice perception
testing will be obvious. Of course, the listener may sometimes
guess, but the overall scoring is arranged to take account of
this (as with any Multiple-Choice test). Each question alone has
some statistical utility anyway: a listener who makes the right
choice five times out of five has only a three percent chance
of doing this by guessing. When all five items have been dealt
with, the students move on to the next question, which will generally
focus upon a different phonetic distinction.
For training, rather than testing, the teacher can of course use a question of this sort to introduce the discrimination in question. One can begin by giving feedback after each item and one can repeat contrasting items in sequence. In a way this merely gives some structure to the kind of improvised repetition and comparison that has always formed part of every good ear-training class in response to students' questions, but it is worth emphasising some differences from the traditional method which this simple example illustrates. First a single clearly-defined phonetic difference is at issue and the expectations for the task clearly set out. Second, the discrimination is tested both thoroughly and reliably (it is unclear what statistical reliability - if any - may attach to results obtained in conventional ear-training dictation). Thirdly, although a ready-made answer form is required (either on paper, or computer-presented), this may be re-used indefinitely with changing dictation material, leading to long-term economies in preparation time for teachers.
Cueing the listeners in
We believe that a very important feature of Analytic Listening
is the way in which the question first cues the listener in to
the point under consideration. When students are asked to transcribe
items in full in the traditional manner, they must generally do
so against a background of essentially arbitrary conventions,
established for the course or even by individual teachers. For
instance, it may be established in advance that some property
(aspiration, nasality, length, etc) is/is not required to be marked,
that all vowel qualities are to be shown as Cardinal Vowels but
without diacritics, and so on. Without such conventions, students'
versions would differ unpredictably and chaotically so that marking
and feedback would be virtually impossible. Learning to do well
in an ear-training class is thus partly a matter of acquiring
and falling-in with conventions. But because Analytic Listening
directs attention to a specific phonetic property in each question,
and gives a direct indication of the type of answer expected,
these arbitrary conventions are largely eliminated. Q2 gives another
illustration of how the cueing-in process works. The wording
of the question tells students to make the assumption that these
items are VCV, then make a judgement about the C part. This is
a very different matter from simply expecting the students to
assume that the sequences are VCV. The orientation suggested in
the rubric is to be taken a sort of working hypothesis. It permits
the use of quite ambiguous sequences as dictation items, for instance
a sequence such as [awa], (where it is questionable whether the
portion represented [w] is a "consonant") or items containing
affricates (where the number of segments is problematic). Further,
the question does not ask "is the consonant a voiced fricative?"
but rather "does it have voiced friction?" That friction
might indeed be within a fricative proper, but it might be in
an affricate, or might be added to a type generally familiar only
as a sonorant, such as a nasal. We are thus able to elicit fine
phonetic judgements about specific parts of sequences without
reinforcing simple assumptions about the nature of those sequences.
Applicability of the technique
The two examples given so far have been based on single segments
of consonantal type, but of course the technique is applicable
to a wide range of segmental and non-segmental effects. Q3 is
an illustration of the technique applied to vocalic sounds.. We
can ask specifically about frontness, rounding, and so on, in
particular vowels, or ask questions which require the comparison
of two vowel tokens (same/different, longer/shorter, etc). Equally,
stress or pitch may be what is focused on; for instance, listeners
may be asked which of two identified syllables within a phrase
has the higher pitch, where the highest pitch within a sequence
is located, and so on. The technique thus offers a way to ask
quite specific questions about prosodic properties, without requiring
that students approach the material with a ready-made ("phonological")
framework for the analysis of intonation.
Since in general there are no phonetic symbols used in the questions or required in the answers, we can employ sound-types which the students have not explicitly met before, or for which there are no generally agreed symbols. Learning of symbols is not infrequently seen as a laborious and unattractive task. Some speech professionals (eg Speech and Language Therapists) who have qualified in phonetics and still require to use phonetics tell us that the forgetting of symbols, or the failure to keep up with changes in symbolisation, are major embarrassments to them. Analytic Listening will therefore have an immediate application in refresher courses, and should help to get across the message that forgetting symbols is not the same as forgetting one's phonetic skills.
Eventually, of course, most of our students do learn a wide range of phonetic symbols, and as they progress students begin to include symbols alongside their answer choices and to attempt here and there complete transcription of the dictated items. This in our view puts symbols in their rightful place: very much secondary to the essential phonetic judgements. Our hope is that we can wean students from the idea that there is a fixed repertoire of sounds, and that everything they hear must be an example of one of them. We do not want our students to be able to cope only with what they have "done" previously. We would prefer them to think of speech as a complex of numerous parameters, that can combine in new and possibly surprising ways. Human attempts to represent the complexity of speech (by transcription) are limited and relatively unreliable; where human listeners excel is in making judgements concerning specific features - even tiny details - within that complexity.
Levels of analysis
It may perhaps be objected that by cueing the listeners in, and
giving a fixed range of choices for responses we are producing
a task that is too easy compared with unrestricted transcription.
Our response to this is that the technique removes largely irrelevant
distractions and pressures, but does not remove or in any way
dilute the essential auditory discriminations to be acquired.
It is certain that if students cannot handle the important sound
discriminations under the controlled conditions of Analytic Listening,
they will not be able to do so in the rough-and-tumble of unrestricted
transcription.
A further advantage of cueing the listeners in, is that the expected level of `narrowness' can be varied from question to question. Most introductory practical phonetics courses, for instance, would not expect as routine the marking of aspiration differences, or of a wide range of secondary articulations. But specific Analytic Listening questions can focus on such matters, as Q4 and Q5 illustrate. We can thus teach and test finer discriminations than is easily possible with conventional dictation material.
Materials - development and future
The first Analytic Listening materials were constructed in 1991
and introduced into course P101 (Introduction to Phonetics and
Phonology A) which forms part of the BA programme in Linguistics
at UCL. In 1994 the technique was introduced into course B104
of the BSc (Speech Sciences) programme, and in summer 1995 assessment
by means of Analytic Listening replaced the nonsense-word component
of the examination for that course. Samples of Analytic Listening
materials were distributed nationally to accompany presentations
at a NetPhon workshop in 1995 and to the British Association
of Academic Phoneticians at the 1996 Colloquium in York.
Quite extensive recordings of Analytic Listening material were completed in 1995 for use in the Department's Listening Centre and for sale to students. For these materials, students record their responses in printed answer books. Printed answer sheets and books are also used in regular classes and for examinations. This simple method of presentation has the advantage that students can listen to the material whenever they wish, and the printed answer books give a permanent record for feedback. But there will be real advantages also in linking the Analytic Listening technique to a system for Computer-Aided Learning. Such a system could deliver training and test materials, give rapid feedback, keep long-term records of progress, and perhaps work adaptively to match the different needs of users.
One promising system is LAPT (London Agreed Protocol for Teaching), developed originally at UCL with input from other London Medical Schools. This is a flexible PC-based system for delivering computer-based teaching. It has so far been used principally by medical and biomedical science students. As the exercises are simply written on a wordprocessor, new subject matter from a range of contributing colleagues is easily incorporated. A unique feature of LAPT is the introduction of confidence assessment into multiple-choice testing. After each answer, students are asked to declare a confidence level of 1, 2, or 3. For a correct answer, the marks gained are 1, 2, or 3. For an incorrect answer, the marks are 0, -2 or -6. The intention is encourage students to think realistically about the confidence they feel in their answers. We believe this scheme will be especially valuable when extended to auditory judgements. In our view, it is very important that those trained in phonetics should be able to form some estimate of the reliability of their perceptions (in terms of repeatability, likely consensus with other observers, and agreement with what instrumental analyses might be likely to show).
At present LAPT does not have facilities for outputting sound, and our intention is to marry it with the system developed by our colleague M. Huckvale which he has called CD-Mike. In this, a PC becomes a sophisticated player for an audio CD, any part of which can be randomly accessed using information in a stored `script' giving identification and timing of the contents of the disk. Advantages of this system are that the audio CD can be a relatively straightforward transfer from existing recorded material, while any new CD will also be usable as a stand-alone recording.
Conclusion
We have outlined the background to the development of Analytic
Listening, described some of its main features and indicated one
line of future work. It remains to put the technique into a broader
context.
Because it involves questions each with a clear perceptual focus, Analytic Listening opens the way to designing the practical phonetics syllabus around an explicit hierarchy of phonetic properties and parameters. Instead of being based upon a certain type of material (nonsense, RP, etc), or upon a fixed number of "sounds" to be covered , our syllabus can be based directly upon auditory skills (ability to make judgements concerning voicing, nasality, pitch, aspiration, and so on). For each programme, and for each course within the programmes, we must specify just what phonetic properties it is relevant to teach and to test at each stage. The Analytic Listening technique than gives us the means to train awareness of these properties, and to test them reliably and directly. There are possibilities for national and international comparisons between courses and examinations and even for standardisation of recognised tests.
© 1996 Michael Ashby, John Maidment and Evelyn Abberton
Back to SHL 9 Contents
Back to Phonetics and Linguistics Home Page
