As promised our remarks on testing. I hope you don't mind that its just a
list of notes and still far from perfect, but we assumed you wouldn't like
to wait that long.
Sebastian
************************************************************************
General questions:
1) What kind of test will we need?
(Assessment vs. theory testing)
CAM: compare presence and absence of spectral detail
YRK: compare pairs of words with phrases whose timing differs due to
gram./phonol. differences that most theories ignore.
UCL: assess how good intonation is or compare presence/absence of
something (pls. specify which)
Cambridge Suggestions for perceptual tests
2) testing temporal modelling
Richard Ogden suggested to test timing of
2nd syll in 2-syll-feet heavy vs. light, e.g.:
(sister vs. sitter)
(beater vs. better)
(windy vs. whinny)
Any other suggestions ?
3) spectral effects (vs.temporal effects)
We want to test coarticulation effects, within a syllable like the
ones described in Hawkins and Nguyen 1998, between adjacent syllables
like Tunley thesis work, or even longer range. These effects don't
seem to be widespread but when they occur they seem to be important.
(we need to identify some)
domains ? (syllable, foot??)
e.g.
within syllable:
coda voicing determines temporal and possible
spectral quality of sonorant consonants at beginning
of syllable
adjacent syllables:
schwa varies depending on the following vowels
possible long range:
r vs l can affect formant frequencies of espiecially
unstressed vowels in non adjacent syllables
(with no intervening blocking consonants)
coarticulation should vary with
a) type of consonants between influencing parts
(blocker vs non-blocker) e.g. :
some 'nerdy `hens (+blk) [jh2133]
some 'nervy `hens (-blk) [jh2134]
some 'nerdy `wrens (+blk) [jh2135]
some 'nervy `wrens (-blk) [jh2136]
b) prosodic structure
UCL and YRK could say what structures they want here. CAM could
fit their segmental sequences into them (constraint: we need
multi-feet structures with succesive weak syllables to have the
long range for the possible long ranging effects)
c) constituent boundaries
(i.e. word or foot boundaries)
see b) above
4) York & UCL doing f0 + timing interactions
What sort of things do you want tested ?
bear in mind that next stage = rate changes
e.g. f0 in long /n/: slow "-\_" fast "-\" ??
Can you (do you want to) use material for this test that can
easily be used in rate change tests too ?
5) How can we test the interaction of the ProSynth?
black box vs glass box
i.e. a) general system output is assessed
b) modules tested for themselves
6) Do we need new recordings now or later?
The obvious suggestion is:
record standard test material for a general speech synthesis
assesment (e.g. Diagnostic Rhyme Test,
Harvard Psychoacoustic Sentences,
Haskins Anomalous Sentences,
Minimal Pairs Intelligibility Test)
Example from the MPI sentences:
The horrid courts scorch a revolution
The horrid courts score a revolution
The advantage of having standardized material would
of course lie in the ease of comparison between our approach
and other speech synthesis systems.
Minimal pairs sentences (like the MPI's) would also give
us a way of testing long distance influences. We might show
that the right modelling just of the surrounding phrase
could enhance the chance to identify the target word.
BUT: the std. systems don't systematically test the types of
structures we're currently working on (some might be in the
test material but only by chance). Does this matter? If it does,
do we want to make up our own material with one/some of the
std. techniques.
We suggest that,
as far as possible, we follow ESCA/COVOSDA Multi-lingual TTS Synth
Evaluation procedures. CAM to make final decision after the
workshop.
*********************************************************************
Sebastian Heid Email: sh276@cam.ac.uk
Phonetics Laboratory Phone: +44 1223 33 50 50
Dept. of Linguistics Fax: +44 1223 33 50 53
University of Cambridge
Sidgwick Avenue
Cambridge CB3 9DA
United Kingdom
*********************************************************************