ProSynth (2).
Notes/text from RAO.

I’m sending everything to everyone: this is in the hope that it’ll make it easier for things not to be repeated.

Bold means comments from us. Italics is material quoted from EPSRC’s documentation. Strikethrough for words that could be removed.
Track record: Jill to put together.
1.3 York (John Local, Richard Ogden, Paul Carter)
John Local has a long history of research in spoken language and worked 1988-1996, and Richard Ogden 1990-1996, on development of YorkTalk. The success of this structure-based approach, with its emphasis on constraint satisfaction rather than arbitrary rules informed and motivated the previous ProSynth proposal. Paul Carter has experience of experimental phonetics and a wide range of instrumental techniques. He has played a vital role in ProSynth, elaborating the labelling system and implementing regression-based statistical models of duration using knowledge of high-level linguistic structure. All three have presented work on the relationship of phonological structure to detailed phonetic content at major international conferences. Work at York has focused on the need for integration of linguistic levels. We have concentrated on developing formal phonological representations for which phonetic interpretations can be stated. York hosted the Sixth International Conference in Laboratory Phonology in July 1998, a recognition of our contribution to the field. Funded work has included: 
(a) British Telecom: Collaborative work on non-segmental speech synthesis, 1988-94 (£233,000). We also acted as linguistic consultants to BT, developing grapheme-to-phoneme rules for British and American English, and an algorithm for stress placement. Outcome: the YorkTalk model of speech synthesis. Some of the linguistic knowledge incorporated in YorkTalk was used by BT in the construction of the LAUREATE system, thus contributing to UK competitiveness.
(b) British Telecom: Generating spoken language statistics from a non-segmental phonological model. Research for development of a test-signal for non-linear communcation devices. P.I. John Local. 1994 (£30,000).
(c) British Council: British-German Acadmic Research Collaboration for work on the phonetics of rhythmic and prosodic systems and the conduct of conversation. 1996 (£3,000). (John Local.)
(d) ESRC Grant R000221880, 1996-7 (£28,920): A declarative account of deletion phenomena in English phonetics and phonology, P.I. Richard Ogden. Based on analysis of MARSEC corpus. Outcome: a declarative, polysystemic analysis of apparent deletion phenomena in the function-word systems of English (Ogden 1999, to appear). Findings from this study will contribute to the present project.
XX Jill: include Sarah in the following?:
cb John Local has collaborated with Ken Stevens and his team at Sensimetrics in the development of the Hlsyn quasi-articulatory synthesis system. We have at York a demonstrated ability to combine phonological analysis with detailed phonetic interpretation, allowing for the generation of natural-sounding formant synthesis and improved timing of diphone synthesis. 
Background: Sarah
Needs to “demonstrate an understanding of past and current work in the subject area both in the UK and abroad”.
RAO: I suggest we use some material from eg. the CSL paper, which is based on the original grant proposal; the background hasn’t changed that much. Perhaps the extract below is helpful? There is also a lot in the early part of section 2 of that paper which might be useful. Suggested edits (to save space) are marked.

Speech synthesised by rule has yet to make a significant impact as an output channel for information systems, despite continued engineering advances in text to speech (TTS) systems. A recurrent complaint is the perceived ‘unnatural’ quality of the synthetic speech: that the speech does not sound as if it could have been produced by a human speaker. Such problems persist despite improvements in textual analysis, pronunciation and signal generation. For example: although the use of a large corpus of recorded speech for polyphone concatenation has produced signals with sections with a highly natural voice quality, utterances still exhibit disfluencies, broken rhythm and lack of coherence.  Contemporary synthetic speech still suffers from unexpressive and often inappropriate prosody, and from poorly modelled coarticulation. These failings arise from the poverty of the linguistic representation underlying the utterance to be produced, as well as a fundamental lack of attention to the systematic fine detail in human production—fine detail that listeners expect and also utilise when listening in noise.
Segmental intelligibility data illustrate the scale of the problem. When heard in noise, most synthetic speech loses intelligibility much faster than natural speech: natural speech is about 15% less intelligible at 0 dB s/n ratio than in quiet, whereas for isolated words or syllables, Pratt (1986) reported that the intelligibility of typical synthetic speech drops by 35%-50%. We can expect similar results today. Concatenated natural speech avoids those problems related solely to voice quality and local segment boundaries, but suffers just as much from poor models of timing, intonation, and systematic variation in segmental quality that is dependent on word and rhythmical structure. Even when the grammatical analysis is right, one string of words can sound good, while another with the same grammatical pattern does not. 
ProSynth is an integrated prosodic (i.e. structure-based) approach to speech synthesis. At its core is a phonological model which allows for structurally important distinctions to be made, even when the phonetic effect of these distinctions is subtle. The phonological model in ProSynth draws together insights from current theories of phonology, and makes it easier to model phonetic and perceptual effects. Recent research in computational phonology (e.g. Bird, 1995) combines highly structured linguistic representations (more technically, signs) with a declarative, computationally tractable formalism. Recent research in phonetics (e.g. Simpson, 1992; Hawkins & Slater, 1994; Manuel, 1995; Zsiga, 1995) shows that speech is rich in non-phonemic information which contributes to its naturalness and robustness. Other work (Local, 1992; 1993; 1995a & b; Ogden, 1992; Local & Ogden, 1997) has shown that it is possible to combine phonological with phonetic knowledge by means of a process known as phonetic interpretation: the assignment of phonetic parameters to pieces of phonological structure. All these strands of work have contributed to the phonological model which ProSynth uses. By mimicking as far as possible the systematic spectral, temporal and intonational detail which is observable in natural speech, we aim to improve the intelligibility of synthetic speech. 
Add something here about how we want to expand ProSynth. (See suggested York text below.)
Why York are interested in function words.
Function words such as prepositions, determiners and auxiliaries, comprise about 50% of all the words of spoken English. 
TTS systems generally model function words in the same way as they model other unstressed syllables. A variety of studies (Fosler-Lussier& Morgan, 1998; Keating 1997 et al. 1994) have shown that function words are phonetically different from open-class items such as nouns, verbs and adjectives. Word frequency and predictability are known to have implications for phonetic detail (van Bergem 1993; Wright, to appear), such that more frequent and more easily predicted words contain more reduced sounds. The systems of phonological contrast are smaller in function words, and this gives rise to the possibility that there may be more systematic variability in the phonetic interpretation of the items in those systems. Given that function words are so frequent and are so phonetically complex, modelling systematic temporal and spectral details pertinent to them in synthetic speech should be expected to increase the percept of naturalness.
Programme and methodology
“Identify the overall aims of the project and the individual measurable objectives against which you would wish the outcome of the work to be assessed.”
“Describe the programme of work, indicating the research to be undertaken and the milestones that can be used to measure its progress. The detail should be sufficient to indicate the programme of work for each member of the research team. Explain how the project will be managed.”
Linguistic representation and structures: MARK
A couple of examples
Word-final bilabial nasals do not assimilate their place of articulation in open-class items, so that lime green cannot be pronounced li[N]e green. On the other hand, the [m] of I’m frequently assimilates to other places of articulation, as in I’[N] going.
The phoneme /D/ has a very limited distribution in English. It only occurs word-initially in function words such as the, this, and they. Despite being classified as a “voiced dental fricative”, it has a range of realisations even in the most careful of speech. These realisations include a dental stop, a dental approximant, a dental nasal or even a dental lateral. These realisations are unique to /D/ and a similar range of variability is not observed for /T/. 
Close front vowels can be heard in even careful speech as the result of labilality spreading in the case of was: /Syz/ for she was and /Oz/ for it was are relatively common
Implications for “informational richness” and “naturalness”
Ogden (1999) has argued that some of the systematic phonetic variability of English is specific to a particular phonological system. Part of speakers’ knowledge of English includes knowledge about the system-specific details of phonetic realisation (or interpretation). Thus the systematic phonetic details which pertain to function words may contribute information about linguistic structure.
A formal model of function word systems.
Fundamental to ProSynth is the representation of linguistic structures. Currently, the model captures two types of relation: hierarchical phonological relations (syllable, foot, accent group, intonational phrase) and simple syntactic relations. The basic phonological structure of ProSynth is <attribute, value> pairs. To allow for more sensitive phonetic interpretation which will capture the kinds of systematic phonetic variability observable in natural speech, it is necessary to develop further an explicit linguistic model of the relations that exist between systems of available contrast at particular places in structure. 
Some of the necessary formal work has already been done (Ogden 1999). 
Methodology
This methodology builds on the success of the current ProSynth model.
This is in chronological order (more or less): might be useful for working out the work packages and milestones.
• Design and record a database (5 speakers) containing a representative set of function words in a variety of metrical contexts. 
Measureable objective: an elaborated parametric labelling system; a labelled database containing function words in a systematically varied range of structures.
• Conduct an analysis of the material in the database using and expanding techniques already established for the current database. These include regression tree analysis, alongside other well-established linear statistical techniques. Establish patterns of systematic variability. 
Measureable objective: a set of phonetic descriptions of function words, including descriptions of ranges of variability.
• Modify the parser so as to (1) identify function words (2) generate appropriate linguistic structures for them. 
Measureable objective: a parser capable of generating phonological structures which integrate syntactic and phonological constraints.
• Establish statements of phonetic interpretation which model the appropriate temporal and spectral details. 
Measureable objective: a device-independent set of phonetic interpretation statements which can be implemented in eg. HLsyn and/or MBROLA.
• Perceptual tests. 
Measureable objective: improved intelligiblity under adverse listening conditions. Greater ‘naturalness’ of synthetic speech.
Database: JOHN
Please look at the comments on Linguistic Data Consortium below. They might have a database we could use.
A rough specification of what York would like in the database.
The database sentences will use a museum context. This will allow the material to be framed in an appropriate discourse context, which will facilitate both production and testing.
Material for each of the datasets will be gathered from five speakers of the same variety (2 female, 3 male), but one speaker (male) will be common to all the datasets for comparison and in order to produce a speech signal that is coherent/sounds like it comes from one speaker. Recording to be done at UCL?
The database material will allow systematic investigation of the phonetic interpretation of the function word system, such as the spectral and durational effects of /D/ (eg. in a vs. in the), the quality of schwa (in a pit vs. in a pot), and systematic parametric reorganisation (eg. she was /Si w@z/ and /Syz/). Attention will be focussed initially on prepositions, determiners and Pronoun+Auxiliary structures. The required contrasts will be placed in a systematically varied range of structures. This will allow for the analysis and modelling of rhythmical, temporal, spectral and intonational effects in a discourse-sensitive manner.
Timeliness and novelty: JOHN
Still to be written. Rough contents: relevance of more kinds of linguistic knowledge in synthesis; novelty of this eg. function words not yet worked on by anyone as a separate system; integration of prosodic knowledge as a whole; to relate phonetics to both grammar and discourse; emphasises results of perceptual tests. SUGGESTIONS WELCOME!
Relevance to beneficiaries: JOHN
The proposed research will increase our expertise and experience in the increasingly important area of speech technology. It will be of benefit to companies and organisations with an interest in speech synthesis and speech technologies. Improvements in the naturalness and variety of synthetically generated speech will greatly increase the applicability of this technology. Commercial applications for high-quality, natural sounding speech synthesis include the next generation of dialogue systems, information enquiry services and machine translation systems. The work will also increase the research community's expertise and experience in the increasingly important domain of speech technology: developer of TTS systems will benefit from the databases, statistical results and prosodic knowledge obtained. The testing of the linguistic model will also contribute to the development of linguistic theory. Further beneficiaries could include users of speech protheses.
Dissemination and exploitation RICHARD
“Indicate the proposed dissemination and technology transfer routes and explain how the transfer of knowledge will take place to both beneficiaries and the general public; 
Identify the mechanisms in place for identification, protection and subsequent exploitation of any exploitable results which may arise from the research (including details of any specific collaborative agreement, where relevant).”
Interim and final results will be disseminated over the web, through conference presentations and their published proceedings (eg. International Conference on Speech and Language Processing; Eurospeech, International Congress of Phonetic Sciences; miscellaneous European Speech Communication Association and domestic workshops), and EPSRC reports. Papers will be submitted to refereed journals e.g. Computer Speech and Language, J. Phonetics, and J. Acoustical Society of America.
Justification of resources RICHARD/JILL co-ordinate: but see below!
NB: If we remember right, we all did our own “justification” in the last proposal, because we all asked for different things. If this is right, then each site needs to work out its own costings and justification.
The RAs appointed to this project should be at post-doctoral level, since they all require a combination of advanced programming skills and linguistic knowledge. The York RA requires advanced skills in declarative and/or computational phonology, preferably combined with a training in phonetics.
Equipment. An Apple PowerBook is requested for portability of demonstrations, running software for acoustic analysis, writing up papers for dissemination. The York site uses predominantly Macintoshes and has support for them. 
Consumables. The DAT recorder and CD-ROM writer and CD-ROMs for data storage are needed for the database recording and analysis. A supported network connection is requested.
Travel. The conferences listed are directly relevant to the project. ICPhS (Barcelona) is the premier experimental phonetics forum; Eurospeech is an important venue for work in speech technology; ICSLP is a major international conference for IT, speech and language. BAAP provides the best national forum for phonetics research. Attendance at these meetings will benefit the project both because of the opportunity to publish and disseminate our results, and because of the valuable contacts with researchers in related fields.
Budget
Travel:
ICPhS, Barcelona 2003: air fare £300, conf. fees £200, accom. £220 x2 people = £1440
Eurospeech x 2 (1 person each time): Ditto. £1440.
ICSLP x 2 (1 person each time, assume intercontinental): air fare £500, conf. fees £200, accom. £220 = £1840
Domestic conferences/workshops 1/year @ £500/each = £1500
Travel for project meetings 4/year, £120 each, 3 people, 3 years = £4320
Total for travel = £10540.
LDC looks as if it has useful databases we should be accessing to bulk out our own work. What do you think? See costs below; previous reviewers’ comments on not creating databases might still be relevant. Also mention MARSEC?
Linguistic Data Consortium membership 3 years @ US$2,000 /year
References
Bergem, Dick R van (1993). Acoustic vowel reduction as a function of sentence accent, word stress and word class. Speech Communication 12, 1-23.
Fosler-Lussier, Eric and Nelson Morgan (1998). Effects of speaking rate and word frequency on conversational pronunciations. In ESCA Tutorial and Research Workshop on Modeling pronunciation variation for automatic speech recognition.
Keating, Patricia A., Dani Byrd, Edward Flemming, and Yuichi Todaka (1994). Phonetic Analyses of word and segment variation using the TIMIT corpus of American English. Speech Communication 14, 131-142.
Ogden, Richard (to appear). A declarative account of strong and weak auxiliaries in English. Phonology 16.
Wright, Richard (to appear).