Speech Processing by Computer

 

LAB 8

SIGNAL GENERATION FOR SYNTHESIS

 

In this lab session we will experiment with three different signal generation methods and compare the results.  We will use methods for prosody manipulation to change the pitch and timing of natural speech.  We will use a diphone concatenation system to produce synthetic copies of a natural utterance.  We will use a formant synthesis by rule system to produce a second set of synthetic copies.  We will then compare the different versions by listening to them.

 

Control file format

 

The prosody manipulation program repros, the MBROLA diphone synthesis system mbrsynth and the formant synthesis system phosynth all use the same control file format.  Control files are plain text files that can be created with Windows notepad.  The format of this file is as follows:

 

            • each line describes 1 phonetic segment

            • a line has three parts separated by spaces:

                        (i) the name of the phone

                        (ii) the duration of the phone in milliseconds

                        (iii) the required Fx contour through the segment

            • the phone names are in SAMPA format:

 

Consonants

p, b, t, d, k, g, tS, dZ, f, v, T, D, s, z, S, Z, h, m, n, N, l, r, w, j, 5 (=dark-l)

Vowels

i:, I, e, {, V, A:, Q, O:, U, u:, @, 3:, aI, OI, eI, @U, aU, e@, I@, U@

Silence

_

 

            • the Fx contour is specified as a list of pairs of numbers, each pair consists of

                        (i) the % duration of the segment at which the Fx is to be set

                        (ii) the Fx value at that point in Hz.

 

Thus the following control file will say "Mark":

 

      _ 100

      m 125 0 125

      A: 500 0 120 100 100

      k 250

      _ 100

 

Read this as: silence for 100ms, [m] lasts 125ms starting at 125Hz, [a:] lasts 500ms, starting at 120Hz, ending at 100Hz, [k] lasts 250ms, end with silence of 100ms.

 

 

1.      Record a natural utterance

a.       Design a short sentence (4/5 words max) with a mix of segments and an interesting prosody

b.      Record Speech and Lx at 16,000 samples/sec

c.       Generate Tx and Fx

d.      Save to file SENT.SFS

2.      Annotate natural utterance

a.       add SAMPA annotations to SENT.SFS

b.      remember to annotate silence as ‘_’

c.       let Mark check your annotations for correctness

3.      Generate .PHO control files

a.       choose Tools/Annotations/Export/Export as MBROLA

b.      save control file as NN.PHO

c.       open NN.PHO in notepad

d.      change the durations of segments to the values specified in the table overleaf.

e.       save as SN.PHO

f.        open NN.PHO

g.       change the pitch contour so that all specifications are deleted except for “0 150” on the first segment and “100 100” on the last segment.  This will cause a uniform fall in pitch from 150Hz to 100Hz over the utterance.

h.       save as NS.PHO

i.         open SN.PHO

j.        change the pitch contour again as in step g

k.      save as SS.PHO

4.      Generate variants of natural utterance

a.       select item 1.01 and choose Tools/Speech/Process/Prosody Change.  Use the SN.PHO control file

b.      select item 1.01 and choose Tools/Speech/Process/Prosody Change.  Use the NS.PHO control file

c.       select item 1.01 and choose Tools/Speech/Process/Prosody Change.  Use the SS.PHO control file

5.      Generate diphone versions

a.       choose Tools/Generate/MBROLA synthesis.  Use the NN.PHO control file and the “en1” database of diphones.

b.      choose Tools/Generate/MBROLA synthesis.  Use the SN.PHO control file and the “en1” database of diphones.

c.       choose Tools/Generate/MBROLA synthesis.  Use the NS.PHO control file and the “en1” database of diphones.

d.      choose Tools/Generate/MBROLA synthesis.  Use the SS.PHO control file and the “en1” database of diphones.

6.      Generate formant versions

a.       choose Tools/Generate/Synthesis by rule.  Use the NN.PHO control file.

b.      choose Tools/Synthesis Data/Synthesize speech to make a speech signal.

c.       choose Tools/Generate/Synthesis by rule.  Use the SN.PHO control file.

d.      choose Tools/Synthesis Data/Synthesize speech to make a speech signal.

e.       choose Tools/Generate/Synthesis by rule.  Use the NS.PHO control file.

f.        choose Tools/Synthesis Data/Synthesize speech to make a speech signal.

g.       choose Tools/Generate/Synthesis by rule.  Use the SS.PHO control file.

h.       choose Tools/Synthesis Data/Synthesize speech to make a speech signal.

7.      Compare versions

a.       you should have 12 different speech items:

 

1.01

Natural Speech

Natural Durations

Natural Pitch

1.02

 

Synthetic Durations

Natural Pitch

1.03

 

Natural Durations

Synthetic Pitch

1.04

 

Synthetic Durations

Synthetic Pitch

1.05

Diphone Synthesis

Natural Durations

Natural Pitch

1.06

 

Synthetic Durations

Natural Pitch

1.07

 

Natural Durations

Synthetic Pitch

1.08

 

Synthetic Durations

Synthetic Pitch

1.09

Formant Synthesis

Natural Durations

Natural Pitch

1.10

 

Synthetic Durations

Natural Pitch

1.11

 

Natural Durations

Synthetic Pitch

1.12

 

Synthetic Durations

Synthetic Pitch

 

b.      what is more important, natural durations or natural pitch?  Compare 2 & 3, 6 & 7, 10 & 11.

c.       does good prosody compensate for poor voice quality?  Compare 5 & 4, 9 & 8.

 

 

 

 

 

Table of Phoneme Durations in milliseconds

 

&

140

3:

240

5

80

@

90

@U

230

A:

220

D

100

I

100

I@

210

N

130

O:

210

OI

240

Q

140

S

180

T

160

U

100

U@

230

V

155

Z

70

_

100

aI

220

aU

240

b

115

d

75

dZ

170

e

125

e@

270

eI

230

f

130

g

90

h

160

i:

140

j

110

k

140

l

80

l~

80

m

110

n

130

p

130

r

80

s

125

t

130

tS

210

u:

155

v

85

w

80

z

140

{

140