Automatic and Data Driven
Pitch Contour Manipulation
with Functional Data Analysis
Michele Gubian, Lou Boves
Radboud University Nijmegen
Nijmegen, The Netherlands
Francesco Cangemi
Laboratoire Parole et Langage
University of Provence, Aix-en-Provence, France
Outline



Pitch Contour Manipulation

Context and problem

Sketch of proposed approach
Use of Functional Data Analysis (FDA)

Case study

Data preparation

Functional PCA

Functional synthesis and listening
Conclusions
2
Context

Languages can express oppositions using intonation

Question/Statement opposition in Neapolitan Italian
QUESTION
STATEMENT
“Milena lo vuole amaro (?)” = Milena drinks it (her coffee) bitter (?)

What are the intonation cues that listeners use?

Perceptual experiments where listeners judge stimuli
whose pitch (F0) contour has been manipulated

STEP 1: extract pitch contours from speech data

STEP 2: modify pitch contours

STEP 3: re-synthesize speech
3
Pitch Contour Manipulation
F0

Use of an intonation model


Stylization
Manual changes
time
POSSIBLE IMPROVEMENTS


Handle dynamic detail

Locally (e.g. concavity/convexity)

Long range correlation
Derive useful variation modes directly and
automatically from data
4
A data driven approach
Functional
Data
Analysis
x
5
Question/Statement opposition
in Neapolitan Italian
DATA

2 male speakers

3 carrier sentences (read speech)

“Milena lo vuole amaro (?)” = Milena drinks it (her coffee) bitter (?)

“Valeria viene alle nove (?)” = Valeria arrives at 9 (?)

“Amelia dorme da nonna (?)” = Amelia sleeps at grandma’s (?)

2 modalities = Q / S

5 repetitions

2 x 3 x 2 x 5 - 3 discarded = 57 utterances
6
Data Preparation

Sampled F0 curves have to be turned into functions

A basis of functions (B-splines) expresses each original curve

Decide how much detail to retain (smoothing)
7
Data Preparation (2)

Landmark registration

Align points in time that are deemed as having the same
meaning across the dataset
8
Classic
Principal Component Analysis (PCA)
PC1
x
x
x x x
x
x xx
xxx xx x x
x x
xx
x
x xx x
x
x x
x
x
x
x
x
x
x
PC2
salary
25
65
age
9
Functional PCA
10
PC-based signal reconstruction
+ 1.65 x
mean(t)
- 0.46 x
PC1(t)
PC2(t)
11
Manipulated stimuli
12
Conclusions

A data driven approach is possible in the exploration of
intonation phenomena

FDA provides automatic tools to describe variation in a set of
pitch contours extracted from real utterances


provided that the relevant landmarks are annotated
The same tools allow to construct artificial contours with
desired perceptual characteristics

Smooth and global variation are applied

Variations come from a statistical analysis of data

The process is automatic
13
14
Descargar

Reductions in the Articulatory Features domain