Phonological Representations from
an Electrophysiological Perspective
Psycholinguistics Class #5 (contd.)
Outline
• Feasible Objectives for Neuroscience of Phonology
• Electrophysiology of Sound Processing
• Abstraction & Levels of Representation
– Phonetic vs. Phonological Categories
– Phonological Natural Classes
• Developmental Issues
• Phonotactics
The Big Questions
• What levels of acoustic/phonetic/phonological representation can
we distinguish in the brain?
• How are these representations created or modified during
development?
• What is the flow of information (in space and time) in the
mapping from acoustics to the lexicon in the brain?
• How does knowledge of native language categories and
phonotactics constrain perception?
• How are phonological representations encoded?
/kæt/
A Category
Another Category
3
III
/kæt/
Discrete Category
Representations
Gradient Category
Representations
The Big Questions
• What levels of acoustic/phonetic/phonological representation can
we distinguish in the brain?
• How are these representations created or modified during
development?
• What is the flow of information (in space and time) in the
mapping from acoustics to the lexicon in the brain?
• How does knowledge of native language categories and
phonotactics constrain perception?
• How are phonological representations encoded?
Sensory Maps
Internal representations of
the outside world. Cellular
neuroscience has discovered
a great deal in this area.
Vowel Space
Notions of sensory maps may be
applicable to human phonetic
representations…
…although attempts to find
them have had little success to
date.
Encoding of Symbols: Abstraction
• But most areas of linguistics (phonology, morphology,
syntax, semantics) are concerned with symbolic, abstract
representations,
...which do not involve internal representations of
dimensions of the outside world.
…hence, the notion of sensory maps does not get us very
far into language
/kæt/
Discrete Category
Representations
Gradient Category
Representations
Some Established Results
• Search for phonetic ‘maps’ in the brain: consistently uninformative
• Electrophysiology of speech perception has been dominated by
studies of the Mismatch Negativity (MMN), a response elicited in
auditory cortex 150-200ms after the onset of an oddball sound
• MMN amplitude tracks perceptual distance between standard and
deviant sound; i.e. measure of similarity along many dimensions
• There are established effects and non-effects of linguistic category
structure on the MMN
– non-effects in comparison of within/across category contrasts
– real effects in comparison of native/non-native contrasts
Some Burning Issues
• How abstract are the representations that the MMN can access, in
terms of
– levels of representation (e.g. underlying forms)
– phonological natural classes (i.e. features)
• Can electrophysiology provide evidence for preserved innate
phonetic representations?
• Can electrophysiological measures provide insights into
phonotactic processing?
Outline
• Feasible Objectives for Neuroscience of Phonology
• Electrophysiology of Sound Processing
• Abstraction & Levels of Representation
– Phonetic vs. Phonological Categories
– Phonological Natural Classes
• Developmental Issues
• Phonotactics
Electroencephalography (EEG)
Event-Related Potentials (ERPs)
John
is
laughing.
s1
s2
s3
pickup coil & SQUID
assembly
160 SQUID
whole-head
array
Brain Magnetic Fields (MEG)
SQUID detectors measure brain
magnetic fields around 100 billion
times weaker than earth’s steady
magnetic field.
Intensity of magnetic signal(T)
How small is the signal?
10
-4
10
-5
10
-6
10
-7
10
-8
10
-9
Earth field
EYE (retina)
Steady activity
Evoked activity
LUNGS
Magnetic contaminants
Urban noise
GI TRACK
Stimulus response
Magnetic contaminations
FETUS
Cardiogram
Contamination at lung
LIMBS
Steady ionic current
-10
10
-11
10
-12
10
-13
10
-14
MUSCLE
Under tension
Heart QRS
Fetal heart
Muscle
Spontaneous signal
(a-wave)
Signal from retina
Evoked signal
Intrinsic noise of SQUID
10
SPINAL COLUMN (neurons)
Evoked by sensory stimulation
HEART
Cardiogram (muscle)
Timing signals (His Purkinje system)
LIVER
Iron stores
10
BRAIN (neurons)
Spontaneous activity
Evoked by sensory stimulation
-15
Biomagnetism
Origin of the signal
MEG
EEG
scalp
B
V
recording
surface
orientation
of magnetic
field
skull
CSF
tissue
current
flow
- noninvasive measurement
- direct measurement.
Evoked Responses
150
140
L a te n c y (m s )
M100
130
120
110
• Elicited by any well-defined onset
100
0
1000
2000
3000
4000
5000
Frequency (Hz)
(Poeppel & Roberts 1996)
• Varies with tone frequency
• Varies with F1 of vowels
• May vary non-linearly with VOT
variation
(Poeppel, Phillips et al. 1997)
• Functional value of time-code unclear
• No evidence of higher-level
representations
(Phillips et al. 1995; Sharma & Dorman 1999)
Mismatch Response
X X X X X Y X X X X Y X X X X X X Y X X X Y X X X...
Mismatch Response
X X X X X Y X X X X Y X X X X X X Y X X X Y X X X...
Mismatch Response
X X X X X Y X X X X Y X X X X X X Y X X X Y X X X...
Latency: 150-250 msec.
Localization: Supratemporal auditory cortex
Many-to-one ratio between standards and deviants
Localization of Mismatch Response
(Phillips, Pellathy, Marantz et al., 2000)
Basic MMN elicitation
©
Risto Näätänen
Basic MMN elicitation
MMN
P300
Näätänen et al. 1978
MMN Amplitude Variation
Sams et al. 1985
How does MMN
latency, amplitude
vary with frequency
difference?
1000Hz tone std.
Tiitinen et al. 1994
Different Dimensions of Sounds
•
•
•
•
Length
Amplitude
Pitch
…you name it …
Amplitude of mismatch response can be
used as a measure of perceptual distance
Impetus for Language Studies
• If MMN amplitude is a measure of perceptual distance,
then perhaps it can be informative in domains where
acoustic and perceptual distance diverge…
Outline
• Feasible Objectives for Neuroscience of Phonology
• Electrophysiology of Sound Processing
• Abstraction & Levels of Representation
– Phonetic vs. Phonological Categories
– Phonological Natural Classes
• Developmental Issues
• Phonotactics
Perceptual Categories & MMN
(Phonetic)
Place of Articulation
• Acoustic variation: F2 & F3 transitions
Place of Articulation
[bæ]
[dæ]
• Acoustic variation: F2 & F3 transitions
Place of Articulation
within
category
between
category
[bæ]
[dæ]
• Acoustic variation: F2 & F3 transitions
Place of Articulation
within
category
between
category
[bæ]
[dæ]
• Acoustic variation: F2 & F3 transitions
Place of Articulation
• No effect of category boundary on
MMN amplitude (Sharma et al. 1993)
• Similar findings in Sams et al. (1991),
Maiste et al. (1995)
but…
Näätänen et al. (1997)
e e/ö ö õ o
Phonetic Category Effects
• Native/non-native vowel
prototypes (Näätänen et al.
1997; Winkler et al. 1999)
• French vs. Hindi contrasts
b-ddental-dretroflex, (DehaeneLambertz 1997)
• English d-t continuum
(Sharma & Dorman 1999)
????????
Phonetic Category Effects
• Measures of uneven discrimination profiles
• Findings are mixed
(…and techniques vary)
• Relies on assumption that effects of contrasts at multiple
levels are additive,
…plus the requirement that the additivity effect be strong
enough to yield a statistical interaction
/kæt/
Discrete Category
Representations
Gradient Category
Representations
Auditory Cortex Accesses Phonological
Categories: An MEG Mismatch Study
Colin Phillips, Tom Pellathy,
Alec Marantz, Elron Yellin, et al.
Journal of Cognitive Neuroscience, 2000
More Abstract Categories
• Phonetic categories show graded internal structure
• At the level of phonological categories, within-category
differences are irrelevant
• Aims
– use MMF to measure categorization rather than discrimination
– focus on failure to make category-internal distinctions
Voice Onset Time (VOT)
60 msec
Categorical Perception
Design
Fixed Design - Discrimination
20ms
40ms
60ms
Design
Fixed Design - Discrimination
20ms
0ms 8ms
16ms 24ms
Grouped Design - Categorization
40ms
60ms
40ms 48ms 56ms 64ms
Design
Fixed Design - Discrimination
20ms
0ms 8ms
16ms 24ms
Grouped Design - Categorization
40ms
60ms
40ms 48ms 56ms 64ms
Results
• 37-channel MEG recordings
• Sensors positioned above left
hemisphere auditory cortex
• 700:100 standard:deviant ratio
• Figure shows difference between
response to dæ-as-standard and
response to dæ-as-deviant
Preliminary Conclusion
• Auditory cortex generator of MMF accesses
representations that treat members of the same category as
identical
• No indication of what might be the form of these
representations, or where they might be stored
EEG Measures of Discrimination and
Categorization of Speech Sound Contrasts
Colin Phillips
Shani Abada
Daniel Garcia-Pedrosa
Nina Kazanina
Design
Fixed Design - Discrimination
20ms
0ms 8ms
16ms 24ms
Grouped Design - Categorization
40ms
60ms
40ms 48ms 56ms 64ms
Design
Fixed Design - Discrimination
20ms
0ms 8ms
16ms 24ms
Grouped Design - Categorization
40ms
60ms
40ms 48ms 56ms 64ms
32-channel EEG recordings
n = 24
Fully counterbalanced
Fixed Condition: between category
Electrode C3
Fixed Condition: within category
Electrode C3
Grouped Condition (d-t)
Electrode C3
Implication
• Presence of MMN in both within- and between-category
fixed conditions might suggest no access to linguistic
category representations
…could be interpreted as an acoustics-only effect
• …but presence of MMN in grouped condition indicates
access to linguistic categories
Discrimination and Categorization
of Vowels and Tones
Daniel Garcia-Pedrosa
Colin Phillips
Some Concerns
• Are the category effects an artifact:
– it is very hard to discriminate different members of the
same category on a voicing scale
– subjects are forming ad hoc groupings of sounds during
the experiment, and are not using their phonological
representations?
– does the ~30ms VOT boundary simply reflect a
fundamental neurophysiological timing constraint?
Vowels
• Vowels show categorical perception effects in
identification tasks
• …but vowels show much better discriminability of withincategory pairs
Vowels & Tones
• Synthetic /u/-/o/ continuum
• F1 varied, all else constant
• Amplitude envelope of F1
extracted for creation of
tone controls
Vowel, F1 = 310Hz
Pure Tone, 310Hz
• Pure tone continuum at F1
center frequency
• Matched to amplitude
envelope of vowel
Design
• Tones
300Hz
320Hz
340Hz
360Hz
400Hz
420Hz
440Hz 460Hz
• Vowels
– First formant (F1) varies along the same 290-470Hz
continuum
– F0, F2, voicing onset, etc. all remain constant
Results: Vowels
Results: Vowels
Results: Tones
Results: Tones
Preliminary conclusions
• MMN appears about 150ms post-stimulus in
vowel but not in tone condition
• Higher amplitude N100 for deviants in both
conditions. Is this evidence for categorization of
tones or just the result of habituation?
• Acoustic differences may be responsible for
greater N100, while categorization elicits the
MMN
Phonemic vs. Allophonic Contrasts
Nina Kazanina
Colin Phillips
in progress
• If we’re looking for category representations used to
encode lexical forms, more is needed…
• Need to distinguish surface and underlying categories
• English syllable-initial contrasts considered so far are not
sufficient
Cross-Language Differences
• Focus on meaning-relevant sound contrasts
Russian
d
t
Korean
d
t
-10
+20
Identification
D/T Identification (N=8)
1
0.8
% D
0.6
0.4
0.2
0
0
10
20
30
40
50
VOT
Perceptual boundary is much shorter than for English
d-t continuum.
Cross-Language Differences
• Focus on meaning-relevant sound contrasts
Russian
d
t
Korean
d
t
Cross-Language Differences
• Focus on meaning-relevant sound contrasts
Russian
d
t
Korean
d
t
…ada ada ada ada ada ada ata ada ada ada ata…
Phonological Features in Auditory Cortex
Colin Phillips
Tom Pellathy
Alec Marantz
Sound Groupings
(Phillips, Pellathy & Marantz 2000)
Phonological Features
Phonological Natural Classes exist because...
• Phonemes are composed of features - the
smallest building blocks of language
• Phonemes that share a feature form a natural
class
Effect of Feature-based organization observed in…
• Language development
• Language disorders
• Historical change
• Synchronic processes
Roman Jakobson, 1896-1982
Sound Groupings in the Brain
pæ, tæ, tæ, kæ, dæ, pæ, kæ, tæ, pæ, kæ, bæ, tæ...
(Phillips, Pellathy & Marantz 2000)
Sound Groupings in the Brain
pæ, tæ, tæ, kæ, dæ, pæ, kæ, tæ, pæ, kæ, bæ, tæ...
(Phillips, Pellathy & Marantz 2000)
Feature Mismatch: Stimuli
(Phillips, Pellathy & Marantz 2000)
Feature Mismatch
Design
(Phillips, Pellathy & Marantz 2000)
Sound Groupings in the Brain
pæ tæ tæ kæ dæ pæ kæ tæ pæ kæ bæ tæ ...
(Phillips, Pellathy & Marantz 2000)
Sound Groupings in the Brain
pæ tæ tæ kæ dæ pæ kæ tæ pæ kæ bæ tæ ...
– –
– –
[+voi]
(Phillips, Pellathy & Marantz 2000)
Sound Groupings in the Brain
pæ tæ tæ kæ dæ pæ kæ tæ pæ kæ bæ tæ ...
– –
– –
[+voi]
–
– –
– – [+voi] – …
(Phillips, Pellathy & Marantz 2000)
Sound Groupings in the Brain
pæ tæ tæ kæ dæ pæ kæ tæ pæ kæ bæ tæ ...
– –
– –
[+voi]
–
– –
– – [+voi] – …
• Voiceless phonemes are in many-to-one ratio with
[+voice] phonemes
• No other many-to-one ratio in this sequence
(Phillips, Pellathy & Marantz 2000)
Feature Mismatch
(Phillips, Pellathy & Marantz 2000)
Feature Mismatch
Left Hemisphere
Right Hemisphere
(Phillips, Pellathy & Marantz 2000)
Control Experiment - ‘Acoustic Condition’
• Identical acoustical variability
• No phonological many-to-one ratio
(Phillips, Pellathy & Marantz 2000)
Feature Mismatch
(Phillips, Pellathy & Marantz 2000)
Hemispheric Contrast in MMF
• Studies of acoustic and phonetic contrasts consistently
report bilateral mismatch responses
Paavilainen, Alho, Reinikainen et al. 1991; Näätänen & Alho, 1995;
Levänen, Ahonen, Hari et al. 1996; Alho, Winkler, Escera et al. 1998;
Ackermann, Lutzenberger & Hertrich, 1999; Opitz, Mecklinger, von
Cramon et al. 1999, etc., etc.
• Striking difference in our finding of a left-hemisphere
only mismatch response elicited by phonological feature
contrast
• Our studies probe a more abstract level of phonological
representation
/kæt/
Discrete Category
Representations
Gradient Category
Representations
Outline
• Feasible Objectives for Neuroscience of Phonology
• Electrophysiology of Sound Processing
• Abstraction & Levels of Representation
– Phonetic vs. Phonological Categories
– Phonological Natural Classes
• Developmental Issues
• Phonotactics
Universal Listeners
• Infants may be able to discriminate all
speech contrasts from the languages of the
world!
When does Change Occur?
• Hindi and Salish
contrasts tested
on English kids
Janet Werker
U. of British Columbia
6-12 Months: What Changes?
Structure Changing
Patricia Kuhl
U. of Washington
Place of Articulation
• Non-native continuum
b -- d -- D
• 3 contrasts
Native
b -- d
Non-native
d -- D
Non-phonetic b1 -- b5
• Conflicting results!
Place of Articulation
• Non-native continuum
b -- d -- D
• 3 contrasts
Native
b -- d
Non-native
d -- D
Non-phonetic b1 -- b5
• Conflicting results!
Dehaene-Lambertz 1997
Place of Articulation
• Non-native continuum
b -- d -- D
• 3 contrasts
Native
b -- d
Non-native
d -- D
Non-phonetic b1 -- b5
• Conflicting results!
Rivera-Gaxiola et al. 2000
Place of Articulation
• Non-native continuum
b -- d -- D
• 3 contrasts
Native
b -- d
Non-native
d -- D
Non-phonetic b1 -- b5
• Conflicting results!
Tsui et al. 2000
Outline
• Feasible Objectives for Neuroscience of Phonology
• Electrophysiology of Sound Processing
• Abstraction & Levels of Representation
– Phonetic vs. Phonological Categories
– Phonological Natural Classes
• Developmental Issues
• Phonotactics
Phonology - Syllables
• Japanese versus French
• Pairs like “egma” and “eguma”
• Difference is possible in French, but not in
Japanese
Behavioral Results
• Japanese have difficulty hearing the difference
Dupoux et al. 1999
EXECTIVE SUITE
ERP Results
• Sequences: egma, egma, egma, egma, eguma
• French have 3 mismatch responses
– Early, middle, late
• Japanese only have late response
Dehaene-Lambertz et al. 2000
ERP Results - 2
• Early response
Dehaene-Lambertz et al. 2000
ERP Results - 3
• Middle response
Dehaene-Lambertz et al. 2000
ERP Results - 4
• Late response
Dehaene-Lambertz et al. 2000
Implications
• Cross-language contrast in MMN mirrors behavioral
contrast
• Relative timing of responses that are same and different
across French & Japanese is surprising from a bottom-up
view of analysis - suggests a dual route
• MMN elicited by non-initial contrast is great news!
• Is this effect specific to comparison in an XXXXY task?
• Is the result robust; does it generalize to other phonotactic
generalizations?
Cross-language Differences
Thai speakers:
Thai *words*:
[da]
[ta]
English *words*:
[daz]
[taz]
DIFFERENT
SAME
Behavioral Result: Patcharee Imsri & Bill Idsardi (2002)
MEG Study in Progress: Imsri, Phillips, & Idsardi
Conclusion
• Electrophysiology is beginning to provide tools that could
be valuable in phonological investigations
• Less abstract, gradient representations: possible to address
questions relating to perceptual distance, similarity, etc.
…but tools need fine-tuning
• More abstract, discrete representations: feasible to identify
and isolate these
…but unlikely to help us understand the form of these
representations
/kæt/
Discrete Category
Representations
Gradient Category
Representations
Some Burning Issues
• How abstract are the representations that the MMN can access, in
terms of
– levels of representation (e.g. underlying forms)
– phonological natural classes (i.e. features)
• Can electrophysiology provide evidence for preserved innate
phonetic representations?
• Can electrophysiological measures provide insights into
phonotactic processing?
Descargar

Phonological Representations from an Electrophysiological