Topics of Research
 Evolution of communication
 Evolution of signalling systems
 Lexicon formation
 Grounding
 Language diversity
 Emergence of grammar
Modelling language origins and evolution
IJCAI-05
1
For each topic
 What it is. The main research question(s).
 Achievements, examples, used techniques.
 Open questions.
Modelling language origins and evolution
IJCAI-05
2
Abbreviations of techniques
 BE Baldwin Effect
 CM Competition Models
 DS Dynamical Systems
 GA Genetic Algorithms
 GT Game Theory
 HC Hill Climbing
 ILM Iterated Learning
Model
 MBL Memory-Based
Learning
 MM Mathematical
Modelling
 NN Neural Networks
 RNN Recurrent Neural
Networks
 ROB Robotics
 LA Language Acquisition
 LG Language games
Modelling language origins and evolution
IJCAI-05
3
Topics of Research
 Evolution of communication
 Evolution of signalling systems
 Lexicon formation
 Grounding
 Language diversity
 Emergence of grammar
Modelling language origins and evolution
IJCAI-05
4
Evolution of communication
Marilyn Monroes (Key West, Florida, 1995)
by Peter Krogh (Nat. Geographic)
 How can communication as such arise as an evolutionary
advantageous strategy?
 What ecological pressures could have caused
communication to have emerged?
Based on biological theories, e.g., (Seyfart et al. 1980, Grafen
1990, Krebs & Dawkins 1984, Zahavi 1975, 1977)
Modelling language origins and evolution
IJCAI-05
5
Examples
Evolution of alarm calls (de Jong 1999).
5 agents, 3 types of predators.
Input: own location, type of predator (if any)
Actions: move 1 horizontal step & and go to a
hiding place (vertical locations)
 90% predators visible, 10% not detected to
an individual  communication may help




eagle
Modelling language origins and evolution
IJCAI-05
6
Examples
Evolution of alarm calls (de Jong 1999).
5 agents, 3 types of predators.
Input: own location, type of predator (if any)
Actions: move 1 horizontal step & and go to a
hiding place (vertical locations)
 90% predators visible, 10% not detected to
an individual  communication may help




Modelling language origins and evolution
IJCAI-05
7
Examples
 Evolving communication to
– enhance cooperation.
• Mate finding, GA (Werner & Dyer 1991),
• Predator-prey simulation, GA (MacLennan & Burghardt
1993)
• Altruistic behaviour, GA & GT (Di Paolo 2000)
• Alarm calls LG (De Jong 2000)
• Handicap principle GA (Bullock 1998)
– enhance competition
• Contests GA (Noble 2000)
Modelling language origins and evolution
IJCAI-05
8
Open issues
 What ecological pressures or cognitive factors
could have facilitated the transition from
using iconic to symbolic communication
systems?
Modelling language origins and evolution
IJCAI-05
9
Topics of Research
 Evolution of communication
 Evolution of signalling systems
 Lexicon formation
 Grounding
 Language diversity
 Emergence of grammar
Modelling language origins and evolution
IJCAI-05
10
Evolution of signalling systems
 How can communication
channels and sound
systems evolve?
– How can sensory-motor
systems evolve that are used
in communication?
– How can particular sound
systems evolve?
Based on phonetic theories &
findings such as (Lindblom et
al. 1984, Lindblom & Madieson
1988, Madieson 1984)
Modelling language origins and evolution
IJCAI-05
11
Examples
 Communication channels:
– Evolving communication without
dedicated communication
channels ROB & GA (Quinn 2001)
2 Robots: Khepera, 2wheels, IR proximity sensors
Task: both robots have to move as far as possible while
remaining at close distance
(i)
(iii)
(ii)
Modelling language origins and evolution
IJCAI-05
(iv)
12
Examples
 Sound systems:
– Vowel systems LG (De Boer 1997; 2000), GA
(Glotin 1995; Berrah et al. 1996)
– Syllable systems GA (Redford et al. 2001), LG
(Oudeyer 2001)
– Phonemic coding LG (De Boer & Zuidema 2003;
Oudeyer 2002)
Modelling language origins and evolution
IJCAI-05
13
Open issues
 Adaptation of (human) vocal tract, auditory
system and their connection.
 Evolution of complex utterances and
consonants.
Modelling language origins and evolution
IJCAI-05
14
Topics of Research
 Evolution of communication
 Evolution of signalling systems
 Lexicon formation
 Grounding
 Language diversity
 Emergence of grammar
Modelling language origins and evolution
IJCAI-05
15
Lexicon formation
 How can a shared vocabulary
emerge in a population?
– Invention
– Language acquisition
Based on language acquisition
literature, such as, e.g., (Clark
1993, Markman…, Tomasello &
Barton 1994).
Modelling language origins and evolution
IJCAI-05
16
Basic approach
m1
m2
m3
m1
m2
m3
w1
0.2
0.2
0.3
w1
1.0
0.1
0.0
w2
0.3
0.2
0.4
w2
0.2
1.0
0.4
w3
0.1
0.2
0.3
w3
0.1
0.0
1.0
m1
m2
m3
m1
m2
m3
w1
0.3
0.1
0.3
w1
1.0
0.0
0.0
w2
0.2
0.2
0.1
w2
0.1
0.9
0.1
w3
0.3
0.2
0.2
w3
0.1
0.1
1.0
Modelling language origins and evolution
IJCAI-05
17
Examples
 Evolution of the Saussurean sign (Hurford 1989)
 Searching for evolutionary good learning strategies
 3 strategies:
Strategy
Input from adult
population
Learner’s
acquired
behaviour
Imitator
Transmission
Reception
Transmission’
Reception’
Calculator
Transmission
Reception
Transmission’
Reception’
Saussurean
Transmission
Transmission’
Reception’
Modelling language origins and evolution
IJCAI-05
18
Examples
 Learning strategies
– Saussurean sign GA (Hurford 1989)
– Obverter learning LG (Batali & Oliphant 1996;
Oliphant 1998)
– Learning biases ILM, NN, GA (K. Smith 2004)
– Joint attention vs. corrective feedback vs. crosssituational learning LG (Vogt & Coumans 2003)
 Interaction strategies
– Language games LG (Steels 1996)
Modelling language origins and evolution
IJCAI-05
19
Open issues
 How can we scale up to realistic lexicon sizes
and population sizes?
 What learning biases have evolved and how?
 How have interaction strategies evolved?
 strategies for producing utterances, listening and
turn taking
Modelling language origins and evolution
IJCAI-05
20
Topics of Research
 Evolution of communication
 Evolution of signalling systems
 Lexicon formation
 Grounding
 Language diversity
 Emergence of grammar
Modelling language origins and evolution
IJCAI-05
21
Grounding
 How can individual agents
use, construct, interpret and
share symbolic
communication
meaningfully?
– Intentionality (Brentano 1874)
or Symbol grounding problem
(Harnad 1990)
Take inspiration from, e.g.,
(Deacon 1997, Searle 1980,
Peirce 1931 , Wittgenstein
1967, Lakoff 1987, Langacker
1987, Clark 1993, Tomasello
1999)
Modelling language origins and evolution
IJCAI-05
From: Pfeifer & Scheier 1999
22
Talking Heads
(Belpaeme et al. 1998; Steels et al. 2002)
 Setup with two cameras
on a tripod.
 Each camera resembles
a Talking Head.
 PowerMac for
processing
 Environment:
geometrical figures on
white-board.
 Experiment: Language
evolution on the
Internet (largely
uncontrolled, because
interaction with human
users)
Modelling language origins and evolution
IJCAI-05
23
Talking Heads
(Belpaeme et al. 1998; Steels et al. 2002)
Evolution of the word-form “wogglesplat” over 90,000 games.
Modelling language origins and evolution
IJCAI-05
24
Examples
 Lexicon grounding
– Mushroom world NN, GA (Cangelosi et al.
2000)
– Naming
• Mobile robots LG, ROB (Steels & Vogt 1997; Vogt
2000) LG, ROB, NN (Billard & Dautenhahn 1999)
• Talking Heads LG, ROB (Belpaeme et al. 1998;
Steels et al. 2002)
• Cross-situational learning LG (A.D.M. Smith
2003; Vogt 2003)
– Survival task LG, ROB (Vogt 2002)
– Emergence of colour categories LG, GA
(Belpaeme & Steels, BBS In press)
Modelling language origins and evolution
IJCAI-05
25
Examples
 Grounding grammar
– Construction grammars LG (Steels 2004)
– Compositionality LG, ILM (Vogt 2005)
– Verbs and nouns GA, ROB (Cangelosi &
Parisi 2001; Marocco et al. 2003)
Modelling language origins and evolution
IJCAI-05
26
Open issues
 How can ‘real meaningful’ communication
emerge in a realistic task environment?
 Emergence of theory of mind or other
intention reading skills.
 Emergence of most linguistic aspects, such as
verbs, case-systems, abstract concepts,
function words, time, etc., largely unexplored
(let alone understood).
Modelling language origins and evolution
IJCAI-05
27
Topics of Research
 Evolution of communication
 Evolution of signalling systems
 Lexicon formation
 Grounding
 Language diversity
 Emergence of grammar
Modelling language origins and evolution
IJCAI-05
28
Language diversity & change
 What are the
conditions that
make languages so
diverse?
–
–
–
–
Dialects
Languages
Language contact
Language change
Based on findings and theories from, e.g., (Dunbar 1996,
Crystal 1987, Labov 1972, Chambers 1995)
Modelling language origins and evolution
IJCAI-05
29
Examples
Social Impact Theory (Latané 1981) models of language
change (Nettle 1999a; 1999b)
 Impact variant p: ip=bp Npa [(si/di2)/Np]
 Impact variant q: iq=bq Nqa [(si/di2)/Nq]
– bp/q is a constant,
– Np/q is nr. of agents speaking p or q,
– a non-linear adoption factor (if linear, all agents will end
up speaking the dominant variant).
– [(si/di2)/Np/q] average impact of variant p or q.
 Learner adopts p if ip>iq and q if iq>ip
 Mutation rate – probability that the above rule is properly
used. (social distance factor)
 Initial population has variant p.
Modelling language origins and evolution
IJCAI-05
30
Examples
Taken from (Nettle 1999b)
Modelling language origins and evolution
IJCAI-05
31
Examples (linguistic diversity)
 Social structures
– Social impact theory CM (Nettle 1999a; 1999b)
 Spatially distributed populations
– Dialect diversity LG, NN (Livingstone 2002)
– Lexicons LG (Steels & McIntyre 1999)
 Ecological influences
– Survival behaviours GA (Arita & Koyama 1996)
 Stochastic dynamical processes
– Macro models of language change MM, DS
(Niyogi & Berwick 1995; Niyogi 2000)
– Micro models of language change DS, LA, GA
(Briscoe 2000a; 2000b)
Modelling language origins and evolution
IJCAI-05
32
Examples (language change)
 Individual level
– Aging structure in language acquisition LG (de Boer & Vogt
1999)
– Critical periods for language acquisition GA (Hurford 1991;
Hurford & Kirby 1998)
 Population level
– Flux of agents, stochasticity in sensorimotor experiences
LG (Steels & Kaplan 1998)
 Language level
– Self-organisation LG, DS (de Jong 1999) LG (de Boer
2000)
– Lexical change without population flux LG, NN (Stoness &
Dircks 1999)
– Rate and pattern of change MM (Pagel 2000)
– Lexical change over populations ILM, LG (A.D.M. Smith, in
press)
– Iterated learning models ILM, LG (Brighton, Kirby, Smith,
Vogt, Zuidema)
Modelling language origins and evolution
IJCAI-05
33
Examples (observations)
 Evolution of signalling abilities GA (Werner &
Dyer 1991)
 Emergence of conventionalised signals NN
(Hutchins & Hazelhurst 1995) LG (Livingstone
& Fyfe 2000)
Modelling language origins and evolution
IJCAI-05
34
Open issues
 What is the influence of language ecology?
(Livingstone 2002)
 Population/language mix
 Mixing of social structures
Modelling language origins and evolution
IJCAI-05
35
Topics of Research
 Evolution of communication
 Evolution of signalling systems
 Lexicon formation
 Grounding
 Language diversity
 Emergence of grammar
Modelling language origins and evolution
IJCAI-05
36
Emergence of syntax/grammar
 Under what
conditions can
(aspects of)
syntactic or
grammatical
structures emerge?
–
–
–
–
Nativist accounts
Cultural accounts
Hybrid accounts
Grounding
Investigate theories from, e.g., (Bickerton 1990, Chomsky
1990, Pinker & Bloom 1990, Tomasello 2003, Wray 1998)
Modelling language origins and evolution
IJCAI-05
37
Iterated learning (Kirby 2002)
 Population dynamics with overlap (i.e. each generation
1 adult, 1 learner)
 Transmission bottleneck
 Predicate logic meaning representation
 Invention mechanism -> holistic, or exploiting existing
rules (words are random strings)
 Heuristic induction mechanism:
– Chunking
S/love(m,j)->marylovesjohn
S/love(h,j)->hannalovesjohn
S/love(x,j)->N/x lovesjohn
N/m->mary
N/h->hanna
– Merging
N/m->mary
M/m->mary
Modelling language origins and evolution
IJCAI-05
N/m->mary
38
Iterated learning (Kirby 2002)
Phase-space plot of expressivity vs. language size shows the
emergence of syntactic language after many generations.
Picture courtesy of Simon Kirby.
Modelling language origins and evolution
IJCAI-05
39
Examples
 Nativist accounts
– Co-evolution of UG/LAD GA (Kirby & Hurford 1997;
Briscoe 2000)
– Evolution of LAD GA (Turkel 2002) GA+BE (Yamauchi
2001)
– Evolutionary constraints for UG MM, GT (Nowak et al.
2000; 2001; Komarova et al. 2001)
 Cultural accounts (transmission bottlenecks)
– Heuristic grammar inducers ILM (Kirby 2000; 2001;
2002; Zuidema 2001)
– Minimum description length ILM (Teal & Taylor 1999;
Brighton & Kirby 2001)
– Hebbian learners ILM, NN (K. Smith 2003; Kirby et al.
2002)
– Static populations RNN (Batali 1998) MBL (Batali 2002)
– Issues in Optimality Theory ILM (Jäger 2003)
Modelling language origins and evolution
IJCAI-05
40
Examples
 Hybrid approaches
– Constructivist evolution GA, LG (Hashimoto &
Ikegami 1996; Zuidema & Hogeweg 2000)
– Learnable languages HC, RNN (Tonkes et al. 2000;
Tonkes & Wiles 2002)
• Sequential learning
– Word order constraints SL, RNN (Christiansen &
Devlin 1997; Christiansen & Ellefson 2002)
 Grounded approaches
– Construction grammars LG, ROB (Steels 2004)
– Compositional structures LG, ILM, ROB (Vogt
2005)
– Verbs and Nouns GA, NN, ROB (Cangelosi & Parisi
2001; 2004; Marroco et al. 2003)
Modelling language origins and evolution
IJCAI-05
41
Open issues
 Increasing complexity to human level
 Is there a biological endowment for the
emergence of grammar? How?
 How much can cultural evolution explain with
respect to the transition towards grammar?
 Modelling co-evolution syntax & semantics
 Evolution of language acquisition mechanisms
(induction mechanism, Theory of Mind, …)
Modelling language origins and evolution
IJCAI-05
42
One big open issue…
 Did the human brain evolve to facilitate
language, was it the other way around or was
there a co-evolution between brain and
language, cf. (Deacon 1997)?
 Few computational models start looking at
this problem (see, e.g., Dominey in press)
Modelling language origins and evolution
IJCAI-05
43
Summary - achievements
 Evolution of
 communication
 vowel systems
 lexicons
 compositional languages and other aspects of
grammar
 Understand aspects of grounding
 Models of language change and diversity
 Some understanding of neural aspects
Modelling language origins and evolution
IJCAI-05
44
Summary – open issues
 Ecological pressures/cognitive factors for evolution of
symbolic communication
 Biological and cultural endowment for emergence of
grammar
 Scaling towards human level complexity






Population sizes
Population dynamics
Vocalisations
Grammars
Semantics
Etc.
 Biological adaptation of





Vocal tract/auditory system
Language acquisition skills
Intention reading skills
Meaningful behaviour
Co-evolution of language and brain
Modelling language origins and evolution
IJCAI-05
45
Take home message
There are many models, based on hypotheses
and scenarios. Although some theories are
mutually exclusive, many are not.
Try not to focus on one model, hypothesis or
explanation when researching language
evolution; better combine the best bits of the
different models.
E.g., different language acquisition strategies can
perform better when applied together, rather than
when used in isolation.
Modelling language origins and evolution
IJCAI-05
46
BREAK
Modelling language origins and evolution
IJCAI-05
47
Descargar

Topics of Research - Tilburg University