Gesture and Language : Mind and Body
Robin Allott
Clarifying the Mind
Stocktaking the Content of Mind
The Process Recorded
1. A Clear Mind - Is there a Clear Mind ?
2. Without Haste - Is it without Haste ?
3. You must Yearn after Truth Do I Yearn after Truth ?
4, You must be Good Am I good ?
5. You must Love Do I Love ?
6. All in Freedom Am I Free ?
7. Avoiding Conceit Do I avoid Conceit ?
8. With a Bit of Zest Is there Any Zest ?
9. An Eye opening on Existence
10. A Rocket carrying Life and Mind through the Emptiness of Existence
11. A Firework in the Dark
12. A Spark in the Night - Family and other Deaths Remembered
13. Katherine Mansfield - Virginia Woolf
14. Hazlitt Wittgenstein A Happy Life ?
15. Clerk Maxwell Michael Faraday Useful Lives ?
16. Alice James
William James
17. St Augustine St Paul
18. Anaxagoras
19. Marcus Aurelius
Sakya Muni
20. All Were Bodies with Minds
21. Complexities balanced on Complexities
22. Complexity of Body External and Internal Organs
23. Complexity of Brain With Language and Consciousness
24. Complexity of Existence Space Now
To beyond the furthest Galaxies Down to the Universal Base
Time (sample) Here The Golden Gates of the Far Past The Silver Gates of the Far Future
25. Space|Time Here|Now
26. A Point in Space|Time
27. A Point of View looking Out on the Umwelt
28. A Point of View among Points of View
29. The Itness of I - The Itness of All I s
30. The I ness of It [the tiger the tree a stone?]
31. The It-ness of I
32. A Point of View looking IN to the Imwelt: the Glassy Essence of the Mind
33. Considering the Frame of Mind: a glassy cube
to the North - the Mountains of Aspiration to the East - the cold grey Seas of Survival
to the West - the soft Lands of Consolation to the South - the warm Lands of Love Hate and Friendship
34. The Frame of Identity - the Glassy Essences of other Individuals
35. At the END - the Individual a congeries of electrical and chemical particles
36. A Temporary Turbulence in Space|Time
37. Above galaxies of galaxies
38. Below the uniform unchanging base of existence
39. Dissipation of the individual pattern into Space|Time
40. Leaving behind ?
The acquisition of language was the turning-point for the
evolutionary separation of humans from apes. From this
flowed the ascent of human intelligence with the
ratcheting up of human mental and cultural advance as a
result of interaction between individual variations in brain
structure and development and continually advancing
complexity of the social, technological and cultural
environment (a manifestation of the Baldwin effect).
The Starting Point:
Language is not an arbitrary construction
Words are not arbitrarily invented forms
Linguistic variability and intellectual development.
The sound [the word] is not “a directly imitative sign but
indicates a quality which the sign and the object have in
common. . . . sounds which partly independently and partly in
comparison with others produce an impression which to the ear
is similar to that which the object makes upon the mind.”
Humboldt, W. von. [1836] Uber die Kawisprache auf der insel Java.Trans.1971
G.C. Buck and F.A. Raven. Univ. of Miami Press.
Whence comes to man the art of changing into sound what is not sound? What has a
color, what has roundness in common with the name that might evolve from it ... why
green is called green and not blue?
I do not suddenly ascribe to man an arbitrary qualitas occulta - a new power
providing him with the ability to create language.
I do not ... proceed on the basis of arbitrary or social forces but from the general
animal economy. An arbitrarily thought-out language is in all senses contrary to the
entire analogy of man's spiritual forces.
Herder J. G. [1772] Essay on the Origin Of Language. Trans. 1986 J. H. Moran, A. Gode. Univ.
Chicago Press.
Both body and brain were involved in the acquisition and advance
of language, phylogenetically and ontogenetically. The relation
between mind-state and body-state was manifested not only in the
patterning of emotion (the James/Lange theory) but also in the
selection of words (articulatory programs) to match patterns of
perception, both of external objects and also innate attitudinal
patterns, "mental" structures, which went to form both language
syntax and motor syntax (cf. Karl Lashley, Kant).
The key aspect of the motor theory of language is that
words, speech and language are the outcome of an
exaptation of the motor control system, that is, a
direct relation between aspects of the motor cortical
system and the characteristic features of phonology,
lexicon and syntax.
“Language did not arrive de novo , but came by differentiation
of and building upon pre-existing systems (such as the
perceptual-motor systems ... )
We see the ability to comprehend and utter words as forming
perceptual and motor schemas that arise in the first place from
abstraction from the sensorimotor schemas that represent the
objects and actions to which those words refer”.
From Schema Theory to Language. Arbib, Conklin, Hill 1987 OUP
Progress in neuroscience, particularly recent experimental
research using fmri and other techniques and presentations in this
conference, has provided material support for the motor basis of
Bernardis Gentilucci Speech and Gesture share the same communication system 2006
Neuropsychologia 44 178-190.
Graziano Taylor Moore Cooke The cortical control of movement revisited. “One
possibility is that the mechanisms for speech were built on a preexisting mechanism
for motor control.” Neuron 36, 349-362.
Holden Origin of speech: The motor route “Abundant behavioral evidence for an
intimate connection between language and motor abilities.” 2004 Science 303:13161319.
Pulvermüller F, Shtyrov Y, Hauk O. Understanding in an instant: neurophysiological
evidence for mechanistic language circuits in the brain. Brain Lang. 2009
Willems Ozyurek Hagoort When Language Meets Action: The Neural Integration of
Gesture and Speech. 2006 Cereb. Cortex .
The origin of words
From the imitation of sounds shapes and actions
Then transduced into articulatory programs
By motor equivalence
Has been the central process in the origin and functioning of language and the
acquisition of words.
What it means is that the same motor program can be executed by different
sets of muscles (and joints) - e.g. writing your signature with your foot or nose
or transducing the articulatory program (articulatory phonology: Browman and
Goldstein) for a word into a gesture or a sound.
Berthoz The Brain’s Sense of Movement Harvard 2002 225-227
Browman, Catherine P. and Louis Goldstein. 1992. Articulatory phonology: An overview. Phonetica 49: 155-180.
Linking of words and gestures (dual expressions of meaning) can be
made overt by specific controlled mind/brain operations
1. to hear sounds produced by the sound-structure of animal names
2. to see picturing gestures produced by the sound-structure of animal
names (for animals which make no distinctive sound)
3. To see gestures produced by the sound-structure of action words
Animal sounds from animal names (in various languages):
Gestures from visual images: Actions from action words
The Inverse
Brain size and structure ?
Baldwin effect ?
EvoDevo ?
The inescapable fact is that brains, and particularly
human brains, have much increased in size in the
course of evolution. The increase in size must have
brought survival benefits and for humans it surely
means that however intelligence is measured greater
size has moved in step with greater intelligence though at the individual level the correlation is not
BUT … all the day-to-day routines of bodily
existence require very little neural mass. Ants,
bees, mice, birds, dinosaurs, manage, or
managed, very well, with small, or extremely
small, brains (the ratio of brain to body may be
more significant, for example, for mice or other
small rodents).
So why and how did humans come to acquire a brainmass much greater than is needed for routine bodily
functions ? From homo habilis to modern homo
sapiens the brain grew from an average 750 cc. to
1350-400 cc.
[Quadrupled in size relative to body mass. No known comparable
increase in brain size in any other species Plotkin 1996]
1. Social complexity
2. Foraging strategies
3. Language development
There has been intense and long-enduring discussion of
these issues and many unresolved arguments with much
speculation and little useful evidence.
That the remarkable increase in human brain size (unmatched by any
similar rate of increase in other animals) should have some relation to the
equally remarkable (unmatched by other animals) evolutionary human
acquisition : language, seems an obvious and plausible hypothesis.
But the debate, confusion and uncertainty about the process by
which humans invented, acquired or developed language, or
languages, still rages, after 2500 years. The question about the
evolutionary relation of human brain-size and language goes with the
unsettled question about the brain or social processes making
possible another remarkable achievement, the untutored, extensive
and rapid acquisition of language, complex syntaxes and massive
lexicons, by children
Evolutionary developmental biology
Behaviour evolution interaction
The accepted evolutionary account of the Baldwin effect was that
humans, and other animals, by changing their behaviour changed
their environment and so created novel potentialities for natural
selection which could operate in succeeding generations, making
it possible for culture to modify evolution.
With the recent growth of the new discipline Evo-Devo
concerned with the relation between developmental and
evolutionary processes, a new application or understanding of
the Baldwin effect in relation to development of the brain
becomes possible.
In the development of the brain, many more neurons are produced than are
ultimately needed to create the mature brain. The neurons are thinned out by
programmed cell death; the initial supply of neurons (twice as many as eventually
survive) are in competition to establish appropriate connections. Those which do
not get the necessary access to a source of NGF (nerve growth factor) die. This
Neural Darwinism (in a sense completely different from Gerald Edelman’s term) is
natural selection at work.
[Joan Styles The Fundamentals of Brain Development Harvard 2008
p.257-268 Programmed cell death]
3 neurons compete for Nerve Growth Factor.
2 win and the 3rd dies(repeating loop)
Through cell death each individual brain is sculpted to match the
environment in which the brain develops (both fetally and for an
extended period after birth). Cell death continues shaping the brain long
after birth. For all cells, body and brain together, from 8-14 the average
child loses approx. 20-30 billion cells a day. The average adult loses
approx.50-70 billion a day.
[ See Biophotonics For Viewing Programmed Cell Death
SciMed – Horizons 7 August 2010]
Cell death can now be recognised as an important link between brain and
environment, operating through life to permit experience to shape the brain.
It is via cell death that the relation between brain-size and culture including
language can be understood. A key point in the operation of this process in
the developing brain is individual brain variation. Each brain is unique in
many ways, including its size and the number of neurons which have to
find appropriate connections to survive.
Natural selection can come into play on differences in brain structure derived
from the culture. Increased brain size means the availability of more neurons
(and more neuronal inter-connections) available to match the extending
experience of the individual- and so increase the selective advantage of the
individual in a changing culture. A virtuous circle is established with the
better adapted and no doubt larger- brained individuals playing a larger part
in shaping the environment in which the next generation will have to face
natural selection and in their turn have their brains sculpted by experience.
Language in the group will account for an ever-larger segment of
total cultural input to the brain and will also act as a powerful
instrument in shaping the social system. A ratchet effect is
established which goes to promote a persisting increase in brainsize (and skull-size co-ordinated by allometry with brain-size)
until the skull-size hits the anatomical constraint of birth-canal
The brain no doubt keeps on growing in complexity to
accommodate continuing language and other cultural
changes but the growth must take the form of internal
re-arrangement to produce the elaborate infolding seen
in the modern human brain.
[The surface area of a flattened human cortex is three times larger than the
inner surface of the braincase Hilgetag, Barbas Sci. Am. Feb. 2 2009]
But how does language drive this? The major new language input to the
brain is through a rapidly expanding lexicon. The growing lexicon, on the
motor theory of language, requires that each word creates an articulatory
motor program (linked to and derived from the visual or action percept)
The influx of novel words involves an increasing demand for neurons and
neuronal connections. Baldwin Evo-devo is the form in which language
drives growth in the size of the human brain in response to the acquisition
of words.
Additional demand for neurons and
connections to accommodate syntax is less
significant. Language syntax can rely
substantially on pre-existing organisation of
the motor and visual systems, motor syntax
and vision syntax, and the neurons and
connections serving these.
But of course this only goes so far in explaining the evolutionary role of
language. Before the Baldwin Evo-Devo process can begin to operate to
increase brain-size, there must already be language and language-related
culture. There must already be words.
The central question remains: WHY and HOW could speech and language
have got going for humans at all? Why humans and not dogs or apes? The
most plausible possibility is, as Jan Wind suggested long ago, not that
there was some massive mutation but a continuing process of cerebral
reorganisation. Relatively minor changes, well within the scope of inherent
brain plasticity, could have made speech possible.
Recent research papers offer some possibilities:
Jurgens: Neuroanatomically, the step from genetically
determined controlled vocal patterns is associated with the
emergence of a direct connection between the motor cortex
and the laryngeal motoneurons, a connection lacking in
subhuman primates.
Jurgens Uwe 2000 A comparison of the neural systems underlying speech and nonspeech vocal utterances in Becoming Loquens ed. Bichakjian et al. Frankfurt am
Main Peter Lang
Kay Cartmill Balow: “Hypoglossal canal and the origin of
human vocal behavior” The hypoglossal canal (which
carries nerves controlling tongue movements) is much
larger in humans than in other primates or in
australopithecus. The larger canal is adapted to carry a
much richer motor innervation of the tongue and so to
make possible language as a uniquely human ability.
"Hypoglossal canal and the origin of human vocal behavior" 1998 Kay
Cartmill Balow PNAS 95 5417-5419.
Brown Ngan Liotti “A larynx area in the human motor cortex” Cerebral Cortex
July 25 2007
“A human evolutionary novelty perhaps related to emergence of voluntary
control of vocalisation.” of vocalisation
"When Language Meets Action: The Neural Integration of Gesture and
Speech.” 2006 Willems Ozyurek Hagoort Cereb. Cortex fMRI evidence that
speech and gesture share a high-level neural integration system.
"Speech and Gesture share the same communication system" 2006
Bernardis Gentilucci Neuropsychologia 44 178-190. Experiment suggests
that word and gesture are related at the levels of execution and processing
with implications for the evolution of language.
Some, or all, of the listed cortical changes could have made speech possible
for humans (but not for other primates). But for the Baldwin Evo-Devo effect to
operate there had to be words. Where did they come from? Where does any
individual word come from?
Herder said that it was totally impossible that words should be arbitrary, that
someone should invent say the word GREEN out of the top of his head for the
distinctive colour Green. If he picked a set of speech sounds at random, say
POGGLE, to mean Green, why should others accept and understand him.
Even more impossible, how could anyone arbitrarily invent the words IF, MIND,
Rizzolatti and Arbib argued that the discovery of mirror neurons linking
responsive motor programming in the brain of an observer with observed motor
patterning of action of another individual, could have been the basis for the
evolution of language. The mirror neurons could have made, and still make,
imitation possible, including imitation of gesture.
Arbib argues that “the ability to imitate is a key innovation”, “a neurobiological
‘missing link’ for the hypothesis that primitive forms of communication based on
manual gesture preceded speech in the evolution of language”. “A possible
evolutionary path from manual skills to language”.
Gallese says “the discovery of mirror neurons may provide a new, though still
sketchy, neurobiological basis to account for the emergence of language”.
Some things can be indicated by imitation or pointing, the sky, a tree, a
direction, up or down, come and go, high and low. Imitation can be used to
point to objects, to indicate hearing, eating or drinking, etc. But for many
things imitation is inadequate: colours - white black red, sounds, different
animals. what would be imitation for a horse, a fish, a rose, a cabbage?
Imitation v mimicry - What matters is stored imitation (requiring
commitment of neurons and interconnecting fibres) in the form of a
link between word and imitated act, sound or shape so that there can
be ready access to the word and what it refers to. Words are anchored
in the motor patterning and are expressible as bodily and articulatory
gesture. Deacon quoted by Gerhard: the “everyday miracle of word
meaning and reference”. So could each individual word be an arbitrary
invention with an equally arbitrary linking to what it related to?
With Herder and Humboldt, we can say ‘surely not’.
How was each gesture invented? Was each gesture as arbitrary
as traditional linguistics says that each word is? Clearly not. A
gesture is patterned by the action seen, the shape of what is
seen, the sound heard, for a vocal gesture. Mirror neurons may
allow transfer of the pattern of an action and the ability to
reproduce the action as a gesture (not in any way arbitrary) - but
imitation has to be possible for much beyond perceived action.
The process by which words were formed was the inverse of the
process by which gestures and sounds can be generated from
existing word-forms - a reverse application of motor equivalence.
On seeing some one hitting something, the action patterning was
by motor equivalence converted into articulatory patterning to
produce a speech-sound structure, a word, directly related to the
action patterning seen. Similarly on hearing an animal sound, the
typical sound of a cat or a lion, the sound-patterning is transduced
by motor equivalence to form a word whose structure is derived
from the sound heard.
Experiment slide
Gestures of all kinds were generated by imitation of actions, shapes and
sound. These were stored as motor programs before humans acquired
When cerebral reorganisation provided new direct connections between the
motor cortex, the tongue and the larynx, there was a great increase in the
innervation of the articulatory apparatus generally.
The motor programs from gestural origins were transduced automatically (via
motor equivalence) into words structured by the gestural programs. The
meanings of words were automatically linked to the action, sound and shape
percepts to which the gestures referred.
There is still the question how words without external reference (including
abstract words) could originate. This remains a matter for separate discussion.
The words exist and must have come from somewhere.[now on this]
Vittorio Gallese at a psychoanalytic seminar

No Slide Title