Total cognitive care for
development of perfect baby brains
Włodzisław Duch
Dept. of Informatics, Nicolaus Copernicus University, Poland
Dept. of Computer Science, School of Computer Engineering,
Nanyang Technological University, Singapore
Google: Duch
Background: cognitive infant development
Few science facts
Total cognitive infant care
Toys that make you smart
Techno remarks
Plans & strategy of development
Endless possibilities ...
Developmental problems
• Brain is the most complex organ and frequently not functioning well.
• About 5-10% of all children have a developmental disability that
causes a delay in their speech and language development.
• If serious hearing problems are not found in the first six months of
life, and the baby given the appropriate treatment, that child will
grow up with significant damage to his or her language abilities.
• Children with normal cognitive development whose hearing losses
are identified before six months can develop language at the same
or a similar rate to a hearing child. Identification of congenital
hearing loss in USA is at 2½ years of age!
• The British Dyslexia Association estimates that 10% of children
have some degree of dyslexia, while about 4% will be affected
severely (an average of one in every class).
Cognitive development
• Brain: the ultimate engineering problem! How to improve it?
• Although genes set a limit for potential individual development
this limit is never reached; genetic manipultion is dangerous.
Early development is critical for unfolding brain’s potential.
• Challenge: prevent abnormalities and boost normal development.
• Since genes specify only roughly where neurons should go (ex:
eye to visual cortex, across the whole brain) too many neurons
are created, and those unused die (apoptosis).
Maximum number of neurons: 1-2 month before birth, although
infant’s brain is only ¼ of the final size.
• General principle:
growing up is specializing = narrowing potential possibilities.
How to keep more possibilities open?
Example: motor development
• Duke University, NC, 2002: velcro-covered mittens help
2-3 month infants develop more quickly.
• Infants are no yet able to grasp objects, but velcro mittens help
them to catch and then touch toys;
this speeds up motor coordination and perceptual development.
• Babies who had experience with the mittens outperformed the
babies who didn't in a number of ways.
• Long-term consequences of this procedure are investigated by
psychologists. News shown by CNN and elsewhere: in this area it
is easy to get a free press ...
• Lynn-Flynn effect: IQ grows everywhere in the
world, 24 points in USA since 1918, 27 points in UK.
Toys and nutrition help to develop better brains?
Infant communication
• Brain is capable of learning, but poor sensory analysis and motor
coordination makes it difficult to communicate with infants.
• High-amplitude suction-methods are used with newborns and
infants to find how much interest they have in different stimuli.
• Sign-language for communication with infants (US patent):
using simple signs with babies to bridge the gap between
understanding and speech development
• Recommendations of developmental psychologists:
pre-natal stimulation: BabyPlus™ “enhanced heartbeat”,
playing music to babies in the womb.
Playing tapes with foreign language lullabies etc.
• All this is passive learning, much less effective than interactive
learning in which the infant is actively engaged.
Can one learn walking just by observation?
• Learning to perceive, speaking and drawing inferences should be
active, guided by well-structured inputs and positive feedback.
Speech perception facts
• Speech perception is based on syllables, combinations of
elementary phonemes; 2-month old recognize syllables.
• There are about 6800 languages, but only 800 phonemes.
• The ability to hear phonetic contrasts of all possible human
languages is in-born, 6 month old are good in any language.
• This ability declines sharply 10 month after birth, only the ability to
distinguish contrast in languages that are spoken to the child is
preserved – one of the earliest specializations.
• At 5 month children are able to categorize a vowel in
a speaker-independent way.
• Learning many languages correlates positively with IQ
• Tonal languages, prevalent in Asia, are phonetically rich,
but even ‘simple’ /ra/-/la/ contrast is impossible for Japanese,
/vi/-/bi/ for Spanish, /s/-/th/ for most non-English native speakers.
Vowels spoken
by 97 speakers,
seen in the first
2 formants only.
Mapping to the
auditory cortex
looks similar.
means that
what we hear
jumps from
a to e with
no gradation
ma – ma diff?
Speech problems
• Up to 4 million primary and secondary school students in USA
have difficulty distinguishing between phonemes, particularly
between consonants like b, d and p; is it pee or bee, pun or bun?
• “Fast ForWord”® software by Scientific Learning helps to develop
learning skills for 4-7 year olds. It is used in the Chicago public
school system.
• Private clinicians provide Fast ForWord training (>$2,500).
It shows remarkable success with kids who suffer from central
auditory processing disorder that leads to speech understanding
problems and retards reading.
• Big market: Learning Company, the producer of Reader Rabbit,
was acquired by Mattel for $3.5 billion.
• Can we eliminate hearing and speech problems improving
perceptual and cognitive skills of babies at the same time?
Active stimulator of brain’s speech centers,
especially infant and children
Application no: 184102, date: 29.07.1997, granted 30.08.2002
Creator and the sole rights holder: Duch Wlodzislaw, Torun, PL
Active stimulator of brain’s speech centers, especially infant and
children, with input device, analog-digital converter, memory
W F , m e m o ry o f thpatterns,
comparator of phonemes received
p h o n e m e p ro to ty p e s
through inputs
w z o rc ó wand retrieved from
D , d e clogical
is io n lo g ic decision unit,
K , com
p a rin memory,
Phonem e
converter, and the output device for sound. The input
F 1converter,
F 1  this
device connects
converter is
M ic ro p h o n e (in p u t d e v ic e )
connected to one input of the comparator,
another input of the
Phonem e
to Dthe
memory containing
phonemic patterns,
/A , d ig ita l – a n a lo g
F1 or F2
c o n v e rte r, s o u n d s
and the comparator’s
S p e a k e r output is connected to a logical decision unit;
the output from this unit connects with the sound
producing device
v is u a l s ig n a lin g d e v ic e
through the digital-analog converter. (p o s itiv e fe e d b a c k )
A /D -W
a n a lo g - digital
c o n v e rter
US5893720 abstract
Hannah R Cohen (US) 1999 patent:
A computer toy for infants that promotes normal speech development
by facilitating the infant's experimentation with babbles and other
elementary sounds. Additionally, the toy provides an enriched
environment for language learning by prompting the infant with a
repertoire of verbal sounds including phonemes, syllables, and simple
spoken words.
The toy includes a microphone input device for detecting vocalizations
by an infant and an audiovisual output device for providing feedback to
the infant including the immediate playback of the infant's own
vocalizations, and a control means for transforming detected
vocalizations into instructions for use by the audiovisual output device.
Philips “Magic Mirror” is based on this principle.
Philips “Magic Mirror”
• Some companies are going in this direction, making interactive
toys, for example:
“Magic Mirror” introduced in 2003
Records and replays baby’s voice,
triggers first speech sounds and
has mirror, pushing buttons
produces some sounds,
encourages experimentation.
It is not aimed at enhancing
specific perceptual skills.
Total cognitive care
• Observe the baby and interpret his/her behavior by monitoring
vocalization, sucking response, movements, GSR etc.
• Challenge the baby to solve perceptual/abstract problems.
Use natural audio-visual-tactile stimulation.
• Reward behaviors that correlate with differences in stimuli.
For example, if a series of sounds is played:
“la la la la ra ra ra ra”, reaction to la-ra change shows that the
baby has noticed an important phonetic contrast.
• PerCog devices for enhancement and therapy of perceptual and
cognitive skills use intelligent AI control to model what the baby
has already learned, what and how often should be presented,
observing, diagnosing, correcting and teaching infants.
• Hypothesis: active learning should gently pressure baby’s brain
to develop perceptual/cognitive skills in the desired direction.
Toys for speech development
• Goal: enhance phonematic hearing.
• Prepare a database of phonemes and syllables, present pairs
that are similar, and if the infant notices phonetic contrasts, apply
positive stimulation. Start with basic contrasts, end with subtle.
• The infant is not able to repeat speech sounds: changes in the
larynx, highly sophisticated control over vocal cords, are slower
than neural development.
la -la … la -ra -ra …
C o n tro l u n it
T e le m e tric
p a c ifie r
W ire le s s
c om m u n ic a tio n
re c e iv e r
A /D c o n ve rte r
D a ta b a s e o f
speech sounds
D /A c o n v e rte r
sound sequences
S p e a ke r
• Hypothesis:
1) Early (~10 month) specialization in correct discrimination of
mother tongue phonetic contrast may be eliminated, giving the
baby ability to learn any language she/he chooses.
2) Sharper discrimination between basic phonemes should
prevent some hearing and speech problems.
A u d io vis u a l
d e v ic e (re w a rd )
D a ta b a s e o f
re w a rd p a tte rn s
N o n -v o la tile
m em ory
Toys for musical ear
• Goal: enhance musical hearing, develop perfect pitch.
• Only one person in 10.000 has perfect pitch; adult professional
musicians try to learn it but it is very hard.
• Prepare a database of musical sounds of different instruments at
different pitch, present pairs of sound samples with few seconds
of silence in between, reward for signaling that sounds were of
the same pitch although tones may differ. Start with pure
frequencies, move to more complex sounds and accords of
several sounds.
• Hypothesis: memory for absolute sound pitch should develop,
increasing the ability to hear subtle musical structures.
Toys for abstract thinking
• Goal: enhance the ability to think!
• Infants habituated for 2 minutes with sentences like
ga ti ga, li na li, of the ABA structure, recognize that wo fe wo has
correct grammatical structure but wo wo fe does not.
G. Marcus et al, “Rule learning by seven-month-old infants”,
Science 1999, Vol. 283, pp. 77 – 80.
• Challenge the infant using sounds and color lights with structures
of increasing complexity, reward for noticing differences.
• Hypothesis: solving problems of this kind has strong influence on
development of the cortex, in particular frontal and temporal
areas, and should increase working memory span, that correlates
well with general IQ.
More PerCog devices
Goal: enhance perceptual discrimination in all modalities, improve
categorical perception, increase working memory span, long-term
perceptual memory, encourage abstractions and faster reactions.
Responses to stimuli, behavioral interaction patterns should allow for
early diagnosis of developmental problems.
Imagine ...
• A world in which people could communicate
without problems and even adults learn languages easily.
• A world in which developmental abnormalities related to speech
and reading would largely vanish.
• A world much more rich in subtle sensory experiences.
• A world in which the potential of each child would always fully
develop, and natural curiosity will grow ...
• We just have to start quite early, adding structure to the
development of the brain – in a natural, effortless way.
• This is possible with PerCog devices, such as cognitive toys!
• Guess what is the market for such devices?
So far
• Polish patent application was restricted to phoneme perception, the
present proposal is much wider and differs in important aspects.
• We have made some research assuming that the main feedback
from the infant will be via vocalization: phonemes from 28
languages have been collected, software to present them,
recognize change from silence/speech and vowel/consonant
change in baby’s vocalization has been used; this is done by
microphone and speaker connected to PC sound card.
• Sucking (HAS), head turning preference, visual fixation and other
procedures used by psychologists for infants < 6 month are better.
• Some work on hardware implementation has been done, but it still
requires a PC to analyze the data; hardware working prototype due
to technical problems.
• First personalized medicine device, not related to molecular level:
adapts to the individual needs/abilities.
• Patent application: “Devices for enhancement and therapy of
perceptual and cognitive skills of infants and children” for all kinds
of PerCog devices and total cognitive care of infants.
• Ready for quick commercialization and more R&D. Largest
market for speech devices is in the Far East, in the USA more
prevention/diagnosis/treatment of speech/reading problems.
• Construction of PerCog devices in form of electronic toys or crib
environments, initially with a modest number of phonemes
covering basic phonetic contrasts, is not difficult.
• Computer games for babies and children may be created using
the same principle.
Research opportunities
• Interdisciplinary group including experts in developmental
psychology, phonetics, neurolinguistics, speech/hearing/language
pathologist, education experts, signal processing, electronics
(sensors + IC), toy design, software development, data mining ...
• Creation of larger phonetic databases: most important phonetic
contrasts for major languages, syllables and words which are hard to
distinguish, like van/ban, thick/sick, extending it to cover subtle
differences, and finally full spectrum of speech sounds.
• Improvement of the feedback recognition systems: wireless sucking
monitoring devices, analysis of infant babbling for sounds that infants
may use for signaling, observation of movements.
• Follow-up studies to investigate the effect of using cognitive speech
toys on phonetic contrast perception (a few month), language
learning (long-term studies) and reduction of speech and reading
abnormalities (long-term).
Brave new world?
New generation of children may grow up to be:
much more intelligent than we are
think much faster than we do
be much more musical
be free of speech and hearing problems
be able to learn any language without accent
see things we hardly notice
Popular book: The Scientist in the Crib,
Alison Gopnik, Andrew N. Meltzoff, Patricia K. Kuhl (William Morrow &
Company 1999.

Title of Talk