Visual Communication: theoretical
issues
Research Seminar Master Informatiekunde
13/09/06
Leonie Bosveld-de Smet
Humanities Computing, RuG
Question



Pictures can be worth ten thousand words.
Visual representations can be superior to verbal ones in
thinking (problem solving), teaching, and
communication.
Can benefits observed in other areas (esp. science) be
exploited in other fields?
10/4/2015
Visual Communication
2
Outline talk

Characteristics of graphical versus sentential languages

Facilitating effects of visual representations

Visual communication: a semantic view

Conclusion and discussion
10/4/2015
Visual Communication
3
Theoretical concepts

Key words:






Representation system
Interpretation function
Form of representation
Perception of representation
Medium
modality
Piece of
information
Structured
information
Representation
system
Form of
representation
link
10/4/2015
Visual Communication
4
Media and modality contrasts


Stenning and Inder(1995)
“Medium” how is a representation perceived?
“Modality” how is a representation interpreted?
10/4/2015
Visual Communication
5
Graphical vs Linguistic
Representations

Stenning and Inder (1995)
the more constrained, the less expressive, the more tractable


Stenning and Oberlander (1995)
limited abstraction, aided “processibility”


Shimojima (2004)




Free ride properties
Auto-consistency
Specificity
Meaning derivation properties
10/4/2015
Visual Communication
6
Free Ride Property
Express information:
Jon defeated Bob.
Ken lost to Bob.
FOL
PD
English
10/4/2015
Defeated(Jon,Bob) & Lost_to(Ken,Bob)
Jon Bob Ken
Jon defeated Bob and Ken lost to Bob.
Visual Communication
7
Specificity
Express information:
Jon defeated Bob.
Ken defeated Bob.
FOL
Defeated(Jon,Bob) & Defeated(Ken,Bob)
PD
Jon Ken Bob
English
Jon defeated Bob and Ken defeated Bob.
10/4/2015
?
Ken Jon Bob
Visual Communication
?
8
Meaning Derivation Property
Express information:
Jon defeated Bob
Gil defeated Jon
Bob defeated Ken
Ken defeated Ron
Defeated(Jon,Bob) & Defeated(Bob,Ken) &
Defeated(Gil,Jon) & Defeated(Ken,Ron)
FOL
Gil Jon Bob Ken Ron
PD
English
10/4/2015
Jon defeated Bob and Bob defeated Ken and
Gil defeated Jon and Ken defeated Ron.
Visual Communication
9
Benefits of graphical systems







Automaticity of inference
Ease of consistency inferences
Difficulty to express “abstract” information
Richness of semantic content
“Transparent” problem solving
Computational offloading
Amplification of cognition
10/4/2015
Visual Communication
10
Monk Puzzle (Winn, 1987)
“A monk went to the temple at the top of a holy
mountain to meditate and pray. He started out early
one morning along the path that led up to the temple.
Because he was an old man, and the way was steep and
arduous, he frequently slowed his pace, and even sat
and rested a while beside the path. Toward evening, he
came to the temple at the top.
After several days of meditation and prayer, it was
time for him to leave. Early in the morning, he set off
back down the path. Again, he frequently changed his
pace and rested by the way. He arrived back at the
bottom in the evening.
Show that there is one single point on the path up
the mountain where the monk will be at precisely the
same time both when he goes up and when he comes
down.”
10/4/2015
Visual Communication
11
“See” the solution
10/4/2015
Visual Communication
12
Larkin and Simon (1987)








“Why a Diagram is (Sometimes) Worth Ten Thousand
Words”
Diagrammatic vs. Sentential representations
Informational equivalence, computational offloading
Pulley problem
Geometry problem
Grouping of information
Exploitation of space
Easy perceptual inferences
10/4/2015
Visual Communication
13
Using vision to think


Kosslyn (1994)
Scatter plots...employ point
symbols (such as dots, small
triangles, or squares) as
content elements. The height of
each point symbol indicates an
amount. These displays
typically include so many points
that they form a cloud;
information is conveyed by the
shape and the density of the
cloud
Simultaneous presentation of
local information and global
information implied by the local
information.
10/4/2015
Visual Communication
14
Visual Communication
Raw
Data
Data
Tables
Data
Transformations
10/4/2015
Visual
Structures
Visual
Mappings
Visual Communication
Views
View
Transformations
15
Semantics of pictures:
Wang’s view
1
2
Graphical Domain
Application Domain
3
1: geometrical characterization of pictures
2: interpretations: graphical entities – application domain objects
3: specifications: application domain concepts – picture classes
10/4/2015
Visual Communication
16
Natural vs. Unnatural Link
c
c
b
b
a
a
10/4/2015
Visual Communication
17
Misleading Link
Gregory
Charles
Bill
10/4/2015
Visual Communication
18
Kinds of visual communication

Via interpretations:




Illustration
Demonstration
Reasoning
Via picture specifications:



Geometric constraint maintenance
Design
Computational art
10/4/2015
Visual Communication
19
Requirements
Meaningful and Easily Tractable diagrams




Clear delimitation of critical information
Description of information by well-defined notions and
properties (elements of application domain)
Graphical devices suitable for representation of
information (elements of graphical domain)
Explicit, natural, not misleading link between graphical
domain and application domain
10/4/2015
Visual Communication
20
Graphical domain








Research in information visualization and diagrams and
diagrammatic reasoning
Bertin
Tufte
Shneiderman
Blackwell
Hegarty
Scaife & Rogers
Novick
10/4/2015
Visual Communication
21
Possible application areas

Language instruction

Linguistic research

History instruction

Historical research
10/4/2015
Visual Communication
22
Towards an automated system
for diagram research
e1: see
e2: stop
“When we saw the spaceship, we stopped the car.”
t
e1: deliver
e2: pay
t’
now
s: have paid
“Two men delivered the sofa. I had already paid for it.”
t
t’
now
e1: make lunch
s: making lunch
“David was making lunch when the phone rang.”
e2: ring
t
10/4/2015
Visual Communication
now
23
3-D Syntactic trees
10/4/2015
Visual Communication
24
Descargar

Exploring visualizations in the humanities