XIV Peru-TESOL
Sharing Teaching and Learning
Experiences in a Global World
Iquitos
August 2, 2006
M. Karen Jogan, Ph.D.
[email protected]
Catching that elusive butterfly:
assessing speaking in five easy
steps
Today we will…
 Outline the process to capture oral competency
of EFL students
 Define oral proficiency – identify proficiency
levels in instructional program
 Design assessable speaking tasks for each level
 Identify or create rubrics used to assess
speaking tasks
 Align assessment tool to a program outcome
 Prepare to implement an oral assessment plan
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
3
What are the steps in the process?
1. Determine minimal outcomes, using
proficiency guidelines
2. Include speaking tasks for assessment in the
classroom
3. Adapt or develop assessment tools to assess
speaking
4. Promote contextualized speaking activities
5. Assess the speaking task, incorporate into
overall assesment framework
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
4
1. Determine minimal student outcomes
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
5
What is proficiency?
Superior
Advanced
Intermediate
Novice
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
6
Specific linguistic functions
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
7
Superior
Advanced
Intermediate
Novice
Tell time, give one´s name, describe
weather, list objects
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
8
Superior
Advanced
Intermediate
Novice
Ask for directions, describe
preferences, ask and answer
questions, order a meal
Tell time, give one´s name, describe
weather, list objects
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
9
Superior
Advanced
Narrate a story in the past, compare
and contrast
Intermediate
Novice
Ask for directions, describe
preferences, ask and answer
questions, order a meal
Tell time, give one´s name, describe
weather, list objects
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
10
Superior
Support opinion, hypothesize,
narrate in detail, abtract topics
Advanced
Narrate a story in the past, compare
and contrast
Intermediate
Novice
Ask for directions, describe
preferences, ask and answer
questions, order a meal
Tell time, give one´s name, describe
weather, list objects
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
11
Discourse – from limited to extended
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
12
Superior
Discourse level
Multiple paragraphs
Advanced
Multiple sentences
paragraph level
Handle complex situation and
transaction
Sentences
create with language;
handle simple situation/transaction
Short memorized phrases
lists
Novice
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
13
Time to think!
In which level of EFL instruction are your
students able to perform the following?
Complain
Summarize
Narrate a past event
Greet a friend
Compare
Ask questions to elicit personal information
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
14
To consider how well students perform
language functions, which (assessment)
activities are appropriate?
What does the student DO to
demonstrate competency???
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
15
 Describe
 Ask and answer
questions
 Narrate in past
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
16
 Describe
 Ask and answer
questions
 Narrate in past
 Oral presentations
 Interview your partner
 Story retelling,
summarizing
 Picture-cued
descriptions
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
17
 Ask/provide
information
 Agree/disagree
 Express opinions
 Support opinions
 Persuade
 Complain
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
18
 Ask/provide
information
 Agree/disagree
 Express opinions
 Support opinions
 Persuade
 Complain
 Role Plays
 Debates
 Information gap
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
19
 Get the gist
 Get the main idea
 Get specific details
 Summarize
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
20
 Get the gist
 Get the main idea
 Get specific details
 Summarize
 Tell/discuss what you
heard
audio file
film
news report
native speaker…
 Tell about what you
read in
newspaper report
story, etc.
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
21
 Summarize
 Describe
 Synthesize
 Analyze
 Summarize/describe/
anaylze
video
program
Article, etc.
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
22
2. Include speaking tasks for assessment
in the classroom
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
23
List all speaking activities which are part of
your course
Designate or design specific speaking
tasks for assessment
Define all elements of the “product” which
you expect your students to produce
Consider which criteria you will use for
assessment
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
24
Interpersonal mode
Roleplays among students
Interviews between students
Oral monologues
Situation cards (simulations)
Performing a play
Presentational mode
Narration or description (oral presentation)
Poster session
Oral presentation with power point
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
25
Time to talk!
Describe an oral activity which your
students have performed successfully.
Why do you think that activity was
successful?
Describe an oral activity that your students
did not perform well. Why do you think
that the activity was unsuccessful?
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
26
Rules to remember
Design the task
carefully.
If you can´t
define it, you
can´t assess it.
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
27
Designing a speaking activity
 Determine level
 Select function or language task
What is the grammar students need to perform this task?
 Select a topic or situation
Vocabulary?
 Define a time frame (present/past/future)
 Define the elements of the context
Who, what, where, when???
 What aspects of culture must be included as part
of the activity?
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
28
It´s your turn!
 Design a speaking activity
1.
2.
3.
4.
5.
6.
Level
Function
Grammar
Context – who, what, where, when
Content – topic or situation
Culture
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
29
3. Adapt or develop assessment tools to
assess speaking
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
30
Recognize that a variety of assessment
tools can be used
Survey avaliable assessment instruments
Examine criteria; recognize that criteria
can differ according to instructional level
Collaborate to develop appropriate
assessment tools
Match assessment tool to appropriate
activity and instructional level
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
31
4. Promote contextualized speaking
activities
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
32
Move from skill building to skill using in the
instructional program
What can students DO????
Move from meaningful to communicative
to situational activities
Provide a framework for students to
structure a conversational exchange
Monitor the process
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
33
Sample role play buildup
Context:
You have just met a new student from
the U.S. who is attending your
university. You want to find out about
this person and what you both have in
common. After you establish common
ground, invite your new friend to join you
for an activity in which you are both
interested.
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
34
Prepare your students for success!
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
35
Sample role play buildup:
instructions to students
 Preliminary step
Key words and
phrases I will use
Question words
Nouns
Verbs
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
36
Sample role play buildup:
instructions to students
1. Write down the question words you will
need to ask
2. Write down possible answers to these
questions
3. Interview someone in the class
4. Write down the answers you hear
5. Let your partner interview you
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
37
Sample role play buildup:
instructions to students
6. Decide with your partner which questions
are the most appropriate
7. Decide with your partner which role each
of you will play
8. Practice acting out with your partner
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
38
And the final step
Now, act out
your role play.
Teacher uses
assessment tool
to evaluate the
student product
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
39
Content
Accuracy
Fluency
Excedes
requirements
Meets
requirements
Needs
improvement
Original and
on task:
excellent use
of new
vocabulary
Grammar
usage is
exceptional
Content is
complete
Incomplete
content or
vocabulary
Grammar
usage is
appropriate
Grammar
needs
improvement
Speaks
without
hesitations
Some
hesitations
Halting and
many
hesitations
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
40
5. Assess the speaking task and
incorporate into the overall
assessment framework
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
41
Address the goals of your program
Make known to students the criteria for
assessment and the specific tasks to be
assessed prior to the activity
Apply an appropriate assessment
instrument to the oral task
Review outcomes
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
42
And finally…
Evaluate the
effectiveness of
your assessment
tool
Consider how to
improve your
course or program,
based on results
from assesment
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
43
Programatic global assessment
 Oral interview
 Oral presentation – final project
 Essay – final product
 Listen or read and summarize/evaluate/analyze
orally or in writing
 Portfolio = collections, selections, and reflections
 Standardized test or mock test
Integrative tests tap all skills!
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
44
How can we catch that elusive butterfly?
1. Build a team
2. Determine goals of your assessment
program.
3. Match course objective to the test.
4. Test like you teach, teach like you test!
5. Review sample assessment tools.
6. Adapt tools to meet your program’s
needs
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
45
7. Use the assessment tool with your class.
8. Critique the tool with your colleagues.
 Are you consistent with your colleagues in its
application?
 Are there ways that the tool or the activity
could be improved?
 Should more “buildup” opportunities be
provided to your students?
9. Collaborate and continue to share,
review, adapt, & critique with colleagues.
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
46
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
47
Time to work…
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
48
Your Assignment…
Define a student outcome for oral
production
Clearly define the task that the student will
complete to demonstrate competency
Identify a rubric for use in evaluating the
task. Consult online resources that follow.
Match rubric to task
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
49
Online references
 Rubistar
Adapt rubrics to meet the criteria you determine to
evaluate student products
 http://rubistar.4teachers.org/index.php
 Rubric Bank – Chicago City Schools
Review grading guidelines for speaking, writing activities
 http://intranet.cps.k12.il.us/Assessments/Ideas_and_Rubri
cs/ideas_and_rubrics.html
 Landmark
Create a rubric from scratch
 http://landmarkproject.com/classweb/tools/rubric_builder.php3
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
50
University of Alaska sample assessment
instruments
Scoring guides
http://www.eed.state.ak.us/tls/frameworks/wrldlang/wl
instr3.html
Kenyon County Schools
Assesment tools and links
http://www.kenton.k12.ky.us/tr/rubrics.html
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
51
Discovery School Assessment tools
Comprehensive collection of assessment tools
http://school.discovery.com/schrockguide/assess.htm
l
Fairfax County Virginia Public Schools
Performance Assessment for Language
Students
http://www.fcps.k12.va.us/DIS/OHSICS/forlang/PALS
/rubrics/
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
52
References
Bachman, Lyle F. 1990. Fundamental
considerations in language testing. New
YOrk: Oxford University Press
Oller, John W. 1980. Language tests at
school. New York: Longman
O’Malley, J. M. and L. Valdez Pierce
(1996). Authentic Assessment in English
Language Learners. New York; Addison
Wesley.
M. Karen Jogan, Ph.D. [email protected] Peru
TESOL 2006
53
Descargar

Catching that elusive butterfly: assessing speaking in