Classroom Implications
of the ACTFL Proficiency
Guidelines
Ray T. Clifford
ALTA
23 March 2006
Seven Major “Implications”
1.
2.
3.
4.
5.
6.
7.
Complexity of Language
Relative Language Difficulty
Time on Task
Student Characteristics
Curriculum Content
Types of Instruction
Assessment Tools
Seven Major “Implications”
1. Complexity of Language
2.
3.
4.
5.
6.
7.
Relative Language Difficulty
Time on Task
Student Characteristics
Curriculum Content
Types of Instruction
Assessment Tools
1. Language is the most complex
of human behaviors.
Read aloud and explain the
“St.” Rule:
ST. PAUL ST.
Read aloud and explain the
“St.” Rule, Part 2:
ST. COLLEGE ST.
Test of Reading Comprehension;
a Newspaper Headline
IRAQI HEAD
SEEKS ARMS
Test of Reading Comprehension;
a Newspaper Headline
DRUNK GETS
NINE MONTHS
IN VIOLIN CASE
Test of Reading Comprehension;
a Newspaper Headline
MSU HONORS STUDENT
ACCUSED OF BEATING
HOUSEMATE WITH BAT
From: Alma Michigan MORNING SUN
Seven Major “Implications”
1. Complexity of Language
2. Relative Language Difficulty
3.
4.
5.
6.
7.
Time on Task
Student Characteristics
Curriculum Content
Types of Instruction
Assessment Tools
2. Relative Language Difficulty
Determining Relative Difficulty
• The relative difficulty of learning a second
language can be estimated by considering
the “distance” and the “direction” between
the language of the learner and language to
be learned.
Determining Difficulty (Cont.)
• Distance can be estimated by noting the
amount of contrast between L1 and L2 in:
–
–
–
–
–
Grammatical structures
Lexicon
Cultural references
Orthography
Pronunciation
• Even a rating scale of “similar = 1,”
“somewhat different = 2,” and “very different
= 3” can be used.
• The total score = relative “difference.”
Determining Difficulty (Cont.)
• Direction is determined by whether the
features found in the L2 are more complex or
simpler than the same concepts in the L1.
– Moving from a language with reduced
grammatical forms to a language with more
complex forms is going “uphill.”
– Moving from a language with complex
grammatical forms to a language with reduced
forms is going “downhill.”
– It is easier to go “downhill” than “uphill.”
English and Other Languages
• Word order
– English:
• subject / verb / object
– Korean:
• subject / object / verb
– Your language?
English and Other Languages
• Gender and plurals
– English:
• Gender for animate and some inanimate objects
• Plurals for most nouns
– Korean:
• No gender
• No plurals
– Your language?
English and Other Languages
• Implied subjects
– English:
• Generally only found in commands.
– Korean:
• Subject must be identified from the context.
• Context is often established in a separate topic
sentence.
• Honorifics often identify the intended subject.
– Your language?
English and Other Languages
• Morphology
• English:
– Distributive grammar.
– Simple, redundant grammatical forms.
• Korean:
– Complex grammatical forms, for instance:
» Subjects marked by 2 different particles
» Each subject marker has 2 phonetic variants
» Objects are also marked.
» The object marker also has two phonetic variants.
• Your language?
English and Other Languages
• Tenses
• English:
– Past, present, and future verb forms.
• Korean:
– Past and present tenses.
– A tentative state or condition rather than a future tense.
• Your language?
Determining Difficulty (Cont.)
• For L1 speakers of English
– The impact of language distance and direction
has been quantified using “average time-toproficiency” results at the Defense Language
Institute (DLI).
– At DLI, students of Korean take approximately
three times as long to acquire Level 2
proficiency as do students of Spanish.
Implications for Academe
• Typical government program.
– Proficiency of graduates is measured.
– Graduates are expected to attain the same levels
of proficiency regardless of language difficulty.
– The length of each language program is
determined by the relative difficulty of the
language for English speakers.
• Typical college program.
– Student proficiency results are not measured.
– The course requirements for a language major
are about the same – regardless of the language.
Seven Major “Implications”
1. Complexity of Language
2. Relative Language Difficulty
3. Time on Task
4.
5.
6.
7.
Student Characteristics
Curriculum Content
Types of Instruction
Assessment Tools
3. Time on Task
“Some considered projections” from 1961
THE NATIONAL INTEREST AND
FOREIGN LANGUAGES
Eight- and ten-year sequences of foreign language study
will become common in the public schools.
The better colleges and universities will require
demonstrated proficiency (not high school “units”) in a
foreign language for entrance, and demonstrated proficiency
in a second foreign language (often non-Western) for
graduation.
WILLIAM RILEY PARKER for
THE U.S. NATIONAL COMMISSION FOR UNESCO,
DEPARTMENT OF STATE, 1961.
After four decades, why are Parker’s
projections not yet a reality?
• Lasting improvements are only sustainable when
they result from a recognition of need.
• Norm-referenced grading gave little evidence of
real-world ability gaps.
• With accreditation focused on process rather than
on outcomes, there was no incentive to change.
• Administrators didn’t read the research results
showing that no improvements in teaching
methods or curricula can compensate for the
current lack of “time-on-task” in our educational
system.
International Research on
Learning English
• G. Bonnet, et al. The Assessment of Pupils’ skills
in English in Eight European Countries: 2002.
European Network of Policy makers for the
Evaluation of Education Systems, 2004.
– Students from those nations where there is more contact
with (and more time spent using) English have higher
levels of competence in English.
– In a language-rich environment, time spent using the
language is more important than the teaching methods
used in the classroom.
International Research on
Learning French
• John B. Carroll, The Teaching of French in
Eight Countries (International Studies in
Evaluation V) John Wiley & Sons, New
York, 1975.
– “… the primary factor in the attainment of
proficiency in French (and presumably, any
foreign language) is the amount of instructional
time provided.” [Page 276]
Consider Bloom’s Taxonomy
• Evaluation and persuasion through refined use of
professional, literary, and rhetorical skills.
• Synthesis of known concepts to produce and
comprehend new, abstract, and hypothetical ideas.
• Analysis and definition of factual relationships
through extended, detailed explanations.
• Application of mental and linguistic skills to create
and understand new communications.
• Comprehension and use of words and phrases.
• Memorization of names, labels, facts.
How much time is required (in one’s
first language) to learn to…?
• Evaluate and persuade through refined use of
professional, literary, and rhetorical skills.
• Synthesize known concepts to produce and
comprehend new, abstract, and hypothetical ideas.
• Analyze and define factual relationships through
extended, detailed explanations.
• Apply mental and linguistic skills to create and
understand new communications.
• Comprehend and use words and phrases.
• Memorize names, labels, facts.
ILR Proficiency Level Summary
LEVEL
FUNCTION/TASKS
CONTEXT/TOPICS
ACCURACY
5
All expected of an
educated NS
All subjects
Accepted as an
educated NS
4
Tailor language, counsel, motivate,
persuade, negotiate
Wide range of
professional needs
3
Support opinions, hypothesize,
explain, deal with unfamiliar topics
Practical, abstract,
special interests
2
Narrate, describe, give directions
Concrete, realworld, factual
Intelligible even if
not used to dealing
with non-NS
1
Ask and answer questions, create
with the language
Everyday survival
Intelligible with
effort or practice
0
Memorized
Random
Unintelligible
Extensive, precise,
and appropriate
Errors never
interfere with
communication &
rarely disturb
Language Proficiency and Cognition
• 4/5 (Distinguished): Evaluation and persuasion
through refined use of professional, literary, and
rhetorical skills.
• 3 (Superior): Synthesis of concepts needed to
produce and understand abstract ideas, complex
arguments, and hypothetical discussions.
• 2 (Advanced): Analysis and explanation of realworld relationships using paragraph-length
communications.
• 1 (Intermediate): Application of language skills
to routine interpersonal communication scenarios.
• 0 (Novice): Memorization / Comprehension of
disjointed words and phrases.
How much time is required (in one’s
second language) to learn to…?
• Evaluate and persuade through refined use of
professional, literary, and rhetorical skills.
• Synthesize known concepts to produce and
comprehend new, abstract, and hypothetical ideas.
• Analyze and define factual relationships through
extended, detailed explanations.
• Apply mental and linguistic skills to create and
understand new communications.
• Comprehend and use words and phrases.
• Memorize names, labels, facts.
Average Instructional Time
for 80 percent of DLI Students to Reach Level 2
• Category I (Romance and Scandinavian
languages)
– 25 weeks
– 750 classroom hours
• Category II (Germanic and Asian
languages)
– 32 weeks
– 960 classroom hours
Average Instructional Time
for 80 percent of DLI Students to Reach Level 2
• Category III (Slavic and some Asian
languages.)
– 47 weeks
– 1,410 classroom hours
• Category IV (Arabic, Chinese, Japanese,
and Korean)
– 75 weeks (Current length is 63 weeks)
– 2,250 classroom hours (Currently 1,890 hours)
How Proficient are Today’s Foreign
Language Majors?
• Results of Oral Proficiency Testing
• Official ACTFL OPI’s administered to
foreign language majors
• Tests were conducted face-to-face and
telephonically
• Double rated and certified results through
the ACTFL Testing Office
About the ACTFL Study
• 501 Undergraduates
– Five Liberal Arts Colleges
– Juniors and Seniors
– Foreign language majors
• Data gathered over five years
– 1998-2002
• Six languages
– Spanish, French, Italian, Chinese, Japanese, Russian
A Look at FL Majors’ Oral Proficiency
ACTFL Rating
Number of
Students
% of total
Cumulative %
Superior
12
2%
2%
Advanced High
24
5%
7%
Advanced Mid
95
19%
26%
Advanced Low
105
21%
47%
Intermediate High
175
35%
82%
Intermediate Mid
86
17%
99%
Intermediate Low
4
1%
100%
Novice High
0
100%
Novice Mid
0
100%
Novice Low
0
100%
Total
501
100%
Percentage of FL Majors Who Would
Qualify for the Following Positions
Diplomat - Superior (ILR 4)
Business Executive - Superior
Court Interpreter - Advanced High
2%
7%
Customer Service Rep - Advanced Mid
Social Worker – Advanced Mid
26%
26%
K-12 Teacher - Advanced Low
47%
Receptionist – Intermediate High
Tour Guide – Intermediate Mid
82%
99%
Seven Major “Implications”
1. Complexity of Language
2. Relative Language Difficulty
3. Time on Task
4. Student Characteristics
5. Curriculum Content
6. Types of Instruction
7. Assessment Tools
4. Student Characteristics
Two Major Student Variables
• Aptitude: The amount of time needed to
learn a language.
• Motivation: The amount of time a learner
is willing to spend learning the language.
Impact of Aptitude and Motivation
on Student Learning
Motivation
Low
High
High Average results
Exceptional
results
Low Below average
Average
results
Aptitude
results
Seven Major “Implications”
1.
2.
3.
4.
Complexity of Language
Relative Language Difficulty
Time on Task
Student Characteristics
5. Curriculum Content
6. Types of Instruction
7. Assessment Tools
5. Curriculum Content
• The adequacy of a curriculum must be
judged against some external criteria.
• For a curriculum with a proficiency
orientation, one would consider the …
– Communication tasks included.
– Topical domains “covered.”
– “Text types” of the language samples.
Proficiency Levels with Text Types
LEVEL
FUNCTION/TASKS
CONTEXT/TOPICS
5
All expected of an
educated NS
4
Tailor language, counsel, motivate,
persuade, negotiate
Wide range of
professional needs
3
Support opinions, hypothesize,
explain, deal with unfamiliar topics
Practical, abstract,
special interests
2
Narrate, describe, give directions
1
Q & A, create with the language
Everyday survival
Sentences
0
Memorized, rehearsed language
Random
Words/phrases
All subjects
Concrete, realworld, factual
TEXT TYPES
Books
Chapters
Pages
Paragraphs
Tasks, content, and text length increase
as one moves up the proficiency scale.
5
4
3
2
1
How can proficiency scales be used
to design a language curriculum?
• Historically there have been two Options:
– Option 1: Interpret the proficiency scale as a
proscriptive set of stages in the students
development.
– Option 2: Identify the proficiency level the
instructional sequence should lead to and build
a curriculum that leads to that goal.
Option 1
• Build a one-dimensional curriculum that
– Begins with the lowest level in the scale
– Progresses vertically through each level in
sequence.
• Therefore, the first objective is to be sure the
student
– shows little real autonomy of expression,
flexibility, or spontaneity.
– Is difficult to understand.
– Has more errors than correct forms.
Option 2
• Recognize that progress in a proficiency-based
curriculum will have three components:
– Communication tasks.
– Topical domains.
– Accuracy of communication.
Option 2
• Build a curriculum that moves horizontally
from “known to unknown” on one dimension
at a time.
– Begins with level-appropriate tasks.
– Uses limited topical domains.
– Reenters those domains to introduce and learn new
tasks.
– Reenters known tasks to add new topical domains.
Option 2
• At the macro level, insure that the
curriculum…
– Keeps the end accuracy goal in mind.
– Provides sufficient time and practice for the
learners to satisfy the requirements of each level
before moving to a new set of progress tests for the
tasks and domains associated with the next higher
level.
Option 2
• Reinforce instruction with proficiency-based
progress tests that for each level…
– Assess mastery of the range of tasks and topical
domains within that level.
– Check for targeted accuracy expectations within
those defined performance areas.
• It often won’t be there.
• The expectation will reinforce the long-range
instructional goal.
Seven Major “Implications”
1.
2.
3.
4.
5.
Complexity of Language
Relative Language Difficulty
Time on Task
Student Characteristics
Curriculum Content
6. Types of Instruction
7. Assessment Tools
6. Types of Instruction
Professional Disciplines
Have Levels of Expertise
• Apprentice
• Journeyman
• Master
Professional Teachers’ Abilities
• Apprentice teachers
– Know what to teach.
• Journeyman teachers
– Know what to teach and how to teach.
• Master teachers
– Know what to teach, when to teach what, how to teach,
and why.
– Also know what they don’t know and are continuously
learning.
A Common Model
Used by Apprentice Teachers
1.Analysts
list high
frequency
language
tasks.
The real
language
2. Textbook writers
include the most
important items
in a textbook.
3. Teachers
present the
textbook.
4. Students
demonstrate
their mastery
of sample
items drawn
from the
textbook.
Textbook
Teaching
Test
The Education Model
Used by Master Teachers
1. Language Needs
Assessments define
the Real-world
Instructional
Real-world Instructional
Domains.
Domains: cognitive
understanding,
psychomotor skills, and
affective insights.
2b.Test developers use a
sample of the real-world
domain areas to create
proficiency tests that are
independent of the
textbook.
Test
Textbook
Teacher
Students
2a.Course developers sample
from the real-world domain areas
to create a textbook.
3. Teachers adapt text materials to
learners’ abilities, diagnose
learning difficulties, adjust
activities and add supplemental
materials to help students apply
new knowledge and skills first in
constrained achievement and
performance areas, and then in
real-world settings.
4. Students practice, expand, and
then demonstrate their
unrehearsed extemporaneous
abilities across a broad range
of real-world settings that are
not in the textbook.
Instructional Methods
• The adequacy of instructional methods can
be judged against four criteria.
– The types of instructional activities used.
– Whether those activities progress from the
current level of the learner to the level targeted
as a learning outcome.
– Where the targeted activities fall on the
proficiency scale.
– The effectiveness of feedback provided the
learner
Students Won’t Learn to Do What
They Haven’t Been Taught to Do
LEVEL
FUNCTION/TASKS
CONTEXT/TOPICS
5
All expected of an
educated NS
4
Tailor language, counsel, motivate,
persuade, negotiate
Wide range of
professional needs
3
Support opinions, hypothesize,
explain, deal with unfamiliar topics
Practical, abstract,
special interests
2
Narrate, describe, give directions
1
Q & A, create with the language
Everyday survival
Sentences
0
Memorized, rehearsed language
Random
Words/phrases
All subjects
Concrete, realworld, factual
TEXT TYPES
Books
Chapters
Pages
Paragraphs
Instructional Activities Hierarchy*
E. Communication in unrehearsed, real-world
settings followed by formative feedback.
D. Semi-structured applications of learner
skills with formative feedback.
C. Learner practice with formative feedback.
B. Guided practice exercises.
A. Presentation of the language to be learned.
* Only instructional programs with activities at levels D and E
can lead to Advanced (Level 2) or higher proficiency.
Feedback in the Language Acquisition Process
Language Input
Diagnosis and
Feedback
Learner Output
Apperception and
Capture In Sensory
Store
Transfer to Long-Term Memory
and internalization in Learner's
Linguistic Expectancy System
(This Process is Accelerated by
L2 Production}
Combination of Semantic and
Syntactic Features Linked to Prior
Knowledge Lead to
Comprehension
Intake Into ShortTerm Memory
Seven Major “Implications”
1.
2.
3.
4.
5.
6.
Complexity of Language
Relative Language Difficulty
Time on Task
Student Characteristics
Curriculum Content
Types of Instruction
7. Assessment Tools
7. Assessment Tools
Four Dimensions of
Language Testing
•
•
•
•
What type of test do we want?
For whom are we testing? (Who will be
getting the results?)
What is the purpose of the tests? (How
will the results be used?)
Who will take responsibility for test
production, maintenance,
administration, and scoring?
What type of test do we want?
• Achievement = Rehearsed or memorized
responses using the content of a specific
textbook or curriculum.
• Performance = Semi-rehearsed ability to
communicate in specific, familiar settings.
• Proficiency = Unrehearsed general ability
to accomplish communication tasks across
a wide range of topics and settings.
Proficiency scales assess unrehearsed ability.
• Broad, thorough elicitation techniques are applied
to identify “hot house” versus sustainable, general
ability.
• For instance,
– At level 1, the learner must create new utterances, not
just recite dialogs.
– At level 2, the learner must handle new work
requirements, not just routine communications.
– At level 3, the learner must hypothesize and defend
opinions in subjects beyond one’s personal interests or
areas of specialization.
• Proficiency indicates transferable skills - not
rehearsed performance in familiar areas.
Contrasts Between Proficiency
And Rehearsed Performance
Unrehearsed
Rehearsed
Proficiency
Performance
Task: Wide range of abilities
Specific Job Skills
Context: Broad, in-depth, variable Focused, restricted
Accuracy: Ascending expectations Situation dependent
Applied Performance
Can be a Subset of Proficiency
Proficiency
Performance
Applied Performance
Can be Distinct from Proficiency
Performance
Proficiency
Applied Performance
Often Overlaps with Proficiency
Performance
Proficiency
For whom are we testing?
(Who will be using the results?)
•
•
•
•
Students
Teachers
Program administrators
Prospective employers
What is the purpose of the tests?
(How will the results be used?)
•
•
•
•
•
Progress checks
Placement
Diagnosis
Program assessment
Selection decisions
Who will take responsibility for
test production, maintenance,
administration, and scoring?
• A classic “buy or build” decision
– What tests exist?
– What needs are being met?
• When needs are not being met, who…
– Has the resources (funding, expertise, and time) to
create the needed tests?
– Can operate a testing center to maintain,
administer, and score the tests once developed?
Where do progress, placement, assessment,
& hiring “purposes” fit?
Achievement
Students
Teachers
Programs
Employers
Progress
Progress
Student
placement
NA
Hiring
Hiring
Performance
Progress
Progress
Student
placement /
Program
assessment
Proficiency
Progress?
Progress?
Program
assessment
So what?
A Self Assessment Quiz
• For each of the following examples, pick
the appropriate test to meet your needs:
a. Achievement
b. Performance
c. Proficiency
Which test type would you pick:
Achievement, Performance,
or Proficiency?
• To assess students’ language learning
after Chapter 3 of a Language 101
course?
Which test type would be best:
Achievement, Performance, or Proficiency?
• To assess students’ language learning after Chapter 3 of a Language
101 course?
• To place students into a university
course sequence?
Which test type would be best:
Achievement, Performance, or Proficiency?
• To assess students’ language learning after Chapter 3 of a Language
101 course?
• To place students into a university course sequence?
• To test students completing an
advanced course?
Which test type would be best:
Achievement, Performance, or Proficiency?
• To assess students’ language learning after Chapter 3 of a Language
101 course?
• To place students into a university course sequence?
• To test students completing an advanced course?
• To screen job applicants for a specific
job?
Which test type would be best:
Achievement, Performance, or Proficiency?
• To assess students’ language learning after Chapter 3 of a Language
101 course?
• To place students into a university course sequence?
• To test students completing an advanced course?
• To screen job applicants for a specific job?
• To document attainment of a given
level of sustainable, general ability in a
language?
Which test type would be best:
Achievement, Performance, or Proficiency?
• To assess students’ language learning after Chapter 3 of a Language
101 course?
• To place students into a university course sequence?
• To test students completing an advanced course?
• To screen job applicants for a specific job?
• To certify attainment of a given level of sustainable, general ability in a
language?
• To qualify prospective FL teachers for
certification?
What type of test(s) does ALTA want?
• There is no single test that will satisfy all of
your assessment needs.
• Care must be taken when deciding what to
test and how to test it.
–
–
–
–
What is to be tested?
Who is the audience or consumer of the results?
What is the purpose of the assessment?
Who can do the testing?
• And remember: “Whatever you test, you’ll
get more of it.”
Application Activities
Practical Exercise 1:
Assessment and Analysis
• Assess the current level of your
students using the “green matrix”
handout.
Practical Exercise 1:
Assessment and Analysis
• Assess the level of your students using
the “green matrix” handout.
• What would be the next step to increase
students’ proficiency? Should it focus
on increasing their…
–
–
–
Ability to perform communication tasks?
Knowledge of topical domains?
Accuracy of communication?
Practical Exercise 2:
Designing Classroom Tests
• How could your classroom tests show
students their progress toward a goal of
real-world proficiency?
– Give credit for accomplishments.
– Point toward higher goals, by reinforcing
the progression of abilities:
• Achievement (rehearsed).
• Performance (semi-rehearsed, familiar).
• Proficiency (unrehearsed, new situations).
Practical Exercise 3:
Grading
• Nothing defines course goals and
objectives like a test!!
• Grading should reinforce both your
immediate and your long-range
goals.
• Grades should be use useful for the
student.
Practical Exercise 3:
Grading
• Grades should be:
– Timely. (Old news isn’t interesting.)
– Specific. (Avoid the appearance of
using an “EGS” grading approach.)
– Informative. (Provide each student
enough information so s/he knows how
to improve.)
Practical Exercise 3:
Grading
• Using the “white matrix” (ACTFL
OPI Rating Grid) handout, design a
grading protocol that would…
– Consider the natural progression from
achievement to proficiency.
– Provide students with useful,
personalized feedback on their abilities.
Practical Exercise 3:
Grading
• Some possibilities:
– Assign separate grades to each of the factors
in the ACTFL OPI Rating Grid.
– Use a rating scale that shows levels of ability
ranging from unintelligible to fully
comprehensible.
– Create an ability profile by assigning a
separate grade for each rating factor for each
test condition (achievement, performance,
and proficiency).
Congratulations!!!
• You can now have a proficiency-oriented
classroom without finding a (new) textbook.
• How you teach and how you test will make the
difference.
– Professional teachers are not just “script readers”
who deliver a textbook.
– Proficient students can use their language skills in
both rehearsed and unrehearsed, real-world
situations.
Descargar

Proficiency, Performance, or Achievement Tests