Increasing Reading
Comprehension
Achievement through
Item Analysis
Monday, February 11, 2008
Auditorium III
2:15 – 3:15 p.m.
Future Ready Schools
Reading Achievement
• State
end-of-grade reading
comprehension scores have
been flat in recent years.
• NAEP reading comprehension
scores have been declining in
recent years.
Future Ready Schools
Reading Achievement
2005, North Carolina 8th
graders fell below the national
average on NAEP.
• The State Board of Education
proposed literacy coaches at
the middle grades in response
to this concern.
• In
Future Ready Schools
1992-93 to 2006-07 EOG General Multiple-Choice Statewide Test Performance Summary for
Grades 3-8, Grade 4, and Grade 8 Reading
100.0%
95.0%
88.6% 88.9% 88.6%
90.0%
85.2%
83.3%
79.5% 79.9% 79.9%
80.0%
74.9% 75.1%
75.0%
79.5%
84.9% 85.3%
85.7% 86.8%
87.6%
87.6%
85.4%
83.7% 83.7% 83.5%
77.1%
Grades 3-8
73.0%
75.0%
77.1%
Grade 4
72.7%
71.1% 72.8%
74.6%
69.0%
69.4%
72.1%
70.0% 66.5%
67.1%
70.9% 71.4%
65.8%
68.0% 67.7%
65.0% 63.4%
65.3%
64.2%
60.0% 62.4%
Grade 8
55.0%
20
06
-07
20
05
-06
20
04
-05
20
03
-04
20
02
-03
20
01
-02
20
00
-01
19
99
-00
19
98
-99
19
97
-98
19
96
-97
19
95
-96
19
94
-95
19
93
-94
50.0%
19
92
-93
Percent Proficient
85.0%
87.7%
89.8%
Year
Future Ready Schools
North Carolina & Nation (Public) Grades 4 & 8 Reading NAEP Results
1992-2007 Average Scale Scores
320
3 10
300
Average Scale Score
290
280
2 70
260
2 50
Na t i on Gr a de 8
262*
2 6 5*
262
260*
261
261
263*
261
2 58
2 59
NC Gr a de 8
240
230
220
2 10
2 15*
2 14 *
2 13 *
2 12 *
2 12 *
2 13 *
222*
2 2 1*
2 17*
2 16 *
Na t i on Gr a de 4
220
2 18
2 17
2 17*
NC Gr a de 4
200
19 0
18 0
170
19 9 2 **
19 9 4 **
19 9 8
2002
2003
2005
2007
*Val ue i s si gni f i cantl y di f f er ent f r om the val ue i n 2007.
**Accommodati ons wer e not per mi tted f or thi s assessment.
NOT E: T he NAEP M athemati cs scal e r anges f r om 0 to 500. Obser ved di f f er ences ar e not necessar i l y stati sti cal l y si gni f i cant. Detai l may not sum to total s because of r oundi ng.
SOURCE: U.S. Depar tment of Educati on, Insti tute of Educati on Sci ences, Nati onal Center f or Educati on Stati sti cs, Nati onal Assessment of Educati onal Pr ogr ess (NAEP), 1992, 1996, 2000, 2003,
Future Ready Schools
Levels of Thinking and Reasoning
(Adapted from Marzano, et al, 1988)

Knowledge

Organizing

Applying
Usually lower order
thinking skills
______________

Analyzing

Generating

Integrating

Evaluating
Usually higher order
thinking skills
Future Ready Schools
Knowledge
• Involves
recall of a fact
• Example:
Where do the
insects live?
Future Ready Schools
Organizing
• Involves
the order of
information, comparing and
contrasting, or classifying
information
• Example: How does Joe’s job
differ from Jim’s job?
Future Ready Schools
Applying
• Involves
demonstrating prior
knowledge within a new
situation
Which statement
from the selection is true?
• Example:
Future Ready Schools
Analyzing
• Involves
clarifying information,
determining characteristics or
parts, identifying relationships and
problems, and identifying main
ideas
• Example: Which words help create
the somber mood?
Future Ready Schools
Generating
• Involves
producing new
information, inferring, predicting,
or explaining something by
adding examples
• Example: How would this
experience help Sally the next
time she goes camping?
Future Ready Schools
Integrating
• Involves
connecting and
combining information and
summarizing
• Example: Which experience
is most similar to Tim’s
experience in the selection?
Future Ready Schools
Evaluating
• Involves
judging the quality
of ideas, determining if ideas
are valid, establishing
criteria, etc.
• Example: Why did the author
begin the selection with an
untrue statement?
Future Ready Schools
Thinking Skills Activity
the passage – Ellis
Island: Doorway to America.
• Work in small groups to
analyze the questions and
determine the appropriate
thinking skill for each question.
• Read
Future Ready Schools
Item Results
Thinking Skill – Analyzing (13)
• North Carolina – 89%
• National Public – 91%
• Below Basic – 210
•
Future Ready Schools
Item Results
Thinking Skill – Knowledge
• North Carolina – 87%
• National Public – 89%
• Below Basic – 228
•
Future Ready Schools
Item Results
Thinking Skill – Applying
• North Carolina – 81%
• National Public – 82%
• Below Basic – 237
•
Future Ready Schools
Item Results
Thinking Skill – Analyzing (11)
• North Carolina – 72%
• National Public – 75%
• Basic – 253
•
Future Ready Schools
Item Results
Thinking Skill – Generating (15)
• North Carolina – 54%
• National Public – 59%
• Basic – 267
•
Future Ready Schools
Item Results
Thinking Skill – Generating (16)
• North Carolina – 32%
• National Public – 39%
• Proficient – 318
•
Future Ready Schools
Strategies
• Ask
higher order thinking
questions in instruction.
• Utilize the higher order questions
in textbook supplementary
materials.
• Provide classroom assessments
with higher order questions.
–
Open-ended as well as multiple-choice
Future Ready Schools
Strategies
• Encourage
students to ask
higher order questions of
text.
– Have
students write questions
for each other.
– Independent reading and
journaling
Future Ready Schools
Strategies
• Teachers
collaborate with
peers to generate higher order
questions for instruction and
assessment
– Assists
with internalizing thinking
skills
– Assists with identifying the higher
order aspects of the text
Future Ready Schools
Strategies
• Change
lower order
questions into higher
questions by rewriting the
question.
• Examples…
Future Ready Schools
Improving Lower Order
Questions
Lower Order Question:
•
What happened to Johnny when he bought
the bicycle?
•
Knowledge because it is stated in the selection.
Higher Order Question:
•
How will this experience most likely
change what Johnny does the next time he
buys a bicycle?
•
Generating because it involves making a prediction
about what will happen next.
Future Ready Schools
Improving Lower Order
Questions
Lower Order Question:
•
What is the personification in
paragraph 2?
•
Knowledge question
Higher Order Question:
•
What is the effect of the use of
personification in paragraph 2?
•
Analyzing question because it requires the student
to not only recognize personification, but also to
analyze the effect it has on the paragraph.
Future Ready Schools
Resources
•
•
•
NAEP Data Explorer –
http://nces.ed.gov/nationsreportcard/nde
NAEP Questions Tool –
http://nces.ed.gov/nationsreportcard/itmrls
NAEP Item Maps –
http://nces.ed.gov/nationsreportcard/itemmaps
•
•
NC EOG Sample Items –
http://www.ncpublicschools.org/accountability/
testing/eog/sampleitems/reading
Marzano, R.J., Brandt, R.S, Hughes, C.S.,
Jones, B.F., Presseinsen, B.Z., Stuart, C., &
Suhor, C. (1988). Dimensions of Thinking.
Alexandria, VA: Association for Supervision
and Curriculum Development.
Future Ready Schools
Questions
Future Ready Schools
Increasing Reading
Comprehension
Achievement through
Item Analysis
Tammy Howard
ELA Test Development Consultant
Marcie Hickman
NAEP State Coordinator
Future Ready Schools
Descargar

Parent Education and Empowerment