Science Assessment:
Teaching and Assessing
Science Core Concepts

What We Know About Successful Programs

TASC2 Workshops

Legislative & OSPI Update

Crisis in Science Education

Science EALRs, WASL, & NAEP

OSPI Science Assessment Resources and
Contact Information
1
What We Know About
Successful Programs
 Districts need a Science Plan and
Leadership for Science
 Teachers must understand the
core concepts and appropriate
best practices
 Teachers need research-based
instructional materials with PD
 Instructional materials need a
focus on core concepts; and
diagnostic, formative, &
summative assessment
enhancements
2
Teaching and Assessing Science Core
Concepts: TASC2 Workshops
TASC2 workshops are offered by OSPI Science Assessment
to model how to use the WASL released materials.
• Seven different TASC2 workshops at OSPI Jan. Conf.
• Three hour workshops for subject/grade specific small
groups of teachers
• Based upon a modification of the unit development
process described in Understanding by Design
Phase I: Establish a core concept as the desired outcome of a
targeted instructional unit
Phase II: Use WASL releases as assessment targets to design
diagnostic, formative, and summative custom assessment tools
Phase III: Enhance the learning plan from the instructional unit
to focus on the core concept and use the custom assessment tools
3
Teaching and Assessing Science Core
Concepts: TASC2 Workshops
Phase I: Core Concept Activity
“You can’t teach what you don’t know”
1. Focus on a systems core concept from the state science
standards (today’s EARLS, GLEs, & WASL EoL)
e.g. HS Energy Transfer: Analyze energy transfers and
transformations within a physical system, including energy
conservation by determining the energy input to an object as work
and explaining how a machine transfers work and/or transforms
force and distance through a force-distance trade-off (ST02 f & g)
4
Teaching and Assessing Science Core
Concepts: TASC2 Workshops
Phase I: Core Concept Activity
(continued)
2. Review and discuss the research on the core concept
through readings identified in Curriculum Topic Study
such as:
a. “As far as physicists are concerned, work is done every time a
force is used to move something …”
Research on Adult Content Knowledge: Science Matters
b. “Work, in the specialized sense used in physics, is often
considered a useful, even necessary, concept for dealing with
ideas of energy …”
Research on Instruction Implications: Benchmarks for Science
Literacy
5
Teaching and Assessing Science Core
Concepts: TASC2 Workshops
Phase I: Core Concept Activity
(continued)
3. Use the activity from a released science WASL scenario to
answer a series of essential questions designed to probe
teachers’ adult and grade level understanding of the core
concept.
e.g.
•
•
•
What are the parts of the Lifting a Load lever system?
How can one square block lift three square blocks?
What is the relationship between the vertical distance a square
block moves down compared … ?
“I never thought to measure the distance an object is lifted by a lever and
finding the work done on that object”
Physics Teacher during a TASC2 Workshop
6
Teaching and Assessing Science Core
Concepts: TASC2 Workshops
Phase II: Assessment Activity
“You need to measure what students learn in an authentic,
valid, and reliable manner”
1. Experience a fleshed-out WASL released scenario focused
on the targeted core system concept.
e.g. Lifting a Load, a 2007 HS application scenario focused on
energy transfer.
Problem: Lift a heavy box up 2 meters.
Gather Information, Explore Ideas, Plan Summary, Steps to do the
Plan, Diagram of Solution, Test Solution, Test Results
7
Teaching and Assessing Science Core
Concepts: TASC2 Workshops
Phase II: Assessment Activity
(continued)
2. Discuss the scoring guide, with student data, for the ten
items of the scenario.
a. How many student correctly answer the multiple-choice
questions? What distractors attracted many students and why
b. How does one use the rubrics guide to validly and reliably
score the short answer items? How did students respond? Why?
c. How did student respond to the extended response item on
scientific design process? Hoe does their response compare to the
only other type of 4-pt science WASL item?
8
Teaching and Assessing Science Core
Concepts: TASC2 Workshops
Phase II: Assessment Activity
(continued)
3. Discuss how use the items to develop custom diagnostic,
formative, and summative assessments for the unit
such as:
a. How can multiple-choice question 1 be used to guide
diagnostic and formative assessments to gather evidence of
learning about the potential and kinetic energy of objects in a
lever system?
b. How can multiple-choice questions 2 and 3 be used to guide
diagnostic and formative assessments to gather evidence of
learning about how to determine the work done on an object?
9
Teaching and Assessing Science Core
Concepts: TASC2 Workshops
Phase II: Assessment Activity
(continued)
3. Discuss how use the items to develop custom diagnostic,
formative, and summative assessments for the unit
such as: (continued)
a. How can short answer (SA) items be modified for classroom
diagnostic, formative, or summative assessments?
For example: Write a conclusion SA items can be modified to use
as a stand-alone, MC with explanation, or a bubbled choice with a
written response
b. How can extended response (ER) items be modified for
classroom diagnostic, formative, or summative assessments?
10
Teaching and Assessing Science Core
Concepts: TASC2 Workshops
Phase III: Enhance Learning Plans of targeted
instructional materials by answering these questions.
W: Help student know Where the unit is going and What is expected?
Help the teacher know Where the students are coming from (prior
knowledge, interested)?
H: Hook all students and Hold their interest?
E: Equip students, help them Experience the key ideas and Explore
the issues?
R: Provide opportunities to Rethink and Revise their understandings
and work?
E: Allow students to Evaluate their work and its implications?
T: Be Tailored (personalized) to the different needs, interests, and
abilities of learners?
O: Be Organized to maximize initial and sustained engagement as
well as effective learning?
11
Teaching and Assessing Science Core
Concepts: TASC2 Workshops
Phase III: Enhance Learning Plans (continued)
Use OSPI Assessment writing templates:
 New Science Instructional Material Enhancement
(SIME) templates to write custom classroom
summative assessments
 Templates for modifying standard WASL items for
classroom diagnostic, formative, and easy-to-score
summative assessment tools
12
Legislative Update
Federal NCLB: Science may be part of AYP
State Legislation
1. HB 1128: Appropriations Bill ($39.6 million for science)
a. Additional District PD days: 2 for all 4th & 5th
grade teachers, 3 for all grade 6-12 science teachers, and 5
for a secondary building science lead ($1,500 support); PD
days for science assessment literacy and to implement new
standards. Talk to your district.
b. Quadruple LASER funding www.wastatelaser.org
c. ESD Science Specialists 2008-09
13
Legislative Update
2. HB 1906: Strengthen Math & Science Standards
because too few students are proficient
a. State Board of Education (SBE) to recommend
revisions to science standards via Science
Advisory Panel. www.sbe.wa.gov
• Consultant: David Heil & Associates with a team including
Rodger Bybee (was BSCS) & Harold Pratt (was U of Pittsburgh)
• 19 member State Panel selected from applicants to represent state’s
diverse interests and experience while giving formal feedback
• Interim report March 2008
• Public feedback in April 2008
• Final recommendation to OSPI by June 2008
14
Legislative Update
2. HB 1906: Strengthen Math & Science Standards
(continued)
b. OSPI (consultant via RFP) to revise science
standards by December 2008.
Note: Mathematics revised standards are to be delivered to
OSPI in Jan 2008. The mathematics standards are being
revised by the Dana Center of University of Texas through a
plan the center developed.
c. OSPI (consultant via RFP) to recommend three
science instructional materials by June 2009.
Note: This has not yet happened in mathematics.
15
Legislative Update
2. HB 1906: Strengthen Math & Science Standards
(continued)
d. 25 secondary science coaches 2008-09
Note: OSPI to accept district proposals & provide training.
e. Science proficiency for graduation in 2013
Note: These students are this year’s 7th graders.
f. Study end-of-course tests for science & math
Note: The governor requested SBE to commission the study.
Report in Jan 2008.
16
OSPI Science & Assessment Update
1. New Assessment Contract Spring 2008 through 2013
a. Science WASL modified for revised standards in 2010.
b. Possible new enhancements to
the entire assessment system:
•
•
•
•
•
•
end-of-course tests in high school mathematics & science
translated into 6 languages for ELL students
accommodations for visual impairments, ADHD,
& reading disabilities
diagnostic assessments
classroom-based assessments
modified WASL items for WAAS
17
OSPI Science & Assessment Update
2. State Science Strategic Planning Team
a. Cary Sneider (consultant lead)
b. ~30 Members (many member represent more than one area)
•
•
•
•
•
•
•
•
•
•
OSPI: Mary McClellan (lead)
LASER & LASER Alliances: Dennis Schatz (lead)
Universities
Educational Service Districts
School Districts
Partnership for Learning representing business
Informal Educational Institutions
State Board of Education
Washington Science Teacher Association
Professional Educators Standards Board
c. Preliminary Plan
presented at OSPI January Conference
18
Crisis in Science Education
For 5 yrs, 2 of 3 students did
NOT understand science.
What will be different
in the next 5 yrs?
100.0
90.0
80.0
G8 2005 NAEP
70.0
10.0
0.0
04
03 04
03 04 05 06 07 08 09 10 11 12 13
05
Grade
5
06 07
05
Grade
8
100
20.0
06 07
2009 NAEP
30.0
44.6
40.0
36.4
35.0
35.2
32.2
31.8
50.0
42.9
35.7
33.0
39.4
35.8
60.0
36.5
35.7
G4 2005 NAEP 29.0
34.9
28.2
Proficient Students on the Science WASL (%)
State Science Proficiencies
High School
Grade 10
19
Crisis in Science Education
2007 Grade 10 Student Performance
Level 1:
Below Basic
Percent of Students
Who Took Test
30.6%
22,924 students
Level 2:
Basic
Level 3:
Proficient
21.4%
33.6%
16,016 students
375 Scale Score
25 of 62 pts
40%
2.5%
25,206students
400 Scale Score
34 of 62 pts
55%
Data Notes from OSPI Website, School Report Card, WASL,
Level 4:
Advanced
1,841 students
458 Scale Score
51 of 62 pts
83%
Detail, 10th
• This data is for the 10th graders who took the High School Science WASL
(74,968 students), not the 9th, 11th, or 12th graders who took the test.
• Included in the Level 1 calculations are 8,882 students (11.8%) who received
no score: unexcused absence/refusal 3,695 students (4.9%), other 5,187
students (6.9%) (no booklet, CSRS but no booklet, incomplete, invalidated, and out of grade level)
• 10th graders not included in this data include - WAAS portfolio: 637 students,
Excused Absence: 5,030 students, Exempt: 1,880 students
• Excluding ‘no score’ students, 41.1% of the 10th graders met standard
20
Crisis in Science Education
2007 Grade 10 Student Performance
Level 1:
Below Basic
Percent of Students
Who Took Test
30.6%
22,924 students
Level 2:
Basic
Level 3:
Proficient
21.4%
33.6%
16,016 students
375 Scale Score
25 of 62 pts
40%
Level 4:
Advanced
2.5%
25,206students
400 Scale Score
34 of 62 pts
55%
1,841 students
458 Scale Score
51 of 62 pts
83%
The Good News
• There were only 9 score points in Level 2 (Basic) this year.
• There is no difference in performance between Basic and Proficient
students on MC questions, on Physical vs. Earth vs. Living system
items, or on content vs. process items.
• Basic students earn an average of ½ a point on 2-pt items.
Proficient students earn an average of 1 point. (11 2-pt items = 22 pts)
• Basic students earn an average of 1 point on 4-pt items.
Proficient students earn an average of 2 points. (3 4-pt items = 12 pts)
21
Crisis in Science Education
22
Crisis in Science Education
Target for the 2011 HS Science WASL
23
Crisis in Science Education
2007 Grade 8 Student Performance
Percent of Students
Who Took Test
Level 1:
Below Basic
Level 2:
Basic
Level 3:
Proficient
23.8%
29.5%
34.9%
18,596 students
23,107 students
375 Scale Score
23 of 62 pts
37%
400 Scale Score
34 of 62 pts
55%
Level 4:
Advanced
9.5%
27,284 students
7,403 students
432 Scale Score
47 of 62 pts
76%
The Good News
• There were only 11 score points in Level 2 (Basic) this year.
• There is no difference in performance between Basic and Proficient students
on MC questions, on Physical vs. Earth vs. Living system items, or on
content vs. process items.
• Basic students earn an average of ½ a point on 2-pt items.
Proficient students earn an average of 1 point. (11 2-pt items = 22 pts)
• Basic students earn an average of 1 point on 4-pt items.
Proficient students earn an average of 2 points. (3 4-pt items = 12 pts)
24
Crisis in Science Education
25
Crisis in Science Education
2007 Grade 5 Student Performance
Percent of Students
Who Took Test
Level 1:
Below Basic
Level 2:
Basic
Level 3:
Proficient
21.2%
41.1%
30.0%
15,716 students
375 Scale Score
18 of 49 pts
37%
30,384 students
22,177 students
400 Scale Score
30 of 49 pts
61%
Level 4:
Advanced
5.9%
4,370 students
427 Scale Score
41 of 49 pts
84%
The Good News
• There were only 12 score points in Level 2 (Basic) this year.
• There is no difference in performance between Basic and Proficient
students on MC questions, on Physical vs. Earth vs. Living system
items, or on content vs. process items.
• Basic students earn an average of ½ a point on 2-pt items.
Proficient students earn an average of 1 point. (10 2-pt items = 20 pts)
• Basic students earn an average of 1 point on 4-pt items.
Proficient students earn an average of 2 points. (2 4-pt items = 8 pts)
26
Crisis in Science Education
27
WA Science Standards
Core Concepts Handout
28
Core Concepts of Physical Systems
• Energy Transfers & Transformations
• Forces to Explain Motion
• Conservation of Matter (& Energy)
Core Concepts of Earth & Space Systems
• Components & Patterns of Earth Systems
• Processes & Interactions in Earth Systems
• Interactions in the Solar System & Beyond
Core Concepts of Living Systems
• Life Processes & the Flow of Matter & Energy
• Interdependence of Life
• Biological Evolution
29
Core Concepts
•
Planning & Conducting Safe Investigations
•
Explaining (the results of an investigation)
•
Evaluating Methods of Investigations
Core Concepts
•
Designing and Testing Solutions
•
Evaluating Potential Solutions
•
Environmental and Resource
Issues
30
WA Core Concepts
2009 NAEP Framework
Physical Systems
Physical Science
Energy Transfers
Conservation of Matter
Forces to Explain Motion
Earth & Space Systems
Components & Patterns of ES
Patterns & Interactions of ES
Interactions in the Solar System
Living Systems
Life Process, Matter & Energy
Interdependence of Life
Biological Evolution
Inquiry in Science
Planning, Explaining, & Evaluating
Investigations
Application of Science
Designing & Evaluating Solutions
Environmental Issues
Energy
Matter
Motion
Earth & Space Science
Earth Structure
Earth System
Earth in Space & Time
Life Science
Structures & Functions of Living Systems
Changes in Living Systems
Note: All test items are coded as science content.
Science Practices
Identifying Science Principles
Using Science Principles
Using Scientific Inquiry
Using Technological Design
Note: Each test item for a particular science
content is also classified as one science practice.
31
Science WASL
Scenario Test Map
Point-balanced in Physical, Earth/Space, & Living Systems
1
Investigating a Physical System
2
Investigating an Earth/Space System
3
Investigating a Living System
4
Designing a Solution
5
Analyzing a System
6
Analyzing a System
Also, there is a Pilot Scenario with 5 items toward the end of a session.
Science WASL Items & Points
Grade Five Exam
Multiple Choice
Number
of Items
21
Total
Points
21
Percent of
Total Points
43%
Short Answer
10
20
41%
2
8
16%
33
49
100%
Type of Item
Extended Response
Total
High School and Grade Eight Exam
Type of Item
Number
of Items
Total
Points
Percent of
Total Points
Multiple Choice
Short Answer
Extended Response
28
11
3
28
22
12
45%
36%
19%
Total
42
62
100%
OSPI Science Assessment Enhancements
and Professional Development
Powerful Classroom Assessment (PCA) Workshops
 3-4 hour workshops for teachers
 Build understanding of a type of scenario and items
 Build understanding of scoring the Science WASL
 Certified SALTers as presenters
Grade 5 PCAs: Scenario, Scoring Guide, 10 annotated
Student Responses for each of the 3 open-ended items
Better Boiling: Physical System Inquiry 2007 WASL
Weather Wonders: Earth System 2007 WASL
Let Us Grow: Living System 2006 WASL
The Birds: Living System Application 2006 WASL
Hold That Soil: Earth System Inquiry 2005 WASL
Grass is Always Greener: Living System Inquiry 2004 WASL
*Acid Rain: Living System Inquiry STC
34
OSPI Science Assessment Enhancements
and Professional Development
Grade 8 PCAs: Scenario, Scoring Guide, 10 annotated
Student Responses for each of the 3 open-ended items
Wild About Radishes: Living System Inquiry 2007 WASL
Soccer Soaker: Earth System Application 2007 WASL
Respiratory System: Living System 2007 WASL
What’s Your Angle: Earth System Inquiry 2006 WASL
Clean Water: Physical System Application 2006 WASL
School Garden: Living System Application 2005 WASL Pilot
In the Doghouse: Physical System Inquiry 2005
Bubble Gum vs. Chewing Gum: Living System Inquiry 2003
*Electricity to Light: Physical System Inquiry STC/MS
35
OSPI Science Assessment Enhancements
and Professional Development
High School PCAs: Scenario, Scoring Guide, 10 annotated
Student Responses for each of the 3 open-ended items
Sliding Away: Earth System Inquiry 2007 WASL
Lifting a Load: Physical System Application 2007 WASL
Aquarium Systems: Living System Inquiry 2006 WASL
Super Grow: Living System Inquiry 2004 WASL
*Mousetrap Cars: Physical System Application NSTA Press
PCA Writing Templates for each grade and type of scenario
http://www.k12.wa.us/assessment/WASL/Science/ClassroomAssessments.aspx
Science Instructional Material Enhancement (SIME)
Scenario Writing Templates for each grade and type of
scenario with a focus on core concept items
36
Don’t Treat Your Students Like Lab Mice
AS A WASL PREP “
37
OSPI Science Assessment Team
Roy Beven
Andy Boyd
[email protected]
[email protected]
Assessment Manager
Cinda Parton
Assessment Specialist
[email protected]
Assessment TOSA (PEM)
Jonathan Frostad
Assessment TOSA (PEM)
[email protected]
Molly Becker Miller
Assessment Support
[email protected]
Science Assessment Leadership Team (SALT)
The 160+ members of this team have developed deep understanding of
assessment and are available to provide PD. Contact information will be
on the science assessment webpage soon.
OSPI Science Assessment Webpage
www.k12.wa.us/assessment/WASL/Science
38
Science Assessment:
Teaching and Assessing
Science Core Concepts
 What We Know About Successful Programs
 TASC2 Workshops
 Legislative & OSPI Update
 Crisis in Science Education
 Science EALRs, WASL, & NAEP
 OSPI Science Assessment Resources and
Contact Information
39
Descargar

Science WASL is