Predictive validities: SAT and PSU
(faculty: Arquitectura)
0.6
0.5
0.4
0.3
0.2
0.1
0
Language
Mathematics
SAT
SAT Writing
PSU Social
Science
PSU Arquitectura
SOURCE: Pearson, Final Report Evaluation of the Chile PSU, January 2013
Predictive validities: SAT and PSU
(faculty: Educacion)
0.6
0.5
0.4
0.3
0.2
0.1
0
Language
Mathematics
SAT
SAT Writing
PSU Social
Science
PSU Educacion
SOURCE: Pearson, Final Report Evaluation of the Chile PSU, January 2013
Predictive validities of the PSU
(CTA v Pearson estimates)
0.6
0.5
0.4
0.3
0.2
0.1
0
Language
CTA
Mathematics
Pearson
SOURCE: Pearson, Final Report Evaluation of the Chile PSU, January 2013; CTA
Incremental Predictive validities (engineering):
(controlling for NEM)
35
30
25
20
PAA
15
PSU
10
5
0
U. Chile
PUC
Language & Math
U. Chile
PUC
Language & Math + subject test
SOURCE: S.A. Prado, Estudio de Validez Predictiva de la PSU y Comparacion con el Sistema PAA,
Universidad de Chile
Content coverage (values from 0% to 100%)
…how much of
the content
domain of a test
has been taught
in the schools.
It is not fair to expect students to master
content to which they have not been exposed.
…or, to compare students who have been
exposed to students who have not.
Percentage curricular coverage in Chilean high schools, by type of school: 2012
Mathematics, Level 1
100
75
50
25
0
Municipal
Subvencionado
Pagado
SOURCE: Centro de Estudios Mineduc, Cobertura Curricular en Ensenanza Media
Lenguaje y Comunicacion – Matematica, Septiembre 2012
Percentage curricular coverage in Chilean high schools, by type of school: 2012
Language & Communication, Level 2
100
75
50
25
0
Municipal
Subvencionado
Pagado
SOURCE: Centro de Estudios Mineduc, Cobertura Curricular en Ensenanza Media
Lenguaje y Comunicacion – Matematica, Septiembre 2012
Percentage curricular coverage in Chilean high schools, by type of school: 2012
Mathematics, Level 3
100
75
50
25
0
Municipal
Subvencionado
Pagado
SOURCE: Centro de Estudios Mineduc, Cobertura Curricular en Ensenanza Media
Lenguaje y Comunicacion – Matematica, Septiembre 2012
Percentage curricular coverage in Chilean high schools, by type of school: 2012
Language & Communication, Level 4
100
75
50
25
0
Municipal
Subvencionado
Pagado
SOURCE: Centro de Estudios Mineduc, Cobertura Curricular en Ensenanza Media
Lenguaje y Comunicacion – Matematica, Septiembre 2012
Percentage curricular coverage in Chilean high schools, by type of curriculum: 2012
Mathematics, Level 4
100
75
50
25
0
Humanista Cientifica
Technico Profesional
Polivante
SOURCE: Centro de Estudios Mineduc, Cobertura Curricular en Ensenanza Media
Lenguaje y Comunicacion – Matematica, Septiembre 2012
Percentage curricular coverage in Chilean high schools, by type of curriculum: 2012
Language & Communication, Level 4
100
75
50
25
0
Numanista Cientifica
Technico Profesional
Polivante
SOURCE: Centro de Estudios Mineduc, Cobertura Curricular en Ensenanza Media
Lenguaje y Comunicacion – Matematica, Septiembre 2012
Percentage of Chilean high schools with full curricular coverage, by subject area: 2012
Levels 1--4
100%
75%
50%
Do NOT Cover 100%
Cover 100%
25%
0%
Mathematics
Language &
Communication
SOURCE: Centro de Estudios Mineduc, Cobertura Curricular en Ensenanza Media
Lenguaje y Comunicacion – Matematica, Septiembre 2012
Subgroup differences
Differences in test scores among
subgroups (e.g., gender, ethnic, school
type) should be due only to differences
in the attribute measured by the test
and not to systematic biases in the test.
Growing gaps in PSU Mathematics raw & adjusted scores,
by type of curriculum: 2002—2010
Brechas PSU Matemáticas para toda la muestra
Brechas PSU Matemáticas para toda la muestra
180
200
160
200
180
140
180
160
120
160
200
140
100
140
180
120
80
120
160
100
60
100
140
80
40
80
120
60
20
60
100
40
0
40
80
20
Brechas
Brechas Brechas
BrechasBrechas
200
20
60
0
0
40
190
180
170
160
150
140
130
120
110
111
100
90
95
80
70
60
50
46
40
43
30
20
2002
2003
2004
2005
10
0
2002
2003
2004
2005
2002
2003
2004
2002
2003
2004
2005
Brechas PSU Matemáticas para toda la 170
muestra
Brechas PSU Matemáticas para toda la muestra
Brechas PSU Matemáticas para toda la muestra
124
85
51
2006
2007
2008
2009
2010
2006
2005
2006
2007
2006
2007
2008
2009
2010
2007
2008
2009
2008
2009
2010
2006
2007
2008
PP Muni-TP
Brecha Sin
Ajustar
PP Muni-TP
Brecha Sin
PP
PP Muni-TP
Muni-TP
Ajustar
Brecha
Brecha Ajustada
Sin
Ajustar
PP Muni-TP
Brecha
Ajustada
PP
Muni-CH
PP Muni-TP
Brecha
Sin
Brecha Ajustada
Sin
Ajustar
Ajustar
PP Muni-CH
Brecha
Sin
PP
Muni-CH
Muni-CH
PP Muni-TP
AjustarAjustada
Brecha
Sin
Brecha Ajustada
Ajustar
PP Muni-CH
Brecha Ajustada
PP Muni-CH
Ajustada
Brecha Sin
Ajustar
PP Muni-CH
Brecha Ajustada
2010
20
0
2002
2003
2004
2005
2009
2010
SOURCE: Koljatic, Silva, & Phelps, Consequential Tests and Conflicts of Interest: The Case of Chile’s PSU,
forthcoming, 2014
Growing gaps in PSU Language & Communication raw &
adjusted scores, by type of curriculum: 2002—2010
Brechas PSU Lenguaje para toda la muestra
Brechas PSU Matemáticas para toda la muestra
180
200
170
160
180
200
140
160
180
120
140
200
160
100
120
180
140
80
100
160
120
60
80
140
100
40
60
120
80
20
40
100
60
0
20
80
40
150
Brechas
Brechas Brechas
Brechas
Brechas
200
0
60
20
160
Brechas PSU Matemáticas para toda la muestra
140
130
120
Brechas PSU Matemáticas para toda la muestra
113
110
100
106
90
79
80
70
86
60
44
50
40
30
20
2002
10
44
36
2003
2004
2005
2006
2007
2003
2002
2004
2003
2005
2004
2006
2005
2007
2008
2009
2010
2006
2007
2008
2009
2002
2003
2004
2005
2006
2007
2008
2009
2010
2002
2003
2004
2005
2006
2007
2008
2009
2010
0
2002
40
0
20
Brechas PSU Matemáticas para toda la muestra
161
2008
2009
2010
PP Muni-TP
Brecha Sin
Ajustar
PP Muni-TP
Brecha Sin
PP
Muni-TP
Ajustar
PP Muni-TP
Brecha
Ajustada
Brecha Sin
PP
Muni-TP
Ajustar
Brecha Ajustada
PP
PP Muni-CH
Muni-TP
PP Muni-TP
Brecha
Brecha Sin
Sin
Brecha Ajustada
Ajustar
Ajustar
PP
Muni-CH
Brecha Sin
PP Muni-CH
Ajustar
PP Muni-TP
PP Muni-CH
Brecha
Ajustada
Brecha Ajustada
Brecha Sin
PP
Muni-CH
Ajustar
Brecha Ajustada
PP Muni-CH
PP Muni-CH
Brecha Sin
Brecha Ajustada
Ajustar
PP Muni-CH
Brecha Ajustada
2010
0
SOURCE: Koljatic, Silva, & Phelps, Consequential Tests and Conflicts of Interest: The Case of Chile’s PSU,
forthcoming, 2014
3. A chronology of mistakes
2000, initial proposal, SIES/PSU project
This proposal attempts a redesign of the tests currently used to select
students for higher education in Chile. It is expected that [this new test
will] have a positive impact in the efficiency of the selection
process, improving the psychometric properties of the measuring
instruments, and establishing a better articulation between the
selection system and the secondary education curriculum.
SOURCE: Proyecto FONDEF, Reformulacion de las Pruebas de Seleccion a la Educacion Superior
A chronology of mistakes (cont.)
2001 (World Bank & MINEDUC)
…the Academic Aptitude Test for entry to the university system is under
revision, together with the universities belonging to the Council of Rectors.
This instrument of entry selection, needs also to be aligned with the new
curriculum and may become an exit exam from the secondary
education system.
SOURCE: World Bank, Implementation Completion Report on a Loan in the Amount of $35 million to the Republic of
Chile for Secondary Education, 2001
Descargar

Descargar-Presentacion-Richard-Phelps-2