Mala Singh
Centre for Higher Education Research and
Information, The Open University, UK
[email protected]
Simple Story Line
 Contemporary higher education is
enveloped in accountability-increased
reporting to range of external stakeholders
 Evaluation is a policy instrument of
accountability-demonstrating compliance
 Quality is a code for performance,
responsiveness and success in competition
MSingh NYU 07May2009
Feel-good or public good proposal?
 If accountability demands/evaluation is
here to stay, should Social Justice be
included in evaluation systems?
 USA and South Africa as examples of
systems where this approach has been
 Extending the scope of evaluation to
include SJ-opportunities, limits and
MSingh NYU 07May2009
Contested Issues:What is at stake?
 What is the measure of excellence in
higher education
 Who is entitled to define it and judge
its achievement-by what means
 Who is able to compel its
MSingh NYU 07May2009
Social Justice in HE Evaluation
 Including the historically excluded-
disjunctures between legislation, policies,
institutional cultures and practices
 Social justice-politics of Recognition?
Distribution? Transformation?
 Premise-that higher education can
contribute to addressing social injustice
both directly and indirectly
MSingh NYU 07May2009
Setting the Scene
 Far-reaching changes to higher education as
learning, research and work setting
 Impressive massification -globally 51 m students in
PSE in 1980 to 140 m in 2006 (Teichler)
 Continuing inequality and exclusion-globally 80%
participation rates in HICs and 7% in LICs;
nationally US 41% white, 32% black and 24%
Hispanic(Altbach); SA 61 % white, 16 % black,
black completion rates after 5 yrs 30 % with 56 %
dropping out(Scott)
MSingh NYU 07May2009
Three Takes on Evaluation
 Evaluation as central part of HE-key to ‘guild
authority’, credentialling, reputational
hierarchies, resource allocation and rewards
(Henkel)-peer review
 Evaluation for social reform and monitoring
social progress-US in 1960s
 Evaluation as a policy instrument of the
‘evaluative state’ to ensure social
accountability and stakeholder
responsiveness of HE
MSingh NYU 07May2009
Evaluation and Social Reform
 Evaluation as a professional field in service of
social reform in the Johnson era (USA 1960s)
 Great Society programmes aimed at racial injustice
and poverty-evaluations of interventions in
education(school curricula, health, welfare)
 Approaches in evaluation studies about social
justice, participatory evaluations, potential for
deliberative democracy-House, Howe, Patton.
MSingh NYU 07May2009
The ‘Evaluative State’
 Trends in UK/European HE in mid
eighties-shift from state control to state
 State use of intermediary agencies to
monitor performance and provide
consumer information; self-regulation
 Increase in private and privatised provision
 Policy isomorphism in reform discourse
 200 members of INQAAHE worldwide
MSingh NYU 07May2009
Key Issues
 The rise of the ‘evaluative state’- juridification
and contractualism(Neave)
 Shifting power balances in the Clark triangle
of state, market and academe
 Systems theory-from inputs and processes to
outputs in judging HE
 Heightened accountability, increased power
of evaluation, continuing inequalities, social
justice underserved by states and markets.
MSingh NYU 07May2009
USA and SA Evaluation Systems
 USA-institutional and subject accreditation –
somewhat voluntary system owned by HEIs,
used for improvement and to demonstrate
quality levels to be eligible for federal funds.
Inclusion of diversity considerations in some
 SA –institutional audit and programme
accreditation-mandatory national system
used for improvement and eligibility for
programme funding. Inclusion of social
justice/social transformation
MSingh NYU 07May2009
Diversity in US Evaluation
 Most evaluation systems look at institutional
mission but not in a directive way iro SJ
 WASC Statement on Diversity and Criteriaresponse to changing demography of the state
and its reflection in HE(1980s)
 Demographic diversity-students, faculty,
governing boards-from affirmative action to
 Curriculum, pedagogy, campus culture and
MSingh NYU 07May2009
Diversity in US Evaluation
 Diversity is a measure of quality education-
cosmopolitanism, tolerance, engagement
with other cultural perspectives,
preparation for world of work and civic
participation-tapping human potential of
all citizens
 Accusations of punitiveness and social reengineering; diversity focus about neither
education nor quality
MSingh NYU 07May2009
Diversity in US Evaluation
 Proposition 209(1996)-no direct focus
on race, ethnicity and gender in public
 Shift from access to retention and
graduation rates
MSingh NYU 07May2009
Social Justice in SA Evaluation
 Post 1994 reconstruction of SA society and HEsocial justice and transformation
 Reflected in evaluation system(2004) –criteria for
institutional and programme evaluation
 Demographic representivity(race, class, gender,
disability) as well as social transformationinstitutional culture, curriculum change, new
pedagogies, new research themes and new
community partnerships
MSingh NYU 07May2009
USA and SA Evaluation Systems
 Different systems-different languages
to designate a similar social concern
about education, quality and inclusion
 Diversity/SJ focus in criteria, training
of evaluators, in the panel engagement
with staff and students, in self-review
and agency report and
MSingh NYU 07May2009
USA and SA Evaluation Systems
 Easier to look at input measures and
measurable achievements
 How to measure SJ competencies-no
standards and indicators set for them-may
help to avoid SJ fundamentalism in
 SJ defined in relation to dominant
contextual imperatives-no Platonic
absolutes-part of democratic debate and
MSingh NYU 07May2009
In search of conclusions
 Opportunities-exploring the interface
of excellence and inequality in HE
 Inserting dialogic engagement on
social justice/transformation issues
pertinent to context into evaluation
 Counterposing search for market
competencies with broader social
competencies and impact
MSingh NYU 07May2009
In search of conclusions
 Limits-what educational outcomes can
be measured in evaluation
 Easier iro numbers-demographic
diversity in governance/ student
body/faculty and administrators,
curriculum change
 More difficult to measure iro value and
worldview outcomes
MSingh NYU 07May2009
In search of conclusions
 Dangers-bringing state/party political
agendas back into academe with no
guarantees as to how evaluation
findings will be used
 Increasing onerousness of evaluation
and burden of evidence on academics
 Using SJ to withhold accreditation
MSingh NYU 07May2009
Unresolved Issues
 Is SJ a constitutive component of quality
and excellence or only a framing
 Who can best organise HE and make it
work in the public interest-states, markets
or academe?
 Evaluation as positivist science or
hermeneutics? Will strong accountability
demands be satisfied with interpretative
rather than quantitative findings?
MSingh NYU 07May2009
Unresolved Issues
 SJ in transnational contexts-whose SJ?
 Expecting too much of HE?
 Populist proposal-save the world
outside of evaluation and HE(Fish)?
 Bringing together the three takes on
evaluation-guild authority, social
reform and excellence in the public
MSingh NYU 07May2009

Opportunities and Dangers: Social Justice in HE Evaluation