Should countries in Africa use PISA for development?

Published on Thursday, 28 July 2016
Should countries in Africa use PISA for development? Credits: UN via flickr

The short answer is no.

Last month, I participated in a conference titled "Learning from Learning Assessments: The Politics and Policies of Attaining Quality Education" which was organized by NORRAG and the Brookings Institute. At the conference I engaged the audience in a discussion about two competing models for international large-scale assessments in Africa: regional assessments and PISA for Development (PISA-D). The regional assessments were established in the mid-1990s with the support of the International Institute for Educational Planning (IIEP) and the international educational community: PASEC serves Francophone countries and SACMEQ serves countries in southern and eastern Africa. PISA-D is a new and enhanced PISA instrument that is, arguably, more relevant for the contexts found in middle- and low-income countries.

While, I am not an expert on Africa or education systems in Africa, I spent the last decade exploring the emergence of international large-scale assessments. The focus of much of my research has been researching the origins of the International Association for the Evaluation of Educational Achievement (IEA). My research has shown that there has been a significant shift in the purpose of these international assessments; from originally addressing distinct research questions to more recently international assessment for the sake of educational governance.

My argument is simple: Low-income countries should avoid PISA-D. This is especially true for African countries that have invested in the development of regional assessments. The rationale for this argument is six-fold.

First, PISA-D does not fit the educational reality in Africa. Both PISA and PISA-D survey a sample of 15-year-old students. In many African countries, however, the net enrollment in lower secondary schools is still low. The sample will not yield a meaningful picture of what 15-year-old students in a given country know or how they can apply their knowledge. PISA-D plan to address this problem by including out-of-school students. But, the value of this effort is unclear. Would out-of-school youth be able to engage with the PISA-D instrument?

Second, and related, various reports suggest that students in Africa have weak competences when entering lower secondary schools. Given that PISA-D is assessing the ways students apply knowledge, how would students with weak competences engage with the PISA-D instrument? Further, if PISA-D will be too difficult to students, the achievements scores will suffer from low variance and from the floor effect (overestimation). How useful would the data be for analysts and policy makers?

Third, because PISA-D is not curriculum-based or grade-based assessment, it offers little information about system variables, such as: teacher’s characteristics, pedagogy, and curriculum. Without these variables, analysts cannot examine how different system’s characteristics matter for student achievement. This limitation raises the question whether PISA-D can inform educational planning.

Fourth, research from high-income countries shows that PISA results have negative consequences for public discourse and opinion. Research in the United States and Israel, for example, demonstrates that exposure to PISA results and discourse decreases public’s confidence in national education system and reduces support for public spending on education. This is often because of scandalized media reporting and a lack of proper dissemination and interpretation of the results. Findings from the 2012/2013 Afrobarometer in 34 countries show that the general public is satisfied with the way governments address educational needs. The general public is more concerned with overcrowded classrooms and poor facilities than from poor teaching. Given that public confidence and endorsement is important, are we willing to take the risk that PISA-D results will change public opinion towards education in African countries?

Sixth, regional assessments are embedded in the local context and are flexible to change. These are real strengths that allow regional assessments to evolve overtime, together with the educational systems they serve. In contrast, PISA is much more institutionalized with a set cycle (every three years) and a focus on monitoring trends. It is unclear what would be the added value of using PISA-D over regional assessments?

My hope is that policy makers will further develop PASEC and SACMEQ and use their insights to shed light on educational policy questions at the country and regional levels.

Leave a comment

Make sure you enter all the required information, indicated by an asterisk (*). HTML code is not allowed.

Oren Pizmony-Levy

Oren Pizmony-Levy is an Assistant Professor in the Department of International and Transcultural Studies at Teachers College, Columbia University. He holds a PhD in sociology and comparative and international education from Indiana University-Bloomington. His research and teaching focus on the intersection between education and global educational movements, such as international large-scale assessments of student achievement (e.g., TIMSS, PIRLS, and PISA).

Other blog posts

The European Dimension of China’s “Going Global” Strategy

Written by
on Thursday, 03 March 2016

There are two major EU-China issues under consideration in 2016. One is whether the EU should grant China Market Economy Status (MES) and the other is the ongoing negotiations on an EU-China Investment Agreement. The two issues are separate, but closely linked. If MES is granted to China, it would have very concrete impact on anti-dumping rules in Europe, which would put up to 3.5 million jobs in Europe at risk. Moreover, it would also be an official sign that…

Read more...

Towards meaningful solidarity

Written by
on Tuesday, 09 April 2013

I became a teacher because I enjoy helping young people learn. I saw teaching as a way to “make a difference” in the world. But at this moment in history, teachers face opponents who are doing great harm, to children, schools, and our profession, harm that we cannot impede solely through our role in classrooms. The rich and powerful, who exploit their stunning wealth to control media and government, use lofty-sounding slogans about “students first” and “making schools work” to…

Read more...
blog archive