Should countries in Africa use PISA for development?

Published on Thursday, 28 July 2016
Should countries in Africa use PISA for development? Credits: UN via flickr

The short answer is no.

Last month, I participated in a conference titled "Learning from Learning Assessments: The Politics and Policies of Attaining Quality Education" which was organized by NORRAG and the Brookings Institute. At the conference I engaged the audience in a discussion about two competing models for international large-scale assessments in Africa: regional assessments and PISA for Development (PISA-D). The regional assessments were established in the mid-1990s with the support of the International Institute for Educational Planning (IIEP) and the international educational community: PASEC serves Francophone countries and SACMEQ serves countries in southern and eastern Africa. PISA-D is a new and enhanced PISA instrument that is, arguably, more relevant for the contexts found in middle- and low-income countries.

While, I am not an expert on Africa or education systems in Africa, I spent the last decade exploring the emergence of international large-scale assessments. The focus of much of my research has been researching the origins of the International Association for the Evaluation of Educational Achievement (IEA). My research has shown that there has been a significant shift in the purpose of these international assessments; from originally addressing distinct research questions to more recently international assessment for the sake of educational governance.

My argument is simple: Low-income countries should avoid PISA-D. This is especially true for African countries that have invested in the development of regional assessments. The rationale for this argument is six-fold.

First, PISA-D does not fit the educational reality in Africa. Both PISA and PISA-D survey a sample of 15-year-old students. In many African countries, however, the net enrollment in lower secondary schools is still low. The sample will not yield a meaningful picture of what 15-year-old students in a given country know or how they can apply their knowledge. PISA-D plan to address this problem by including out-of-school students. But, the value of this effort is unclear. Would out-of-school youth be able to engage with the PISA-D instrument?

Second, and related, various reports suggest that students in Africa have weak competences when entering lower secondary schools. Given that PISA-D is assessing the ways students apply knowledge, how would students with weak competences engage with the PISA-D instrument? Further, if PISA-D will be too difficult to students, the achievements scores will suffer from low variance and from the floor effect (overestimation). How useful would the data be for analysts and policy makers?

Third, because PISA-D is not curriculum-based or grade-based assessment, it offers little information about system variables, such as: teacher’s characteristics, pedagogy, and curriculum. Without these variables, analysts cannot examine how different system’s characteristics matter for student achievement. This limitation raises the question whether PISA-D can inform educational planning.

Fourth, research from high-income countries shows that PISA results have negative consequences for public discourse and opinion. Research in the United States and Israel, for example, demonstrates that exposure to PISA results and discourse decreases public’s confidence in national education system and reduces support for public spending on education. This is often because of scandalized media reporting and a lack of proper dissemination and interpretation of the results. Findings from the 2012/2013 Afrobarometer in 34 countries show that the general public is satisfied with the way governments address educational needs. The general public is more concerned with overcrowded classrooms and poor facilities than from poor teaching. Given that public confidence and endorsement is important, are we willing to take the risk that PISA-D results will change public opinion towards education in African countries?

Sixth, regional assessments are embedded in the local context and are flexible to change. These are real strengths that allow regional assessments to evolve overtime, together with the educational systems they serve. In contrast, PISA is much more institutionalized with a set cycle (every three years) and a focus on monitoring trends. It is unclear what would be the added value of using PISA-D over regional assessments?

My hope is that policy makers will further develop PASEC and SACMEQ and use their insights to shed light on educational policy questions at the country and regional levels.

Leave a comment

Make sure you enter all the required information, indicated by an asterisk (*). HTML code is not allowed.

Oren Pizmony-Levy

Oren Pizmony-Levy is an Assistant Professor in the Department of International and Transcultural Studies at Teachers College, Columbia University. He holds a PhD in sociology and comparative and international education from Indiana University-Bloomington. His research and teaching focus on the intersection between education and global educational movements, such as international large-scale assessments of student achievement (e.g., TIMSS, PIRLS, and PISA).

Other blog posts

Beyond the Summit

Written by
on Wednesday, 16 March 2016

The sixth International Summit on the Teaching Profession (ISTP) was held in Berlin on the third and fourth of March 2016. The first such summit, in New York in 2011, was a major breakthrough for teachers, which also recognised Education International (EI) as the global representative of teachers and of the profession. The New York meeting, like subsequent summits, brought EI together with governments and the Organisation for Economic Cooperation and Development) (OECD), with its considerable expertise and bountiful data…

Read more...

Collective bargaining – the real teachers’ voice.

Written by
on Wednesday, 22 June 2016

Many official policy documents and academic papers make the case for ‘teacher voice’. They highlight the need to develop genuine collaborative cultures in schools where teachers can have a say in making decisions on professional issues. This may be so obvious as to be a ‘common sense’. However, the reality of teachers’ experience is often very different. In many parts of the world teachers experience a huge amount of top-down centralised control that fails to recognise their skills and professionalism.…

Read more...
blog archive