Why have annual national assessments?

One thing typically lacking in discussions about the ANAs is a clear understanding of their purpose. There are three broad goals that one might want a system of national assessments to achieve:

• First the ANA should be seen as a tool for teachers to use in their own assessment processes in order to diagnose learning problems and design remedial strategies (Goal 1).

• Second, the ANA could be used to measure the progress of the country (or of smaller sub-groups such as provinces) towards improving literacy and numeracy outcomes (Goal 2).

• Third, the ANA could be used to promote accountability throughout the system for achieving learning outcomes (Goal 3). Here I use a broad definition of accountability, which includes empowering parents and communities with information about learner and school performance.

All three of these goals are envisaged in the Department of Basic Education’s sector plan, albeit not in the exact same words.

To some extent the ANA have made limited contributions to all three of these goals over the last few years. On goal 1, research just prior to the introduction of ANA had revealed extremely weak assessment practices within the majority of our schools, thus pointing to the need to provide teachers with good assessment tools to use in the process of teaching and learning. Therefore, even if the test development process in ANA could be improved, government has at least provided new assessments that are an improvement on what was previously available.

On goal 2, the ANA has confirmed the low and unequal nature of learning outcomes within the primary school sector, highlighting the urgency to address foundational literacy and numeracy.

On goal 3, there is now a greater emphasis on academic outcomes within primary schools, something that previously only existed at the matric level when it is too late to address fundamental learning gaps.

However, these goals are only bluntly or partially achieved by ANA in its current design. On goal 1, many teachers do not yet effectively incorporate the ANA into their learning programme, and sometimes experience ANA as a distraction. On goal 2, we cannot yet compare the overall performance in ANA from one year to the next since a new set of test items is developed each year. On goal 3, one cannot exert any strong accountability if teachers themselves administer and mark the tests – any attempts to label particular teachers will therefore not improve accountability but will simply provide an incentive for teachers to inflate ANA performance.

Importantly, each of these goals, if they are to be achieved, requires different design features within ANA. And there is a real risk that if we aim for ANA to serve all three goals we will end up achieving none of them. For example, if teachers administer and mark ANA themselves since they are expected to use it as a classroom assessment tool, but those teachers also face rewards and sanctions based on the test results, then we may end up in a situation where teachers don’t use the tests effectively and the test results will not be reliable for accountability purposes.

To provide teachers with good assessment tools, teachers across all grades should receive ANA every year. If the assessments are to function effectively for this purpose, they should be administered and marked by the teacher, and they should remain low-stakes so that teachers administer and mark them without any fear of negative consequences. For this purpose of ANA, the department could introduce training opportunities and tools for teachers dealing with how to use the ANA to diagnose learning difficulties and inform remedial strategies. To some extent, these sorts of initiatives are already being introduced by the department, but over time they need to be refined and implemented on a larger scale. There may also be a need to investigate the curriculum content covered in the ANA relative to the timing of when ANA is administered in order to optimize the alignment of ANA with lesson planning and curriculum sequencing within schools.

For goal 2, the best way to measure changes in learning outcomes within the system is to test a representative sample of students, as is presently pursued through the Verification ANA process. This process requires independent fieldworkers to administer and mark the tests. In order to compare performance from one year to the next, the tests administered to this sample of students need to contain some test items that are common across the years. These items must therefore be kept secure. The department has been exploring possibilities to achieve this but it is a somewhat technical process and has not yet been finalised. For this purpose of ANA, it is not necessary to test every grade; one could focus only on certain priority grades.

Using national assessments to hold schools or teachers accountable for learning outcomes is a far more ambitious pursuit. This would require independently administered assessments for all schools, which would have major logistical implications and may not be compatible with the current design where assessments are administered simultaneously in all schools. Even then, it is a statistically complex task to isolate the impact of a teacher on learning outcomes over and above the influence of other factors such as the home background of students. This is of course not to mention the objections teacher unions would probably have to holding teachers accountable in this way. My view is that we should rather be content with “soft” and “generic” accountability whereby the importance of achieving learning outcomes in the early grades of schooling is recognised by all parties – teachers, learners, parents and government.

ANA is potentially a very important intervention in our school system. All the research indicates that to meaningfully improve educational opportunities in South Africa we have to get learning foundations in the early grades of school right. The assessments help raise awareness throughout society of the importance of achieving these early learning foundations.

So why is there opposition to the ANA from the teacher unions?

I am sympathetic with the desire to improve the design and implementation of ANA in order to increase their usefulness. However, I suspect that the opposition to ANA is at least partly motivated by a general anti-testing view that I often encounter. The anti-testing sentiment is strongly influenced by experiences and educationalists in the USA. In South Africa, the context is very different. Prior to the ANA, there was effectively no measure of school performance aside from the matric examinations, leaving primary schools in the dark. Arguments like “weighing the pig doesn’t make it any fatter” are of course ridiculous – if this was applied to medicine there would be no place for diagnosis. Besides, assessment is fundamental to learning and teaching. The argument that teachers are teaching to the test would perhaps be valid if there was an unhealthy over-emphasis on academic outcomes in South Africa relative to a more holistic education. But in a situation where the majority of primary school learners are failing to acquire basic literacy and numeracy skills as defined by international benchmarks, encouraging teachers to prepare explicitly for a literacy or numeracy test is hardly a bad thing.

The argument that the money spent on the ANA could perhaps rather be spent on teacher training opportunities is a red herring and a distraction. Given that the ANA costs less than 0.2% of what government spends on teacher salaries, the ANA are one of the most cost-effective ways in which we spend on education. Improving opportunities for professional development of teachers is of course important, and a lot of work is happening in this area, but this should not be seen as an alternative to the ANA, which is a key systemic intervention.

But the ANA clearly need to be improved in their design and implementation. This improvement will have to begin with a clear and shared understanding of the purposes of ANA. The ANA of 2015 has unfortunately been negatively affected. Administering the ANA in December is obviously not ideal: there will be less opportunity for teachers to use the tests to inform their teaching and the marking and data collection may have to occur during the school holidays.

On a positive note, the recent concerns raised by teacher unions could provide a prompt for government to fast-track its efforts to improve various design features of ANA. In the end, we must use this situation to redesign and improve the ANA in the interests of one key stakeholder – the children.

Taylor is a researcher in the South African Department of Basic Education. His work includes impact evaluation of education interventions, measuring educational performance and equity in educational outcomes. In 2010 he completed a PhD in economics at the University of Stellenbosch, analysing educational outcomes of poor South African children. This article is written in his personal capacity. Views expressed are not necessarily GroundUp’s.