Every year, without fail, throughout my career in education, there has been intense discussion about the current year’s exam results and whether standards are rising or falling. This speculation is tiresome at the best of times, and tends to focus upon the performance of students at the top end of the grade brackets, leading to pot-shots being taken at different school sectors, whilst nobody discusses the support needed for the forgotten third of the cohort.
In the exceptional times we are living through, the added speculation on the value of this year’s results is not only unhelpful but incredibly stressful for students who, like last year’s cohort, are also grappling with concerns that in the future, nobody will take their results seriously because they didn’t sit the “real” exam.
There is also much debate about the possibility of, and reasons for, any gap in attainment between maintained and private schools. Opinions will be voiced on the effectiveness of “sharp elbowed parents” in giving their children an edge in a coming storm of appeals from the middle classes who have the time and resources to engage legal advice.
“Students are also grappling with concerns that nobody will take their results seriously because they didn’t sit the ‘real’ exam.”
Now, it is clear that the advice on exam arrangements was given to schools by the Department for Education far too late, and left schools to pull off a frankly heroic feat in ensuring that the criteria set were correctly interpreted, and met by every subject, for every student, within the timeframe. These Teacher Assessed Grades have been drawn from a range of evidence schools held on students and quality assured by the exam boards.
So why the angst about the validity and reliability of the results? According to the OED, validity can be defined as “the state of being logical and true”, whilst reliability is defined as the quality of being likely to be correct or true.
In a “normal” year, validity and reliability are not so much examined, even though these results are norm-referenced and adjusted each year, based upon prior attainment data. As the DfE’s disastrous dalliance with algorithms showed last year, prior attainment data for a school can give a semblance of validity on a macro level, whilst being deeply unfair to the individual, or cohort within a particular school, which does not fit the model. Internationally, new schools do not have historic exam data to rely upon.
Norm referencing does not ensure validity in exams, and in its purest form indicates a student’s rank, not their capability in the subject. If reliability is defined as a measure of how likely a student’s performance on a paper is to be a true reflection of their ability, then the high-stakes exam isn’t the best guarantee of this, either.
“Norm referencing does not ensure validity in exams, and in its purest form indicates a student’s rank, not their capability.”
So what about this year? Students have had the opportunity to demonstrate their capability based upon what they have been taught, and across a range of different tasks. The grades arrived at by schools are therefore based upon multiple reference points. This means that, in terms of validity, for each individual student they are more likely to be in that “state of being logical and true”. In addition, the gathering of multiple data points ensures a degree of consistency, and therefore reliability around the judgements reached.
Of course, all predictions have pointed to grades increasing this year. This isn’t necessarily an indication of a relaxation of standards and it certainly doesn’t devalue the outcomes. Students and schools have had the time and opportunity to be their “best selves” in a way that the usual exam routine does not allow. Nor have their results been adjusted to fit a distribution curve.
Schools have also achieved this under incredibly challenging circumstances, with students who have had incredibly disrupted and variable educational experiences throughout their A level (or GCSE) courses. All they have achieved should be a cause for celebration and congratulations.