SCHOOLS NorthEast issued a Call for Evidence to all serving head teachers of secondary schools in the region in response to concerns at volatility in exam assessment in summer 2015 and a significant degradation in trust for the process and the organisations responsible for delivering it.
The Education Select Committee will be looking at submitted evidence and speaking to Ofqual Chief Regulator Glenys Stacey and Chair of Ofqual Amanda Spielman in a session on the work of Ofqual taking place Wednesday 14 October 2015.
Here is the SCHOOLS NorthEast submission:
A significant number of head teachers (20% of all North East secondaries) responded to the Call to give detailed evidence on issues they had personally experienced. This number is expected to rise as a proportion of schools were still awaiting the outcome of re-marks at the time the Call was issued.
Collectively, the themes of the response are as follows:
- Confidence in examination boards is at an all-time low
- Consistency is patchy, leading to significant downward shifts in cohort results
- Examples of wild grade inflation upon re-mark – in some cases by as many as 30 marks on an individual module
- Concern over the capacity and quality of marking at certain examination boards
- Grade boundary fluctuations having a significant impact on school performance
- Lack of confidence in the system of re-marking
- Impact on staff morale with individual teachers experiencing a significant loss of confidence with regard to predicting student outcomes
- One school reported threats of legal action by parents against teachers due to grade disparities between predictions and results
The impact is clear in the associated comments received by school leaders, best illustrated by this head teacher: “I have been in the profession for 36 years and I have never before felt such distrust and despair with regard to the national examination landscape.”
In submitting this evidence, SCHOOLS NorthEast has taken account of the following:
Ofqual acknowledges that one of, if not the most important aspects of its work is to “maintain standards and confidence in qualifications”.
Its About Us section on www.gov.uk states:
We’re responsible for making sure that:
- regulated qualifications reliably indicate the knowledge, skills and understanding students have demonstrated
- assessments and exams show what a student has achieved
- people have confidence in the qualifications that we regulate
The 44th meeting of Ofqual Board – minutes May 2014 – looked at ‘Activities that undermine confidence’ which included the following:
– factors that could affect an increase in the number of remarks requested,
for example confidence in the marking of subjects such as English and history
The minutes reflect: The Board recognised the difficulties that were faced in tackling these issues and welcomed the work to date.
SCHOOLS NorthEast received over 20 A4 sides of evidence from schools, many detailing specific issues with individual papers. In recognising the Committee is seeking short written submissions, this information has been condensed into the following points of detail. Additional information is available upon request.
Schools report significant issues in the results both of examination papers and assessment of coursework. This was experienced across a range of subjects, with particular concerns raised in English – both GCSE and iGCSE.
AQA stands out as the examination board with the greatest degree of variance to predicted results in English and several other subjects, and CIE’s IGCSE English Language papers marking saw centres report candidates underperforming by 1-2 grades.
This is illustrated by the following report from one school which reported an annual drop of 23% in A*-C grades on its Cambridge iGCSE English summer 2015 examination and a reduction of 36% in the number of students achieving three levels of progress. This was despite the same group of teachers delivering the course in the same way as the previous year.
The school’s head teacher reported: “Our coursework was downgraded by 2 marks across the cohort – a situation which has never before occurred within English in our school”.
While the bulk of concerns were with English papers and coursework, there was a large volume of issues with a wide range of other subjects
Even in schools where there has been a significantly above average intake of pupils (based on RAISE Online), there was distinct results degradation in assessed coursework despite no changes to staffing or approach.
A number of schools report hiring external, independent moderators – at a cost of up to £3,000 – to re-assess coursework that has been downgraded across entire cohorts.
Concern is raised about the practice of assessors using only small samples of coursework assessment to downgrade entire centre submissions which has seen a shift of between 0.5-2 grades which impacts greatly on a school’s overall performance at A*-C. In one school with a cohort of 260 pupils, just 7 candidates’ work was used to assess English coursework. The result was a 0.5 grade reduction for all students.
An example of the response from school leaders – “We have had three current and former Heads of English, one of whom is an experienced examiner, mark them independently; all three awarded the papers an identical mark which in each case was significantly higher than the mark which had been given and later confirmed by (the exam board)”.
One head teacher reports the impact on staff morale as follows: “The last five weeks have been the worst of my career and characterised by dozens of parental complaints and several threats to sue.”
There is a strong response to marking schemes, with one head teacher saying “I have specific concerns related to“the creation of a mark scheme which imposes artificial restrictions on the marker; restrictions which could never have been known by candidates, but which have led to students receiving less credit”.
The volume of papers sent for re-assessment has been phenomenal. Schools report sending entire cohorts back to examination boards for re-marking at great cost to the school because the grades have differed so greatly from predictions.
There is a strong sense of a ‘hold the line’ approach to re-marked scripts and requests for re-moderation.
A number of schools report sending large volume requests for re-marking and then receiving a response within 2 hours that there was no grade change.
Where requests for re-moderation have led to changes in marking, schools report wild variance in the results. In one cohort, there was a variance of 40 points difference across the entry – some pupils marked up by 20 while one was marked down by the same amount for one single module.
There is still the belief that having a competitive market place for each individual subject area leads to an unhelpful variance in grade outcomes. The role of Ofqual in ensuring consistency across the board is vital.
It is recognised that the exam period is logistically difficult to manage. That said, schools report a larger than normal volume of lost papers and some report papers being delivered to primary schools
In one example, a school challenged AQA on a number of candidates’ results and then further challenged the re-marks. At this point, the exam board admitted losing two papers and it had estimated the re-mark grades based on the exam centre.
Another school reported the following: “Edexcel contacted the school on many occasions asking where coursework was as they received it, only for me to have actually sent it to where it had supposed to have been sent. The exam board changed the moderators’ details quite frequently and so whilst things were being couriered to one place, another moderator had been assigned and then they received the coursework. Also Edexcel on several occasions lost scripts from certain pupils and so I had to email them the seating plans, confirmation of attendance, and proof of posting each time.”
The quality and quantity of assessors is clearly a defining factor in confidence in the entire assessment process.
A number of what was described by one school leader as “horror stories” from their teaching staff who also work as assessors is insightful. These include reports of:
- significant staff shortages resulting in some papers remaining unmarked
- offers of payment three times in excess of day rate made by exam boards to incentivise markers to work over-time
- assessors drafted in at short notice from other subjects to mark scripts that were not in their field of expertise and who had no previous experience of marking in that subject
- marking schemes only being finalised days before standardisation, resulting in some quality responses not being considered
- team leaders reporting extreme variations in marking – up to 40 marks out on one paper – and assessors not following the mark scheme correctly
One team leader reported: “I have marked for many years now and and I am increasingly appalled by the fact that even when I have given someone the lowest grade and recommended not to use them, they then turn up a year later as a team leader”.
The stakes are too high for pupils, schools and individual school leaders to have the levels of disparity and distrust that this summer’s invigilation prompted. This has compounded previous experiences such as the grading issues of 2012.
Schools are operating in unprecedented times with heightened levels of scrutiny and the potential for ‘one strike and you are out’ leadership change. Outcomes data is the predominant tool Ofsted uses for assessing overall school effectiveness which makes accurate assessment at GCSE and A-Level essential.