The awarding of qualifications in 2021
CIEA Vice-Chair Dr Mick Walker presents a plan for the awarding of qualifications in 2021.
It's 12:00am on the 15th April 1912 and we're stood on the decks of the Titanic debating the future of shipping.
The recent decision to cancel examinations for 16- and 18-year-olds this summer is hardly a surprise given the current state of the pandemic. What might be a surprise, even shock, is that given the experience of 2020, there is no plan B ready for immediate implementation. The CIEA, CCT and Sir John Dunford wrote to the Secretary of State (SoS) last September offering an outline plan to cover the possibility of cancellations in 2021 and the offer of help in implementing such a plan. The SoS didn’t respond, but the Minister for Schools replied on his behalf on the 15th December. Plans were put in place to push back the examination timetable, to provide guidance on the content of examinations and notification that grades ‘would be generous’. None of these measures related to the offer of assistance for developing a plan B should examinations be cancelled, and the measures announced by ministers were ‘broadly welcomed’ by the teacher associations.
Our pupils don’t want ‘generous grades’, they want fair and equitable outcomes. But we are now in a situation where Ofqual has been given the impossible task of coming up with an approach that will be fair, hold water technically and be acceptable to a wide range of interested parties – but only after a consultation. Meanwhile others are pitching in with ideas for the long-term role of examinations and tests pointing to all the shortcomings in our system and those who run it. I’m not for a second against debates about education and assessment, but there is a time and a place, and the starting point for any debate should be based on what we want out of our education system and not about tinkering with examinations: approaches to assessment should follow.
So, we are now in a position where we are stood on the decks of the Titanic around 12:00am on the 15th April 1912 debating the future of shipping.
The focus of interest right now has to be on setting in place a process that is as fair as possible to the current cohort, whilst at the same time recognising that we have to be equally fair to the cohorts of 2019, 2020, 2022 and beyond.
So, I’m resisting debate and in its place I offer my outline plan (below) for dealing with the awarding of grades in summer 2021.
It’s not technically perfect by some considerable distance, and some of it is quite demanding in terms of what we can do in the little time left. Rather it is pragmatic. It puts the bulk of responsibility and faith in the teaching profession. But it also recognises shortcomings in teacher-based assessment and the need for the profession to step up to the plate and make every effort to provide assurance on the quality of their judgments. The grade inflation of 2020 has to be recognised and every school and college has to play its part in addressing any form of bias: the approach has to be as objective as possible. The use of examination results as measures of accountability has coloured perceptions of assessment and generated accusations of gaming, teaching to the test and ‘our assessments are fair – it’s that lot down the road you should be worrying about’.
The profession can, and must, rise above this and show its true worth. If it succeeds, it offers new approaches to assessment that sit alongside examinations and tests in the future. But it requires immediate support, Lead Assessors in every centre and guidance from Ofqual and the awarding bodies. As a contingency, the CIEA has already designed such support for Lead Assessors which will help now and lay the foundations for the development of professional knowledge and understanding of assessment so evidently lacking in our system.
My outline plan was shared with ministers, Ofqual and the DfE last week and has evolved a little in the light of some very helpful feedback from trusted colleagues. It’s not perfect by a long shot, but it’s a lifeboat.
A suggested approach to the awarding of general qualifications: summer 2021
- The suggested approach broadly recommends a continuation of the approach adopted as a final resort in 2020 using centre assessed grades. This is not presented in any way as an ideal model as the grade inflation of last year underlines. But, using a similar approach over the two ‘Covid years’ brackets a period of similarity rather than a series of awards covering 2019, 2020, 2021 and 2022 all at different standards of performance. However, serious consideration of how to take forward the assessment of general qualifications in 2022 and beyond should already be under consideration.
- Teachers are now best placed to know and understand the impact of ‘learning loss’ or ‘teaching loss’ at individual pupil level. This allows for very localised disruption – or not – to learning and course coverage.
- Centre Assessed Grades (CAGs) will be awarded by centres following centrally set procedures and advice guided by the inclusion of reference test1 outcomes wherever possible.
- A CAG is the level of achievement a candidate would have reasonably achieved under normal circumstances. The award is generated through the professional judgment of teachers informed by all available contextual information.
- Students are looking for fair and creditable results that can be used alongside results from pre- and post-Covid awards. They want recognition of their worth relative to their peers and not ‘generosity’ – nor do future users of qualifications such as universities and employers.
- Schools and colleges should appoint Lead Assessors in each subject to oversee the quality assurance procedures including standardisation and moderation procedures. This is a pivotal role and should be undertaken by senior personnel.
- Schools will be given designated time to conduct training and assessment.
- A series of randomised centre audits should be included to check recommended or required procedures have been adopted and to provide public and professional confidence in the veracity of the adopted approach.
The suggested approach for 2021
The following sections provide guidance on the suggested approach. The facility to utilise all of the suggested elements will depend on the speed of agreeing the approach and the time required to set up the process and make resources available to centres. If time is shortened, some elements could be reduced in intensity, removed or made optional. However, a balance has to be struck between the provision of rigour and pragmatism.
a. Re-designate GCE and GCSE examination papers developed by the awarding bodies as optional reference tests to support teacher-based assessments. This will include mark schemes and marking guidance. These tests are of good quality, already produced and will give the best insight into the impact of Covid-19 and provide schools and colleges with a baseline assessment to complement other contextual information such as coursework, mock exams, quizzes and student’s workbooks.
b. Reference tests could also be formed from a question bank made available by awarding bodies along with guidance on expected levels of performance per question. This approach would make it possible to produce randomised tests and therefore prevent widescale sharing to minimise the chance of predictability of reference tests and pre-prepared answers being shared by students.
c. Advise schools to run reference tests wherever possible under ‘normal’ examination conditions, that is in supervised, timed sessions. These tests should be administered by schools and colleges at times designated by teachers.
d. Give schools the option of internal-marking or offer an awarding body marking service to reduce workload However, external marking would maximise consistency of the standard of marking, but all reference test results should be for school or college use only. Awarding bodies should not make the results more broadly accessible in order to prevent fear of interpretation or use by external parties.
e. Awarding bodies should publish grade boundaries following the marking exercise. This will assist schools in interpreting reference test scores.
f. Grades achieved as outcomes of the reference tests should only be used to inform teacher judgments.
g. If a centre chooses to use items from the reference test question bank, guidance should be provided as to how cut scores could be determined at school level with reference to grade descriptions provided by awarding bodies.
h. The outcomes of the reference test papers will provide centres with an indication of student’s performance in comparison to the population. Teachers will be able to identify areas contained in the exams but not covered during teaching or impacted by limited or distanced teaching.
i. If a centre utilises the item bank approach, awarding bodies could provide item analysis showing schools for example the percentage or range of students who performed well on that item,
j. Centres should be provided as soon as possible with guidance on standard setting, moderation and handling bias.
k. Heads of centre should appoint Lead Assessors in each subject. Lead Assessors play a vital role in ensuring that the guidance materials are used and understood and to oversee the quality assurance of the assessment process in each subject. Lead Assessors therefore need to hold positions of authority. The CIEA has already produced such modules that can be delivered remotely; see https://www.herts.ac.uk/ciea/training/ciea-lead-assessor-programme
l. Examination boards under the auspice of Ofqual should produce grade descriptions and exemplification of expected standards against each and every grade with senior examiners providing commentary to guide schools.
m. Examination boards should run the equivalent of what were ‘grade recognition’ sessions under the old CSE regime. Here, Chief or Senior Examiners explain the expected/required standards of performance with reference to the exemplification materials. These sessions were run face to face but could now be offered on-line. Lead Assessors should attend these sessions in order to fulfil their role.
n. Teachers should award internally derived grades following the awarding body guidance materials and using the standardisation and moderation processes as explained in the support materials.
o. Ofqual and the awarding organisations should make available school/college level and national data showing previous qualification level performances over at least three years and with guidance to help centres in awarding defensible grades and to understand the potential effects of bias and grade inflation.
p. Centre assessed grades should be awarded after consideration of the school/college data, reference test performance if available and work produced during the course taking into consideration the impact of lost learning and projected performance if learning had not been interrupted.
q. Before grades are submitted by schools, heads of centres and Lead Assessors must declare they have followed and implemented guidance procedures as required by the awarding organisations and Ofqual and ensuring that they have challenged recommended grades against previous data and all other contextual information wherever possible.
r. A system of external moderation by a sample of students’ work selected by visiting moderators would assist in assuring consistency across centres. Lead Assessors who have attended the grade recognition sessions could administer this function. It needs to be clear here that external moderators are ensuring the understanding and application of a standard and not the amount of work used completed by students.
s. Randomised centre visits to monitor the application of requirements should be put in place to further support public confidence. This broadly follows current monitoring conducted by the Joint Council for Qualifications (JCQ).
t. Communications supporting these, or indeed any other adopted arrangements will be key. There needs to be good quality guidance, particularly around the role of reference tests, the need for equity and fairness, a firm and un-changed timeline and the allocation of time for teachers to engage with the guidance AND for any marking and grade awarding procedures.
u. Teachers of years 11 and 13 should be taken ‘off-timetable’ to cover the work.
- Dr Mick Walker, January 2021