By John Marks

Examination Results - The Truth

At a recent conference of the Campaign for State Education (CASE), sweeping claims were made about the substantial gains in GCSE scores across the country that could be made if all grammar and secondary modern schools were to be replaced by comprehensive schools.

The claims were made by Professor David Jesson. Here the NGSA, using reasoned arguments, refutes those claims. The truth about the examination results of Comprehensive Schools compared with Grammar and other Secondary Schools. Professor David Jesson's 'findings' exposed and refuted.

1. All Headlines & No Story

On November 1, 1999 Professor David Jesson made sweeping claims - in public and to the national press and television - about substantial gains in GCSE results across the country if all grammar and secondary modern schools were to be replaced by comprehensives schools. His claims made headlines the next day.

But when he was asked for details of his research - the data he was working with and the calculations he had done - he refused to reply, claiming that the work was not yet finished.

Conclusion

It is wrong - both in public life and in academic research - to announce results when you can't give the evidence on which they are based.

2. Real Results not Adjusted Figures

Three weeks later, on November 22, Professor Jesson sent a brief note to newspapers making it clear that his findings used not real unadjusted GCSE results but adjusted results relating to the difference between the standards achieved by pupils at 14 and 16. His work is, therefore, open to two major criticisms.

First, it is the unadjusted results which pupils will take with them when they go on to the next stage of their studies and their careers. So these unadjusted results have a meaning and a reality which is inherently not the case with Professor Jesson's adjusted results. He should, therefore, specify the actual GCSE results achieved by pupils alongside the adjusted values derived from his calculations.

Conclusion

Real unadjusted results matter and should always be quoted alongside adjusted data.

3. Flawed Research

The second major criticism is that adjusted results of the kind used by Professor Jesson should, in order to be fair to the schools involved, relate to the difference between the standards achieved by pupils at 11 and at 16 - the time when pupils have been at their secondary schools - and not to the difference between the standards achieved by pupils at 14 and at 16.

If substantial progress is made by pupils from 11 to 14, as in many good schools, there may be relatively little more to be gained from 14 to 16. This is particularly likely if there is a ceiling effect with GCSE scores for able pupils as many good teachers have frequently claimed.

It was for these reasons, advanced by headteachers, teacher unions and the press, that the School Progress column in the National Performance Tables for 16 year olds - based on the progress schools had made between the average of their Key Stage 3 results in 1996 and an average measure of their 1998 GCSE performance - was first, in 1998, drastically curtailed by David Blunkett and then, in 1999, abandoned completely. It was widely recognised that it is not fair to make a comparison based on only the last two years of pupils' progress rather than over the full five year period from 11 to 16.

Conclusion

Results fairly adjusted to show progress should cover the whole 11 to 16 period and not just the last two years.

4. What Are the Real Results - Professor Jesson's Claims Exposed

Professor Jesson claims are that:

* Grammar schools get high results - but for a MINORITY of selected pupils

Grammar schools do get high results - one of the very few fully correct statements in Professor Jesson's summary - but it is likely that a majority, not a minority, benefit from this. In GCSE grammar schools do well (%5A*C above 90%) and they do even better at A-level. Results for selective schools taken together (grammar and secondary moderns) are as much as 35% or so better relatively than for comprehensive schools. (See, for example, J Marks A Selective of Comprehensive System: which works best? The Empirical Evidence, Centre for Policy Studies, 1998 and J Marks, C Cox and M Pomian-Srzednicki, Standards in English Schools, National Council for Educational Standards, 1983)

On the only occasion the Department of Education investigated the effect of selection on examination performances at 16+ it was found that - after due allowance had been made for social background data - pupils right across the ability range obtained significantly higher results in those local education authorities which had retained selection (DES Statistical Bulletin 13/84; see Hansard 15 May 1987, Columns 465 and 466).

The results obtained by the National Council for Educational Standards using regression methods, which showed that a fully selective system would obtain results 30-40% better relatively than a comprehensive system) were said by the Department to be similar to the DES findings in Bulletin 13/84, notwithstanding its different data sources (Hansard, 15 May 1987, Column 458).

Pupils in the selective system in Northern Ireland:

- at the age of 14 are 18 months ahead of pupils in England in both English and Mathematics;
- at 16 achieve GCSE results (%5A*C) which are about 10% better absolutely (52% compared with 42%) and about 25% better relatively than for pupils in England.
- at 18 achieve A-level results (%2+A) which are about 12% better absolutely (32% compared with 20%) and 60% better relatively than for pupils in England.

These results indicate that selection is better for all pupils - those at the widely under-rated secondary modern or high schools as well as those at grammar schools.

* Pupils 'not selected' get lower results than they would elsewhere

Results for comprehensive and secondary modern schools in England vary widely with some having 5 or 6 times as many pupils obtaining good GCSEs as others even in the same area. About 700 comprehensive schools - a quarter of the total - perform less well than the average for secondary modern schools.

* Very able pupils in Comprehensive schools do as well as their counterparts in Grammar schools - ie there is NO ADVANTAGE for these pupils in attending a Grammar as opposed to a Comprehensive School.

Pupils in comprehensive schools make up 85% of the age group but obtain only about 75% of good GCSE passes. At A-level their proportion of passes falls to about 65% and to about 50% for A grades. Pupils at grammar schools make up about 3% of the age group but obtain about 13% of all Mathematics and Science passes and points at A-Level and about 15% of the A Grades; the proportions are slightly smaller for Arts subjects.

* The impact of Grammar schools DEPRESSES the OVERALL LEVEL of performance in LEAs where they exist in substantial numbers

Comparisons between the GCSE results for all pupils in LEAs in England and in the Northern Ireland Library Boards show that all the Library Boards and many LEAs with grammar schools are in the top third of the table and that there is a marked tendency for LEAs with grammar schools to achieve better results, on average, for all their pupils.

* Opening ALL schools to ALL pupils would improve results by an average of 3 percentage points for %5+ A* - C passes, and an additional two to three higher grades for every pupil in the system. (Equivalent to THREE YEARS improvement at current rates of progress)

The results for selective schools described above indicate substantial under achievement by many comprehensive schools and a possible further 60,000 pupils who could achieve good GCSEs if we had a wholly selective system in England.

The results are consistent with those emerging over a number of years from studies conducted at the Universities of Sheffield, York, Newcastle-upon-Tyne, Durham and the London Institute of Education.

I know of no such findings relating to Selective as opposed to Comprehensive systems of schools and when I asked Professor Jesson for some he failed to reply.

Conclusion

The real results show that Grammar Schools and Secondary High Schools do better in many ways than Comprehensive Schools - the very opposite of what Professor Jesson claims.

5. The Camp Followers

Professor Jesson's findings have been quoted without any qualification by:

- the Campaign for State Education (CASE) in their December 99 newsletter to parents and schools
- the CASE Say No to Selection website.
- Professor Peter Mortimore, Director of the London University Institute of Education and current President of the British Educational Research Association (BERA), in the Guardian-sponsored debate at the London University Institute of Education on November 16 1999.

Conclusion

Too many commentators are repeating unfounded claims before checking their sources.

6. Value Added Research Results

The DfEE are currently developing value-added measures by tracing individual pupils from 11 (using their National Curriculum Key Stage 2 results) to 16 (using some measure of their GCSE performance). It will not be possible to make calculations of this kind until the year 2003. Even then the calculations involved will be difficult to explain to the layman even if the experts can agree on how the calculations should be done.

Two pieces of research - from GCSE to A-level - do exist and they do not support Professor Jesson's claims.

The first is based on an analysis of GCSE to A-level scores for different kinds of schools and colleges (see DfEE Statistical Bulletin 9/94). It shows that, on most indices, grammar schools do better than comprehensive schools although independent schools do considerably better than any state schools or colleges.

The second report - by the DfEE in 1996 - showed that students in grammar schools make on average 2 A level points more progress than those in comprehensive schools.

Conclusion

The only existing value added research - from GCSE to A-level - shows that Grammar School pupils do better than those from Comprehensive Schools

7. A Simple Suggestion, by John Marks

There is however one simple possibility - a proposal I made in my paper to the CASE Conference on 1 November - that school performance tables for 16-year olds should have a column giving the average standard on National Curriculum tests of pupils when they enter the school at 11.

We could then all judge whether proper progress has taken place and public light could be thrown on discussions about school intakes and the value-added by individual schools, in a way that would be readily understood by parents, governors and everybody else who is interested in the performance of schools.

This would in no way prevent more detailed studies of pupil progress being made by researchers and would indeed complement such studies.

The same thing could and should be done for junior school performance tables for 11-year olds which should have a column giving the average standard on National Curriculum tests of pupils when they enter the school at 7.

Conclusion

A simple form of value added, which all could understand, should be introduced now - by including a column in the National Performance tables for 16 year olds showing the average standard reached by the same pupils in National Curriculum tests at the age of 11.