Are the Standardized Tests Good Measure of Academic Ability or Progress

In 2009 two Indian states, Himachal Pradesh and Tamil Nadu, took part in the Programme for International Student Assessment, the test of fifteen year olds run by the Organisation for Economic Cooperation and Development. They came 72nd and 73rd out of 74 participants. The result, published in 2012, shocked many Indians and became a reference point in the debate on education. In another case, an investigation in the American city of Atlanta found in 2011 that teachers and principals in dozens of schools had altered student answer sheets to raise scores. Both stories concern standardized tests, and they point to the two sides of the topic. Such tests can reveal the truth, and they can distort it.

A standardized test is one administered and scored in the same way for all test takers, so that results can be compared. It may be a school board examination, an entrance test or an international assessment. The topic asks whether these tests are good measures of academic ability, which is what a student can do, or of academic progress, which is how much he has learnt over time.

The case for them is strong. They offer objectivity. In a system without them, grades depend on the judgement of individual teachers, who may be biased or inconsistent, and in a society with deep inequalities, standard tests can open doors to the talented. The system of competitive examinations in India for the civil services and institutes of technology has allowed students without connections to rise. Standardized tests are also cheap to administer to large numbers, and they provide data on how systems perform. The Annual Status of Education Report, based on simple tests administered to children in villages since 2005, has shown that many children in the fifth grade cannot read a second grade text, a finding that led to a national debate on learning. Without such measures, failure would remain hidden.

Tests can also motivate. Students and teachers work toward clear goals, and measurement allows early identification of those who need help. Properly designed, with questions that probe understanding and are validated for reliability, a test can predict later performance reasonably well. Studies of the SAT and similar tests find modest but real links with success in college.

Without such measures, failure would remain hidden.

Yet the limits are serious. First, what is tested is narrow. A written test of a few hours can sample reading, arithmetic and recall of facts, and perhaps some reasoning. It cannot capture creativity, persistence, collaboration, curiosity or moral character, which matter in life. Howard Gardner proposed in 1983 that intelligence has many forms, including musical, spatial and interpersonal, and a pen and paper test favours only some. The paediatrician Alfred Binet, who devised the first intelligence test in 1905, warned that intelligence could not be reduced to a single number, and Stephen Jay Gould in The Mismeasure of Man criticised the misuse of such scores.

Second, high stakes distort behaviour. The social scientist Donald Campbell stated in 1976 that the more any quantitative indicator is used for decisions, the more subject it will be to corruption pressures and the more apt it will be to distort the process it monitors. Schools drill children in the form of the test, teach only what will be examined, and neglect art, sport and discussion. The coaching industry that has grown around entrance examinations in India is an example. Pupils learn tricks for choosing answers, and not the subject. In extreme cases the pressure leads to cheating, as in Atlanta, or the mass copying and paper leaks reported from several states.

Third, tests are poor at measuring progress unless designed for it. A single score at one time shows where the student stands, which depends on his home and earlier schooling, and not what the school added. A child from a prosperous family will score higher than one from a poor family, even if the second has learnt more in the year. Fairer measures follow the same students over time, to see the gain. Where a test is in a language the child does not speak at home, or reflects the experience of urban children, it measures that gap.

The experience of nations is instructive. The United States brought in the No Child Left Behind Act in 2002, which tied school funding and sanctions to test scores, and critics found that it led to narrowing of the curriculum and gaming. Finland, which ranks high in international tests, relies little on standardized testing in its early years, trusting trained teachers to assess pupils. China's gaokao, the college entrance examination, is credited with fairness and criticised for stress. India's own board examinations and entrance tests carry heavy consequences, and the Right to Education Act of 2009 discouraged detention of children and brought in continuous and comprehensive evaluation, which has been uneven in practice.

The reasonable position is that standardized tests are useful tools if used for what they can do. They are good at measuring basic skills at scale, at comparing systems and at spotting problems. They are poor as the sole basis for judging a child's worth, a teacher's quality or a school's merit. Decisions should draw on several kinds of evidence, including classwork, projects and teacher observation. Tests should assess understanding and application, and not memory alone, and should be used mainly to help teaching, with low stakes for the child.

Reform of the tests themselves is possible. Questions can ask students to explain, to apply and to argue, and not only to recall. Tests can be given more than once a year so that no single day decides a future, and results can be reported with a margin of error, as honest measures should. Scores can be adjusted to account for a child's circumstances. Countries such as Singapore have moved to assess a wider range of skills. Technology may make adaptive testing, which adjusts questions to the student's level, more widespread.

A test measures what its makers thought worth measuring and what could be measured cheaply. The first is a choice about values, and the second is a limit of method, and the result looks like a fact. If the aim of education is to prepare a person for a life that nobody can fully predict, the most important abilities are those the test is least able to see, and the question is whether a society will accept a measure it cannot see.