Skip to content

Leaning Tower of P.I.S.A – Part II

money tower#

As I said in Part I, my original reason for wanting to write about the Programme for International Student Assessment had nothing to do with digging up more conspiracy theory dirt about the corporate takeover of education. Now that the dust has settled, it was to reflect on what it is the tests actually measure and on the educational models of the nations whose schools led the most recent PISA rankings.

The first thing that comes to mind when I think about the use of standardized tests to measure  knowledge and understanding is the famous Albert Einstein quote, “Not everything that counts can be counted, and not everything that can be counted counts.” Who better than Einstein to try to lead us away from the old Newtonian paradigm’s obsession with quantifying everything? Meanwhile, contemporary neuroscientific and developmental theory is steadily bearing him out. We now know there’s nothing linear at all about learning and development, and so the idea that the snapshot measurements taken by standardized tests have any real meaning outside the narrow confines of schooling metrics is patently absurd.

The second thought that comes up for me is that there is one, and only one, thing that standardized tests do accurately measure—socioeconomic status. This isn’t my opinion; its sociological fact supported by a solid body of research. In a 2007 study, for example, researchers found that SES was the most significant individual predictor of performance on the grade 3-8 Ohio Achievement Tests, and that test performance was “vastly more indicative of the out-of-school, lived experience of the students than of [in-school] academics.”

Similarly, a study in Washington State comparing the test scores of students in one of the highest income school districts in the state with students in one of the lowest found that 80.7% of the wealthy district students passed the tests, as opposed to only 19.6% from the low-income district. The author of the report cites similar results from around the country, and also refers to an analysis of SAT scores that showed a 97% correlation between test scores and SES. A 2012 study by the College Board itself  confirms this finding by showing that SAT scores go up almost exactly 20 points with each $20,000 increase in family income, with the effect flattening out in the $100,000-$200,000 range and then increasing to nearly 30 points after that.


A state-by-state breakdown of the U.S. PISA scores published last month in the Huffington Post reveals the same tight correlation between SES and test performance. The report showed that if Massachusetts, a state with one of the highest per capita incomes, were its own country, it would currently rank sixth out of the 65 nations in PISA reading results and on par with other high-scoring European countries in math and science. A poor state like Florida, on the other hand, would rank 27th in reading, 38th in science, and 41st  in math.

A study by Helen Ladd at Duke University discovered the same pattern across 14 other Western nations, with the highest-income students nearly doubling the scores of their lowest-income peers. Ladd lamented that Finland, Korea, and Canada, which are well-known for their efforts to ensure educational equality, were among the 14.

Meanwhile, Shanghai, which for reasons I don’t entirely understand represents all of mainland China in the PISA contest and ranked #1 in the world for the second straight cycle, is a highly prosperous city with the highest level of disposable income in the country. There are also reports that Shanghai’s high schools admit only the best and brightest students from peasant families who have recently emigrated from the countryside, thus excluding a sizable pool of students who would be likely to bring down the city’s PISA average.

And the Winners Are …

While I neither believe that what the PISA tests allege to count can really be counted, nor that what they actually do count is worth all that much, a look at the PISA top ten still provides interesting food for thought. For starters, it should come as no surprise that the first seven nations are all located in Asia. Countries like China, Japan, and Korea are notoriously education-obsessed. In Shanghai, for instance, the competition to attend higher ranked schools, which is determined by standardized entrance exams, is so fierce that test prep virtually becomes the sole aim of education almost from the beginning. Parents are heavily invested in the process because high scores are an important status symbol, so much so according to Jiang Xueqin, the deputy principal of one of China’s most famous public schools in an op-ed piece for CNN, that many wealthy parents try to bribe their children’s way into the schools that produce the highest scores.

“Cram schools,” before- and after-school test prep centers that children begin attending in elementary school, are now ubiquitous throughout Asia. They’ve even existed in New York City’s Chinatown for decades, and according to the New York Times a rising number of white students are now enrolling in them too.

In his whistle-blowing exposé, Jiang Xueqin refers to research indicating that performance-based incentives like standardized tests are bad for students and teachers. “Incentives do not just make students stressed, lonely, and unhappy,” he wrote, “they also kill student’s innate curiosity, creativity, and love of learning.”

Jiang Xueqin then went on to praise the education system in Finland. Even though its model excludes the use of standardized tests for the same reasons stated above, Finland is a perennial PISA frontrunner, and actually ranked #1 in 2000, the first time the tests were given. Jiang Xueqin, who visited Finnish schools to learn why they are so successful, noted that while Shanghai children leave school at 4 p.m., then go on to cram school, and then do homework until bedtime, Finnish kids are done at one or two and then go home to play for the rest of the day.

Play is also valued part of the school day in Finland, where elementary-level students enjoy a 15-minute recess after every 45-minute lesson. Grades are de-emphasized, too, and never used to rank students because the Finns are in tune with how children actually learn and recognize that external motivation, pressure, and competition impede the learning process.

So isn’t it instructive that Finland’s PISA scores are substantially higher than here in the U.S., where our educational model is heading in the opposite direction, and almost as high as Shanghai’s even though Finnish students do nothing to directly prepare for the tests? I think they just might be on to something.

2 Comments Post a comment
  1. David B. Easton #

    After reading these two pieces on the PISA tests, it strikes me as some terrible irony that the results from a test that basically identify your ranking on the Socio-Economic scale motivates the spending of billions of dollars that don’t directly go toward fixing the real problem: poverty.

    In my mind there can and will never be any real education reform until we have poverty reform.

    February 8, 2014
    • Chris Mercogliano #

      You’ve anticipated the post I’m working on now about the work of the late Jean Anyon. As I just said to Bunty Ketcham, great minds think alike!

      February 10, 2014

Leave a Reply

You may use basic HTML in your comments. Your email address will not be published.

Subscribe to this comment feed via RSS