Putting PISA Results to the Test

Analysis by Eric Yu and Pamela Cantor, M.D.

The 2015 PISA results are out and the mainstream media headlines are in:

• 15-year-old students in the U.S. are middling at best in science and reading
• American 10th-graders are performing poorly in math and getting worse

These results have led to calls for myriad reforms, ranging from increased standards and testing, to greater emphasis on STEM, to more funding for school choice. However, an analysis of U.S. schools by students’ family income reveals a more nuanced story.

The Impact of Child Poverty on U.S. PISA Results

On the surface, the PISA results are indeed very disappointing. The U.S. ranks 31st in math, 20th in reading and 19th in science out of 35 OECD nations, dropping in two out of three subjects compared to 2012.But there is more to this data story. Let’s take a look at these results through the lens of child poverty. According to a 2015 report by UNICEF, the U.S. has the second-highest child poverty rate (23.1%) among industrialized nations from the European Union and OECD; only Romania’s is higher (25.5%). In fact, the majority of children attending U.S. public schools – 51% – are growing up in low-income households, the highest percentage since the federal government began tracking the figure.This very high prevalence of child poverty in the U.S. and the struggle to educate large numbers of disadvantaged students are two of the most important factors underlying the country’s discouraging showing on the PISA exam.

• U.S. schools where <25% of students qualify for free- and reduced-price lunch (FRPL) fare extremely well in PISA rankings. In fact, they would rank first in reading and science and third in math among OECD countries.

• By contrast, U.S. schools where >75% of students qualify for FRPL fare very poorly, ranking nearly last in all subjects. Their scores are so low that they drag the overall U.S. average below the median, just above Mexico and Chile.

Science helps explain the connection between poverty and poor academic performance. Children growing up in poverty often experience chronic stress. Neuroscientific research demonstrates that chronic stress can affect the developing learning centers of the brain, with impact on attention, concentration, working memory and self-regulation. It also shows that the developing brains of children are malleable into adulthood. This means there is an important opportunity for educators to reverse the effects of stress through the intentional design of teaching and learning environments, specifically through personalization of the learner experience.

Trends in U.S. Mathematics Performance

The PISA rankings beg further analysis of U.S. performance in math, where average scores (31) have fallen almost as low as high-poverty scores (33).

2015 PISA math scores by school income level show a growing number of schools trailing the global OECD average in math performance:U.S. schools with high levels of poverty (>50% FRPL) continue to lag the OECD average in math by significant amounts. However, in 2015, U.S. schools with moderate levels of poverty (25-50% FRPL) also scored below the OECD average, after exceeding it handily in 2012.

Educators report problems with both the fragmented structure of the math curriculum in many U.S. schools and approaches to instruction. Studies have shown that a number of countries leading the PISA math rankings focus on only six or seven major math concepts per grade, while U.S. schools touch upon up to 75. At the same time, leading countries require students to understand “connections” problems, where children must apply concepts to solve previously unfamiliar problems, whereas U.S. coursework in math focuses almost exclusively on “procedure” problems, which test memorization of a math skill, rather than its application to new, more difficult problems.

Impact on U.S. Workforce Competitiveness

The consequences of the U.S.’s weak PISA performance are already being seen in the country’s workforce. The OECD conducts a survey of workforce skills (the Program for International Assessment of Adult Competencies, or PIAAC) that is similar to PISA for adult workers, measuring two key skill areas: literacy and numeracy.

In the most recent PIAAC survey released in June, the U.S. ranks 17th in literacy and 27th in numeracy among 34 nations and territories surveyed, similar to the PISA rankings.A deeper dive into the data shows an even more concerning trend:

The U.S. adult skill rank is especially low among younger cohorts of workers. While older (55-65) U.S. workers’ skills exceed their peers in most countries, U.S. 16-24-year-olds trail nearly all countries.Unfortunately, this trend suggests that U.S. workforce competitiveness has been declining over time, with the most recent high school graduates ranking near the bottom globally.For the U.S. to maintain its economic competitiveness, Americans with a high school education or below must be able to demonstrate the higher order literacy and numeracy skills that today’s workforce demands.

What are the key takeaways from this data?

1)  Low U.S. PISA rankings continue to be attributable to the very high prevalence of child poverty in the country.

• Schools with lower levels of poverty (<25% FRPL) continue to rank very well in reading (1), science (1), and even math (3)
• High-poverty schools (>75% FRPL) continue to struggle. Their scores are roughly equal to those of Chile.

2)  U.S. performance in PISA math showed alarming deterioration; in particular, a steep drop in the performance of moderate-poverty schools (25-50% FRPL).

• For the first time ever, moderate-poverty schools’ math PISA scores lagged behind the OECD average. This begs a number of questions:
  • Are we seeing the disruptive effects of poverty on learning even in moderate-poverty schools?
  • To what degree are shortcomings in the U.S. math curriculum and math pedagogy having an impact on performance compared to other countries?

3)  The consequences of weak preparation in K-12 schools are already being seen in rapidly-declining U.S. workforce skills.

• U.S. workers rank especially poorly in math-related skills and the trend shows a steady worsening among the youngest American workers
• The discouraging performance of low-income U.S. schools can be seen in the lagging skills of U.S. workers who do not attend college or drop out of high school

Conclusion

The 2015 PISA data tell a story that goes well beyond the headlines of disappointing U.S. rankings. Low-poverty U.S. schools are actually performing well, but high-poverty schools drag U.S. rankings below the global median, particularly in mathematics. The consequence is a growing segment of young U.S. workers who are less prepared for careers in an increasingly competitive world.

If the U.S. wants this to change, it must understand and address the impact of poverty on school environments and student learning. This analysis shows that these two areas of focus have the greatest potential to reverse an alarming and persistent downward trend in the preparation of American students for participation in the global economy.

Eric Yu is a Special Advisor to Turnaround for Children. Pamela Cantor, M.D. is President and CEO of Turnaround for Children.