The most relevant lessons for improving the U.S. education system may be found in successful states, rather than in other countries, according to a new paper from Stanford University professor Martin Carnoy, EPI economist Emma García and Tatiana Khavenson, a researcher at the Institute of Education at National Research University Higher School of Economics in Moscow. In Bringing it back home: Why state comparisons are more useful than international comparisons for improving U.S. education policy, Carnoy, García, and Khavenson argue that differences in countries’ societies and educational systems make international test scores an ineffective way of judging U.S. student achievement, and recommend that we look instead to states that have made large gains in national tests as examples of how to craft education policy.

The U.S. educational system has long been criticized because of students’ relatively poor performance on international tests, such as the Program for International Student Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS). Many policymakers and pundits have concluded, based on U.S. students’ standings, that the United States lags woefully behind comparable nations.

However, the authors caution against attributing U.S. student performance entirely to the quality of U.S. education. There are vast differences between state performances. Students in Massachusetts and Connecticut, for example, scored higher in reading on the 2012 PISA than France, Germany, and the UK. Furthermore, after adjusting for factors such as mother’s education or the number of books in the home, U.S. students perform considerably better than the raw scores indicate.

It is difficult to craft policy recommendations based on the educational systems of high-scoring countries or on reforms made in countries that have made large gains on international tests. International school systems are simply too dissimilar to draw valid conclusions about how to improve U.S. education, and we have very little insight into why some countries score higher on international tests than others.

“There’s no evidence that students in South Korea, for example, score higher on international tests because of better schooling, rather than large investments families make in academic activities outside of school,” said García. “For that reason, it is difficult to use high-scoring countries as examples for how to improve the U.S. education system.”

(In a complementary report released today by the National Education Policy Center, Carnoy discusses other misuses of international data that cast doubts their usefulness for policy making.)

In contrast, school systems among U.S. states are relatively similar to one another, teacher labor markets are not drastically different, and education systems are regulated under the same federal rules. If students with similar family academic resources in some states make much larger gains than in other states, those gains are more likely to be related to specific state policies that could then be applied elsewhere in the United States.

“While education systems differ among states, these differences are much smaller than the difference between lower-performing states and higher-performing countries. Many states have made impressive gains in student performance and can serve as models for others,” said Carnoy. “It makes more sense for Alabama to look to North Carolina for lessons, before turning to Finland, Poland, or Korea.”

The most important lessons we can learn about improving education come from examining why some states have made large gains while others haven’t. The authors turn to state scores on the National Assessment of Educational Progress (NAEP) exam, adjusted for a wide-range of factors, to show which states have made large gains. The lessons embedded in how these states increased student achievement are much more relevant to improving student outcomes in other states than looking to high-scoring countries with social, political, and educational histories that differ markedly from the U.S. experience.

For example, in the past two decades, eighth grade students in Massachusetts made much larger gains in math than students in Connecticut, and students in New Jersey made larger gains than students in New York. Students in North Carolina, meanwhile, made much larger gains than students in neighboring Kentucky and Tennessee, and students in Minnesota made larger gains than students in Iowa.

“These state comparisons become all the more useful in light of the 2015 NAEP results released this week,” García said. “They could provide important insights into the kinds of policies that could help states rebound after disappointing news of student performance.”