A recent academic paper by economists Michael Lovenheim and Alexander Willén argues that men who lived as school-age children in states where teachers were allowed to bargain collectively are less likely to work as adults and, when they do work, they earn significantly less than men who grew up in states where teachers were not allowed to bargain collectively.

There are at least three reasons to be deeply skeptical of their findings.

First, the chain of causal links is extremely circuitous. The reasoning runs from a student’s initial potential “exposure” to teachers’ right to collective bargaining all the way through to the conclusion that this “exposure” significantly worsened labor market outcomes decades later as an adult. In most of their analysis, the authors rely on data that let them know the state where a person was born and the employment situation of that same person in a single year between the ages of 35 and 49. The researchers use this information to construct a simulated educational history for each adult, where they assume that the person attended K-12 school in the state where they were born. The researchers, however, don’t actually know that an individual lived in the state of birth while at school age, or whether the school the individual attended was unionized, or even whether the individual attended a public or private school. Instead, the paper’s conclusions hinge on the idea that students born in states with collective bargaining for teachers were more likely to be “treated” by collective bargaining than students in “control” states where teacher collective bargaining was not permitted. This is possible, of course, but the methodology leaves substantial room for other factors that might explain the observed differences in labor market outcomes of adults who were born in different states. The states that denied teachers the right to bargain collectively, for example, include 11 southern states, which have many long-term trends in common other than collective bargaining rights, including industry and age structure, income distribution, climate, and rapid population growth.

Second, the paper finds no effect of collective bargaining on direct measures of educational outcomes, such as the number of years of schooling. This result is puzzling because educational attainment is probably the single most important determinant of earnings, but the study finds no effect of collective bargaining rights on students’ educational attainment. The paper, instead, argues that the effect of exposure to collective bargaining operates through the impact on a combination of cognitive and “noncognitive” skills that reduce earnings without having any effect on the amount of schooling received. Even more puzzling is that the impact on earnings appears only to affect men—not women—and some ethnic and racial groups, but not others. The estimated effect on earnings for women is not statistically significant; the effect for white and Asian men is relatively small and barely statistically significant; the effect for black and Hispanic men is statistically significant and much larger than for white and Asian men. Again, this is possible, but given the long chain of links between the data and the theory, and the strong tendency of men and women and minority groups to be segregated across different occupations and industries, the different gender-specific and race-specific results suggest that the findings may actually point to contemporary features of state labor markets, rather than educational experience two or three decades earlier.

Third, other recent research suggests much more directly that collective bargaining by teachers is associated with positive educational and social outcomes. Economist Eunice Han, for example, has demonstrated that teacher unions are associated with better outcomes on a range of measures of teacher quality and with a lower high-school dropout rate. Han used three datasets from the National Center for Education Statistics to build a district-teacher matched panel from 2003–2012, included several measures of union strength, and studied what happened after Idaho, Indiana, Tennessee, and Wisconsin placed restrictions on collective bargaining in 2010 and 2011, and found that districts with strong teacher unions had more teachers with stronger qualifications, higher retention rates for high-quality teachers, higher dismissal rates for low-quality teachers, and lower high-school dropout rates. In addition, she argues that because “the dropout rate of an area is negatively associated with future earnings and upward mobility of children of the area (Chetty et al., 2014), unions are more likely to improve educational attainment and the welfare of all children in the area” (p. 47-48).

How and why teacher unions impact student outcomes and how these impacts are transformed in the long run are important questions to be addressed. But we will learn little about them unless research produces conclusions and recommendations that are based on thorough technical and conceptual frames—ones that do not ignore what is already known on the issues (or, if unaligned, clearly explain why), and that offer a more balanced and rigorous insight.