Do Current State K-12 Education Rankings Give An Accurate Picture?

Stan Liebowitz, an endowed professor and director of the Center for the Analysis of Property Right and Innovation (CAPRI), and Matt Kelly, a research fellow at the Colloquium for the Advancement of Free-Enterprise Education (CAFÉ), both in the Naveen Jindal School of Management at the University of Texas at Dallas challenge the way state K-12 education rankings are determined.

They state how states are currently ranked don’t give an accurate picture and those rankings are inherently biased.

They wrote a piece in The Hill that I found interesting:

Conventional rankings also include metrics that aren’t directly related to learning. Some conventional rankings, like Education Week’s ranking, erroneously treat government spending on education as a purely positive factor, rewarding states that spend lavishly regardless of actual student performance.

We recently completed a study of state education systems and found that fixing these problems changes rankings substantially. Conventional rankings are thus severely flawed, as is the consensus of which states educate best.

We graded states based on how well they educate each type of student; i.e., how much “value added” in learning they create. Our analysis utilized the same Department of Education student achievement test data included in most conventional state rankings, but removed metrics unrelated to learning.

Crucially, we disaggregated scores by grade, race, and test subject in order to more accurately measure the value added by state educational systems.

One of the criteria that they dropped was how much a state spends on education which rewarded states that spent “lavishly” on K-12 education spending. “It would seem parents must force politicians to spend into bankruptcy or else doom their illiterate, innumerate children to a menial existence,” they write.

In the introduction to their study they make this excellent point:

Frankly, the entire enterprise of ranking state education systems is a blunt instrument for judging school quality. There exists substantial variation in educational quality within states, and schools differ from district to district. We generally dislike the idea of painting the performance all schools in a given state with the same brush. However, since state rankings currently play so prominent a role in the public debate on education policy, their more egregious methodological flaws demand rectification. We hope that the ranking presented in this report can provide a more nuanced, and more accurate, picture of education systems in U.S. states.

They compare my home state of Iowa with Texas. Iowa traditionally has had higher rankings than Texas in K-12 education. They found when they desegregated NAEP scores:

Texas students have very different characteristics than Iowa students; Iowa’s student population is predominantly white, while Texas is much more ethnically diverse. NAEP data includes average test scores for various ethnic groups. Using the four most populous ethnic groups (White, Black, Hispanic, and Asian), at two grade levels (fourth and eighth), and three subject area tests (math, reading, science), there are twenty-four disaggregated scores that could, in principle, be compared between the two states in 2017. This is much more than just the two comparisons (eighth grade reading and math) USNWR considers.

Given that Iowa students outscore their Texas counterparts on each of the three tests in both fourth and eighth grade, one might expect that most of the disaggregated groups of Iowa students would also outscore their Texas counterparts in most of the twenty exams that both states take. But the exact opposite is the case. In fact, Texas students outscore their Iowa counterparts in all but one of the disaggregated comparisons. The only instance where Iowa students beat their Texas counterparts was the reading test for eighth grade Hispanic students. This is indeed a near shut-out, but one in Texas’ favor, not Iowa’s.

Ouch. Read their report below:

[scribd id=383592941 key=key-I7xvNRMS7oppEIj4yrZv mode=scroll]