Why Gates’ Big Data Experiment Assessing Teacher Performance Failed

Cathy O’Neil, a mathematician, hedge-fund analyst and data scientist, points out in a piece for Bloomberg yesterday why the Gates’ Foundation’s Intensive Partnerships for Effective Teaching experiment not only failed but did actual harm to public schools.

Gathering data from assessments, principal observations of teachers, and evaluations from students and teachers they used an algorithm to determine whether a teacher was adding value.

She writes that the goal was to reward the good teachers and root out the bad.

She writes:

Laudable as the intention may have been, it didn’t work. As the independent assessment, produced by the Rand Corporation, put it: “The initiative did not achieve its goals for student achievement or graduation,” particularly for low-income minority students. The report, however, stops short of drawing what I see as the more important conclusion: The approach that the Gates program epitomizes has actually done damage. It has unfairly ruined careers, driving teachers out of the profession amid a nationwide shortage. And its flawed use of metrics has undermined science.

The program’s underlying assumption, common in the world of “big data,” is that data is good and more data is better. To that end, genuine efforts were made to gather as much potentially relevant information as possible. As such programs go, this was the best-case scenario.

Still, to a statistician, the problems are apparent. Principals tend to give almost all teachers great scores — a flaw that the Rand report found to be increasingly true in the latest observational frameworks, even though some teachers found them useful. The value-added models used to rate teachers — typically black boxes whose inner workings are kept secret — are known to be little better than random number generators, and the ones used in the Gates program were no exception. The models’ best defense was that the addition of other measures could mitigate their flaws — a terrible recommendation for a supposedly scientific instrument. Those other measures, such as parent and student surveys, are also biased: As every pollster knows, the answer depends on how you frame the question.

Read the whole thing.

It must be awesome to get to purchase education policy. How many schools, teachers, and students will they experiment on when they finally learn that this is not the way to go about determining education policy?

2 thoughts on “Why Gates’ Big Data Experiment Assessing Teacher Performance Failed

  1. We have seen many reports about the failure of America 2000, Goals 2000, No Child Left Behind and most recently the largest federal misstep Race to the Top but they continue to move forward now don’t they. That should tell a troubling story to people but most just don’t see the truth behind all of these schemes to drive an agenda that is anti education. The sheeple just continue to blindly follow and trust those that truly do not have our children’s best interest in their heart. I believe it is time for us to get up off our knees and stop begging people to fix something that is not broken. It is working perfectly as planned. There is only one real solution and it is something we can do ourselves without government. We must create alternative power. It is time to STARVE THE BEAST.

Comments are closed.