Robo-grading PARCC

Photo credit: Jonathan McIntosh (CC-By-2.0)
Photo credit: Jonathan McIntosh (CC-By-2.0)

Politico Morning Edition yesterday had a follow up to Pearson’s development of a computer algorithm that would grade the essay section of the PARCC assessment.  The vote on whether to approve Pearson’s plan has been delayed.  Inquiring minds want to know why and inquiring minds were left to inquire.  PARCC is not extremely forthcoming with information.

ROBO-GRADING THE COMMON CORE: The PARCC consortium was supposed to have decided by now whether to approve Pearson’s plan to use computer algorithms, rather than human beings, to grade student essays on Common Core exams in the coming years. But that deadline appears to have been postponed — and both PARCC and Pearson are tight-lipped about why.

— Some background: The PARCC exams are designed to challenge students to read closely, think deeply and write sophisticated analyses of complex texts. But hiring people to read all that student writing is expensive. So Pearson’s four-year contract to administer the exams bases the pricing on a phase-in of automated scoring. All student writing will be scored by real people this coming spring. The following year, the plan calls for two-thirds to be scored by computer. The year after that, all the writing is scheduled to be robo-graded, with humans giving a small sampling a second read as quality control.

— The contract required Pearson to submit a proof-of-concept study demonstrating the validity of automated grading by mid-October. It required PARCC to give “final approval” to proceed with the phase-in by Oct. 31. Any changes to these deadlines required a formal “scope change” to the contract. There have been no scope changes to this portion of the contract, according to Larry Behrens, a spokesman for the New Mexico Public Education Department, which oversees the contract. So where’s the Pearson study? PARCC spokesman David Connerty-Marin told Morning Education it’s being revised — but he declined to say who had asked for the revisions or what they entail. Pearson wouldn’t answer any questions on the subject, referring them all to PARCC.

— And what of the provision that PARCC provide “final approval” of the phase-in of automated grading? Connerty-Marin wouldn’t answer questions about whether a vote has already taken place or will be held in the future. He at first told Morning Education that PARCC states are currently “conducting studies” on the efficacy of using computer algorithms. But he later acknowledged that states aren’t doing their own studies; they’re relying on the Pearson report. Morning Education contacted all the states using PARCC tests to ask if they had made a decision on automated grading. Only Colorado and D.C. replied; both said no decision had been made.

Robo-grading essays, what could possibly go wrong?  If you think this is a disaster waiting to happen, you are not alone.