I was reading the deposition in David Delk v. Ohio County (WV) Board of Education, a lawsuit currently before the Circuit Court of Ohio County, WV. Delk is an attorney and the father of a third grader in Ohio County Schools. He filed a FOIA request to see the questions on the Smarter Balanced after results were released. His request was denied and he then sued. The person being disposed is Sue Ellen McGuire who is the assessment director for Ohio County Schools.
The following jumped out at me.
Q. And who is the vendor for this test?
A. American Institutes for Research, AIR.
Q. And do you have an understanding how those people who read the assessments are trained?
A. I do not, sir.
Q. Do you have any idea what their background is?
A. I do not, sir. Q. Do you have any idea what their level of education is?
A. I do not, sir.
Q. Do you know whether it’s a part-time or temporary job? A. I have no idea, sir. Q. Do you have any idea what the job pays? A. I have no idea, sir.
So humans are grading the written sections of the assessment and we essentially know nothing about how they trained and what kind of background is required. Nice.
Then right after talking about who is grading they discuss cut scores.
Q. Does Ohio County Schools have any indication or knowledge what level of performance is required to hit the various cut levels on the assessment?
A. Ohio County Schools has no knowledge of that.
Q. Does the West Virginia Department of Education have that information, to your knowledge?
A. I would say they probably do not.
This report uses a scale score, and the scale scores would be a conversion of the student’s raw score to a common scale. So they may or may not know, and I would be reluctant to answer on behalf of the department of ed.
Q. Ohio County Schools doesn’t know the raw score?
A. We do not.
Q. The parents are not provided the raw score?
A. No.
Q. I mean, it’s fair to say whatever scale score the student receives on this test, we have no idea to know whether that means they answered 25 percent right or 99 percent right?
A. You’re right, sir.
To not know what level of performance is needed to reach a cut score is appalling.
I encourage you to read the deposition for yourself which you can see below:
Karen Effrem, MD says
Thanks for posting this Shane! Too bad the attorney didn’t have this info about the AIR subcontractor Data Recognition Corporation (DRC) test scorers: According to a teacher (http://www.teachertalkstruth.com/blog/score-its-all-about-the-children) and reported by EdWeek (http://www.edweek.org/ew/articles/2015/05/20/thousands-of-scorers-take-on-the-common-core.html?cmp=ENL-EU-NEWS1), DRC hires temporary and at least partially non-teacher personnel who are merely required to have a college degree, but no teaching experience, to grade the tests. They use websites like Craig’s list and Indeed as shown below to hire these workers. The flyer DRC uses for hiring is available at http://www.datarecognitioncorp.com/Documents/Reader-Flyer-2015.pdf. This raises all sorts of concerns about the quality of the training, the subjective nature of various individuals’ interpretations and the validity of scores for these items.
Chip Douglas says
Mr. Delk should be figting the State Department that agreed to utilize this test not the local county that is under the States rule. Just seems like This Delk fella has something personal or he would went after the correct department.
Judy Hinnerichs says
This is so stupid. I move to WV to retire and get away from the stress of DC. I was wrong. The mental angst and stress of living in a state that has so little regard for its children’s future is beyond belief. This state is the worst for unemployment, the worst in education and the people who control this are perpetuating another generation of the same. There is a cure for ignorance, it’s called education there is no cure for stupid.