I was reading the deposition in David Delk v. Ohio County (WV) Board of Education, a lawsuit currently before the Circuit Court of Ohio County, WV. Delk is an attorney and the father of a third grader in Ohio County Schools. He filed a FOIA request to see the questions on the Smarter Balanced after results were released. His request was denied and he then sued. The person being disposed is Sue Ellen McGuire who is the assessment director for Ohio County Schools.
The following jumped out at me.
Q. And who is the vendor for this test?
A. American Institutes for Research, AIR.
Q. And do you have an understanding how those people who read the assessments are trained?
A. I do not, sir.
Q. Do you have any idea what their background is?
A. I do not, sir. Q. Do you have any idea what their level of education is?
A. I do not, sir.
Q. Do you know whether it’s a part-time or temporary job? A. I have no idea, sir. Q. Do you have any idea what the job pays? A. I have no idea, sir.
So humans are grading the written sections of the assessment and we essentially know nothing about how they trained and what kind of background is required. Nice.
Then right after talking about who is grading they discuss cut scores.
Q. Does Ohio County Schools have any indication or knowledge what level of performance is required to hit the various cut levels on the assessment?
A. Ohio County Schools has no knowledge of that.
Q. Does the West Virginia Department of Education have that information, to your knowledge?
A. I would say they probably do not.
This report uses a scale score, and the scale scores would be a conversion of the student’s raw score to a common scale. So they may or may not know, and I would be reluctant to answer on behalf of the department of ed.
Q. Ohio County Schools doesn’t know the raw score?
A. We do not.
Q. The parents are not provided the raw score?
Q. I mean, it’s fair to say whatever scale score the student receives on this test, we have no idea to know whether that means they answered 25 percent right or 99 percent right?
A. You’re right, sir.
To not know what level of performance is needed to reach a cut score is appalling.
I encourage you to read the deposition for yourself which you can see below: